ScraperConfig constructor

ScraperConfig({
  1. List<String> pathPatterns = const [],
  2. bool requiresHtml = true,
  3. UrlCleaner? urlCleaner,
  4. required List<Parser> parsers,
  5. ProxyAPIConfig? proxyAPIConfig,
  6. Map<String, String>? cookies,
  7. Map<String, String>? headers,
  8. bool forceRefresh = false,
  9. UserAgentDevice userAgent = UserAgentDevice.mobile,
})

Creates a new ScraperConfig instance.

Parameters:

  • pathPatterns: URL path patterns to match
  • parsers: List of parsers for data extraction
  • requiresHtml: Whether HTML fetching is required (default: true)
  • urlCleaner: URL preprocessing configuration (optional)
  • proxyAPIConfig: Proxy API configuration (optional)
  • cookies: Cookies to include in the HTTP request (optional)
  • headers: Headers to include in the HTTP request (optional)
  • forceRefresh: Force fresh HTTP requests (default: false)
  • userAgent: User agent device type (default: mobile)

Implementation

ScraperConfig({
  this.pathPatterns = const [],
  this.requiresHtml = true,
  this.urlCleaner,
  required this.parsers,
  this.proxyAPIConfig,
  this.cookies,
  this.headers,
  this.forceRefresh = false,
  this.userAgent = UserAgentDevice.mobile,
});