canScrape method

bool canScrape({
  1. required Uri url,
  2. required ScraperConfigMap scraperConfigMap,
})

Checks if the given URL can be scraped with the provided configuration.

This method determines if there's a matching ScraperConfig for the given URL by checking if the URL's path matches any of the path patterns defined in the scraper configurations.

Parameters:

  • url: The URL to check for scraping capability
  • scraperConfigMap: Map of domain names to lists of scraper configurations

Returns:

  • true if a matching scraper configuration is found
  • false if no matching configuration exists

Implementation

bool canScrape({
  required Uri url,
  required ScraperConfigMap scraperConfigMap,
}) {
  ScraperConfig? scraperConfig = findScraperConfig(
    url: url,
    scraperConfigMap: scraperConfigMap,
  );
  return scraperConfig != null;
}