extractStructuredDataEfficient method

List<Map<String, String>> extractStructuredDataEfficient({
  1. required String html,
  2. required Map<String, String> selectors,
  3. Map<String, String?>? attributes,
  4. int chunkSize = 1024 * 1024,
})

Extracts structured data using memory-efficient parsing for large HTML documents

html is the HTML content to parse selectors is a map of field names to CSS selectors attributes is a map of field names to attributes to extract (optional) chunkSize is the size of each chunk to process (default: 1024 * 1024 bytes)

Implementation

List<Map<String, String>> extractStructuredDataEfficient({
  required String html,
  required Map<String, String> selectors,
  Map<String, String?>? attributes,
  int chunkSize = 1024 * 1024, // 1MB chunks
}) {
  _logger.info(
    'Using memory-efficient structured extraction for HTML (${html.length} bytes)',
  );
  return _memoryEfficientParser.extractStructuredData(
    html: html,
    selectors: selectors,
    attributes: attributes,
    chunkSize: chunkSize,
  );
}