Any Logger JSON HTTP #

A JSON HTTP appender extension for Any Logger that enables sending structured logs to REST APIs, Logstash, centralized logging services, and custom HTTP endpoints with automatic batching, retry logic, and compression support.

Features #

• Automatic Batching - Intelligently batches logs to reduce network overhead
• Retry Logic - Configurable retry with exponential backoff for failed sends
• Multiple Auth Methods - Supports Bearer tokens, Basic auth, and custom headers
• Compression Support - Optional batch compression to reduce bandwidth
• Flexible Configuration - Works with config files, builders, or programmatic setup
• Error Recovery - Buffers logs during network failures and retries when connection resumes
• Statistics Tracking - Monitor successful/failed sends and buffer status

Installation #

dependencies:
  any_logger: ^x.y.z  
  any_logger_json_http: ^x.y.z  // See Installing

To register the JSON HTTP appender you have to import the library

import 'package:any_logger/any_logger.dart';
import 'package:any_logger_mysql/any_logger_mysql.dart';

and call:

AnyLoggerJsonHttpExtension.register();

Quick Start #

Simple Setup #

await
LoggerFactory.builder
().console
(
level: Level.INFO)
    .jsonHttp(
url: 'https://logs.example.com/api/logs',
level: Level.WARN,
authToken: 'your-api-key',
)
    .build();

Logger.info('This goes to console only');
Logger.warn('This goes to both console and HTTP endpoint');
Logger
.
error
(
'
Errors are batched and sent efficiently
'
);

With Authentication #

// Bearer token (most common)
await
LoggerFactory.builder
().jsonHttp
(
url: 'https://api.logservice.com',
authToken: 'sk-1234567890',
authType: 'Bearer',
level: Level.ERROR,
)
    .build();

// Basic authentication
await LoggerFactory.builder()
    .jsonHttp(
url: 'https://logs.example.com',
username: 'logger',
password: 'secure_password',
level: Level.INFO
,
)
.
build
(
);

Configuration Options #

Using Builder Pattern #

final appender = await
jsonHttpAppenderBuilder
('https://logs.example.com
'
)
.withLevel(Level.ERROR)
    .withBearerToken('api-key-123')
    .withBatchSize(100)
    .withBatchIntervalSeconds(30)
    .withHeaders({
'X-Application': 'MyApp',
'X-Environment': 'production',
})
    .withStackTraces(true)
    .withMetadata(true)
    .withMaxRetries(5)
    .withExponentialBackoff(true)
.
build
(
);

Using Configuration Map #

final config = {
  'appenders': [
    {
      'type': 'JSON_HTTP',
      'url': 'https://logs.example.com',
      'endpointPath': 'v2/ingest',
      'authToken': 'your-token',
      'authType': 'Bearer',
      'level': 'INFO',
      'batchSize': 100,
      'batchIntervalSeconds': 30,
      'headers': {
        'X-Application-Id': 'myapp',
        'X-Environment': 'production',
      },
      'includeMetadata': true,
      'includeStackTrace': true,
      'maxRetries': 3,
      'exponentialBackoff': true,
      'timeoutSeconds': 10,
    }
  ]
};

await
LoggerFactory.init
(
config
);

Configuration Parameters #

Alternative Field Names #

For compatibility with different configuration formats:

• bufferSize → batchSize
• flushIntervalSeconds → batchIntervalSeconds
• enableCompression → compressBatch

JSON Payload Structure #

The appender sends logs in this format:

{
  "timestamp": "2025-01-20T10:30:45.123Z",
  "count": 3,
  "metadata": {
    "appVersion": "1.2.3",
    "deviceId": "a3f5c8d2",
    "sessionId": "e7b9f1a4",
    "hostname": "server-01"
  },
  "logs": [
    {
      "timestamp": "2025-01-20T10:30:45.100Z",
      "level": "ERROR",
      "levelValue": 40000,
      "message": "Database connection failed",
      "logger": "DatabaseService",
      "tag": "DB",
      "class": "DatabaseService",
      "method": "connect",
      "line": 42,
      "error": {
        "message": "Connection timeout",
        "type": "TimeoutException"
      },
      "stackTrace": "...",
      "mdc": {
        "userId": "user-123",
        "requestId": "req-456"
      }
    }
  ]
}

Presets #

Logstash Integration #

final appender = await
jsonHttpAppenderBuilder
('https://logstash.example.com
'
)
.withLogstashPreset(
) // Optimized for Logstash
.
build
(
);

// Preset configures:
// - batchSize: 200
// - batchIntervalSeconds: 30
// - includeMetadata: true
// - includeStackTrace: true
// - exponentialBackoff: true

High Volume Logging #

final appender = await
jsonHttpAppenderBuilder
('https://logs.example.com
'
)
.withHighVolumePreset(
) // Optimized for high throughput
.
build
(
);

// Preset configures:
// - batchSize: 500
// - batchIntervalSeconds: 10
// - includeStackTrace: false (performance)
// - includeMetadata: false (reduce payload)
// - compressBatch: true

Critical Errors Only #

final appender = await
jsonHttpAppenderBuilder
('https://alerts.example.com
'
)
.withCriticalErrorPreset()
    .withBearerToken('alert-api-key')
    .build
(
);

// Preset configures:
// - level: ERROR
// - batchSize: 10 (send quickly)
// - batchIntervalSeconds: 5
// - includeStackTrace: true
// - maxRetries: 5

Integration Examples #

With Centralized Logging Service #

// Datadog, Loggly, Papertrail, etc.
await
LoggerFactory.builder
().console
().jsonHttp
(
url: 'https://http-intake.logs.datadoghq.com',
endpointPath: 'v1/input',
authToken: process.env['DD_API_KEY'],
headers: {
'DD-EVP-ORIGIN': 'my-app',
'DD-EVP-ORIGIN-VERSION': '1.0.0',
},
level: Level.INFO,
)
    .build(
);

With Custom Backend #

await
LoggerFactory.builder
().jsonHttp
(
url: 'https://api.mycompany.com',
endpointPath: 'logging/v1/ingest',
authToken: await getAuthToken(),
headers: {
'X-Client-Version': '2.1.0',
'X-Platform': Platform.operatingSystem,
},
batchSize: 50,
includeMetadata: true,
)
.
build
(
);

Error Alerting System #

// Send only errors immediately to alerting system
await
LoggerFactory.builder
().file
(
filePattern: 'app', level: Level.DEBUG) // All logs to file
    .jsonHttp(
url: 'https://alerts.example.com/critical',
level: Level.ERROR, // Only errors to HTTP
batchSize: 1, // Send immediately
authToken: 'alert-key',
)
    .build();

Monitoring & Statistics #

final appender = logger.appenders
    .whereType<JsonHttpAppender>()
    .firstOrNull;

if (
appender != null) {
final stats = appender.getStatistics();
print('Successful sends: ${stats['successfulSends']}');
print('Failed sends: ${stats['failedSends']}');
print('Buffer size: ${stats['bufferSize']}');
print('Last send: ${stats['lastSendTime']}');
}

Best Practices #

1. Choose Appropriate Batch Sizes #

• Low volume: 50-100 logs per batch
• High volume: 200-500 logs per batch
• Critical errors: 1-10 for immediate sending

2. Set Reasonable Intervals #

• Production: 30-60 seconds
• Development: 5-10 seconds
• Critical systems: 1-5 seconds

3. Handle Network Failures #

// Configure retry strategy
    .withMaxRetries(5)
    .withExponentialBackoff(true)
    .withTimeoutSeconds(
10
)

4. Optimize Payload Size #

// For high-volume, non-critical logs
    .withStackTraces(false) // Reduce size
    .withMetadata(false) // Only if not needed
    .withCompression(
true
) // Compress batches

5. Secure Your Credentials #

// Use environment variables or secure storage
final apiKey = Platform.environment['LOG_API_KEY'];
// Or use a secrets manager
final apiKey = await
SecretManager.getSecret
('log-api-key
'
);

Troubleshooting #

Logs Not Reaching Server #

1. Check network connectivity
2. Verify authentication: Ensure token/credentials are correct
3. Check URL and endpoint: Verify the full URL is correct
4. Enable self-debugging:

await
LoggerFactory.builder
().jsonHttp
(
url: 'https://logs.example.com')
    .withSelfDebug(Level.DEBUG
)
.
build
(
);

High Memory Usage #

• Reduce batchSize to limit buffer memory
• Enable compressBatch for large payloads
• Disable includeStackTrace if not needed

Logs Lost During Shutdown #

Always flush before app termination:

// In your app shutdown handler
await
LoggerFactory.flushAll
();

Testing #

For unit tests, use test mode to avoid actual HTTP calls:

final appender = await
jsonHttpAppenderBuilder
('https://test.example.com
'
)
.withLevel(Level.INFO)
    .build(test: true
); // No actual HTTP calls made

License #

MIT License - see LICENSE file for details.

Support #

• Main Package: any_logger
• Issues: GitHub Issues
• Examples: See /example folder in the package

Part of the Any Logger ecosystem.

💚 Funding #

• 🏅 https://github.com/sponsors/raoulsson
• 🪙 https://www.buymeacoffee.com/raoulsson

Happy Logging! 🎉