blobinator 0.1.2 copy "blobinator: ^0.1.2" to clipboard
blobinator: ^0.1.2 copied to clipboard

Tools and HTTP service for temporary binary blob storage.

Dart package and standalone HTTP service for temporary binary blob storage.

Why? #

Blobinator provides a simple way to exchange temporary binary data between systems. It's designed for:

  • Temporary data exchange - Binary blobs with random identifiers, no metadata required
  • Short-lived storage - Data may be evicted or lost, applications must handle retries
  • Expendable content - Data written to disk can be deleted automatically
  • Processing pipelines - Intermediate data storage between processing steps

⚠️ Do not use for permanent storage - data will be automatically evicted.

Features #

Two-Tier Storage #

  • Memory tier: Fast access for recent blobs
  • Disk tier: Slower access for evicted blobs
  • Automatic promotion: Memory → Disk when limits reached
  • Configurable limits: Items, size, and TTL for each tier

Configuration #

  • Memory: 1M items, 1 GiB, 3 days TTL (default)
  • Disk: 100M items, 512 GiB, 90 days TTL (default)
  • Flexible units: 1m items, 2GiB size, 6h TTL
  • Optional disk storage: Memory-only mode available

HTTP API #

  • GET /blobs/{id} - Retrieve blob data
  • PUT /blobs/{id} - Store blob data
  • HEAD /blobs/{id} - Get metadata (size, last-modified)
  • DELETE /blobs/{id} - Remove blob
  • POST /flush - Move memory blobs to disk storage
  • GET /status - Service statistics and metrics

Flush Endpoint

The /flush endpoint moves blobs from memory to disk storage and supports query parameters:

  • limit - Maximum number of blobs to flush (e.g., 100, 1k, 5m, 2b)
  • age - Only flush blobs older than specified duration (e.g., 30s, 5m, 2h, 1d)

Examples:

  • POST /flush - Flush all memory blobs to disk
  • POST /flush?limit=1000 - Flush oldest 1000 blobs to disk
  • POST /flush?limit=5k&age=1h - Flush up to 5000 blobs older than 1 hour

Alternatively, the PUT /blobs/{id}?flush=true can be used to immediately flush a blob to disk.

Blob IDs #

  • Characters: [a-zA-Z0-9._~-] only
  • Length: 4-512 characters
  • Client-generated: No collision detection, use sufficiently random IDs

Eviction Strategy #

  • Memory limits: Checked after every write
  • Memory TTL: Checked hourly
  • Disk limits: Checked every 8 hours
  • Algorithm: Oldest entries evicted first (by last-modified time)
  • Statistics: 7-day eviction history (in-memory only)

CLI use #

Start the blobinator service:

# Basic usage (defaults: memory-only, port 8080)
dart run blobinator serve

# With disk storage
dart run blobinator serve --disk-storage-path ./blobs

# Custom configuration
dart run blobinator serve \
  --port 3000 \
  --mem-items 500k \
  --mem-size 2GiB \
  --mem-ttl 6h \
  --disk-items 50m \
  --disk-size 1TB \
  --disk-ttl 30d \
  --disk-storage-path ./data/blobs

Unit formats:

  • Items: k (thousands), m (millions), b (billions)
  • Size: KiB/KB (kilobytes), MiB/MB (megabytes), GiB/GB (gigabytes)
  • TTL: s (seconds), m (minutes), h (hours), d (days)

HTTP client use #

import 'package:blobinator/blobinator.dart';

final client = BlobinatorClient('http://localhost:8080');

// Store and retrieve bytes
await client.putBytes('my-blob', [1, 2, 3, 4]);
final data = await client.getBytes('my-blob'); // Uint8List?

// Store and retrieve files (streaming)
await client.putFile('large-blob', './input.dat');
await client.getFile('large-blob', './output.dat'); // bool (success)

// Check existence and metadata
final exists = await client.exists('my-blob'); // bool
final size = await client.getSize('my-blob'); // int?
final lastMod = await client.getLastModified('my-blob'); // DateTime?

// Get detailed metadata
final metadata = await client.head('my-blob'); // BlobMetadata?

// Delete blobs
await client.delete('my-blob'); // bool (found and deleted)

// Service status
final status = await client.getStatus(); // ServiceStatus

// Flush memory blobs to disk
final flushed = await client.flush(); // int (number of blobs flushed)
final flushedLimited = await client.flush(limit: 1000); // Flush max 1000 blobs
final flushedOld = await client.flush(age: Duration(hours: 1)); // Flush blobs older than 1 hour
final flushedCombined = await client.flush(limit: 500, age: Duration(minutes: 30)); // Combined

// Clean up
client.close();
0
likes
160
points
42
downloads

Publisher

verified publisheragilord.com

Weekly Downloads

Tools and HTTP service for temporary binary blob storage.

Repository (GitHub)
View/report issues

Topics

#binary #server #tool

Documentation

API reference

License

BSD-3-Clause (license)

Dependencies

args, crypto, http, json_annotation, path, shelf, shelf_router

More

Packages that depend on blobinator