voxa_beauty #

Real-time beauty for voxa_rtc_engine, on the camera capture path — what the streamer sees is what the room receives.

Smoothing and tone are free. A Beauty key adds colour filters, face shaping, makeup looks, stickers, and a cloud catalog that grows without an app update.

dependencies:
  voxa_beauty: ^0.4.7

That is the whole integration. No gradle edit, no MainActivity change, no ProGuard rule, no native code — the filter registers itself.

final beauty = VoxaBeauty(engine);
await beauty.enable();
await beauty.setRetouch(smooth: 0.5, white: 0.4);

What you get #

Gate your control on beauty.isSupported rather than assuming. It is false on platforms this package does not cover yet, so you show no toggle instead of one that does nothing.

Retouch #

Skin is found by colour, not by a face-shaped mask, so hair, brows, eyes and the room behind you stay exactly as they were — and no face tracking has to run for the free controls. Brightening is a gamma lift rather than a push toward white: mid-tones come up, black stays black, a lit forehead never blows out.

await beauty.setRetouch(smooth: 0.5, white: 0.4);
await beauty.setRetouch(slim: 0.2, enlarge: 0.2, narrow: 0.1);  // needs a key

All controls are 0..1 and take effect on the next frame, so they are cheap to drive from a slider.

setSmoothing, setBrightness, setWarmth and setSaturation are the same four controls under their 0.3 names, so an app written against 0.3 keeps working.

Colour filters #

Five filters ship inside the package, so the first frame never waits on the network.

final bundles = await VoxaBeautyBundles.builtIn();
final peach = bundles.firstWhere((b) => b.id == 'fresh_peach');

await beauty.setFilter(peach, strength: 0.7);
await beauty.setFilterStrength(0.4);   // dial it back live
await beauty.setFilter(null);          // off

Looks and stickers #

Two makeup looks and five stickers ship with the package. A look follows the mouth and eyes rather than sitting on top of them; a sticker hangs off face landmarks, and some react — a part shows while the mouth is open, or plays once on a blink. The face is tracked in every frame the camera produces, so a sticker stays on it through a turn or a shake rather than catching up after.

await beauty.setMakeupLook(bundles.firstWhere((b) => b.id == '604zc'), strength: 0.8);
await beauty.setSticker(bundles.firstWhere((b) => b.id == '2676'));

The cloud catalog #

Over eight hundred more effects live in the catalog and need no app update.

final catalog = await VoxaBeautyCatalog.load();

for (final item in catalog.stickers) {
  // item.name, item.category and item.thumbUrl are what a picker needs
}

final bundle = await catalog.stickers.first.download(
  onProgress: (received, total) => setState(() => _progress = received / total),
);
await beauty.setSticker(bundle);

Effects are cached once installed, and VoxaBeautyBundles.installed() lists everything on the device, built-in and downloaded, so one picker can show it all.

A key can be limited to a chosen set of effects, so what your app lists is the slice picked for you rather than everything published. Catalog effects are encrypted and open only for an app whose key is live, so a copy of the file on its own is nothing. A download that is refused says why, and there is a daily download limit per device — both are on the key's card in the console.

With another engine #

The effects run inside voxa_rtc_engine's own capture path, where the frame never leaves the GPU. An engine that will not lend out its capture path — Agora among them — can still have them: this package runs the camera instead, and hands finished frames back.

final beauty = VoxaBeauty.standalone();   // no engine to attach to
final camera = VoxaBeautyCamera();

await beauty.setLicenseKey('vbk_…');
await beauty.setRetouch(smooth: 0.5, white: 0.35);

// Agora: publish a track of our own, so the camera can be released.
// Do not call setExternalVideoSource — a custom track needs only its id.
final trackId = await agora.createCustomVideoTrack();
await agora.enableLocalVideo(false);
await agora.updateChannelMediaOptions(ChannelMediaOptions(
    publishCameraTrack: false,
    publishCustomVideoTrack: true,
    customVideoTrackId: trackId));

await camera.start();                     // camera.preview is a widget

camera.frames.listen((f) {
  agora.getMediaEngine().pushVideoFrame(
    videoTrackId: trackId,
    frame: ExternalVideoFrame(
      type: VideoBufferType.videoBufferRawData,
      format: VideoPixelFormat.videoPixelI420,
      buffer: f.data,
      stride: f.width,
      height: f.height,
      rotation: f.rotation,
      timestamp: f.timestampMs,
    ),
  );
});

Two things there are not optional, and both fail quietly. The camera can only be held by one thing at a time, so the engine has to let go of it before this package can open it — for Agora that means enableLocalVideo(false), and stopPreview() is not enough. But that also stops the camera track carrying anything, which is why the frames go to a track of their own instead. Get one without the other and you have either a frozen picture or a room watching an unfiltered one.

Leave setExternalVideoSource out. It is Agora's older single-source API, and Agora does not switch sources inside a channel. Call it and switching the effects on during a live session leaves the room frozen on the last camera frame while every frame you push is quietly dropped — with no error anywhere. A custom track needs only its id: create it, publish it, push to it.

Media options belong to a channel, so apply them again after joining if beauty was switched on before.

Set the encoder configuration, or the room sees half the frames. Agora's default is 960x540 at 15 fps and it applies to an external video source exactly as it does to Agora's own camera. The camera captures 30, the preview shows 30, and the encoder throws half of them away — which reads as "the effects made the video slow" when the effects had nothing to do with it. Measured on a Pixel 6, same build, one call apart:

Then run the camera at the encoder's rate. VoxaBeautyCamera.start(fps:) takes the number; give it the same one you gave the encoder. A camera at 30 feeding an encoder set to 24 makes Agora drop one frame in six, and dropped frames never leave evenly — the host preview, which shows every frame, looks fine while the room stutters. Measured on the same call: a 29 fps camera into a 24 fps encoder had viewers decoding anywhere from 22 to 33 fps from one second to the next; the camera at 24 into the same encoder gave them a steady 24, and used less bandwidth doing it.

await agora.setVideoEncoderConfiguration(const VideoEncoderConfiguration(
    dimensions: VideoDimensions(width: 540, height: 960),
    frameRate: 24,
    degradationPreference: DegradationPreference.maintainFramerate));

Pick the number deliberately: 24 roughly doubles the bitrate of 15, which a multi-seat room pays once per publisher.

Nothing else in the app may start the camera while the effects are on. This is the one that bites late, because it looks harmless until you are in a call. camera.start() holds the camera; an app whose go-live path then calls enableLocalVideo(true) or startPreview() has two components reaching for one device. Outside a channel the engine's attempt just fails and you see nothing wrong. Inside a channel the engine wins — this package's capture session is closed under it, the preview stops on its last frame, and the room loses the video. Ask before you start capture:

if (!camera.isRunning) {
  await agora.enableLocalVideo(true);
  await agora.startPreview();
}
// the camera is running either way

The package no longer suffers this in silence. It notices, reports through cameraStatus, and keeps trying to open the camera again:

camera.cameraStatus.listen((s) {
  switch (s.state) {
    case VoxaBeautyCameraState.interrupted:   // frozen; coming back
    case VoxaBeautyCameraState.recovered:     // frames again
    case VoxaBeautyCameraState.failed:        // still held by whatever took it
  }
});

A brief interruption — backgrounding the app, something borrowing the camera for a moment — recovers on its own. failed means it did not, and is the point to tell the user or to turn the effects off and give the camera up.

This route costs more, and it is worth knowing why. Turning a GPU texture into bytes an engine will accept is a copy per frame. On a Pixel 6 the same effects hold the camera's full rate through voxa_rtc_engine and around 24 fps this way. Frames are handed over as I420 rather than RGBA, which halves what crosses the channel, and one that arrives while the last is still being delivered is dropped rather than queued.

The camera needs a WebRTC media library in the app. Apps on voxa_rtc_engine already have one.

Beauty key #

Create a key under Beauty in the console and hand it over once.

await beauty.setLicenseKey('vbk_…');
beauty.licenseStatus.listen((status) {
  if (status.isRefused) debugPrint('beauty key refused: ${status.code}');
});

The key binds to the first app that uses it, up to the number of apps it allows. Bound apps are listed in the console, where the key — or one app — can be switched off, and switching one off stops the next download rather than waiting out the day.

A refused key is never silent. licenseStatus carries the reason, and so does logcat:

Why it costs so little #

The filter runs in the graphics context the capturer and the hardware encoder already share. A frame arrives as a GPU texture, goes through the shader passes, and leaves as a GPU texture — no readback, no CPU copy, no pixel ever touching system memory, and nothing crossing into Dart.

Measured on a Pixel 6 at 640×360, cycling through every effect in the catalog in turn: 30 fps, held for the whole run — the camera's own capture rate, with a colour filter, retouch and a sticker on the frame. Face tracking runs on its own thread and drops frames rather than queueing them, so capture never waits for it. The camera itself slows in poor light; the beauty pass does not add to that.

There is no per-frame millisecond figure here on purpose. GL commands are queued rather than executed, so timing the draw call measures how long it took to submit work, not what the GPU spent on it. Sustained capture rate is the honest number.

Platforms #

Documentation #

Full setup, the catalog API and the console walkthrough are at voxartc.com/docs/beauty.

_{The face in these images is computer-generated and is not a real person. Every effect shown on it was produced by this package's own shaders; the panel is the example app.}