Skip to main content

Log in to comment and vote

Comments2

  • Riki DBag

    •

    Sep 28

    Here is a technical breakdown of why this happens with Hedra Realtime Avatars (Beta) and how to resolve it:
    1. Architectural Differences: Batch Diffusion vs. Real-Time Streaming

    • Standard Video Pipeline (Character-1 / Character-2): Offline generation relies on full frame-by-frame latent diffusion inference. Because latency is not a factor, the pipeline can run full UNet passes at high spatial resolutions (1080p+) and encode the final video using high-bitrate containers (H.264/HEVC with low CRF values).

    • Realtime Avatars (Beta): To achieve glass-to-glass latencies under 500ms for live interaction, the realtime pipeline utilizes a compressed/distilled neural rendering model running at a lower native tensor resolution (typically 512x512). The output is streamed live over WebRTC.

    2. High-Frequency Spatial Detail & Codec Artifacts

    • Quantization & Bitrate Limits: Your source image contains an extreme density of high-frequency spatial detail (knitted wool patterns, individual hair strands, micro-snowflakes).

    • WebRTC Adaptive Encoding: Real-time WebRTC encoders (H.264/VP8) dynamically adjust Quantization Parameters (QP) to preserve motion fluidity and prevent packet drop/buffering. High-frequency details consume massive spatial bandwidth; when constrained by real-time bitrate caps, the codec aggressively drops high-frequency DCT coefficients, resulting in heavy macroblocking, temporal flickering, and pixelation.

    • Resampling Aliasing: When Hedra resamples your ultra-high-res source image down to the realtime model's internal input tensor, fine recurring textures (like knitwear) introduce spatial aliasing before neural inference even begins.

    How to Fix / Optimize

    1. Optimize the Source Image for WebRTC:
      Reduce High-Frequency Noise: Avoid source images with heavy textures (wool knitwear, snow/grain, fine mesh). Smooth surfaces, clean studio lighting, and soft backgrounds stream significantly cleaner over real-time codecs.
      Pre-Filter High-Detail Areas: If you want to use this specific character, run a subtle Gaussian blur (0.5–1px) or low-pass spatial filter on the clothing and background textures in Photoshop to soften high-frequency details. This frees up bandwidth for facial features (eyes/mouth).
      Pre-Crop to 1:1: Crop the image to a 1:1 square ratio at 1024x1024 before uploading so you control the downsampling algorithm rather than relying on automated browser/server cropping.

    2. Choose the Right Pipeline for the Output:
      Use Realtime Avatars exclusively for interactive WebRTC live sessions where low latency takes priority over image fidelity.
      For non-interactive, high-definition promotional videos or showcases, use the standard Video / Character Generation pipeline in batch mode.

  • djgruber

    •

    Nov 30, 2025

    •

    Merged request

    •

    1 vote

    Low quality with Realtime Avatars... :(

    I have seen many online examples with the new Realtime Avatars, and they look stunning.

    Mine looks terrible. :(

    I have tried several HI RES, clear image as my avatar. But when chatting with my avatar the quality is terrible, pixelated. :( I have a screaming fast internet… so am I doing something wrong? HEDRA server traffic, or compression?

    It may be hard to tell in my screenshot, but the avatar looks horribly pixelated. :(

    Please advise… hoping I can get these looking good! :)

    thx!
    DJ