How to Decode in Wasm/Web?

[R2rho]

First off, nice work so far, I'm trying to integrate this into my app. However, I'd like to stream the frames to a web app to render the video stream. Currently only RgbFormat seems to return Err when I try to decode the buffer (using leptos for web rendering, implemented my own type to serialize the Buffer via websocket):

let packet = camera_feed.get();

        // Convert the Media Packet to a raw Buffer
        let raw_buffer: Buffer = packet.into();

        // Decode the image to RGB format
        if let Err(err) =  raw_buffer.decode_image::<RgbFormat>() {
             log(format!("ERROR: {}",e));
        }

This is producing this error in the browser console:

INFO: "ERROR: This operation is not implemented yet: Not available on WASM"

From what I can tell the only supported decoder is RgbFormat. Here is my camera constructor:

pub async fn connect_to_camera(config: CamConfig) -> Result<Camera, NokhwaError> {
    println!("Connecting to camera!");

    let index = CameraIndex::Index(0);
    
    let requested: RequestedFormat = RequestedFormat::new::<RgbFormat>(RequestedFormatType::AbsoluteHighestFrameRate);
    let camera = Camera::new(index, requested)?;

    Ok(camera)
}

Is there a recommended way that you would suggest achieving a web stream of a nokhwa frame capture? I've tried going about this a few different ways and feel stuck.

Originally posted by @R2rho in #86

[l1npengtul]

AFAIK, the browser returns an already decoded format for you, so you can use the buffer directly in e.g. ImageBuffer.

[RReverser]

Even better, if you just want to display the video stream, it's better to bypass copying each frame to Wasm / Rust and back entirely and let the browser do it all for you. It will be much more efficient. https://developer.mozilla.org/en-US/docs/Web/API/MediaStream_Recording_API/Recording_a_media_element