Problems with opus/pcm audio with RTSP translation in WebRTC

[sanek11591]

I'm trying to broadcast an rtsp stream from a camera running on openIPC firmware. and I get big problems with sound in pcm and opus modes. The sound seems to be robotic and cropped, there are no such problems with sound in the MSE broadcast. There are no audio problems in the RTSP stream if you watch it in VLC.

{
  "producers": [
    {
      "id": 8,
      "format_name": "rtsp",
      "protocol": "rtsp+tcp",
      "remote_addr": "192.168.1.76:554",
      "url": "rtsp://root:[email protected]:554/stream=0",
      "sdp": "v=0\r\no=StreamingServer 38990265062388 38990265062388 IN IP4 0.0.0.0\r\ns=RTSP Session\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\na=range:npt=0-\r\nm=video 0 RTP/AVP 96\r\na=control:video\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 profile-level-id=002996; packetization-mode=1; \r\na=framerate:25\r\nm=audio 0 RTP/AVP 98\r\na=control:audio\r\na=rtpmap:98 opus/48000/2\r\n",
      "user_agent": "go2rtc/1.9.8",
      "medias": [
        "video, recvonly, H264",
        "audio, recvonly, OPUS/48000/2"
      ],
      "receivers": [
        {
          "id": 9,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video"
          },
          "childs": [10, 14],
          "bytes": 5004512,
          "packets": 4506
        },
        {
          "id": 11,
          "codec": {
            "channels": 2,
            "codec_name": "opus",
            "codec_type": "audio",
            "sample_rate": 48000
          },
          "childs": [12, 15],
          "bytes": 287666,
          "packets": 2239
        }
      ],
      "bytes_recv": 5373118
    }
  ],
  "consumers": [
    {
      "id": 7,
      "format_name": "webrtc",
      "protocol": "ws+udp",
      "remote_addr": "192.168.1.150:60363 prflx",
      "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 YaBrowser/24.12.0.0 Safari/537.36",
      "medias": [
        "video, sendonly, VP8, VP9, AV1, H264",
        "audio, sendonly, OPUS/48000/2, G722/8000, PCMU/8000, PCMA/8000, L16, PCML"
      ],
      "senders": [
        {
          "id": 10,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video"
          },
          "parent": 9,
          "bytes": 5004512,
          "packets": 4506
        },
        {
          "id": 12,
          "codec": {
            "channels": 2,
            "codec_name": "opus",
            "codec_type": "audio",
            "sample_rate": 48000
          },
          "parent": 11,
          "bytes": 287666,
          "packets": 2239
        }
      ],
      "bytes_send": 5373706
    },
    {
      "id": 13,
      "format_name": "probe",
      "protocol": "http",
      "remote_addr": "[::1]:64116",
      "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 YaBrowser/24.12.0.0 Safari/537.36",
      "medias": [
        "video, sendonly, ALL",
        "audio, sendonly, ALL",
        "audio, recvonly, ANY"
      ],
      "senders": [
        {
          "id": 14,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video"
          },
          "parent": 9
        },
        {
          "id": 15,
          "codec": {
            "channels": 2,
            "codec_name": "opus",
            "codec_type": "audio",
            "sample_rate": 48000
          },
          "parent": 11
        }
      ]
    }
  ]
}

[AlexxIT]

I have same problem with OpenIPC camera. But sound bad in all situations - WebRTC/MSE/RTSP/etc.

[sanek11591]

I have same problem with OpenIPC camera. But sound bad in all situations - WebRTC/MSE/RTSP/etc.

What can be done about it?
Everything works fine in MSE. Everything is fine in the stream received directly from the camera, too. Maybe it's worth digging around to fix it?
Moreover, this applies not only to the OPUS codec, but also to PCM

[AlexxIT]

I thought I just had a bad camera. Or that's just how OpenIPC works with it.

[sanek11591]

I thought I just had a bad camera. Or that's just how OpenIPC works with it.

I've been testing with multiple cameras, do you think this is a problem on the OpenIPC side?

How can I help to fix this problem? Can I run some tests or do I need additional data?

[AlexxIT]

I needs to make some additional tests with my camera.

[sanek11591]

I needs to make some additional tests with my camera.

Thank you, if you need more information from me or a remote connection to the camera, I am always ready.

[sanek11591]

@AlexxIT I recorded a short video for understanding. It's not a big project on the left side also for RTSP translation to WebRTC, go2rtc on the right side. I hope it helps. codec OPUS

expl.mp4

[AlexxIT]

Heh. On the left side is my friend's project. I started with it before I was even familiar with WebRTC and the go language.
Only then did I have a lot of ideas for go2rtc project. I'll check where there might be differences.

[sanek11591]

@AlexxIT

Good afternoon, I just found a link to your friend's project on your project page :)

Today I conducted another experiment with a similar mediamtx project, it has the same problem with opus, but it is missing from pcm, webrtc pion is also present in the project dependencies, could there be a problem somewhere in this library?

[AlexxIT]

RTSPtoWeb also based on pion library. Probably it has some fix for this case.