[HELP] Recording videos using the scrcpy server jar results in abnormal video duration and playback speed. Any suggestions?

[codematrixer]

Environment

• OS: macOS
• Scrcpy version: 2.1.1
• Device model: HUAWEI SEA-AL10

Problem Description

Hello, could you please help me with this issue? I’d greatly appreciate it.

I implemented a screen recording client in Python, which retrieves the H.264 video stream provided by the scrcpy server. Then, I use FFmpeg to convert it into an MP4 file. However, the video duration and playback speed are incorrect. For instance, in this demo, the screen recording lasts for 10 seconds, but the final video is only 2 seconds long. I’m seeking assistance.

Below is my demo code:

import os
import socket
import struct
import subprocess
import time
import threading
import queue
from typing import Optional, Tuple

from adbutils import AdbDevice, adb, Network, AdbError


class ScrcpyClient:
    SOCKET_TIMEOUT = 5.0

    def __init__(
        self,
        serial: str,
        max_size: int = 1080,
        bitrate: int = 1300000,
        max_fps: int = 30,
        lock_screen_orientation: int = -1,
        encoder_name: Optional[str] = None,
        connection_timeout: int = 6000,
        output_file: str = "recording.mp4"
    ):

        self.max_size = max_size
        self.bitrate = bitrate
        self.max_fps = max_fps
        self.lock_screen_orientation = lock_screen_orientation
        self.connection_timeout = connection_timeout
        self.encoder_name = encoder_name
        self.output_file = output_file

        self.serial = serial
        self.device: AdbDevice = adb.device(serial=serial)

        self.device_name: Optional[str] = None
        self.resolution: Optional[Tuple[int, int]] = None
        self.alive = False
        self.video_socket: Optional[socket.socket] = None
        self.control_socket: Optional[socket.socket] = None
        self.scrcpy_process: Optional[subprocess.Popen] = None
        self.recording_thread: Optional[threading.Thread] = None

    def _deploy_server(self):

        self.device.sync.push("scrcpy-server-v2.1.1", "/data/local/tmp/scrcpy-server-v2.1.1.jar")

        commands = [
            "adb",
            "-s",
            self.serial,
            "shell",
            "CLASSPATH=/data/local/tmp/scrcpy-server-v2.1.1.jar",
            "app_process",
            "/",
            "com.genymobile.scrcpy.Server",
            "2.1.1",
            "log_level=info",
            f"max_size={self.max_size}",
            f"max_fps={self.max_fps}",
            f"video_bit_rate={self.bitrate}",
            "tunnel_forward=true",
            "send_frame_meta=false",
            "control=true",
            "audio=false",
            "stay_awake=true",
            "power_off_on_close=false",
            "clipboard_autosync=false"
        ]

        print(" ".join(commands))
        self.scrcpy_process = subprocess.Popen(commands, stdout=subprocess.PIPE, shell=False, preexec_fn=os.setsid)

    def _init_server_connection(self):
        """
        Connect to android server, there will be two sockets, video and control socket.
        This method will set: video_socket, control_socket, resolution variables
        """
        print("Connecting video socket & control socket")
        for i in range(self.connection_timeout // 100):
            try:
                self.video_socket = self.device.create_connection(
                    Network.LOCAL_ABSTRACT, "scrcpy"
                )
                self.control_socket = self.device.create_connection(
                    Network.LOCAL_ABSTRACT, "scrcpy"
                )
                print(f"[{self.serial}] Connected scrcpy video socket and control socket")
                break
            except AdbError:
                print(f"[{self.serial}] waiting connect...")
                time.sleep(0.1)
        else:
            raise ConnectionError("Failed to connect scrcpy-server after 6 seconds")

        self.video_socket.settimeout(self.SOCKET_TIMEOUT)
        self.control_socket.settimeout(self.SOCKET_TIMEOUT)

        dummy_byte = self.video_socket.recv(1)
        if not len(dummy_byte) or dummy_byte != b"\x00":
            raise ConnectionError("Did not receive Dummy Byte!")

        # Read the device name
        device_name_bytes = self.video_socket.recv(64)
        self.device_name = device_name_bytes.decode("utf-8").rstrip("\x00")
        if not len(self.device_name):
            raise ConnectionError("Did not receive Device Name!")
        print("Device Name: " + self.device_name)

        # Read the new protocol format
        header = self.video_socket.recv(12)
        if len(header) != 12:
            raise ConnectionError("Did not receive the expected header!")

        # Unpack the values from the header
        codec_id, initial_width, initial_height = struct.unpack(">III", header)

        print(f"Serial: {self.serial}")
        print(f"Codec ID: {codec_id}")
        print(f"Initial Video Width: {initial_width}")
        print(f"Initial Video Height: {initial_height}")

        self.resolution = (initial_width, initial_height)

    def _socket_reader(self, video_queue: queue.Queue):
        """
        Reads data from video_socket and places it into the queue.
        """
        while self.alive:
            time.sleep(.001)
            try:
                data = self.video_socket.recv(4096)
                if not data:
                    break
                video_queue.put(data, timeout=0.1)
            except socket.timeout:
                print("Socket timeout during recording.")
            except Exception as e:
                print(f"Error in socket reader: {e}")

        self.alive = False  # Stop recording if the reader stops

    def _stream_writer(self, video_queue: queue.Queue):
        """
        Reads data from the queue and writes it to ffmpeg.
        """
        # Use fps based on scrcpy settings
        ffmpeg_command = [
            "ffmpeg",
            "-y",  # Overwrite output file without asking
            "-f", "h264",  # Input format is raw H.264
            "-i", "pipe:0",  # Read input from stdin
            "-c:v", "copy",  # Copy codec (no re-encoding)
            self.output_file,  # Output file
        ]

        start_time = int(time.time())

        process = subprocess.Popen(ffmpeg_command, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
        try:
            while self.alive or not video_queue.empty():
                try:
                    data = video_queue.get(timeout=0.1)  # Reduce wait time
                    process.stdin.write(data)
                except queue.Empty:
                    continue
        except Exception as e:
            print(f"Error in stream writer: {e}")
        finally:
            process.stdin.close()
            process.wait()
            print(f"Recording finished and saved to {self.output_file}, duration: {int(time.time() - start_time)} seconds")

    def _record_stream(self):
        """
        Main recording function that coordinates the socket reader and stream writer.
        """
        video_queue = queue.Queue()

        # Start reader and writer threads
        reader_thread = threading.Thread(target=self._socket_reader, args=(video_queue,))
        writer_thread = threading.Thread(target=self._stream_writer, args=(video_queue,))

        reader_thread.start()
        writer_thread.start()

        # Wait for threads to finish
        reader_thread.join()
        writer_thread.join()

    def start(self):
        if self.alive:
            raise RuntimeError("Already started")

        self._deploy_server()
        self._init_server_connection()
        self.alive = True

        # Start recording
        self.recording_thread = threading.Thread(target=self._record_stream)
        self.recording_thread.start()

    def stop(self):
        self.alive = False

        if self.recording_thread:
            self.recording_thread.join()

        if self.video_socket:
            self.video_socket.close()
        if self.control_socket:
            self.control_socket.close()


# Example Usage
if __name__ == "__main__":
    client = ScrcpyClient(serial="6HJDU20506005768")
    try:
        client.start()
        time.sleep(10)  # Record for 10 seconds
    finally:
        client.stop()

Run the demo

pip3 install adbutils

python3 demo.py 

log

adb -s 6HJDU20506005768 shell CLASSPATH=/data/local/tmp/scrcpy-server-v2.1.1.jar app_process / com.genymobile.scrcpy.Server 2.1.1 log_level=info max_size=1080 max_fps=30 video_bit_rate=1300000 tunnel_forward=true send_frame_meta=false control=true audio=false stay_awake=true power_off_on_close=false clipboard_autosync=false
Connecting video socket & control socket
[6HJDU20506005768] waiting connect...
[6HJDU20506005768] waiting connect...
[6HJDU20506005768] waiting connect...
[6HJDU20506005768] Connected scrcpy video socket and control socket
Device Name: SEA-AL10
Serial: 6HJDU20506005768
Codec ID: 1748121140
Initial Video Width: 496
Initial Video Height: 1080
Recording finished and saved to recording.mp4, duration: 10 seconds

[rom1v]

I implemented a screen recording client in Python, which retrieves the H.264 video stream provided by the scrcpy server.

A H.264 raw stream has no timestamp information, so with send_frame_meta=false you don't have enough information to mux it correctly.

[codematrixer]

Even though I set send_frame_meta=true, the issue persists. I don't have much experience with video and h264, so I'm not sure what steps to take. Could you offer guidance on how to fix this?

[rom1v]

There is no way you can capture the H.264 stream from scrcpy-server, save it to a file "as is" and use the ffmpeg command line tool to convert it (unless you develop a FFmpeg demuxer reading the input stream in the scrcpy format).

You must use the ffmpeg library and mux the packets programmatically (the same way as recorder.c does).

[codematrixer]

Got it. I will refer to the recorder.c, thank you very much.

[rom1v]

For simplicity, you should check recorder.c from a verion older than 2.0 (when audio was introduced), where it simply recorded the single video stream. Checkout v1.25 for example.

[codematrixer]

For simplicity, you should check recorder.c from a verion older than 2.0 (when audio was introduced), where it simply recorded the single video stream. Checkout v1.25 for example.

THanks a lot.

[yume-chan]

-use_wallclock_as_timestamps 1 (before -i) can be used as a workaround. It uses frame receive time as timestamp, so not as precise as server timestamp.

[codematrixer]

-use_wallclock_as_timestamps 1 (before -i) can be used as a workaround. It uses frame receive time as timestamp, so not as precise as server timestamp.

After adding this parameter, the slow-motion and duration issues were indeed resolved. However, the video is still stutter and doesn't match the smooth performance of scrcpy --record