How LinuxPlay Remote Desktop Works

Note: LinuxPlay is a toy project designed for beginners who want to learn advanced remote desktop streaming concepts. If you're comfortable with the terminal, you can copy and paste the commands below to run LinuxPlay directly, no need to use the GUI launcher (start.py).

1. Capturing the Screen

The host captures the screen using FFmpeg's x11grab input at the specified resolution and framerate.

ffmpeg -f x11grab -framerate 30 -video_size 1920x1080 -i :0

2. Encoding the Video Stream

Captured frames are encoded with either software encoders (like libx264, libx265, or libaom-av1) or hardware-accelerated encoders (such as NVIDIA NVENC or VAAPI).

ffmpeg -c:v h264_nvenc -b:v 8M -preset llhq -f mpegts udp://239.0.0.1:5000

3. Negotiation & Authentication

A TCP handshake on port 7001 ensures both the host and client agree on the encoder type and if used, verifies the password.

# Client sends:
PASSWORD:yourpassword

# Host replies:
OK:h264
  

4. Transmitting the Video and Audio Streams

The encoded video is sent via UDP multicast (239.0.0.1:5000), while audio captured from PulseAudio is streamed on port 6001.

# Video stream:
ffmpeg ... -f mpegts udp://239.0.0.1:5000

# Audio stream:
ffmpeg -f pulse -i default.monitor -c:a libopus -b:a 128k -f mpegts udp://239.0.0.1:6001
  

5. Decoding on the Client

The client decodes the UDP stream using PyAV (which binds to FFmpeg) and uses hardware decoding (NVIDIA NVDEC or VAAPI) if available.

av.open("udp://@239.0.0.1:5000?fifo_size=5000000&overrun_nonfatal=1", options={"hwaccel": "h264_nvdec"})
  

6. Rendering the Video

Decoded frames are converted into images and rendered in a PyQt5 GUI, ensuring smooth playback.

7. Sending Input Events

User input (mouse/keyboard) on the client is sent as UDP control messages (port 7000) to the host, which uses xdotool to simulate these events.

# Example control message:
MOUSE_MOVE 400 300

# Executed on host:
xdotool mousemove 400 300
  

8. Clipboard Sharing

Clipboard content is synchronized between host and client via UDP multicast (port 7002) using xclip on the host and the clipboard API in PyQt5 on the client.

xclip -o -selection clipboard

9. User Interface and Adaptive Bitrate

A GUI launcher (written in PyQt5) allows configuration of host/client options such as encoder/decoder, resolution, framerate, bitrate, audio, password, and adaptive bitrate management.

Summary

• The host captures and encodes video/audio with FFmpeg (using hardware acceleration if available).
• A TCP handshake on port 7001 negotiates settings and handles password authentication.
• Streams are transmitted over UDP multicast (239.0.0.1) on dedicated ports for video (5000), audio (6001), and clipboard (7002).
• The client decodes and renders the video with PyAV and PyQt5.
• User input is sent via UDP (port 7000) and executed with xdotool.
• Clipboard syncing uses UDP and xclip.
• An integrated GUI launcher lets you configure and start the host/client, with optional adaptive bitrate adjustments.