socket_003:Sending and Receiving Video over Sockets

In this tutorial, we will explore how to send and receive video frames over a network using Python sockets. We will break down both the server and client code to understand how video data is transmitted and displayed in real-time.

Overview

1. Server Code: The server listens for incoming connections, receives video frames from the client, and displays them.
2. Client Code: The client reads video frames from a video file, sends them to the server, and displays them locally.

Server Code Breakdown

import socket
import cv2
import pickle
import struct

def recv_exact(sock, size):
    """Receive exactly `size` bytes from the socket."""
    data = b""
    while len(data) < size:
        packet = sock.recv(size - len(data))
        if not packet:
            return None
        data += packet
    return data

# Create a TCP socket
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.bind(('localhost', 12345))
server_socket.listen(1)
print("Waiting for client connection...")

client_socket, addr = server_socket.accept()
print("Connected to:", addr)

try:
    while True:
        # Receive the frame size (4 bytes)
        packed_size = client_socket.recv(4)
        if not packed_size:
            break
        frame_size = struct.unpack("I", packed_size)[0]

        # Receive the complete frame data
        frame_data = recv_exact(client_socket, frame_size)
        if frame_data is None:
            break

        # Deserialize and decode the frame
        frame = cv2.imdecode(pickle.loads(frame_data), cv2.IMREAD_COLOR)

        # Display the frame
        cv2.imshow("Server Camera", frame)

        if cv2.waitKey(1) & 0xFF == ord('q'):
            break

except Exception as e:
    print("Error:", e)

finally:
    cv2.destroyAllWindows()
    client_socket.close()
    server_socket.close()
    print("Connection closed.")

Key Functions and Concepts in Server Code

1. recv_exact(sock, size):

• This function ensures that exactly size bytes are received from the socket. It is necessary because recv() may not always return the exact number of bytes requested.

Parameters:

• sock: The socket object.
• size: The number of bytes to receive.

2. socket.socket(socket.AF_INET, socket.SOCK_STREAM):

• Creates a TCP socket.
• AF_INET specifies the address family (IPv4).
• SOCK_STREAM specifies the socket type (TCP).

3. server_socket.bind(('localhost', 12345)):

• Binds the socket to the localhost on port 12345.

4. server_socket.listen(1):

• Listens for incoming connections. The argument 1 specifies the maximum number of pending connections.

5. client_socket, addr = server_socket.accept():

• Accepts a connection from a client. Returns a new socket object (client_socket) and the address of the client (addr).

6. struct.unpack("I", packed_size)[0]:

• Unpacks the frame size from the received 4-byte data. "I" specifies an unsigned integer.

7. cv2.imdecode(pickle.loads(frame_data), cv2.IMREAD_COLOR):

• Deserializes the frame data using pickle.loads() and decodes it into an image using cv2.imdecode().

8. cv2.imshow("Server Camera", frame):

• Displays the received frame in a window titled “Server Camera”.

9. cv2.waitKey(1):

• Waits for 1 millisecond for a key event. If the ‘q’ key is pressed, the loop breaks.

Client Code Breakdown

import socket
import cv2
import pickle
import struct
import time

client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client_socket.connect(('localhost', 12345))

video_path = 'mix.mp4'  # Replace with your video file path
cap = cv2.VideoCapture(video_path)

fps = cap.get(cv2.CAP_PROP_FPS)
print(f"Video FPS: {fps}")

try:
    while True:
        start_time = time.time()
        ret, frame = cap.read()
        if not ret:
            break

        # Resize and compress the frame
        frame = cv2.resize(frame, (640, 480))
        ret, buffer = cv2.imencode('.jpg', frame, [cv2.IMWRITE_JPEG_QUALITY, 80])
        data = pickle.dumps(buffer)
        message_size = struct.pack("I", len(data))
        client_socket.sendall(message_size + data)

        cv2.imshow("Client Camera", frame)
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break

        time_to_sleep = max(0, (1 / fps) - (time.time() - start_time))
        time.sleep(time_to_sleep)

except KeyboardInterrupt:
    print("Client stopped.")

finally:
    cap.release()
    client_socket.close()
    cv2.destroyAllWindows()

Key Functions and Concepts in Client Code

1. client_socket.connect(('localhost', 12345)):

• Connects the client socket to the server at localhost on port 12345.

2. cv2.VideoCapture(video_path):

• Opens the video file specified by video_path.

3. cap.get(cv2.CAP_PROP_FPS):

• Retrieves the frames per second (FPS) of the video.

4. cap.read():

• Reads the next frame from the video. Returns ret (a boolean indicating success) and frame (the actual frame).

5. cv2.resize(frame, (640, 480)):

• Resizes the frame to 640x480 pixels.

6. cv2.imencode('.jpg', frame, [cv2.IMWRITE_JPEG_QUALITY, 80]):

• Encodes the frame as a JPEG image with 80% quality.

7. pickle.dumps(buffer):

• Serializes the encoded frame data into a byte stream.

8. struct.pack("I", len(data)):

• Packs the length of the serialized frame data into a 4-byte unsigned integer.

9. client_socket.sendall(message_size + data):

• Sends the frame size followed by the frame data to the server.

Conclusion

This tutorial demonstrated how to send and receive video frames over a network using Python sockets. The server receives and displays the frames, while the client reads frames from a video file, sends them to the server, and displays them locally. By understanding the key functions and concepts, you can build more complex video streaming applications.