TCP Protocol

Reinforced race TCP protocol

The learning algorithm communicates with either the virtual environment or the model race car via TCP in a binary format. All network traffic has network byte order (big endian).

Virtual Environment / Model Car (Server)

Both the virtual environment and the model car open a TCP server and listen for incoming connections from the Reinforcement Algorithm. By default, this is port 2851. Only a single client should be connected at a time.

It responds to the following protocol

Read Sensors

Responds with whether the car has been disqualified since the last read sensors request, whether it reached the finish line and the current image as specified from the front camera.

Receive

struct {
  byte requestType = 1;
  int width;
  int height;
}

Respond

struct {
  bool disqualified;
  bool finished;
  int velocity; // float encoded as realVelocity * 2^16
  byte image_data[width*height]; /* each pixel is one byte grayscale
                                    in the shape [height, width]
                                    encoded C-Style (last axis running fastest) */
}

This request should only be dealt with as soon as the car is in a valid state, i.e. it has been positioned by a human and is ready to go or has driven back to its previous position and therefore reports disqualified = true in this request.

Write action

This request executes the same given action until the next action is requested.

Receive

struct {
  byte requestType = 2;
  int actionVertical; // -1 for backwards, 0 for standing still, +1 for forward
  int actionHorizontal; // -1 for left, 0 for straight, +1 for right
}

No response

struct {}

Reinforcement Learning Algorithm (Client)

As soon as the python module is started up it connects to the respective server and starts sending any of the previously given requests.