Neural network controlled self-driving (RC) car, inspired by ML-class

Link to his blog post with construction details, schematics and code can be found here: How David Singleton built a neural network controlled self-driving (RC) car!

Link to his G+ post and comments

Updated January 2nd: the source code is now open source and available on github.

Video of car in action: http://www.youtube.com/watch?v=ndSiW9Zmd6g

The system consists of:

• Android phone — mounted on the car, captures video frames of the road ahead using its built-in camera at ~15 fps. An app running on the phone connects to a server running on a laptop computer via wifi and streams 176x144 grayscale video frames across the connection.

• Computer — runs a little Java app called "Driver" which acts as both a TCP server, receiving streamed image frames from the phone and a user interface allowing a human driver to control the car with the cursor keys or mouse. In record mode, the video frames are saved to disk, labelled with the current control input coming from the human driver. The neural network is trained using these labelled frames in a separate environment on the computer. Trained parameters are saved out to files which are in turn read by the Driver app... which in auto mode can feed incoming video frames directly to the neural network and steer according to its predictions, by sending instructions over a serial interface connected to an...

• Arduino Uno — connected to the computer via USB and hacked to connect to and simulate keypresses on the car's radio controller PCB (as described below).