Abstract

How do you design a program that can pilot a self-driving race car? By using machine learning. In this project, you will train your own machine learning model for an autonomous vehicle, the AWS (Amazon Web Services) DeepRacer. You can run your car's machine learning model on a simulated racetrack (Figure 1), or you can purchase a 1/18 scale model vehicle that can race on a physical track (Figure 2).

There are many different machine learning strategies. The AWS DeepRacer uses a type of machine learning called reinforcement learning. Reinforcement learning is similar to training a dog: you give rewards for good behavior and remove rewards for bad behavior. For your dog, the desired behavior might be something like sitting instead of jumping when waiting for a treat. For an autonomous car, the desired behaviors could include things like staying in the middle of a lane, driving below the speed limit, avoiding collisions with other cars, and stopping at red lights.

The unique thing about reinforcement learning is that the model does not "know" in advance what the desired behaviors are—it has to learn through feedback. Other types of learning require more information from the start. For example, a supervised machine learning model intended to categorize images of cats and dogs would first need access to pre-classified sets of images.

A reinforcement learning model, in contrast, learns through trial and error which behaviors are rewarded. This video gives an overview of how reinforcement learning works and why it is a good machine learning strategy for teaching a complex behavior—like driving a race car—that is too hard to explain to a machine step-by-step.

In the AWS DeepRacer console, you will create your own reward function in Python. (You can start out with several simple example functions). You get to decide what kind of behavior you want to reward. For example, do you want to reward your race car for staying close to the center of the track? For driving as fast as possible on straight sections? For avoiding over-steering on turns? A weighted combination of different factors?

Your reward function determines how your car's behavior will evolve over time as it learns. The virtual racing console will show you a 3D simulation of your car on a racetrack as well as a graph indicating how its performance improves over time (Figure 1).

To get started, go to AWS DeepRacer and follow the instructions. You will need to create an AWS account. You can set up a free trial account with a limited amount of storage space and model training time, but you will be required to enter credit card information. Up-to-date pricing information is available on DeepRacer's pricing page, and step-by-step instructions on how to create an account are available from AWS's online support center.

Once you have an AWS account, the best way to get started with DeepRacer is to take the free 90-minute e-learning course offered by AWS. This course will teach you everything you need to know about reinforcement learning in order to get your first machine learning model up and racing on the virtual track. (Note that several videos at the beginning of this course focus on the physical car, so you can skip through those if you only plan on using the simulator.)

Once you have trained an autonomous race car model, you can test it out on different virtual racetracks and even enter it in the virtual racing league to compete against other people.

Here are a few questions to consider if you would like to use your virtual race car for a science project:

• How do models with different reward functions compare to each other? For example, compare a model that prioritizes high speed with one that prioritizes staying close to the center of the track.
• Can you write a reward function that rewards multiple behaviors? What happens if you change the relative amount of reward between the different behaviors?
• How do models with different hyperparameters (see AWS training course) compare against each other?
• How do models trained on one track perform when you test them on other tracks? For example, if your model learns on a track that has gradual turns, can it perform well on a track with sharp turns?
• How well does a virtually trained model perform on the real car? Are there things in the physical world, like bumpiness in the track or variations in lighting, that the virtual model has difficulty accounting for?

Careers

If you like this project, you might enjoy exploring these related careers:

Related Links

• Science Fair Project Guide
• Other Ideas Like This
• Computer Science Project Ideas
• Self-Driving Cars Project Ideas
• Artificial Intelligence Project Ideas
• My Favorites

Cite This Page

General citation information is provided here. Be sure to check the formatting, including capitalization, for the method you are using and update your citation, as needed.

MLA Style

APA Style