Teach a Computer to Recognize Road Signs
Summary
Introduction
How do driverless cars know how to recognize different road signs? They use something called machine learning. In this activity you will teach a computer to recognize different road signs using photos that you take yourself and a tool called Teachable Machine by Google®. The better the training data, the more accurately the program will be able to recognize different types of signs.
Materials
- Smartphone or digital camera
- Access to locations where you can safely take pictures of road signs
- Computer with Internet access
Prep Work
The computer program you will use is called a machine learning model. Before you can teach this program to recognize road signs, you need to take lots of pictures of the signs. These pictures are your training data. You will use them to train the model to recognize the signs.
Choose at least two different types of road signs. Take at least a few dozen pictures of each type of sign. You can even take hundreds of pictures if you want. The more pictures you take, the more data you will have available to help train your model.
Ideally, you should take pictures of signs from different angles, in different locations and with different backgrounds. For example, notice how the picture has a clear blue sky in the background. If you only take pictures of signs against a blue sky, your model might have difficulty recognizing signs in pictures with other backgrounds, like buildings or trees.
After you take all your pictures, load them onto your computer and organize them into folders, with one for each type of sign.
Instructions
Note: Websites can change frequently. If the instructions here do not match exactly what you see on the Teachable Machine website, follow the on-screen instructions on the website. The general process should be similar.
- Go to Teachable Machine and click Get Started.
- Click Image Project.
- Click Standard image model.
- Edit the names of the two Classes in the model. For example, you could name them "Stop sign" and "Curve sign."
- For your first class, under Add Image Samples, upload the pictures of that type of sign from your computer. It may take a while to upload if you have a lot of images.
- Repeat Step 5 to add images for your other class.
- Click Train Model. Wait for the model to finish training.
- Under Preview, change the selection in the drop-down menu from Webcam to File.
- Under Preview, upload one of your pictures of a road sign.
Look at the Output section. Does your model correctly classify the picture?
- Now put your model to the test. It should work well if you simply re-use images that you already used in the training data. What happens if you try new images that your model has never seen before? You can go take more pictures or search for pictures of road signs online and use those.
How well does your model work with new images? What if you try images with different backgrounds, in different weather or at different times of day?
- Try to improve your model by giving it more training data. For example, maybe your model works well to identify signs on sunny days but does not work well for pictures taken on cloudy, rainy or snowy days. In this case, try searching for pictures taken in a variety of weather conditions. Upload them and re-train your model. Does this improve your model's performance?
What Happened?
When you train your model, it learns to identify the two classes of images. When you upload a test image, the model uses what it has learned to assign a class to the image, along with a level of certainty.
If you trained your model on two signs that are visually very different (for example, an octagonal red stop sign and a diamond-shaped yellow curve sign), the model might work very well, even without a lot of training data. Notice that the model identifies the stop sign with 99% certainty.
Your model might have more trouble with similar-looking signs, like "curve ahead" and "bump" signs, which are both yellow and diamond-shaped. In this case, note how the model only assigns an 84% probability that the test image shows a bump sign. You could improve this model by loading more training photos that show signs in a wider variety of environments.
screenshot shows uploaded training images of curve signs and bump signs on the left. On the right is an output preview showing the results for a photo of a bump sign. The program has classified the image as a bump sign with 84% certainty.
Digging Deeper
There are many different types of machine learning. In this activity, you used a type called supervised learning, where the model is given a set of pre-classified images. This type of learning requires "supervision" by a human who knows how to correctly categorize the images in advance. Watch this video to learn more about supervised learning:
Supervised learning models can be subject to biases present in their training data, even if the bias is not intentional on your part. For example, as mentioned earlier, if you only show your model pictures of road signs with blue sky in the background, it might become biased toward blue-sky pictures and fail to properly identify signs with other backgrounds. This video discusses machine learning and human bias:
By providing a lot of training data with signs in a wide variety of environments and backgrounds, you can help avoid bias in your model.
Ask an Expert
For Further Exploration
- Try training a model that can identify three or more types of signs. You can also include a class for pictures with no sign.
- Try training a model to identify other objects that an autonomous car might encounter when driving, like red, yellow and green lights, pedestrians, cyclists or other cars.
- Click the Advanced drop-down under the Train model button to access several different parameters that affect how your model learns. Hover your mouse over the question mark icon next to each parameter to learn more about it. Try adjusting these parameters to see how they influence your model.
