Explore Perceptrons: A Simple Type of Neural Network

Abstract

Have the phrases "machine learning" or "neural network" caught your interest, but you are not really sure how to use them in a science project? This project is a great place to start for beginners. You will learn about a very simple type of neural network—a computer program modeled after the human brain—called a perceptron. The perceptron can make a simple yes/no decision based on several inputs. You can "train" your perceptron to improve its decision-making abilities. Try it and get started on the path to learning about machine learning!

If you do not think you are ready for computer programming, you can also check out a version of this project where you do the math by hand.

Summary

Areas of Science

Artificial Intelligence

Difficulty

Method

Engineering Design Process

Time Required

Very Short (≤ 1 day)

Prerequisites

None

Material Availability

Readily available

Cost

Very Low (under $20)

Safety

No issues

Credits

Ben Finio, PhD, Science Buddies

Science Buddies is committed to creating content authored by scientists and educators. Learn more about our process and how we use AI.

https://www.youtube.com/watch?v=SV2huMdNs9c

Objective

Write a computer program that acts like a very simple neural network that can make yes/no decisions based on certain inputs.

Introduction

Imagine that you are trying to decide whether or not to go to the playground. You can make this decision because your brain is full of billions of neurons, or cells that can send signals to each other. Your neurons gather information (inputs) from your senses and help your brain process this information to decide on outputs or actions. What if you wanted a computer to make the same decision? Having a computer "think" like a human is a field of science called artificial intelligence (AI). Sometimes it is hard to program a computer to tell it exactly what to do. Instead, we can "teach" a computer to accomplish something without telling it exactly what to do first. The computer can learn through trial and error, just like a human baby learning to walk or talk. This process is called machine learning (ML).

When deciding whether to go to the playground, your brain may use inputs like, is it sunny out? Are your friends going? Are you done with your homework? Your decision would depend on the answers to these questions. In this project you will explore the basics of how AI can make very simple decisions, without needing to do any computer programming yourself.

You can represent a decision-making process with an algorithm, or a set of instructions for a computer to follow. An algorithm can have inputs (information like whether or not it is sunny) and outputs (like the yes/no decision to go to the playground). You do not need to write any computer code to design an algorithm. You can just write down the instructions in everyday language. For example, you could say "if at least two of the inputs are yes, then I will go to the playground." You could also represent your algorithm with a diagram like Figure 1. If it is sunny out, your friends are going, but you are not done with your homework, this algorithm would tell you go to the playground because at least two of the inputs are yes.

Image Credit: Ben Finio, Science Buddies / Science Buddies

Aria text: a circle labeled 'check if at least 2 inputs are yes.' Three input arrows point toward the circle, each with yes/no options. Input 1: is it sunny out? Input 2: are my friends going? Input 3: Am I done with my homework? An output arrow, should I go to the playground? (yes/no) points away from the circle.

Figure 1. Diagram representing an algorithm to decide whether or not to go to the playground.

That simple approach works if all of the inputs are equally important to you. However, what if you really want to see your friends, and do not care about the weather as much because you can just put on a jacket? You might assign a different level of importance, or weight, to each one of the inputs. You might weigh the different inputs in your head when deciding whether or not to go. A computer program can do the same thing using math. To do this, you need to represent each input for the computer with a 1 for "yes" and a 0 for "no." Then, you can follow the process shown in Figure 2:

• Multiply each input (1 for "yes" and 0 for "no") by its weight
• Add up all the resulting values
• Compare those values to a threshold value to determine whether the output is 1 or 0

Image Credit: Ben Finio, Science Buddies / Science Buddies

Aria text: a circle labeled 'add up the values and check if the result is greater than or equal to the threshold.' The circle has three input arrows, each of which can be one or zero: is it sunny out? Are my friends going? Am I done with my homework? Each input value is multiplied by a respective weight. The output, which can be 1 or 0, is 'Should I go to the playground?'

Figure 2. Diagram representing an algorithm with weights.

You can also write the process from Figure 2 using equations:

$Sum = (input\: 1 \times weight\: 1) + (input\: 2 \times weight\: 2) + (input\: 3 \times weight\: 3)$

If $\: (Sum \ge threshold)\: \rightarrow \:Output = 1$
If $\: (Sum \lt threshold)\: \rightarrow \:Output = 0$

Do not worry if that seems confusing—we will walk through a complete example using numbers. First, we have to "pick" the weights and the threshold to use. For a very simple example like this, will just hand-pick a few random weights and a threshold to start: weights of 3, 5, and 1 for the sunny, friends, and homework inputs respectively, and a threshold value of 6. Using the same scenario from earlier (it is sunny, your friends are going, and you are not done with your homework), you can calculate the sum using the equation above:

$Sum = (1 \times 3) + (1 \times 5) + (0 \times 1) = 8$
Then, comparing this sum to the threshold:

$8 \ge 6 \rightarrow \:Output \:is\: 1$

You see that the answer is 1, or "yes" you should go to the playground. It might help you do the math if you write the problem out in table format, like Table 1.

Now that you have an equation and/or table set up, you can quickly see what happens if you change one of the inputs. For example, say that it is no longer sunny out, so the input for sunny changes to a 0. You can go through the same calculation to see that:

$\: Sum = (0 \times 3) + (1 \times 5) + (0 \times 1) = 5$

$\: 5 \lt 6 \rightarrow \:Output \:is\: 0$

The new output is 0, or "no," and you should not go to the playground.

Since this is a very simple example and we can do the math by hand, you could look at the new output and think "hey, that's not right! I really want to see my friends, I don't care if it's not sunny!" You could then change the weights to change the output (for this simple example, we will keep the threshold constant). For example, you could increase the weight for "friends" to 7. This would give:

$\: Sum = (0 \times 3) + (1 \times 7) + (0 \times 1) = 7$

$\: 7 \ge 6\: \rightarrow \:Output \:is\: 1$

And now the output says that you should go to the playground even though it is not sunny. You could continue teaching, or training your algorithm, by testing different combinations of inputs, evaluating the output, and adjusting the weights. Your algorithm might be "wrong" most of the time at first, but eventually, as you fine-tune the weights, you can get it to be "right" most or all of the time.

The process shown in Figure 2 is a model of the behavior of a single artificial neuron. This is all you need for a simple decision like our playground example. More complex artificial intelligence programs, like those used to recognize objects in pictures, have many neurons. These programs are called artificial neural networks, and they are designed to mimic the behavior of biological neural networks in the brain. A neural network that has a single "yes or no" output is called a perceptron. These networks can be very complex, and typically have far more weights and thresholds than a person could ever adjust by hand. Instead, the programs adjust the weights and thresholds automatically depending on whether their outputs are right or wrong. This process involves more advanced math, so in this project, we are keeping it simple and you are adjusting the weights "by hand."

Now it is your turn to try it out. In this project, you will write a simple computer program that does the calculations for you. Based on the inputs and weights, it will output whether or not you should go to the playground. Can you help your program "learn" by adjusting the weights to produce the correct output?

Terms and Concepts

• Neuron
• Artificial intelligence (AI)
• Machine learning (ML)
• Algorithm
• Input
• Output
• Weight
• Threshold
• Training
• Model
• Neural network
• Perceptron

Questions

• What are some simple yes/no decisions you make every day?
• What are the inputs you consider when making those decisions?
• Are some of the inputs more important than others?

Materials and Equipment

• Computer with Python installed, or a website that lets you run Python code online, like Trinket
• Lab notebook

This project follows the Engineering Design Process. Confirm with your teacher if this is acceptable for your project, and review the steps before you begin.

Part 1: Practicing with Python

1. In this project you will use a programming language called Python. Follow the instructions on the Python download page to download the correct version for your operating system, or create an account at Trinket. Ask an adult if you need help.
2. Python comes with an Integrated Development and Learning Environment (IDLE). You will use IDLE to write and run your code if you installed Python on your computer (if you are using Trinket, you will edit your code in your web browser).
   1. Download perceptron_example.py and save it on your computer.
   2. Open IDLE on your computer. This is not the same as opening the "Python" app.
   3. Select File→Open in the IDLE menu, navigate to where you saved perceptron_example.py, and open it.
3. Perceptron_example.py is a program that demonstrates the "should I go to the playground?" example from the background section, but a few lines of code are missing. Carefully read through the code and try to fill in the missing lines by following the instructions.
4. When you are done, select Run→Run Module from the instructions. Follow the on-screen prompts to enter 1 or 0 (press enter after typing the number) to answer each question. Watch the output to see whether or not the program says you should go to the playground.
   1. Note: if you get an error message in the output, that means there is a problem with your code. You will need to try and fix your code depending on what the error message tells you. If you do not understand the error message, try copy and pasting it into a search engine like Google to see if you can find an explanation.
5. Re-run the program, enter different values for the inputs this time, and see what happens to the output.
6. Try changing one of the weights. Save the program (File→Save), run it again, and see what happens to the output.

Part 2: Use Your Own Questions

1. Save your program with a different file name (File→Save As).
2. Now, change the program to use a decision and inputs that you think of yourself, instead of the playground example. Make sure the output decision and all of the inputs are "yes/no" questions. For example, you can use "is it hot today?," but not "what is the temperature today?"
3. Create a table like Table 2, with a row for every possible combination of inputs. For three inputs with two options each, there should be eight possible combinations.

Input 1	Input 2	Input 3	Output (your decision)	Program output (trial 1)	Program output (trial 2)	Program output (trial 3)
0	0	0
0	0	1
0	1	0
0	1	1
1	0	0
1	0	1
1	1	0
1	1	1

Table 2. All possible input combinations and the corresponding output (as decided by a human).

4. For each row in table 2, fill in what you would decide for the output based on the three inputs. You will compare these "correct" answers to the output of your program.
5. Pick random weights between one and six for each of your three inputs. You can do this, for example, by rolling a die, or writing numbers 1–6 on pieces of paper and drawing them out of a cup. If you want to, you can also look up how to generate random numbers in Python. These initial weights might not be correct, but that is OK! You will adjust them later as your neural network "learns." To keep the project simple, you will only adjust the weights and not the threshold. Use a fixed threshold value of 9 instead of choosing a random threshold.
6. Run your program and enter the first series of inputs (all zeroes, from the first row in Table 2). Record the output in the column for trial 1.
7. Repeat step 6 for each row in Table 2. Do not adjust your weights yet. Use the same weight values for all of trial 1.
8. Now, compare the "your decision" column to the "trial 1" column. By comparing the two sets of outputs, can you tell if one of the inputs is weighed too much or too little in your program?
9. Adjust one of the weights in your program (do not change multiple weights at once) and then repeat steps 6–7. Are the results of your model closer to the "right" answers this time?
10. Adjusting one weight at a time, continue to repeat this process. You can add more columns to Table 2 for additional trials if needed. Remember that in a larger neural network, the computer would automatically adjust the weights and thresholds during the training process. In this simple example, you are making the adjustments by hand.
11. Analyze your results.
    1. Can you reach a point where your model is correct (its output matches the "your decision" column) 100% of the time?
    2. Do you get "stuck," or reach a point where you cannot get the model to be correct 100% of the time no matter how you adjust the weights?
    3. Would you trust your model to make this decision for you if you just gave it the inputs?

Ask an Expert

Do you have specific questions about your science project? Our team of volunteer scientists can help. Our Experts won't do the work for you, but they will make suggestions, offer guidance, and help you troubleshoot.

Post a Question

Variations

Careers

If you like this project, you might enjoy exploring these related careers:

Computer Software Engineer

Log in to add favorite More Menu

• Read More

Career Profile

Are you interested in developing cool video game software for computers? Would you like to learn how to make software run faster and more reliably on different kinds of computers and operating systems? Do you like to apply your computer science skills to solve problems? If so, then you might be interested in the career of a computer software engineer. Read more

Mathematician

Log in to add favorite More Menu

• Read More

Career Profile

Mathematicians are part of an ancient tradition of searching for patterns, conjecturing, and figuring out truths based on rigorous deduction. Some mathematicians focus on purely theoretical problems, with no obvious or immediate applications, except to advance our understanding of mathematics, while others focus on applied mathematics, where they try to solve problems in economics, business, science, physics, or engineering. Read more

Data Scientist

Log in to add favorite More Menu

• Read More

Career Profile

Many aspects of peoples' daily lives can be summarized using data, from what is the most popular new video game to where people like to go for a summer vacation. Data scientists (sometimes called data analysts) are experts at organizing and analyzing large sets of data (often called "big data"). By doing this, data scientists make conclusions that help other people or companies. For example, data scientists could help a video game company make a more profitable video game based on players'… Read more

News Feed on This Topic

 

, ,

Explore Our Science Videos

Breast Cancer Diagnosis with Python & KNN: A Step-by-Step Coding Tutorial

Can Bacteria Survive in Highly Acidic (or Basic) Water? | Science Project

Raspberry Pi Scratch 2 Tutorial (Science Buddies Kit)