Machine Learning Variables and Trials

[braindog11]

Hi,

I doing a science fair project to see how well teachable machine can be trained to classify happy, sad, surprised and angry faces. I'm confused about the variables and trials for this type of project. I understand confusion matrixes, but what should my three trials test? I think the independent variable would be the number of images the tool is trained with. My dependent variable would be the AI's performance. My controlled variables would be the AI (Teachable Machine), Number of Each Emotion, Number of Each Gender, etc. Does that sound OK? How do I make the trials so I can compare the tools when I am testing new images each time?

Also, I am in 5th grade, and this is a project idea from the Happy or Sad? project on Science Buddies.

[Administrator note: Your posts have been combined. Project url: https://www.sciencebuddies.org/science- ... sification

[bfinio]

Hello - your understanding of variables in the project is correct! If you want to do multiple trials, you can re-upload the same images and re-train the model to see if you get the same results. Note that in general, some machine learning algorithms have an element of randomness to them, so you will not necessarily get exactly the same results each time (which is why it could help to do multiple trials, even if it's with the same data). This is not true for ALL computer programs however - some of them will give the exact same results each time you run them (for example, if you wrote a program to add two numbers, and always gave it the same two inputs, it would always give you the same results).

You can also learn more from this page: https://www.sciencebuddies.org/science- ... procedures

Hope that helps!

[braindog11]

So, would this be training different tools like is mentioned in the example project? Then, the three trials are comparing the different tools? The example mentions testing so many different things that it gets confusing. Do just pick one difference to test? And do I keep all the test images the same for each trial? I am planning to have friends and family draw images to use for my test data.

[janna_sciencebuddies]

Hello! I’m happy to help clarify things:

For each trial, you'll repeat the same process using one of the tools. You will conduct three tests (trials) with the tool trained on 20 images and three tests (trials) with the tool trained on 200 images.

Start by training the tool with 20 images, then test it and record the results using the confusion matrix. For the second trial, open a new project, upload the same 20 images, and test it again using the same test images. Repeat this once more for the third trial. The results should be similar, which is expected. You can then average the results from all three trials.

Next, follow the same process with the tool trained on 200 images: upload the 200 images, test the tool three times, record the results, and take an average.

There are many things you can test in your project, so I understand why it might feel confusing. You can choose to test one, a few, or all of the cases listed in Step 15, but be sure to consistently measure the same factors for each trial and each tool.

You're doing great by focusing on the variables and trials! Machine learning can feel tricky, but your approach of testing and using a confusion matrix shows you're thinking like a scientist. Keep up the good work!

[MadelineB]

As a science fair judge, I'd like to add a suggestion to the excellent advice from these experts ...

in addition to taking the average of your measure of model performance, judges like to see a plot showing the measures for each of the 3 trials. This gives a sense of the range of variability for those 3 trials.

Also note the comment from Ben:

" ... some machine learning algorithms have an element of randomness to them, so you will not necessarily get exactly the same results each time (which is why it could help to do multiple trials, even if it's with the same data). ... some of algorithms will give the exact same results each time you run them ... "

Think of these 2 possibilities when you are discussing the variability or lack thereof for your three trials!

Good luck with your project!

Madeline