Scientific Method or EDP? Machine Learning

Ask questions about projects relating to: aerodynamics or hydrodynamics, astronomy, chemistry, electricity, electronics, physics, or engineering

Moderators: AmyCowen, kgudger, bfinio, MadelineB, Moderators

UNDEFINEDNULL
Posts: 9
Joined: Sat Apr 01, 2023 9:33 am
Occupation: Student: 5th Grade
Project Question: How does sample size and variety impact machine learning bias?
Project Due Date: 4/26/23
Project Status: I am conducting my experiment

Scientific Method or EDP? Machine Learning

Post by UNDEFINEDNULL »

Hi there ScienceBuddies,
I am a 5th-grade student and want to do your "Identify Road Signs With Machine Learning" project. However, I am concerned about which process to use. I am thinking about using the EDP, but I don't want to have to brainstorm alternative solutions. By the way, I want to make my machine have more than 2 sign classes. The Scientific Method seems like a good choice, but I'm unsure what my testable question should be. I considered, but dismissed "Can I train an AI to recognize road signs". THank you for assisting me.
bfinio
Expert
Posts: 748
Joined: Mon Aug 12, 2013 2:41 pm
Occupation: Science Buddies Staff
Project Question: Expert
Project Due Date: n/a
Project Status: Not applicable

Re: Scientific Method or EDP? Machine Learning

Post by bfinio »

Hi - this project is better suited to the scientific method than the engineering design process. Teachable Machine is an existing tool that you can use to run experiments. You can change some of the parameters in the model, but you are not really "designing" your own machine learning system or writing your own code etc. I'm not sure what the requirements for your testable question are, but you could make it more specific, for example "Can I train an AI to tell the difference between (all the types of signs you plan to test)"?

Hope this helps!
UNDEFINEDNULL
Posts: 9
Joined: Sat Apr 01, 2023 9:33 am
Occupation: Student: 5th Grade
Project Question: How does sample size and variety impact machine learning bias?
Project Due Date: 4/26/23
Project Status: I am conducting my experiment

Re: Scientific Method or EDP? Machine Learning

Post by UNDEFINEDNULL »

Actually, would the Engineering Design Process work? I found a couple of alternative solutions
Train the AI to recognize based on sign color
Train the AI to recognize using sign shape
etc
They are not as good as the other solution(training based on a variety of real world signs) but will work for the purpose of having alternatives.
UNDEFINEDNULL
Posts: 9
Joined: Sat Apr 01, 2023 9:33 am
Occupation: Student: 5th Grade
Project Question: How does sample size and variety impact machine learning bias?
Project Due Date: 4/26/23
Project Status: I am conducting my experiment

Re: Scientific Method or EDP? Machine Learning

Post by UNDEFINEDNULL »

P.S. I'm still super evenly split. Each has its pros and cons, and I can't even decide. The "scientific method" part of me agrees with you, but the EDP part of me argues that training an AI image model IS designing. I'm in a bit of a quandary.

P.P.S The question would be best if it was in a who, what, where, when, why, how, or which format IF possible. Would that be possible for my project(I am not looking at bias or any one aspect, instead my goal is to create a model that is accurate enough to be used in a self driving car)?

P.P.P.S. Would "Can I train an AI to recognize the differences between stop, curve, and bump signs with at least 95 percent accuracy" be a good question that can be testable?

P.P.P.P.S. Thank you for EVERYTHING!!!
bfinio
Expert
Posts: 748
Joined: Mon Aug 12, 2013 2:41 pm
Occupation: Science Buddies Staff
Project Question: Expert
Project Due Date: n/a
Project Status: Not applicable

Re: Scientific Method or EDP? Machine Learning

Post by bfinio »

Hi - before I provide any more answers - are you able to ask your teacher or another mentor about any of this? Which approach you take may depend on the specifics of the science fair you plan to enter.
bfinio
Expert
Posts: 748
Joined: Mon Aug 12, 2013 2:41 pm
Occupation: Science Buddies Staff
Project Question: Expert
Project Due Date: n/a
Project Status: Not applicable

Re: Scientific Method or EDP? Machine Learning

Post by bfinio »

Also, I should have mentioned this the first time, but have you seen our page about comparing the scientific method and engineering design process? If not, it might help to read this page and watch the video at the bottom: https://www.sciencebuddies.org/science- ... fic-method. That might help you decide.
UNDEFINEDNULL
Posts: 9
Joined: Sat Apr 01, 2023 9:33 am
Occupation: Student: 5th Grade
Project Question: How does sample size and variety impact machine learning bias?
Project Due Date: 4/26/23
Project Status: I am conducting my experiment

Re: Scientific Method or EDP? Machine Learning

Post by UNDEFINEDNULL »

Thank you for the prompt reply! I did ask my teacher before this, and she was also evenly split. The science fair project has to be either "a well-designed investigation using the Scientific Method" OR "an original invention using the Engineering Design Process"—no other info about allowed project types. The video was also quite helpful. I think I want to go with the Scientific Method after all, but would the testable question mentioned in "P.P.P.S." be good? THANK YOU!!!!!!!!!
bfinio
Expert
Posts: 748
Joined: Mon Aug 12, 2013 2:41 pm
Occupation: Science Buddies Staff
Project Question: Expert
Project Due Date: n/a
Project Status: Not applicable

Re: Scientific Method or EDP? Machine Learning

Post by bfinio »

Hi - given that description I would also lead toward scientific method, since Teachable Machine is an existing tool designed by someone else and not "an original invention."

Regarding the question - we have an entire page about coming up with a good question, reading this may help: https://www.sciencebuddies.org/science- ... t-question. There isn't necessarily a single "best" way to phrase the question, but I think you are on the right track. You have to make sure the question is something you can answer with an experiment, which you're already thinking about. If you aren't sure how to phrase the question or how specific you need to be, you could talk about that with your teacher more. For example, "Can I train an AI to recognize road signs?" is pretty broad, but your most recent version gets pretty specific; you might not need the "with 95% accuracy" part in the question itself.

Hope that helps!
UNDEFINEDNULL
Posts: 9
Joined: Sat Apr 01, 2023 9:33 am
Occupation: Student: 5th Grade
Project Question: How does sample size and variety impact machine learning bias?
Project Due Date: 4/26/23
Project Status: I am conducting my experiment

Re: Scientific Method or EDP? Machine Learning

Post by UNDEFINEDNULL »

Hi. I noticed that I would have trouble conducting research and formulating a hypothesis for this question since:
1. This question is not really a "Does doing X lead to Y?" question. In fact, the outcome is more me-controlled. If I want to, it is entirely possible to, indeed, create a model that CAN "recognize the differences between stop, curve, and bump signs with at least 95 percent accuracy". Therefore, my hypothesis is kinda going to be off.
2. It is hard to identify the independent and dependent variables in this question. It seems to be too broad.
3. I am having a hard time identifying what EXACTLY to research, using the Background Research Plan. (https://www.sciencebuddies.org/science- ... earch-plan)
Because of this, will I have to abandon this idea and move on to something more specific(such as "How does sample size and variety impact machine learning bias?")
Thank you so much! :D
P.S.(Does a hypothesis HAVE to have an independent and dependent variable? For example, if, hypothetically, my topic was “Can Humans Recognize ChatGPT’s Text”, could my hypothesis be “Humans cannot recognize ChatGPT’s text from human text since ChatGPT has passed the Turing test…”?
(Also, what is the point of doing background research, besides forming a hypothesis (since my experimental procedure is already there)?)
chloemkohl
Student Expert
Posts: 53
Joined: Sat Jun 11, 2022 7:17 am
Occupation: Student

Re: Scientific Method or EDP? Machine Learning

Post by chloemkohl »

Hi! I read through this thread, and you ask some really good questions!
1. Not all experiments need to have a traditional hypothesis! While some hypotheses will follow the "If _____ _____, then _____[this]_____ will happen" format, your hypothesis just needs to include your prediction of the results. However, I do think you would be able to fill in the blanks if you really wanted to, but it's definitely not necessary. Also, saying that it is possible to successfully create this model is a hypothesis, as without actually creating the model, you are predicting the outcome. I think this may also answer your question in the "P.S.," but let me know if not!
This quick article https://www.sciencebuddies.org/science- ... hypothesis may also help you come up with your hypothesis.
2. The independent variable is whichever variable is being tested, or in other words, the variable that is being changed. The dependent variable refers to what you are measuring in response to the independent variable. Science Buddies has an article linked here (https://www.sciencebuddies.org/science- ... /variables) that can help to explain the difference.
3. For background research, I would recommend reading the introduction to the experiment as well as watching any of the videos. It could also be helpful to research more about bias in machine learning. Additionally, you may want to be familiar with all of the different terms and concepts. Background research is important to ensure that you have sufficient knowledge in order to analyze your data and understand all of your results.

Hopefully this helps to answer your questions, but please feel free to reach out for further clarification! Good luck with your project :)
- Chloe
Last edited by chloemkohl on Fri Apr 07, 2023 10:47 am, edited 1 time in total.
UNDEFINEDNULL
Posts: 9
Joined: Sat Apr 01, 2023 9:33 am
Occupation: Student: 5th Grade
Project Question: How does sample size and variety impact machine learning bias?
Project Due Date: 4/26/23
Project Status: I am conducting my experiment

Re: Scientific Method or EDP? Machine Learning

Post by UNDEFINEDNULL »

chloemkohl wrote: Thu Apr 06, 2023 8:07 pm Hi! I read through this thread, and you ask some really good questions!
1. Not all experiments need to have a traditional hypothesis! While some hypotheses will follow the "If _____ _____, then _____[this]_____ will happen" format, your hypothesis just needs to include your prediction of the results. However, I do think you would be able to fill in the blanks if you really wanted to, but it's definitely not necessary. Also, saying that it is possible to successfully create this model is a hypothesis, as without actually creating the model, you are predicting the outcome. I think this may also answer your question in the "P.S.," but let me know if not!
This quick article https://www.sciencebuddies.org/science- ... hypothesis may also help you come up with your hypothesis.
2. The independent variable is whichever variable is being tested, or in other words, the variable that is being changed. The dependent variable refers to what you are measuring in response to the independent variable. Science Buddies has an article linked here (https://www.sciencebuddies.org/science- ... /variables) that can help to explain the difference.
3. For background research, I would recommend reading the introduction to the experiment as well as watching any of the videos. It could also be helpful to research more about bias in machine learning. Additionally, you may want to be familiar with all of the different terms and concepts. Background research is important to ensure that you have sufficient knowledge in order to analyze your data and understand all of your results.

Hopefully this helps to answer your questions, but please feel free to reach out for further clarification! Good luck with your project :)
- Chloe

Hi! Thank you so much for the clarification about my hypothesis and background research! However, I still have a super tiny question - did you suggest doing "Can an AI be trained to recognize..." or my more recent question about bias? Also, for the bias question, would these 3 groups work?
Group 1 - Small sample(15 photos for each sign type)
Group 2 - Medium Sample(50 photos for each sign type)
Group 3 - Large Sample(100+ photos for each sign type)
AmyCowen
Site Admin
Posts: 581
Joined: Mon Aug 22, 2016 4:39 pm
Occupation: Administrator
Project Question: -
Project Due Date: -
Project Status: Not applicable

Re: Scientific Method or EDP? Machine Learning

Post by AmyCowen »

I think you have questions listed now in your profile, but to make it easiest for someone to provide some answers to your remaining question, can you write both questions you are considering out in a single reply.

It's clear that you have one possible project about "whether or not" you can train a computer to recognize signs and another about bias. You could do a project related to either (provided you are clear on what bias means in this context). Experts here won't be able to tell you "which" project you should do, but if you can write out where you are now .... what the two questions you are considering are (and what hypothesis you've constructed for each), it would be helpful. (Thinking through your hypothesis would help you understand what the experiment you would be doing will be since your variables will come up in the context of the hypothesis.) Chloe gave you linked to the hypothesis and variables pages in the Project Guide in a previous post.

Amy
Science Buddies
UNDEFINEDNULL
Posts: 9
Joined: Sat Apr 01, 2023 9:33 am
Occupation: Student: 5th Grade
Project Question: How does sample size and variety impact machine learning bias?
Project Due Date: 4/26/23
Project Status: I am conducting my experiment

Re: Scientific Method or EDP? Machine Learning

Post by UNDEFINEDNULL »

Hi Amy,
The first question I am considering is "Can I train an AI to recognize the differences between stop, speed limit, and no parking signs". The other question is "Does sample size have a correlation to the amount of bias/accuracy?" My hypothesis for the 1st one is,"Yes, I can train an AI to do so." while the 2nd one's hypothesis is "The correlation between sample size and bias is negative(i.e, as one decreases, the other increases). I honestly want to go with question #2. I am doing background research but have partially started setting up my experiment. Finally,(besides the bibliography of the project), what would be some places I could look to find out more about bias in machine learning?
AmyCowen
Site Admin
Posts: 581
Joined: Mon Aug 22, 2016 4:39 pm
Occupation: Administrator
Project Question: -
Project Due Date: -
Project Status: Not applicable

Re: Scientific Method or EDP? Machine Learning

Post by AmyCowen »

Hi - It is great to hear that you've picked a direction and started your research.

Thank you for replying with your two questions. I think (especially at your grade level) that it would help you to use the Hypothesis resource to craft a hypothesis statement that clearly sets up how the variables work in your experiment. (This is especially true for your question 1 option.) To do a controlled experiment, you will be modifying a variable (independent) and seeing what happens (to the dependent variable). You want to be testing something specific. Machine learning, after all, is a huge topic. Your project will involve a specific, manageable test. Just saying "yes this will happen" isn't a well-formed hypothesis. Question 1 would map to the procedure in the project. Shifting to your bias angle, you'll be creating your own procedure.

If you are more interested in bias, that's great. Keep in mind that sample size alone doesn't create or solve bias. You can upload 5, 1000, or 50,0000 images of yellow and green shirts and end up with a machine that has learned something but also may have built-in bias. How will you be evaluating or testing for bias?

There is another project, similar to this one, that also talks a little bit about bias in machine learning: https://www.sciencebuddies.org/science- ... sification You might find it helpful to read through that one, too. It may have other bibliographic resources to help with your research, but a Google search will lead you to many other sources.

Amy
Science Buddies
UNDEFINEDNULL
Posts: 9
Joined: Sat Apr 01, 2023 9:33 am
Occupation: Student: 5th Grade
Project Question: How does sample size and variety impact machine learning bias?
Project Due Date: 4/26/23
Project Status: I am conducting my experiment

Re: Scientific Method or EDP? Machine Learning

Post by UNDEFINEDNULL »

Hi! I will actually be testing for accuracy, NOT bias. However, I understand that bias is just a systematic error (that occurs for a couple of reasons) that impacts accuracy negatively. Therefore, if accuracy is higher, I will conclude that bias is lower(I will mention this in my tri board). I have finished background research and made a hypothesis thanks to your(and other experts') help. However, the "Experiments" guide (https://www.sciencebuddies.org/science- ... procedures) said to repeat the experiment 3 times. How would this apply to me?

Thanks,
UNDEFINEDNULL
Post Reply

Return to “Grades K-5: Physical Science”