Ask an Expert: How many tests do I run for each Trial?
Moderator: berkeleywebs

 Posts: 1
 Joined: Tue Dec 04, 2007 6:03 pm
How many tests do I run for each Trial?
I am an 8th grade Science Teacher. We have a Science Fair at our school every year. Each year, I struggle with how to explain the concept of a fair test and significant results. Do you have a "rule of thumb" that you recommend to the students for the number of runs in each Trial. We ask the students to do three Trials. Within each Trial, they have control and experimental tests (one variable). I always ask them to run each control and test three times in order to take an average of each Trial. This way, they will be able to tell if their results were significantly different from the controls. Is this what you recommend? And if not, what experimental design do you recommend?
Thanks
Thanks
Hi Cara,
The ScienceBuddies website has a great link on experimental procedure that discusses the number of recommended trials. Hope it helps!
http://www.sciencebuddies.org/mentoring ... dure.shtml
The ScienceBuddies website has a great link on experimental procedure that discusses the number of recommended trials. Hope it helps!
http://www.sciencebuddies.org/mentoring ... dure.shtml
"There is a single light of science, and to brighten it anywhere is to brighten it everywhere." Isaac Asimov

 Former Expert
 Posts: 1297
 Joined: Mon Oct 16, 2006 11:47 am
The analysis of Statistical Significance is difficult to explain without some higher math involving probabilty distribution curves and convolution integrals so I'm not surprised you find it diffucult to explain it at an eighth grade level.
For students who understand things by seeing graphs, the use of a multicolor scatter plot is helpful. If you plot the control samples in one color and the samples for the samples in each experiment group in different colors (one color per experiment group) and the averages for each color in the same color but with a different symbol, then you can visually demonstrate variation, variance, out lying points, and intermingling of samples. By using various amounts of contrived dummy data (start with one point, then use a well formed clump of samples that is skewed, then add more sample points that move the average to show the first set was skewed), you can demonstrate why you do multiple trials.
The key to understanding statistical significance is in understanding that there are some complex mathmatical calculations that analyze each point with respect to all others for determining whether you have enough samples to conclude to some confidence factor that
1) the samples are sufficiently disjoint (not intermingled) to conclude separation (statistical significance), or
2) the samples are sufficiently intermingled to conclude identify (statistical insignificance), or
3) there aren't enough samples to draw either conclusion (statistically inconclusive).
One common method used is called a "T" or "tail" test that involves the equivalent of a convolution integral of some percentage of the tail (out lying points) with respect to the whole to generate a probability. These are straight forward number crunching tests if you can assume that the error probability is gaussian (equally likely to be off in either direction with decreasing probability the further away from the midpoint).
Unfortunately not all problems have symetric error probabilties. For example things with time factors. If you have anything where you have a given that something hasn't occured in the past, the probability is NOT gaussian and the standard gaussian based "T" tests are inappropriate. This subtlety lost on many scientists and engineers.
If an experiment ends up in the inconclusive state, then the obvious solutions are to run more trials or figure out how to refine the experimental proceedure to reduce experimental error or variation.
For students who don't understand things by seeing graphs, I'm clueless on how you can convey statistical significance at this grade level without first teaching how to interpret graphs.
If there is a related math probability lesson and you are team teaching, then you ideally you should try to allign the coverage so that the math probability starts just before you go into this in science and then run concurrently.
For students who understand things by seeing graphs, the use of a multicolor scatter plot is helpful. If you plot the control samples in one color and the samples for the samples in each experiment group in different colors (one color per experiment group) and the averages for each color in the same color but with a different symbol, then you can visually demonstrate variation, variance, out lying points, and intermingling of samples. By using various amounts of contrived dummy data (start with one point, then use a well formed clump of samples that is skewed, then add more sample points that move the average to show the first set was skewed), you can demonstrate why you do multiple trials.
The key to understanding statistical significance is in understanding that there are some complex mathmatical calculations that analyze each point with respect to all others for determining whether you have enough samples to conclude to some confidence factor that
1) the samples are sufficiently disjoint (not intermingled) to conclude separation (statistical significance), or
2) the samples are sufficiently intermingled to conclude identify (statistical insignificance), or
3) there aren't enough samples to draw either conclusion (statistically inconclusive).
One common method used is called a "T" or "tail" test that involves the equivalent of a convolution integral of some percentage of the tail (out lying points) with respect to the whole to generate a probability. These are straight forward number crunching tests if you can assume that the error probability is gaussian (equally likely to be off in either direction with decreasing probability the further away from the midpoint).
Unfortunately not all problems have symetric error probabilties. For example things with time factors. If you have anything where you have a given that something hasn't occured in the past, the probability is NOT gaussian and the standard gaussian based "T" tests are inappropriate. This subtlety lost on many scientists and engineers.
If an experiment ends up in the inconclusive state, then the obvious solutions are to run more trials or figure out how to refine the experimental proceedure to reduce experimental error or variation.
For students who don't understand things by seeing graphs, I'm clueless on how you can convey statistical significance at this grade level without first teaching how to interpret graphs.
If there is a related math probability lesson and you are team teaching, then you ideally you should try to allign the coverage so that the math probability starts just before you go into this in science and then run concurrently.
Craig

 Former Expert
 Posts: 27
 Joined: Wed Oct 31, 2007 3:33 pm
 Occupation: Student
Hi,
I agree with Craig that it can be difficult to teach statistics to students who do not yet have the mathematical background to understand the math behind them. However, it can definetly be done. If you focus on the broader concepts, sidestepping the nittygritty math, I think that eigth graders should be able to understand.
Essentially, the goal of any statistical analysis is to find out how likely it is that the results of an experiment occured through random chance. Your students should definetly be able to understand this. From here it gets more difficult, because, as Craig mentioned, different projects require different forms of statistics. I'm not sure what kinds of projects your students are doing, so I don't know what kind of statisticlal tests you would need to explain.
For most experiments, it would be helpful to explain the concept of a normal distribution, such as a bell curve. A graph would certainly help with this, but for those who don't do well with graphs, you could try using an example. For instance, say everyone in a class throws a ball at some easy target. The majority (assuming it is an easy enough target) will hit the target. Others, through random error, will be slightly to the right of the left, while a few are even farther off, in either direction. With enough throws in the sample, this would be a random distribution. If you were to graph the number of balls that went to each place, ou would get a roughly bellshaped curve. If you have time and a suitable location, you could even actually do this experiment.
Obviously, random variation may cause a bell curve to not look like a completely uniform diagram from a textbook. However, an abnormal distribution can also indicate that the independent variable had some efect on the dependent variable. The point of statistics, in this case, is to figure out how likely it is that these are two seperate distributions, centered around different points, and how likely it is that it is just random variation within a single distribution.
The chances that they are two seperate distributions will go up for three reasons: The distributions are centered far apart, they have small standard deviations (can be explained as thinner graphs that don't overlap as much), or they have so much data that it is unlikely for there to be much abnormality due to randomness)
That is the basic sort of explanation that I learned for my science fair project in sixth grade. You can expand from there as necesary. Hopefully some of that gave you some good ideas, and good luck!
 Emily
I agree with Craig that it can be difficult to teach statistics to students who do not yet have the mathematical background to understand the math behind them. However, it can definetly be done. If you focus on the broader concepts, sidestepping the nittygritty math, I think that eigth graders should be able to understand.
Essentially, the goal of any statistical analysis is to find out how likely it is that the results of an experiment occured through random chance. Your students should definetly be able to understand this. From here it gets more difficult, because, as Craig mentioned, different projects require different forms of statistics. I'm not sure what kinds of projects your students are doing, so I don't know what kind of statisticlal tests you would need to explain.
For most experiments, it would be helpful to explain the concept of a normal distribution, such as a bell curve. A graph would certainly help with this, but for those who don't do well with graphs, you could try using an example. For instance, say everyone in a class throws a ball at some easy target. The majority (assuming it is an easy enough target) will hit the target. Others, through random error, will be slightly to the right of the left, while a few are even farther off, in either direction. With enough throws in the sample, this would be a random distribution. If you were to graph the number of balls that went to each place, ou would get a roughly bellshaped curve. If you have time and a suitable location, you could even actually do this experiment.
Obviously, random variation may cause a bell curve to not look like a completely uniform diagram from a textbook. However, an abnormal distribution can also indicate that the independent variable had some efect on the dependent variable. The point of statistics, in this case, is to figure out how likely it is that these are two seperate distributions, centered around different points, and how likely it is that it is just random variation within a single distribution.
The chances that they are two seperate distributions will go up for three reasons: The distributions are centered far apart, they have small standard deviations (can be explained as thinner graphs that don't overlap as much), or they have so much data that it is unlikely for there to be much abnormality due to randomness)
That is the basic sort of explanation that I learned for my science fair project in sixth grade. You can expand from there as necesary. Hopefully some of that gave you some good ideas, and good luck!
 Emily
Reach for the stars and, if you miss, grab the moon!

 Former Expert
 Posts: 1297
 Joined: Mon Oct 16, 2006 11:47 am
I did not intend for my message to be that statistics was hard to teach to 8th graders. Quite the contrary. The math involved in generating simple statistics and probability is straight forward to explain. Unfotrunately, teaching how to appropriately make use of it isn't.EmilyDolson wrote:I agree with Craig that it can be difficult to teach statistics to students
It is a major challenge to go from the a rudimentary understanding of basic gaussian statistics to conveying the concept of statistical significance, statistical insignificance, and statistically inconclusive which is followed by another major hurdle in figuring out what it means wrt the hypothesis.
EmilyDolson wrote:Obviously, random variation may cause a bell curve to not look like a completely uniform diagram from a textbook. However, an abnormal distribution can also indicate that the independent variable had some efect on the dependent variable. The point of statistics, in this case, is to figure out how likely it is that these are two seperate distributions, centered around different points, and how likely it is that it is just random variation within a single distribution.
The chances that they are two seperate distributions will go up for three reasons: The distributions are centered far apart, they have small standard deviations (can be explained as thinner graphs that don't overlap as much), or they have so much data that it is unlikely for there to be much abnormality due to randomness)
Collectively this maybe obvious once you have mastered the application of statistics at the college level; however, I've helped enough graduate level people who passed two 3 hour college level courses in statistics really learn how to analyze significance related to something they were working on to know that it isn't easy to explain even when everything involved is gaussian.
The experimental analysis needs to go futher than just determining that you are obtaining two different central moments (two seperate distributions, centered around different points) to determining if there might be other factors (dependent relationships) besides the hypothesis that can explain the separation. There maybe a different hypothesis that is equally likely or there maybe two (or more) factors that jointly account for the data. Statistical analysis can help figure it out if you understand it beyond simply plug and chug on a formula. Without the understanding the number is meaningless. What often happens is somebody states a rule of thumb: if the number is above something it means this, if it is below something else it means that, and if it is between it means something else. The problem is that the rule of thumb has some assumptions related to what the rule deals with that aren't understood.
The challenge at the 8th grade level is to expose them to enough tools so they can tackle some problems and at the same time provide an understanding that what you exposed them them to is a simplification and it might not apply to everything. This dichotomy can be very frustrating to 8th graders; however, not explaining when something is a simplification can be far more damaging.
Craig

 Former Expert
 Posts: 27
 Joined: Wed Oct 31, 2007 3:33 pm
 Occupation: Student
I'm sorry, that came off sounding a lot different than I meant it to. I didn't mean to say that you were implying that it would be particularly difficult to teach eigth graders statistics, although I totally see how it sounded that way. I was agreeing with you that it isn't.
I agree that the suggestion I outlined are not sufficient for the entire class, because it is unlikely that all of their experiments fit this sort of statistical model. I was just trying to provde another, slightly different, way of looking at the basics essential to each project, because different ways of saying and looking at things may make sense to different people in the class.
I'm really sorry if I offended you. That was not my intention.
 Emily
I agree that the suggestion I outlined are not sufficient for the entire class, because it is unlikely that all of their experiments fit this sort of statistical model. I was just trying to provde another, slightly different, way of looking at the basics essential to each project, because different ways of saying and looking at things may make sense to different people in the class.
I'm really sorry if I offended you. That was not my intention.
 Emily
Reach for the stars and, if you miss, grab the moon!

 Former Expert
 Posts: 1297
 Joined: Mon Oct 16, 2006 11:47 am
Emily, I'm not easily offended and you didn't get anywhere close!
After reading your first post I was concerned that I didn't communicate my thoughts well enough in my first post. I hope I didn't offend you by taking pieces of your post out of context to point out some teaching difficulties I've run into.
I believe we totally agree that being able to present a concept in multiple ways to reach more students is what all teachers should strive for. I know multiple approaches makes a huge difference in being able to teach abstract math concepts.
Statistical significance is an area where I've tried a few approaches and failed miserably for people who don't get it from scatter plots when they understand convolution integrals (well beyond the 8th grade level).
After reading your first post I was concerned that I didn't communicate my thoughts well enough in my first post. I hope I didn't offend you by taking pieces of your post out of context to point out some teaching difficulties I've run into.
I believe we totally agree that being able to present a concept in multiple ways to reach more students is what all teachers should strive for. I know multiple approaches makes a huge difference in being able to teach abstract math concepts.
Statistical significance is an area where I've tried a few approaches and failed miserably for people who don't get it from scatter plots when they understand convolution integrals (well beyond the 8th grade level).
Craig

 Former Expert
 Posts: 1297
 Joined: Mon Oct 16, 2006 11:47 am

 Former Expert
 Posts: 27
 Joined: Wed Oct 31, 2007 3:33 pm
 Occupation: Student
Oh no, you didn't offend me at all. I'm fine with you using the quotes! I'm just paranoid about offending others , especially because it is so easy to misunderstand people when communicating over the internet.
I think your explanation was great, by the way!
I think your explanation was great, by the way!
Reach for the stars and, if you miss, grab the moon!