Science Buddies: "Ask an Expert"

My Project is:

"Does doing homework affect test grades positively?"

I had 12 volunteers, 6 took homework the night before the test, while the other 6 only took the test without any homework.

The homework consisted of 10 math questions from the SATs, as did the test.

I want to know if there are any holes in this project, and how I might be able to fix them. Any help is greatly appreciated!

Lets think about what could vary here between the groups that you might need to consider and try to control for.
Are all of your volunteer test subjects exactly the same?
I think it's unlikely unless they are robots! So you need some way to attempt to control for this variation between people - when studying humans this is always tricky!
The type of study you are doing could be described as an intervention study. The ones that didn't do homework before are the controls and the group that got the homework/practice questions received an intervention. Ideally the control group should be as similar to the intervention group before the intervention as possible.

One way researchers do this is to look at the group and try and make pairs of similar people - you could do this based on some previous test score or current grade GPA or something. The one of each matched pair goes in the control group and the other goes in the intervention group. The difficulty is in making sure each pair is really similar, hard to do with humans, but if you have a large enough group and look at the average scores in each group it might show something you could test with a chi square test. Using averages can help get around the problem of variability between people (inter-individual variation) but it relies on the group size being large enough to even things out.

Another way if the whole group is pretty similar (eg. they all usually get A on a test) is to randomly assign people to each group - this is kind of what you have now - and then look at the average scores.

Another way researchers try and do this is to use each person as their own control - ie. do the experiment once where no one gets homework then do it again and intervene by giving homework then compare each persons score for the first and second test. This has its own problems though because maybe just the action of having done the test before will help them learn to do better in the second test. So you could do what is called a cross over study - one group do the test no homework first then after a period of time do the test after homework, group 2 do the test with homework first then test no homework after same period of time. (You could do this using what you have already done as the first step). This also compares each person against their own score so accounts a bit for inter-individual variation.

All of these methods have their own flaws. The more people you can study the better for getting significant results but it can be hard getting people to participate. Either way learning about types of study designs is a useful thing as these are the kinds of ways that new drugs are tested or ways to prevent diseases (lifestyle interventions) are tested.
I hope some of this information is useful. Post back with your ideas and questions.

best of luck,
Caroline

Caroline gave you some great advice. All I would add is that the SAT covers a lot of different math. So, it's possible that the 10 homework problems may not be very good 'practice' for the actual 'test' in this case. I personally would suggest having everyone do more problems (both homework and test) so that you get a broader range of examples. Alternatively, you might try asking a math teacher for a test on a particular subject, and have students do homework problems on that same subject--which is usually the case when people do homework!