Is my project question too wide?

[Miss-C]

So I am doing a science fair project for my co-op, and I am giving several different subjects three different puzzles. The question is, "Does age, gender, or personality affect problem solving skills?" I am going to time my subjects and see if they can get the right answer within a time limit. The first puzzle is going to be a trick question, the second one is going to involve math, and the third would be an "impossible situation" in which the subject would have to figure a way out of the made up situation.

So I guess that my question is, am I on the right track??? Help please!!

[deleted-140482]

It sounds like you are absolutely on the right track., but I would consider narrowing down on your variables a bit. Right now your variables are 1) age, 2) gender, 3) personality, and you also have three different questions 1) trick, 2) math, 3) "impossible situation" which I'd call a measure of creativity. It's going to be very difficult to get enough people involved in your study to be able to accurately determine which variable is most affecting the outcome. For example, if you enroll 3 people: 1 is young, female, and outgoing, 2 is old, male, and shy, and 3 is old, female and outgoing, how would you determine whether the differences in their ability to answer questions is due to the difference in age, gender, or personality? Additionally, since you have three different kinds of questions, if subject 1 does really well on the trick question and the math question, but not the creativity question, can you conclude that young, female, outgoing people are good at math and trick questions but not creative? I'd argue that it's impossible to tell whether she's not good a all creative situations, happened to not be good at that one, and whether you can extrapolate any of that to the entire group of young, female, and outgoing people.

In order for you to be able to draw conclusions from your data, you should decrease your variables to one or two (I would strongly consider ruling out personality, because that is a much more subjective measure than age or gender), and I'd also stick with one "type" of question, although you would ask multiple questions of that "type." For example, if I were structuring this experiment, I would ask "Does gender affect math skills?" Then you recruit a number of subjects from both genders (the more subjects, the better, since you won't be able to control for person to person variability) and give them all a math quiz (say 10 questions). At the end you can compare the average scores on your quiz for the males v. the females, and answer your question. This is only an example, you could certainly pick a different variable or different skill question. The important part is to try to only test one variable at a time, and to enroll as many people as possible.

I hope this helps, and if you have more questions or I wasn't entirely clear, please feel free to post again here.
JMP

[Miss-C]

Thank you so much for your advice!!!! My dad and I talked about it and we both agreed that personality should be ruled out, but now we aren't sure if it should just be gender, or age and gender? Any thoughts on that? Also I will probably do one type of question (like you sugested) but I am still not sure what type. My dad and I looked around at a bunch of logic puzzles and riddles and optical things, but if I chose to do age & gender as varibles I would need a problem that would be able to be hard for people my age, not too easy for adults, and not impossible for say my five year old sister.........

Do you have any pointers or suggestions for the way I am going now?

[deleted-189740]

Hello Miss C,
I think you are on the right track to narrow down to one factor. Before deciding the type of question you would like to put on a test, it would be a good idea to formulate a hypothesis for predicting the outcome and the reasonings behinds it. In my opinion, It would be useful to ask how does ages/ gender/ personality affect the problem solving skills . Once you come up with your explaination, it would help you to decide what type of tests you would like to put on and the types of the subjects you would use to prove your claim .

Hope it helps!
Best,
Pak

[deleted-3443]

The question you are struggling with is trying to find a measurement that is reliable and accurate.

I will give you a little hint -- suppose you are a teacher, trying to determine how much a classroom of students know about American history. Suppose you try to do this with a one question test: "In what year was the battle of Antietam?"

What can you reliably and accurately say about the class knowledge of history with a test like that? Would the students think that was a fair test?

Would this question be any better? "Describe some of the causes of the American civil war."

Similarly, by trying to find ONE question and giving that same question to a varied group of test subjects, you are making your experimental design problem very hard. Think about using multiple questions and using open-ended questions. And, then ask yourself, looking at the test as a whole, does this fairly, reliably, and accurately measure what I am trying to find out?