Science Buddies: "Ask an Expert"

My daughter (9th grade) presented her planetary science fair project yesterday. ISEF has changed the "Earth and Planetary Science" category to "Earth and Environmental Science" and limited that category to Earth. Projects like hers have moved to "Physics and Astronomy." On her crit sheet, the judge says that she needs to do:

Hypothesis --> Mathematical equations --> Model --> Scientific data --> Fit to model --> Conclusion

This is pretty much how I did physics (but I don't have any physics background beyond into level classes). Newton's laws --> roll the cart down the ramp --> fit the distance/time data to the equation.

BUT, since she became interested in Mars, we have NEVER done a science fair project this way.

It looks more like:

1. Choose an aspect about Mars (craters, ice, etc.)
2. Background reading about that aspect
3. Think about what sorts of things you can measure, background research about missions and datasets
4. Site selection (compromise between interesting location and data set availability)
5. Design site analysis procedure
6. Download/analyze data
7. Think about what it means --> usually have to circle back to step 2 or 3 here
8. Write conclusions, supported by various charts and graphs

She's done, I guess, more statistical analysis (is the cloud more strongly associated with the change of season or the change in dust level?), rather than model-fitting (if the season is X and the dust is Y, I predict the atmospheric water ice to be Z based on this equation).

She's actually pretty good at reading the math in her background research, but she's not necessarily able to do math at the level she can read it. For example, seeing notation like dt/dz for the change in temperature versus the change in depth is not an issue, but she hasn't actually taken a calculus class. And, even if she could do the math, it's not clear that for a particular model, she'd be able to get all the measurements to fill in all the variables (really depends on the topic and site, and what instruments have data available).

So, I'm looking for suggestions for next year, for a topic that hasn't been determined yet, but which will almost certainly revolve around Mars:

1. What are some good reference materials for mathematical models of Mars to read up on?
2. What are some good examples to look at for Mars research that are done in a more "physics" fashion?

On the other hand, despite not getting a great score, she did come home from science fair with a smile on her face... Any experts want to tack the philosophical question here: How much should I try to modify her science to fit the criteria for her category (so she can win prizes, etc.) and how much should I just let her do things her own way? I can see the educational benefit in learning about how a physicist sees the world versus how a geologist might, and I think over time that she might not want to do science fair if she doesn't get much positive feedback. But, really, she isn't doing anything wrong with her projects, they are (in my own humble opinion) good science for a kid her age, but there's a square peg in a round hole problem here.

Hi Janet_425,

I'm glad your daughter had a positive science fair experience, despite the fact that judging didn't turn out the way you were hoping. I have a few different thoughts--hopefully one or two of them will be helpful to you.

Thought 1:
The outcome of judging is not unique. With different judges, on a different day, and a slightly different set of projects (or even the same projects), it's entirely possible that different students would have won awards. It's not likely that the outcome would be vastly different, but the order and distribution of 1st, 2nd, 3rd, honorable mention, etc. might have been. This isn't because the judges don't know what they are doing. Instead, it reflects the fact that different experts sometimes emphasize different parts of a project. One set of judges might think mathematical models are a really important part of the scientific method. Another set of judges might hone in on how well the conclusions align with the data collected. Yet another set of judges might put more emphasis on a student's background research and knowledge. Detailed judging rubrics help mitigate this variability, but the human element is always present in the judging process. So, not winning a prize at one fair doesn't mean that someone did a "bad" project. But, it can still be disheartening. The fact that this human element plays a role in the judging process also means that the comments you get from one judge don't reflect the comments you might have received from a different judge. Feedback of any kind is helpful and informative, but it isn't necessarily an edict decreeing what you have to change to make the project "fit".

Thought 2:
Planetary science projects at ISEF have been shifted around a lot. There used to be a Space Science category, which included planetary science. Then, the space a science category was dissolved and planetary science was wrapped into the Earth and Planetary Science category and astronomy moved to physics. Now, a few years later this latest change has happened. Your analogy of a square peg/round hole is spot-on. It's hard to find a place where planetary science project fit perfectly. Looking at the ISEF category descriptions (https://student.societyforscience.org/i ... categories), if categories stay the same for 2016, the project your daughter did would probably fit better in the Earth and Environmental Sciences: Atmospheric Science category, despite the fact she is studying Mars. Yes, she is looking at a different planet (Mars) and not the Earth. But, the strategies she is using reflect more of a geological approach than a physics/astronomy approach. You would need to talk this over with your fair, but I think it's worth the conversation. The metaphorical hole might a little more square-like in the Earth and Environmental Sciences category than in Physics and Astronomy.

Thought 3:
Here is a link to a short paper about Martian craters that two colleagues of mine will be presenting at a meeting next week. It's dense. But, it does show one way of linking geological observations to a mathematical model.

http://www.hou.usra.edu/meetings/lpsc2015/pdf/1272.pdf

The takeaway message from this abstract is that geological observations motivated the mathematical modeling that they did. They formed a hypothesis based on the geology and then developed a model to test that hypothesis.

Thought 4:
Statistical analysis is, to me, a perfectly valid type of mathematical analysis. Your daughter could perhaps highlight in more detail the statistical tools that she used to arrive at her conclusion.

Thought 5:
The judge's comments, ignoring the business of mathematical equations, suggest that s/he wasn't able to clearly see how your daughter followed the scientific method. The procedure you outlined for your daughter's project (steps 1 to may have been the way that she did the project, but she can present the project with the steps in a different order. What do I mean by this? I mean that your daughter can present a brief introduction, question, hypothesis, background research, procedure, data, results, analysis, and conclusions in that order, even if that wasn't the exact order she did the project in. When she was designing the analysis procedure, your daughter probably had a question she was trying to find the answer to. She probably also had a hypothesis she was testing. (If she didn't have a hypothesis to test, she should formulate one as she is preparing for her next project.) Even if, in reality, she may not have arrived at that question and hypothesis until step 4 or 5 of the process you outlined, she can still organize her presentation by putting the question and hypothesis first. Even geological/statistical projects are hypothesis-driven.

Perhaps an example will clarify this. Right now I'm working on a project related to Ceres, the dwarf planet that NASA's Dawn mission just started orbiting. When I first started looking at Ceres and at the literature about Ceres, I didn't have a specific question I was trying to answer. I was just exploring what was out there. But, as I dug deeper, my thoughts started crystallizing and a question emerged: where did the strange surface of Ceres come from? I did some preliminary experiments to try a few different ideas. Based on those preliminary experiments, I hypothesized that the strange surface of Ceres formed as impacts delivered pieces of asteroids and meteorites to the surface of Ceres. I then constructed a detailed set of experiments, which I conducted and analyzed. When I publish this work, however, my paper will look something like this:

1) Introduction to Ceres
2) The question I'm trying to answer
3) My hypothesis
4) My experiments
5) My results and analysis
6) My conclusions - does my hypothesis work?

This wasn't the exact series of steps that I went through as I worked on the project, but it is how I will report the research to the scientific community.

These thoughts probably don’t answer all of your questions, but hopefully they answer some of them. Let me know if you have other thoughts/questions, and I will do my best to answer them!

I definitely understand that judging will turn out differently on different days and fairs. This is actually a brand new fair for us (K-8 and 9-12 have science fairs sponsored by different organizations around here). I guess the reason why I am trying to figure out how to put this judge's comments to use is because there are so many of them. This is the first fair where we've ever gotten an entire page of written feedback. Usually it's a few boxes checked and a sentence or two. This was obviously someone who liked my kid and wanted to take some time to help her to do better in the future.

As far as my square peg goes, she's been the only Martian at every fair we've done, and it's highly unlikely we'll see a planetary scientist for a judge. For this new fair this year, we actually picked the earth science category (we don't specify subcategories), and got moved to Physics, Astronomy, and Mathematics after entries were turned in. So, not only don't we pick our subcategories on the entry form, but the categories are collected by the fair into bigger groups if there are not enough entries in certain categories. (And, I have to say the earth/environmental science group this year was not a great fit for us, either: I saw lots of environmental science projects on coal trains, waste handling, clean water, etc. I don't honestly remember if there were any geologists.) I think we're just square pegs, but if I decide to try talking to the fair coordinators next year, I'll be back to ask about how to do that.

This year's project has less math and statistics than last year's. She's basically struggling with image calibration issues and while she thinks her pictures are close qualitatively, she doesn't really trust the numbers. So, she has pictures, detailed captions, and what she thinks are the strengths and weaknesses of her results, but no firm conclusions. (So, really, not one that we expected to be a championship project so much as a learning experience.) But, our board VERY clearly shows the scientific method. We have worked hard on clearly communicating her work and it's usually one of the clearest boards to navigate at the fair. LMK if you'd like to see pictures.

As I hinted in the end of my original posting, I'm still trying to figure out exactly how much of the advice to take and how much to just chalk up to having a kid who doesn't quite fit the science fair mold. I will be looking at your examples closely, and it will be easier to figure out how much mathematical modeling is possible once she starts thinking about a topic for next year. This year's project is what it is. I am going to look through the examples you gave more carefully, and keep my eyes open for mathematical models we might try to use, but without a topic, it's hard to say what we can do.

Hi Janet425,

It's great that you got so much feedback from judges! And, it's wonderful that s/he took the time to give detailed comments. If the comments are fairly consistent (and, based on your post, it sounds like they are), then I agree it's advice worth considering. In my post, I was remembering a time when one of the judges at a fair I was running made a bunch of comments that were far different from what the other judges thought. It also sounds like you and your daughter did a very clear job of showing the scientific method in her display board. That's excellent, and I apologize for misunderstanding that in your first post.

If your daughter runs into calibration issues in the future, I'm happy to help her work through those challenges. Some of my colleagues specialize in taking orbital data from spacecraft and turning it into useful data products. We really are happy to help, if that would be useful to your daughter. As she is brainstorming ideas, I'm happy to look at a few of them and see if anything comes to mind about how to incorporate modeling into her study. If you have questions about the example I posted, do let me know so that I can clarify things as needed.

As far as taking advice versus not fitting the science fair mold goes, there's not a clear-cut answer. Planetary projects definitely aren't common at science fairs, which means people aren't very familiar with them. But, planetary science is definitely worth doing and incredibly fascinating.

Thanks for all the offers of assistance.

She finally decided that she'd worked it on her own enough and emailed the author of the paper she was working from and got a good hint.