Survey and the IRB, etc.

[rpeebles]

I had one more thought to share - the concerns of survey bias are actually always true with survey research - both online and "in person". You NEVER force someone to fill out a questionnaire, so subjects ALWAYS choose to participate in your survey. So actually, there is always a selection bias involved. In the real non-online world, you cannot reduce that bias, but instead you keep track of how many people refuse to participate, and then publish your response rate, and that is how someone reading your study determines how large a role bias played. For example, if your response rate for a survey in a GI clinic was 90% of all patients visiting that clinic in a 2 month period of time, that's unbelievably good, and implies very little selection bias, in that everyone participated. You have other biases then, because you're surveying a clinic population that may not represent everyone with a stomach problem, but your survey selection bias is not as huge an issue. On the other hand, if your response rate is 15%, then there are a ton of people who didn't participate, and the question becomes "Why?" If you can explain that answer well, beyond the issue of bias, the survey may still be useful and publishable. If you can't, it's usually a big problem.

With online methodology, there are ways to measure hits to your survey, and completed surveys, and even track IP addresses, and all of these things can help. With small online communitites you can often find out how many active members there are as well. These things cannot give you a true response rate, but can help you approximate that for your analyses.

Hope that helps and clears that up. Basically: survey = selection bias, whether online or not. - Rebecka

[methionine]

Hi,
thanks for your help!

in response to what you 've written-- wouldn't it be sort of a privacy violation-- to track IP addresses and such? One of the main obstacles in my project (if i decide to carry on with it) is that I need IRB approval. While my school is willing to get an IRB together during the summer for me, it's probably difficult to actually get around the rules-- that is, since the ED community is classified as an "at risk" group, and each survey is required to have some sort of parent signed form, it'll be especially difficult, since a significant percentage of parents don't even know that their child goes on pro-ED web sites....
I may choose to email the people who run the competition direrectly.

Also, can you tell me or point me to any resources that can help me design an internet survey? I understand that surveys should be as short as possible (so the subjects don't stop caring in the middle of the test), but there are also many other things I'm probably not aware of.

-M

[rpeebles]

So, to answer:

1. It is not a privacy violation if they give informed consent and know you're doing it. We did not use IP address tracking for our larger study, but instead used hits, but for a one site survey (we were surveying 298 sites) it is not unreasonable. The key is that you don't do it secretly, but let people know that that is how you will be tracking to make sure no one is duplicating their responses. But you do not have to do it that way.

2. However you will do this, you will need informed consent and IRB approval.

3. Many people on these sites are 18 or older, so not everyone will need parental consent. It is certainly a lot more challenging to study people under 18 without parental consent.

4. I think the first thing you have to think about in designing any survey - web included - is that you need to think long and hard about what questions you most want to answer with your study. What are you most interested in learning? What is your hypothesis? What do you think this research will contribute? Once you have answers to these questions, or at least some direction, then you start to design your survey. It's always best if you include as many "validated" questions or measures as you can - those that have already been used in other studies, for example. Making up your own questions or phrasing should only be a last resort. If you let me know your questions, I can help. They don't always have to be short, although you don't want it any longer than necessary. You DO want it well-written and thoughtful.

5. With web research, just like you don't want people to lie to you, you need to be careful not to be deceptive with them. I would definitely not recommend "trolling" their site and asking a casual survey and THEN saying it was a research project.

So - catch me up as your ideas take form - R

[methionine]

Hi,
Sorry it took a while to respond -- I was away and studying for finals--

In response to what you wrote: although a significant part of people on the ED sites are 18 or older, there still are many that are under 18 years of age. Wouldn't that skew my results in a way?

Anyway, more about my ideas. I might still need some more time to sort them out, but ...

I wanted to learn about Asian-Americans' attitudes towards eating disorders, and whether or not that influenced the outcome/prevalence, etc of the disorder in any way. The reason why I was curious about this topic was because I had several friends who suffered from eating disorders (while they were not officially diagnosed, I'm fairly sure that it would have been considered anorexia judging by the criterion listed in the DSMIV).. and their parents did not "believe in" eating disorders themselves.

Sorry if this part is somewhat inconsistent to what I had been writing about earlier-- I'm still trying to get my ideas together. Part of the reason why I thought about doing an online survey was because I thought that this was a feasible way for me to contact those who had been afflicted with disordered eating. On second thought, however, do you think it would be better to simply survey the parents of patients instead? That presents a problem as well: the only way I can think of to contact parents of patients is from hospital lists (if it's even legal to get to those in the first place, that is). That means that the only parents I will be able to even distribute the survey to will have already had their child seen a psychiatrist and/or hospitalized, which would also skew the results drastically. ... I wanted to concentrate on those people whose parents might not even be aware of eating disorders in the first place. A significant number of those people now are online, etc. -- in fact, I know that there have been online groups called "Asians with EDs," etc. ... That's where it all started from.


I really appreciate your help.
-M

[rpeebles]

OK.

1. All surveys are skewed. There are limitations to every choice you make. That's why you develp your question carefully, and then design the best (and most feasible) study you can to answer it as responsibly as possible. Enough said.

It will be MUCH harder to study people under 18. Near impossible, if you want it to be published. This is because it will be very hard to get IRB approval to study them without parental consent, but to get parental consent you REALLY select out an abnormal group of kids for these sites - kids that are comfortable w/their parents knowing they're visiting such a site, which is a very unusual subset.

If you start with people 18 or older, you will start to get information, and potentially could get data that might support the need to study people under 18 - data you would really need if you wanted to convince an IRB that they should waive parental consent. But it does skew your data. It's just that I'm not sure how else you would practically go about this.

2. If you survey the parents, you will get different information. And you're right, you will only be able to access parents that have recognized that their kid has a problem and have sought care - again, an unusual subset. You could screen a random group of parents about their attitudes about their kids' eating and weight, but their kids might not have an eating disorder.

3. So, your question is developing, and that's fine. But right now, it's still not really a question yet. Take a stab at an honest-to-goodness question (ex: Do Asian parents' knowledge about eating disorders impact the severity of disordered eating in their children?) and then take an honest-to-goodness stab at a hypothesis, or a possible answer, to your question. I would recommend doing a literature review first to develop your question further - by looking at http://www.pubmed.org and at the psychologic/social science literature as well - to see what others have already published on your areas of interest. If you don't have access to any medical/psychology literature through your library, let me know and I'll see if I can maneuver some access for you.

4. I think you're right - with what you are expressing an interest in, I think online is a great way to go, and I think the over-18 group will give you a good start. Research is often done in stages. This would be your first stage.

OK, you're making progress.

Let me know when you have more.....-R

[rpeebles]

you might also want to check out the Pew Internet and American Life project - which surveys teens on the Internet - and also the kaiser Family Foundation work - and find out how they got consent for teens to answer their surveys....