?validity of postal survey with low returns

Ask specific questions about preparing for a science fair, such as: judging, how to set up your display board, preparing a presentation, preparing for questions...

Moderator: berkeleywebs

Locked
cchang
Posts: 4
Joined: Tue Feb 03, 2009 5:04 am
Occupation: NHS
Project Question: Social Survey
Project Due Date: n/a
Project Status: I am conducting my research

?validity of postal survey with low returns

Post by cchang »

28,438 (about 10% of local population) postal survey was sent out and there were 781 correct returns. 781 is 2.7% of ramdomly chosen people for the survey. Therefore the return containing the views of the people is 0.27% of the local population in total. Is this low figure an "acceptable" return i.e. this figure of return is viewed to be an expected average? Can this result be considered to be a valid survey i.e. could one said the opinion of the 781 returns be representative of the whole population of 280,000? Thanks. Carl
MelissaB
Moderator
Posts: 1055
Joined: Mon Oct 16, 2006 11:47 am

Re: ?validity of postal survey with low returns

Post by MelissaB »

There's an interesting discussion of response rates in surveys on Wikipedia: http://en.wikipedia.org/wiki/Response_rate . You might want to consider searching for response rates for surveys like the one you sent out; unfortunately, I am not sure what sort of response rates are typical of postal surveys.

As the link above and this one: http://www.dummies.com/how-to/content/s ... respo.html explain, a low return rate can indeed bias your sample. However, if you have done this survey there is little you can do besides explain what happened and include your response rate.
cchang
Posts: 4
Joined: Tue Feb 03, 2009 5:04 am
Occupation: NHS
Project Question: Social Survey
Project Due Date: n/a
Project Status: I am conducting my research

Re: ?validity of postal survey with low returns

Post by cchang »

Dear Melissa B, thank you for all the info and the links. I have to come clean that I am not actually "preparing for science fair"(I do not know which button to click!). The 28,481 postal survey with only 2.7% (781) return was done by the local health authority. I am trying to find out if their results were biased or not as the patients' "wants" did not reflect the MORI poll nor any other surveys. So I wonder if this low return survey in fact is creditable or not. I am concerned that the health authority trying to use the result of their skewed survey to impose their demands and I am trying to find and prove if possible that there may be some flaws in their work. The first link in your reply is very interesting stating the non-responders may say more about the credibility of the survey. In their survey there are >97% non-responders! Is there any reference on postal survey expected responder rates anywhere? Thanks, Carl
agm
Former Expert
Posts: 289
Joined: Mon May 05, 2008 10:34 am
Occupation: graduate student
Project Question: n/a
Project Due Date: n/a
Project Status: Not applicable

Re: ?validity of postal survey with low returns

Post by agm »

Hi cchang,

Some other concepts that it might be helpful to apply to this situation are self-selection and sample bias:

http://en.wikipedia.org/wiki/Self-selection
http://en.wikipedia.org/wiki/Biased_sample

Basically, the idea is that there can be a correlation between the type of responses provided by a person and his/her likelihood of responding. For example, if the survey consisted of a question like, "Do you like feature X of our service?", people who want it to be changed might be enthusiastic about responding because they see the survey as an opportunity to voice their opinions and get X changed, while people who like X might not be as strongly moved to respond because they are happy with the status quo. Thus, the average opinion of responders can differ quite a bit from the average opinion of the target population.

Amanda
cchang
Posts: 4
Joined: Tue Feb 03, 2009 5:04 am
Occupation: NHS
Project Question: Social Survey
Project Due Date: n/a
Project Status: I am conducting my research

Re: ?validity of postal survey with low returns

Post by cchang »

Dear agm,

Thank you for your help and the links. They are very useful as there were also other surveys that the health authority did definitely with a "biased population". I just try to find out if the low return (2.7%) can in fact render their survey/result invalid! Anyone else out here can give advice ? Thanks Carl.
ChrisG
Former Expert
Posts: 1019
Joined: Fri Oct 28, 2005 11:43 am
Occupation: Research Hydrologist
Project Question: n/a
Project Due Date: n/a
Project Status: Not applicable

Re: ?validity of postal survey with low returns

Post by ChrisG »

cchang wrote:I just try to find out if the low return (2.7%) can in fact render their survey/result invalid! Anyone else out here can give advice ? Thanks Carl.
I think it is impossible to say whether the survey study is valid or invalid without knowing more about the methods and results. Even a very tiny sample % can give accurate results if it is a truly random sample, or if enough is known to be able to correct for observed biases. Also, without knowing the interpretation of the survey, it is impossible to say whether the interpretation is correct, or whether it warrants the position held by the health authority.

For more info, you might want to get a copy of this paper from a local university, or by sending a reprint request to the authors.
Title: Addressing Nonresponse Bias in Postal Surveys
Author(s): MacDonald SE, Newburn-Cook CV, Schopflocher D, et al.
Source: PUBLIC HEALTH NURSING Volume: 26 Issue: 1 Pages: 95-105 Published: JAN-FEB 2009

That paper includes references to several other relevant studies.

Good luck!
Chris
cchang
Posts: 4
Joined: Tue Feb 03, 2009 5:04 am
Occupation: NHS
Project Question: Social Survey
Project Due Date: n/a
Project Status: I am conducting my research

Re: ?validity of postal survey with low returns

Post by cchang »

Dear Chris,

Thank you for your help and the info link. I just read the abstract of that article regarind "Non response bias" and will try to get the full article. I am most grateful for all the experts spending time to response to my question and giving me advice to find out more reading material. I found all the responses very helpful. Thank you once again. Carl
Locked

Return to “Preparing for the Science Fair”