SMHS Surveys
Contents
Scientific Methods for Health Sciences - Surveys
Overview
Survey methodology studies on the sampling of individual units from population then apply survey data collection techniques such as questionnaires to improve the number and accuracy of responses to surveys. The ultimate goal is to make statistical inferences about the population studied, which would of course, depends strongly on the survey questions provided. The commonly used survey methods include polls, public health surveys, market research surveys, censuses and so on. Surveys provide important information for public information and research fields and are widely applied in varieties of fields such as marketing, health professionals, sociology and so on. In this lecture, we are going to present a general introduction to surveys and various methods used in surveys will be illustrated with examples.
Motivation
Surveys may be one of the most commonly used methods for data collection. The questions used in the survey are of significant importance in collecting enough data to make statistical inference of the population. There are various ways of collecting data in surveys and they all have their strengths and weakness and are applied to different kinds of data. So, how do surveys work? And how to perform a good survey?
Theory
1) Surveys: Surveys are made of at least one sample (or the full population in the case of a census), a method of data collection and individual questions that become data, which can be further analyzed statistically.
- A single survey may focus on different types of topics such as preferences, opinions, behavior or factual information depending on the purpose of the study.
- Given that survey is based on one sample of the population, the success of research depends largely on the representativeness of the sample with respect to the target population.
- Surveys aim to identify principles about the sample design, data collection instruments, statistical adjustment of data, data processing, and final data analysis that can be used to create systematic and random survey errors, which can sometimes be analyzed in connection with survey cost. The cost constraints are sometimes framed as improving quality within cost constraints, or alternatively, reducing costs for a fixed level of quality.
2) Survey methodology topics The most important challenges of a survey method include making decisions on how to: (1) identify and select potential sample members; (2) contact sampled individuals and collect data from those who are hard to reach or reluctant to respond to the questions; (3) evaluate and test questions; (4) select the mode for posting questions and collecting responses; (5) train and supervise interviewers if they are involved; (6) check data files for accuracy and internal consistency; (7) adjust survey estimates to correct for identified errors.
- Selecting samples: there are mainly two types of survey samples: probability samples and non-probability samples. Stratified sampling is a method of probability sampling where the sub-population within the population are identified and included in the sample selected in balanced ways.
- Modes of data collection: the choice between various modes of administering a survey can be influenced by several factors including costs, coverage of the target population, flexibility of asking questions, respondents’ willingness to participate and response accuracy. Mode effect created by different methods can change the way the respondents answer. The most commonly used modes of administration can be summarized into some main categories including telephone, mail, online surveys, personal in-home surveys, personal mall or street intercept survey and hybrids of the above.
- Telephone: use of interviewers encourages sample persons to respond, leading to higher response rates; interviewers can increase comprehension of questions by answering respondents’ questions; fairly cost efficient, depending on local call charge structure; good for large national sampling frames; some potential for interviewer bias; cannot be used for non-audio information; three main types of telephone include traditional telephone interviews, computer assisted telephone dialing and computer assisted telephone interviewing (CATI).
- Online surveys: (1) Web surveys are faster, simpler, and cheaper. However, lower costs are not so straightforward in practice, as they are strongly interconnected to errors. Because response rate comparisons to other survey modes are usually not favorable for online surveys, efforts to achieve a higher response rate (e.g., with traditional solicitation methods) may substantially increase costs. (2) The entire data collection period is significantly shortened, as all data can be collected and processed in little more than a month. (3) Interaction between the respondent and the questionnaire is more dynamic compared to e-mail or paper surveys.[6] Online surveys are also less intrusive, and they suffer less from social desirability effects. (4) Complex skip patterns can be implemented in ways that are mostly invisible to the respondent. (5) Pop-up instructions can be provided for individual questions to provide help with questions exactly where assistance is required. (6) Questions with long lists of answer choices can be used to provide immediate coding of answers to certain questions that are usually asked in an open-ended fashion in paper questionnaires. (7) Online surveys can be tailored to the situation (e.g., respondents may be allowed save a partially completed form, the questionnaire may be preloaded with already available information, etc.). Online questionnaires may be improved by applying usability testing, where usability is measured with reference to the speed with which a task can be performed, the frequency of errors and user satisfaction with the interface.
- Mail: the questionnaires may be handed to the respondents or mailed to them but in all cases they are returned to the researcher via mail; the advantage is that the cost of mail survey is very low and there is no interviewer bias, however there might be long delays and are not suitable for issues that may require clarification. The response rates can be improved by using mail panels, monetary incentives and improve the class of mail through which the surveys were sent.
- Face-to-face: suitable for locations where telephone or mail are not developed; potential for interview bias; easy to manipulate by completing multiple times to skew results.
- Mixed-mode surveys: with the introduction of computers to the survey process, survey mode now includes combinations of different approaches (mixed-mode designs). Some commonly used methods include computer-assisted personal interviewing (CAPI), audio computer assisted self-interviewing (audio CASI), computer-assisted telephone interviewing (CATI) and interactive voice response (IVR).
- Cross-sectional and longitudinal surveys: the former involves a single questionnaire administered to each sample member and the latter refers to surveys, which repeatedly collect information from the same people over time. Longitudinal surveys are usually considered analytical advantages but can be challenging to implement successfully. As a result, specialist methods have been developed to select longitudinal samples to collect data repeatedly, to keep track of sample members over time, to keep respondents motivated to participate and to process and analyze longitudinal survey data.
- Response formats: there are two kinds of formats of questions used in surveys: the open-ended questions, which requires the respondents to formulate their own answers and closed-ended questions, which require the respondent to pick and answer from a given list of mutually exclusive and exhaustive options. There are four types of response scales for closed-ended questions including dichotomous (two options), nominal-polytomous (more than two unordered options), ordinal-polytomous (more than two ordered options) and bounded continuous (continuous scaled questions). A respondent’s answer to open-ended question can be coded into a response scale afterwards or analyzed using more qualitative methods.
- Nonresponse reduction in telephone and face-to-face surveys: (1) Advance letter: sent in advance to inform the sampled respondents about the upcoming survey. It should be personalized but not overdone; (2) Training: the interviewers thoroughly trained in how to ask respondents questions, how to work with computers and making schedules for callbacks to respondents not reached; (3) Short introduction: the interviewer should always start with a short instruction about him or herself about their names, the institute she is working for, the length of the interview and goal of the interview; (4) Respondent-friendly survey questionnaire: the questions asked must be clear, non-offensive and easy to respond to for the subjects under study.
- Interviewer effects: The effects of the surveys may be affected by physical characteristics of the interviewer including race, gender, and the relative body weight (IBM). These characteristics of the interview are particularly influential when the questions are related to the interviewer trait. While interviewer effects have been investigated mainly for face-to-face surveys, they are also shown to exist for interview modes with no visual contact such as telephone surveys and in video-enhanced web surveys.
3) Simple examples of survey can be viewed here:
Applications
1) This article characterize response rates for mail surveys published in medical journals; and determined how the response rate among subjects who are typical targets of mail surveys varies by evaluating the contribution of several techniques used by investigators to enhance response rates. Methods. One hundred seventy-eight manuscripts published in 1991, representing 321 distinct mail surveys, were abstracted to determine response rates and survey techniques. In a follow-up mail survey, 113 authors of these manuscripts provided supplementary information. Results. The mean response rate among mail surveys published in medical journals is approximately 60%. However, response rates vary according to subject studied and techniques used. Published surveys of physicians have a mean response rate of only 54%, and those of non-physicians have a mean response rate of 68%. In addition, multivariable models suggest that written reminders provided with a copy of the instrument and telephone reminders are each associated with response rates about 13% higher than surveys that do not use these techniques. Other techniques, such as anonymity and financial incentives, are not associated with higher response rates. Conclusions. Although several mail survey techniques are associated with higher response rates, response rates to published mail surveys tend to be moderate. However, a survey's response rate is at best an indirect indication of the extent of non-respondent bias. Investigators, journal editors, and readers should devote more attention to assessments of bias, and less to specific response rate thresholds.
2)This article aims to survey operating theatre and intensive care unit staff about attitudes concerning error, stress, and teamwork and to compare these attitudes with those of airline cockpit crew. This study used the cross sectional surveys involving urban teaching and non-teaching hospitals in the United States, Israel, Germany, Switzerland and Italy and included 1033 doctors, nurses, fellows and residents working in operating theatres and intensive care units and over 30000 cockpit crew members in the study and measured the perceptions of error, stress and teamwork. This study concluded that: medical staff reported that error is important but difficult to discuss and not handled well in their hospital. Barriers to discussing error are more important since medical staff seem to deny the effect of stress and fatigue on performance. Further problems include differing perceptions of teamwork among team members and reluctance of senior theatre staff to accept input from junior members.
3) This article provided a review of epidemiological studies of pervasive developmental disorders (PDD), which updates a previously published article. The design, sample characteristics of 32 surveys published between 1966 and 2001 are described. Recent surveys suggest that the rate for all forms of PDDs are around 30/10,000 but more recent surveys suggest that the estimate might be as high as 60/10,000. The rate for Asperger disorder is not well established, and a conservative figure is 2.5/10,000. Childhood disintegrative disorder is extremely rare with a pooled estimate across studies of 0.2/10,000. A detailed discussion of the possible interpretations of trends over time in prevalence rates is provided. There is evidence that changes in case definition and improved awareness explain much of the upward trend of rates in recent decades. However, available epidemiological surveys do not provide an adequate test of the hypothesis of a changing incidence of PDDs.
Software
Problems
Suppose, you want to study on the effectiveness of a new released drug on headache, can you come up with a short survey (5 or 6 questions) on a group of patients with headache who have been using this drug for the past three months? What kind of survey mode would you choose here and why? (open question)
References
Statistical inference / George Casella, Roger L. Berger
Sampling theory and methods / S. Sampath
- SOCR Home page: http://www.socr.umich.edu
Translate this page:
