Survey Design

Question 1

Surveys vary according to the following main dimensions:

Answer 1

1. the nature of unit – institutional (e.g. schools, villages, health facilities, households), individual (e.g. general adult population, individuals with a specified disease) or events (e.g. practitionerpatient consultations).
2. method of data collection – face-to-face interviews, telephone interviewing, self-completion questionnaires (postal or otherwise), observation, clinical examination, laboratory analysis of specimens, or a combination of these.
3. single measures or multiple measures – single-round or cross-sectional surveys versus longitudinal or prospective studies.
4. whether sample is designed to be representative (i.e. a probability sample) or not (informal, convenience, purposive and snowball samples, etc): quota sampling is an intermediate strategy.

Question 2

Feasibility

Answer 2

Large-scale, representative surveys are most feasible when:
• the subject matter of the investigation is familiar and well-explored;
• major measurement problems have been overcome and reasonable expectations of high data reliability/validity exist;
• a sampling frame is available or can be developed;
• subject recruitment is straightforward

Question 3

Value

Answer 3

Representative surveys are particularly valuable – indeed indispensable – when:
• the magnitude of a specific problem needs to be assessed;
• trends over time or differences between groups are to be assessed;
• interventions have to be evaluated;
• repeatability (over time or by other investigators) is important.

Question 4

Survey

Answer 4

For present purposes, “collection of same or similar information using standardized methods from a sample of units that yields numerical results”.

Question 5

Sample size

Answer 5

In general, the larger the sample the more precise will be the estimates and the greater will be the potential to produce estimates for sub-groups of the study population BUT costs and logistical difficulties of maintaining high-quality data collection will increase, i.e. a trade-off often exists not just between sampling error costs but between sampling and non-sampling error. When the precision of a single estimate (e.g. % with specific disease) or the power to detect a key difference (e.g. incidence of specific disease is 20% lower in the intervention group than control group) is crucial, then standard statistical formulae can be applied to calculate required sample size. With multi-purpose surveys, the decision, in reality, is often pragmatic. Note though that costs are approximately proportional to sample size but precision increases approximately in proportion to the square root of sample size – as one spends more, one gains less from doing so.

Question 6

Sample design

Answer 6

To draw a representative sample, a sample frame (an exhaustive list of units) must be found or generated. This is sometimes straightforward e.g. list of medical practitioners,patients, school children.

Information about the units that is available in the sampling frame can be used to stratify the population into sub-groups of relevance (e.g. urban and rural settlements). A sample is then drawn for each stratum. If it is important to be able to make separate estimates for small sub-groups (e.g. the urban population of a predominantly rural country), it can be over-sampled and then weighted down during analysis when making estimates for the entire population. Implicit stratification involves ordering the list of units in ways that may be relevant to outcomes (e.g. listing children in a school by school year, school class and home address rather than alphabetically) and then selecting from it systematically (i.e. with random start but at a fixed interval). Stratification reduces sampling error.

Nearly all population surveys that involve face-to-face interviews or other forms of field work use multistage clustered designs to reduce travelling time and therefore data collection costs. First, primary sampling units (PSUs) are selected (e.g. areas or clinics) and then the units of interest are sampled within the selected PSUs (e.g. households, individuals or consultations). When no sampling frame exists for all units of interest, such designs allows the survey to sample PSUs and only list the final units of interest within the selected PSUs. (N.B. in developing countries that lack lists of addresses, listing usually involves mapping). Because households in the same locality tend to resemble each other more than a random selection of households from the whole of a population, clustering increases sampling error. I

Question 7

Data collection methods

Answer 7

Leaving aside biomedical data and direct observation, the main options in industrialized countries are self-completion questionnaires, telephone interviews, and face-to-face interviews. In countries in which phone ownership is low and levels of illiteracy high, there is no real alternative to the face-to-face method. Self-completion questionnaires (often in form of postal surveys) are cheap and have advantage of potential anonymity (therefore often preferred for sensitive topics – sex, drugs, etc). BUT response rates are often low, the questionnaire must be kept short and simple, no opportunity exists to ensure the respondent understands the questions, and missing answers common. Telephoning interviewing, in general, yields similar results to face-to-face method and is much cheaper BUT it is difficult to ensure representative sample.

Question 8

Maximising data quality

Answer 8

Achieve higher response rates by: advanced warning (letters, community liaison, appearing on radio programmes, etc); several call-backs when selected respondents are not at home; use of different interviewers to try to overcome refusals. Achieve high quality of responses by: pre-testing of instruments; translation and back-translation of instruments into local languages; a pilot survey to test the entire field operation, thorough training of field staff including practice interviews; close supervision of fieldwork; and spot checks by senior staff.