Chapter 11: Designing Surveys and Questionnaires

Question 1

What is a survey?

Answer 1

A set of questions that elicits quantitative or qualitative responses

Question 2

What are types of survey formats?

Answer 2

Questionnaire
Interview

Question 3

What is a questionnaire?

Answer 3

A standardized survey, usually self-administered, that asks individuals to respond to a series of questions

Questionnaires are particularly useful as a research method for examining phenomena that can be best assessed through self-observation, such as attitudes, values, and perceptions. The primary disadvantages of the written questionnaire are the potential for misunderstanding or misinterpreting questions or response choices, and unknown accuracy or motivation of the respondent.

Question 4

What is an interview?

Answer 4

the researcher asks respondents specific questions and records the answers.
- Structured: standardized set of questions
- Semi-structured: list of questions and topics
- Unstructured: conversational, informal, discuss issues of concern

Question 5

What are the steps in developing content for a survey?

Answer 5

Question writing
Review the literature
Develop content
- Use existing instruments when applicable
Expert review of draft questions
Pilot testing
Revisions

Question 6

What is an open ended question?

Answer 6

Ask respondents to answer in their own words
Useful for probing respondents’ feelings and opinions without biases or limits imposed by the researcher

Question 7

What is a closed-ended question?

Answer 7

ask respondents to select and answer from among several fixed choices

Question 8

What is branching?

Answer 8

Used to follow up specific answers with more detailed questions
May use skip patterns for persons to who questions are not applicable
May include “none of the above” as a choice
Challenges for data analysis

Question 9

Things to remember when writing questions:

Answer 9

Every question should be answerable by every subject
Questions should be easy to answer
Consider recall of information
Consider if respondents will be honest
Try to use a variety of question types
Questions should generate varied responses

Question 10

Things to remember when wording questions:

Answer 10

Purposeful language
Avoid bias
Clarity
Avoid double-barreled questions
Frequency and time-measures
Sensitive questions

Question 11

Things to remember when formatting a survey:

Answer 11

Question order
- Follow thought process
Length of the survey
- Long surveys less likely to be completed
Demographics
- Include questions that can characterize the sample of respondents

Question 12

What is sampling?

Answer 12

Probability vs non-probability sampling
Sampling errors
Response rates
Sample size

Question 13

What should you remember when contacting respondents?

Answer 13

Contact them prior to the survey
Send cover letter
Follow-up communication
- Reminders to facilitate an increased return rate
- Thank the participants

Question 14

How do you analyze survey results?

Answer 14

Coding
- Closed choice responses
- Open-ended responses
Summarizing survey data
- Figures
- Cross-tabulations: relationship between 2 categorical variables
- Missing data: invalid response

Question 15

Describe ethics in survey research?

Answer 15

Institutional Review Board (IRB) review and approval
Informed consent
- Submission of questionnaire is considered informed consent
- May be included in the cover letter
- Face-to-face interviews provide a form to sign
- For telephone interviews, provide full information at the start of the call to obtain verbal consent

Question 15

Describe ethics in survey research?

Answer 15

Institutional Review Board (IRB) review and approval
Informed consent
- Submission of questionnaire is considered informed consent
- May be included in the cover letter
- Face-to-face interviews provide a form to sign
- For telephone interviews, provide full information at the start of the call to obtain verbal consent

Question 16

What are descriptive surveys?

Answer 16

Descriptive surveys are intended to characterize knowledge, behaviors, patterns, attitudes, or demographics of individuals within a given population. Such surveys are often used to inform marketing decisions, public policy, or quality improvement.

Question 17

How can surveys test hypotheses?

Answer 17

Data can be used to establish outcomes following intervention, explore relationships among clinical variables, or examine risk factors for disease or disability. Surveys can be used for data collection in experimental studies to compare outcomes across groups when studying variables such as attitudes and perceptions.
Standardized questionnaires are also used extensively as self-report instruments for assessing outcomes related to broad clinical constructs such as function, pain, health status, and quality of life. These instruments are designed to generate scores that can be used to evaluate patient improvement and to support clinical decision making, requiring extensive validation procedures.

Question 18

What formats are used for closed-ended questions?

Answer 18

Multiple choice questions
Dichotomous questions: 2 choices
Check all that apply.
Measuring intensity: usually 3-5 options
Checklists
Measurement scales
Visual analog scales
Rank-order questions
Branching

Question 19

What is a probability sample?

Answer 19

one that is chosen at random from a population, where every individual has the same chance of being chosen

Question 20

What is a nonprobability sample?

Answer 20

not chosen at random

Question 21

What are sampling errors?

Answer 21

Sampling error refers to the difference between collected data and the true scores in the population, potentially occurring when the sample is too small or not randomly drawn.

Coverage or nonresponse errors

Question 22

What is a coverage error?

Answer 22

In survey research, coverage error occurs when all the members of a population are not included in the sampling process and, therefore, some will have no chance of being selected.

Question 23

What is a nonresponse error?

Answer 23

Nonresponse error occurs when the people who respond to the survey are different from those who do not respond in a way that is relevant to the study.

Question 24

What is a response rate?

Answer 24

Response rate is the percentage of people who receive the survey who complete it. This is a constant threat to validity in survey research, as response rates tend to be disappointingly low.

Question 25

Research has shown that respondents are more likely to return surveys if:

Answer 25

They know and value the sponsoring organization.
The purpose and focus of the survey are clear.
The invitation to participate is personalized.
They understand why they are being asked and feel that they are a stakeholder.
The topic is important to them.
The questionnaire is clearly designed with a simple layout.

Question 26

Survey sample size determinations are based on three projections regarding the sample and expected responses:

Answer 26

The size of the population from which respondents will be recruited.
The acceptable margin of error
The confidence level desired, typically 95%

Question 27

What should be included in a cover letter?

Answer 27

Start with the purpose of the study
Indicate why the respondent has been chosen for the survey.
Assure the respondents that answers will be confidential and, if appropriate, that they will remain anonymous
Some electronic survey platforms allow for a respondent to enter a personal identification number (PIN) so that their results can be tied to other information or a follow-up survey. If this is an important part of the study, it should be explained in the cover letter as a way of maintaining anonymity.
Give instructions for completing the survey, including how long it should take to fill out.
Provide contact information if they have any questions.
Thank respondents for their cooperation.
If you are providing incentives, indicate what they are and how they will be offered.
Offer an opportunity for them to receive a summary of the report.
Sign the letter, including your name, degrees, and affiliation.