CHD4630 Midterm

Question 1

The four elements of research design

Answer 1

Variables, Sample, Location, and Method

Question 2

Element of Research Design: Variable

Answer 2

what kind of data to collect

Question 3

Element of Research Design: Sample

Answer 3

from whom to collect data

Question 4

Element of Research Design: Location

Answer 4

where to collect your data

Question 5

Element of Research Design: Method

Answer 5

how to collect and analyze your data

Question 6

Clarifying your research focus

Answer 6

Clarify & focus in definitions help your research question be understood by readers
-What you aim to achieve
Sub-questions provide direction and structure for your research
-Ex: What role do family and friends play in motivating Facebook use? Extent of particular software applications motivate?

Question 7

Strengths of Qualitative research Design

Answer 7

Emergent research design
-info. explore may influence future data gathering decisions (flexibility)

Exploration
-useful as first step in understanding ideas, perspectives and phenomena

In-depth understanding
-can enable deeper understanding of individuals’ perspectives and experiences

Question 8

Weaknesses of Qualitative research

Answer 8

Lack of Breadth
-hamper ability to gather data from a broad range or large number of cases
-possible skewed understanding

Time-consuming
-voluminous data
-higher costs and longer time to prep data for analysis

Limited ability to generalize to population
-experiences or perspectives of small numbers

Question 9

Strengths of quantitative research

Answer 9

Easier to generalize results

Large scale
-collecting data from large numbers, greater confidence not biased

Statistical analysis
-complex and powerful analysis that can identify subtle statistical patterns in the data

Visualizations
-easily translate data

Question 10

Weaknesses of Quantitative data

Answer 10

Loss of depth and context
-individual-level patterns can be obscured, important contextual factors may be neglected

Limitations in accuracy
-limited depth of measurement or respondents not fully understanding or engaging with survey questions

Disconnected
-“Operationalizing” concepts and turning into Qs can create measures that are removed from real-world phenomenon
-may measure what not intended

Question 11

Elements of Action research

Answer 11

Cycle of taking action
-Ex: Implementing intervention

Evaluating an action’s effectiveness
-Ex: Did intervention address research question effectively

Using that knowledge to inform future action
-Ex: modifying intervention

Question 12

Action research does what?

Answer 12

Enables researchers to intervene with aims of improving or affecting something about your sample while creating new research knowledge
-may be specific system, service, product or outcome
-refined product & new knowledge
-usually continue until resources dry up or evaluation results become satisfactory

Question 13

Evaluation research

Answer 13

= focus on assessing whether objectives or intended outcomes have been achieved by a particular intervention or initiative
-real life events or contexts
-Social, policy, learning, or business
-aims to provide useful information on how and why particular interventions or processes work or fail
-goal is to inform future decision-making

Question 14

Process of writing literature review includes

Answer 14

1. Focus you literature review while accounting for the big picture
2. Search for books and articles
3. Organize your sources
4. Continually concept map
5. Begin-in-depth reviews of categories
6. Write your literature review

Question 15

Process of writing literature review: Focus you literature review while accounting for the big picture

Answer 15

Place your topic within a relatively broad research and policy context to establish why your topic is interesting and orient your readers
Save time by defining your focus as narrowly as possible
The more tightly focused your topic, the more efficient and effective your research can be
If you don’t yet have a specific research question, try to sharpen your focus by talking to your colleagues/supervisor or doing initial reading about general topic
Keep making adjustments as you develop your literature review

Question 16

Process of writing literature review: Search for books and articles

Answer 16

Investigate a lot of different approaches and ideas (open and curious, while maintaining goal)
Develop a search plan (words and phrases likely to lead to information)
-Enables you to maintain good coverage in your search and to explain how you went about your search in your research report
Evaluate & access potential sources
Weigh strengths and weaknesses
Peer review is gold standard for source credibility
-Widely accepted quality control mechanism in social research (can vary, so be critical)
Start with electronically accessible, saving difficult access for a later detailed search
Ensure relevance by first reading the title, then read the abstract or summary, and then by downloading or recording the details of the publication and its citation

Question 17

Process of writing literature review: Organize your sources

Answer 17

Organize them into categories, with a folder on your computer for each category
Place downloaded publications into these folders along with a document containing copied and pasted details of publications you couldn’t download
Once you’ve created a preliminary set of categories, take stock to see if you can combine categories, if you need to expand your search or if you should re-focus away from any categories

Question 18

Process of writing literature review: Begin-in-depth reviews of categories

Answer 18

Shouldn’t just report, but critically evaluate your sources’ claims
Evaluate whether the methods used in the study provide sufficient evidence for its claims
Don’t assume that established researchers are producing good research
Take 2 different sets of detailed notes on your sources
First set summarize the methods, findings and a few quotes that effectively describe the research results or conclusions
Second, start identifying connections (similarities or divergences) with other literature

Question 19

Process of writing literature review: Write your literature review

Answer 19

Clear organization
Include introduction that emphasizes its importance, as well as stating what research has been conducted and how the section will be structured
Next, body section being the main part where you present your summaries, evaluation and application of the literature to your topic
Conclude by summarizing the gaps your research will be addressing

Question 20

Reviewing your literature includes

Answer 20

Evaluation of core content
Assessment of evidence
Coverage of the topic

Question 21

Reviewing your literature: Evaluation of core content

Answer 21

Has the author clearly defined the topic? How significant?
Whats author’s theoretical framework?
Does the information come across as opinion or propaganda?
Are assumptions reasonable?
Does the author take a radically different approach from others? Could it have been better if approached from a different perspective?

Question 22

Reviewing your literature: Assessment of evidence

Answer 22

Are there methodological shortcomings? If so, how fundamental to the author’s claims are they? What are the implications for the author’s claims? Is the chosen research method leading the author to make certain claims?
Are claims supported with sufficient and credible evidence? Is the author overreaching with the publication’s claims?
Does the article rest on primary or secondary research (or just the author’s arguments or application of the theory)?

Question 23

Reviewing your literature: Coverage of the topic

Answer 23

Does the source confirm or support other sources you’ve read, update other material, or provide new insights?
Does the publication appear to be of good quality?

Question 24

What does it mean to be analytical?

Answer 24

Summarize publications, presenting any major patterns in the literature and relationships between studies
Use quotations only when the author says something in a particularly concise way, or when you are going to be disagreeing with the author’s claims
Keep your own ‘voice’ as you write
-may feel pressure to mimic the style or terminology of your sources, but staying authentic helps ensure the review presents your analysis

Question 25

Identifying key gaps in the literature

Answer 25

Finding a research gap that you can address is a good way to demonstrate the value of your research Helpful to ask self: Are there areas of worthwhile study relating to your topic that have not been covered by other researchers? Are relevant studies neglecting a particular type of evidence? Is there a relevant theoretical concept that has not yet been applied to the topic?

Question 26

What is research ethics?

Answer 26

= values or principles that are integral to your research objectives and define your responsibilities to your participants, your institution and yourself -Provides guiding principles during the research process -Essential in order to maintain a positive relationship with your research participants that will stand the test of time -Participants need to feel they are protected and not taken advantage of

Question 27

Purpose of research ethics is to

Answer 27

Maintain integrity Protect the welfare of others Build support for you and your research Give you direction when facing challenging situations

Question 28

Purpose of research ethics: Maintain integrity

Answer 28

Promotes good research by establishing that you should be consistently truthful, seek to gain knowledge and avoid error Established a set of expectations for your conduct that prohibits falsifying information, deception or other acts that might bring disrepute to you, the quality of your research or your institution

Question 29

Purpose of research ethics: Protect the welfare of others

Answer 29

A fundamental principle of research ethics is to do no harm to others You must ensure your research does not put at risk the welfare or reputation of those directly or indirectly involved in your research

Question 30

Purpose of research ethics: Build support for you and your research

Answer 30

Those who participate in or view your research will have a more positive view of both you and your research Improve likelihood of recruiting participants for your future research and gain help from colleagues

Question 31

In regards to using data for purposes other than specified

Answer 31

Always make clear to participants how you will use the data collected from them The person giving you permission to use their information has a right to know how it is going to be used and then decide whether this is an acceptable purpose or not You can only use the information gained from participants for the specific purpose that you declare to them and they authorize

Question 32

2 different methods of achieving informed consent

Answer 32

Writtent and Oral conset

Question 33

Written consent

Answer 33

Generally viewed as gold standard Offers the most protection from a legal standpoint

Question 34

Oral consent

Answer 34

Generally considered a weaker form of consent, although we contest this view In some situations, gaining consent verbally is superior to using a written form, such as when a signature can be a threat to maintaining a participant’s anonymity

Question 35

How to achieve informed consent

Answer 35

Carefully weigh the relative strength of the form of informed consent vs privacy needs to your respondents When seeking informed consent, make clear: Participation is voluntary They have a right to refuse to participate They can withdraw from participation at any point for any reason (they don’t have to tell you what that reason is) Should specify early in the process whether the data will be anonymized or held confidentially When obtaining, you may need to make it clear that there are limits to the confidentiality you can offer (ex: report violent behavior to authorities) Permission to electronically record a research interaction must be explicitly granted

Question 36

Ethics an vulnerable populations

Answer 36

Additional ethical implications Carefully consider whether you are prepared and qualified to tackle Potential for causing harm to your participants is greater Consider that you may need additional training before embarking with vulnerable population Assess whether the vulnerable participants are capable of giving informed consent Must be able to understand what participating in your research would mean, as well as the nature of the risks associated with it If there are guardians responsible for the well-being of your prospective participants, you should also seek their informed consent When working with children, you generally need the consent of their parent, guardian or school (if it has the delegated authority to grant consent) Also important to gain the direct consent of the child if possible Participants of all ages may also be vulnerable Difficult decision about whether to engage someone in research that will likely be personally upsetting (e.g., victim of abuse)

Question 37

Employ neutral phrasing

Answer 37

Investigate phrasing that may upset or insult your participants (Useful resources:) Speaking to gatekeepers or those with knowledge of the participants’ social group Viewing your research from the perspective of your participants Creating a list for yourself of verbal ‘no-go’ terms or topics Be familiar with your participants’ preferred phrasing and mirror it where possible Employing the phrase ‘so called’ can help distance yourself from controversial term

Question 38

Stress and decision making

Answer 38

Stressful emotions can distort your research perspective Your emotional response can be instructive, pointing you towards important insights But it can also override your objectivity, making you vulnerable to bias Repeated exposure to content can make you think that upsetting incidents are more common than they really are May cause you to lose perspective and overestimate a phenomenon’s importance or relevance to your research Raw emotional experiences can make you identify too closely with your participants While you certainly want to be able to empathize with your participants’ perspectives, you must maintain emotional boundaries Getting too emotionally involved in your participants’ problems can get in the way of successfully completing your research

Question 39

Indicators of emotional risk in research: lower risk

Answer 39

Grant-funded Familiar environment Risk to physical health, security and property are better understood Familiar participant community or topic Team-based research Allows you to share the workload and responsibilities Strong social support network

Question 40

Indicators of emotional risk in research: higher risk

Answer 40

Self-funded Language barriers Unknown environment Risk to physical health, security and property are poorly understood Solidarity research Emotionally-charged or unfamiliar topic Research requires developing new skills, such as a new statistical test

Question 41

What is sampling?

Answer 41

Used when your population is too large for you to study everyone (almost always) Pick a sub-set of population; a sample Can be used for both qualitative and quantitative research

Question 42

Quantitative goal:

Answer 42

make accurate generalizations about the population’s characteristics

Question 43

Qualitative goal:

Answer 43

get beneath surface of phenomenon

Question 44

Supporting your knowledge claims

Answer 44

Always include a methods section which allows your readers to assess the validity of the claims you make, and to identify any flaws in your research design A good methods section includes: Why you opted for a partiular sampling strategy and who you collected data from How you gathered your data, identify and operationalize the variables you investigated How you analyzed the data

Question 45

Probability Sampling includes

Answer 45

Simple Random Sampling Stratified Random Sampling Cluster Sampling Multistage Sampling Systematic Sampling

Question 46

Non-Probability Sampling

Answer 46

Don’t meet the standard of every member of equal probability of selection for all members of your target population Generally much cheaper and easier than probability sampling, but should be used with caution Can’t use inferential statistics to generalize from your sample to the population as a whole Your conclusions should therefore be more tentative

Question 47

Non-probability Sampling includes

Answer 47

Convenience Sampling Quota Sampling Snowball Sampling Purposive/Theoretical Sampling

Question 48

Simple Random Sampling

Answer 48

Ideal to aspire to as any of your individuals in your target population have an equal chance of being selected to be part of your sample Members of your population are selected using random numbers for inclusion in sample Requires little or no information about your population’s characteristics an is easily the most robust for of sampling Becomes exponentially harder as the population size increases and the accessibility of your population decreases

Question 49

Stratified Random Sampling

Answer 49

Involves dividing the population you are researching into exclusive sub-groups, or strata (age, gender, country, ethnicity, etc.) Useful if you are looking to compare the attitudes of particular sub-groups (e.g. men to women) Also useful if you are trying to compare sub-groups with drastically different sizes Requires significantly more information about your population and is much more complicated to perform than simple random sampling

Question 50

Cluster Sampling

Answer 50

Useful when your population is dispersed across a large geographical area Divide the population into geographical groups, or clusters, often based on pre-existing boundaries Then randomly select clusters: your sample consists of everyone in the cluster you selected Can also be used to gather a representative sample when you have a large group of people in a space (e.g. professors, festivalgoers, etc.) Physical space can be divided into imaginary grids

Question 51

Multistage Sampling

Answer 51

Combination of sampling techniques Example: First randomly select clusters (cut down on cost by limiting the geographic disbursement of your sample) Then stratify the individuals in those clusters (to ensure your sample is representative) Then randomly select participants from the stratified groupings (to avoid sampling bias)

Question 52

Systematic Sampling

Answer 52

Occasionally you will find that it’s just not practical, or possible, to use a truly random sampling method Use ‘selection rules’ to mimic the objectivity of the random selection Often, the selection rule is as simple as picking every Xth person to meet some criteria (like entering a room) Not as good as strict random sampling, it can come relatively close to producing a random sample

Question 53

Convenience Sampling

Answer 53

Selects the members of a population who are the easiest for the researcher to access Can involve surveying your family or people you stop on the street, your Facebook friends, or anyone else who is easy to access Unlikely that the people sampled through this method are representative of a wider population Can be an effective way to test that survey questions, or conduct other preliminary pilot testing activities

Question 54

Quota Sampling

Answer 54

Selection of a sample according to a predetermined quota (or number) on a non-random basis Two ways: Proportional quota sampling: selecting participants for your sample in proportion to some characteristic of the population as a whole Non-proportional quota sampling: having minimum number of people you wish to sample within each category, rather than a maximum

Question 55

Snowball Sampling

Answer 55

A technique where participants in your research are used to find additional participants Almost guarantees your sample won’t be representative However, it’s used when there may be no other way to access your population other than personal introductions Ex: Those with an illness

Question 56

Purposive/Theoretical Sampling

Answer 56

Uses researcher’s judgment to select participants who are likely to offer particularly useful insights Need to use ‘special knowledge or expertise about some group to select subjects who represent this population Selection process will be guided by earlier data collection within the project, prior research or theory, as well as the researcher’s instincts

Question 57

Response rate

Answer 57

percentage of those you invite to complete your surveys that actually do so

Question 58

Should keep a refusal log where you write down basic information including

Answer 58

Data/Time of refusal Apparent age of non-respondent Apparent gender of non-respondent Apparent ethnicity of non-respondent Explicit reasons given for the refusal Other possible reasons for the refusal

Question 59

Sampling frame

Answer 59

a summary list of all the members of a population who can be sampled (i.e. those who can be identified, located and contacted)

Question 60

Type 1 error

Answer 60

alternatively known as ‘false positives’; the rejection of a null hypothesis that is actually true Example: conclude that a new medical treatment helps to cure cancer, when it does not, the results would be tragic

Question 61

Type 2 error

Answer 61

‘false negatives’; failure to reject a null hypothesis that is actually false Ex: serious error if you were to dismiss a potential cure for cancer as ineffective when it actually cures the disease

Question 62

The Incentive categories are

Answer 62

Individual, Social, and Ethical

Question 63

Individual Incentives

Answer 63

Direct compensation -Vouchers -Tickets -Gift certificates -Prize draw -Cash (fair market value) Reimbursing expenses -Travel costs -Stipend for meals or alternatively provide refreshments

Question 64

Social Incentives

Answer 64

Charitable -Donation to charity of researcher’s choice -Donation to charity of participant’s choice Community/Society -Details of how findings will benefit the participant’s community or society

Question 65

Ethical Incentives

Answer 65

Access Giving respondents access to the research findings

Question 66

Manage concerns over privacy

Answer 66

Help assuage concerns about privacy by indicating to your participant that: Their information will not be passed to third parties under any circumstances There is no reason to believe that anyone would be specifically hacking your system Every reasonable precaution (such as using password protected computers) will be used to maintain privacy Collected data will be assigned a unique identifying number and the correlating respondent names will be stored on a medium not connected to the internet Establish your credibility as a researcher Take time to build up trust between you and the participant, especially if the topic is particularly personal

Question 67

Soliciting participants: Decision tree includes either

Answer 67

Face-to-Face or Online

Question 68

Soliciting participants: Decision tree for Face-to-face

Answer 68

Sufficient resources to access a big enough sample Potential participant locations are clear Credibility more likely to be gained through face-to-face interaction (depending on topic area) Participants will feel comfortable discussing the topic in public (even if just an introduction)

Question 69

Soliciting participants: Decision tree for Online

Answer 69

Small budget but large sample required Large geographic area needs to be covered Uncertainty around where participants may be located Topic area is sensitive- participants will seek anonymity

Question 70

Conclusion: Strategies and incentives

Answer 70

You should be prepared to demonstrate basic mastery of information about your research topic to assure your participants of your credibility as a researcher You should also avoid terms and phrases that may sound condescending, pompous or unapproachable The greater the time commitment, inconvenience and risk faced by your respondent, the more you will need to offer incentives for participation You may need to work through gatekeepers in order to reach your participants effectively

Question 71

What are surveys a good method for?

Answer 71

Can be used for describing patterns in a large population Can determine individual’s characteristics Can be used to assess social/political conditions from individuals’ perspectives

Question 72

Layout and sequencing

Answer 72

Don’t include too many questions as it may be intimidating and harm your response rate The question order affects the readability of the survey Questions should go from general to specific and from easy to hard Gradual introduction to the topic area Save the demographics

Question 73

Types of survey questions include

Answer 73

Open-ended, close-ended, classification and demographic questions, ranked responses, multiple choice answers: select one, multiple choice answers: all that apply

Question 74

Strengths of Open-ended questions

Answer 74

Respondents can answer in their own terms Limited researcher bias Useful for exploring areas of limited researcher knowledge

Question 75

Weaknesses of Open-ended questions

Answer 75

Time-consuming analysis (coding) Require greater effort from respondents

Question 76

Strengths of Close-ended questions

Answer 76

Easy to administer and complete Easy to process and analyze Enhance comparability of responses

Question 77

Weaknesses of close-ended questions

Answer 77

Generate limited detail Sensitive to design errors Time-consuming to design

Question 78

Classification and Demographic questions

Answer 78

Objective characteristics about respondents Biological, sex, ethnicity, religious affiliation Can be asked in different forms E.g. Age can be an open-ended question, a multiple-choice question or by enquiring about the date or birth Depends on the researcher’s preference Recommended to collect demographics according to Census data questions for consistency (in US)

Question 79

Ranked responses

Answer 79

Used to find out the respondents’ relative preferences about a list of options This can be a good question type for researching comparisons

Question 80

Multiple choice answers: select one answer

Answer 80

This question type provides pre-determined response options for a respondent to choose one answer to a given question

Question 81

This criteria for Multiple choice answers: select one answer is that all response options should be

Answer 81

Exhaustive, Exclusive, Unambigous

Question 82

Exhaustive

Answer 82

everyone fits into at least one category

Question 83

Exclusive

Answer 83

everyone fits into only one category

Question 84

Unambiguous

Answer 84

response categories mean the same to everyone

Question 85

Multiple choice answers: all that apply

Answer 85

This type requires more time to analyze then types where only one answer is selected This can be very useful question type in some cases Ex: How did you hear about this event? Check all that apply/ If this is only used for descriptive statistics, the time taken to analyze the data is reduced

Question 86

Types of survey

Answer 86

Cross-sectional, longitudinal, self-administered, verbally administered

Question 87

Cross-sectional survey

Answer 87

Surveys are done at a single point in time Most commonly used as they are a simple, low-cost option

Question 88

Longitudinal survey

Answer 88

Studies use 2+ linked studies measuring change over a period of time Lapse between studies can be hours, days, weeks, or years Often seen as impractical: time-consuming, costly

Question 89

Self-administered survey

Answer 89

Where respondents read and complete the survey themselves Usually online, but can be distributed face-to-face

Question 90

Verbally administered survey

Answer 90

Where researcher or trained assistants go through the respondents providing their answers verbally or pointing to a chosen category

Question 91

Types of survey bias

Answer 91

Researcher Expectancy effect, Acquiescence bias, Demand characteristics, Social Desirability Bias,

Question 92

Researcher Expectancy Effect

Answer 92

= researchers unintentionally introduce bias by designing survey questions and response options around their existing assumptions Ex: a business’s customer service team expecting positive feedback might unintentionally bias their survey by asking leading questions

Question 93

Acquiescence Bias

Answer 93

= respondents tend to agree with Likert scale (level of agreement) statements Ex: If all such Likert scale statements are framed positively, the results may skew towards agreement

Question 94

Demand Characteristics

Answer 94

= respondents may alter their answers based on what they think the researcher’s preferred result Ex: Being asked to give feedback about a hospital by a uniformed hospital worker may result in more positive responses

Question 95

Social Desirability Bias

Answer 95

= respondents may over-report views and behaviors that are widely praised in society and to make themselves look better Ex: inaccurately reporting higher levels of recycling or charitable donations in order to appear more caring is typical of this bias