Introduction to Management Flashcards

Question

What is the difference between a Likert scale and a numerical rating scale?

Answer 1

Likert and numerical rating scales are nearly identical. What makes a numerical rating scale unique is that each response option (and sometimes points between un-labeled options) corresponds to a number.

Answer 2

Vagueness: What is the definition of skipping a meal? Anything less than 3 meals per day? What is the definition of a meal?

Answer 3

Telescoping bias occurs when a respondent includes a behavior, action or event outside of the reference period and is particuarly common with "lumpy purchases"

Answer 4

Social desirability bias occurs when respondents give an answer that they believe is "socially acceptable or desirable". In this case, it is not a measurement effect, as measurement effect affects behvior itself and not only the response to the question

Answer 5

Return to both treatment and control villages and re-conduct the endline with paper surveys Using the original electronic data would likely introduce attrition bias, since some villages in the control group are more likely to have misisng data since they are less likely to have electricity. Conducting the endline in the control group with a different method or at a different time might introduce systematic error that biases our results. Only surveying both groups at the same time with the same method would ensure we have no error.

Answer 6

``` Changes necessary to achieve the final outcomes. Usually changes in: • Knowledge & beliefs • Attitudes & aspirations • Capacity & ability • Decisions, behaviors & actions ```

Answer 7

To measure outcomes (long-term, intermediate, first order, second order, inputs, outputs, etc.); covariates (provide background on respondents, classify respondents behaviors, reduces standard error); treatment compliance (individual & group level; predictors of compliance); heterogenous treatment effects; context for external validity

Answer 8

Impact (goal/overall objective) Outcome (project objective) Outputs Inputs (activities)

Answer 9

Objectives/Hierarchy Indicators Sources of Verification Assumptions/Threats

Answer 10

* What data do you collect? * Where do you get it? * When do you get it?

Answer 11

• Obtained from other sources – Publically available – Administrative data – Other secondary data Collected by researchers – Primary data

Answer 12

Information provided by a respondent ○ Could be through a survey, exam results, etc. ○ Information about a person, household, possessions Automatically generated ○ Automatic tollbooths – detailed individual data ○ Or, a sensor picking up data all the time (not about a single person) Information NOT about a person/household/possessions Pollution monitors, etc. Still an active data collection process most likely, but not based on a person

Answer 13

* Surveys * Exams, tests, etc. * Games * Vignettes * Direct Observation * Diaries/Logs * Focus groups * Interviews

Answer 14

• Interviewer administered – Paper-based – Computer-assisted/ Digital – Telephone-based • Self-administered – Paper – Computer/Digital

Answer 15

``` • Baseline • During the intervention – Process, Monitoring of intervention • Endline • Follow-up • Scale-up • Intervention: M&E ```

Answer 16

Construct --> Indicators --> Data Collection ("Response") --> Data

Answer 17

Accuracy Unbiasedness Validity Precision Reliability

Answer 18

How well does the indicator map to the outcome? (e.g. IQ tests -> intelligence) Construct --> (Validity) --> Indicators

Answer 19

The measure is consistent and precise vs. "noisy" Construct --> (reliability) --> Indicators --> (reliability) --> Data Collection ("Response")

Answer 20

1. Comprehension of the question 2. Retrieval of information 3. Judgement and estimation 4. Reporting an answer

Answer 21

How well the respondent understands the question i.e. How many times did you consume rice this month? Does this mean just pure rice, or any product that contained rice? Rice flour/rice crackers?

Answer 22

When the respondent thinks about the question, and retrieves the information required to answer Question - When you received your first measles vaccination, on a scale of 1-5, with 1 being painless, and 5 being unbearable painful: what was the level of pain? Probably received this vaccination as a child; too long ago to retrieve accurate information; data that you do collect is likely very inaccurate

Answer 23

When the respondent has to estimate/judge the answer (this should be minimized) For example, I did "X" thing twice last week, so over the past month...that's 2x's a week over four weeks...so probably about 8 times.

Answer 24

Respondent actually gives a verbalized response at this point. Even after the respondent has gone through comprehension/ retrieval/judgement - there may be some breakdown between the enumerator and respondent. For example - if you ask about illegal drug use, the respondent might have an accurate answer but give an inaccurate one for fear of implications.

Answer 25

Objective - facts Subjective - an opinion, attitude, perception, aspiration, expectations (not verifiable by external observation or records) Quasi-facts - race, religion, ethnicity, gender Here, the response could be motivated by objective factors (biology, etc.) or by subjective factors (personal identity)

Answer 26

state of being | actions and behaviors

Answer 27

Permanent facts i.e. date, district of birth

Answer 28

Can change at any time (i.e. age; district of residence) ...but sometimes in a predictable way ( age, years of education, years of experience); could change due to outside circumstances (HH size, marital status, number of children); others could be fluctuation but at some point become static (highest education level; number of children a woman has had)

Answer 29

i.e. regularly attending school | Useful when asking about frequency

Answer 30

One time and/or infrequent behaviors | i.e. the purchase of a TV

Answer 31

Clearly define variables – What is a household? – Who can be considered household members? – What is considered a room in a household? etc. • Determine the level of precision you need for your study • Useful to look at standard questionnaires for framing these questions • Be aware of local context and culture

Answer 32

Beliefs (Cognitive) Expectations (behavior intentions) Attitudes (Evaluative)

Answer 33

Beliefs (Cognitive) - set of beliefs about the attitudinal object (i.e. how healthy are cigarettes on the following dimensions?)

Answer 34

Expectations (behavior intentions) - Respondents' attitudes in relation to their future actions. (i.e. Do you plan to quit smoking in the next year?)

Answer 35

Attitudes (Evaluative) - Evaluation of the object. (i.e. Do you associate smoking with being cool?)

Answer 36

Subjective questions are usually never the ultimate impact outcome we intend to measure. However, they are very useful as an intermediate outcome. If individuals are unaware of the dangers of cigarette smoke, we probably need to change their beliefs before expecting them to stop smoking. Or subjective measures can help us understand the context and possibly assumptions under which our theory of change will work. For example - an information campaign on the dangers of smoking may not be as effective if people are already aware, but rather think it's worth the cost to look cool.

Answer 37

Sensitive questions will usually cause the respondent to not be honest about their answer, even if they know it. Sometimes the respondent does not know the answer- Unknown Use proxy indicators! Rather than directly asking them the sensitive question - can ask about correlated measures. Must be correlated with construction, and ensure the correlation is dynamic (in terms of the outcome we are measuring, etc.)

Answer 38

Only one proxy indicator is needed to measure the construct For example, as rice yields increase, a larger proportion could be used to replant for the following season, or they may now actually be able to sell some of that rice in the market or traded for other goods. Therefore, unless we are certain that all rice grown is consumed, then rice yielded is not an exclusive indicator of nutrition.

Answer 39

We sometimes want an exhaustive list of indicators for our construct. And we're not always confident that the indicator always moves in the same direction as our construct. We may not want to rely only on one proxy alone. So, for example, if we want to know total calorie consumption, we may need a full accounting of all the food our respondents consume. For each indicator or food item, we ask about the quantity consumed in terms of weight or serving. And then we can ask--then we can convert it into calories and add up all the calories at the end to get total caloric intake.

Answer 40

The middle ground between a single indicator and a comprehensive accounting of all things consumed would be an index. For example - where we create a sample of food items--a consumption basket-- and weigh each item's contribution to the value of index in proportion to the amount a typical household consumes that item.

Answer 41

An item in an index

Answer 42

Sum of all components, no weighting If we believe our index components are comprehensive, or at least representative

Answer 43

Weight components by themes

Answer 44

Need to take into account components that might be negatively correlated We don't want to naively add positives and negatives both signs of empowerment together and have them cancel each other out in the end. We just need to make sure we adjust the signs.

Answer 45

Sometimes, expert judgment is used. For many exams, the teacher may weight a section by how much time was spent on the topic during the semester, or by how important he or she feels it is to getting to the next level of difficulty.

Answer 46

there are statistical methods that weight index components by their actual or potential explanatory power. What these methods all have in common is that they remove any correlation between components so that their latent attributes are not double or triple counted when contributing to the constructor we care about.

Answer 47

a method that standardizes individual components within the index before compiling them. This is also called a z-score index. So for example, if one component was ranked on a scale between 1 and 100 and the other between 1 and 10, the former's contribution would not be 10 times that of the ladder. On the other hand, the more components that are included within a particular theme, the more weight that theme has in the overall index.

Answer 48

• Determine the comparison group against which you will standardize – Baseline – Control group of the same round We can use the entire sample from the baseline, or we can just use the control group observations from the end line. Obviously, if you don't have a baseline measure, you'll need to use the control group. • Standardize individual components of the index – Standardized variable = (variable – variable mean)/ variable standard deviation we demean each observation. In other words, we take the average value of that component from the comparison group, either the baseline sample or the end line sample of the control group, and subtract it from each observation. Then we divide the demeaned value by the standard deviation of the comparison group. • Average components together • Standardize the final index – Standardized index = (index – index mean)/ index standard deviation We take the mean value of the entire index--and that's the entire index not just the comparison sample--and divide it by the standard deviation of the index.

Answer 49

* Forgetting to recode variables that measure bad * Forgetting to recode missing values * Forgetting to account for missing values

Answer 50

• Principal component analysis (PCA) – Reduces multidimensionality of data • Seemingly Unrelated Regression Estimation (SURE) With a principal component analysis or other similar methods, we ensure components are combined into themes to maximize variation. Or in other words, we adjust for inter-component correlation within each theme. And then we adjust for the inter-themed correlation at the end. The SURE method (or seemingly unrelated regression estimation) does basically the same thing without creating themes first.

Answer 51

Open-ended questions; Responses: Verbatim, Numeric Field-coding questions Close-ended questions: Single response, Multiple responses, response filters, ratings, rankings

Answer 52

Respondents are allowed to talk through responses, rather than respond to pre-selected codes

Answer 53

``` – Coding free response material is: • Time consuming • Costly • Induces coding error Requires good interviewer skills in recognizing ambiguity of responses and probing (if required) ```

Answer 54

• Verbatim responses – E.g.: What are the most significant health concerns faced by you and your family? – Best to use when you don’t know too much about the likely responses – Requires good interviewer skills in recognizing ambiguity of responses and probing (if required)

Answer 55

Numeric responses – E.g.: What is your age? – Often associated with demographic variables – E.g. How many times did you visit the hospital in the past 30 days? – Units must be explicit – E.g. How many times did you consume in the past 7 days? The fact that we're asking for a numeric response is restricting enough that it's easy to convert into data. However, we may run into problems if there are implied units, but those units are not made explicit.

Answer 56

Respondents are presented with a set of pre-coded responses to choose from • Response categories usually generated through cognitive interviews, focus group discussions and pretesting • Respondents are given both the topic and the dimensions on which answers are wanted Often field-tested first, to ensure there isn't a lot of misinterpretation

Answer 57

yes/no or true/false question Did you work as a hired laborer in the autumn season? A. Yes B. No

Answer 58

Another example is a typical standardized exam where you fill in the bubble A, B, C, D, or E. Sometimes these are called multiple choice questions. However, they should be not confused with multiple choice responses. Because of the potential for confusion, we usually refer to multiple choice responses as choose all that apply or select all that apply questions ``` In which seasons did you work as a hired laborer during this year? (SELECT ALL THAT APPLY) A. Autumn B. Spring C. Summer D. None ```

Answer 59

– Likert – Numeric – Frequency

Answer 60

First, it could mitigate privacy concerns the respondent may have. Or if it's a question that requires estimation or recall--for example, how much rice did you consume last month--it could help the respondent if they're unsure of the precise answer. Forcing them to give a numeric answer rather than choosing a range may lead them to round up or round down, which could bias the results. Sometimes we want to know the reasoning behind people's opinions, actions, behaviors, or decisions. Even if we see a pattern amongst a majority of the population, that pattern may not hold for each and every respondent. It would be presumptuous to assume it did.

Answer 61

Do you strongly agree, agree, neither agree nor disagree, disagree, or strongly disagree? Central tendency bias

Answer 62

Sometimes it's set up so that the first option is very positive like strongly agree. And the second option is very negative like strongly disagree. If it were numeric, it might range from say plus 2 to minus 2. This would be called a bipolar scale.

Answer 63

Other times, it's set up so that the last option is equivalent to a zero. So do not agree rather than strongly disagree. So the numeric equivalent might be zero to four.

Answer 64

• One thing to note here is the inclusion of a middle alternative. In this case, the neutral option. A middle alternative offers an indifference point between being for or against a particular view. Providing a middle alternative has pros and cons. It may be the best option for those who are truly indifferent. However, sometimes respondents will subconsciously default to a middle option because cognitively it is least taxing. This is known as central tendency bias. Central tendency bias may lead a disproportionate number of responses taking the middle option, meaning it won't reflect the true underlying distribution of our population.

Answer 65

If we want a larger spread than a typical five point Likert provides us, or if we want to quantify the responses as a continuous variable, we can use numeric scales. Numeric scales help us extract a bit more granularity out of our sample if we believe there is true underlying variance in opinions. So for example, on a scale from 0 to 10, how much do you agree with the following statement?

Answer 66

Only two options given - agree or disagree

Answer 67

Similar to a Likert scale For example, how often do you visit your child's school? One set of responses could be similar to a Likert--never, rarely, sometimes, often, always. The responses represent some implied quantity. But the magnitude of those quantities is subjective. Alternatively, we can use frequency scales that are closer to reflecting true numbers. For example, daily, weekly, yearly. Again, this doesn't give us a precise answer and makes analysis slightly more difficult if we hope to estimate a linear relationship between this variable and some other. But it may make it easier for our respondent to translate a vague estimate into a response.

Answer 68

This is particularly useful when respondents do not have an absolute sense of quantity, but they do have a relative sense. They can sort the response options in some way or the other. Which teaching-learning materials do you use most often? (Rank the top 3) Options: Workbooks, flipcharts, textbooks, maps, flash cards, games etc.

Answer 69

a version of an open response question when the response options are not given to the respondent. So the respondent answers in his or her own words. Then the surveyor codes these responses using pre-determined response categories. This is useful when we don't want to prompt the respondent with possible options, because perhaps out of convenience, they may just select one, several, or all of the responses given to them without even thinking. It is especially true if the response options might signal to the respondent which answers the researcher considers quote-unquote acceptable. However, it requires a bit more faith in our surveyors, or at least a bit more skill.

Answer 70

Vague concepts where respondents may interpret the question in a different way Make sure to define vague concepts

Answer 71

The response categories do not include all categories that can be expected as a response Pilot question to make sure that categories are exhaustive

Answer 72

Questions that include negatives can be confusing to the respondent and lead to misinterpretations. Avoid unnecessary negatives

Answer 73

The categories overlap each other. Make sure that all categories are mutually exclusive

Answer 74

The question assumes certain things about the respondent Use filters and skip patterns

Answer 75

People react to a particular choice in different ways depending on how it is presented i.e. prefer gains over losses Try to be neutral when framing questions

Answer 76

People may retrieve recollections regarding events or experiences differently You can ask respondents to keep a diary or save their receipts

Answer 77

People tend to rely too heavily on the first piece of information seen Avoid adding anchors to your questions

Answer 78

People perceive recent events as being more remote than they are (backward telescoping) and distant events as being more recent than they are (forward telescoping) Visit once at the beginning of the reference period. Then ask, “since the last time I visited you, have you…?”

Answer 79

Tendency of respondents to answer questions in a manner that is favorable to others i.e. emphasize strengths, hide flaws, or avoid stigma Ask indirectly, ensure privacy

Answer 80

* Act of being surveyed changes subsequent behavior * Particularly relevant for panel surveys where there are multiple interactions with respondents More data are better for analysis, But measurement effects could change the interpretation of subject behavior. Consider which questions could be asked at endline only, or obtained through non-survey methods

Answer 81

Imagine we have a measurement instrument that systematically overestimates the value of our outcome, but it does so the same amount in both the treatment and control groups If we look at either the difference in our outcome between baseline and endline, or between the treatment and control groups, we'll estimate the true difference and the true impact without bias. This is an example of where the magnitude of bias is totally uncorrelated with the treatment assignment.

Answer 82

One group (treatment or control) is over-reporting positive or negative results, leading to an inaccurate assessment of whether or not there was an impact (i.e. exaggerating positive behavior in treatment group would make it seem like there was a large impact, when, in fact it's wrong & biased)

Answer 83

For all treatment groups we should collect data with: – Same enumerators - Blinding to the treatment assignment – Same time period – Same methods – Same incentives We want to make sure that the differences that we measure between the two groups is due to the treatment and the treatment alone, not surveyor characteristics or biases.

Answer 84

It depends. If the bias differs systematically between the treatment and control groups, it will introduce error. If the bias is identical for both groups, then it may not introduce error.

Answer 85

Validity. Oppenents take issue with whether standardized exams are a valid measure of learning levels since one can do well on a test without having mastered concepts, or vice versa.