Test 3: Regression Equations, Types of Measurement, Survey Design and Sampling. Flashcards

Question

In Psychology:  The most common type of variable, scale of measurement, used is ___; why? >anova t-tests

Answer 1

i.e. interval data. Why? o Because many inferential statistics such as ANOVA and T-Tests have an assumption of equal spacing between points on a scale and normal distribution. o Ordinal and Nominal data cannot meet these assumptions!! o Non-Parametric tests must be used if nominal or ordinal scales are used and these assumptions cannot be meet. e.g. Wilcoxon signed rank test, Friedman test, Kruskal-Wallis test etc. o You can only use Nominal variables in a parametric test (test for interval and ratio data) if it is being used an IV/predictor variable.

Answer 2

How people report back to us about their thoughts, feelings, and beliefs. Advantages: 1. Who knows better than the individual in question? Self-reports are a useful tool because there are some topics only the individual can report on - i.e. depression, anxiety, mindfulness where an observer such as their parents or peers are not able to observe or accurately report on another person’s internal state. 2. Easier and more efficient? Potentially more accurate method than obtaining data through observation, reports from others or physiological measures which are highly variable. Problems: 1. Awareness/Memory: If people are unable to recall something, then they are not able to report it, or worse people will just make something up (places limitations on the reliability of the data obtained) 2. Response set/Bias: If one doesn’t want to admit to something or report it people tend to self-censor and provide an answer that makes them socially desirable. 3. Format the question: The way a question is formatted can distort the answers we get back. For example, with a multiple-choice format if the answer the participant thinks of is not there, then it is easy to pick another option even though it does not accurately reflect your beliefs. 4. Questions tailored for samples: The questions need to “fit” the sample, this is so questions are socio-culturally appropriate i.e. do not cause offence or distort the answers we get back. For example, technical terms are specific to individuals with the correct domain-specific knowledge. For example, Jargon goes out of fashion quickly and should be avoided to ensure that participants understand the question.

Answer 3

(A) Yes/No questions This format limits the range of responses significantly and is not sensitive to controversial topics in which respondents would like to give a more personal response. - Simplicity is best for working with children samples and is not appropriate for adults. - Can be extended to yes/no/maybe questions. - Close-ended format (B) Fill-in-the-blank e.g. “when a child lies a parent should ___” o Advantage of a fill in the blank format for survey items is that answers are not constrained. However, this means responses will require coding and take more time, money and resources and only produces categorical data. o Categorical data severely limits the inferential statistics that can be conducted (i.e. not t-tests and ANOVA). o Open-ended format o Participants may produce answers that you did not expect or predict. (C) Multiple choice e.g. “when I’m depressed, I:”  Cry a lot  Stay in bed  Put on a happy face o Can either ask people to select one or tick all that apply. o Is an okay measure but is constrained by overlap between answers i.e. you may cry a lot in bed? o It’s an okay measure but it is insensitive to people’s responses. (D) Likert scale o Most common format for items because it is more sensitive than other styles. o Usually has 5-7 alternatives within to extremes (strongly agree and strongly disagree). o Uses interval data so we need equal distances between points, sometimes this is hard to do.

Answer 4

Labels: o The two extreme points HAVE to be labelled. o If you have a middle point it is GOOD to label it. o You can CHOOSE to label all other points on the scale or leave then blank. Middle point: o It is good to include a middle point for instances where participants truly are neutral on a topic or don’t understand what the question is asking (i.e. do not have the knowledge and are ignorant to the topic). o Not having a middle point forces participant to choose a side and this can upset participants who wish to be neutral on a topic. o To do this you would use an even numbered scale. Using an even numbered Likert scale will affect: o Reliability rather than validity because it would increase the number of errors in responses if the neutral point is omitted. o If the scale is not reliable in producing consistent data or internal reliability then this may impact the validity of the scale-whether it measures what it is supposed to measure-validity is tested over multiple studies.

Answer 5

1. Complexity: o Keep questions short and simple so participants are able to understand the question. o Grammatically complex items with multiple nouns or verbs distract the reader from point of the question. 2. Technical Terms: o Generally, avoid using abbreviations and if you have to use them you need to clearly define what they mean. e.g. DD is drink driving. o Word choice needs to match the socio-cultural context the survey is conducted with. For instance, a survey designed for a USA sample will not be suited for a NZ sample. 3. Ambiguity: o Don’t write a statement that could be interpreted in two or more ways. e.g. “we should treat politicians as they deserve” o This question is ambiguous and does not clearly state what the point of the question is. How does someone respond to a question like this? o Really hinders the reliability of the scale and the data obtained. e.g. visiting family can be fun 4. Double-Barreled Questions: o Questions with two clauses e.g. “we should spend more money on defense and education” o It is hard for someone to respond to the question if they agree with one and not both of these clauses. o Typically, people will respond in a neutral position if the question is unclear. e.g. Is it wrong for female patrons in bars to swear and accept drinks from unknown men? 5. Negatives: o Negative phrasing can confuse the reader. Do not include double negative’s it requires more cognitive effort and time to understand the meaning of the question. o There should be a mix or positively worded and negatively worded items. e.g. “one should not support people who do not look for work”. 6. Response Acquiescence: o Avoid a having too many items that are framed the same way (i.e. positive or negatively). o This can lead to response acquiescence where people respond on autopilot without carefully reading each question. o Solve this by framing 50/50 positive or negatively worded items and mix them throughout the survey to avoid response acquiescence. 7. Emotive Language (loaded question): o Avoid using inflammatory language e.g. “the government should cut off support for people who are too lazy to work” o This framing is leading towards a particular direction of the scale and can distort the responses given. o For example, strong language leads to people using extreme ends of the scale and impairs the reliability of the findings. 8. Leading Questions o Don’t make questions that are hard to disagree with. o “we should improve the lives of New Zealanders” 9. Invasion of Privacy o Be aware that questions about illegal activities (drug use or crime) or sensitive topics (sexuality) are known to produce controversial and strong emotional reactions. o Sometimes including questions like this are hard to avoid depending on the topic you are researching (I.e. drug use or sexual behavior) but they should be used with sensitivity. o Only use when appropriate and be extra cautious to come across as judgmental i.e. “do you use dangerous drugs”. o You should always warn participants at the beginning of the study if sensitive topics will be covered and an example of the types of questions that will be asked. This way they can decide prior to commencing the study if they are comfortable disclosing that kind of information. 10. Sensitive Topics with Young People: o Always need parental consent for any children who are under 16 years of age. o Does surveying children about sensitive topics like suicide, graffiti, drugs or sexual behavior end up encouraging said behavior. o Research found that completing a suicidal tendencies survey did not reinforce this behavior in children but actually made it less likely to occur. o With drug use: people are not likely to admit to illegal behaviors in fear of negative repercussions. Thus, data is not likely obtain an accurate estimation of these behaviors’ frequency in the population- could use anonymous surveys it’s a new method that increases honesty on controversial topics.

Answer 6

A) Visual Analogue Scales e.g. like a temperature gauge. B) Beck Depression Inventory: Multiple Choice o Mixture of multiple choice and Likert scale survey o Where each point on the Likert scale are ranked ordinally from I don’t agree to I do agree. o (4) point Likert scale is presented like it was a multiple-choice survey. C) Smiley-Frowny faces: - Again, great for hospitals to indicate level of pain or severity of symptoms. - Great for children or individuals with autism. - More personal scale compared to the graded box scale. - Liking or disliking of pain and pleasure D) Graded Boxes: - Graded boxed are easy for children to visually see which point is larger than the other and choose the appropriate one. E) Use of Don’t Know: o Inclusion of an IDK option on a Likert scale… it’s placed at the end of the scale to indicate that it is an alternative option to the Likert scale. o To account for memory errors or cases where the participant truly does not know what the question is referring to does not have the knowledge required to answer the question properly.

Answer 7

Why Digital Administration?  Completing an online survey is much easier and faster than filling out a pen and paper survey.  Easier to compile the data into excel or SPSS formats without human error or paying a third party to help compile data.  Easier to create and edit your self-report survey.  Almost everyone has a screen to complete a survey digitally. However, there are issues with screen size and the mechanics of completing the survey (i.e. readability).  Can “skip and branches” more easily (i.e. subsections of the survey on a topic which if you answer to doing you can skip that section of the survey). e.g. Demographics: A) European NZ. Tailored questions about immigration of ancestors. B) Maori tailored questions about iwi, hapu etc. Disadvantages with Digital Surveys:  You cannot skim forward or backwards through the survey.  Can’t determine how far through the survey you are (participant fatigue)  The fonts can be too small and hard to read.  Participants is tied to their screen (typically their desk computer or laptop).  Computers can die or internet connection can be lost (data can be lost completely or missing parts).

Answer 8

Sampling A fundamental aspect of the research process which is pivotal to the reliability and validity of a study (for both prospect and retrospective study). Note: Even if you did not collect the data yourself, you should always evaluate how it was conducted to determine if your findings can be generalized to the population.  External Validity: to what degree can we generalize the findings of a sample to a larger group (the goal for quantitative work but not so important for qualitative work).  The sample needs to be REPRESENTATIVE of the population!  The attributes of the sample need to be identical to the characteristics of the population.

Answer 9

A) Survey design | B) Sampling methods

Answer 10

then It calls into question the reliability of our sampling methods (possible self-selection bias-overrepresentation or underrepresentation of attributions i.e. sample is not representative of the population).

Answer 11

1. Simple Random Sampling o Every member of the specified population is placed onto a list and then members are randomly selected from the list to be a part of the sample. o This means that everyone has an equal chance of being drawn from the population into the sample. o However, for larger populations obtaining a list can be very difficult and sometimes impossible. o Method is very rarely used in psychological research due to these constraints. 2. Stratified Random Sampling o We divide the population by attributes or dimensions (gender, Socio-economic status, ethnicity, age etc.). o We randomly sample from these groups with the goal of keeping the attributions from the population proportional identical to the sample. o i.e. population 60% female then sample will have 60:40 (female:male) out of 100. o The attribute used can be any demographic information. However, the researcher can decide which attribute is most relevant to their study and needs to be kept proportional. 3. Cluster Sampling o Frequently used by national polling organizations. o Obtain participants from pre-existing clusters (groups) of people, that have one specific characteristic in common. o E.g. schools or required/core curriculum classes. o You would randomly select a few clusters form a large group of them in order to obtain variety and a more representative sample. ``` School=cluster o Public vs private o Rural vs city o Size o Age range o Maori integrated etc. ``` - If clusters are too big you can use stratifying techniques to make them smaller. e. g. 16 apartments in total, 6 clusters, 40 per floor, 3 randomly selected floors from each cluster = 720 sample (rather than 9000 previously).

Answer 12

Is a cheaper and easier method of sampling but raises the concerns about the obtained sample being representative of the population. 1. Convenience Sampling: o When you sample from a readily available and easily obtainable source. Although it is convenient for the researcher it is flawed by sampling biases. E.g. sampling off of the street from people who passerby o Poor external validity o Too biased (only people who are businesspeople, on lunch, shopping or students). o Underrepresents large subgroups of people 2. Quota Sampling: o The non-probability version of stratified sampling. o Where the goal is to obtain the appropriate percentages of attributes (or types of participants who vary in gender or age etc.). o The key difference is that for non-probability sampling people are selected from a readily available source (i.e. people from the population do not have equal chances of being selected to be a part of the sample). E.g. IPRP sample o Since psychology students have 80:20 female to male ratio (which is not representative of the population). o Researchers can put constraints on the sampling process by limiting the number of female students who can sign up (i.e. close sign ups for women once their quota is reached). 3. Purposive Sampling: o You select individuals who fit within a particular category to fit a purpose. o e.g. selecting people who have had an episode of depression by recruiting them from mental health clinic. o This can lead to biases such as… o Severity of depression i.e. limits it to people who have been severe enough to seek professional help. o Excludes ongoing patients who still experience depression that are on prescription and do not frequent the clinic. o You can reduce the level of biases by using multiple methods approach to sampling (sample from schools, clinics, neighborhoods, online etc.). 4. Snowball Sampling o We recruit from an initial group of participants that are easy to obtain i.e. friends, family, peers. o Then we ask participants to refer us to people within their social network that they think would be willing to participate in their study. o Method suited for studies that wish to obtain data from hard to reach or rare populations (i.e. prostitutes or surfers). *Snowball and Quota sampling have a degree of bias in their sampling procedures which limits the conclusions one can make to the general population!

Answer 13

• Most psychologists use non-probability sampling because it is easier and cost effective relative to probability sampling. o There is an assumption within psychology that if a relationship between variables or psychological phenomenon is significant enough that it will be generalizable to the population regardless or our sampling reliability. o The validity of this claim depends on the situation. e.g. If people were studying how adolescence cope with stress but we only sample from children in wellington then we have likely missed: rural and non-Pacifica adolescents. • Paul’s stance on this is that psychologists should always be concerned with obtaining representative data because this can be determined by subtle differences in researchers sampling methods. e.g. Cognitive psychologists tend to use university students as their samples due to convivence. However, university students are at a peak age for cognitive ability and have expertise that majority of the population do not have.

Answer 14

1. Non-representativeness of the sample: When the sampling frame significantly differs from the population. e.g. research question is about time perspective, but your sample frame is all available commuters in wellington- people who use public transport will have a different perspective on time then the population. 2. Response Rate: o When response rate is low. o For example, when researchers mail out their survey and less than 60% of people return their survey (10-20% is very poor). o This generates a self-selection bias where people who agree to complete the survey and those who don’t significantly differ and a likely important characteristic. e.g. when people who are stressed or depressed do not fill it out and thus, or sample has an overrepresentation of healthy participants in the sample. 3. Ethical Permission: o Common problem when working with adolescence where you need to obtain to sets of consent (one from parents and one from the child). o There are many reasons why people may refuse: religion, concerned about child’s mental health, did not see the form etc. o This systematically removes data from the sample that can significantly reduce the representativeness of the sample to the population.

Answer 15

1. Passive ethical consent: o When gaining consent for adolescence to participate in your study sometimes researchers can obtain permission from the school to study their students. o Then they send home a letter addressed to the parents to advise them about the purpose of the study and advise them that the students will be asked to participate on the day if they do not get in contact with the researcher remove their child from the study. o Controversial Method. ``` 2. Compensation and incentives: o Using money, candy, time out of class or CD’s etc. to increase participation but researchers should be aware that this may also introduce biases into the sample (giving us crappy data just to obtain the incentive-researchers need to make sure the incentive is appropriate for the sample i.e. not too big). ``` 3. New ways to collect data: o Laptops, IPads, internet, testing on cell phones, zoom or diary studies etc. 4. The “personal touch”: o Some people prefer face to face interviews with the researcher. o This is time consuming and expensive method that is not often used nowadays. 5. Under-utilized samples: o Traveling around NZ to unrepresented samples i.e. Nelson, kids in camp or kids at malls etc.) o Introduces travel costs.

Answer 16

* ESM= Event Sampling Method * The experience sampling method, also referred to as the dairy study method, is an intensive longitudinal research methodology that involves asking participants to report their thoughts, feelings, behaviors and/or environment on multiple occasions over time. * Short-term longitudinal study where people report their feelings hourly, daily or weekly basis. Advantages: -Good for rapidly changing variables (unstable) such as mood in relation to bullying. - Capturing data on psychological phenomena closer to the time that it occurs leads to more accurate data (not flawed with memory biases). - Obtaining multiple assessments of variables of interest to the researcher, and a larger quantity of them increases the studies reliability and validity. - Can identify contexts for important psychological states e.g. what contextual factors may trigger depressive thoughts or cravings. Disadvantages: Due to the intensity of this data collection method: o It is hard to recruit participants and keep them engaged in the study (people do not enjoy being interrupted multiple times a day to answer a survey). o Participants must be familiar with technology in order to fill out the survey (i.e. may exclude older people from being a part of the study). o Lots of missing data (people sometimes ignore the notification to fill out the survey if they are busy). o The data you obtain is rich (i.e. within subjects repeated measures) and hard to analyze. Data Analysis: o This rich data requires a specific kind of data analysis Multi-level Modeling. o This is a method of analysis which is taught at honors level because of its difficulty. o Is the best method to analysis ESM data.

Answer 17

Obtaining: r= -.26, p < .05 is meaningless.  Because the data is nominal the correlation reflects how the data was arbitrarily coded NOT an underlying pattern in the data. If we were to recode our data, we would find the same correlational value for the new category.

Answer 18

 Then finding a correlation is meaningful.  E.g. r (218) = .16, p = .018. what does this mean? • Means there is a positive correlation towards females having higher levels of anxiety than males. • How? Females = 1, Males = 0 Females have been arbitrarily coded to be higher ranked as 1 (relative to 0). • Thus, a positive correlation = directed towards females. • A negative correlation = directed towards males. Interesting crossover: The exact same statistical finding can be found using a t-test (it is the same mathematically and statistically):  Has to be two dichotomous category variables and a continuous variable.  The same statistically and mathematically analyzing it with a t-test or a correlation.  Key difference is the type of conclusion you can make- o Correlation is suited for association between variables, especially over time. o A t-test is best for making comments about differences in means. Note: A key assumption for a t-test or ANOVA is equal variance (something nominal data cannot meet).

Answer 19

When analyzing statistical data in science there are two main camps of inferential statistics: (A) Mean Group Differences: e.g. ANOVA or T-Tests are common in cognitive psychology. (B) Associations: e.g. correlations, regressions, factor analysis. - Two ways of looking at the same thing (both under the GLM: General Linear Model). - However, the choice between which method is used is tied to the type of variable in question: * Categorical variables are used as IV’s in an ANOVA * Continuous variables can be used in either t-tests or ANOVA.

Answer 20

For the movie rating example: With ordinal (ranked) data: - The numerical value indicates the movies relative standing (higher or lower). - Smaller numbers indicate a higher ranking and lower numbers indicate a lower ranking. - There can be no double ups i.e. one movie for each ordinal rank. - distance between each position is equal making it an insensitive measure. - is used sometimes because it's easier than creating and implementing a likert scale. - limited analysis only correlation doable is a phi correlation. With Interval (ratings) Data: - In the form of ratings from a Likert scale meaning... - there can be more movies in the 1st position - higher ratings indicate a higher relative standing - a sensitive measure because the distance between movie's rating are not equally distanced- it can vary. - flexible measure can do many types of inferential statistics because it meets the assumptions of equal spaces between points on a scale and normal distribution.

Answer 21

4. Ratio Data: - Used on the rare occasions where you want 0 to be meaningful (i.e. indicate absence of attribute like errors). - Is treated much like interval data, so conceptually it is just interval data with a meaningful zero.

Answer 22

(A) Categorical Data: Can: o Can generate frequencies o Dichotomous category variable can be used as IV’s in analysis of ANOVA or MANOVA o Can do a Chi-Square Test Wilcoxon signed rank test, Friedman test, Kruskal-Wallis test. Can Not: o Generate means because assumption of normal distribution is not met o Cannot use category data as an DV (B) Ordinal Data: - Is a mixture of Interval and Categorical data. - Usually, we just report the rank ordering that is obtained. - There are non-parametric tests which can be used for ordinal data e.g. phi correlation - No assumption of normal distribution. You need to venture off the beaten path into the world of non- parametric analysis. Wilcoxon signed rank test, Friedman test, Kruskal-Wallis test. (C) Interval (continuous) data? - The most flexible scale of measurement for quantitative analysis - Most common variable in psychology. - Can calculate a mean and standard deviation. Thus, have the required information to conduct an ANOVA, t-test and many other analyses. (D) Ratio Data: - Used on the rare occasions you want a true point of zero - Treated much like interval data.

Test 3: Regression Equations, Types of Measurement, Survey Design and Sampling. Flashcards

(46 cards)