Project Prep Benchtest Flashcards

Question

Internal

Answer 1

- Do you get same results if you use different tests to measure the same thing - Strong correlation supports reliability

Answer 2

Does the test relate to high level theories

Answer 3

Does test appear to test what it aims to test

Answer 4

- Does the test relate to an existing similar validated test - Work is built on findings of another test and matches their work

Answer 5

Does the test predict performance in a later developed test

Answer 6

Concerns the responsibility of researchers to be honest and respectful to all individuals who are affected by their research studies or their reports of the studies’ results

Answer 7

Conducting research in a way that allows others to have trust and confidence in the methods used and findings that result from this

Answer 8

Conscious or unconscious influencing of the study and its results

Answer 9

- Recall bias - Selection bias - Observation bias - Confirmation bias - Publishing bias

Answer 10

- Survey respondents asked to recall events - different types of events more likely to be remembered than others

Answer 11

Samples can sometimes under-represent certain people and over represent others

Answer 12

- Hawthorne Effect - When participants are aware that they’re being observed they, either consciously or unconsciously, alter the way they act or the answers they give

Answer 13

- Occurs during interpretation of study data - Researchers consciously or unconsciously look for information or patterns that confirm the ideas or opinions that they already hold

Answer 14

- Studies with negative findings (nothing found) are less likely to be submitted by scientists or published by journals - Perceived as less interesting

Answer 15

- Bias in per-course survey (unbalanced data) - automatic profiling - Bias in learning about user instead of type of user (stereotyping) - different users in training and test sets - Bias in future data predicting past - train on past, test on future - Bias in unbalanced data sample - stratified sampling

Answer 16

- A survey of scholarly sources on a specific topic(s) - Provides an overview of current knowledge allowing you to identify relevant theories methods and gaps in existing research

Answer 17

- Summarises current state of understanding on a topic - Surveys and summarises previously published studies - rather than report on new facts or analysis - Gives roadmap on future research - Can be used to back up the validity of your question

Answer 18

Any method focused on asking Participants for responses

Answer 19

- Gather information not available from other sources - Ubiased representation of population interest - Collect information from many individuals to understand them as a whole - Allows massive information gathering

Answer 20

Mainly quantitative but qualitative methods can be used too

Answer 21

- Can get info from large samples - Can have different types and numbers of variables - Gets info that's hard to observe - Easy and cheap - Standardised stimulus - no observer subjectivity

Answer 22

- Intentional misreporting to hide inappropriate behaviour - Poor recall - Response rates are critical - Can introduce bias from wording of questions - Inflexible - can't be changed during data gathering - Not ideal for controversial issues

Answer 23

- Exploratory - form general ideas about research questions - Descriptive - collect more specific descriptions of the variables of interest - Explanatory - develop understanding of relationships among variables of interest

Answer 24

- Need to validate bias in question design - Ask positive and negative questions - should be given opposite answers - Validity of survey comes from the representativeness of the sample and the precision of the questions - Face validity - Do questions appear reasonable and acquire data you want - Content validity - Are questions all about issue and other subjects related to it - Internal validity - Do questions imply the outcome you want to achieve - External validity - Do questions elicit answers that are generalizable

Answer 25

- Correlational questions - Less technical questions - usability - Exploratory questions

Answer 26

- Random sampling- each member has equal chance of being picked - Stratified sampling- use subsets of the population to sample - lower sampling error - Systematic sampling- every Nth name is selected - Quota sampling- researcher chooses necessary number of participants per stratum - Purposive sampling- researcher selects participants according to criteria

Answer 27

To understand how people naturally interact with products and people and the challenges they face

Answer 28

- Can get more subtle data - Allows richly detailed description - Viewing or participating in unscheduled events - Improves quality of data collection - Can see things you weren't expecting - Useful for formulating hypothesis - Doesn’t depend on information provided by respondents - Can deal infants/animals

Answer 29

- Less structured responses - Get huge amount of data - analysing and not including bias is hard - Difficult to replicate - lots of variables you don't have control of - Different researchers gain different understanding of what they observe - Male/female researchers have access to different information - Many events are uncertain in nature - difficult for researcher to determine time and place - Can't generalise - Long and expensive

Answer 30

- Exploratory - Explanatory

Answer 31

Typically qualitative but can be quantitative

Answer 32

- Complete observer - Observer as Participant - Participant as Observer - Complete Participant

Answer 33

- Detached observer - Researcher is neither seen nor noticed by participants - Minimises Hawthorne effect - participants more likely to act natural - Most likely to raise ethical questions

Answer 34

- Researcher is known and recognised by participants - Participants know research goals of the observer - Some interaction with participants but limited - Researchers aim is to play a neutral role

Answer 35

- Researcher is fully engaged with the participants - More of a friend or colleague than neutral third party - Full interaction with participants but they still know its a researcher

Answer 36

- Fully embedded researcher - Observer fully engages with the participants and partakes in their activities - Participants aren’t aware that observation and research is being conducted

Answer 37

Use multiple independent researchers to observe

Answer 38

- Quantitative technique - Explicitly counting the frequency and/or intensity of specific behaviours - Most direct observation data collection done by actual observers - Don’t require human data collector - audio/video can be used - Ordinal data/ purely factual description - Structured form of data collection

Answer 39

- Process enabling researchers to learn about the activities of the people under study in the natural setting through observing and participating in those activities - Qualitative, interactive and unstructured - Information collected is unique to the individual collecting the data

Answer 40

Explore the views, experiences, beliefs and/or motivations of individuals on specific matters

Answer 41

- Group of respondents are interviewed together - Obtain data from purposely selected group of individuals rather than representative sample

Answer 42

- Can get qualitative data - Preferable when researcher wants subjective perspective rather than generalisable understandings

Answer 43

- Time consuming - Not the best for researching sensitive topics

Answer 44

- Better at drawing people out of their shells - increased validity - Allows for discovery - Can build on each others comments for richer contextual data

Answer 45

- Time consuming - Anonymity is hard - Less reliable - Participants can be influenced by other group members - conformity, social desirability, oppositional behaviours - Need skilled interviewer to prevent these problems

Answer 46

- Exploratory questions - Theory testing/creation questions - Confirmatory research questions

Answer 47

- Almost always qualitative

Answer 48

Structured: - Quantitative method - Closed-ended questions - List of questions - Everyone asked same questions in the same order - Easy to replicate - Easy to test for reliability - Quick to conduct - Not flexible Unstructured: - Do not use any set questions - Guided discussion - Most useful for qualitative research - Rarely provide valid basis for generalisation - More flexible - Increased validity - can probe for deeper understanding - Time consuming to conduct and analyse the data - Employing and training interviewers is expensive Semi-Structured: - Set questions but can investigate answers more - Gets qualitative and quantitative data - Can explore around answers - Gathers useful info but respondents can answer more on their own terms - More flexible - More time-consuming

Answer 49

- Dual moderator - Two moderators - Two-way - Two seperate groups having discussions at the same time - second group listens to the firs tbefore having teh discussion - Mini- 4-5 participants instead of 6-10 - Client-involvement - clients ask for focus group and invite those who ask - Participant-moderated- one or more participants are moderators - Online

Answer 50

Allows researchers to look at cause-and-effect relationship Used when: - There is time priority in a causal relationship - There is consistency in a causal relationship - The magnitude of the correlation is great

Answer 51

- Allows for reproducibility - Generalisation is easier - Can take bias into account using statistics

Answer 52

- Equipment might be more expensive - Highly prone to human error - Errors can reduce validity - Eliminating real-life variables can result in inaccurate conclusions - Time-consuming process - Researchers can control variables to suit personal preferences - Results are not descriptive

Answer 53

Correlational questions

Answer 54

Quantitative

Answer 55

Researcher manipulates one variable and controls the rest of the variables

Answer 56

Hypothesis invented after testing is done to try and explain contrary evidence

Answer 57

variable manipulated

Answer 58

variable measured

Answer 59

not changed

Answer 60

- Take data from previous research and examine it for new question - Look for datasets that other people have created

Answer 61

- Discover new things from old data - Can use data that you wouldn’t have the resources to gather - Access to historical data - Ease of Access - Inexpensive - Time-saving

Answer 62

- May be issues with the data e.g bias - Might twist yourself to fit the data you’ve got - If you don’t know how the data is collected - don’t know the validity - Because data is hugely heterogeneous in many cases - have to make decisions to remove, ignore or add sections - can lead to confirmation bias - Many critical decisions in processing the data - Irrelevant Data - have to find the relevant data from the irrelevant data

Answer 63

- Often explorational - Every question can be asked

Answer 64

- Because data is hugely heterogeneous in many cases - have to make decisions to remove, ignore or add sections - can lead to confirmation bias - Need to know a lot about the data to prove that any changes in adding or ignoring have valid assumptions and rationale

Answer 65

To validate secondary data, find the: - Purpose for which the material was collected/created - Specific methods used to collect it - Population studied and validity of the sample - Ccredibility of the collector - Limits - Historic and/or political circumstances - And consider how the data is coded/categorised - Consider whether data must be adapted/adjusted

Answer 66

- Correlational - Causation - The how questions

Answer 67

- The why questions

Answer 68

- Mix of qualitative and quantitative data - Usually use different methods to collect them - When you have a small sample size - want to do quantitative but don't have enough people - Qualitative used to underpin quantitative - For exploration

Answer 69

- Expressed in numbers and graphs - Used to test or confirm theories and assumptions - Can be used to establish generalisable facts about a topic - Methods include experiments, observations recorded as numbers and surveys with closed-ended questions - At risk for research biases icl. Information bias, omitted variable bias, sampling bias or selection bias

Answer 70

- Expressed in words - Used to understand concepts through experiences - Gather in-depth insights on topics - Methods include interviews with open-ended questions, observations described in words, focus groups, Ethnographies and literature reviews - At risk of research biases incl. Hawthorne effect, observer bias, recall bias and social desirability bias

Answer 71

- Don’t draw samples from large-scale data sets due to time and costs involved - Problem of adequate validity or reliability is major concern due to subjective nature - Contexts, situations, events, conditions and interactions cannot be replicated - Generalisations can't be made to a wider context than the one studied - Lengthy time required - Expert knowledge of an area is required to interpret the data

Answer 72

- Researcher gains an insider’s view of the field - can find issues that are often missed - Can be important in suggesting possible relationships, causes, effects and dynamic processes - Allows for ambiguities/contradictions in the data which reflect social reality - Uses a descriptive, narrative style

Answer 73

- Do not take place in natural setting - Do not allow participants to explain their choices - Poor knowledge of the application of the statistical analysis may negatively affect analysis and subsequent interpretation - Large sample sizes needed for more accurate analysis - Confirmation bias - researcher might miss observing phenomena because of focus on theory or hypothesis testing rather than on theory/hypothesis generation

Answer 74

- Scientific objectivity - data can be interpreted with statistical analysis - Useful for testing and validating already constructed theories - Data analysis and collection can be performed quickly - Data can be checked by others and replicated - Hypotheses can be tested

Answer 75

Collect data to determine if a claim about the population is true

Answer 76

-Testable statement that you want to accept or reject - You never "prove" a hypothesis

Answer 77

- Needs to be testable - Need to be able to prove it false - Be specific - don’t use ambiguous words e.g “athlete” or “better” - Don’t be too specific - overlap with methodology "If (one variable) 'is related to'/'is affected by'/'causes' (other variables) then (comment on relationship)"

Answer 78

- Two tailed test - doesn't state direction - One-tailed test - states direction

Answer 79

Null Hypothesis is true but is rejected - false positive

Answer 80

Null hypothesis is false but is not rejected - false negative

Answer 81

- Compare p-value to a threshold value (significance level/alpha) to reject null hypothesis - P > alpha - fail to reject - P <=alpha - reject

Answer 82

- Some tests return a list of critical values and their associated significance levels and a test statistic - Test statistic < critical value - fail to reject - Test statistic >= critical value - reject

Answer 83

- Observational data - Experimental data - Simulation data - Dervived/Compiled data

Answer 84

Open surveys, observational studies, focus groups etc. ...

Answer 85

Collected via experimentation - easier to reproduce

Answer 86

Scenario simulation allows for generation of predictive data

Answer 87

Utilises existing data to generate new data - secondary data analysis

Answer 88

- Basic analysis of the data giving a general overview - Only describes what the data is or what it shows - Allows for simple analyses - No extrapolation of inference - Measures of frequency - Measures of central tendency - Measures of dispersion or variation - Measures of position

Answer 89

Count, percent, frequency

Answer 90

- Mean, median, mode - Used to show an average or most commonly indicated response

Answer 91

- Range, variance, standard deviation - Variance/standard deviation - difference between observed score and mean - When you want to show how spread out the data is

Answer 92

- Percentile ranks, Quartile ranks - Describes how scores fall in relation to one another - Relies on standardised scores - Use when you need to compare scores to a normalised score

Answer 93

- Examine or explore data and find relationships between variables which were previously unknown - Does not describe the cause - Useful for discovering new connections

Answer 94

- Use statistics to look beyond the collected data to identify new conclusions - Using a small sample of data to infer about a larger population - Based on laws of probability and confidence intervals - Central Limit Theorem - T-test

Answer 95

- Distribution sample means approximates a normal distribution and the sample size gets larger, regardless of populations distribution - Average of sample means and standard deviations will equal the population mean and standard deviation

Answer 96

- Tells how likely the difference between two groups is a real difference rather than sampling artefact - ‘P-value’ - probability that the data collected occurs by random chance

Answer 97

- Using historical or current data to find patterns to make predictions about the future - Simulations can both generate data for prediction as well as using existing data - Accuracy of predictions depends on input variables/data - Accuracy depends on types of models - linear model generally works well - Using variable to predict another doesn’t denote a causal relationships

Answer 98

- Step beyond inferential analysis - Examines the cause and effect relationships between variables focused on finding the cause of a correlation - Generally large, complex and expensive studies - Four important components 1. Correlation 2. Temporal sequence - cause must occur before effect 3. Concomitant variation - variation must be systematic between the two variables 4. Nonspurious association - Any covariation between a cause and an effect must be true and not due to another variable

Answer 99

- Similar to predictive but instead of general data driven predictions - utilise highly specific changes in variables that lead to changes in linked variables - Generally used in high precision disciplines e.g engineering and physics - Often used in high precision computer models

Answer 100

1. Validity - degree to which data conforms to defined business rules or constraints 2. Accuracy - ensure data is close to true values - E.g put in positive and negative questions in questionnaire - person should answer 1 to the negative if they answered 5 to the positive 3. Completeness - degree to which all required data is known 4. Consistency - ensure data is consistent within the same dataset/ across multiple datasets 5. Uniformity - degree to which data is specified using the same unit of measure

Answer 101

- Nominal (categories, no ordering) e.g male, female - Ordinal (categories, ordered) e.g small, medium, large

Answer 102

- Discrete (countable, integers) - Continuous (measurable) e.g Age, temperature - can subdivide it

Answer 103

Two variables in the individuals of a population that are linked together in order to determine the correlation

Answer 104

- Nominal variable - McNemar's Test - Ordinal (Ordered categories) - Wilcoxon - Quantitative (Discrete or Non-Normal) - Wilcoxon - Quantitative (Normal) - Paired t test

Answer 105

- Make assumptions about the parameters of the population distribution from which the sample is drawn - Often that the population data are normally distributed - Can only apply parametric tests (e.g T-test) if you have a sample big enough (in regards to population) to assume that the central limit theorem applies

Answer 106

- “distribution-free” - Can be used for non-Normal variables

Answer 107

- Reducing the chances of a type I error increases the chances of a type II error and vice versa - In science it is better to miss something than draw incorrect conclusions - reduce type I errors - Bonferroni correction - Reduces instances of type I errors but increases type II errors - Types II error reduction not as easy as Bonferroni: - Increase sample size - Change alternative value in the alternate hypothesis

Answer 108

- test looking at 3 or more groups - reduces type I errors - Used for comparing the means of three or more groups or variables

Answer 109

- In uncertain scenario - allows for exploration of the problem/solution space - One of the most popular techniques for calculating effect of unpredictable variables on a specific output variable - Ideal for risk analysis

Answer 110

- Large well-structured questionnaire - Trying to address multiple things - Many questions may investigate the same ‘factor’ - Method allows for grouping variables into set of underlying factors - Confirmatory factor analysis - know what the factors are and have set them - Exploratory Factor analysis - assume there are factors but not setting them

Answer 111

- Form of behavioural analytics - Ideal for examining user behaviour - Allow for exploration between cohorts - Group of people who share common characteristics over a given time frame

Answer 112

- Works by organising items into groups or clusters on how associated they are - K-means clustering - n data points in k clusters - Setting different number of clusters gives different results - Works at a data-set level - every point is assessed relative to the others - data must be as complete as possible - Intracluster distance - distance between clusters - Intercluster distance - distance within clusters

Answer 113

- Useful to see how variable changes over time - Forecasting via trends

Answer 114

- Natural language processing technique to determine whether data is positive, negative or neutral - Not terribly refined - can’t figure out sarcasm

Answer 115

Research for curiosity vs research to answer a specific question

Project Prep Benchtest Flashcards

(140 cards)