Study Guide - questions B Flashcards
(162 cards)
1
Q
What type of sampling has been shown to lead to significant bias?
A
Convenience Sampling - asking subjects easy to identify
2
Q
In rare event analysis, it may be advantageous to bias the sampling toward sampling those individuals most likely to have….
A
Experience the event of interest. Known as stratified random sampling.
3
Q
This sampling method ensures that each subgroup of a given population is adequately represented within the whole sample population of a research study?
A
Stratified random sampling
4
Q
It is common to use __________ (and especially regression) to specify the value of interest as a function of the covariates (characteristics).
A
Response surface modeling
5
Q
When the variable is ratio scale, _________ are often used to achieve normality.
A
Box-Cox Transformations
6
Q
When the dependent variable is categorical, the regression model is typically ______?
A
logistic
7
Q
When the dependent variable is ordinal, the regression model is typically ordered ______?
A
logit
8
Q
When the dependent variable is ratio, ________ is often used?
A
standard regression
9
Q
If Y is the dependent variable and X1…Xn represent the independent variables, then the typical regression model has the form ________?
A
y=E[Y] + e
where is e is a normally distributed error term, and E[Y] the expected value of Y is a parameterized function (X1…Xn).
10
Q
Time series analysis typically corrects for _____?
A
Season patterns, and provides a natural way of identifying trends
11
Q
Sampling plan - A simple rule of thumb is that _______ the number of individuals sampled reduces the uncertainty in half.
A
Quadrupling
12
Q
Sampling plan - _______ is a common way to measure uncertainty
A
Standard deviation
13
Q
Sampling plan - If standard deviation does not exist, then the difference between the ____________, is more appropriate.
A
third and first fractile of the uncertainty distribution
14
Q
Sampling plan - If our uncertainty is described by an exponential family distribution, it will have how may parameters?
A
two
15
Q
Determining questions to be asked - A key issue in designing the experiment is determining what?
A
The nature of the variable being assessed - i.e. categorical
16
Q
What type of scale asks YES/NO questions or multiple choice for ______
A
Nominal scales
17
Q
For ordinal scales, it is possible to define the normalized quantity for each response x by the fraction of responses __________?
A
Less than or equal to x
18
Q
Semantic differential survey responses with a form “very hard, somewhat hard, OKAY, somewhat easy, very easy”, where two ends of the scale represent opposites, the response is _______?
A
Ordinal
19
Q
What survey approach asks individuals to rate various factors in order of importance?
A
Rank- order
20
Q
Determining a control group - measurements are typically only meaningful if there is reference to some kind of _____________?
A
Underlying standard
21
Q
When the item is an uncertain quantity, the score of an item is the probability of the item outranking a randomly chosen item from the __________?
A
Benchmark group
22
Q
The benchmark group is commonly referred to as a _____ with the item’s score being called its _______?
A
control……effect size
23
Q
The purpose of extraction is to collect all this data from the many sources so that it can eventually be loaded into a common ________.
A
database
24
Q
In extracting data, it is critical to know the _______ from which each data element was taken.
A
data source
25
What is it called if there is a change in the clients analysis, and its important to transition the database to reflect the data sources which the new clients consider important?
traceability - and typically requires careful documentation
26
What are three reasons why survey quality may be deficient?
1. Respondents get fatigued and put in any value
2. Respondents may be offended by questions and deliberately fill in false answers
3. Respondents refuse to fill out the survey
27
Data cleaning involves the following 6 items:
1. Identifying the range of valid responses
2. Identifying invalid data responses
3. Identifying inconsistent data encodings
4. Identifying suspicious data responses
5. Identifying suspicious distribution of values
6. Identifying suspicious interrelationships between fields.
28
A key part of data cleaning is determining whether the data makes sense, and also involves handling _______.
Null or missing values
29
What are four possible solutions to missing values?
1. Deletion
2. Deletion when necessary
3. Imputing a value
4. Randomly imputing a value
30
What are the 10 "Cs" checks on quality of the data?
1. Completeness
2. Correctness
3. Consistency (is data under a given field consistent with definition of that field?)
4. Currency (is data obsolete?)
5. Collaborative (is data based on one opinion or a consensus of experts?)
6. Confidential
7. Clarity (is data legible and comprehensible)
8. Common format
9. Convenient (can data be conveniently and quickly accessed)
10. Cost-effective (is cost of collecting data commensurate with its value).
31
A data warehouse is generally used to describe these three things:
1. A staging area
2. Data integration in centralized source
3. Access layers in OLAP data marts
32
Data marts are organized along a single point of view for efficient data retrieval. It allows analysts to do these 5 things:
1. Slice data (filtering)
2. Dice data (grouping)
3. Drill down
4. Roll-up
5. pivot
33
What are three examples of fact tables?
1. Transaction fact tables
2. Snapshot fact tables (at point in time)
3. Accumulating fact tables (aggregate facts)
34
Do dimension tables have a larger or smaller number of records compared to fact tables?
smaller
35
What are 5 examples of dimension tables?
1. time
2. geography
3. product
4. employee
5. range
36
Discovering relationships in data - what are 5 methods to reduce dimensions in the data?
1. PCA or factor analysis (can determine if there is correlation across different dimensions)
2. Frequency-inverse document frequency
3. Feature hashing (creating fixed number of features)
4. Sensitivity analysis and wrapper methods
5. Self-organizing maps and Bayes nets
37
When data has a variable number of features, _________ is an efficient method of creating a fixed number of features which form the indices of an array.
Feature hashing
38
For unstructured text data, __________ identifies the importance of a word in some document in a collection by comparing the frequency with which the word appears in the document...
frequency-inverse document frequency
39
_______ and _______ are typically essential when you don't know which features of your data are important.
Sensitivity analysis and wrapper methods
40
Wrapper methods, unlike sensitivity analysis, typically involving identifying a set of features on a small sample and then testing that set on a ________.
holdout sample
41
________ and _______ are helpful in understanding the probability distribution of the data.
Self-organizing maps and Bayes nets
42
Extracting features - ________ is required to ensure your data stays within common ranges.
Normalization
43
Format conversion is typically required when data is in __________?
binary format
44
Fast Fourner Transformations and Discrete wavelet transformations are used for _________?
frequency data
45
Coordinate transformations are used for geometric data defined over ________?
Euclidian
46
Collecting and summarizing data - These three plots provide compact representations of how data is distributed?
1. Box plots
2. Scatter plots
3. box and whisker plots
47
Collecting and summarizing data - when the data can be reasonably described in parametric distributions, ___________ are even more efficient ways of summarizing data.
distribution fitting
48
Collecting and summarizing data - ___________ aggregation is an effective way of summarizing all the information available on an entity
Baseball card
49
Adding new information to the data - ________ is recommended for tracking source information and other use-defined parameters.
Annotation
50
Adding new info to the data - ____________ and _______ can be helpful in processing certain data fields together or in using one field to compute the value of another.
Relational algebra rename and feature addition
51
What are the 6 methods for segmenting data to find natural groupings?
1. Connectivity-based methods (hierarchical clustering)
2. Centroid-based methods
3. Distribution-based methods
4. Density-based method
5. Graph-based methods
6. Topic modeling (text data)
52
segmentation -A connectivity-based method called _________ generates an ordered set of clusters with variable precision.
Hierarchical clustering
53
segmentation - A centroid-based method with a known number of clusters
K-means clustering
54
segmentation - A centroid-based method with an unknown number of clusters.
x-means clustering
55
segmentation - A centroid-based method that is an alternate way of enhancing k-means when the number of cluster is unknown
canopy clustering
56
segmentation - A distribution-based method that typically uses the expectation-maximization (EM) algorithm and is appropriate if you want any data elements' membership in a segment to be 'soft'
Gaussian mixture models
57
segmentation - Two density-based methods used for non-elliptical clusters are _________?
fractal and DB scan
58
segmentation - _________ methods are often based on constructing cliques and semi-cliques, and are useful when you only have knowledge of how one item is connected to another.
Graph-based models
59
segmentation - For text data, this method allows for segmentation of the data.
topic modeling
60
variable importance - When the structure of the data is unknown, these methods are helpful.
tree-based methods
61
variable importance - If statistical measures of importance are needed, these models are appropriate.
Generalized linear models
62
variable importance - if statistical measures of importance are NOT needed, these two methods are useful.
1. regression with shrinkage (e.g. Lasso or elastic net)
| 2. stepwise regression
63
classifying data into groups - These two methods are helpful if you're unsure of feature importance.
1. neural nets
| 2. random forests
64
classifying data into groups - If you require a highly transparent model, this type of model can be preferable.
decision trees (i.e. CART, CHAID)
65
classifying data into groups - What method should you use if the number of data dimensions is less than 20?
k nearest neighbor methods
66
classifying data into groups - If you have a large dataset with an unknown classification, what method should you use?
Naive Bayes
67
classifying data into groups - These models are useful in estimating an unobservable state based on observable values.
Hidden Markov models
68
Refining BP and AP statements- You may find at this point that the true _______ of the system isn't what you thought it was, and that therefore the analytics problem needs to be reframed around the newly surfaced constraint.
constraint
69
APF - When reformulating the "what" of the business problem into the "how" of the analytics problem, what are the four questions you need to ask?
1. What result do we want?
2. Who will act?
3. What will they do?
4. What will change in the organization as a result of the new information generated?
70
APF - This formal method of decomposition is a rigorous process that maps the translation of requirements from one level to the next (i.e. from business level to the first analytics level)
quality function deployment
71
APF - If you are formally decomposing and parsing a complex business statement, or less formally brainstorming with a project sponsor, it is critical to account for these two types of requirements.
Tacit and Formal
72
APF - This is the best known model for decomposing and parsing requirements.
Kano's requirements model
73
APF - Kano's requirements model distinguishes between unexpected customer delights, known customer requirements, and customer ________ that are not explicitly stated.
must-haves
74
APF - When you ask business stakeholders for a list of requirements, they will tend to focus on the "normal" requirements not the _______ requirements.
expected
75
APF - Your _________ functions are strongly related to your assumptions about what is important about this problem as well as the key metrics by which you'll measure the organizational response to the problem.
input/output functions
76
APF - Once you have inputs and general sense of their predicted effects, what is the next step?
communicate them to the team
77
APF - What are two simple approaches for communicating back to the team?
1. Input table
| 2. black box sketch
78
APF - Key business metrics need to be negotiated, published, committed to, and _______
tracked
79
APF - the output of the stakeholder agreement will vary by organization, but should include the following 5 items:
1. budget
2. timeline
3. interim milestones
4. goals
5. any known effort that is excluded as out of scope
80
APF - translation of problems from business domain to analytics domain requires that all parties agree to __________
definitions and terms
81
APF - Requirements should be these three things:
1. unitary (no conjunctions such as and, but, or)
2. positive
3. testable
82
APF - __________ is the act of breaking down a higher-level requirement to multiple lower-level requirements.
decomposition
83
Methodology - Almost all analytical methods can be classified into one of these three categories
1. Descriptive
2. Predictive
3. Prescriptive
84
Methodology - Generally speaking, this type of model answers the question "what is the best action or outcome?"
prescriptive
85
Methodology - three types of prescriptive techniques are:
1. Optimization
2. Simulation-Optimization
3. Stochastic Optimization
86
Methodology - 7 types of Optimization techniques:
1. Linear programming
2. Integer programming
3. non-linear programming
4. Mixed integer programming
5. Network optimization
6. Dynamic programming
7. Metaheuristics
87
Methodology - These types of methodologies include any forecasting models such as time-series models, moving averages, and auto-regression models. Answers the question "What could happen?"
predictive models
88
Methodology - List 7 types of predictive models:
1. Simulation
2. Regression
3. Statistical inferences
4. Classification
5. Clustering
6. Artificial Intelligence
7. Game Theory
89
Methodology - List three types of simulation techniques:
1. Discrete event
2. Monte Carlo
3. Agent-based modeling
90
Methodology - List 4 types of statistical inference techniques:
1. Confidence intervals
2. Hypothesis testing
3. Analysis of variance
4. Design of experiments
91
Methodology - Descriptive methodologies can be conveyed through these 2 methods:
1. Charts and graphs
| 2. numerical presentations (mean, median, mode, etc)
92
Methodology - These techniques answer the question "What happened?"
Descriptive
93
Methodology - Prescriptive analytics evaluates and determines new ways to operate, targets business objectives, and balances __________.
constraints
94
Methodology - What are the 7 primary factors that an analyst generally considers to select an appropriate methodology?
1. Time
2. Accuracy of the model
3. Relevance of the methodology and scope of project
4. Accuracy of the data
5. Data availability and readiness
6. Staff and resource availability
7. Methodology popularity
95
Methodology - ________ methods are most helpful when there is a need to pinpoint certain decisions to the level of quantifying the variables that enhance the performance under study.
Prescriptive
96
Methodology - common methods - This type of method is often used to understand bottlenecks in systems, handles cases that cannot be handled in queueing theory, and is often used for multistage processes modeling with variations in their arrivals and service times and utilizing shared resources to perform multiple operations.
Discrete event simulation
97
Methodology - common methods - This method is designed to identify the most efficient pathway to solution. i.e. it might identify the number of tellers needed to satisfy customers in a particular time frame such as no more than 10 minutes waiting
Queuing model
98
Methodology - common methods - This method is used primarily to estimate dependent variable randomness out of a set of independent variable randomness. This is necessary when distributions of the input variables are not normally distributed and the relationship to estimate the dependent variable is not simple (i.e. additive). Use when Queuing model is not needed.
Monte Carlo simulation
99
Methodology - common methods - This method is simulated as a collection of autonomous decision making entities that are used to discover emergent behavior that is hard to predict without simulation.
Agent-based modeling (ABM)
100
Methodology - common methods - This is a simulation approach used to understand the interaction of a complex system over time.
System dynamics (SD)
101
Methodology - common methods - This is the study of strategic decision-making processes through competition and collaboration
Game theory
102
Methodology - common method (econ) - Discounted rate used in capital budgeting to compare returns on investment opportunities.
IRR - Internal rate of return
103
Methodology - common method (econ) - Difference between present value of income vs. outgo
NPV - Net present value
104
Methodology - value of a future event or item based on current value that is adjusted by some standard
FV - Future value
105
Methodology - period of time after which an expenditure is fully amortized and income begins to accrue in excess of expsense
Payback period
106
Methodology - common methods - A class of statistical methods used to map dependent variables with independent variables and understand the significance between the variables and their correlations.
Regression
107
Methodology - common methods - method of model building that successively adds or deletes variables based on performance
Stepwise regression
108
Methodology - common methods - a regression analysis often used to predict the outcome of categorical variables
logistic regression
109
Methodology - common methods - What are two types of statistical inferences:
1. Confidence intervals
| 2. Hypothesis testing
110
Methodology - what are three types of AI models?
1. Artificial neural networks
2. Fuzzy logic
3. Expert systems
111
Methodology - common methods - What is a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event.
Markov chains
112
Methodology - ________ mapping requires more aggregate data compared to a discrete-event simulation model.
Value-stream
113
Methodology - Lower level aggregation is more accurate and descriptive, but is harder to ______ and will certainly lead to more mistakes.
validate
114
Methodology - Higher level aggregation usually provides _______ results that are easier to understand.
faster
115
Methodology - The general rule of thumb is to model at the highest level of aggregation possible that will ensure a satisfactory level of _______
accuracy
116
Methodology - It is often advisable to run scenarios on the "back of the envelope" often referred to as QnD
quick and dirty
117
Methodology - products that specialize in visualization, optimization, simulation, data mining, and statistical are ________
software tools
118
Methodology - After the model is developed, the _______ step refers to making certain that the model is built the way it was designed and meant to be.
verification
119
Methodology - After the model is developed, the _______ step refers to making certain that the model is representing real-life to a certain level of accuracy.
validation
120
Methodology - The help the testing process, it is advisable to divide data into these three portions:
1. Building
2. Testing
3. Validation
121
Model building - A logistic regression, a decision tree, and a neural network can all predict in _____ target.
binary - (this is NOT typically done a priori. Instead you might identify several types of models, fit them all, and select a champion)
122
Model building - It should be possible to perform ______ in a real-time production environment where specialized analytics software might not be available.
Scoring
123
Model building - A predictive model should always be selected using an honest assessment of the model on ______ data.
holdout
124
Model building - A model that will be used to select the "top x%" from a sample should be assessed using a metric that evaluates the rank order of predicted values such as these three things:
1. concordance
2. discordance
3. ROC/c-statistic
125
Model building - validation assessment techniques can vary and include the following 3 types:
1. data splitting
2. k-fold cross validation
3. leave-one-out cross validation
126
Model building - Honest validation assessment - It is critical that the observations used to fit the model and estimate parameters are not observations that are ________ in the assessment.
scored
127
Model building - honest assessment with data splitting on binary target - You must select a large sample of data for modeling, for a binary target a good practice is to ensure that you have at least _______ observations in the small of the two classes.
2000
128
Model building - honest assessment - use stratified random sampling without replacement to create two data sets with appx. the same proportion of ______ target levels.
0 and 1
129
Model building - Fit models and estimate parameters using the _______ data.
training
130
Model building - using assessment statistic, score observations in the _______ data set.
validation
131
Model building - If the model uses stop training, pruning, or model selection without stopping rules, then those selection should always be based on the ________ data performance
validation
132
Model building - selection - you might select the champion based on a combination of model performance and ________
interpretability - (models like neural networks might not be selected because they are difficult to interpret, but might be used as a benchmark against which other models are compared)
133
Model building - Segmentation through clustering, rule generation through market basket/association analysis, deriving links among nodes through social network analysis, measurement of latent variables through common factor analysis are all examples of:
unsupervised techniques
134
Model building - Techniques for validating unsupervised analyses are not as straightforward and typically rely on the analysts _________
best judgement
135
BPF - Popular way to frame business opportunity or problem is to obtain reliable info on the 5 Ws. What are they?
1. Who are the stakeholders
2. What problem are we trying to solve
3. Where does the problem occur
4. When does the problem occur
5. Why does the problem occur
136
BPF - Of the 5 Ws, which is the most critical to the long term success of the project
Who - stakeholders
137
BPF - In determining if problem is amenable to analytics solution, most important question is can the organization accept and ______ the answer.
deploy
138
BPF - If there is no feasible way forward, the ethical analyst will notify who?
stakeholders
139
BPF - After initial analysis, it may be necessary to refine the problem statement to make it more accurate, more appropriate to the stakeholders, or _______
more amenable to available analytic tools/methods
140
BPF - It will be necessary to define constraints. These constraints could be any of the three:
1. analytical
2. financial
3. political
141
BPF - If an optimization problem has a large number of constraints, it may need to be restated with fewer constraints and/or a less complex _________.
objective function
142
BPF - List 4 types of potential constraints:
1. Desired accuracy and repeatability
2. Program cost
3. Timeframe
4. Number of stakeholders impacted
143
BPF - After problem statement is set, you define business benefits which can be quantitative or qualitative. This is also known as the _______
business case
144
overall - What are the 5 E's that are the pillars of the Certified Analytics Professional?
1. Ethics
2. Education
3. Experience
4. Examination
5. Effectiveness
145
Model Building - what are the 4 overall objectives in the Model Building phase?
1. Identify and build effective model structures
2. Run and evaluate
3. Calibrate models and data
4. Integrate the models
146
Deployment - what are the two methods used for deployment?
1. CRISP-DM (cross industry standard process for data mining)
2. DMAIC (6 sigma - define, measure, analyze, improve, control)
147
Deployment - What are the four steps for CRISP-DM deployment?
1. Planning deployment - your methods for integrating data mining discoveries into use
2. Planning monitoring and maintenance
3. Reporting final results
4. Reviewing final results
148
Deployment - After deployment, it is necessary to ensure your answer is still tied to the original question. However, discrepancies can creep in. It is common for business context to have changed, which can invalidate key ________?
Assumptions
149
Deployment - For organizations to accept the results of the process, those results must be integral and acknowledged as having _______?
Integrity (not just what senior mgmt wants to hear)
150
Deployment - What are the 2 key items to consider as a model becomes the basis for an organization taking action?
1. Plan the deployment
| 2. Plan monitoring and maintenance
151
Deployment - When surveying key stakeholders use of the model, pay attention to functional areas where the model is being ignored - this will tell you where key assumptions have been invalidated and use that as a way to ____________ the model.
Strengthen and update
152
Life cycle mgmt - A good lifecycle process helps with the following 3 items:
1. Keep the process orderly
2. minimizes cost and efforts
3. provides business users with clear roles
153
Life cycle mgmt - An effective process requires defining the roles of the various departments involved and the _______ process that will be used to iron out differences and make decisions.
Governance
154
Life cycle mgmt - For the model to be trusted it has to be _______.
Repeatable
155
Life cycle mgmt - Documentation should include the following 6 items:
1. Key assumptions made about the business context and analytics problem
2. Data sources and schema
3. Methods used to clean and harmonize the data
4. Model approach and model review artifacts
5. Documentation for any software code written
6. Recommendations for future improvements to the model
156
Life cycle mgmt - Evalution criteria should be created up front both in terms of the business results expected and the ______ and ______ expected from the model.
Accuracy and Confidence
157
Life cycle mgmt - What are 5 useful model evaluation criteria?
1. Value of the model in terms of the business
2. Does the model discover/predict something that is new and useful?
3. Is the model reliable across a wide range of data?
4. Can a "lift" or "gain" graph be constructed to show how well the model is predicting?
5. Check if the model's predictions on unknown data vs. train/test data
158
Life cycle mgmt - When the model quality starts to decay, it is time for the next step of _______ the model and rechecking its _______.
Recalibrating, assumptions
159
Life cycle mgmt - The results of the model should be tracked over the long term because a model may degrade if either of these 2 things happen:
1. input data changes
| 2. user requirements change
160
Life cycle mgmt - If there has been a fundamental change in a key assumption or two, then the project needs to be....
revalidated against the business problem (to see if the overall approach is still valid)
161
Life cycle mgmt - One of the keys to a successful analytics project or engagement is appropriate _______ for the users of the model and its results
training
162
Life cycle mgmt - One way to demonstrate business benefits of a model is to compare how your organization is doing against industry _______ during the time period in question.
benchmarks
