Domain 3 - Data

Question 1

Completeness

Answer 1

Are all the fields of the data complete

Question 2

Correctness

Answer 2

Is the data accurate

Question 3

Consistency

Answer 3

Is the data provided under a given field and for a given concept consistent with the definition of that field and concept

Question 4

Currency

Answer 4

Is the data obsolete

Question 5

Collaborative

Answer 5

Is the data based on one opinion or on a conses of experts in the relative area

Question 6

Confidential

Answer 6

Is the data secure from unauthorized use by individuals other than the decision maker

Question 7

Clarity

Answer 7

Is the data legible and comprehensivle

Question 8

Common Format

Answer 8

Is the data in a format easily used in the application for which it is intended

Question 9

Convenient

Answer 9

Can the data be conveniently and quickly access by the intended user in a time-frame that allow for it to be effectively used

Question 10

Cost-effective

Answer 10

Is the cost of collecting and using the data commensurate with its value

Question 11

Data warehouses typically describe (three things)

Answer 11

1. A Staging area
2. Data integration
3. Access Layers

Question 12

Data warehouse staging area

Answer 12

The operational data sets from which the information is extracted

Question 13

Data integration

Answer 13

The centralized source where the data is conveniently stored

Question 14

Access layers

Answer 14

Multiple OLAP data marts which store the data in a form which will be easy for the analysis to retrieve

Question 15

Data mart

Answer 15

A subset of the data warehouse organized along a single point of view (e.g., time, product type, geography) for efficient data retrieval.

Usually oriented to a specific business line or team. Whereas data warehouses have an enterprise-wide depth, the information in data marts pertains to a single department.

Question 16

Data marts allow analysts to… (five things)

Answer 16

1. Slice Data
2. Dice Data
3. Drill-down/up
4. Roll-up
5. Pivot

Question 17

Slice data

Answer 17

filtering data by picking a specific subset of the data-cube and choosing a single value for one of its dimensions

Question 18

Dice data

Answer 18

grouping data by picking specific values for multiple dimensions

Question 19

Drill-down/up

Answer 19

allow the user to navigate from the most summarized (high-level) to the most detailed (drill-down)

Question 20

Roll-up

Answer 20

summarize the data along a dimension (e.g., computing totals or using some other formula)

Question 21

Pivot

Answer 21

interchange rows and columns (`rotate the cube’)

Question 22

Fact tables

Answer 22

used to record measurements or metrics for specific events at a fairly granular level of detail

Question 23

Transaction fact details

Answer 23

record facts about specific events (like sales events)

Question 24

Snapshot fact tables

Answer 24

record facts at a given point in time (like account details at month end)

Question 25

Accumulating snapshot tables

Answer 25

record aggregate facts at a given point in time

Question 26

Dimension tables

Answer 26

Hhave a smaller number of records compared to fact tables although each record may have a very large number of attributes. Dimension table includes time dimension tables, geography dimension table, product dimension table, employee dimension table, and range dimension tables.

Question 27

What to do with missing data (4 things)

Answer 27

Deletion of record Deletion when necessary Imputation Imputation at random

Question 28

Filtering

Answer 28

Filtering can involve using relational algebra projection and selection to add or remove data based on its value. Filtering usually involves outlier removal, exponential smoothing and the use of either Gaussian or median filters.

Question 29

Filling in missing data with imputation

Answer 29

If other observations in the dataset can be used, then values for missing data can be generated using random sampling or Monte Carlo Markov Chain methods. To avoid using other observations, imputation can be done using the mean, regression models or statistical distributions based on existing observations.

Question 30

Dimensionality reduction options for structured data

Answer 30

Principle component analysis or factor analysis can help determine whether there is correlation across different dimensions in the data

Question 31

Dimensionality reduction options for unstructured text data

Answer 31

term frequency-inverse document frequency (tf-idf): is a numerical statistic that is intended to reflect how important a word is to a document in a collection or corpus

Question 32

Feature hashing

Answer 32

Dimensionality reduction technique for when data has a variable number of features. Feature hashing is an efficient method for creating a fixed number of features which form the indices of an array

Question 33

Sensitivity analysis and wrapper methods

Answer 33

Used when you don't know which features of your data are important. Wrapper methods involve identifying a set of features on a small sample and then testing that set on a holdout sample.

Question 34

Self-organizing maps and Bayes nets

Answer 34

Used to understand the probability distribution of the data

Question 35

Normalization

Answer 35

Used to ensure data stays within common ranges. Prevents scales of data from obscuring interpretation and analysis

Question 36

When is format conversion used?

Answer 36

When data is in binary format

Question 37

When are Fast Fourier Transforms and Discrete wavelet transforms used?

Answer 37

With frequency data

Question 38

When are coordinate transformations used?

Answer 38

For geometric data defined over a euclidean space.

Question 39

Connectivity-Based clustering methods

Answer 39

AKA Hierarchical clustering | Generates an ordered set of clusters with variable precision

Question 40

Hierarchical clustering

Answer 40

AKA Connectivity-Based methods | Generates an ordered set of clusters with variable precision

Question 41

Centroid–Based clustering methods

Answer 41

When the number of clusters is known, k-means is a popular technique. When the number is unknown, x-means is a useful extension of k-means that both creates clusters and searches for the optimal number of clusters. Canopy clustering is an alternate way of enhancing k-means when the number of clusters is unknown.

Question 42

Distribution-based clustering methods

Answer 42

Gaussian mixture models, which typically used the expectation-maximization (EM) algorithm. Used if you want any data element’s membership in a segment to be `soft.’

Question 43

Density-based methods

Answer 43

Clustering method for non-elliptical clusters - fractal and DB scan can be used.

Question 44

Graph-Based methods

Answer 44

Clustering method for when you have knowledge of how one item is connected to another. Cliques and semi-cliques

Question 45

Topic modelling

Answer 45

Clustering method for text data

Question 46

How to determine important variables when structure of data is unknown?

Answer 46

Tree-based methods

Question 47

How to determine important variables when statistical measures of importance are needed?

Answer 47

GLM models

Question 48

How to determine important variables when statistical measures of importance are not needed?

Answer 48

Regression with shrinkage (e.g., LASSO, elastic net) and stepwise regression

Question 49

How to classifying data into existing groups when unsure of feature importance?

Answer 49

Neutral nets and random forests are helpful

Question 50

How to classifying data into existing groups when unsure of feature importance but require a transparent model?

Answer 50

Decision trees (e.g., CART, CHAID)

Question 51

Key problem with neutral nets and random forests

Answer 51

Difficult to explain, "black box", less transparent than decision trees

Question 52

How to classifying data into existing groups with fewer than 20 dimensions?

Answer 52

K-nearest neighbours

Question 53

When to use Naive Bayes?

Answer 53

When you have a large dataset with an | unknown classification signal

Question 54

When to use Hidden Markov Chains?

Answer 54

When estimating an unobservable state based on observable values