09 - Data Analysis

Question 1

What is data analysis?

Answer 1

Data Analysis is a process of collecting, transforming, cleaning, and modeling data with the goal of discovering the required information.

The idea is to do a separation of a whole into its component parts. For example in criminal data analysis:

• Build criminal pattern to detect and prevent crimes
• Breakdown the pattern of crimes to evaluate when, where, and why they are occurring

Question 2

What are data limitation issues?

Answer 2

• Missing data
  
  • Missing altogether
  • Specific items
• Altered data
  
  • In the rest, in-transit, in the possession
  • Intentional vs. unintentional
• Different definitions of the same data (Possibility of misunderstanding)
• Non-existent data (Data of interest do not exist or is not in a usable form)
• Data is available in different forms

Question 3

What are the four steps of data analysis?

Answer 3

1. Planing
   
   • Identification of data, data availability & quality
2. Data Collection (Inventory / Pre-Processing)
   
   • Improving data quality
   • Information gathering
3. Data Preparation
   
   • Structure data for analysis tool
   • Data cleaning
   • Depends on type of data
4. Data Analysis
   
   • Visualization
   • Statistics
   • Relationship linking

Question 4

What are the major tasks of data pre-processing?

Answer 4

1. Data summarisation
   
   • Understand the distribution of data
2. Data cleaning
   
   • Fill in missing values, remove outliers / noisy data
3. Data integration
   
   • Integration multiple data sources
4. Data transformation
   
   • Aggregation of data
5. Data reduction
   
   • Take whats relevant
6. Data discretisation
   
   • Reduction with focus on numbers

Question 5

What are the four types of central tendency?

Answer 5

• Mean (sensitive to extreme value)
• Median (remove outliers)
• Midrange (min + max / 2)
• Mode (Value that occures most frequently)

Question 6

What is the five number summary?

Answer 6

The five-number summary is a set of descriptive statistics that provide information about a dataset. It consists of the five most important sample percentiles:

min, max, med, Q1 (first quartile), Q3 (third quartile)

The five-number summary can be used to examine the credit card summary and its distribution and detect outliers as fraud values.

Question 7

What is a data warehouse?

Answer 7

A data warehouse, also known as an enterprise data warehouse, is a system used for reporting and data analysis, and is considered a core component of business intelligence. DWs are central repositories of integrated data from one or more disparate sources.

Question 8

What is Behaviour profiling?

Answer 8

Finding of suspicious behaviour by analyzing databases. Capability to recognize patterns of criminal activity. Predict when and where crimes are likely to take place.

Question 9

What is data mining and what is the ultimate goal?

Answer 9

Data mining (knowledge discovery in databases). Extraction of interesting (hidden) information or patterns from data in large databases.

The ultimate goal of data mining is the prediction of human behaviour.

Question 10

What are potential applications for data mining?

Answer 10

• Database analysis and decision support.
• Market analysis and management
• Risk analysis and management
• Fraud detection and management
• Text mining (news group, email, documents)
• Web analysis
• Intelligent query answering

Question 11

How can data mining help to detect fraud?

Answer 11

Data mining can help to detect fraud by using the historical data to build models of fraudulent behavior and then to use data mining techniques to identify similar instances.

Question 12

Name a few different data mining techniques?

Answer 12

• Link analysis
• Intelligent agents
• Text mining
• Neural Networks
• Machine-learning algorithms

Question 13

What is clustering?

Answer 13

Clustering is a technique to group a set of data objects into clusters.

Question 14

What are the three main components of link analysis?

Answer 14

• Entities
• Events
• Associations

Question 15

Name a few data mining tools?

Answer 15

• Open-Source
  
  • Weka
  • Orange
  • Rapid Miner
  • NLTK
• Commercial
• SAS
• IBM SPSS
• Web site
• kdnuggets.com

Question 16

What is the data mining process?

Answer 16

1. Creating a target data set
2. Data Cleaning & Pre-Processing
3. Data reducation & transformation
4. Data Mining
5. Knowledge Presentation