8.2 Data engineering, extraction and mining

Question 1

Data engineering is the practice of

Answer 1

• designing and building systems for collecting, storing and analysing large sets of data

Question 2

The three stages in blending data from: (Pg 209)

Answer 2

• multiple sources into a destination system (eg data warehouse) are (ETL):
  
  • Extraction
  • Transformation
  • Loading
• ETL system provides data in a clean and ready to use format

Question 3

Extraction is the process of

Answer 3

-harvesting data from multiple sources
- prior to this is data profiling which involves analysing data to understand it’s content, format and structure

Question 4

Transformation takes the

Answer 4

• extracted data and transforms it into a suitable format for the destination database and intended use
• this is done using code and rules designed to interrogate the source data and then turn it into new format as per code instructions

Question 5

Loading is when the

Answer 5

• newly cleaned and prepared data is uploaded into the destination database ready for use

Question 6

A data warehouse is a

Answer 6

• store for data that has been loaded in ETL process
• it will be stored in a systematic and logical way ready for further interrogation and analysis by bus intelligence function.

Question 7

Business intelligence (BI) is the

Answer 7

• technology driven process of analysing bus data to create insightful and actionable info to help improve the operations or products of bus
• present finding using data visualisation techniques and traditional summaries and reports
• Data mining is an important component of BI

Question 8

Data mining is the

Answer 8

• process of uncovering patterns and other valuable information from large sets of data in the data warehouse

Question 9

Big data, ETL and BI - main challenges for traditional ETL programs:

Answer 9

• Rate of growth - data volumes are growing at unprecedented levels
• Types and sources - data for modern bus comes in all shapes and sizes and from multiple internal and external sources
• New data tech - use innovative techniques to handle the challenges of big data - volume, variety, veracity and velocity

Question 10

The FF’s role in data engineering, extraction and mining

Answer 10

• they will use their unique set of competencies to act as an interface between data specialists and the bus
• making the generated data commercially relevant and transforming it into valuable info

Question 11

Data engineering, extraction and mining and the information to impact framework (pg 212) - they will add value through:

Answer 11

• Assembling info - raw data from variety of relevant sources is collected and cleaned and transformed into meaningful info
• Generating insights - financial and non financial info is analysed for insights to improve performance
• Influencing decision makers - insights are used to advice and influence the relevant stakeholders
• Achieving impact - guide actions to help achieve the desired impact