Big Data Overview

Question 1

Big Data

Answer 1

refers to non-conventional strategies and
innovative technologies used by businesses and
organizations to capture, manage, process, and make
sense of a large volume of data

Question 2

challenges of big data

Answer 2

*Capturing, transporting, and moving the data

*Managing - the data, the hardware involved, and the software

*Processing - to provide insight

*Storing - safeguarding and securing

Question 3

conventional BI & DWH architecture

Answer 3

App Servers
Network Switches
Database Servers
SAN Switch
Storage Array
proprities : SQL based
High availability
Enterprise database
Right design for structured data

Question 4

Analytics Architecture

Answer 4

Edge node
Network switches
Data nodes
porprities :Not only SQL based
High scalability, availability, and flexibility
Compute and storage in the same box for reducing network latency
Right design for semi-structured and unstructured data
Data and Application are in the same machine (Data nodes)

Question 5

The Vs of Big Data

Answer 5

Volume Variety Velocity{the speed at which vast amounts of data are
being generated, collected and analyzed} Veracity {is the quality or trust of the data} Value

Question 6

Volume

Answer 6

how much data is there?

Question 7

Variety

Answer 7

• how many different types of sources are there?

Question 8

Velocity

Answer 8

• how quickly is the data being created, moved, or
  accessed?

Question 8

Veracity

Answer 8

can we trust the data?

Question 9

Validity

Answer 9

• is the data accurate and correct?

Question 9

Viability

Answer 9

• is the data relevant to the use case at hand?

Question 10

Volatility

Answer 10

• how often does the data change?

Question 11

Vulnerability -

Answer 11

can we keep the data secure?

Question 12

Visualization

Answer 12

• how can the data be presented to the user?

Question 13

Value

Answer 13

• can this data produce a meaningful return on
  investment?

Question 14

Types of Big Data

Answer 14

Structured semi-structured unstructured

Question 15

Structured

Answer 15

Data that can be stored
and processed in a
fixed format, aka schema

Question 16

Semi-structured

Answer 16

Data that does not have a formal structure of a data model, i.e. a table
definition in a relational DBMS, but nevertheless it has some
organizational properties like tags and other markers to separate semantic
elements that makes it easier to analyze, aka XML or JSON

Question 17

Unstructured

Answer 17

Data that has an unknown form and cannot be stored in RDBMS and
cannot be analyzed unless it is transformed into a structured format

Question 18

5’Vs and Data : Volume Velocity Variety Veracity Value

Answer 18

Data at rest : not in use
Data in motion : analyzing data on the fly
Data in many forms
data in doubt
Data into money

Question 19

Hadoop

Answer 19

Apache open source software framework for reliable,
scalable, distributed computing of massive amount of data

Question 20

What Hadoop is good for

Answer 20

Massive amounts of data through
parallelism

A variety of data (structured, unstructured,
semi-structured)

Inexpensive commodity hardware

Question 21

Hadoop is not good for

Answer 21

Not to process transactions (random access)

Not good when work cannot be parallelized

Not good for low latency data access

Not good for processing lots of small files

Not good for intensive calculations with little data

Question 22

Data Lake

Answer 22

a large storage repository and processing engine

Question 23

Data munging/Data wrangling

Answer 23

is the process of
transforming and mapping data from one
“raw” data form into another format with the intent of
making it more appropriate and valuable for analytics

Question 24

Oceans of data

Answer 24

data at rest

Question 25

Streams of data

Answer 25

data in motion

Question 26

The main
categories of data are

Answer 26

• Structured
• Unstructured
• Natural language
• Machine-generated
• Graph-based
• Audio, video, and image
• Streaming

Question 27

The six
design
principles in
Industry 4.0

Answer 27

Interoperability
Virtualization
Decentralization
Real-time Capability
Service Orientation
Modularity