Big Data

Question 1

What is continuous data?

Answer 1

Numerical data with infinite options such as age or weight.

Question 2

What is discrete data?

Answer 2

Numerical data with finite options such as shoe size.

Question 3

What is ordinal data?

Answer 3

Categorical data that is hierarchical in nature such as pain severity or mood

Question 4

What is nominal data?

Answer 4

Non hierarchical categorical data such as eye colour or dog breed.

Question 5

What is Data Aggregation?

Answer 5

Combining all the data together in a uniform format

Question 6

What is Normalization?

Answer 6

Scaling data into a regularised range so it can be compared more accurately.

Question 7

What is Apache Kafka and what are its uses?

Answer 7

Apache Kafka is a distributed messaging system for real time data streams, it is used for real-time data pipelines, log aggregation and stream processing.

Question 8

What is Apache Flume and what are its uses?

Answer 8

Apache Flume collects, aggregates and moves large amounts of log data from multiple sources to a centralised data store, it is primarily used for log data collection.

Question 9

What is AWS Kinesis and what are its uses?

Answer 9

AWS Kinesis is a real time data streaming service for ingesting and processing data streams and it is used real-time event tracking and IoT data ingestion

Question 10

What are the pros and cons of Kafka?

Answer 10

Pros: Scalable, low latency, fault tolerant
Cons: requires setup, complex to manage.

Question 11

What are the pros and cons of Flume?

Answer 11

Pros: simple, strong Hadoop integration
Cons: Limited to log data, not real-time

Question 12

What are the pros and cons of AWS Kinesis?

Answer 12

Pros: fully manages, AWS integration
Cons: AWS dependent, costs may rise

Question 13

What is a data warehouse?

Answer 13

A centralised repository for storing structured data. Optimised for querying and reporting on large scale datasets

Question 14

What are the uses of a data warehouse?

Answer 14

Business intelligence, reporting, analytics applications and historical analysis

Question 15

What are the pros and cons of a data warehouse?

Answer 15

Pros: fast queries on structured data
Cons: expensive, less flexible for unstructured data

Question 16

What is a data lake?

Answer 16

A centralised repository for storing unstructured, semi structured and structured data in raw format, supports later processing.

Question 17

What are the uses of a data lake?

Answer 17

Machine Learning, Big Data analytics

Question 18

What are the pros and cons of a data lake?

Answer 18

Pros: Cost Effective, stores all types of data
Cons: lack of governance, complex for analysis

Question 19

What is a data lakehouse?

Answer 19

Modern data architecture combining features of a data lake and data warehouse allowing for efficient storing and processing of both structured and unstructured data

Question 20

What are the uses of a data lakehouse?

Answer 20

Predictive analytics, AI, large scale data processing.

Question 21

What are the pros and cons of a data lakehouse?

Answer 21

Pros: combines flexibility and governance
Cons: still evolving, potential integration challenges

Question 22

What is Hadoop Distributed File System (HDFS)?

Answer 22

A primary storage system used by Hadoop applications which allows for distributed storage and processing of large datasets across clusters of computers.

Question 23

What is the architecture of HDFS?

Answer 23

Files are split into large blocks (Default 128 or 64MB) and distributed across multiple machines, each block is replicated across multiple nodes for fault tolerance

Question 24

What is HDFS used for?

Answer 24

Scalable storage of unstructured data such as logs, media or large datasets in scientific research

Question 25

How does HDFS compare to traditional storage?

Answer 25

HDFS is optimised for write-once, read-many operations and offers large scale and high fault-tolerance.

Question 26

What is MongoDB?

Answer 26

A NoSQL document-oriented database that stores data in flexible JSON-like documents. It's suited for applications requiring hierarchical or complex structures like a CMS.

Question 27

What are the pros and cons of MongoDB?

Answer 27

Pros: schema flexibility, easy to scale Cons: Not optimized for heavy write workloads

Question 28

What is Cassandra?

Answer 28

A distributed wide-column store designed for high availability and scalability across multiple nodes, often used in IoT apps.

Question 29

What are the Pros and Cons of Cassandra?

Answer 29

Pros: highly available, optimized for writes Cons: Limited complex querying, eventual consistency

Question 30

What is Hadoop MapReduce?

Answer 30

A programming model for processing large datasets in parallel across a distributed cluster.

Question 31

Where is Hadoop MapReduce used?

Answer 31

processing log files, web indexing, and aggregating large-scale data in parallel.