Big Data, Lambda Architecture, & SQL

Question 1

Lambda architecture

Answer 1

A data processing architecture designed to handle massive volumes of data, often associated with big data applications. It was introduced by Nathan Marz to address the challenges of real-time data processing and batch processing in big data systems. Lambda architecture combines both batch and real-time processing to provide a comprehensive and scalable solution for processing large datasets.

Question 2

Purposes of Lambda Architecture

Answer 2

Real-Time Data Processing
Batch Data Processing
Scalability
Fault Tolerance
Consistency of Results

Question 3

Fault Tolerance in Big Data

Answer 3

Big data systems are distributed and complex, making them prone to failures. Lambda architecture’s fault tolerance ensures that the system can recover from failures and maintain data consistency.

Question 4

“Consistency of Results”

Answer 4

Lambda architecture guarantees that the data processed by both real-time and batch layers eventually converges, ensuring consistent results across the entire system.

Question 5

Scalability in Big Data Systems

Answer 5

Big data systems need to scale horizontally to accommodate the increasing volume of data and processing requirements. Lambda architecture’s design allows for horizontal scaling of both the real-time and batch processing components.

Question 6

Batch Data Processing

Answer 6

In addition to real-time data, big data systems often deal with historical data and large datasets that require batch processing. Lambda architecture includes a batch processing layer to handle these vast amounts of data efficiently.

Question 7

Real-Time Data Processing

Answer 7

Big data systems often receive continuous streams of data from various sources, such as sensors, social media, or clickstreams. Lambda architecture incorporates a real-time processing layer to handle these streams of data and provide low-latency processing and analytics.

Question 8

Three Layers of Lambda Architecture

Answer 8

1. Batch Layer
2. Speed Layer
3. Server Layer

Question 9

Batch Layer for Large-Scale Data Processing

Answer 9

Big data applications deal with massive volumes of data that cannot be processed in real-time due to computational limitations. The Batch Layer is designed to handle these large-scale datasets efficiently by breaking them into manageable batches and processing them in parallel.

Question 10

Batch Layer for Historical Data Processing

Answer 10

The Batch Layer is well-suited for processing historical data, which accumulates over time. It enables the system to process and analyze the entire historical dataset to produce accurate and comprehensive batch views.

Question 11

Batch Layer for Precomputing Results

Answer 11

The Batch Layer precomputes batch views by running computationally intensive algorithms and data processing tasks on the entire dataset. This precomputation provides consistent and reliable results for later queries.

Question 12

Batch Layer for Fault Tolerance

Answer 12

The Batch Layer’s batch processing is typically executed on distributed data processing frameworks, such as Apache Hadoop MapReduce or Apache Spark. These frameworks provide fault tolerance by handling failures and ensuring that the batch processing completes even in the presence of node failures.

Question 13

Batch Layer for Scalability

Answer 13

The Batch Layer can scale horizontally by distributing data and computation across multiple nodes in a cluster. As the dataset grows, additional nodes can be added to handle the increased workload, making it suitable for big data scenarios.

Question 14

Batch Layer for Consistant Results

Answer 14

By processing the entire dataset in batches, the Batch Layer ensures that the results are consistent and complete. It avoids the issues of partial or incomplete data views that may arise in real-time processing.

Question 15

The Batch Layer

Answer 15

The Batch Layer is one of the three main layers designed to handle big data processing. It is responsible for processing large volumes of historical data in batches. The Batch Layer’s primary function is to compute batch views or batch-processing results from the entire dataset.
The Batch Layer’s primary goal is to provide a comprehensive and accurate view of historical data, which complements the real-time processing provided by the Speed Layer in the Lambda Architecture. The Batch Layer’s precomputed batch views are stored and updated periodically, enabling low-latency query processing and efficient retrieval of historical data.

Question 16

The Speed Layer

Answer 16

The Speed Layer is one of the three main layers designed to handle real-time data processing in big data applications. The Speed Layer is responsible for processing and analyzing continuous streams of data in near real-time, providing low-latency results and insights.
The Speed Layer’s primary focus is to process and analyze real-time data streams, ensuring that the system can respond promptly to incoming events and provide real-time insights and analytics. By working in conjunction with the Batch Layer in the Lambda Architecture, the Speed Layer enables big data systems to handle both real-time and historical data efficiently, providing a complete and up-to-date view of the data for various use cases.

Question 17

Speed Layer for Real-Time Data Processing

Answer 17

Big data applications often deal with continuous streams of data from various sources, such as sensor data, social media feeds, logs, or clickstreams. The Speed Layer is designed to handle these streams of data in real-time, ensuring that data is processed and analyzed as it arrives.

Question 18

Speed Layer for Low-Latency Processing

Answer 18

The Speed Layer aims to provide low-latency results to support real-time decision-making and provide timely insights. It is essential for applications where immediate actions or responses are required based on incoming data.

Question 19

Speed Layer for Event-Driven Architecture

Answer 19

The Speed Layer is based on an event-driven architecture, where it continuously processes events as they occur. It responds to events as they arrive, making it well-suited for time-sensitive and dynamic data scenarios.

Question 20

Speed Layer for Stream Processing

Answer 20

The Speed Layer utilizes stream processing technologies, such as Apache Storm or Apache Flink, to process and analyze data streams efficiently. These technologies enable parallel processing and support fault tolerance in distributed environments.

Question 21

How the Speed Layer Compliments the Batch Layer

Answer 21

While the Batch Layer handles historical data processing, the Speed Layer complements it by processing real-time data. Both layers work together to provide a comprehensive view of the data, including both historical and up-to-date information.

Question 22

Speed Layer for Data Integration

Answer 22

The Speed Layer integrates with various data sources to ingest real-time data streams. It can process and aggregate the data, enrich it with contextual information, and make it available for real-time analytics.

Question 23

Speed Layer for Incremental Updates

Answer 23

Unlike the Batch Layer, which processes the entire dataset in batches, the Speed Layer performs incremental updates on the data as new events arrive. This enables it to provide real-time insights and responses.

Question 24

Speed Layer for Complex Event Processing

Answer 24

The Speed Layer can handle complex event processing tasks, identifying patterns, correlations, and anomalies in real-time data streams.

Question 25

Ther Serving Layer in Lambda Architecture

Answer 25

The Serving Layer is responsible for combining and serving the results from both the Batch Layer and the Speed Layer to provide a unified and up-to-date view of the data. Overall, the Serving Layer plays a crucial role in Lambda Architecture by combining and presenting the results from both the Batch Layer and the Speed Layer to users and applications. It ensures that big data applications can efficiently process and serve both historical and real-time data, enabling users to make informed decisions based on a comprehensive and up-to-date data view.

Question 26

Serving Layer for Unified Data Views

Answer 26

The Serving Layer combines the precomputed batch views from the Batch Layer and the real-time views from the Speed Layer to create a comprehensive and unified data set. This unified view ensures that queries can access both historical and real-time data.

Question 27

Serving Layer for Low-Latency Query Processing

Answer 27

The Serving Layer provides low-latency query processing by leveraging the real-time views generated by the Speed Layer. This allows users or applications to receive up-to-date data insights without significant delays.

Question 28

Serving Layer for Consistency of Results

Answer 28

The Serving Layer ensures that the results from both the Batch Layer and the Speed Layer eventually converge. This convergence guarantees that the data presented to users or applications is consistent and reflects the latest available information.

Question 29

Serving Layer for Scalability

Answer 29

The Serving Layer needs to handle query requests efficiently, even in the face of high data volumes and complex queries. It should be designed for horizontal scalability to accommodate increasing user demand.

Question 30

Serving Layer for User-Facing Interface

Answer 30

The Serving Layer provides an interface for users and applications to interact with the data. It offers APIs, web services, or other methods for querying and accessing the processed data.

Question 31

Serving Layer for Data Visualization and Reporting

Answer 31

The Serving Layer may also include tools and components for data visualization and reporting, allowing users to gain insights and analyze data in a user-friendly manner.

Question 32

Serving Layer for Load Balancing

Answer 32

To maintain low-latency query response times, the Serving Layer may use load balancing techniques to distribute incoming queries across multiple servers or nodes.

Question 33

Four Techniques for Load Balancing

Answer 33

1. Round-Robin Load Balancing: Requests are distributed sequentially across available resources in a circular manner. 2. Weighted Load Balancing: Each resource is assigned a weight based on its capacity or capabilities, and requests are distributed proportionally. 3. Least Connections Load Balancing: Requests are sent to the resource with the fewest active connections to evenly distribute the workload. 4. Dynamic Load Balancing: Load balancers continuously monitor resource utilization and dynamically adjust the distribution of workloads based on real-time data.

Question 34

Six Purposes of Load-Balancing

Answer 34

1. Optimizing Resource 2. Utilization 3. Reducing Processing 4. Bottlenecks 5. Minimizing Response Times 6. Fault Tolerance and High Availability 7. Scaling and Elasticity 8. Data Distribution in Distributed Storage

Question 35

Distributed Storage System Technologies

Answer 35

1. Apache Cassandra 2. Apache HBase 3. Amazon S3 (Simple Storage Service) 4. Google Cloud Storage 5. Microsoft Azure Blob Storage 6. Apache GlusterFS 7. Ceph 8. IBM Specturm Scale (GPFS) 9. OpenStack Swift 10. Red Hat Gluster Storage

Question 36

Load Balancing

Answer 36

Load balancing is a technique used in big data and distributed computing systems to distribute processing workloads evenly across multiple computing resources (e.g., servers, nodes, or clusters). The goal of load balancing is to optimize resource utilization, enhance performance, and prevent overloading specific components, ensuring that the system operates efficiently and reliably. Load balancing can be implemented using dedicated load balancer hardware or software, as well as through software-defined load balancers in cloud environments. It is a fundamental technique in building scalable, fault-tolerant, and high-performance big data systems that efficiently handle the challenges posed by large data volumes and processing requirements.

Question 37

Technologies used in the Batch Layer

Answer 37

1. Apache Oozie 2. Cascading 3. Apache Kylin 4. Apache Beam 5. Apache Crunch 6. Apache Pig 7. Apache Five 8. Apache Flink 9. Apache Spark 10. Apache Hadoop MapReduce

Question 38

Technologies used in the Speed Layer

Answer 38

1. Apache Storm 2. Apache Flink 3. Apache Kafka 4. Apache Samza 5. Amazon Kinesis 6. Google Cloud Dataflow 7. Microsoft Azure Stream Analytics 8. Spark Streaming 9. NATS 10. RabbitMQ

Question 39

Technologies used in the Serving Layer

Answer 39

1. Apache HBase 2. Apache Cassaandra 3. Apache Druid 4. Elasticsearch 5. AmazonDynamoDB 6. Google Cloud Bigtable 7. Apache Ignite 8. Redis 9. MongDB 10. Apache Solr

Question 40

Least Connections Load Balancing

Answer 40

Least Connections Load Balancing is a load balancing technique used in distributed computing and networking to distribute incoming requests or connections to a group of resources (such as servers or nodes) based on the current number of active connections on each resource. The basic principle is to direct new requests to the resource with the fewest active connections, aiming to evenly distribute the workload and prevent overloading any specific resource. Least Connections Load Balancing is a load distribution strategy that plays a significant role in big data systems, contributing to their scalability, fault tolerance, and efficient utilization of computing resources. It helps ensure that big data applications can efficiently process vast amounts of data and deliver timely insights to users and applications.

Question 41

Batch Processing

Answer 41

Batch processing is a data processing method in which data is collected, processed, and analyzed in fixed-size, discrete batches. In this approach, data is accumulated over a certain period or until a specific volume is reached before it is processed as a group. The processing occurs offline, away from real-time interactions, and is typically done during non-peak hours or when the system load is low. Batch processing is a fundamental concept in big data systems, allowing them to efficiently manage and process vast amounts of data. It complements real-time processing, such as stream processing, in the Lambda Architecture, providing a comprehensive and efficient solution for handling big data workloads.

Question 42

Benefits of Batch Processing for Big Data

Answer 42

1. Handling Large Volumes of Data 2. Scalability 3. Resource Optimization 4. Reduced Latency Sensitivity 5. Consistency and Determinism 6. Complex Computations 7. Error Handling 8. Reduced Overhead

Question 43

Consistency and Determinism in Batch Processing

Answer 43

Batch processing provides consistent and deterministic results. By processing an entire batch of data at once, the system ensures that all data points within the batch are processed using the same set of rules and algorithms, avoiding potential discrepancies due to real-time fluctuations.

Question 44

Resource Optimization

Answer 44

Resource optimization in the context of big data refers to the efficient and effective utilization of computing resources, storage, and other infrastructure elements to handle the large volumes of data and processing requirements of big data applications. It involves maximizing the performance, scalability, and cost-effectiveness of the resources while minimizing waste and inefficiencies. Resource optimization in big data involves various strategies, including load balancing, efficient data partitioning, smart scheduling of processing tasks, storage optimization, and leveraging specialized hardware or cloud-based services. It requires a careful understanding of the application's requirements, data characteristics, and performance goals. Overall, resource optimization is a fundamental aspect of big data management and infrastructure design. It plays a key role in ensuring that big data applications can handle large-scale data processing efficiently, meet performance expectations, and achieve cost-effectiveness, contributing to the success and sustainability of big data projects.

Question 45

Best way to reduce costs in big data

Answer 45

1. Cloud Adoption 2. Serverless Architectures 3. Data Compression (Storage Optimization) 4. Data Lifecycle Management 5. Efficient Data Processing 6. "Spot and Preemptible Instances" 7. Auto-Scaling 8. Data Archiving and Backup 9. Open Source Technologies 10. Resource Optimization 11. Data Governance and Quality 12. Data Security and Compliance 13. Data Visualization and Reporting

Question 46

Strategies for making big data more eco-friendly

Answer 46

1. Cloud Computing and Virtualization 2. Data Center Efficiency 3. Renewable Energy Sources to power big data operations 4. Consolidation and Data Sharing 5. Data Compression and Deduplication 6. Energy-Efficient Hardware 7. Server and Resource Optimization 8. Data Lifecycle Management 9. Smart Data Replication 10. Energy-Aware Algorithms 11. Data Caching and Preprocessing 12. Real-time Data Pruning 13. Data Center Location 14. Power Management Policies 15. Monitoring and Reporting

Question 47

The 12 Distributed computing architectures (Layers) for the structure of Big Data systems

Answer 47

1. Data Sources 2. Data Ingestion Layer 3. Storage Layer 4. Data Processing Layer 5. Batch Processing 6. Speed Processing 7. Serving Layer 8. Query and Analytics 9. Data Visualization and Reporting 10. Security and Governance 11. Monitoring and Management 12. Scaling and High Availability

Question 48

The Foundation of all Big Data Systems

Answer 48

Data is at the core of big data systems. The entire ecosystem is built around collecting, storing, processing, and analyzing vast amounts of data from various sources.

Question 49

5Vs of Data

Answer 49

The 5Vs are fundamental characteristics of big data. They define the key challenges and opportunities posed by big data systems. 1. Volume 2. Variety 3. Velocity 4. Veracity 5. Value These 5Vs collectively define the complexity and potential of big data and highlight the importance of efficient data management, processing, and analysis in modern data-driven environments. Big data systems are designed to address the challenges posed by these 5Vs while leveraging the opportunities they present for data-driven innovation and business growth.

Question 50

Data Volume

Answer 50

Volume refers to the vast amount of data generated and collected in big data environments. It involves dealing with massive datasets that may range from terabytes to petabytes or even exabytes in size. The ability to handle and process such large volumes of data is a defining characteristic of big data systems.

Question 51

Data Velocity

Answer 51

Velocity represents the speed at which data is generated, collected, and processed. In the era of real-time data streams, big data systems must be capable of ingesting and analyzing data as it is generated to provide timely and up-to-date insights. Velocity is crucial for time-sensitive applications and real-time analytics.

Question 52

Data Variety

Answer 52

Variety refers to the diversity of data types and sources in big data environments. Data can be structured, semi-structured, or unstructured, and it comes from various sources such as social media, sensors, log files, videos, images, and more. The challenge lies in managing and analyzing this diverse data effectively.

Question 53

Data Veracity

Answer 53

Veracity represents the quality, reliability, and trustworthiness of data. Big data often involves dealing with data from various sources, and the veracity of this data can vary. Ensuring data quality, cleaning, and validation are critical to making accurate and reliable decisions based on big data insights.

Question 54

Data Value

Answer 54

Value refers to the insights and meaningful information that can be extracted from big data. The ultimate goal of big data is to derive actionable insights that lead to valuable outcomes, such as data-driven decision-making, business optimization, and improved customer experiences.

Question 55

The 16 Basic Processing Paradigms

Answer 55

1. Batch Processing 2. Real Time Processing 3. Micro-Batch Processing 4. Stream Processing 5. Interactive Processing 6. Near-line Processing 7. Asynchronous Processing 8. Graph Processing 9. In-Memory Processing 10. Predictive Processing 11. Interactive Batch Processing 12. Adaptive Processing 13. Parallel Processing 14. Complex Event Processing (CEP) 15. Edge Processing 16. Machine Learning Inference

Question 56

Distributed Computing

Answer 56

A computing model in which a group of interconnected computers work together to solve a computational problem or perform a task. In this model, processing tasks are divided into smaller sub-tasks and distributed across multiple nodes (computers) within a network. Each node processes its assigned sub-task independently, and the results are combined to obtain the final result. The goal of distributed computing is to achieve higher performance, scalability, fault tolerance, and resource efficiency compared to traditional centralized computing models. In the context of big data, distributed computing plays a crucial role in handling the massive volume of data generated and processed in big data systems. Big data often exceeds the capacity of a single computer or server to handle, making distributed computing essential for managing and analyzing large datasets efficiently. The two main aspects of distributed computing relevant to big data are: 1. Data Storage 2. Data Processing The combination of distributed data storage and distributed data processing enables big data systems to address the challenges posed by the 5Vs of big data (Volume, Velocity, Variety, Veracity, and Value)

Question 57

Distributed Computing for Data Storage

Answer 57

Distributed storage systems, such as Hadoop Distributed File System (HDFS) and distributed NoSQL databases like Apache Cassandra and MongoDB, are used to store and manage vast amounts of data across multiple nodes. Data is partitioned and replicated across nodes to ensure data availability, fault tolerance, and data locality.

Question 58

Distributes Computing for Data Processing

Answer 58

Distributed computing frameworks like Apache Hadoop, Apache Spark, and Apache Flink are employed for distributed data processing. These frameworks divide data processing tasks into smaller chunks, and each node processes its portion of the data independently. The results are then combined to form the final output. This parallel processing allows big data systems to handle massive datasets efficiently.

Question 59

Horizontal Scalability v. Vertical Scalability

Answer 59

Horizontal scalability (aka Scale-out) -Ability of a system to handle increasing workloads and growing demands by adding more nodes or resources to the system. As data volume or processing requirements increase, new nodes can be added to the existing infrastructure, distributing the workload across the added nodes. This allows the system to maintain or improve performance, throughput, and responsiveness as the demand grows. Vertical scalability, (aka scale-up) -Involves increasing the capacity of individual nodes (e.g., adding more memory or CPU to a single server) to handle increased workloads. While vertical scalability can be effective up to a certain point, it eventually faces limitations due to hardware constraints, and it becomes more expensive and challenging to scale further.

Question 60

Benefits of Horizontal Scalability

Answer 60

1. Handling Massive Data Volumes 2. Performance and Throughput 3. Fault Tolerance/High Availability 4. Cost-Effectiveness 5. Flexibility and Elasticity 6. Future-Proofing

Question 61

Flexibility in Big Data

Answer 61

Flexibility refers to the system's ability to adapt and accommodate varying workloads, data sources, and processing requirements. A flexible big data system can handle different data types (structured, semi-structured, unstructured) from diverse sources, allowing for easy integration and processing of data with varying formats and characteristics. Key aspects of flexibility in big data include: 1. Data Variety: A flexible big data system can ingest, process, and analyze data in different formats, such as text, images, videos, log files, social media feeds, and sensor data. 2. Data Schema Evolution: A flexible system can accommodate changes in data schema over time without disrupting existing data processing pipelines or applications. 3. Support for Various Processing Paradigms: A flexible big data system can support batch processing, real-time processing, interactive processing, and streaming data processing, depending on the specific use case and requirements. 4. Data Exploration and Ad-Hoc Queries: Flexibility allows data scientists and analysts to explore data and perform ad-hoc queries efficiently, enabling data discovery and deeper insights.

Question 62

Elasticity in Big Data

Answer 62

Refers to the system's ability to automatically and dynamically scale its computing resources up or down in response to changing workloads or demands. An elastic big data system can add or remove resources (e.g., nodes or computing instances) on-the-fly to handle varying data processing needs, ensuring optimal performance and resource utilization. Key aspects of elasticity in big data include: 1. Automatic Scaling: An elastic system can automatically scale its resources based on predefined rules or thresholds. For example, when data processing demand increases, more nodes are added to distribute the workload. 2. Resource Optimization: An elastic system optimizes resource allocation to match the current workload, avoiding overprovisioning or underutilization of resources. 3. Cost Efficiency: Elasticity allows for cost optimization by scaling resources based on demand. During periods of low demand, unnecessary resources can be removed, reducing costs. 4. Resilience: Elasticity enhances system resilience by enabling it to adapt to sudden spikes in data processing demands without sacrificing performance or availability.

Question 63

Ad-Hoc Queries

Answer 63

Refer to on-the-fly, user-initiated queries that are performed in an exploratory and interactive manner to gain immediate insights and answers from large and diverse datasets. Ad-hoc queries are typically unplanned and ad hoc, meaning they are not predefined or part of a regular processing workflow. Instead, they are formulated based on specific analytical needs or questions raised by data analysts, scientists, or business users.

Question 64

Processing Paradigms

Answer 64

Approaches for handling data in big data computing systems. Processing paradigms are employed to analyze and extract insights from large and diverse datasets. Each processing paradigm has its characteristics and use cases.

Question 65

Extract, Transform, Load (ETL)

Answer 65

Extracting data from various sources, transforming it into a suitable format, and loading it into a data repository for analysis.

Question 66

Big Data Analytics

Answer 66

Analyzing and deriving insights from massive datasets to identify patterns, trends, and correlations. This includes tasks like data mining, predictive modeling, and machine learning on large datasets.

Question 67

Real-Time Data Processing

Answer 67

Processing and analyzing data as it arrives in real-time, such as streaming data from IoT devices, social media feeds, and financial markets.

Question 68

Large-Scale Data Warehousing

Answer 68

Storing and managing vast amounts of structured and unstructured data in a data warehouse for historical analysis and reporting.

Question 69

Search and Information Retrieval

Answer 69

Conducting searches and retrieval operations on large databases or search engines

Question 70

Recommendation Systems

Answer 70

Generating personalized recommendations for users based on their preferences and historical data

Question 71

Social Network Analysis

Answer 71

Analyzing relationships and connections between entities in social networks

Question 72

Log Analysis

Answer 72

Processing and analyzing log data from various systems to detect anomalies, monitor system health, and identify potential issues

Question 73

Genomic Data Analysis

Answer 73

Analyzing genetic data to study genetic variations, gene expression, and disease-related factors.

Question 74

Image and Video Processing

Answer 74

Analyzing and processing large volumes of image and video data, such as in computer vision applications and video surveillance.

Question 75

Natural Language Processing (NLP)

Answer 75

Analyzing and processing natural language text data for tasks like sentiment analysis, language translation, and information extraction.

Question 76

Financial Data Analysis

Answer 76

Analyzing financial transactions, market data, and economic indicators to support financial decision-making and risk analysis.

Question 77

Scientific Data Analysis

Answer 77

Analyzing data from scientific experiments, simulations, and observations in fields like astronomy, climate research, and bioinformatics.

Question 78

Geospatial Data Analysis

Answer 78

Analyzing geographic and spatial data for applications in geographic information systems (GIS) and location-based services

Question 79

In-memory computing

Answer 79

In-memory computing is a computing technique that involves storing and processing data directly in the main memory (RAM) of a computer, as opposed to storing data on disk or other slower storage devices. In this approach, data is accessed and manipulated at a much faster speed since RAM has significantly lower access times compared to disk storage. This allows for real-time data processing and analysis, making it ideal for handling data-intensive and time-sensitive tasks. In-memory computing is widely used in various domains and applications, including big data analytics, real-time data processing, financial trading systems, online transaction processing (OLTP) systems, and high-performance computing (HPC). It is often utilized in conjunction with distributed computing frameworks, enabling large-scale data processing and analytics with high performance and efficiency.

Question 80

The 5 V's of Big Data

Answer 80

Volume, Velocity, Variety, Veracity, and Value

Question 81

Relational Databases in Big Data

Answer 81

Organizes data into tables, where each table consists of rows and columns. The rows represent individual records or entries, and the columns represent attributes or characteristics of those records.

Question 82

Rows/Tuples in a datatable

Answer 82

Each row in a table represents a single record or data entry containing specific values for each column.

Question 83

Columns/Attributes in a datatable

Answer 83

Columns represent the specific characteristics or attributes of the data being stored in the table.

Question 84

Keys in datatables

Answer 84

Primary Key: A unique identifier for each row in a table, ensuring each row is distinct. Foreign Key: A field in a table that refers to the primary key of another table. It establishes relationships between tables.

Question 85

Database Normalization

Answer 85

The process of organizing data to minimize redundancy and dependency by dividing large tables into smaller tables and defining relationships between them.

Question 86

Structured Query Language (SQL)

Answer 86

The language used to interact with RDBMSs. SQL allows for the creation, manipulation, and retrieval of data from databases

Question 87

Candidate Keys for a data table

Answer 87

A subset of attributes that can be used to uniquely identify a record withing a table (since no two records will ever contain the same values for all those attributes) -These are called candidate keys because you will choose one of them to be your primary key *each table will likely have more than one Candidate Key

Question 88

Primary Key for a data table

Answer 88

One of the Candidate keys: the one specifically chosen to be the unique identifier

Question 89

Foreign Keys in RDBs

Answer 89

Used to enforce relationships between two tables. Each Foreign key corresponds to an existing primary key in another table in the RDBMS

Question 90

Relational Databases (RDBs)

Answer 90

A type of database that organizes and stores data in a structured format using rows and columns within tables. It is based on the principles of the relational model of data

Question 91

Security Vulnerabilities of a database

Answer 91

1) Aggregation: If the enemy collects enough non-sensitive data, they can sell it to someone who will know how to exploit it (can be prevented by zero trust/least privilege) 2) Inference: If the enemy obtains aggregated non-sensitive data, they can use it to draw conclusions that are actually sensitive (can be prevented by "blurring data" and "database partitioning") 3) Database ransomware: Attackers can lock you out of your database so that you cannot use the data anymore.

Question 92

Reasons to avoid putting too much data in one database

Answer 92

1) Performance Degradation: As the amount of data stored in a database grows, the performance can degrade. Retrieving, updating, or deleting data becomes slower, affecting response times for applications and end-users. 2) Scalability Issues: Large databases can become difficult to scale. Scaling vertically (increasing the resources of a single machine) has limits, and scaling horizontally (adding more servers) can be complex and costly, especially if the database was not designed for distributed architecture. 3) Increased Backup and Recovery Times: Backing up and recovering a large database takes more time and resources. The longer the recovery time, the higher the potential impact on business continuity and data availability during downtime. 4) Higher Maintenance Complexity: Managing and maintaining a large database is more complex. Tasks like indexing, optimizing queries, and performing maintenance activities (e.g., reorganization, updates) become time-consuming and resource-intensive. 5) Difficulty in Data Management: As the database grows, it becomes challenging to manage and organize the data efficiently. Data may become fragmented, leading to suboptimal data retrieval and storage processes. 6) Security Risks: Larger databases present a more attractive target for cyber threats. A breach can expose a substantial amount of sensitive data, increasing the potential impact on individuals and the organization. Proper security measures become even more critical. 7) Compliance Challenges: Compliance with regulatory requirements becomes more complex as the amount of data increases. Ensuring adherence to data privacy and protection laws, as well as auditing and tracking data access, becomes more challenging. 8) Resource Contention: A large database can consume a significant portion of system resources, causing contention with other applications or services running on the same server. This can lead to overall system instability and affect the performance of other applications. 9) Reduced Flexibility and Agility: It becomes harder to adapt and make changes to the database structure, schema, or applications when the database is excessively large. Changes may require significant planning and testing to avoid disruptions.

Question 93

Primary Storage in RDBMS

Answer 93

Represents the memory directly accessible by the processor (usually volitile RAM)

Question 94

Secondary Storage in RDBMS

Answer 94

Inexpensive non-volitile resources available for long-term use

Question 95

Virtual Memory

Answer 95

Allows a system to simulate additional primary memory through the use of secondary memory

Question 96

Volatile Storage

Answer 96

Loses its contents when the system powers down (RAM is the most common example)

Question 97

Nonvolatile Storage

Answer 97

Does not depend on the presence of power to maintain its contents (magnetic/optical media and non-volitile ram)