Big Data Lecture 02 Lessons Learnt

Question 1

Explain data independence

Answer 1

Logical model (interface) of the data (queries, and displaying) is independent from the physical storage (can be swapped).

Question 2

What 4 pieces constitute the architecture of data storage?

Answer 2

<ul><li>Language (how you query),</li><li>model (representation, driver of independence),</li><li>compute (execution of computation),</li><li>storage (physical hardware).</li></ul>

Question 3

What does the data model describe? (2)

Answer 3

<ul><li>What the data looks like,</li><li>what you can do with it (manipulation primitives).</li></ul>

Question 4

What is a table?

Answer 4

Collection of rows with different attributes.

Question 5

What is a row? 

Answer 5

One record in the table.

Question 6

What is an attribute?

Answer 6

One column in a table.

Question 7

What is a primary key?

Answer 7

Unique key that identifies the record in a table.

Question 8

What is a value?

Answer 8

One input in a row and a column of a table.

Question 9

What is relational algebra?

Answer 9

Algebra to express operations on a table.

Question 10

Relation table expressed formally in relational algebra?<br></br><br></br>What are its two components?

Answer 10

Each attribute has its domain, the relation is a subset of cross product of these domains, tuples of which we now put into a table.<br></br><br></br>Components:<br></br>1. set of attributes (schema),<br></br>2. set/bag/list of tuples.

Question 11

Explain: set, list, and bag.

Answer 11

<ul><li>Set: unordered collection without duplicates,</li><li>list: ordered collection, can have duplicates,</li><li>bag: unordered collection, with duplicates.</li></ul>

Question 12

How can tuple be seen as a function?

Answer 12

It assigns to each attribute of a table a value.

Question 13

What is relational integrity?

Answer 13

All the attributes must have a correct reference, meaning that the keys point to valid records in other tables.

Edit: I dont think this is true. The real answer should be that:
All records must have identical support. Eg there cannot be missing values.

Question 14

What is atomic integrity?

Answer 14

There are no tables in a table, every value is atomic.

Question 15

When is table 1st normal form?

Answer 15

Table must follow atomic integrity.

Question 16

What is domain integrity?

Answer 16

All the values must come from the same type, i.e. all are bools, or strings.

Question 17

What is NoSQL?

Answer 17

When we break all the given constraints, we get outside, that is what we study in Big Data!

Question 18

What is selection?

Answer 18

Selecting rows of a table.

Question 19

What is projection?

Answer 19

Selecting columns of a table.

Question 20

What is grouping?

Answer 20

Merging values of one table on the same attribute or condition.

Question 21

What is sorting?

Answer 21

Sorting a table based on some order.

Question 22

What is Cartesian product?

Answer 22

Taking product (each with each) of two tables.

Question 23

What is join?

Answer 23

Merging two tables on a common attribute.

Question 24

What are anomalies?

Answer 24

If some data is duplicated, but not properly linked, it might happen that on update/delete/insert there is anomaly.

Question 25

What is functional dependency?

Answer 25

When one attribute depends on another in a table, can be seen as fully function of the other.

Question 26

What is superkey?

Answer 26

Value or set of values, such that the values in the row depend on it.

Question 27

What is candidate key?

Answer 27

Candidate key is any minimal superkey, one of which we can pick to be the primary key.

Question 28

Define non-prime attribute.

Answer 28

Attribute that is not in any candidate key.

Question 29

When is table in 2nd normal form?

Answer 29

Proper subset of a candidate key cannot determine a non-prime attribute.

Question 30

When is table in a 3rd normal form?

Answer 30

Non-superkey cannot determine a non-prime attribute.

Question 31

When is table in 3.5th (Boyce-Codd) Normal Form?

Answer 31

Non-superkey cannot determine anything else.

Question 32

What is data denormalization?

Answer 32

Putting data from more advanced normal forms to the 1st normal form, to allow easy parallel querying.

Question 33

What is the difference between proto-imperative and functional/declarative language?

Answer 33

In proto-imperative language, we have to define everything explicitly, however, in functional/declarative language we just say what we want to happen and it happens.

Question 34

What kind of language is SQL?

Answer 34

Declarative, we declare what we want and it happens.

Functional, we can nest it like math.

Question 35

Give all SQL clauses and explain them.

Answer 35

SELECT column_name1, column_name2
FROM table_name
WHERE condition
GROUP BY attribute
HAVING condition
ORDER BY attribute and direction
LIMIT number_to_display
OFFSET number_to_skip

Question 36

What set operations can we use in SQL?

Answer 36

All queries are sets, so we can UNION, UNION ALL (with duplicates), MINUS or INTERSECT them.

Question 37

What is the difference of theta join, full outer join, right and left join, and natural join?

Answer 37

• Theta join only matches on a selected attribute or condition,
• right and left joins, join one table onto the other on matching records and fill in the rest using NULL,
• full outer join does both right and left join,
• natural join joins on matching attribute names.

Question 38

Explain ACID and all its characteristics.

Answer 38

Good old day of databases gave us a lot of guarantees on transactions, this is not the case in Big Data anymore:

• atomicity: either everything or nothing is executed,
• consistency: everytime you update, all the data will be consistent,
• isolation: more people are using the database, but if feels like you are the only one,
• durability: updates that are carried out are persistent.

Question 39

How can Big Data become big? What do we have more of?

Answer 39

We can have a lot of

• rows,
• columns,
• nesting.