technology and tools

Question 1

what is Hadoop?

Answer 1

open source distributed computing framework

Question 2

what is Hadoop written in?

Answer 2

java

Question 3

what are the 4 main compensates of Hadoop?

Answer 3

map reduce
YARN
HDFS
Hadoop common

Question 4

what size blocks does a Hadoop distributed file system (hdfs) use?

Answer 4

128 mb blocks

Question 5

in the HDFS of Hadoop is failure normal?

Answer 5

yes as its highly fault tolerable

Question 6

what is the name node?

Answer 6

master server

Question 7

what does the name node do?

Answer 7

holds file system
undertakes file and directory operations
maps blocks to datanodes

Question 8

what is the data node?

Answer 8

a file split into more than one block

Question 9

what do data nodes do?

Answer 9

read and write requests

reports back to namenode

Question 10

what is bad about HDFS?

Answer 10

not good for small reads
not good for many small files
append not amend

Question 11

what is map reduce?

Answer 11

java based programming paradigm

Question 12

when to use map reduce?

Answer 12

problems that are embarrassingly parallel

Question 13

what does the Map from map reduce do?

Answer 13

Performs a map function on input key-value pairs to generate intermediate key-value pairs

Question 14

what does the reduce from map reduce do?

Answer 14

Performs a reduce function on intermediate key-value groups to generate output key-value pairs

Question 15

name a case where you would use map reduce?

Answer 15

data mining
spam detection
ad optimisation
index building in search engines 
article clustering for news
statistical machine translation

Question 16

What does YARN stand for?

Answer 16

Yet another resource negotiator

Question 17

What does yarn do?

Answer 17

Manages and monitors workloads

Question 18

What are the main features of yarn?
A shared
B fast
C scalability
D flexibility
E efficiency

Answer 18

A
C
D
E

Question 19

What is pig

Answer 19

Data flow language

Question 20

What is hive/hiveQL

Answer 20

SQL style query language

Question 21

What is hbase

Answer 21

Column-orientated database

Question 22

What is mahout

Answer 22

Machine learning library

Question 23

What is spark

Answer 23

In memory processing

Question 24

In Hadoop what are the data ingestion programs 
Flume
Hbase
Sqoop 
Storm

Answer 24

Flume
Sqoop
Storm

Question 25

In Hadoop what are the analytic and machine learning programs Spark Giraph Mahout

Answer 25

Giraph | Mahout

Question 26

``` What are the no sql programs on Hadoop Tez Hbase Cassandra Spark ```

Answer 26

Hbase | Cassandra

Question 27

In Hadoop what programs are the engines Spark Storm Tex

Answer 27

Spark | Tez

Question 28

What is zookeeper in hadoop

Answer 28

Cluster and workflow management

Question 29

What does hive do?

Answer 29

Coverts sql queries into java jobs

Question 30

What does hbase allow you to do?

Answer 30

Read/write operations on large datasets and works in real time

Question 31

What does spark do?

Answer 31

Analytic engine for large scale data processing

Question 32

What is different with sparks data sharing?

Answer 32

It’s in memory and not disk

Question 33

What is greenplum

Answer 33

Open source data platform

Question 34

What is postgresql

Answer 34

Rdbms with object oriented features

Question 35

What is MADlib

Answer 35

Open source library for in database analytics

Question 36

In greenplum what is the intersect operation

Answer 36

Rows from all answer sets

Question 37

In greenplum what is the except operation

Answer 37

Rows from first answer set minus rows from second

Question 38

In greenplum what is the union all operation

Answer 38

Rows from all answer sets with repeating rows

Question 39

In greenplum what is the union operation

Answer 39

Rows from all answer sets minus repeating rows

Question 40

In greenplum what is the group by operation

Answer 40

Group results based on one or more specified columns

Question 41

In greenplum what is the group by with union all operation

Answer 41

Add sub totals and grand totals

Question 42

In greenplum what is the roll up operation

Answer 42

Replaces union all

Question 43

In greenplum what is the cube operation

Answer 43

Creates sub totals of all possible combinations

Question 44

In greenplum what is the grouping function

Answer 44

Distinguishes NULL from summary markers

Question 45

In greenplum what is a window function.

Answer 45

Performs a calculation across a set of rows that are related to the current roe

Question 46

In greenplum and window functions what clause should you apply to specify which data window

Answer 46

OVER

Question 47

In greenplum window functions how would you define window partitions

Answer 47

PARTITION BY

Question 48

what does MAD stand for in MADlib?

Answer 48

magnetic agile deep

Question 49

what are the MADlib in-database analytical functions a) regression b) classification c) validation d) text analysis e) descriptive analytics f) clustering and top modelling g) association rule mining

Answer 49

a) regression b) classification c) validation e) descriptive analytics f) clustering and top modelling g) association rule mining

Question 50

what does MADlib do?

Answer 50

creates models without moving data out of DBMS