SQL

Question 1

SQL is pronounced

Answer 1

Spark Sequel

Question 2

Spark SQL extends RDDs to a

Answer 2

“DataFrame” object

Question 3

DataFrames contain _____ objects

Answer 3

row

Question 4

DataFrames have a ____, which leads to more efficient storage

Answer 4

schema

Question 5

DataFrames can run _____ queries

Answer 5

SQL

Question 6

parquet is a

Answer 6

popular column data store object

Question 7

Spark SQL can read and write

Answer 7

Hive, JSON, parquet

Question 8

import

Answer 8

from pyspark.sql import SQLContext, Row

Question 9

To use SQL first thing you do is create a

Answer 9

Hive context

Question 10

create a Hive context

Answer 10

hiveContext = HiveContext(sc)

Question 11

get Hive data from JSON

Answer 11

inputData = hiveContext.jsonFile(dataFile)

Question 12

JSON is pronounced

Answer 12

Jay Sahn

Question 13

infer schema from inputData

Answer 13

inputData.registerTempTable(“myStructuredStuff”)

Question 14

run a query and make a DataFrame

Answer 14

myResultDataFrame = hiveContext.sql(‘”“‘SELECT foo FROM bar ORDER BY footer’””’)

Question 15

alternative to HiveContext

Answer 15

SQLContext

Question 16

Difference between HiveContext and SQLContext

Answer 16

Hive on top of SQL. Hive compatibility. Hive has heavier dependencies, but also a bit ahead of SQL at least in 1.5.