Big Data Solutions

Question 1

What is Cloud Pub/Sub?

Answer 1

A messaging event service that is fully managed and used for data pipelines

Question 2

Which services align with the Cloud Dataflow pipeline?

Answer 2

Cloud Dataflow is based on Apache Beam

Question 3

Which services align with the Cloud Dataproc pipeline?

Answer 3

Cloud Dtaproc is used for Apache Spark and Hadoop clusters

Question 4

What is Big Query?

Answer 4

BigQuery is a fully managed anaylitics service used to help analyze large amounts of data

Question 5

What language are queries executed in BigQuery?

Answer 5

SQL language

Question 6

What services can Cloud Pub/Sub integrate with?

Answer 6

Cloud Logs, Cloud API, Cloud Dataflow, Cloud Storage, and Compute Engine

Question 7

What is the primary difference between Cloud Dataflow and Cloud Dataproc?

Answer 7

You must provision your own servers in Cloud Dataproc

Question 8

What types of instances are available for Cloud Dataproc jobs?

Answer 8

Compute Engine instances, preemptible instances

Question 9

What is Cloud IoT Core?

Answer 9

Cloud IoT Core is a fully managed Google service that offers secure connections, management, and ingestion of data from IoT devices

Question 10

What type of Pub/Sub protocol does Cloud IoT Core use?

Answer 10

It typically uses MQTT Pub/Sub protocol more effectively than HTTP although it can use both

Question 11

What can you do with Cloud IoT Core?

Answer 11

You can register, configure, update, and control IoT devices

Question 12

How much data can be loaded into BigQuery?

Answer 12

BigQuery can be scaled to petabytes of data, although it must always contain at least 1 dataset

Question 13

What is a publisher?

Answer 13

An application that can create and send messages to a topic

Question 14

What is a topic?

Answer 14

A topic is a resource to which messages are sent by publishers

Question 15

What is a message?

Answer 15

Data a publisher will send to a topic (data in transit)

Question 16

What is a subscription?

Answer 16

Subscriptions are the stream of messages from a single topic to be delivered to the subscribing application

Question 17

What are common uses for Cloud Pub/Sub?

Answer 17

Distributed Even notifications, Balancing workloads, and Logging

Question 18

What is a data pipeline?

Answer 18

A pipeline is a piece of code that determines how we wish to process our data

Question 19

What is Cloud Dataflow typically used for?

Answer 19

Cloud Dataflow ingests data from Cloud Pub/Sub and transforms it into the data that we need to use as a part of the data pipeline

Question 20

What should you do if data being uploaded into BigQuery is only being used temporarily?

Answer 20

Data tables in BigQuery can be given a table expiriation in order to cut down on storage costs when the data set is being created

Question 21

How long must data be in BigQuery for it’s storage price to drop?

Answer 21

Data must be in BigQuery for 90 days unedited before it’s storage costs drop to 50%

Question 22

What is an advantage of using CLoud Dataproc clusters?

Answer 22

Clusters are only used for job’s lifetime and are therefore cost effective

Question 23

What types of machines are available in a Cloud Dataproc cluster?

Answer 23

Master notes, worker nodes, and preeptible worker nodes

Question 24

What type of cluster configuration in Cloud Dataproc has one master nodes and N worker nodes?

Answer 24

Standard; In flight jobs will fail and the file system will be inaccessible until the master node reboots if there is a compute failure

Question 25

What type of cluster configuration in Cloud Dataproc has includes 3 master nodes and N worker nodes?

Answer 25

High Availability; Designed to allow uninterrupted operations in the event of a compute engine failure

Question 26

What type of cluster configuration for Cloud Dataproc combines both master and worker nodes?

Answer 26

Single Node; Not suitable for large data processing and should be used for PoC or small scale non-critical data processesing

Question 27

What components from Apache Hadoop are auto installed onto a Cloud Dataproc ecosystem?

Answer 27

Apache Spark, Apache Hadoop, Apache Pig, Apache Hive, Python, Java, and the Hadoop Distributed File System

Question 28

What is MQTT?

Answer 28

MQTT is a Publish/Subscribe protocol that is often used with devices because it is data focused (often considered better for IoT jobs)

Question 29

What is HTTP in relation to Pub/Sub?

Answer 29

HTTP is a Publish/Subscribe protocol that is connectionless and can maintain a connection to the core. They are considered document focused.

Question 30

How do protocols communicate with Cloud IoT Core?

Answer 30

They communicate via a protocol bridge which provides MQTT and HTTP protocol endpoints, automatic load balancing and Global data access via Pub/Sub