U5 Flashcards
(421 cards)
1
Q
As its name suggests, “big data” is huge and fast-growing data. Big data initially:
With the dramatic increase in data, database costs, e.g., hardware, software, and operating costs, have accordingly increased. Hence,
A
attributed to search engines and social networks is now making its way into enterprises.
2
Q
There exist several challenges while working with big data, including?
A
how to store it and how to process it.
3
Q
Among these challenges is enabling the databases to meet the needs of?
A
high concurrent reading and writing with low latency.
4
Q
there is an immense need to lower the costs of storing big data?
A
Because, with the dramatic increase in data, database costs,
e.g., hardware,
software,
and operating costs,
have accordingly increased.
5
Q
The traditional relational databases, e.g., structured query language (SQL), are a?
A
collection of data items with pre-defined relationships between them.
6
Q
These items are organized as a set of tables with:
A
columns and rows.
7
Q
Unfortunately, these relational databases have some inherent limitations which emerge with:
A
the rapid growth of data.
8
Q
In these cases, relational databases are:
A
widely prone to deadlocks and other concurrency issues.
9
Q
These situations lead to rapid declines in?
A
the efficiency of reading and writing.
10
Q
Furthermore, the multi-table correlation mechanism that exists in —————————————-represents a major limitation of database scalability. To overcome these problems, ——————–databases were proposed instead of the traditional database. NoSQL is an —————–term for ——————- databases which do not use the SQL structure.
A
relational database
NoSQL
umbrella
non-relational
11
Q
NoSQL databases are useful for ?
A
applications that deal with very large semi-structured and unstructured data.
12
Q
Unlike relational databases, NoSQL databases are designed to ?
A
scale horizontally and can be hosted on a cluster of processors.
13
Q
In most of these databases, each row is a ?
A
key-value pair.
14
Q
NoSQL databases contains truly elastic databases, e.g., MongoDB and Cassandra, which allows?
A
the addition/removal of nodes to/from a cluster without any observable down-time for the clients.
15
Q
To this end, routing algorithms are used to decide when to move the inter-related data chunks, for instance, ?
A
when data must be moved to newly added node B. During the copying process, the data is served from the original node A. When the new node B has an up-to-date version of the data, the routing processes start to send requests to the node B.
16
Q
In general, there are some important aspects related to distributed databases that need to be thoroughly addressed, including :
A
scalability,
availability,
and consistency.
17
Q
First, scaling is typically achieved through?
A
“sharding” to meet the data volume.
18
Q
Sharding is ?
A
a type of database partitioning that separates very large databases into smaller, faster, more easily managed parts, referred to as data shards.
19
Q
NoSQL databases support an auto-sharding mode in which?
A
the shards are automatically balanced across the nodes on a cluster.
20
Q
Additional nodes can be easily added as ?
A
necessary to the cluster to align with data volume.
21
Q
Second, availability can be achieved via replication, i.e., ?
A
master-slave replication or peer-to-peer replication.
22
Q
With master-slave replication, two types of nodes are typically implemented including:
A
a master node where all the write operations go to the master node.
23
Q
Data can be read from any node, either a —————————– If a master node goes down, a slave node gets promoted to a ————–, and continues to replicate to the —————.
A
master or a slave.
master node
third node
24
Q
When a failed master node is resurrected, it joins the cluster as a slave. Alternatively,?
A
peer-to-peer replication is slightly complex where all the nodes receive read/write requests.
25
In terms of consistency, two major types of inconsistencies exist:
read and write.
26
Read inconsistencies arise in ------------------------- replication when a user tries to read of a ----------------- before changes propagate from the -----------------------, while in ---------------------------- replication the user runs into both read and write inconsistencies, as write (update) is allowed on ----------------s.
master/slave
slave
master node
peer-to-peer
multiple node
27
It is obvious that availability and consistency are ?
two contradicting metrics.
28
Achieving the right balance between these metrics highly depends on ?
the nature of the IoT application.
29
For example,
a user can prohibit read and write inconsistencies through considering slaves as hot standby without reading from them.
30
MongoDB is a prominent example of a document-oriented, scalable NoSQL database system which has?
a powerful query language.
31
MongoDB supports complex data types, e.g.,?
BJSON data structures.
32
It allows most functions like?
-query in single-table of relational databases,
-and it also supports indexing.
-Furthermore, MongoDB has the advantage of supporting high-speed access to mass data.
33
When the stored data exceeds 50 GB, the access speed of MongoDB is?
ten times higher than MySQL (Yan, 2015).
34
Thanks to these characteristics, many system designers are?
considering MongoDB instead of relational database.
35
Another example of a NoSQL database is?
the Apache Cassandra. It offers good scalability and high availability without compromising performance.
36
Cassandra demonstrated?
fault-tolerance on commodity hardware (i.e., cloud infrastructures)
and linear scalability,
thus making it the ideal platform for mission-critical data.
37
Cassandra features allow?
replication across multiple datacenters,
offering lower latency for data availability during regional outages.
38
With Cassandra, columns can be easily indexed with ?
powerful built-in caching mechanism.
39
Netflix, Twitter, Urban Airship, Reddit, Cisco, OpenX, and Digg are examples of the companies that use?
Cassandra to deal with huge, active, online interactive datasets.
40
The largest known Cassandra cluster has over?
300 TB (terabytes) of information in over 400 machines.
41
Processing a massive amount of data, i.e., big data, demands?
a shift from the client server model of data processing where a client node pulls the data from a server node.
42
Instead, data can be processed on the -------------------------------------. In addition, this processing can be carried out independently in parallel as the underlying data is already --------------------------across different nodes.
cluster nodes
partitioned
43
This approach of data processing is referred to as ?
the MapReduce framework and it also interestingly uses key-value pairs.
44
MapReduce makes use of hundreds or even thousands of “pluggable” nodes in a cluster to?
process data in parallel, which significantly shortens the time between the operational events and presenting the analytics results.
45
The -------------------------------- framework offers an effective method for the efficient analysis of the collected -------------data, especially when the computations involve linearly computable -----------functions over the elements of the data streams, e.g.,
-------------------------------------------------------.
MapReduce
sensor
statistical
MIN, MAX, SUM, and MEAN
46
Google’s original MapReduce framework was designed for ?
Therefore, this framework represents an ideal candidate for sensor data analytics. The figure below demonstrates the MapReduce architecture for processing sensor data in parallel on different processing nodes.
analyzing large amounts of web logs, and more specifically deriving such linearly computable statistics from the logs.
47
In fact, the sensor-generated data has many conceptual similarities to web logs. Specifically,?
they are similarly repetitive, and the typical statistical computations which are often performed on sensor data for many applications are linear in nature.
48
this framework represents an ideal candidate for sensor data analytics?
the sensor-generated data are often performed on sensor data for many applications are linear in nature.
49
The figure below demonstrates the MapReduce architecture for processing sensor data in parallel on different processing nodes.
50
To understand the MapReduce framework, consider a case where?
the data can be in the form of (year, value) where the year is the key. The Map function takes a list of pairs (year, value) from one domain and then returns a list of pairs (year, local max value). The local max value denotes the local maximum in the subset of the data processed by that node. This computation is typically performed in parallel by dividing the key value pairs across different distributed computers.
the maximum temperature each year is to be determined from sensor data recorded over a long period of time.
To this end, the “Map” and “Reduce” functions of MapReduce are defined with respect to data structured in (key, value) pairs.
51
For example,
the data can be in the form of (year, value) where the year is the key.
The Map function takes a list of pairs (year, value) from one domain and then returns a list of pairs (year, local max value).
The local max value denotes the local maximum in the subset of the data processed by that node.
This computation is typically performed in parallel by dividing the key value pairs across different distributed computers.
52
Subsequently, the MapReduce framework combines ?
all pairs with the same key from all lists, thus creating one group for each one of the different generated keys.
53
This grouping step requires -------------------between the different -----------------------. However, the cost of this communication is much lower than moving the------------------------------- around because the ----------- has already generated a ---------------- summary of the processed data.
communication
computers
original data
Map step
compact
54
It is worth mentioning that the exact implementation of the Map step widely depends upon?
the implementation of the adopted MapReduce,
and also on the exact nature of the distributed data.
55
For instance, the sensor data may be distributed over a local cluster of computers (with the use of an implementation such as Hadoop). An alternative solution is to geographically distribute the sensor data because?
the data is originally created at different locations, and it is too expensive to move the data around.
56
The latter scenario is much more suited for ------ applications. Nevertheless, the steps for collecting the intermediate results from the different Map steps may depend upon the specific ----------------------------------------------- in which the MapReduce framework is -----------------.
IoT
implementation and scenario
utilized
57
After performing the grouping step, the Reduce function is applied in?
parallel to each group.
58
Such a step generates a collection of?
values in the same domain.
59
Next, we apply Reduce ---------------- in order to create list------------.
(k2, list(V2))
(v3)
60
Each execution of the Reduce function returns only one value, although it is also?
possible for the function to return more than one value.
61
For instance,
the input to the “Reduce” function will be a list in the form (Year [local max1, local max2, …, local maxr]),
where the local maximum values are determined by the execution of the different Map functions.
The Reduce function determines the maximum value over the corresponding list in each call of the Reduce function.
62
A Hadoop client typically submits jobs to the MapReduce framework through what is called?
the “jobtracker” running on the master server.
63
Subsequently, the --------------------------- automatically assigns the jobs to --------------------------- running on many -----------------------------.
jobtracker
“tasktrackers”
slave nodes
64
The tasktrackers regularly send heartbeats to the jobtracker to update the status, e.g.,?
alive, idle or busy.
65
If a job fails or timeouts, or a node is dead, the jobtracker can automatically reschedule ?
the jobs to run on available nodes.
66
In general, HDFS comprises two components, namely ?
name-nodes
and data-nodes.
67
A name-node is?
responsible for keeping the metadata about the data on each data-node.
68
When a client application reads or writes data into HDFS, it must communicate with?
the name-node to get the locations of data block to be read from or written to.
69
The metadata is read into main memory when ?
Hadoop starts, and is dynamically maintained.
70
A data-node updates name-node the metadata of its local data blocks through?
heart beats.
71
Hadoop also has?
a secondary name-node mainly used to store the latest checkpoints of HDFS states.
72
Although the Hadoop MapReduce framework has the goal of
high scalability
and better fault-tolerance,
it is not ?
optimized for input/output efficiency. Specifically, both the Map and Reduce functions are “block operations” in which data transition cannot proceed to the next stage until the tasks of the current stage have finished.
73
Accordingly, the output of mappers needs to be ?
first written into HDFS before shuffled to the reducers.
74
Such blocking, the one-to-one shuffling strategy, and the runtime scheduling ?
degrade the performance of each node.
75
The MapReduce framework lacks:
a database management system
and does not optimize data transferring across various nodes.
76
it is more suitable for batch jobs than real-time processing.
Because Hadoop has a latency problem due to its inherent nature.
77
Large-scale IoT applications,
e.g.,traffic monitoring,
weather forecasting,
homeland security,
entertainment,
and disaster response, often have ?
the challenge of capturing too much data with too little inter-operability.
78
Such challenges are accompanied with ?
too little knowledge about the ability to utilize different resources which are available in real time.
79
To overcome these challenges, the Sensor Web Enablement initiative ?
defines service interfaces which enable developers to make all types of sensors, transducers, and sensor data repositories discoverable, accessible, and usable via the Web.
80
Such standardized interfaces are extremely beneficial since?
they hide the heterogeneity of the underlying IoT devices from the applications that use them.
81
In this context, the term “Sensor Web” defines ?
an infrastructure enabling access to IoT devices and archived sensor data.
82
Such data can readily be discovered and accessed using -----------------------------------------------. The goal of the Sensor Web is to enable real-time “---------------------” in order to ensure timely --------------------- to a wide variety of events.
standard protocols and APIs
situation awareness
responses
83
The major benfits of the IoT sensor data can only be realized if we have?
the infrastructure and mechanisms to synthesize, interpret, and apply this data intelligently via automated means.
84
The Sensor Web enables automated applications to
understand,
interpret,
and reason with basic but critical semantic notions such as ?
“nearby,” “far,” “soon,” “immediately,” “dangerously high,” “safe,” “blocked,” or “smooth.” Ontologies are at the heart of the semantic sensor web technology.
85
An ontology is a mechanism for ?
resources, the Resource Description Framework (RDF) data model is widely used to describe resources.
knowledge sharing and reuse.
86
Ontologies are generally knowledge representation systems. To represent ?
resources, the Resource Description Framework (RDF) data model is widely used to describe resources.
87
Literally, a resource is ?
any device or concept, e.g., person, place, restaurant. Each resource is uniquely identified by a URI.
88
Aside from describing resources, RDFs are capable of ?
specifying how resources are inter-related through performing inference.
89
The building blocks of RDF are triples, where a triple is?
a 3-tuple of the form where subject, predicate, and object are interpreted as in a natural language sentence.
90
It is most helpful to perceive RDF as a
-----------, where subject resources are represented in ---------, literals in ------------, and predicate (relationships) represented as directed ------------------------- or between ---------------------------------.
graph
ovals
rectangles
edges between ovals
ovals and rectangles
91
For instance,
the triple representation of the sentence, “Washington, D.C. is the capital of the United States,” is illustrated in the following figure.
92
The Web Ontology Language (OWL) is ?
another ontology formalism that was developed to overcome the drawbacks of RDF.
Specifically, RDFs do not provide ways to represent constraints,
e.g., domain or range constraints.
93
Furthermore, ------------------------------------------------------------- cannot be represented in RDF data model.
transitive or inverse properties
94
Extending RDF(s) makes it straightforward to provide?
a formal specification in OWL.
95
Both RDF and OWL ontology formats have extensive developer community support in terms of?
the availability of tools for ontology creation and authoring.
96
An example is ?
Protege, which supports RDF and OWL formats, data storage, and management stores, such as
OpenSesame, for efficient storage and querying of data in RDF or OWL formats.
97
Furthermore, there is significant availability of actual ontologies in a variety of domains in the ---------------------------- formats.
RDF and OWL
98
The Semantic Sensor Network (SSN) is an example of ?
an ontology which relies on the OWL data model to describe sensors and observations.
99
It describes the IoT sensors in terms of ?
their capabilities,
measurement processes,
observations,
and deployments.
100
The SSN ontology is conceptually organized into -----------------------. In fact, the ontology can be seen from ---------- main perspectives, namely ?
ten modules
four
sensor perspective,
observation perspective,
system perspective,
and feature and property perspective.
101
The full ontology consists of?
41 concepts and 39 object properties.
102
The ontology can describe?
sensors,
the accuracy and capabilities of such sensors,
observations,
and methods used for sensing.
103
Concepts for operating and survival ranges are also included, as?
these are often part of a given specification for a sensor, along with its performance within those ranges.
104
Finally, a structure for field deployments is included to?
describe deployment lifetimes and sensing purposes of the deployed macro instrument.
105
To achieve automatic processing and interpretation of the IoT data, we need ?
common agreements on providing and describing the IoT data.
106
To evaluate the quality aspects of data, the source provider,
device,
and environment-specific information also need to be?
associated to the data.
107
Considering the diversity of data types, device types, and potential providers in the IoT domain, common description frameworks are essential to ?
describe
and represent the data to make it seamlessly accessible
and processable across heterogeneous platforms.
108
Considering the diversity of data types, device types, and potential providers in the IoT domain, common description frameworks are essential to ?
describe
and represent the data to make it seamlessly accessible
and processable across heterogeneous platforms.
109
The semantic descriptions and annotations must be provided at different layers of the IoT framework, including:
the “Things” level, device,
and network level (e.g., SSN ontology), and the interaction and business process
model to?
enable autonomous processing and interpretation of the IoT data.
110
In fact, the effective discovery, access, and utilization of the IoT resource require?
machine-interpretable descriptions of different components and resources in the IoT framework,
e.g., sensors, actuators, and network resources.
111
The current Semantic Web technologies and ontologies can efficiently describe various aspects of?
the IoT data and resources.
112
Description models and representation frameworks that can describe the IoT data and services need to consider the constraints and dynamicity of the IoT domain.
Since IoT environments are often dynamic and pervasive,
113
In this context, the concept of “linked data” emerges to connect?
individual data items to support semantic query and inferences on the data coming from the physical and virtual objects.
In other words,
linked data simply refers to data published on the Web in such a way that it is machine-readable, its meaning is explicitly defined, and it is readily linked to other external data sets.
114
The linked data, represented using formal knowledge representation ---------------------such as ------------------------------, provides potential for information reuse and ----------among ----------------------sources. The information published for the linked data is typically structured and connected to one another.
formalism
RDF and OWL
interoperability
heterogeneous
115
In general, publishing linked data widely encourages the reuse?
(1) using URIs as names for data; (2) providing HTTP access to those URIs; (3) providing useful information for URIs using the standards such as RDF and SPARQL; and finally (4) including links to other URIs.
of existing information rather than creating new information.
116
This implies that human users can exploit the existing knowledge base by?
simply providing links to the data in it.
117
For instance,
the DBpedia project
-extracts structured information from Wikipedia.
-DBpedia enables sophisticated queries over the information that exists in Wikipedia.
- Moreover, it provides new ways of browsing and navigation through the semantic links.
118
Nevertheless, semantic descriptions without being linked to other existing data on the Web would be mostly?
processed locally and according to the domain descriptions (i.e., ontologies) and their properties.
119
The linked data offers four main principles to publish linked-data:
(1) using URIs as names for data;
(2) providing HTTP access to those URIs; (3) providing useful information for URIs using the standards such as RDF and SPARQL; and finally
(4) including links to other URIs.
120
In fact, the emergence of sensor data as linked data enables IoT applications and sensor network providers to?
connect sensor descriptions to potentially endless data existing on the Web.
Specifically, the action of relating sensor data attributes, such as location, type, and measurement features, to the other resources on the Web of data enables the users to integrate physical world data and the logical world data.
121
The results of such an integration are:
drawing beneficial conclusions,
creating business intelligence,
enabling smart environments,
and supporting automated decision-making systems.
122
In order to get the most out of the integration of IoT and cloud computing, the use of ?
microservices is recommended.
123
Microservices represent an architectural approach for developing applications as a set of small services, where?
each service is running as a separate process, communicating through simple mechanisms .
124
Most of the advantages of the microservices architecture stem from ?
decomposing a service or an application into smaller components, i.e., microservices.
125
Each of these components should implement a specific functionality. As a result, :
we can independently develop,
deploy,
upgrade,
and scale every microservice.
126
each microservice can be separately scaled?
Since the different microservices may have different workloads.
127
Accordingly, we can?
-use an optimal amount of resources making microservices architecture a natural fit for achieving both scalability and elasticity.
-We can separately control every microservice where they are easily manageable thanks to being small.
128
Developing microservices separately enables?
the employment of different technologies, e.g., different programming languages for each microservice
129
Furthermore, the task of releasing an update for a part of our application or service does not require?
the redeployment of the whole application,
but only
the corresponding microservice.
130
Microservices often communicate through web services, such as:
REST,
or through remote procedure calls (RPC).
131
there is a need to reduce the communication between the different microservices to the minimum. ?
as communication between processes may become costly.
132
The advantages of microservices architecture are best identified when?
we compare it to the traditional monolithic architecture.
133
Monolithic application has all of its components packed together.
For instance,
monolithic web applications have the client-side,
the server-side
, and the database in a single logical executable.
Similarly, monolithic IoT applications have the whole logic for communication with IoT devices,
processing of devices’ data, communication with databases,
and visualization, in a single logical executable.
134
To achieve scalability and elasticity in monolithic applications, more instances of the whole application must be?
deployed or terminated. However, different application functionalities rarely have an equal share of the workload.
135
Alternatively, every microservice is packed as?
an independent component in the microservices approach.
136
We can scale every ----------------------------independently and change the number of instances for each microservice separately. In this context, the application -----------------------can be controlled according to the workload of each of the microservices. To summarize, microservices ------------------------------------------------------ application development.
microservice
scalability
enable a scalable
elastic,
and resource-efficient
137
Another major difference between microservices and monolithic architecture is that?
a change in how an application communicates with IoT devices and receives data from them must have no impact or only minimal impact on how we process the data. Therefore, we should “componentize” the application into microservices in a way that would allow for the communication between microservices to be minimal.
the latter usually runs as a single process.
138
If we want to release an update of the application, the whole application must be?
redeployed for the changes to take effect;
139
it does not matter which component we have changed. With microservices, the update of one microservice has to cause?
no changes or only minor changes to the other microservices.
140
In this realm, we can highlight a potential challenge of the microservices approach. As mentioned previously, the communication between components is?
relatively expensive and has to be deliberately minimized.
141
If a change in a single microservice imposes many changes in other microservices, ?
the advantages of the microservices architecture might be lost.
142
For instance,
we should “componentize” the application into microservices in a way that would allow for the communication between microservices to be minimal?
Because a change in how an application communicates with IoT devices and receives data from them must have no impact or only minimal impact on how we process the data.
143
In some applications, the monolithic architecture could become excessively large. In these cases, several drawbacks emerge, such as:
if a microservice which communicates with a certain group of sensors crashes, such a crash will not affect or stop the processing of the data provided by microservices which communicate with other sensors. The other components of the application will still be up and properly running.
the difficulty of software management, being more vulnerable, and being harder to update.
144
Bugs in monolithic applications could be expensive,?
as they cause the whole application to crash, whereas in microservices architecture only the corresponding microservice collapse.
145
Bugs in monolithic applications could be expensive, as they cause the whole application to crash, whereas in microservices architecture only the corresponding microservice collapse. In this case, the microservices-based application can continue?
running and only the specific functionality implemented by the malfunctioned microservice is unavailable.
146
Such behavior of ----------------------------------------------is highly important in the IoT domain. For example, ?
microservices-based architectures
if a microservice which communicates with a certain group of sensors crashes,
such a crash will not affect or stop the processing of the data provided by microservices
which communicate with other sensors.
The other components of the application will still be up and properly running.
147
In general, IoT applications have high requirements regarding?
scalability.
148
These scalability requirements fundamentally push toward?
designing distributed architectures rather than monolithic ones.
149
In general, the microservices architecture is ?
adaptable to the requirements of IoT applications.
150
When developing applications, it is generally good practice to break down the application into several --------------------. Such components that programmers frequently use are referred to as-------------.
components
libraries
151
The concept of services in microservices architecture is similar to the libraries concept with one major difference:
libraries are essentially linked to a main program and when the program is running, there is only one process.
On the other hand, the microservices architecture tends to componentize a project into services, where each service is running in its own separate process.
152
each microservice could be deployed and scaled independently?
As the microservices architecture tends to componentize a project into services, where each service is running in its own separate process.
153
each microservice could be deployed and scaled independently. By ?
componentization into microservices, the problem of vast heterogeneity of devices could be simply addressed.
154
To this end, distinctive microservices can be implemented as?.
proxies for the IoT devices that communicate using different protocols, e.g., Wi-Fi, LoRa, BLE
155
Furthermore, adding new devices, which may communicate using unsupported protocol, is usually resolved by?
adding a microservice acting as a proxy between protocols.
156
In general, there exist two common approaches for decomposition of applications into microservices:
verb-based
and noun-based strategies.
157
The former strategy deals with the --------of an application around single use cases. Such a decomposition strategy is ------------- for ------ applications.
decomposition
ill-suited
IoT
158
If we are dealing with multiple groups of devices, we might group the logic for communication with certain type of devices, e.g.,?
temperature sensors, the data processing logic, and the visualization logic for this certain group of sensors in one microservice.
159
In fact, the approach is not a natural fit for IoT applications, ?
as the scaling of different modules is dependent upon different factors.
160
For instance:
The communication with devices is most dependent upon the number of devices and the amount of data they generate, while the visualization application must consider the number of users which access it simultaneously.
161
In the noun-based decomposition, a microservice is responsible for?
every operation related to a certain functionality.
162
A single microservice communicates with ? data visualization.
the devices and exchanges data with them
163
a second microservice processes the data, e.g., ?
CEP engine;
164
a third microservice might store the data in ?
a database for later processing;
165
and finally, a fourth microservice might be responsible for?
data visualization.
166
Such a decomposition leads to the design of a dynamic application, where ?
each functionality can be separately scaled.
167
A combination of the verb-based and noun-based approach is also ?
possible.
168
fault tolerance can be easily considered?
Since microservices are independent components.
169
The microservices architecture brings more freedom for?
deployment and management of applications in cloud infrastructures.
170
For a scalable IoT application, every microservice must use?
an optimal amount of resources.
171
To this end, microservices typically employ the platform as ?
a service (PaaS) cloud computing mode.
172
PaaS offers a platform, which is responsible for?
low-level operations like management of virtual machines (VMs), application deployment, load balancing, etc.
173
PaaS enables :
easier management of applications
and is especially suitable for microservices.
It enables system designers to focus on the IoT aspect and not on the essential low-level part of the application.
174
On the contrary, it is naturally hardly to find a PaaS provider that supports ?
monolithic applications.
175
In case of a monolithic application, system designers need to?
perform the job of configuration and management of the application.
176
The first scaling strategy of microservices is referred to a---------------------------In vertical scaling, more resources, e.g., ----------------------------units, are added to a single specific IoT node to incorporate additional---------------- so that it can successfully handle the application requirements.
“vertical scaling.”
memory and CPU
capabilities
177
Nevertheless, such a scaling strategy is relatively limited in?
its nature, as it is impractical to continuously add resources due to some physical constraints.
178
the vertical approach is not common in the IoT domain, unless it is combined with other horizontal scaling approaches.?
Because IoT applications often deal with a huge number of events and thus require lots of resources, which a single node often cannot provide.
179
The following scaling approaches are the three dimensions of a scale cube:
X-axis scaling
Y-axis scaling
Z-axis scaling
180
X-axis ?
X-axis scaling is a typical horizontal scaling in which several application instances are used to distribute the workload evenly.
It is common for monolithic applications to use this kind of scaling.
181
Having N instances of an application, each instance gets?
1/N of the workload.
182
Alternatively, each microservice is ?
separately scaled
183
PaaS environments further facilitate the scaling of microservices, as they allow us to ?
specify the number of instances for a specific microservice.
Some PaaS providers also offer automatic scaling, i.e., microservices are dynamically scaled depending on the current workload.
184
Similarly, Y-axis scaling splits ?
the application into multiple, different services.
Each service is responsible for one or more closely related functions.
185
In practice, a combination of X-axis and Y-axis scaling is commonly used for ?
scaling of IoT applications.
186
In the third class of the horizontal scaling strategies, referred to as the Z-axis scaling, each server runs?
an identical copy of the code.
187
The big difference between Z-axis and X-axis is that ?
each server is responsible for only a subset of the data.
Some component of the system is responsible for routing each request to the appropriate server.
A common use case where Z-axis scaling is applicable is division according to the user category.
188
For example, ?
requests of users who have paid for a service are routed to different, more powerful servers.
189
Premium users requests are not routed together with the requests of non-premium users, as higher performance must be ensured?
since Premium users usually have higher service level agreements (SLA) than non-premium users.
190
Better performance is usually provided by?
less restrictive policy on the creation of new instances or redirection to more powerful instances.
191
Premium users might also have access to additional services. For example, ?
we might process and visualize data from additional sensors, available to premium users only.
Therefore, they will be redirected to servers which are hosting these additional services.
192
A component for routing might be implemented as an additional microservice.
In fact, Z-axis scaling has many benefits?
Since each server only deals with a subset of the data, the cache utilization is improved while reducing the memory usage and the input/output traffic.
193
In addition, Z-axis scaling improves fault isolation since?
a failure only makes part of the data in accessible.
194
Unfortunately, Z-axis scaling has some drawbacks. One major drawback is ?
the increased application complexity.
It is necessary to implement a partitioning scheme, which can be tricky especially if we ever need to repartition the data.
195
Complex event processing (CEP) is?
an approach that identifies data and application traffic as events, correlates these events to reveal predefined patterns, and reacts to them by generating actions to systems, people, and devices.
196
CEP is based on?
the observation that actions are mostly triggered not by a single event, but by a complex composition of events, happening at different times, and within different contexts.
197
It can provide companies with ?
the ability to define,
manage,
and predict events,
situations,
conditions,
opportunities,
and threats.
198
CEP is as an event-driven processing engine because ?
the computation is triggered by the receipt of event data.
199
the computation is triggered by the receipt of event data. The events being analyzed can be happening across different parts of an organization as sales?
leads, orders, or customer service calls.
200
These data types can include:
news items, text messages, social media posts, stock market feeds, traffic reports, weather reports, or other kinds of data.
201
An event may also be defined as?
a change of state, when a measurement exceeds a predefined threshold of time, temperature, or other value—that is really where IoT comes in.
202
The challenge of real-time analysis continues to grow as ?
more IoT devices are deployed to collect more data.
203
Being able to react quickly in a mission-critical situation can?
save companies millions of dollars and is one of the pillars of IoT functionality.
204
Being able to react quickly in a mission-critical situation can. save companies millions of dollars and is one of the pillars of IoT functionality. This is why CEP is becoming ?
a more mainstream solution for IoT deployments.
205
In a large-scale IoT application, event processing applications must process events that arrive from ?
various kinds of sources such as sensors which constitute a WSN, RFID readers, GPS, social media, etc.
206
For instance,
each reading operation of the RFID reader at a garage generates a simple event but a complex event like “a car leaves the garage” is the kind of event that a user is really concerned with.
To get such a complex event, we need to combine many simple events based on some rule.
207
Many events, generated by IoT devices, are used as?
an input to a CEP engine.
208
The CEP engine correlates the complex events with already-stored pattern/rules. Once a match between the complex events and the patterns is found?
an action is generated to react to the complex events.
209
In general, the CEP engine can be either ----------------------------------------------. The centralized CEP architecture requires greater --------------------------------------capability as the number of clients increases, and the system is hardly robust and scalable because of-------------------------------------- .
centralized or distributed
bandwidth and computational
a single point failure or network break.
210
Distributed CEP architecture becomes more common in the IoT domain. In fact, employing CEP in the IoT domain poses special requirements on CEP.
As many IoT applications are naturally distributed and hence complex events are to be detected from the distributed system.
211
To optimize various processes in IoT applications, such as logistics, smart grid, traffic management, it is crucial to ?
react to the occurrence of critical situations with low latency.
212
Examples of such situations can be?
accidents, load variations in power consumption, or changing weather conditions.
213
The monitoring of situations of interest is enhanced by?
a steadily growing infrastructure of globally deployed sensors.
214
By continuously capturing and analyzing sensor streams, CEP systems can help to?
detect the relevant situations.
215
To react on complex events, CEP engines trigger decisions such as:
updating a traffic route
or changing the configuration in power consumption.
216
In this context, the utility of such decisions widely depends on
(1) the time at which events are delivered by the CEP system,
and (2) the view of the application on the set of delivered events at decision time.
217
This view should be consistent, i.e., at time of decision all relevant events are present (-------------------------------------------) and no false events (------------------------------------) are delivered.
no false-negatives
no false-positives
218
For example,
consider a scenario where it is required to establish an overtaking ban on a certain region of a highway.
To this end, two camera-based sensors are deployed at the beginning and end of the no-passing zone.
219
Readings from these sensors are used as?
an input to a CEP operator to detect when a vehicle overtakes another one.
220
Obviously, such a scenario requires both?
the consistent and low-latency detection of complex events that signal when a vehicle has overtaken another one.
221
In case of false-positive detection, an unjustified ticket maybe --------------, while a false-negative detection would leave the ----------------unpunished. On the other hand, low-latency detection of an overtaking maneuver allows for direct
-------------------- to drivers who have
-----------------the traffic rules.
issued
transgressor
feedback
violated
222
In controlled environments, e.g., data centers, it is often uncomplicated to find?
acceptable bounds for communication and processing latencies.
223
Nevertheless, understanding communication and processing delays is not enough when?
the CEP operator suffers from overload, i.e., the arrival rate of events exceeds the achievable processing rate.
224
Nevertheless, understanding communication and processing delays is not enough when the CEP operator suffers from overload, i.e., the arrival rate of events exceeds the achievable processing rate. In this case, the CEP operator may adopt one of two strategies:
“load shedding”
or “buffering.”
225
With load shedding, the CEP operator simply discards events that it is?
not
capable of processing in time.
226
Obviously, such a strategy may lead to?
inconsistencies, which are not tolerable in several IoT applications.
227
it is necessary to keep a buffering limit in the operators to support low-latency event detection?
since In the second strategy, a high number of events are to be buffered before being processed, which can cause an unacceptable latency in event detection.
228
In fact, keeping a buffer limit is highly challenging ?
since many applications, e.g., traffic monitoring and smart grids, comprise very high and fluctuating event rates.
229
Efficient and consistent parallel execution of dozens of processing entities is necessary to?
speed up the CEP systems.
230
In this manner, the parallel CEP operators are capable of ?
meeting the buffering limits.
231
When deploying resources while considering the worst-case workload, the resources could be?
underutilized for a large part of the day when the traffic intensity is lower.
232
This underutilization causes unnecessary cost for resource provisioning. In fact, the costs already become?
high by reacting to dozens or even hundreds of different patterns that can occur in large-scale IoT applications, such as a smart city.
233
This underutilization causes unnecessary cost for resource provisioning. In fact, the costs already become high by reacting to dozens or even hundreds of different patterns that can occur in large-scale IoT applications, such as a smart city. One possible solution to such a problem is?
to dynamically adjust the parallelization degree of operators according to the current workload given by the arrival rate of events.
234
In this manner, significant cost gains can be achieved through?
avoiding underutilization of the system resources.
235
CEP operators need to support parallelization, dynamic reconfigurations, and adaptive provisioning of resources such that ?
the operator can meet a buffering limit for a given workload at minimal cost.
236
In fact, there exist several classes of CEP parallelization techniques, including :
intra-operator parallelization and data parallelization.
237
In the former class, internal processing steps are identified so that ?
the operator logic is split accordingly,
and the identified processing steps are executed in parallel on the incoming event streams.
238
This approach offers only a limited achievable parallelization degree depending on?
the number of variables in the query.
239
Alternatively, the data parallelization class involves?
First, a splitter assigns the events, according to a partitioning model, to several identical operator instances. The execution of an operator instance is controlled by a runtime environment (RE). The RE receives information about the assigned partitions and the corresponding events. Afterward, it manages the operator execution so that the assigned partitions are processed. Finally, a merger ensures that an ordering between all produced events is established. The ordered events are consecutively assigned sequence numbers.
applying stream partitioning.
240
In lieu of splitting the operator logic, the incoming event streams are split into?
partitions that can be processed by a number of identical instances of the operator.
241
The architecture of such a framework consists of:
a splitter,
many operator instances,
and a merger.
First, a splitter assigns the events, according to a partitioning model, to several identical operator instances.
The execution of an operator instance is controlled by a runtime environment (RE).
The RE receives information about the assigned partitions and the corresponding events.
Afterward, it manages the operator execution so that the assigned partitions are processed.
Finally, a merger ensures that an ordering between all produced events is established.
The ordered events are consecutively assigned sequence numbers.
242
To divide the event stream among the various operator instances, two common partitioning models can be utilized, namely:
insufficient to support consistent partitioning for operators that detect patterns of an unknown size.
key-based and batch-based.
243
The former model partitions the event stream by?
a key that is encoded in each event.
244
key-based partitioning only offers limited expressiveness?
The parallelization degree is limited to the number of different key values.
245
Alternatively, the batch-based stream partitioning model splits the streams into ?
batches that are large enough to fit any match to a queried pattern.
246
the communication overhead which occurs when?
patterns fluctuate in their size in terms of comprised events,
247
Aside from the communication overhead which occurs when patterns fluctuate in their size in terms of comprised events, the approach is?
insufficient to support consistent partitioning for operators that detect patterns of an unknown size.
248
In fact, machine learning is not a new computer science development. Instead, these mathematical models for data fitting and probability go back to?
the early 1800s to Bayes' theorem and the least squares method of fitting data. Both models still widely used in machine learning models.
249
The first neural network devices, referred to as ----------------------------, were developed in the early 1950s when computing machines and learning were ----------------------. At that time, mathematics was ---------------------------------------------------------------------, e.g., IBM S/360. Evolutionary computation, e.g., genetic algorithms and swarm intelligence, became a research focus in the late 1960s and 1970s and was used for ---------------------------------------------------------------.
perceptrons
unified
beyond the reasonable resources of existing computers
solving complex engineering problems
250
In fact, genetic algorithms are still employed today in?
a form of probabilistic AI, such as Bayesian inference models, and was successfully applied to research in gesture recognition and bioinformatics.
mechanical engineering and automatic software design.
251
In the mid-1960s, the concept of hidden Markov models was introduced as?
a form of probabilistic AI, such as Bayesian inference models, and was successfully applied to research in gesture recognition and bioinformatics.
252
With the advent of logic systems, the field of AI known as a logic-based AI emerged. The logic-based AI enabled programmers to ?
easily describe symbolic expressions.
253
One major drawback of logic-based AI models at this time was?
the inability of logic-based semantics to think like a human.
254
In the 1980s, expert systems were invented as another form of logic-based systems for ?
a well-defined problem trained by experts in that particular domain.
One could think of them as a rule-based engine for a control system.
255
Expert systems had been successfully used in ----------------------------------- and became the first -------------------------- available AI systems sold.
corporate and business settings
commercially
256
Along a different line, fuzzy logic first manifested in research by Lotfi Zadeh at University of California Berkeley in 1965, but it was not until 1985 that?
researchers at Hitachi discovered how fuzzy logic could be successfully applied to control systems.
257
Thanks to this discovery, significant interest was sparked in Japanese ------------------------------------ firms to adopt fuzzy systems into actual products. While expert systems and fuzzy logic seemed to be the basic building blocks for -----, researchers in the early 1990s observed that -------------systems, or------------- systems, could never ------the mind.
automotive and electronics
AI
expert
logic-based
emulate
258
Support vector machines (SVMs) became popular between ?
1970s and the early 1980s.
using a novel technique to find the best hyperplanes to categorize data sets.
259
They became the foundation for linear and nonlinear classification by?
using a novel technique to find the best hyperplanes to categorize data sets.
260
This technique became popular with?
handwriting analysis, before being evolved into uses for neural networks.
261
Recursive neural networks (RNNs) also became a topic of interest in the
1990s.
Such networks were different to traditional deep learning neural networks,
e.g.,?
convolutional neural network (CNN), because RNN could be applied to a problem involving the notion of time, such as audio and speech recognition.
262
The utilization of RNN and CNN were prohibitive until ?
the advent of GPU processors.
263
Thanks to all the aforementioned efforts to develop intelligence, today we find AI-based systems everywhere,:
from self-driving cars to speech recognition in Siri, to tools emulating humans in online customer service, to medical imaging, to retailers using machine learning models to identify consumer interest in shopping and fashion as they move about a store.
264
The following figure demonstrates the full spectrum of Al algorithms where machine learning represents the heart of AI-based systems.
265
In the context of IoT, machine learning offers interesting solutions while processing the massive amount of data generated by the IoT devices. In general, the value of a system of sensors is based on?
what a collection of sensors measures.
266
Some of that generated data will be structured, e.g.,?
time-correlated series.
267
Other data will be unstructured, e.g., ?
cameras, synthetic sensors, audio, and analog signals.
268
a manufacturing plant could plan to optimize ?
operational expenses and potentially capital expenses by adopting IoT and machine learning.
269
In such industrial environments, the manufacturers typically have many interdependent systems. They may have some assembly tool to produce?
a widget, another robot to cut parts out of metal or plastic, conveyor belts, lighting and heating systems, packaging machine, and inventory control systems.
270
In this case, IoT devices are to be attached to these systems to generate different types of data, before being intelligently processed by means of ?
various AI-based models.
271
In this context, machine learning and AI can predict ?
machine learning can handle big data that cannot be handled by humans.
behavioral effects.
272
Currently, experienced technicians are able to predict possible machine failures by?
observing some behavioral effects, e.g., a machine may start creaking in a certain way.
273
Machine learning can perform this role much earlier to implement?
a predicative maintenance system.
274
Sensors can surround such machines to monitor their actions. An entire factory could be perceived in such a case to ?
understand how that factory is performing at that very instant based on a collection of millions or even billions of events from every machine.
275
With this massive amount of data, only machine learning can overcome the noise and find the required patterns. In other words,?
machine learning can handle big data that cannot be handled by humans.
276
In fact, there is no single machine learning algorithm that can optimally fit for ?
all IoT data.
In particular, each model has its specific strength and the use case it serves.
277
In terms of the training data, there exist three types of learning systems to consider:
278
In terms of the training data, there exist three types of learning systems to consider:
supervised learning.
The training data provided to the model has an associated label with each entry. For instance, a dataset may be a collection of pictures each labeled with the content of that image, e.g., cat, dog, banana, car. Many machine learning models today are supervised. Supervised learning is mainly used for solving classification and regression problems.
unsupervised learning.
It requires no labels for the training data. This type of learning model utilizes mathematical rules to reduce redundancy. A typical use case is to find clusters of like things.
semi-supervised learning.
It mixes labeled data and unlabeled data. The main objective is to force the machine learning model to organize data as well as make inferences.
279
In general, the three fundamental uses of machine learning are :
classification,
regression,
and anomaly detection.
280
Classification :
is a form of supervised learning where labeled data is used to detect a name, value, or category.
For instance, neural networks can be used to scan images for the sake of detecting pictures of a shoe.
281
The following figure depicts three sets of data, i.e., blue, green, and red data points which are to be classified using the SVM algorithm.
282
The hyperplane is
a straight line; thus, it is called a linear classifier.
283
Through a training phase,:
SVM can find the best hyperplane, i.e., creates artificial segments, to divide the colored data points.
After several thousand iterations, the division is somewhat optimal where the artificial segments properly separate the different data sets,
i.e., the red data points exist in the red hyperplane.
In fact, SVM algorithm cannot support nonlinear relationships which are common in machine learning.
In fact, employing a linear model over nonlinear data would cause severe error rates.
284
Aside from classification algorithms concerned with predicting a discrete value, regression models are used to?
predict a continuous value.
285
For instance,
regression analysis can be used to predict the average selling price for a home based on the selling prices of all the homes in the neighborhood.
286
Several techniques exist to perform regression analysis, including:
least squares method, linear regression, and logistic regression.
287
“Random forest” is?
(1) they are easy to interpret and make for straightforward visualizations; (2) they can handle both numerical and categorical data; (3) they perform well on large datasets; and (4) they are extremely fast. A major shortcoming of decision trees is that they are prone to overfitting, especially when a tree is particularly deep. In this context, overfitting means the generation of a machine learning model that corresponds too closely or exactly to a particular set of data and may therefore fail to fit additional data or predict future observations reliably.
a subset of another machine learning model called the decision tree.
288
A decision tree is ?
a statistical learning algorithm that simply takes several variables into consideration and produces a single output that classifies the dataset.
289
Decision trees are?
-typically used for both classification and regression.
-The decision tree generates a set of probabilities that a path has taken based on the input.
-The model behaves with “if this, then that” conditions ultimately yielding a specific result.
290
Decision trees have several advantages:
(1) they are easy to interpret and make for straightforward visualizations;
(2) they can handle both numerical and categorical data;
(3) they perform well on large datasets; and (4) they are extremely fast.
291
A major shortcoming of decision trees is that they are?
prone to overfitting, especially when a tree is particularly deep.
In this context, overfitting means:
the generation of a machine learning model that corresponds too closely or exactly to a particular set of data and may therefore fail to fit additional data or predict future observations reliably.
292
To overcome the shortcoming of decision trees, random forest is?
usually used for classification and regression.
It is a supervised learning algorithm where it creates a forest of decision trees.
In other words, random forest builds multiple decision trees and merges them together to get a more accurate and stable prediction.
293
If the outcome of several trees--------with one another even though they arrived at that decision through -------- paths and a single tree is an ----------, one will naturally side with the majority. This creates a model with low variance, compared to a -------------- tree model that can be extremely -----------.
agrees
different
outlier
single decision
biased
294
Aside from random forest, Bayesian models are?
based on Bayes' theorem which describes the probability that an event will occur based on prior knowledge of the system.
295
For instance,
Bayesian models can estimate the probability that a machine will fail based on the temperature of the device.
296
As analog to random forest, a Bayesian network is?
an extension of Bayes' theorem in the form of a graphical probability model, i.e., a directed acyclic graph (DAG).
297
In DAGs, the graph flows?
in one direction and there are no loopbacks to previous states.
298
The various probabilities of each state come from ?
expert knowledge, historical data, logs, trends, or a combination thereof.
299
A Bayesian network is well suited for?
environments in IoT that cannot be completely observed or when the data is unreliable.
300
One major advantage of Bayesian networks is ?
their robustness against poor sampled data, noisy data, and missing data.
301
Furthermore, Bayesian methods avoid the overfitting problem. Additionally, Bayesian networks have been deployed to find?
time-correlated series from sensors and find and filter malicious packets in networking.
302
Various IoT platforms are currently available that can be used for?
developing an IoT solution.
303
platforms that are widely used for data storage and processing in IoT solutions, e.g.,?
Google Cloud and Amazon Web Services (AWS).
304
Microsoft Azure is a collection of ?
Microsoft-managed cloud services that connect, monitor, and control billions of IoT devices.
305
An IoT solution is typically made up of ?
one or more IoT devices and one or more back-end services running in the Azure Cloud that communicate with each other.
306
The back-end services provide several important functions:
(1) receiving telemetry, data collected by the IoT devices, at large scale, and determining how to process and store that data;
(2) analyzing the telemetry to provide insights, either in real time or in batch processing;
(3) sending commands from the cloud to a specific IoT device; and finally
(4) controlling the state of the IoT devices and monitor their activities.
307
For instance,
in a predictive maintenance scenario, the cloud back-end service stores historical telemetry.
The solution uses this data to identify potential anomalous behavior on specific machines before they cause a real problem.
Data analytics can be used to identify an appropriate solution, before sending a command back to the device to take a corrective action.
This process generates an automated feedback loop between the device and the cloud that greatly increases the solution efficiency.
308
The first component of Microsoft Azure is referred to as?
In other words, IoT Hub provides reliable device-to-cloud (D2C) and cloud-to-device (C2D) messaging at scale together with enabling secure communications. Accordingly, customers can use D2C telemetry data to understand the state of their machines and assets so that proper actions can be taken. Using the C2D messages, it is possible to reliably send commands and notifications to any connected devices. To develop an IoT application, an IoT Hub comprises device libraries for the most popular languages and platforms, thus making it versatile and common.
the Azure IoT Hub service, i.e., it represents the first endpoint of any IoT application running over Microsoft Azure.
308
The first component of Microsoft Azure is referred to as?
In other words, IoT Hub provides reliable device-to-cloud (D2C) and cloud-to-device (C2D) messaging at scale together with enabling secure communications. Accordingly, customers can use D2C telemetry data to understand the state of their machines and assets so that proper actions can be taken. Using the C2D messages, it is possible to reliably send commands and notifications to any connected devices. To develop an IoT application, an IoT Hub comprises device libraries for the most popular languages and platforms, thus making it versatile and common.
the Azure IoT Hub service, i.e., it represents the first endpoint of any IoT application running over Microsoft Azure.
308
In other words, IoT Hub provides?
reliable device-to-cloud (D2C) and cloud-to-device (C2D) messaging at scale together with enabling secure communications.
309
Accordingly, customers can use D2C telemetry data to understand?
the state of their machines and assets so that proper actions can be taken.
310
Using the C2D messages, it is possible to reliably send?
commands and notifications to any connected devices.
311
To develop an IoT application, an IoT Hub comprises device libraries for?
the most popular languages and platforms, thus making it versatile and common.
312
the most popular languages and platforms, thus making it?
versatile and common.
313
The Azure IoT Hub has an “identity registry” which stores?
all information about provisioned devices.
314
In fact, this information is related to ?
identity and authentication of the devices.
315
Specifically, it provides monitoring information such as?
Additionally, IoT Hub exposes another endpoint, referred to as the?
After receiving the telemetry data, the Stream Analytics service plays the role of?
connection status (i.e., connected or disconnected) and last activity time. Accordingly, customers can easily enable and disable the devices using this registry.
“device identity management,” which is used to create, retrieve, update, and delete devices.
monitoring the Azure IoT Hub and performing operations on the received data streams.
316
Such operations range from reformatting and transferring the data, to looking for?
specific patterns that may result in triggering some sort of action, depending on the event.
316
For instance,
in a healthcare application, the Stream Analytics service will constantly monitor the raw sensor data of the IoT Hub that is related to a user’s heart rate.
Once a sudden change occurs, an appropriate action is to be taken, e.g., calling a person or notifying an ambulance.
317
Such simple events can be simply identified using hard-coded rules through?
an SQL-like language which Microsoft has developed for the Stream Analytics services.
318
It is necessary to employ machine learning techniques that allow the system to recognize complex patterns.
In fact, detecting more complicated patterns and events can be more challenging.
319
Luckily, Microsoft Azure provides seamless integration with Azure’s Machine Learning Studio. Hence, it is ?
straightforward to create statistical models of sensor use cases using previous historical data, deploy them, and then use them as functions on the Stream Analytics service to help easily recognize new similar events that match the patterns of the trained models.
320
Such a pattern generation scheme is extremely helpful in ?
many IoT applications.
321
For instance,
predictive monitoring, i.e., predicting when an engine will break, can be performed based on similar sensor data patterns of engines that previously broke.
322
Finally, the last stage of Microsoft Azure deals with?
To the outside world, an IoT application can seem like a black-box that takes in raw sensor data, stores it, and performs the necessary actions depending on the application. Accordingly, Microsoft Azure’s services can be used together to build extremely efficient and scalable IoT solutions easily.
storing the received data and possibly visualizing it.
323
Microsoft Azure already provides several solutions such as:
SQL Database,
SQL Data Warehouse,
and DocumentDB for storage,
and PowerBI for visualization.
324
In fact, the major advantage of the Microsoft Azure IoT architecture is that:
all the services can interact with each other seamlessly.
325
In fact, the major advantage of the Microsoft Azure IoT architecture is that:
all the services can interact with each other seamlessly.
326
Such interaction typically occurs without ?
delay,
availability,
or security issues,
since all the services reside on the same data centers used to host the entire solution.
327
Google Cloud Platform (GCP) is ?
a collection of cloud computing services including data storage, data analytics, and machine learning.
328
GCD runs on the same infrastructure that Google internally utilizes for?
its end-user products, e.g., Google Search and YouTube.
329
From the architectural prospective, GCP consists of a set of physical servers, as well as?
virtual resources like VMs that are contained in Google’s data centers around the globe.
330
Through GCP, IoT developers can code, test, and deploy their applications with?
highly scalable and reliable infrastructure provided by Google.
331
Accordingly, system developers must solely focus on?
the code and Google handles issues regarding infrastructure, computing power, and data storage facility.
332
In fact, GCP is one of the popular IoT platform thanks to?
(1) its fast global network,
(2) providing better pricing than its competitors,
(3) offering support of various available services of cloud such as Firebase, PubSub, and Telit Wireless Solutions, and
(4) its powerful mechanism for connecting IoT devices to Firebase and Cassandra services.
333
There are several stages of IoT data management in ?
the Google Cloud Platform.
334
After the IoT project is up and running, many devices will?
produce a massive amount of data.
335
In this case, we need an efficient, scalable, and affordable way to both:
manage those devices and handle all that information and make it suitable for application.
336
Google Cloud is an ideal solution for ?
storing, processing, and analyzing the big data.
337
For the sensors which typically use an operating system, Google offers a ?
stripped-down version of Android for IoT devices called “Android Things.”
338
Android Things lets developers configure and push ?
over-the-air software updates to the operating system and applications on the IoT devices.
339
In general, Cloud IoT Core is a platform that enables?
large-scale data collection from IoT devices.
340
Furthermore, it automatically performs load balancing and scaling to support?
data ingestion from millions of devices.
341
Any application that needs to read data from certain IoT devices can subscribe to?
a single channel to receive the required data.
342
Cloud Functions are used to-------- the data to its final destination. A Cloud Function is triggered to execute when --------- occur. A Cloud Function may react to incoming data by writing it to a document database called -------as well as to ---------.
route
certain events
FireStore
BigQuery
343
BigQuery is also a fully managed data warehouse that can run?
SQL-like queries against terabytes of data in seconds.
344
Using BigQuery enables data scientists to ask questions against terabytes of data without needing to?
deploy and manage Hadoop clusters and to do so at reasonable costs.
344
Using BigQuery enables data scientists to ask questions against terabytes of data without needing to?
deploy and manage Hadoop clusters and to do so at reasonable costs.
345
Once we collect enough data, the Cloud Machine Learning service is used to?
train the models that can be later exported to run elsewhere on different data.
346
Firebase Firestore, a NoSQL document database, is utilized when ?
a database that supports location-aware queries in real time is highly required.
347
For instance,
Pokémon Go (2016) was built on a predecessor of Firestore, referred to as Datastore.
348
Amazon Web Services (AWS) are a ?
collection of services that form a cloud computing platform (Raj and Raman, 2017).
349
The most common services are :
Amazon Elastic Compute Cloud (EC2)
and Amazon Simple Storage Service (S3).
350
AWS provides large computing capacity faster and cheaper than ?
the privately built and maintained servers.
351
AWS automatically handles :
resource provisioning,
load balancing,
scaling,
and monitoring.
352
The main concept in AWS platform is the ?
device state.
353
The devices are able to report their state by ?
publishing messages to the message broker through topics.
354
The broker delivers the received messages to?
all clients subscribed on the specific topics.
355
The default settings for any application is ?
to run a single Amazon EC2 micro instance and an Elastic Load Balancer.
356
If required, additional instances can be added to efficiently handle any peaks in the ------. AWS IoT has a “---------” database that contains devices-related information. Through the ------ command line interface (CLI), a thing registry can be used to ------------------------------------------- things.
workload
thing registry
AWS
create, delete, and update
357
The AWS IoT Device SDK is a set of ?
software components that easily and quickly enables connecting an IoT device or a mobile application to AWS IoT Core.
358
The AWS IoT Device SDK can not only establish a connection, but?
it also enables authentication and messages exchange with AWS IoT Core using the MQTT, HTTP, or WebSockets protocols.
359
The AWS IoT Device SDK can be implemented using?
C and JavaScript, where it supports Arduino hardware platforms.
360
At the AWS cloud, the Device Gateway represents the?
entry point for IoT devices connecting to AWS.
361
This component manages all active device connections and implements multiple protocols to ensure that?
the IoT devices can securely and efficiently communicate with AWS IoT Core.
361
The Device Gateway supports several communication protocols, e.g.,?
MQTT, WebSockets, and HTTP 1.1.
361
For devices that connect using MQTT or WebSockets, the Device Gateway can establish long-lived, bidirectional connections, thus enabling these IoT devices to?
send and receive messages at any time with low latency.
361
The Device Gateway is designed to be ?
fully managed and scales automatically to support over a billion devices.
362
The Message Broker is a pub/sub message broker that securely transmits messages to and from?
the IoT devices and applications in a timely manner.
363
It supports various messaging patterns, such as:
control messaging and broadcast notification systems.
364
Additionally, fine-grained access controls can be implemented to enable managing the permissions of?
individual connections at the topic level, thus ensuring that the IoT devices and applications only send and receive the relevant data.
365
In general, AWS IoT Core provides:
mutual authentication
and encryption at all points of connection.
366
Accordingly, the collected data is never ?
exchanged between devices
and AWS IoT Core without a proven identity.
367
Specifically, AWS IoT Core offers several methods of authentication, such as ?
SigV4, X.509 certificate-based authentication,
and customer-created token-based authentication.
368
HTTP connections may use any of these methods, while MQTT connections use?
certificate-based authentication,
and WebSockets connections can use SigV4 or custom authorizers.
369
With AWS IoT Core, customers can authorize certain IoT devices or applications to have?
access, or revoke access without ever touching the IoT device.
370
In general, it is straightforward to create, deploy, and manage certificates and policies for?
the IoT devices from the console or using the API.
371
AWS IoT Core can also provide temporary AWS credentials after a device has authenticated with?
an X.509 certificate, so that the device can more easily access other AWS services such as DynamoDB or S3.
372
Another component of the AWS IoT Core is referred to as ?
the Device Shadow.
373
A Device Shadow of a certain device includes?
the device’s latest state so that applications
or other devices can read messages and interact with the device.
374
Even if the IoT device is offline, the Device Shadow provides?
the last reported state and desired future state of each device.
375
Thanks to the Device Shadow tool, we can retrieve the last reported state of?
a device or set a desired future state through the API or using the rules engine.
376
Applications can set the desired future state of a device without ?
accounting for the device’s current state.
377
AWS IoT Core compares the difference between
the desired and last reported state, and then command the device to make up the difference.
378
In general, the Device Shadow enables IoT developers to ?
store the state of your devices for up to a year.
379
Finally, the Rules Engine component enables building IoT applications that gather, process, analyze, and act on data generated by ?
the IoT connected devices at a global scale without having to manage any infrastructure.
380
The Rules Engine evaluates messages published into?
AWS IoT Core before delivering them to another device or a cloud service, based on customer-defined business rules.
381
A rule can be applied to data generated from one or many IoT devices, and then?
it can simultaneously take one or many actions.
382
The Rules Engine can route messages to several AWS endpoints including :
AWS Lambda, Amazon Kinesis, Amazon S3, Amazon Machine Learning, Amazon DynamoDB, Amazon CloudWatch, Amazon Simple Notification Service (SNS), Amazon Simple Queue Service (SQS), AWS IoT Analytics, Amazon Elasticsearch Service with built-in Kibana integration, and AWS Step Functions.
383
In addition to the existing rules, IoT developers can write ?
new rules using an SQL-like syntax.
384
Rules can be authored to behave differently depending upon?
the application.
385
For instance,
if a temperature reading exceeds a certain threshold it could trigger a rule to transmit data to AWS Lambda.
386
Rules can also be authored to take into?
account other data in the cloud, such as data from other devices.
387
For example,
a rule can be designed to make an action if the temperature is more than 15 percent higher than the average of 5 other devices.
388
The Rules Engine :
1-provides dozens of available functions that can be used to process the collected data.
2- Moreover, a myriad of other functions can be created through the AWS Lambda component.
For instance, if we are dealing with a wide range of values, we could take the average of the incoming numbers.
389
Rules can also trigger the execution of Java, Node.js, or Python code in AWS Lambda, giving developers?
maximum flexibility and power to process device data.
390
Data storage and data processing are two important components of an-----system architecture. Traditional relational database systems based on ----- fail to accommodate rapid change of ------ data and to provide an efficient means for -------------------.
IoT
SQL
unstructured
storage and elaboration
391
NoSQL database systems have been designed to?
scale horizontally and to run on clusters.
392
Briefly speaking, they are non-relational and distributed database systems which enable ?
rapid organization and analysis of huge amount of heterogeneous data.
393
Scalability, availability and consistency are achieved with?
ad hoc approaches (e.g., sharding).
393
Examples of widely used NoSQL database systems are?
MongoDB and Apache Cassandra.
394
A common approach to process the amount of data stored into databases effectively and efficiently is?
the MapReduce pattern.
395
the MapReduce pattern.
It uses hundreds or thousands of pluggable nodes in a cluster to process data in parallel, which significantly shortens the time between the events and the presentation of their analysis.
396
An open source implementation of the MapReduce framework can be found in ?
the Apache Hadoop.
397
The major benefits of IoT can be realized only if ?
a suitable infrastructure and mechanism to synthesize and interpret the data to make actionable decisions is set up.
In other words, there must be a way to generate and represent knowledge extracted from the heterogeneous data base.
398
Ontologies are knowledge representation systems which find applications in IoT in ?
describing resources and connections/relations among them.
399
The Resource Description Framework (RDF) is one way to describe such resources, which can be?
any device or concept (e.g., person, place).
400
RDF allows the description of the connections by means of ?
a subject, predicate, object triple.
401
The Web Ontology Language (OWL) formalism is another approach to ?
resource description that extends the RDF by solving some of its limits.
402
An implementation of OWL for sensors networks is? It can describe sensors and their characteristics, observations, and methods used for sensing.
the Semantic Sensor Network (SSN) ontology.
403
the Semantic Sensor Network (SSN) ontology.
It can describe sensors and their characteristics, observations, and methods used for sensing.
404
To achieve automatic processing and interpretation of the IoT data, we need ----to describe the IoT data including the ------ of the data. The concept of linked data emerges to connect individual data items to support ------------------------------------------on the data coming from physical and virtual objects.
standards
source
semantic query and inferences
405
The linked data is represented by the RDF or OWL formalism and provides the basis for?
information reuse and interoperability among heterogeneous sources.
DBpedia, which enables sophisticated queries over the information contained in Wikipedia.
406
One example of implementation of the linked data concepts is
DBpedia, which enables sophisticated queries over the information contained in Wikipedia.
407
Processing of IoT data can be performed with several technologies. Complex event processing (CEP) is?
processing of IoT data, in particular to perform classification, regression and anomaly detection.
an approach that identifies data and application traffic as events, correlates these events to reveal some patterns, and reacts to them by generating actions to systems, people, and devices.
408
It is an event-driven processing approach because ?
the computation is triggered by the receipt of event data, and aims at a real-time reaction.
409
One main challenge with CEP is ?
the overload which occurs when the arrival rate of events exceeds the achievable processing rate.
410
Solutions to this problem are based on ?
buffering, load shedding or parallelization.
411
Machine learning, as a subset of artificial intelligence methods, is another widely adopted approach for?
processing of IoT data, in particular to perform classification, regression and anomaly detection.
412
Various IoT platforms are available to ?
develop IoT applications and solutions.
Examples
are Microsoft Azure, Google Cloud Platform and Amazon Web Services.
413
Growth opportunities provided by IoT and insight from IoT data across the enterprise can give companies in any industry a ---------------------------. IoT helps companies realize greater -----------------------, more effective -----------------, and increased customer and employee --------------------.
competitive advantage
innovation
operations
engagement
