Computer Vision (15-20%)

Question 1

Use this feature for general, unstructured documents with smaller amount of text, or images that contain text.

Answer 1

Azure AI Vision - OCR

Question 2

Use this service to read small to large volumes of text from images and PDF documents.

Answer 2

Azure AI Document Intelligence

Question 3

Which service do you use to read text from street signs, handwritten notes, and store signs?

Answer 3

OCR

Question 4

Which API would be best for this scenario? You need to read a large number of files with high accuracy. The text is short sections of handwritten text, some in English and some of it is in multiple languages.

Answer 4

Image Analysis service OCR feature

Question 5

What levels of division are the OCR results returned?

Answer 5

Results contain blocks, words and lines, as well as bounding boxes for each word and line.

Question 6

You’ve scanned a letter into PDF format and need to extract the text it contains. What should you do?

Answer 6

The Document Intelligence API can be used to process PDF formatted files.

Question 7

You want to use the Azure AI Vision Analyze Image function to generate an appropriate caption for an image. Which visual feature should you specify?

Answer 7

To generate a caption, include the Description visual feature in your analysis.

Question 8

What is the purpose of the Azure AI Vision service?

Answer 8

The Azure AI Vision service is designed to help you extract information from images through various functionalities.

Question 9

In AI Vision Service what features are available in the VisualFeatures enum?

Answer 9

VisualFeatures.TAGS: Identifies tags about the image, including objects, scenery, setting, and actions
VisualFeatures.OBJECTS: Returns the bounding box for each detected object
VisualFeatures.CAPTION: Generates a caption of the image in natural language
VisualFeatures.DENSE_CAPTIONS: Generates more detailed captions for the objects detected
VisualFeatures.PEOPLE: Returns the bounding box for detected people
VisualFeatures.SMART_CROPS: Returns the bounding box of the specified aspect ratio for the area of interest
VisualFeatures.READ: Extracts readable text

Question 10

What functionality exists in Azure Video Indexer?

Answer 10

Facial recognition - detecting the presence of individual people in the image. This requires Limited Access approval.
Optical character recognition - reading text in the video.
Speech transcription - creating a text transcript of spoken dialog in the video.
Topics - identification of key topics discussed in the video.
Sentiment - analysis of how positive or negative segments within the video are.
Labels - label tags that identify key objects or themes throughout the video.
Content moderation - detection of adult or violent themes in the video.
Scene segmentation - a breakdown of the video into its constituent scenes.

Question 11

What extensions can be made to Azure Video Indexer for custom insights?

Answer 11

People. Add images of the faces of people you want to recognize in videos, and train a model. Video Indexer will then recognize these people in all of your videos.
Note
This only works after Limited Access approval, adhering to our Responsible AI standard.

Language. If your organization uses specific terminology that may not be in common usage, you can train a custom model to detect and transcribe it.
Brands. You can train a model to recognize specific names as brands, for example to identify products, projects, or companies that are relevant to your business.

Question 12

You want Azure Video Indexer to analyze a video. What must you do first?

Answer 12

Upload the video to Azure Video Indexer and index it.

Question 13

You want Azure Video Indexer to recognize brands in videos recorded from conference calls. What should you do?

Answer 13

Edit the Brands model to show brands suggested by Bing, and add any new brands you want to detect.

Question 14

What resources need to be provisiioned to use the AI Custom Vision Service?

Answer 14

A training resource (used to train your models). This can be:
An Azure AI Services resource.
An Azure AI Custom Vision (Training) resource.
A prediction resource, used by client applications to get predictions from your model. This can be:
An Azure AI Services resource.
An Azure AI Custom Vision (Prediction) resource.

Question 15

Explain multiclass classification

Answer 15

there are multiple classes in the image dataset, but each image can belong to only one class

Question 16

Explain multilabel classification

Answer 16

an image might be associated with multiple labels

Question 17

What steps are performed in the Azure AI Custom Vision portal?

Answer 17

Create an image classification project for your model and associate it with a training resource.
Upload images, assigning class label tags to them.
Review and edit tagged images.
Train and evaluate a classification model.
Test a trained model.
Publish a trained model to a prediction resource.

Question 18

You want to train a model that can categorize an image as “cat” or “dog” based on its subject. What kind of Azure AI Custom Vision project should you create?

Answer 18

To train a model that classifies an image using a single tag, use an Image classification (multiclass) project.

Question 19

Which of the following kinds of Azure resource can you use to host a trained Azure AI Custom Vision model?

Answer 19

You can publish a trained Azure AI Custom Vision model to either an Azure AI Custom Vision (Prediction) resource, or an Azure AI Services resource.

Question 20

What features are available in the Face service within Azure AI Vision?

Answer 20

Face detection (with bounding box).
Comprehensive facial feature analysis (including head pose, presence of spectacles, blur, facial landmarks, occlusion and others).
Face comparison and verification.
Facial recognition.
Facial landmark location
Facial liveness - liveness can be used to determine if the input video is a real stream or a fake

Question 21

What method do you take to detect and analyze faces using the Azure AI Vision service?

Answer 21

call the Analyze Image function (SDK or equivalent REST method), specifying People as one of the visual features to be returned.

Question 22

What attributes can be returned in the Facial Attribute analysis?

Answer 22

Head pose (pitch, roll, and yaw orientation in 3D space)
Glasses (NoGlasses, ReadingGlasses, Sunglasses, or Swimming Goggles)
Blur (low, medium, or high)
Exposure (underExposure, goodExposure, or overExposure)
Noise (visual noise in the image)
Occlusion (objects obscuring the face)
Accessories (glasses, headwear, mask)
QualityForRecognition (low, medium, or high)

Question 23

How can the Face service be provisioned?

Answer 23

You can provision Face as a single-service resource, or you can use the Face API in a multi-service Azure AI Services resource.

Question 24

Describe the Face detection process in the Face API

Answer 24

When a face is detected by the Face service, a unique ID is assigned to it and retained in the service resource for 24 hours. The ID is a GUID, with no indication of the individual’s identity other than their facial features.

While the detected face ID is cached, subsequent images can be used to compare the new faces to the cached identity and determine if they are similar (in other words, they share similar facial features) or to verify that the same person appears in two images.

Question 25

What steps do you take to train a facial recognition model in the Face service?

Answer 25

Create a Person Group that defines the set of individuals you want to identify (for example, employees). Add a Person to the Person Group for each individual you want to identify. Add detected faces from multiple images to each person, preferably in various poses. The IDs of these faces will no longer expire after 24 hours (so they're now referred to as persisted faces). Train the model.

Question 26

You need to create a facial recognition solution to identify named employees. Which service should you use?

Answer 26

Use the Face service to create a facial recognition solution.

Question 27

You need to verify that the person in a photo taken at hospital reception is the same person in a photo taken at a ward entrance 10 minutes later. What should you do?

Answer 27

Verify the face in the ward photo by comparing it to the detected face ID from the reception photo. This is the most efficient approach as the photos were taken within 24 hours

Question 28

What is the easiest option for labeling images for object detection?

Answer 28

use the interactive interface in the Azure AI Custom Vision portal. This interface automatically suggests regions that contain objects, to which you can assign tags or adjust by dragging the bounding box to enclose the object you want to label.

Question 29

What do the bounding box measurement units quantify in image labeling?

Answer 29

The values are the proportion of the full image size. Consider the example: Left: 0.1 Top: 0.5 Width: 0.5 Height: 0.25 This defines a box in which the left is located 0.1 (one tenth) from the left edge of the image, and the top is 0.5 (half the image height) from the top. The box is half the width and a quarter of the height of the overall image.

Question 30

What must you do before taking advantage of the smart labeler tool when creating an object detection model?

Answer 30

To take advantage of the smart labeler, tag some images and train an initial model. Subsequently, the portal will suggest tags for new images.

Question 31

What steps are needed to use the Read API for OCR?

Answer 31

call the ImageAnalysis function (REST API or equivalent SDK method), passing the image URL or binary data, and optionally specifying a gender neutral caption or the language the text is written in (with a default value of en for English). result = client.analyze( image_url=, visual_features=[VisualFeatures.READ] ) https:///computervision/imageanalysis:analyze?features=read&...

Question 32

You build an app named App1 that uses the Azure AI Face service. You need to optimize the app for images that contain blurry faces. What should you do?

Answer 32

Set the detection model to detection_02.

Question 33

You are building an app that will be deployed to an edge device and will use Azure AI Custom Vision to analyze images of fruits. You need to select a model domain for the app. The solution must support running the app without internet connectivity. Which model should you use?

Answer 33

Compact domain is correct. The Azure AI Custom Vision service only exports compact domains, and the models generated by compact domains are optimized for the constraints of real-time classification on mobile devices.