Week 4 - 3D Computer Vision

Question 1

What is the fundamental problem with trying to recover the 3D structure of the scene from points matched between two images?

Answer 1

Fundamental ambiguity - Any point on the ray OP projects to the same image location, called P.

Question 2

What is Stereo Correspondence?

Answer 2

Find matching pixels/features in 2 or more images and convert their 2D positions into 3D depths

Question 3

What can resolve the fundamental ambiguity in stereo correspondence?

Answer 3

A second camera can resolve the ambiguity enabling measurement via triangulation

Question 4

How do you achieve depth recovery using two cameras?

Answer 4

You use triangulation, which requires:
- Knowledge of absolute and relative camera geometry i.e. Calibration
- Point correspondence i.e. which rays to intersect

Question 5

What are the properties of camera calibration?

Answer 5

It recovers the intrinsic parameters of the cameras e.g. focal length, pixel size, principal point, lens distortion
Relative poses between cameras, also called extrinsic parameters, are also factored in e.g. rotation, translation, scale that transforms left image on to right

Question 6

What is the easiest way to perform camera calibration?

Answer 6

Simplest approach is to use a known calibration target object

Question 7

What additional geometric distortions are present within lenses used in cameras?

Answer 7

• Decentering errors: Displacement of the lens centre from optical axes
• Radial distortion: Variations in light refractions, mostly in wide angle lenses

Question 8

What is the image warping parameter?

Answer 8

Image warping parameter is estimated to warp the ideal projected coordinate to the distorted coordinate. K contains warping parameter.

Question 9

What is the equation for image warping in stereo correspondence?

Answer 9

x’ = warp(x, k)
Where:
- x = Ideal image (no distortion)
- x’ = Observed image with distortion

Question 10

How are points generally defined in 3D space in stereo correspondence?

Answer 10

Points in the 3D space are expressed in terms of a different coordinate frame known as the world coordinate frame. The relation given between the coordinates of P in camera and world coordinate system are given by: Xcam = R(Xw - c)
Where:
- c = 3x1 vector representing the coordinates of the camera in the world coordinate system
- R = 3x3 matrix representing the orientation of the camera

Question 11

What is the purpose of camera calibration in a mathematical sense?

Answer 11

It is to calculate the intrinsic, extrinsic and distortion parameters.

Question 12

What is epipolar geometry?

Answer 12

Given the two optical centres and a point in one image, we can compute the epipolar plane and so the corresponding epipolar line in the other image

Question 13

Why is epipolar geometry important for camera calibration?

Answer 13

Given two calibrated cameras, it’s possible to retrieve the actual 3D coordinate of a corner in the image

Question 14

How can correspondence be used to measure depth of an object in an image?

Answer 14

Correspondence allows measurement of disparity: The difference in the image coordinates of the projections of a given world point into each camera.
Depth is inversely proportional to disparity.

Question 15

How does correspondence search work?

Answer 15

• Find a window in the original image
• Slide it along the right scanline and compare the content of that window with that of the reference window in the original image

Question 16

What is the effect of a window size in correspondence search?

Answer 16

• Larger window size: Smooth disparity maps but less detail captured
• Smaller window size: More detail, but also more noise captured

Question 17

What are some problems with correlation-based stereo?

Answer 17

• Window size is fixed across the image, but viewed objects differ in size and depth
• Uniform regions always match
• Can provide a dense disparity map, but values are only reliable where there is some local variation in intensity e.g. near edges
• Dense disparity is computationally expensive in spatial domain

Question 18

How do you get ground truth data?

Answer 18

• Alternative/competing sensors
• Artificial images
• Real images

Question 19

What are some problems that can be encountered when gathering ground truth data?

Answer 19

• Automatic methods can have errors
• Manual methods are slow, subjective and also error prone
• What if standard sets don’t have the properties you are attempting to evaluate your images on?

Question 20

What is True Positive defined as?

Answer 20

True Positive - The algorithm makes a correct prediction about the presence of an object in an image

Question 21

What is False Positive defined as?

Answer 21

The algorithm predicts the presence of an object but that object is not present in the image

Question 22

What is False Negative defined as?

Answer 22

The algorithm misses an object

Question 23

What is Precision’s equation, and how is it defined?

Answer 23

Precision = TP/(TP + FP)
Fraction of responses that were correct

Question 24

What is Recall’s equation, and how is it defined?

Answer 24

Recall = TP/(TP + FN)
Fraction of correct classifications that were identified

Question 25

How do you evaluate recognition?

Answer 25

Using Precision-Recall curves as a visualisation tool

Question 26

What are some properties of Precision-Recall curves?

Answer 26

- Plot of precision against recall as some parameter is varied - Parameter is the threshold used to decide if the model and image are similar enough to be considered equal - Increasing threshold imposes a tighter requirement on matching, which reduces False Positives, but increases False Negatives

Question 27

How would you measure accuracy in a classification problem?

Answer 27

For classification, you can use a confusion matrix, which shows what category images are confused with others

Question 28

How is ground truth defined?

Answer 28

Ground truth is a set of manually-drawn bounding boxes on an image

Question 29

What are two key elements of measurement and recognition?

Answer 29

- Accuracy - Robustness

Question 30

What does a Precision Plot measure?

Answer 30

A Precision Plot measures the percentage of frames whose estimated location is within a given threshold distance of the ground truth.

Question 31

When using algorithms that learn, what properties must it fulfill?

Answer 31

- Must be representative of the data - Must not be too specific - Must not use training data in the evaluation of performance

Question 32

What does the term 'confining research leftward' mean in terms of Disparity Maps?

Answer 32

When you know which image is the left image and which image is the right image, then you can make safe estimations such as a point x in the left image could never be further to the right in the right image, thereby narrowing the search space.