Video Processing Flashcards
Video Processing:
What is Video?
- sequence of frames captured over time
- image data is a function of space and time
(Motion Analysis) What information can be extracted from time varying sequences of images?
– Camouflaged objects easily visible when
moving
– Size and position of objects are more easily
determined when the objects move
– Even simple image differencing provides edge
detector for objects moving over any static
background
Into how many stages can analysis of visual motion be divided?
2
– measurement of the motion
– use of motion data to segment scene into objects and
to extract information about shape and motion.
How many types of motion are there to consider?
– movement in the scene = static camera,
– movement of the camera = ego motion.
* should be the same (motion is relative) but not always true -
if scene moves illumination, shadow and specular effects
need to be considered
2
– static camera=movement in the scene
– ego motion = movement of the camera
It should be the same (motion is relative) but not always true -
if scene moves:
-illumination, shadow and specular effects
need to be considered
In what are we interested in? (Optical Flow and Motion)?
We are interested in finding the movement of scene objects from time-varying images (videos).
Which are the uses of motion analysis?
-Track object behavior
-Align images (mosaics)
-3D shape reconstruction
-Correct for camera jitter (stabilization)
-Special effects
What is the Motion field? And
-projection of the 3d scene motion into the image
- length of flow vectors inversely proportional to depth Z of 3D point (points closer to the camera move quickly across the image plane
What is the Optical flow?
- apparent motion of brightness patterns (or colors) in the image
Ideally, optical flow should be the same as the motion field?
Yes
How can apparent motion be caused?
It can be caused by lighting changes without any actual motion.
What is necessary to estimate pixel motion from image?
We have to solve the pixel correspondence problem.
What is the pixel correspondence problem?
Given a pixel in frame t, look for nearby pixels
with same characteristics (colour, brightness, …)
in frame t − 1.
How to estimate pixel motion from image H to image I? (pixel correspondence problem)
We need to find pixel correspondences:
- Given a pixel in H, look for nearby pixels of the same
color in I
What are the key assumptions in order to solve the pixel correspondence problem?
-color constancy: a point in H looks “the same” in image I
(for example: In grayscale images, this is brightness constancy)
– small motion: points do not move very far
What is the Lucas Kanade method (most popular Optical
Flow Algorithm) main goal? And how do you achieve it?
To get more equations for a pixel
The basic idea is to: impose additional constraints
What is the most common Lucas Kanade aditional constraint?
It is to assume that the flow field is smooth locally by pretending the pixels neighbors have the same (u,v).
But If we use a 5x5 window, that gives us 25 equations per pixel!
What is the problem that comes from the most common Lucas Kanade aditional constraint?
Having more equations than unknowns.
How to solve the problem that comes from the most common Lucas Kanade aditional constraint?
Solve least squares problem.
-The sum are over all pixels in the K x K window
When is the problem that comes from the most common Lucas Kanade aditional constrain solvablet?
A^T * A should be :
-invertible
- not to small due to noise (λ1 and λ2 should not be too small)
- well-conditioned (λ1 and λ2 should not be too large (λ1 = larger)
What is the Algorithms goal in Optical Flow?
They try to approximate the true motion field of the image plane.
What is background subtraction?
-it allows looking at video data as a spatio-temporal volume.
- if camera is stationary, each line through time corresponds to a single ray in space
When are Background subtraction techniques commonly used for?
They are commonly used for segmenting out objects of interest in a static camera scene:
- surveillance
- robot vision
- object tracking
- traffic applciations
- human motion capture
- augmented reality
How does the background subtraction allow the segmentation of objects of interest in a static camera scene?
Trough the foreground mask (binary image) that it creates, containing moving objects in static camera setups:
- subtracting the observed image from the estimated image and thresholding the result
What is Foreground detection?
How the object areas are distinguished from the background
