Lecture 11 - Optical Flow Flashcards
(38 cards)
What is a video?
A sequence of frames captured overtime, where the image data is a function of space and time
Why is motion useful?
- We know what is moving and not moving
- Detect moving objects and make decisions
- Determines direction (i.e. which way is traffic is moving)
What is optical flow?
The movement of pixels overtime
What is the goal of optical flow?
- To generate a motion vector for each pixel in an image between t0 and t1 by looking at two images
- By computing the motion vector between each frame in a video we can track the flow of objects or the brightness patterns over an extended period of time
What’s an issue with optical flow?
It is only limited to representing the apparent motion
What are the assumptions (constraints) of optical flow?
- Brightness should be constant
- Small motion (large motions make it hard to track)
What is a motion field?
Image velocity of a point that is moving in the scene
What is the formula for motion?
- I (x,y,t)
- Where:
○ The horizontal component is u(x, y)
○ The vertical component is v(x, y)
When does optical flow not equal motion field?
- If source is moving and object is stationary it an optical flow
- If the object is moving and the source is stationary we have motion field
REFER TO EXAMPLES
What is the formula for displacement (shift)?
I(x+u, y+v, t+1) = I(x,y,t), as you can see it is the same a the orginal formula for motion as brightness doesnt change
What are the two assumptions of optical flow?
- Brightness of the point will remain the same
○ Encapsulates Spatial smoothness - neighbouring pixels are lilely to belog to the same surface - Small motion
○ Displacement with change is x and y
What is the formula of optical flow?
Formula for optical flow: (u,v) = (change in x (x/time), change in y (y/time))
What is the brightness constancy equation for optical flow?
- Ixu + lyv + It = 0
- Allows for change across two frames
What do the terms of brightness constancy equation represent?
- Ix and Iy represent the image gradient at point p
- u and v represent the flow velocities
- It represents the temporal gradient
- We can compute Ix, Iy and It from the image
How do you compute Ix and Iy derivatives?
- Using forward difference
- Sobel filter
- Derivative of Gaussian Filter
- More specifically
○ You can apply [-1 0 1] to find Ix
○ You can apply [-1 0 1]^T to find Iy
How do you compute temporal derivative?
- Frame differencing
○ Which is subtracting one frame from another
○ t - (t+1) = It
What is the aperture problem?
Where the movement is ambiguous, looks like it’s moving in one direction but if going in another direction
- We recover the v of the optical flow but not the u
Why does the aperture problem happen?
- Due to viewing only constant areas
- This problem does not occur in corners where you can see the movement
What is the Lucas-Kanade Solution?
- Same as the brightness constancy equation but uses k and l instead
- Ix(k,l)u + iy(k,l)v + It(k,l) = 0
Where do we get more equations to determine u and v?
We know that u and v are unknown (cannot find in image) so we need two equations to find it
- We use the constant flow of the surrounding patch (which can be of varying size for example 5x5)
○ This means we will get 25 total equations that can be written as:
§ X = (A^T * A)^-1 * A^T *b
What are the properties of the equations to determine u and v??
- A^T *A should be invertible
- A^T * A should not be too small (if small it’s not a corner)
- A^T * A should we well-conditioned
Why is the smooth region for optical flow bad?
- Cannot reliably compute the flow
- As the pixel moved, but the brightness did not change so we can’t find optical flow
Why are edges bad in optical flow?
- We don’t know how the edges are moving
- Basically aperture problem
- Badly conditioned and prominent gradient in one direction
Why are Textured areas good for optical flow?
- Well-conditioned
- Large and diverse gradient magnitudes
- Can easily compute
