How does the LKT defines and solves tracking?
- [1] By the brightness constancy equation we have that I(x,y,t) = I(x+u,y+v,t+1)
- Assuming a linear displacement of pixels, we can say that I(x+u,y+v,t+1) =' I(x,y,t) + I_x*u + I_y*v + I_t
- Hence I(x+u,y+v,t+1) - I(x,y,t) = I_x*u + I_y*v + I_t (and by using [1], all = 0). Written as nabla I*[u,v]^T + I_t = 0
- There's one equation with two variables so the solution would have the barber-pole problem. The solution is to take neighboring pixels (a window) and assume they all move the same.
- That's too many. On a 5x5 window, there are already 25*3(RGB) equations. Instead of solving for each, solve the matrix A d = b --> [I_x p_0, I_y p_0; ...; I_x p_n, I_y p_n] * [u,v]^T = -[I_t p_0;...;I_t p_n] using least squares which imply solving (A^T A) * d = A^T b (way less dimensions)