计算机视觉15 7.2 structure from motion II

计算机视觉,机器视觉,中国科学技术大学

Multi-frame Structure from Motion

计算机视觉,机器视觉,中国科学技术大学

Issues in SFMTrack lifetime Nonlinear lens distortion Degeneracy and critical surfaces Prior knowledge and scene constraints Multiple motions

计算机视觉,机器视觉,中国科学技术大学

Track lifetime

every 50th frame of a 800-frame sequence

计算机视觉,机器视觉,中国科学技术大学

Track lifetime

lifetime of 3192 tracks from the previous sequence

计算机视觉,机器视觉,中国科学技术大学

Track lifetime

track length histogram

计算机视觉,机器视觉,中国科学技术大学

Nonlinear lens distortion

计算机视觉,机器视觉,中国科学技术大学

Nonlinear lens distortion

effect of lens distortion

计算机视觉,机器视觉,中国科学技术大学

Prior knowledge and scene constraints

add a constraint that several lines are parallel

计算机视觉,机器视觉,中国科学技术大学

Prior knowledge and scene constraints

add a constraint that it is a turntable sequence

计算机视觉,机器视觉,中国科学技术大学

Factorization[Tomasi & Kanade, IJCV 92]

计算机视觉,机器视觉,中国科学技术大学

Problem statement

计算机视觉,机器视觉,中国科学技术大学

Notationsn 3D points are seen in m views q=(u,v,1): 2D image point p=(x,y,z,1): 3D scene point : projection matrix : projection function qij is the projection of the i-th point on image j ijq ij ( depth of qij ( x, y, z ) ( x / z, y / z ) projective j p i )

ij z

计算机视觉,机器视觉,中国科学技术大学

Structure from motionEstimate Mj and pi to minimize

(Π1 , , Π m , p1 , , p n ) wij log P( (Π j pi ); qij )j 1 i 1

m

n

1 wij 0

if pi is visible in view j otherwise

Assume isotropic Gaussian noise, it is reduced to

(Π1 , , Π m , p1 , , p n ) wij (Π j pi ) qijj 1 i 1

m

n

2

计算机视觉,机器视觉,中国科学技术大学

SFM under orthographic projection2D image point orthographic projection matrix 3D scene point image offset

q Πp tTrick

2 1

2 3 3 1 2 1

Choose scene origin to be centroid of 3D points Choose image origins to be centroid of 2D points Allows us to drop the camera translation:

q Πp

计算机视觉,机器视觉,中国科学技术大学

factorization (Tomasi & Kanade)projection of n features in one image:

q

q 2 qn p1 p 2 pn 2 n 3 n 2 3 projection of n features in m images1

q11 q12 q1n Π1 q 21 q 22 q 2 n Π 2 p p p 2 n 1 3 n q m1 q m 2 q mn Π m 2m 3 2m n

W measurement

M motion

S shape

Key Observation: rank(W) <= 3

计算机视觉,机器视觉,中国科学技术大学

Factorizationknown

2m n

W M S

2 m 3 3 n

solve for

Factorization Technique– W is at most rank 3 (assuming no noise) – We can use singular value decomposition to factor W:2m n

W M ' S'

2 m 3 3 n

– S’ differs from S by a linear transformation A:

W M' S' (MA )( AS )– Solve for A by enforcing metric constraints on M

1