2021. Computer Vision.
Computer Vision at the University of Helsinki. September to October 2021.
106 flashcards · Computer Science · by Wehzie
How would ρ\rhoρ be initialized when implementing the Hough transform?
In polar coordinates, ρ\rhoρ represents distance from the origin. In polar coordinates we may bind ρ\rhoρ as 0≤ρ≤ρmax 0 \leq \rho \leq \rho_{\max}0≤ρ≤ρmax . An image's origin is defined to be the top left corner. The maximum distance from the origin can be the bottom left corner. Therefore, ρmax=w2+h2\rho_{\max} = \sqrt{w^2 + h^2}ρmax=w2+h2 , where w,hw, hw,h are image width and height.
Given the polar coordinates ρ=xcosθ+ysinθ\rho = x \cos \theta + y \sin \thetaρ=xcosθ+ysinθ derive the Cartesian coordinates y=ax+by = a x + by=ax+b . Then give the equations for aaa and bbb in terms of ρ\rhoρ and θ\thetaθ .
ρ=xcosθ+ysinθ\rho = x \cos \theta + y \sin \thetaρ=xcosθ+ysinθ −xcosθ+ρ=ysinθ- x \cos \theta + \rho = y \sin \theta−xcosθ+ρ=ysinθ −xcosθsinθ+ρsinθ=y- x \frac{\cos \theta}{\sin \theta} + \frac{\rho}{\sin \theta} = y−xsinθcosθ+sinθρ=y −xcotθ+ρcosecθ=y-x \cot \theta + \rho \cosec \theta = y−xcotθ+ρcosecθ=y y=−cotθ x+ρcosecθy = - \cot \theta \; x + \rho \cosec \thetay=−cotθx+ρcosecθ y=ax+by = a x + by=ax+b a=−cotθ=−cosθsinθa = - \cot \theta = - \frac{\cos \theta}{\sin \theta}a=−cotθ…
What is the size of the homography matrix HHH ?
3 x 3
A homography is another term for a projective transformation.
Other transformations are translation, Euclidean-, similarity- and affine transformations.
Name three steps to apply a homogeneous transformation to point ppp .
Convert point ppp to homogeneous coordinates. Multiply ppp by the homography matrix HHH . Convert ppp to heterogeneous coordinates.
How many degrees of freedom does a homography matrix HHH have?
8
Why does a homography matrix HHH have 8 instead of 9 degrees of freedom?
The entire matrix can be scaled by a scalar without changing the resulting transformation. The transformation is characterized by the relationship of its components. One parameter can always be scaled to the value 1. In relation to this parameter there are only 8 independent factors.
How many matching feature points between two images are at needed to apply a homography?
At least 4.
Explain the nature of ppp in the Least Squares Error function ELS=∑i∣∣f(xi,p)−x′∣∣2E_{LS} = \sum_i || f(x_i, p) - x' ||^2ELS=∑i∣∣f(xi,p)−x′∣∣2 .
Entity ppp is a parameter vector. The LSELSELSE is used to adjust ppp such that f(x,p)f(x, p)f(x,p) approaches x′x'x′ . An example of f(xi,p)f(x_i, p)f(xi,p) is y=ax+by = a x + by=ax+b where p=[a b]p = [a \; b]p=[ab] .
Imagine to take a photograph with a bare sensor. What is the result and why?
The result is a blurred image because each scene point contributes to all sensor pixels.
Assume no lens is used when taking a photo. What can an aperture achieve?
It can make each scene point contribute to only one sensor pixel.
Parallel light rays enter a convex lens. What is the relationship between the light rays upon exit?
A convex lens focuses parallel rays onto a single focal point.
What is depth of field and how does change in aperture affect it.
Depth of field is the range in which an object is approximately in focus. A smaller aperture increases the depth of field.
What is the name of this transformation? What is the result of the matrix multiplication? [10tx01ty001][xy1]= ?\begin{bmatrix} 1 & 0 & t_x\\ 0 & 1 & t_y\\ 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} x\\y\\1 \end{bmatrix} = \; ?⎣ ⎡100010txty1⎦ ⎤⎣ ⎡xy1⎦ ⎤=?
This is a translation. [10tx01ty001][xy1]=[x+txy+ty1]\begin{bmatrix} 1 & 0 & t_x\\ 0 & 1 & t_y\\ 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} x\\y\\1 \end{bmatrix} = \begin{bmatrix} x + t_x\\ y + t_y \\ 1 \end{bmatrix}⎣ ⎡100010txty1⎦ ⎤⎣ ⎡xy1⎦ ⎤=⎣ ⎡x+txy+ty1⎦ ⎤
What is the name of this transformation? What is the result of the matrix multiplication? [cosθ−sinθ0sinθcosθ0001][xy1]= ?\begin{bmatrix} \cos \theta & -\sin \theta & 0\\ \sin \theta & \cos \theta & 0\\ 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} x\\y\\1 \end{bmatrix} = \; ?⎣ ⎡cosθsinθ0−sinθcosθ0001⎦ ⎤⎣ ⎡xy1⎦ ⎤=?
This is a rotation. [cosθ−sinθ0sinθcosθ0001][xy1]=[xcosθ−ysinθxsinθ+ycosθ1]\begin{bmatrix} \cos \theta & -\sin \theta & 0\\ \sin \theta & \cos \theta & 0\\ 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} x\\y\\1 \end{bmatrix} = \begin{bmatrix} x \cos \theta - y \sin \theta\\ x \sin \theta + y \cos \theta \\ 1 \end{bmatrix}⎣ ⎡cosθsinθ0−sinθcosθ0001⎦ ⎤⎣ ⎡xy1⎦ ⎤=⎣ ⎡xcosθ−ysinθxsinθ+ycosθ1⎦ ⎤
What is the name of the following transformation? Compute the result of the matrix multiplication. [sx000sy0001][xy1]= ?\begin{bmatrix} s_x & 0 & 0\\ 0 & s_y & 0\\ 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} x\\y\\1 \end{bmatrix} = \;?⎣ ⎡sx000sy0001⎦ ⎤⎣ ⎡xy1⎦ ⎤=?
This is scaling. [sx000sy0001][xy1]=[sx⋅xsy⋅y1]\begin{bmatrix} s_x & 0 & 0\\ 0 & s_y & 0\\ 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} x\\y\\1 \end{bmatrix} = \begin{bmatrix} s_x \cdot x\\ s_y \cdot y \\ 1 \end{bmatrix}⎣ ⎡sx000sy0001⎦ ⎤⎣ ⎡xy1⎦ ⎤=⎣ ⎡sx⋅xsy⋅y1⎦ ⎤
Affine space is a generalization of Euclidean space.
Affine space defines no origin.
Give a general definition for the additive inverse of a number.
A number bbb forms the additive inverse of a number aaa when the sum of aaa and bbb is 000 . a+b=0a + b = 0a+b=0
What is the additive inverse of 3 in Z\mathbb{Z}Z ?
In Z\mathbb{Z}Z , the additive inverse of 3 is -3. 3+(−3)=03 + (-3) = 03+(−3)=0
Give a general definition of additive identity.
a+i=i+a=aa + i = i + a = aa+i=i+a=a
What number has the property of additive identity?
In Z\mathbb{Z}Z , n+0=0+n=nn + 0 = 0 + n = nn+0=0+n=n , so 000 is the additive identity. In Zmod n\mathbb{Z} \mod nZmodn , idem Zn\mathbb{Z}_nZn , a+k⋅n=k⋅n+a=aa + k \cdot n = k \cdot n + a = aa+k⋅n=k⋅n+a=a , so 000 and its congruents k⋅nk \cdot nk⋅n are additive identities in Zn\mathbb{Z}_nZn . Example in Z5\mathbb{Z_5}Z5 , 3+1⋅5=1⋅5+3=3≡−2,3,8,13,…3 + 1 \cdot 5 = 1 \cdot 5 + 3 = 3 \equiv -2, 3, 8, 13, \dots3+1⋅5=1⋅5+3=3≡−2,3,8,13,… , so 000 and its congruents k⋅5k \cdot 5k⋅5 are additive i…
Give a general definition for the multiplicative inverse of a number.
A number bbb forms the multiplicative inverse of a number aaa when the product of aaa and bbb is 111 . a⋅b=1a \cdot b = 1a⋅b=1
What is the multiplicative inverse of 111 in Z\mathbb{Z}Z ?
The multiplicative inverse of 111 in Z\mathbb{Z}Z is 111 . 1⋅1=11 \cdot 1 = 11⋅1=1
What is the multiplicative inverse of 222 in Z\mathbb{Z}Z ?
There exists none. While, 2⋅12=12 \cdot \frac{1}{2} = 12⋅21=1 , 12∉Z\frac{1}{2} \not \in \mathbb{Z}21∈Z .
Give a general definition of multiplicative identity.
a⋅i=i⋅a=aa \cdot i = i \cdot a = aa⋅i=i⋅a=a
What number has the property of multiplicative identity?
In Z\mathbb{Z}Z , 1 has the property of multiplicative identity.
Give a definition for the commutative property on binary operators.
A binary operator ∗*∗ is commutative in set SSS if swapping its first argument with the second always gives the same result. Idem, the order of arguments to the function doesn't matter. a∗b=b∗aa * b = b * aa∗b=b∗a in SSS f(a,b)=f(b,a)f(a, b) = f(b, a)f(a,b)=f(b,a) in SSS
Give a definition of the associative property on binary operators.
A binary operator ∗*∗ is associative in set SSS if the order of grouping does not matter. Idem, the order in which functions are executed doesn’t matter. (a∗b)∗c=a∗(b∗c)(a * b) * c = a * (b * c)(a∗b)∗c=a∗(b∗c) in SSS f(f(a,b),c)=f(a,f(b,c))f(f(a, b), c) = f(a, f(b, c))f(f(a,b),c)=f(a,f(b,c))
Give an example of an operation that is commutative but not associative.
Let f(a1,…,an)f(a_1, \dots, a_n)f(a1,…,an) be the arithmetic mean of it's arguments. 1n∑i=1nai=a1+a2+⋯+ann{\frac {1}{n}}\sum _{i=1}^{n}a_{i}={\frac {a_{1}+a_{2}+\cdots +a_{n}}{n}}n1∑i=1nai=na1+a2+⋯+an Then f(a1,a2)=f(a2,a1)f(a_1, a_2) = f(a_2, a_1)f(a1,a2)=f(a2,a1) , so fff is commutative. But f(f(a1,a2),a3)≠f(a1,f(a2,a3))f(f(a_1, a_2), a_3) \neq f(a_1, f(a_2, a_3))f(f(a1,a2),a3)=f(a1,f(a2,a3)) , so fff is not associative.
Give an example of an operation that is associative but not commutative.
Let f(a,b)f(a, b)f(a,b) be the concatenation of strings aaa and bbb . Then f(a,b)≠f(b,a)f(a, b) \neq f(b, a)f(a,b)=f(b,a) , so ff f is not commutative. But f(f(a,b),c)=f(a,f(b,c))f(f(a, b), c) = f(a, f(b, c))f(f(a,b),c)=f(a,f(b,c)) , so fff is associative.
Give the formal definition of a field in set theory.
A field FFF is a set with two operations, addition and multiplication. Under addition the elements in FFF form a commutative group. Under multiplication the non-zero elements in FFF form a commutative group. Addition and multiplication are linked by the distributive property. Set FFF with two operations: +++ , ⋅\cdot⋅ ⟨F,+⟩\langle F, + \rangle⟨F,+⟩ commutative group ⟨F×,⋅⟩\langle F^\times , \cdot \rangle⟨F×,⋅⟩ a⋅(b+c)=a⋅b+a⋅ca \cdot (b + c) = a \cdot b + a \cdot ca⋅(b+c)=a⋅b+a⋅c (b+c)⋅a=b⋅a+c⋅a…
Give loose definitions for groups, rings and fields in set theory.
Groups, rings and fields are types of sets. There are many groups, rings and fields. Whether a set is classified as group, ring or field depends on the support for operations. Each valid operation in a set yields an entity that is also a member in that set. Loosely, a group supports addition and each element has an additive inverse. A ring supports addition and multiplication; there are additive inverses but no multiplicative inverses for each member in the set. A field supports addition and mu…
Name an infinite field in set theory. How many infinite fields exist?
The rational numbers form an infinite field. Q\mathbb{Q}Q There are infinitely many infinite fields.
Name a finite field in set theory. How many infinite fields exist?
The field of integers modulo 5 forms an infinite field. Z5\mathbb{Z}_5Z5 , idem Zmod 5 \mathbb{Z} \mod 5Zmod5 There are infinitely many finite fields.
Zp\mathbb{Z}_pZp , idem Zmod p\mathbb{Z} \mod pZmodp , forms a series of sets known as prime fields.
Z5\mathbb{Z}_5Z5 , idem Zmod 5 \mathbb{Z} \mod 5Zmod5 is an example.
Given sets AAA and BBB what is the result of the Cartesian product A×BA \times BA×B ?
The result of the Cartesian product is the set of all ordered pairs (a,b)(a, b)(a,b) . A×B={(a,b)∣a∈A,b∈B}A \times B= \{(a,b) \mid a \in A, b \in B \}A×B={(a,b)∣a∈A,b∈B}
Loosely, what is a vector space?
A vector space is a set of vectors VVV for which axioms about the operations of vector addition and scalar multiplication hold.
Give a definition of a linear transformation (linear map)?
Consider a field KKK under vector spaces VVV and WWW . Take any two vectors v⃗,u⃗∈V\vec{v}, \vec{u} \in Vv ,u ∈V and any scalar c∈Kc \in Kc∈K . A function fff between vector spaces VVV and KKK is a linear map f:V→Wf: V \rightarrow Wf:V→W when fff satisfies conditions of additivity and homogeneity of degree 1. In other words, fff preserves the operations of vector addition and scalar multiplication. Additivity: f(u⃗+v⃗)=f(u⃗)+f(v⃗)f(\vec{u} + \vec{v}) = f(\vec{u}) + f(\vec{v})f(u +v )=f(u )+f(v…
What properties does an affine transform preserve and not preserve?
An affine transformation, in >= 2 dimensional space, preserves the following properties. Lines map to lines. Parallelism (of lines). Ratios (between lines, areas, ...). Compositions of affine transforms are affine transforms. Affine transformations don't map origin to origin.
Give an example of a linear transformation (linear map).
Rotation and reflection are linear transformations (linear maps).
What properties does a projective transform preserve?
Preserved are two properties. Lines map to lines. Compositions are projective transforms.
What does the camera parameter ttt denote?
Symbol ttt , translation, denotes the camera position in world coordinates.
What does the camera parameter RRR denote?
Symbol RRR , rotation, denotes camera orientation in world coordinates.
What does the camera parameter KKK denote?
Symbol KKK denotes the camera intrinsic matrix.
What are the two extrinsic camera parameters?
Translation ttt and rotation RRR .
What are the three intrinsic camera parameters?
Focal length, fff . Principal point (px,py)(p_x, p_y)(px,py) , idem u,vu, vu,v Pixel size (sx,sy)(s_x, s_y)(sx,sy)
What does the camera parameter PPP denote? Between which frames does PPP describe a mapping?
PPP is the projection matrix. It describes the relationship between the world frame, the camera frame and the image frame.
What is described by the parameters in −c-c−c ?
Parameter −c-c−c describes the translation between the world frame and the camera frame.
What mapping do the extrinsic parameters (right part) do? What mapping does the intrinsic KKK matrix do?
The extrinsic parameters maps 3D points in world coordinates to 3D rays in the camera's coordinate system. The KKK matrix maps 3D rays in the camera's coordinate system to 2D image coordinates.
Explain the components of the equation, P=K[R ∣−t] P = K [ R \; | -t ] P=K[R∣−t] where, −t=−Rc-t = -Rc−t=−Rc
PPP , is the camera projection matrix. KKK , is the camera's intrinsic matrix. RRR , is rotation between the world and camera frames. ∣|∣ , is the matrix concatenation operator. −t-t−t , is the translation of the world frame with respect to the camera frame over the camera center point.
Explain why −t=−Rc-t = -Rc−t=−Rc in the context of the camera projection matrix.
We have −t=Rc −t=Rc −t=Rc because we define translation with regards to the camera center. Rotation is not meaningful at the center point.
Explain the components in the equation x=PXx = PXx=PX in the context of camera calibration.
Here xxx are image points, PPP is the camera projection matrix and XXX are known 3D world coordinates.
Give a synonym of camera calibration.
Pose estimation, referring to the pose of the camera.
Explain the relationship between PPP , P′P'P′ and P′′P''P′′ in terms of the equations below.
x,yx, yx,y represent a real world point PPP . x′′,y′′x'', y''x′′,y′′ represent the point P′′P''P′′ ; this is a distorted mapping of PPP onto an image. x′,y′x', y'x′,y′ represent the corrected point P′P'P′ by multiplying x′′,y′′x'', y''x′′,y′′ with λ\lambdaλ .
Write out the equation to resolve the double fraction, abcd=?\frac{\frac{a}{b}}{\frac{{\color{red}c}}{{\color{blue}d}}} = ?dcba=?
abcd=ab:cd=ab⋅dc\frac{\frac{a}{b}}{\frac{{\color{red}c}}{{\color{blue}d}}} = \frac{a}{b} : \frac{{\color{red}c}}{{\color{blue}d}} = \frac{a}{b} \cdot \frac{{\color{blue}d}}{{\color{red}c}}dcba=ba:dc=ba⋅cd
What two properties characterize an inverse matrix?
A matrix is singular when it doesn't have an inverse. A matrix is singular when its' determinant is zero. Example, detA=det[3162]=3⋅2−6⋅1=0\det A = \det \begin{bmatrix}3 & 1\\ 6 & 2\end{bmatrix} = 3 \cdot 2 - 6 \cdot 1 = 0detA=det[3612]=3⋅2−6⋅1=0 .
How do points xxx and x′x'x′ in two images relate to epipolar lines lll and l′l'l′ ? Give two equations.
x⊤l=0x^\top l = 0x⊤l=0 and x′⊤l′=0x'^\top l' = 0x′⊤l′=0 .
How is the fundamental matrix FFF different from the essential matrix EEE ? Give FFF in terms of EEE and the calibration matrix KKK .
The equation for the fundamental matrix FFF is similar to that of the essential matrix EEE . We have, x′⊤Fx=0x'^\top F x = 0x′⊤Fx=0 and x^′⊤Ex^=0\hat{x}'^\top E \hat{x} = 0x^′⊤Ex^=0 . The essential matrix operates on normalized coordinates with respect to the camera. Because we have x^′=K−1x′\hat{x}' = K^{-1} x'x^′=K−1x′ and x^=K−1x\hat{x} = K^{-1} xx^=K−1x , EEE depends on the calibration matrix KKK . FFF depends on both intrinsics, via calibration matrix KKK and extrinsics via translation ttt…
What is the essential matrix EEE in stereo vision? Give the Longuet-Higgins equation.
In stereo vision, the essential matrix EEE relates each point in the first image to a point in the second image. The Longuet-Higgins equation is, x′⊤Ex=0x'^\top E x = 0x′⊤Ex=0 , where xxx is a point in the first image and x′⊤x'^\topx′⊤ is the transpose of the related point in the second image.
Relate the three conditions Camera calibration (pose estimation) Triangulation Reconstruction to knowledge about Structure (scene geometry) Motion (camera geometry) by creating a table.
What or where is the optical center, idem focal center, within a camera?
The optical center is the point where all light rays that enter through the camera's pinhole or lens cross. After crossing, the rays fall onto the image sensor (projection area). The image appears inverted on the projection area.
What are the two epipoles (epipolar points) in stereo vision (stereopsis)?
An epipole is the projection of one image's optical center onto the other image's virtual image plane. The virtual image plane is located in front of the optical center.
What three points span the epipolar plane in stereo vision (stereopsis)?
The two cameras' optical centers ooo and o′o'o′ and the point of interest ppp span the epipolar plane. The point ppp is observed by both cameras.
What are the two epipolar line in stereo vision (stereopsis)?
An epipolar line is the intersection between a virtual image plane and the epipolar plane. There are two epipolar lines, one for each camera.
What is the epipolar constraint in stereo vision?
Take image 1 and image 2 of real world point ppp , where image 1 is taken by camera 1 and image 2 is taken by camera 2. Assume that ppp projects onto image 1 at point xxx . And x′x'x′ is the unknown point where ppp projects onto image 2. Then x′x'x′ must lie on the epipolar line of image 2. This is known as epipolar constraint.
Where are the epipoles when two cameras are parallel in stereo vision?
The epipoles would be at infinity, outside the image.
What is a skew-symmetric matrix?
Matrix AAA is skew-symmetric when −A=A⊤-A = A^\top−A=A⊤ . In terms of matrix entries we have −Ai,j=Aj,i⊤-A_{i, j} = A^{\top}_{j, i}−Ai,j=Aj,i⊤ . Example, A=[02−45−20−44540]A={\begin{bmatrix}0&2&-45\\-2&0&-4\\45&4&0\end{bmatrix}}A=⎣ ⎡0−245204−45−40⎦ ⎤ is skew-symmetric because, −A=[0−245204−45−40]=AT-A={\begin{bmatrix}0&-2&45\\2&0&4\\-45&-4&0\end{bmatrix}}=A^{\textsf {T}}−A=⎣ ⎡02−45−20−44540⎦ ⎤=AT .
What is a square matrix?
A square matrix has the same number of rows nnn as columns mmm . AAA has dimensions n×mn \times mn×m where n=mn = mn=m .
What is the transpose of a matrix and how is it formed?
The transpose of matrix AAA is denoted A⊤A^\topA⊤ . We have Ai,j=Aj,i⊤A_{i, j} = A^{\top}_{j,i}Ai,j=Aj,i⊤ where the first index accesses rows and the second index accesses columns. Algorithmically, we can write the first row as first column, ..., until the last row as last column. We can also write the first column as first row, ..., until the last column as last row. Visually, fix a matrix along its diagonal. Then pull up the bottom-left corner, lift the corner over the diagonal, then push i…
What is the main diagonal of a matrix?
The main diagonal of a matrix AAA is a list. It is formed by those entries in AAA where row and column indices are equal. Ai,jA_{i, j} Ai,j where i=ji = ji=j .
What is the relation between the cross product and skew-symmetric matrices?
The cross product of two 3×13 \times 13×1 dimensional vectors a⃗×b⃗\vec{a} \times \vec{b}a ×b can be represented by forming a skew-symmetric matrix of a⃗\vec{a}a which is denoted [a⃗]×[\vec{a}]_\times[a ]× . [a]×=[ 0 −a3 a2 a30 −a1 −a2 a1 0][\mathbf {a} ]_{\times }={\begin{bmatrix}\,\,0&\!-a_{3}&\,\,\,a_{2}\\\,\,\,a_{3}&0&\!-a_{1}\\\!-a_{2}&\,\,a_{1}&\,\,0\end{bmatrix}}[a]×=⎣ ⎡0a3−a2−a30a1a2−a10⎦ ⎤
Given an image, what is the task definition of object detection?
The task of object detection on an image is to identify a set of objects. For each object a category label is provided and a bounding box.
Given an image, what is the task definition of semantic segmentation?
The task of semantic segmentation on an image is to label each pixel with a label. Object instances are not important, for example two adjacent cows form one area of pixels labelled cow.
Given an image, what is the task definition of classification; what is not in the scope?
The task of image classification is to provide one label for the provided image. A label itself could be quite complex in theory; however this is uncommon and inefficient.
What is the receptive field in the context of convolution?
The receptive field quantifies the many to one relation between an input and output object of a convolution.
How is camera motion between to images estimated? In other words how do translation vector ttt and rotation matrix RRR relate to the essential matrix EEE or the related fundamental matrix FFF ?
Motion between two images is given in terms of translation vector ttt and rotation matrix RRR . We have, E=[t^]xR=UΣV⊤=[u1u2t^][110][v0⊤v1⊤v2⊤]E = [\hat{t}]_x R = U \Sigma V^\top = \begin{bmatrix} u_1 & u_2 & \hat{t}\end{bmatrix} \begin{bmatrix}1 & &\\ & 1 &\\ & & 0 \end{bmatrix} \begin{bmatrix} v_0^\top \\ v_1^\top \\ v_2^\top \end{bmatrix}E=[t^]xR=UΣV⊤=[u1u2t^]⎣ ⎡110⎦ ⎤⎣ ⎡v0⊤v1⊤v2⊤⎦ ⎤ where EEE is the essential matrix, [t^]x[\hat{t}]_x[t^]x is the skew-symmetric representati…
What is LiDAR and how does it work?
Light detection and ranging (LiDAR) relies on controlled steering of laser beams followed by a distance measurement. Distance is inferred by the duration of a signal returning to the system and a known wavelength.
What are some downsides of LiDAR?
Some downsides of LiDAR are, Expensive technology. Energy hungry. Slow to process. Physically large systems. Glass may be challenging. Output is sparse, meaning a point cloud is given.
Show and name the tensor when working with the geometry of three views in stereo vision. Also explain the rank and shape of the tensor.
With three views we are dealing with a 3 x 3 x 3 tensor. Therefore we have a third order tensor known as trifocal tensor. See the following tensor examples, 1 vector of length 5: 1x5 tensor, 1 matrix with 2 rows and 3 columns: 1x2x3, 3 matrices with 3 rows and 3 columns: 3x3x3. The trifocal tensor doesn't describe the tensor of having 3 images. It represent the motion between the three views in the form of essential matrices or fundamental matrices.
What are RGB-D cameras and how do they work?
RGB-D cameras give a depth value for each pixel. Depth is calculated by projecting a pattern onto the scene with an infrared light. An infrared camera picks observes how the known pattern is distorted. Then differences between the original and distorted pattern are computed to infer depth.
What does the normalized 8-point algorithm do? Describe the algorithm; also mention preprocessing steps necessary.
The normalized 8-point algorithm computes the fundamental matrix FFF from two views. Preprocessing involves, Load image Convert to grayscale Detect features Match features Normalized 8-point algorithm involves, Normalize points with matrices TTT and T′T'T′ Build constraint matrix AAA UΣV=U \Sigma V = UΣV= singular value decomposition (svd) on AAA F=F =F= column in VVV mapped to smallest singular value F′=F' = F′= svd (F)(F)(F) such that FFF has rank 2 Denormalize F′F'F′ by TTT and T′T'T′
How does the essential matrix EEE relate to rotation RRR and translation ttt between two views?
We have E=R[t×]E = R [t_\times]E=R[t×] where [t×][t_\times][t×] is the skew-symmetric representation of translation vector ttt and RRR is rotation.
How do epipolar lines lll and l′l'l′ for two views relate to points xxx and x′x'x′ and the the essential matrix EEE ? Give two equations.
l′=Exl' = E xl′=Ex and l=E⊤x′l = E^\top x'l=E⊤x′ .
How do the epipoles eee and e′e'e′ in stereo vision relate to the essential matrix EEE ? Given an equation.
We have e′⊤E=0e'^\top E = 0e′⊤E=0 and Ee=0E e = 0Ee=0 .
What is structure from motion (SFM)? Does SFM eliminate the scale problem? What does structure mean in this context?
Structure from motion (SFM) is an algorithm which solves for 3D object locations and the camera parameter matrix PPP at the same time. SFM doesn't eliminate the scale problem. Structure refers to inferring a 3D point XXX from a set of matched 2D points.
In the context of structure from motion, what is bundle adjustment in simple terms?
Bundle adjustment means minimizing an error function. The function is the sum of squared reprojection errors over mmm views and nnn matched points. The function is called ggg for global structure from motion. Global as opposed to a sequential form of the algorithm where views and points are added after each other.
What is the SLAM algorithm? Is a prior knowledge needed? Is the algorithm on-line?
Simultaneous localization and mapping (SLAM) does two things at the same time. First localization, the act of finding and object's location on a map. Second mapping, the act of building a map of the environment. SLAM does both at the same time. The algorithm is online and doesn't require prior knowledge.
What is reprojection error in bundle adjustment?
First a 3D real-world point is estimated from 2D image points. Second the estimated 3D point is projected onto 2D views. The difference between observed and reprojected 2D image points gives the reprojection error.
What is loop-closure in simultaneous localization and mapping (SLAM) and why is it important?
A known landmark is revisited via a new path. The current position should lay on top of the landmark; due to drift this may not be the case. The position is updated and the path can mapped path can be improved to connect with previous knowledge. This is known as loop closure.
How is depth computed for stereo cameras? Where lies the difficulty?
Matching points for stereo rectified images have the same y coordinates. Disparity is the difference in the x coordinate between the matching points. The variable zzz indicates the distance to a real world point XXX . The baseline is the distance between the focal points C,C′C, C'C,C′ of both views and fff is the focal length. The difficulty in distance estimation for stereo cameras comes from finding the best matching points x,x′x, x'x,x′ . The points lie on a horizontal epipolar line. There a…
Are stereo cameras always completely aligned?
Yes, the optical axes are parallel so that depth can be computed. If optical axes are not parallel stereo image rectification may be used.
What is optical flow detection in simple terms and how does it compare to other feature extraction and matching methods?
Other feature extraction and matching approaches such as corner or SIFT detection are good when motion between views is small. Optical flow detection is used in analyzing motion in consecutive video frames. It operates on spatio-temporal brightness variations for each pixel.
What is the aperture problem in optical flow detection?
The aperture problem describes the difficulty of finding the direction in which a line is moving in an image patch. Consider a vertical line shown in a patch. We can take Iy=0I_y = 0Iy=0 because the gradient in the y direction is 0. We can recover uuu but not vvv , this is the aperture problem. Where Ix,IyI_x, I_yIx,Iy are the spatial derivatives in x and y direction, u,vu, vu,v the optical flow in x and y direction and ItI_tIt the temporal derivative. Ixu+Iyv+It=0I_x u + I_y v + I_t = 0Ix…
Give the brightness constancy equation for optical flow computation. In simple words explain the terms and how to compute them.
Ixu+Iyv+It=0I_x u + I_y v + I_t = 0Ixu+Iyv+It=0 where, Ix=δIδx,Iy=δIδy,I_x = \frac{\delta I}{\delta x}, I_y = \frac{\delta I}{\delta y},Ix=δxδI,Iy=δyδI, u=dxdt,v=dydtu = \frac{d x}{d t}, v = \frac{d y}{d t}u=dtdx,v=dtdy , It=δIδtI_t = \frac{\delta I}{\delta t}It=δtδI . Here Ix,IxI_x, I_xIx,Ix are the spatial derivatives. They can be computed with simple filters such as Sobel, Scharr. Then ItI_tIt is the spatial derivative. The difference between previous and current frame at each…
What are two key advantages of convolutional neural networks (CNN) over fully connected neural networks for computer vision?
Firstly, CNNs reduce the number of parameters in the network; therefore also decreasing computational cost. Secondly, convolution encodes spatial information of surrounding pixels into one node or unit.
What is a synonym for pooling in convolutional neural networks (CNN)?
Subsampling is a synonym for pooling.
What is the innovation in residual networks (ResNets)?
ResNets introduce skipping connections between layers. This allows using deeper network architectures. The vanishing gradient problem is mitigated. And training is speed up.
Name three upsampling methods and explain them. What is a use case of upsampling in CNNs?
In CNNs upsampling is used to build an encoder-decoder architecture. Through the bottleneck essential features are retained. Some upsampling methods are 1) max unpooling 2) nearest neighbor 3) bed of nails. For max unpooling the position max value during pooling is stored. At that position the upsampled pixel will be positioned while other values in the pool will be zero. Nearest neighbor multiplies the upsampled element in the pool across all fields. Bed of nails places the upsampled element i…
Why is the role of the decoder and the encoder components in encoder-decoder networks, respectively?
The decoder component reconstructs an approximation of the input. This approximation is filtered for expressive features via the processing applied by the encoder.
Name three challenges in object detection.
Firstly, the number of outputs per image is variable; the number of objects is unknown. Second, there are two types of output, the bounding box and the category label. Third, images must be large. At least around 800x600 pixels is required. In comparison classification works on around 224x224 pixels.