Write the warp$\_$image and image$\_$rectification functions. WRITE ONLY UNDER WRITE YOUR CODE HERE

$1\mathrm{.}4\mathrm{.}4$ Problem $3\mathrm{.}4$: Uncalibrated Stereo Image Rectification $[15$ points$]$

In Assignment $2,$ you performed epipolar rectification with calibrated stereo cameras. Rectifying a pair of images can also be done for uncalibrated camera images. Using the fundamental matrix we can find the pair of epipolar lines $\backslash ($ l$\_\{$i$\}\backslash )$ and $\backslash ($ l$\_\{$i$\}\backslash )$ for each of the correspondences. The intersection of these lines will give us the respective epipoles $\backslash ($ e $\backslash )$ and $\backslash ($ e $\backslash ).$ Now to make the epipolar lines to be parallel we need to map the epipoles to infinity. Hence, we need to find a homography that maps the epipoles to infinity.

The rectificaton method has already been implemented for you. You can get more details from the paper Theory and Practice of Projective Rectification by Richard Hartley.

Your task is to:

$1)$ complete the warp$\_$image function $($Hint: You may reuse some of the codes from Homework$2,$ but this time we perform the warp of the full image content, i$.$e$.,$ the output image will contain the full image content in the source image. Thus, the size of the output image may not be same as the size of the source image. Calculate the targeted size of the output image by piping four corners through the forward transformation$).$

$2)$ complete the image$\_$rectification function to find the rectified images. Use the provided compute$\_$matching$\_$homographies to compute the rectification transformation matrices.

$3)$ plot the parallel epipolar lines using the plot$\_$epipolar$\_$lines function from above.

The figure below gives you an idea on how the final results look $($Note that the two images may not be in the same shape$).$ Show your result for matrix and warrior.

$```$

def warp$\_$image$($image$,$ H$)$:

$"$n$"$

Performs the warp of the full image content.

$```$

Calculates bounding box by piping four corners through the transformation.

Args:

image: Image to warp

H: The image rectification transformation matrices.

Returns:

Out: An inverse warp of the image, given a homography.

min$\_$x$,$ min$\_$y: The minimum$/$maxmum of warped image bound.

$"""$

### YOUR CODE HERE

### END YOUR CODE

return out, min$\_$x$,$ min$\_$y

$[]$: from math import floor, ceil

def compute$\_$matching$\_$homographies$($e$2,$ F$,$ im$2,$ points$1,$ points$2)$:

$""$ "This function computes the homographies to get the rectified images.

Args:

e$2$: epipole in image $2$

F : the fundamental matrix $($think about what you should be passing: F or F$.$T $!$

im$2$: image$2$

points$1$: corner points in image$1$

points$2$: corresponding corner points in image$2$

Returns:

H$1$: the image rectification transformation matrix for image $1$

H$2$: the image rectification transformation matrix for image $2$

$"""$

# calculate H$2$

width $=$ im$2.$ shape$[1]$

height $=$ im$2.$shape$[0]$

T$=$np$.$identity $(3)$

T$[0][2]=-1\mathrm{.}0**$ width $//2$

T$[1][2]=-1\mathrm{.}0**$ height $//2$

e$2$hat $=$ T$.$ dot $($e$2)$

norm$\_$v $=$ np$.$linalg.norm$($e$2$hat $[$:$2])$

cos$\_$theta $=$ e$2$hat$[0]/$ norm$\_$v

sin$\_$theta $=$ e$2$hat$[1]/$ norm$\_$v

if cos$\_$theta $<mtext></mtext><mn>0</mn>$:

cos$\_$theta $=-$cos$\_$theta

sin$\_$theta $=-$sin$\_$theta

R $=$ np$.$array$([[$cos$\_$theta, sin$\_$theta, $0],$

$[-$sin$\_$theta, cos$\_$theta, $0],$

$[0,0,1]])$

G $=$ np$.$identity$(3)$

G$[2,0]=-$e$2$hat$[2]/($e$2$hat$[0]*$ cos$\_$theta $+$ e$2$hat$[1]*$ sin$\_$theta$)$

H$2=$ np $|$ dot$($np$.$dot $($G$,$ R$),$ T$)$

calculate H$1$

e$\_$prime $=$ np$.$zeros$((3,3))$

e$\_$prime$[0][1]=-$e$2[2]$

e$\_$prime$[0][2]=$ e$2[1]$

e$\_$prime$[1][0]=$ e$2[2]$

e$\_$prime$[1][2]=-$e$2[0]$

e$\_$prime$[2][0]=-$e$2[1]$

e$\_$prime$[2][1]=$ e$2[0]$

v $=$ np$.$array$([1,1,1])$

M $=$ e$\_$prime.dot$($F$)+$ np$.$outer$($e$2,$ v$)$

points$1\_$hat $=$ H$2.$dot$($M$.$dot$($points$1.$T$)).$T

points$2\_$hat $=$ H$2.$dot$($points$2.$T$).$T

W $=$ points$1\_$hat $/$ points$1\_$hat $[$:$,2].$reshape$(-1,1)$

b $=($points$2\_$hat $/$ points$2\_$hat$[$:$,2].$reshape$(-1,1))[$:$,0]$

least square problem

a$1,$ a$2,$ a$3=$ np$.$linalg.lstsq$($W$,$ b$,$ rcond$=$None$)[0]$

HA $=$ np$.$identity$(3)$

HA$[0]=$ np$.$array$([$a$1,$ a$2,$ a$3])$

H$1=$ HA $|$ dot$($H$2)$ dot$($M$)$

return H$1,$ H$2$

$]$: def image$\_$rectification$($im$1,$ im$2,$ points$1,$ points$2)$:

$"""$This function provides the rectified images along with the new corner

$-$points as

images with corner correspondences

Args:

im$1$: image$1$

im$2$: image$2$

points$1$: corner points in image$1$

points$2$: corner points in image$2$

Returns:

rectified$\_$im$1$: rectified image $1$

rectified$\_$im$2$: rectified image $2$

new$\_$cor$1$: new corners in the rectified image $1$

new$\_$cor$2$: new corners in the rectified image $2$

n$""$

YOUR CODE HERE

END YOUR CODE

return rectified$\_$im$1,$ rectified$\_$im$2,$ new$\_$cor$1,$ new$\_$cor$2$

$[]$:

This code is for you to plot the results.

The total number of outputs is $4$ images in $2$ pairs

imgids $=["$matrix$",$ "warrior"$]$

for imgid in imgids:

print$("./$p$3/"+$imgid$+"/"+$imgid$+"0.$png$")$

I$1=$ imread$("./$p$3/"+$imgid$+"/"+$imgid$+"0.$png$")$

I$2=$ imread$("./$p$3/"+$imgid$+"/"+$imgid$+"1.$png$")$

cor$1=$ np$.$load$("./$p$3/"+$imgid$+"/$cor$1.$npy$")$

cor$2=$ np$.$load$("./$p$3/"+$imgid$+"/$cor$2.$npy$")$

rectified$\_$im$1,$ rectified$\_$im$2,$ rect$\_$cor$1,$ rect$\_$cor$2=$

$4$mage$\_$rectification$($I$1,$ I$2,$ cor$1,$ cor$2)$

rect$\_$F $=$ fundamental$\_$matrix$($rect$\_$cor$1,$ rect$\_$cor$2)$

plot$\_$epipolar$\_$lines$($rectified$\_$im$1,$ rectified$\_$im$2,$ rect$\_$F$,$ rect$\_$cor$1,\backslash$sqcup

rrect$\_$cor$2)$