Uses Python!

We must construct a mosaic from a central image and a set of peripheral images since we then need to find just one homography for each peripheral image. You should flesh out the code below to allow the user to manually select at least 4 pairs of corresponding points between each central and peripheral image. These points should be located in distinctive locations that you can easily identify between the images such as high contrast corners.

Write code below that loads in the central image and then loops over the remaining images and for each image allows you to select four or more points. You should encapsulate the basic procedure of getting the point correspondences in a function called get_correspondences to make it easier to run on different sets of images to produce your final results

Some code is provided which takes care of getting user clicks in the file selectpoints.py. It takes as input the plot axis and the number of points you want from the user. As you click, the points are numbered so that you can make sure points correspond in the two images you are trying to align. sp = select_k_points(ax,npoints)

Once you have finished marking npoints locations in the plot, you can access their coordinates via the fields sp.xs and sp.ys

------------------------------the code----------------------------------

%

matplotlib notebook

#import a few key modules, these should be sufficient to complete the assignment import numpy as np import matplotlib.pyplot as plt from scipy.interpolate import griddata from scipy.ndimage import gaussian_filter from scipy.ndimage import zoom import pickle

#import the provided functions to support point selection in the notebook import selectpoints

def get_correspondences(imnames,baseim,npoints=4): """ The function loads in a set of images which are going to be assembled in to a mosaic. It displays the central (base) image along with each peripheral image and allows the user to click to to select corresponding points between the base image and the peripheral image. It returns the images along with the user input. If the original image files were in color the function converts them to grayscale. Parameters ---------- imnames : list of str Filenames of image files that are going in to the mosaic baseim : int An index which specifies which of the image files is the base image npoints : int How many points are required from the user. Defaults to 4 Returns ------- imgs : list of 2D float arrays The arrays for the corresponding images given in imnames. These are gray scale images represented as floats. pointmatches : list of SelectPoints objects Returns an object whose fields xs and ys contain the point coordinates once the user has clicked (see selectpoints.py) """

nimages = len(imnames)

#loop over images and load in each one and convert to grayscale imgs = list() for fname in imnames: print('loading...',fname) I = plt.imread(fname) #convert to float data type if necessary # downsample the image to 1/4 resolution just to make things run quicker I = zoom(I,0.25) ## your code here #convert from color to grayscale if necessary ## your code here #finally, store the array in our list of images imgs.append(I) #loop over each pair of overlapping images and have the user #click to specify corresponding points pointmatches = list() for i in range(nimages): if (i==baseim): continue fig = plt.figure() #select points in base image ax1 = fig.add_subplot(2,1,1) ax1.imshow(imgs[baseim],cmap=plt.cm.gray) sp1 = selectpoints.select_k_points(ax1,npoints)

#corresponding points in overlapping image ax2 = fig.add_subplot(2,1,2) ax2.imshow(imgs[i],cmap=plt.cm.gray) sp2 = selectpoints.select_k_points(ax2,npoints) pointmatches.append((sp1,sp2))

return imgs,pointmatches

imnames = ('atrium/IMG_1347.JPG','atrium/IMG_1348.JPG','atrium/IMG_1349.JPG') baseim = 0 #index of the central base image that we will align everything to

#make the default figure size larger to make clicking points easier #feel free to adjust this to suit your monitor / display size plt.rcParams['figure.figsize'] = [9, 14]

#call your function to get user clicks imgs,pointmatches = get_correspondences(imnames,baseim)

#reduce the figure size a bit for visualizations later in the notebook plt.rcParams['figure.figsize'] = [9, 9]

# collect together all the point coordinates into numpy arrays nimages = len(imnames) srcpts = list() basepts = list() for i in range(nimages-1): basepts.append(np.array((pointmatches[i][0].xs,pointmatches[i][0].ys))) srcpts.append(np.array((pointmatches[i][1].xs,pointmatches[i][1].ys))) # save everything out to a file so we don't have to click again. f = open('atrium_correspondeces.pckl','wb') pickle.dump((baseim,imgs,srcpts,basepts),f) f.close()

# later on we can quickly read back in the variables and visualize them f = open('atrium_correspondeces.pckl','rb') baseim,imgs,srcpts,basepts = pickle.load(f) f.close()

fig = plt.figure()

#plot all the base points on the base image fig.add_subplot(2,2,1).imshow(imgs[0],cmap=plt.cm.gray) for i in range(2): ## your code here

#plot the src points on the peripheral images for i in range(2): ## your code here

plt.show()

--------------------------selectpoints.py-----------------------------

import matplotlib.pyplot as plt

class SelectPoints: """ Class that encapsulates allowing the user to click on a matplotlib axis and renders the points they clicked on """ def __init__(self,ax,npoints): self.ax = ax self.pts, = ax.plot([0],[0],'r.') self.npoints = npoints self.xs = list() self.ys = list() self.cid = self.pts.figure.canvas.mpl_connect('button_press_event',self)

def __call__(self, event): #ignore clicks outside the figure axis if event.inaxes!=self.pts.axes: return #otherwise record the click location and draw point on the plot self.xs.append(event.xdata) self.ys.append(event.ydata) self.pts.set_data(self.xs,self.ys) self.ax.text(event.xdata+150,event.ydata,('%s'%len(self.xs)),bbox=dict(facecolor='red',alpha=0.3),verticalalignment='top') self.pts.figure.canvas.draw() #once we have npoints clicked, stop listening for #click events so that we don't accidentally add more if (len(self.xs) >= self.npoints): self.pts.figure.canvas.mpl_disconnect(self.cid)

def select_k_points(ax,npoints): """ Function to allow for interactive selection of points, displaying a number along side each point you click in order to make it easier to maintain correspondence. Parameters ---------- ax : matplotlib figure axis Indicates the axis where you want to allow the user to select points npoints : int How many points are needed from the user Returns ------- SelectPoints object Returns an object with fields xs and ys that contain the point coordinates once the user clicks """

ax.set_title(('click to select %d points' % npoints)) selectpoints = SelectPoints(ax,npoints) plt.show() return selectpoints

-----------