Through this problem you will gain an additional insight into the math that enables a more efficient

implementation of aspects related to convolutional neural networks on modern hardware. For exam$-$

ple, a matrix$-$vector multiplication can be carried out very efficiently $($i$.$e$.,$ can be very fast$)$ on GPU's.

In this problem, you need to show, by an algebraic construction, that the convolution operation can

be expressed using only a single matrix$-$vector multiplication and a single vector$-$addition operation.

Specifically, consider the convolution of a $mm1$ input tensor $x($i$.$e$.,$ a $mm$ matrix$)$ with

the kernel $($filter$)K$ of size $kk,$ that results in the $nn$ matrix $Z($convolution operation result,

consisting of $n^2$ elements$).$

Let $g$ be a vector of size $k^{2}1,$ containing all entries of the kernel matrix $K,$ i$.$e$.,g$ is a flattened $K.$

Let $b$ be a vector such that all of its components are equal to the bias $b,$ where the scalar value $b$ is

a bias that applies to each element of the convolution operation.

Let vector $z$ of size $n^{2}1$ contain all elements of the convolution. This implies that $z$ components

could be rearranged $($i$.$e$.,$ vector $z$ reshaped$)$ into $nn$ dimensional matrix $Z.$

Find the algebraic expression for the vector $z($as a function of $Y,g,$ and $b),$ consisting of only a

single matrix$-$vector product and a single vector$-$sum.