Non-Negative Matrix Factorization

Non-negative Matrix Factorization (NMF) is a typical linear system and a matrix factorization problem. Here we use least square technique to find solutions for NMF. At the same time we use an image data set as an example to show how NMF works and how to use some library routines to process images.

1) In science and engineering fields, a data set can be represented by a matrix. For example, an image which has 196 pixels can be represented by a 14 by 14 matrix or by any other two dimensional matrix. If we have a set of images, we can use a 3 dimensional matrix to represent it. For example, there is a set of 16 images, each image has 625 pixels. Then this set of images can be represented by a 25 x 25 x 16 matrix. Given a data set, do the following operations. a) Check the dimensionalities of the given data set using library routine size().

b) Define a two dimensional matrix to represent the given data set. Represent each image as a vector, a column of a matrix, all the images together form a two dimensional matrix. (Hint: search library function: reshape()).

c) Define a constant as 16. This constant is the second dimension of factor matrix W, and the first dimension of factor matrix H. Then define W, and H matrices. In Matlab, a two dimension matrix with all 0 values can be defined as A = zeros(m,n), m, n are the dimensionalities of A. If defining a three dimension matrix with all 1 values we can define it as A = ones(m,n,k), m, n, k are the dimensionalities of A, k is the third dimension. Here we should use rand() to generate W, H matrices with random values.

d) Set up an iteration criterion. One way is simply to set up a big enough number as iteration criterion, the other way is to compute the residual of VWH, set an acceptable residual value as the iteration criterion. e) Given the updating rules of NMF, provide the corresponding Matlab codes to implement these updating rules. The updating rules should be repeatly run until it meets the iteration criterion.

2) Show some samples of original images in the image set. Show the factor matrix W, and H. For W, each column actually represents an image. (Hint: first convert column vectors in W to matrices, then display the matrices as images.) a) How to show images. Try library functions imshow(), (imagesc(), colormap()). (Hint: when showing an image, the image should be in a matrix format but not vector format) b) How to draw a figure. Use library function plot(), subplot() .