2. [20] Consider a linear machine which performs supervised learning. Suppose we regard its weights as probabilities, so that b + d =1 wi =

2. [20] Consider a linear machine which performs supervised learning. Suppose we regard its weights as probabilities, so that b + d

κ=1 wi

= 1. Formulate learning as MaxEnt, where the entropy is maximized under the constraints imposed by the training set. Discuss the corresponding solution.

Hint: The constructed solution is somewhat opposite with respect to sparse solutions.