S Generalised linear models. Let .Q/2 be an exponential family with respect to the statistic T D id and a function a W !

S Generalised linear models. Let .Q/2ƒ be an exponential family with respect to the statistic T D id and a function a W ƒ ! R, see p. 211. (By assumption, a is invertible.)

Also, let n 2 N, ‚ D ƒn, P# D Nn iD1Q#i for # D .#1; : : : ; #n/>

2 ‚, and # be the density of P#. It is assumed that the quantities a.#i / appearing in # depend linearly on an unknown parameter  2 Rs . That is, there exists a real n s matrix A (determined by the design of the experiment) such that a.#i / D Ai; here Ai is the i th row vector of A. The vector a.#/ WD .a.#i //1in is thus given by a.#/ D A, and conversely we have

# D #./ D a1.A/ WD .a1.Ai//1in. Let X D .X1; : : : ; Xn/> be the identity map on Rn. Show the following.

(a) Writing ./ WD E.Q/ and  ı a1.A/ WD . ı a1.Ai//1in, one has grad log #. / D A>

X   ı a

1.A/



:

Therefore, a maximum likelihood estimator O of  is a zero of the right-hand side and can be determined numerically, for example by Newton’s method.

(b) The Gaussian linear model fits into this framework, and the maximum likelihood estimator is equal to the estimator O from Theorem (12.15a).