Optimisation formulations in ML Training = Fitting = Parameter estimation Typical formulation e argmin L(data, 0) * argmin because we want a minimiser not the minimum Note: argmin can return a set (minimiser not always unique!) * denotes a model family (including constraints) * L denotes some objective function to be optimised E.g. MLE: (conditional) likelihood E.g. Decision theory: (regularised) empirical risk Lemma: Consider any objective function L(0) and any strictly monotonic f. * is an optimiser of L(0) if and only if it is an optimiser of f(L(0)). Proof: Try it at home for fun! Optimisation formulations in ML Training = Fitting = Parameter estimation Typical formulation e argmin L(data, 0) * argmin because we want a minimiser not the minimum Note: argmin can return a set (minimiser not always unique!) * denotes a model family (including constraints) * L denotes some objective function to be optimised E.g. MLE: (conditional) likelihood E.g. Decision theory: (regularised) empirical risk Lemma: Consider any objective function L(0) and any strictly monotonic f. * is an optimiser of L(0) if and only if it is an optimiser of f(L(0)). Proof: Try it at home for fun