Your colleague is trying to evaluate two different forecasts using the MSE method. He says, "I've used an exponentially smoothed forecast for Product $A,$ with an alpha of $0\mathrm{.}70$ for sensitive forecasting, and my MSE is $104,000.$ I ran a weighted moving average forecast for Product B$,$ with three period weights of $0\mathrm{.}70,0\mathrm{.}20,$ and $0\mathrm{.}10,$ with the most weight on the most recent time period. That MSE came out to be $104.$ I$\text{'}$m thinking that because we know in the MSE that this represents the error measure, and that lower errors are better, that zero errors would be the best.

So I$\text{'}$m thinking that my forecast for B is much better, because the MSE is lower."

Do you agree? If so$,$ why? If not, why not?

Select the most correct answer.

Using the MSE is not advisable here because Product A and Product B may have different units. In this setting, we would prefer the RMSE to make the measurement unit$-$agnostic. A lower RMSE is preferrable, but it is more robust than the MSE and is specifically designed for cross$-$domain comparisons.

The MSE is not the preferred error measure; we should instead be focused on the SSE. Likewise, when focusing on the SSE, we would prefer a larger value. The SSE is also unit$-$agnostic, meaning it can be leveraged to compare across products without any issue.

Your colleague should not be trying to compare two distinct types of forecasting methods. We are only able to make meaningful comparisons between forecasts in the same "family". Therefore, for his comparisons to be valid, he should compare, e$.$g$.,$ a weighted$-$moving average for Product A vs$.$ another for Product B using the MSE as a metric.

The MSE is indeed a proper error measure, and lower is better, but we need to do an apples to apples comparison. Namely, the MSEs between Products A and B are not necessarily comparable, e$.$g$.,$ if the products are measure in different units or serve very different purposes.