DEMYSTIFYING MODELLING: QUANTITATIVE MODEL$$S

One of the many impacts of the COVID$-19$ crisis has been to highlight the role of quantitative models in our lives. Ideas

associated with modelling, such as flattening the curve of disease transmission, are now regularly discussed in the media

and among families and friends. Across the globe, we are trying to understand the numbers and what they mean for us$.$

Forward$-$looking models aren$$t new. They have long played an important but unseen role in day$-$to$-$day life$$for instance, in

pricing homeowners$$ insurance, anticipating the weather, and deciding how many iPhones to manufacture. However, in the

COVID$-19$ pandemic, the scale of impact and the level of uncertainty have introduced new challenges$$and notoriety$$for

modelers. Used properly, models provide information that can present a framework for understanding a situation. But they

aren$$t crystal balls that state with certainty what will happen, and they don$$t in themselves answer the difficult question of

what to do$.$ The eminent British statistician George Box summarized the point with his famous aphorism: $$All models are

wrong, but some are useful.$$ And he refined it by saying, $$Since all models are wrong, the scientist must be alert to what is

importantly wrong. It is inappropriate to be concerned about mice when there are tigers abroad.$$

Making decisions in the face of uncertainty is challenging, particularly during a pandemic. Quantitative models can help us

understand systems and behaviors in several useful ways that help navigate this ambiguous environment. Models structure

data in support of reasoned decision making by restricting variables to those that matter for a particular question. For

example, when developing a demographic model to help civic leaders plan future community needs, key drivers could be

birth rates, death rates, and new$-$job creation. Models can help users understand what is known about each element and

identify the areas of continuing uncertainty.

Models are well suited to exposing sensitivities: they show how even small changes in key assumptions can produce large

variations in outcomes, helping decision makers establish priorities. An obvious case in point related to the COVID$-19$

pandemic is the massive impact of even small adjustments in the transmission rate of infection. By establishing

sensitivities$,$ models pinpoint areas for investment of effort or money to reduce uncertainty. Models expose how different

assumptions lead to different outcomes. Through discussion of modelling results, decision makers can form a collective

judgement on scenarios to plan for, based on the multiple variables considered, and thus reach practical decisions. For

example, models were used to enable policy makers to weigh the benefits of requiring seatbelts against the moral hazard of

encouraging people to drive faster. Not only do models trigger discussion, but they may force a more nuanced and

evidence$-$based approach to decision making. In many cases, that is more important than the specific output itself.

A model is simply a tool, and, as with any tool, its value highly correlates with the way it is used. Models can be broken

down into three main components: raw data, assumptions that define what the model does with the data, and final output.

The relative importance of assumptions and data varies by model. Google Search$$s autofill, for example, is mostly data

driven, while the adage about waiting an hour before swimming is driven by assumptions. Each part must be viewed with a

critical lens$$ failure to do so can lead to poorly informed decisions. A model is only as good as its underlying data, and data

in a time of extreme uncertainty, such as a global pandemic, present a serious challenge. Just as rotten ingredients won$$t

produce a tasty dish no matter how good the recipe is$,$ poor data lead to poor output from a model. Data can be wanting for

various reasons: too few data points, inconsistency, inaccuracy, or incorrectly generalizing from a particular data set.

Modelling anything related to a novel virus entails the risk of using bad data. Virtually all the data series being collected

about the COVID$-19$ crisis are incomplete or subject to caveats. For example, using data on the impacts of the COVID$-19$

pandemic in one geography to model potential impacts in another community can be problematic. Data might not be

generalizable if the populations differ in important dimensions, such as age.

Ultimately, when using models to make decisions or when interpreting their outputs, there are several key questions to ask:

How has this model simplified the world? What inputs does the model require, and how knowable, certain, and stable are

those inputs? What are the outputs telling us$,$ and what is the level of uncertainty? And lastly, how have users engaged with

this model in the process of making decisions? Satisfactory answers to th