The Norwegian parliament (‘Stortinget’) has decided to buy 52 new JSF jet fighters type F35. When the decision was taken, the price for one F35 was unknown because the plane was still under construction. Therefore, the Stortinget demanded a total cost forecast that had a 50% chance of being accurate. In 2012, the price estimate was NOK 61 billion (approximately EUR 6 billion). In addition, the Stortinget wanted to have a cost estimate that had an 85% chance of being accurate. This estimate was NOK 72 billion; see https://www.regjeringen.no/no/dokumenter/prop-73-s-20112012/id676029/sec9, Section 2.4.4.

Requesting a probabilistic cost forecast with a 50% or an 85% chance of being accurate is established practice for large (>NOK 500 million) public projects in Norway. Probabilistic forecasts are necessary because the future is uncertain. There are many different possible futures, and some are more likely to come to pass than others. A probabilistic forecast, as opposed to a deterministic forecast, quantifies the uncertainty about future developments. While a probabilistic forecast is not necessarily more accurate than a deterministic forecast, the former contains more information, which is useful for planning purposes.

Let us assume that in a specific area, only a deterministic forecast of the relevant variable(s) is available. This may leave room for political decisions to be made. An example is the ‘shutdown’ of the U.S. government in December 1995. President Clinton proposed a seven-year budget plan that would produce a $115 billion deficit over the seven-year period, according to the economic forecasts of the Congressional Budget Office (CBO). However, the Office of Management and Budgets (OMB) estimated rather more optimistically that the budget would be balanced at the end of the period. Republican leaders demanded that Clinton propose a plan that would be balanced when using the CBO numbers, rather than the OMB’s; see https://www.washingtonpost.com/news/wonk/wp/2013/09/25/here-is-every-previous-government-shutdown-why-they-happened-and-how-they-ended/?utm_term=.4bb10057f0d9, shutdown # 17.

The following example illustrates why probabilistic forecasts are much more useful than deterministic forecasts. Let us assume that we are casting a die. The outcome is random, and has yet to occur. Before we cast the die, someone asks us to predict the outcome. This can be any value between one and six – all outcomes are equally likely. The expected value (equal to the median value in this particular case) is 3.5, which is our best guess. However, the probability that 3.5 will come up is zero. The probability of a result in the interval [2, 5] – i.e., a two, a three, a four, or a five – is two out of three. A die with five faces numbered one, two,…, five has an expected value of three. This is our best guess if we have to predict the outcome. The probability that three will come up is only 20%, while the probability of a result in the interval [2, 4] is 60%.

Now let us assume that we are predicting the population. The outcome is random. Why should we give only one (best guess/expected value, the probability of which is near zero) or three (high, medium, low) numbers, and not the whole set of possible results, like when casting a die? Many would argue that we cannot give the whole set because we do not know the probability distribution. However, we can estimate this distribution. Users should be told what the chance is of the real population falling between the high and the low value: is it 30%, 60%, or even 90%?

In the field of population forecasting, it is established practice to express uncertainty by means of variant predictions for fertility, mortality, and migration (UNECE 2017, 25). This leads to logical flaws in high and low variants of forecast results (Alho and Spencer 2005). Assume that [BL,BH] and [DL,DH] are high-low intervals for births B and deaths D. There is no guarantee that the two intervals will cover an equal probability range. Now assume that the intervals are given a probabilistic interpretation, such as 80% prediction intervals. I=B−D is natural population growth. [BL-DH,BH-DL] then has a higher than 80% probability of occurring (unless B and D are perfectly negatively correlated, which is unrealistic).

Another potential complication is that two variants that may be extreme for a certain variable need not be extreme for a different variable.

To illustrate additional problems that typically arise when forecast variants are used to reflect uncertainty, Table 1 shows the results for the 2014-based population forecast for Norway. The predicted number of people aged 20–66 varies from 3.42 to 4.05 million. The interval between these two extremes is 17.2% wide compared to the number in the variant with medium population growth. For the age group 67+, the interval is slightly wider (24.3%). However, the dependency ratio (defined as the ratio of the number of elderly people to the number of people aged 20–66) has a relative interval width of no more than 7.5%, which is much smaller than we would intuitively expect to observe. The explanation for this surprising finding is that the strong population growth variant combines high life expectancy with high fertility. These two variables move in the same direction for the population numbers in the first and second rows, but they counterbalance each other in the dependency ratio.

Traditional deterministic population forecasts are computed by means of the so-called cohort-component model Kt+1=PtKt, used recursively. Here, Kt is a vector of population numbers broken down by age and sex as of time t; and Pt is a matrix of transition probabilities during the time interval (t,t+1) that account for fertility, mortality, and emigration. Immigration by age and sex is added as a vector. A forecast for the first year is obtained by combining the jump-off population K0 with assumed values of the matrix P0; for later years, the model is used recursively. This forecasting approach was first developed by Edwin Cannan in 1895. By the 1930s, it had become generally accepted by the statistical agencies of many countries (De Gans 1997). In the 1970s, the model was extended to include a regional breakdown of the population vector (multiregional model; see, e.g., Rogers, 1995), or an extra dimension in general, such as educational level, labour market status, or household status (multistate/multidimensional model).

Thus, a deterministic forecast can be derived by combining specific assumptions about fertility, mortality, and migration. Different projections or scenarios can be produced by systematically combining different assumptions. Collectively, those different projections can give some impression of the degree of uncertainty, but not in any quantified way. The probability that an outcome will be above or below a given scenario is unknown (Dunstan and Ball 2016).

Various methods for probabilistic population forecasting have been developed since the 1960s, although Törnquist (1949) was probably the first to integrate probabilistic thinking into population forecasting. In this approach, the elements of the matrix Pt are random variables. Early contributions were made by Pollard (1966), Sykes (1969), Schweder (1971), and Cohen (1986). The initial aim was to find analytical solutions for the predictive distributions of the variables of interest. Due to correlations between components, ages and sexes, and autocorrelations in all variables, approximations were necessary. Later work (e.g., Keyfitz, 1985; Kuijsten, 1988; Lee and Tuljapurkar 1994) was based on Monte Carlo simulation.

In the past two decades, Alho’s Model for Scaled Error has received some general acceptance. The simulation model is extremely flexible in specifying various correlation structures. The corresponding computer program PEP (‘Program for Error Propagation’) is available free of charge. The model assumes normal distributions (in the log scale) for the input parameters of the cohort-component model. See Alho (1998) and Alho and Spencer (2005). In the UPE (Uncertain Population of Europe) project, it was assumed that the expected values of the normal distributions would coincide with the mean variant of deterministic projections. The numerous (co-)variances and (auto-)correlations came from time-series analyses, errors in historical forecasts, and expert judgements. The volatility in the UPE forecast results was constructed to be similar to historical patterns of volatility. See http://www.stat.fi/tup/euupe/upe_final_report.pdf for the final project report and a general project description, and http://www.stat.fi/tup/euupe/index_en.html for stochastic forecast results for 18 countries in Europe.

During the second half of the 1990s, Wolfgang Lutz became one of the pioneers (together with Sergei Scherbov, Warren Sanderson, and Alexander Hanika) of probabilistic demographic forecasting. For example, applications were developed for Austria by Hanika et al. (1997) and Lutz and Scherbov (1998a); for Germany by Lutz and Scherbov (1998b); and for the global population by Lutz (1996) and Lutz et al. (1997, 1998, 2001).

Lutz and colleagues developed the so-called expert-based approach. Its main idea can be summarised as follows. Given a deterministic population forecast with high-low intervals for key forecast indicators (such as the total fertility rate, the life expectancy, or the level of net migration) together with point forecasts for these indicators (medium variant values), all in a future year t, experts can be asked to define the probability content for these intervals, such as 90%. Let us assume that the probability distribution at time t for the variables is normal, and perform random draws from that distribution by Monte Carlo simulation. Variables for intermediate years are found by straight-line interpolation between the value in the jump-off year and the drawn value for year t. In constructing such ‘random slopes’, perfect autocorrelation is assumed. This unrealistic assumption has been relaxed in later work to a time-series approach based on a moving average model (e.g., Lutz et al. 2001; Sanderson et al. 2004). The advantage of relying on the expert-based approach is that it is simple, and is thus easy to explain to demographers and forecast users with little or no statistical experience. However, a major drawback of using this approach is that there is no empirical basis for the (co-)variances implicitly selected by the experts. At the same time, experts tend to be overconfident in their opinions, which could result in the generation of forecasts that have a lower degree of variability than would be desirable; and in the selection of prediction intervals that are too narrow (Alho and Spencer 2005). Such overconfidence is problematic, given that sensitivity analyses by Lutz et al. (1996) have shown that assumptions of perfect or zero correlation between fertility and mortality might have a big impact on prediction intervals for total population size and broad age groups in the long run.

The expert-based approach stimulated further refinements. For instance, Billari et al. (2012, 2014) addressed some of the weaknesses with respect to autocorrelation. They proposed applying a Bayesian interpretation to expert-based predictions, and treating expert inputs as data. While there are good reasons to be sceptical about the use of a method that depends heavily on the opinions of experts (overconfidence, selection), the Bayesian approach is emerging as the state of the art in probabilistic population forecasting (Bijak and Bryant 2016).

Official probabilistic population forecasts have been published by the statistical offices of Italy, the Netherlands (since 1999), and New Zealand.¹ 1

See ISTAT (2018), http://statline.cbs.nl/Statweb/publication/?DM=SLEN&PA=82683eng&D1=0,7,10-12,14-15,21&D2=0-4&D3=0-1,6,11,16,21,26,31,36,41,l&LA=EN&VW=T and http://www.stats.govt.nz/browse_for_stats/population/estimates_and_projections/NationalPopulationProjections_HOTP2016.aspx.

The case of Statistics New Zealand is worth a closer look. As Dunstan and Ball (2016) reported, the shift from using a deterministic approach to using a probabilistic approach is less difficult to make than might be expected. Uncertainty in the different components can be modelled simply or with more complexity. The authors mentioned a number of benefits that the producers and users of New Zealand’s probabilistic forecasts have noticed since such forecasts were first published in 2012. First, while the probability distribution of a certain forecast result is often skewed, this message is not conveyed in a deterministic forecast. For example, the share of elderly people in the future population is usually around 15–20% in developed countries (depending on the definition of ‘elderly’). Since any share is restricted to the interval [0,1], an outcome that is five percentage points below the expected value is less likely than an outcome of equal magnitude above the expected value. Second, the fan chart for a particular forecast result tells us how much more likely small or large deviations from the expected value or the median value are. Third, any user who wants one number can use the median value. Fourth, the stochastic results indicate how far into the future we can reasonably provide a precise forecast, and the horizon is different for each variable. The New Zealand Treasury is developing a stochastic version of its fiscal model in order to get a better idea of how uncertain their variables of interest are.

Indeed, “… a probabilistic projection clearly gives more useful information than the usual high and low variants…” (Lutz and Scherbov 1998a, p. 13).

Finally, as UN Population Division director John Wilmoth stated in 2013: “… I expect that demographers will continue to be surprised by trends that do not follow our prior expectations. It is for this reason that the Population Division has worked hard in recent years to be more explicit and precise about the degree of uncertainty affecting projections of future population trends”. http://www.un.org/en/development/desa/news/population/population-division-director.html

The book ‘World population and human capital in the 21^st century’, edited by Lutz, Butz, and KC and published in 2014, represents an impressive summary of the work by Wolfgang Lutz and his team in the field of forecasting the world population broken down by age, sex, and educational level. It uses techniques of multistate modelling, but the forecasts are entirely deterministic. Here I would like to comment on two points: namely, the role of education and the use of scenarios.

Wolfgang Lutz and colleagues have made major contributions to the field of probabilistic population forecasting. While some of the details can be criticised, the pioneering work of Wolfgang Lutz et al. in the 1990s was important because it showed that probabilistic population forecasting is feasible, and does not have to be very complicated. At the same time, it should be stressed that the current world population forecasts (including education) made by Wolfgang Lutz et al. and the IPCC forecasts based on SSP scenarios will be much more useful and relevant for the user once they are formulated in probabilistic terms.