Combining risks II: zero-inflated distributions

Zeroes mess things up

Imagine I'm modeling risk: I have a set of low-probability risks, each of which has a distribution of possible losses, and each of which is independent. It's possible that none, some, or all of these risks happen. What I want is to know the overall risk distribution, in other words, what's the probability of losing $x? If each of these risks was certain to occur, it would be easy to combine them, but the presence of zeroes really messes things up as we'll see.

Let me make the problem clearer by laying out a simple case where we're combining the losses from two events:

• for an event \(F\), if it happens, there's a probability distribution for the range of losses \(f\), the probability of the event happening is 5% (\(P(F)\)),
• for an event \(G\), if it happens, there's a probability distribution for the range of losses \(g\), the probability of the event happening is 6% (\(P(G)\)),

We have four possibilities:

1. None of the events occur. 
2. Event \(F\) occurs and event \(G\) does not occur.
3. Event \(G\) occurs and event \(F\) does not occur.
4. Event \(F\) and \(G\) occur.

How do we model the overall loss distribution? Let's start with the simple part.

Combining probabilities

Working out the probabilities is standard and simple probability theory: 

1. None of the events occur: \(1 - P(F))(1-PG)))\)
2. Event F occurs and event G does not occur: \(P(F)(1-P(G))\)
3. Event G occurs and event F does not occur: \(P(G)(1-P(F))\)
4. Event F and G occur: \(P(F)P(G)\)

The sum of these four probabilities will be 1.

All of this doesn't tell us anything about the overall distribution of losses; how likely is it I will lose $x?

If there were no zeroes

If there were no zeros, combining the loss probabilities would be simple. In this case, we would have to assume the events did occur (so, \(P(F) = P(G) = 1\)). The overall loss distribution \(h\) is given by:

\[ h = f * g = \mathcal{F}^{-1}(\mathcal{F}(f)\mathcal{F}(g)) \]

in other words, it's the convolution of the distributions \(f\) and \(g\), where \(\mathcal{F}\) is the Fourier Transform.

Model zeroes with the dirac delta function

The problem comes down to how we represent zero losses as a continuous function so we can use the same math we use to handle continuous probability distributions. Fortunately, quantum physics has solved that problem for us with the dirac delta function. This isn't really a function, but I won't dig into that too much. 

Here's its definition.

\[ \delta(x) = \begin{cases} +\infty, & x = 0 \\ 0, & x \neq 0 \end{cases}\]

Here are some properties we're going to use later.

\[ \int_{-\infty}^{\infty} \delta(x)\,dx = 1 \]

\[ \int_{-\infty}^{\infty} f(x)\,\delta(x)\,dx = f(0) \]

\[ \int_{-\infty}^{\infty} f(x)\,\delta(x-a)\,dx = f(a) \]

\[ \delta(x) = \lim_{\epsilon \to 0^+} \frac{1}{\epsilon\sqrt{\pi}} e^{-x^2/\epsilon^2} \]

Zeroes and probability distributions

I started this blog with an example:

• for an event \(F\), if it happens, there's a probability distribution for the range of losses \(f\), the probability of the event happening is 5% (\(P(f)\)),

Here's how we can represent this using the dirac delta function:

\[ h(x) = P(F)\,f(x) + (1-P(F))\,\delta(x) \]

In words, this says we multiply the dirac delta function by the probability of it happening (\((1 - P(f))\)) and \(f\) by the probability of it happening (\(P(F)\)). We can see why this is so from the equation below.

\[ \int_{-\infty}^{\infty} h(x)\,dx = P(F)\int_{-\infty}^{\infty} f(x)\,dx + (1-P(F))\int_{\infty}^{\infty}\delta(x)\,dx = P(F) + (1-P(F)) = 1\]

As an aside if the dirac delta function is at \(a\) instead of zero, then:

\[ h(x) = P(F)\,f(x) + (1-P(F))\,\delta(x-a)\]

What this means is we have a way of representing the zero loss case and the loss case together. This is called a zero-inflated probability distribution, If we were to plot it as a probability distribution, we'd see a spike at zero and the probability distribution \(f\).

Zero inflated probability distributions and overall risk

Two risks

To go back to my starting example, we have two risk, \(h\) and \(j\) which are zero inflated probability distributions. We know that:

\[ h(x) = P(F)\,f(x) + (1-P(F))\,\delta(x) \]

\[ j(x) = P(G)\,g(x) + (1-P(G))\,\delta(x) \]  

To get the combined probability distribution we use convolution and Fourier Transforms:

\[ k = h * j = \mathcal{F}^{-1}(\mathcal{F}(h)\mathcal{F}(j)) \]

which is:

\[\begin{equation} \mathcal{F}\{h\}\,\mathcal{F}\{j\} = pq\,F G + p(1-q)\,F + (1-p)q\,G + (1-p)(1-q). \end{equation}\]

We can define these symbols:

\[F(\nu) = \mathcal{F}\{f\}(\nu)\]

and:

\[G(\nu) = \mathcal{F}\{g\}(\nu)]\]

Using these relationships:

\[\mathcal{F}^{-1}\{FG\} = f * g\]

\[\mathcal{F}^{-1}\{F\} = f\]

\[\mathcal{F}^{-1}\{G\} = g\]

\[\mathcal{F}^{-1}\{1\} = \delta\]

\[\mathcal{F}\{\delta\} = 1\]

We have:

\[\begin{equation} \begin{aligned} k(x) ={}& P(f)P(g)\,(f*g)(x) + P(f)\bigl(1-P(g)\bigr)\,f(x) \\ &+ \bigl(1-P(f)\bigr)P(g)\,g(x) + \bigl(1-P(f)\bigr)\bigl(1-P(g)\bigr)\,\delta(x). \end{aligned} \end{equation}\]

This equation tells us how to calculate the overall probability distribution, and so calculate the risk of losing \($x\) (or more formally, the risk of losses in the range \($x\) to \($y\)).

Notice there are contributions from zero events happening, \(F\) only happening, \(G\) alone happening, and \(F\) and \(G\) happening.

Multiple risks

The formula for two events is annoying, but you can work with it. What happens if there are multiple risks, defined like this?

\[h_i(x) = P(f_i)\,f_i(x) + \bigl(1-P(f_i)\bigr)\,\delta(x)\]

In this case, the overall probability distribution is given by:

\[ k = \mathcal{F}^{-1}\!\left(\prod_i \Bigl[ P(f_i)\,\mathcal{F}\{f_i\}(\nu) + \bigl(1-P(f_i)\bigr) \Bigr]\right) \]

That's much harder to calculate, especially with what I'm going to tell you next.

Fourier pairs and real-world difficulties

In some cases, the Fourier Transform of some distributions gives another distribution, or at least a nice closed formula. Sadly, most of the time, that isn't true. Here's what that means in practice; we start with some nice distributions, we zero inflate them, and then we have a calculation involving their Fourier Transform and the inverse of the combined formula, neither of which is a closed form. So what can we do? The answer is to use discrete approximations.

Using the discrete approximation

Using the discrete approximation involves digitizing the data, i.e., taking samples at regular discrete intervals. To do this, we have to use the same step size and the same starting point for all the equations. Once we've digitized the data, we can use the discrete Fourier Transform to get the Fourier Transform, do the products, then use the inverse discrete Fourier Transform to get to the overall loss distribution. This turns out to be difficult for all kinds of reasons. I'm just going to go over a few of the problems here. Bear in mind, we have finite computers with finite memory, and in most cases, we want the overall distribution calculation to be done quickly.

How do we represent distributions that go off to infinity? Many continuous distributions, including the normal, range from \(-\infty\) to \(\infty\). We can't digitize an infinite range, so we have to decide where to stop and start. A good rule of thumb is to stop when the value of the probability function is below some threshold number, e.g., \(10^{-8}\). What that threshold is depends on the calculations you're performing. Bear in mind that this truncation will change the overall calculations. You have to choose your threshold so that the effect is unimportant.

What grid size (step size) do we choose? If we choose too narrow a grid, computation time goes up a lot. If we choose too wide a grid, we may well miss peaks in the data.

Where do we start the grid from? If we start from the wrong place, we can miss edges of sharp functions. In practice, we start from zero to get the zero-inflated part of the distribution (which also involves another careful calculation).

The list of difficulties goes on. None of them are showstoppers, but what it does mean is that you need a level of expertise to perform these calculations correctly. This is not a place for beginners.

Central limit theorem to the rescue?

The central limit theorem (CLT) may well be helpful here. To put it simply, the CLT says that when you take samples from a distribution and sum them, the distribution of the sum follows a normal distribution. In most undergraduate texts, the CLT is formulated in a restrictive way; they say the samples have to be from the same distribution. That's not actually true; the samples have to be from distributions with certain properties.

In principle then, if we have a large enough number of risks, then the overall risk will follow a normal distribution, and we can figure out the properties of the normal distribution from the properties of the underlying  risk distributions. This would collapse the calculation time by orders of magnitude. To figure out when the CLT applies, we can use the Berry-Esseen theorem.

An alternative to all this

There is a way to get to the overall probability distribution without running into the closed form problem, and that's to use Monte Carlo analysis.

This problem is a particularly simple form of Monte Carlo. At its base, all you do is choose random numbers from the distributions and multiply them. In reality, things are a little more complicated, but still pretty straightforward. In practice, you might use quasi Monte-Carlo to increase the efficiency. 

The problem with Monte Carlo is accuracy. How many samples do you need to take to get an accurate representation of the overall probability distribution? It's likely over 32,000 for any moderately complicated problem, and maybe more. Quasi Monte Carlo gives you a lot of benefits, but ties you to powers of 2 for the sample size.

If an analytic solution is possible, the analytic solution will always be more accurate than Monte Carlo, and it may even be faster to calculate.

Discussion

I have to say, this whole process leaves me unsatisfied. 

I love the fact that we can write down an equation combining multiple risks, but I hate the fact that the digitization process is so awkward; if we could push digitization after the initial Fourier Transforms, that would be so much easier. Simplification comes down to finding the closed-form formula for the Fourier Transforms of distributions, but only a few distributions have closed form formula. 

I really dislike the arbitrary nature of selecting a grid size.

I like the fact that the CLT should kick in and simplify the process if you have enough risks, but I'm uncomfortable because of what this implies for computation time. It means that computation time increases with every risk, up to the point where the CLT kicks in, then computation time falls off a cliff. That feels a little arbitrary to me.

I dislike choosing Monte Carlo because of the effort needed to do it and the risk of an inaccurate result. It feels to me like if you have the equations for the underlying distributions, you really ought to be able to combine them analytically.

Conclusions

You can combine zero-inflated distributions to get overall distributions, but the process is complex and requires a strong knowledge of calculus and advanced probability theory.

AI statement

This blog was entirely human written with no help from AI. I formatted the formula with the help of AI.