Estimating Monte Carlo Runs and Error Terms

Theoretical Estimate of Runs and Convergence

Monte Carlo is a statistical measure. This means we can estimate the exact number of runs necessary to reach a given confidence interval in order to measure risks of a portfolio.

In a Monte Carlo simulation the standard error of the mean of the distribution is:

where

z= to the number of standard deviations necessary to reach a TWO tailed confidence interval under the normal distribution. For a 95% confidence, z=2. With 99% z=3.

= is the portfolio’s standard deviation.

Runs = the number of runs in the Monte Carlo simulation.

From this one can see that the error term can be reduced either by:
decreasing the numerator  
            or 
increasing the denominator.

In the first case, we must improve the distribution’s volatility estimate. This is done by implementing  importance sampling, stratified sampling, variance reduction etc.

In the second case, we must increase the sheer number of runs, which is akin to brute force.  The problem with the latter is that accuracy only improves  as the square root  of the ratio of the number of additional Runs !
Hence, if say, we are running 1000 simulations and we want to reduce the error term by 10 we must actually increase the number of simulations by 100!

We must therefore set our number of runs to 100000 in order to achieve an improvement of one order of magnitude!
 
For example, if the portfolio has a standard deviation of 15% and we a running 1000 simulations, we have  95% chances that the true mean of the distribution lies within 1% of our estimate with 1000 runs.
 (2*0.15/1000^0.5)=0.9045 %.

Now, if we increase the number of runs by one order of magnitude, our error term will be (2*0.15/10000^0.5)=0.3 %, which indeed reduces the error term by  0.9045/0.3 (1000/10000)^0.5=10^0.5=3.16

Empirical  Estimate of Quality of Runs and Convergence

A practical estimate usually starts at one standard deviation of the risk measure.  
This approach is somewhat akin to a tracking error.
We then move to two standard deviations with two different portfolios in order to estimate stability. Once this desirable property has been obtained we can compute the optimal number of runs for our portfolio.

Convergence estimate is very different from  back testing. This approach only describes convergence of the mean error during Monte Carlo simulation, not the results themselves.

The procedure which is usually carried out quarterly or semi-annual starts off by defining at least two (or more) portfolios that are well distributed across asset classes and instruments. If we are running credit we will also ensure they are well distributed across counterparty, master agreements and countries.

The first portfolio should include a smaller sample size than the second. You can obviously proceed with more than two portfolios, provided coverage across asset classes and products is different.

The portfolio is then run with, for example 1000, 2000, 5000 & 10000 runs for Market Risk. A minimum number of 50 000 simulations is recommended for Credit Risk.

Each Simulation is then run 20-30 times.

For each Risk Factor, accumulate the mean “risk factor” and then compute the volatility in percent terms. You then plot the results for each portfolio with one and two standard deviations.

Once you have plotted results, you can identify rapidly outliers.

For each estimate, if the mean of the larger sample portfolio is not within two standard deviations of the smaller portfolio,  your portfolio is not stable enough to draw any acceptable conclusion.

In this case, the largest portfolio is either not well distributed across asset classes and products, counterparties, etc or you do not have enough positions in your portfolio. If this is the case, you must start over until results present sufficient stability.

 Once mean and standard deviations of the risk  measures are stable enough, you can proceed to compute the required number of runs.

 Finding the appropriate number of runs for your own organization is extremely simple:

For each portfolio, plot the standard deviation from the mean by connecting the points between each number of runs.

The numbers on your graph should show a clear relationship between a decrease in standard deviations and an increase in number of runs. 
In this case we assume standard deviations and numbers of runs can be interpolated linearly. 

From this you should be able to pinpoint immediately the necessary number of runs needed to reach the percentage error you are seeking !