Modeling the Reality of Risk: The Cornerstone of Enterprise Risk Management

The Problem with Standard Statistical Modeling Tools

The risk modeling tools most used today are based on statistical methods. Managers generally know that the tools are "imperfect" for modeling operational risks. But in our view, the standard statistical modeling tools are also inadequate for modeling financial risks. In fact, they expose managers to the very risks they were trying to avoid by using the models—poor decision-making. Fortunately, there are better tools available: "structural" models of risk that capture cause-and-effect relationships between risk factors and outcomes. Using structural risk models enable managers to develop, with confidence, appropriate risk mitigation strategies for their organizations.

Statistical tools look like plausible approaches to modeling enterprise-wide risks because they use hard data captured in databases over fairly extensive periods of time. Financial managers have used these tools to model individual financial risks beyond the direct control of their organizations. These include such macroeconomic risks as changes in interest rates and exchange rates, and asset performance. They also include insurable risks, such as mortality and property/casualty claims.

When financial managers, especially in banking, have been challenged to treat financial risks more holistically, their approach has generally been to simply aggregate the risks. That is, they build models of each financial risk separately and then combine the risks using a statistical approach.

For example, if the risks were interest rates, equity returns, and liability volatility, these financial managers would build statistical distributions representing each of the risks and then mathematically combine (i.e., convolute) the distributions. To do that means specifying the form and parameters of each of the three risk distributions and the nature of the linkage between them. If the probability distribution for each risk is part of a family of distributions with special mathematical characteristics—such as symmetric distributions with constant covariance—the manager determines the aggregate distribution by simply doing the math.

That couldn't be simpler, faster, or easier to implement. But it also can lead to incorrect decisions, as we will discuss in a moment.

Using Statistical Modeling Tools with Operational Risks

What is not so simple, managers find, is using the statistical modeling tools with operational risks. Those are the risks that arise from such things as the entry of a new product or company into a market, poor business judgment on the part of a senior manager, or the decision to use a new product distribution system such as the Internet (or even direct telemarketing). Financial managers who are comfortable with using statistical tools to model financial risks find themselves frustrated when trying to use those tools with these sorts of risks. The problem, they say, is that there is not enough historical data on operational risks to build valid statistical models. The solution, they say, is to start building databases of operational risks—and many of them, especially in the banking industry, have begun to do exactly that.

In our view, they will never get there on that horse alone. That's because the problem is not with the richness of the data, but with the adequacy of the tool. Statistical tools don't work for operational risks because those risks, by their nature, do not lend themselves to a simple statistical description. Moreover, the inadequacy of statistical tools for modeling operational risks actually reveals the inadequacy of those tools for modeling integrated financial risks. In both cases, the statistical tool fundamentally doesn't work because it oversimplifies reality. As Einstein once said, "Solutions should be simple, but not too simple."

Take the integration of financial risks, for instance, which many financial managers assume can be represented by aggregating the individual financial risks as we described above. If you think about it a moment, it is pretty clear that reducing the interrelationship between two risks to a single number—their correlation—is too restrictive to capture the nature of most risks in the real world.

For example, consider the behavior of equity markets around the world. At most times, the behaviors of the stock markets in New York, London, and Tokyo are only partially correlated. While they react to one another, most of the time they also react to local conditions. But sometimes they do move in almost lockstep, when, for example, one of the markets declines precipitously. The usual statistical models do not and cannot capture this sort of behavior; they are simply incapable of representing the varying levels of correlation among the markets that depend on different, complex circumstances.

The Limitations of Using Statistical Models with Operational Risks

The weakness of statistical models is even clearer in the case of operational risks, again because of their fundamental nature. In the first place, operational risks vary significantly, based on how a company manages its internal operations, so the data needed to apply standard statistical approaches would need to be company-specific. It couldn't merely be industry-specific. Moreover, the data should be representative of the current operations environment. Because these operations are dynamic, changing with adjustments to the basic business model, as well as technology and work processes, the likelihood of gathering sufficient representative data is fairly remote.

Second, operational risks are managed through changes in processes, technology, people, organization, and culture. They are not generally managed by using financial tools such as hedging in the capital markets. Managers want to know how operational risks would change if they were to implement alternative operational decisions. It's highly unlikely that historical data—the foundation of statistical approaches—will be, or can be, segmented on that basis.

Third, operational risks are of two sorts: event risks and business risks. Event risks are isolated occurrences that generate loses, such as a technology failure. Business risks are created by business decisions. Although it might be possible to gather sufficient historical data on event risks to build a rough statistical model, it is highly unlikely managers could do the same thing for losses that arise from business decisions.

In short, the very nature of operational risks makes them ill suited for statistical modeling. It's the problem with statistical modeling we saw in integrated financial risk modeling writ large. For both financial risk modeling and operational risk modeling, financial managers need much more flexible, robust tools. Those tools are a family of approaches called structural modeling.

Structural Modeling Methods for Operational Risks

Structural methods differ from statistical models because they simulate the dynamics of a specific system by developing cause-effect relationships between all the variables of that system. Structural models can range from the very mathematically rigorous, such as stochastic differential equations (particularly useful in modeling complex financial risks), to methods that rely on a mixture of mathematical calculations and expert opinion, such as system dynamics simulation, fuzzy logic, and Bayesian belief networks (BBNs). These latter methods are especially useful for modeling operational risks.

Using structural methods for modeling financial risks solves the problem of oversimplifying complex, variable relationships. For example, financial managers can model the complexity of a wide range of macroeconomic risks—including short- and long-term interest rates, real GDP, price and wage inflation, equity earnings yield and the like—by representing their interaction in a "cascade" structure in which each variable in the cascade is dependent on the variables "above" it.

That arrangement captures the causal relationship among the variables. And, luckily, experts have already figured out most of the macroeconomic relationships higher up in the cascade. The task of the financial manager is to link these relationships to those company-specific risks at the bottom of that organization's cascade.

Structural approaches to modeling financial risks are especially useful in a multi-period context. A stochastic scenario generator, for instance, can simulate internally consistent paths for interest rates, inflation, equity markets, currencies, and the like over multiple time horizons. Using structured equations, managers can induce appropriate levels of mean reversion, spread reversion, etc. The approach is clearly superior to assuming that these variables behave in a "random walk" over time.

Generating scenarios using a structural model eliminates the constraints on risk modeling associated with statistical models, such as limitations on the form of probability distributions and constant correlation. Thus, structural modeling provides a much more reliable representation of the nature of financial risks and their interaction. It allows financial managers to make much sounder decisions, decisions based on how financial variables actually behave in the real world.

System Dynamics Simulation

As useful as structural methods are for capturing the reality of financial risk, they may be the only way to model the reality of operational risks. System dynamics simulation, developed in the 1950s by Dr. Jay Forrester at MIT, is a particularly robust structural method and illustrates the value of all such approaches.

The starting place for a system dynamics simulation is expert "testimony," which overcomes the problem of the lack of historical data. The model builders gather information from experts in a given domain in order to develop a graphical system map of the cause-effect relationships among the key variables to represent the dynamics of a specific risk. A system map for the risk associated with the decision to use the Internet as a distribution channel to launch a new product, for instance, would capture the impact of variables such as brand name, marketing, and advertising expenditures, complexity of product features, use of a financial services portal, process cycle times, availability of on-line support, and the like.

Then, the model builders quantify each cause-effect relationship, using a combination of available historical data and expert input, to again overcome the problem of gaps in the historical data and to adjust for data that may not be representative. To the extent that the expert input is uncertain, the cause-effect relationship is represented as a probability distribution around a point estimate.

Next, the model builders run simulations to develop a range of outcomes for key operational and financial variables. The output across the simulation runs is summarized as a probability distribution of financial variables. The probability distributions represent the operational risk, given the current operating environment.

Finally, the model builders perform "what-if" analyses by modifying the decision variables that represent changes in operations and then rerunning the simulations. This step allows the managers who are directing the model building to evaluate the effect of alternative decisions on operational risk. They can then make and implement the decision that will be most likely to get them the outcome they want, at the risk level they and their organization can tolerate.

Other Advantages of Using Structural Models with Operational Risks

Structural models have other advantages, over and above avoiding the limitations of statistical methods. These include facilitating—or "forcing"—interaction among managers in an organization who normally don't think about how their individual decisions affect one other and the organization as a whole, as well as focusing the members of an organization on the specific kind of data they actually need to clarify the assumptions on which they base their key decisions (a much more cost-effective approach than trying to "learn everything about everything").

But, from the broad standpoint of enterprise risk management, the fundamental value of structural modeling is that it removes the biggest barrier to managing risks holistically: it's the tool managers have been looking for.