Existing methods for monitoring risk in financial markets can be enhanced through greater use of independent verification and testing.
Financial regulators use many techniques to carry out their responsibilities. These techniques include rulemaking to establish regulatory standards and procedures, enforcement to impose sanctions when violations occur, and ongoing review of market participants to evaluate compliance and to assess risk.
This article focuses on the methods used by regulators for monitoring risk. These methods can be placed into four levels: review of data, auditing of data, review of risk management procedures, and testing of risk management procedures.
Opportunities currently exist for greater implementation of Level 4 techniques that would enable regulators to conduct more independent assessments of the risks posed by participants in the financial markets. In particular, as the use of clearing grows, Level 4 methods will facilitate the monitoring of that risk.
Level 1 - Review of Data
Regulated institutions are typically required to provide periodic reports to regulators. These reports give information such as the current financial condition of market participants or the positions that they hold in various markets or instruments.
For example, futures commission merchants file detailed reports to the Commodity Futures Trading Commission about their capital and the customer funds that they hold. Derivatives clearing organizations file detailed reports with the CFTC about positions being cleared and the margin posted as collateral for those positions.
Review of reports like these enables regulators to assess the safety and soundness of regulated entities, identify trends, and engage in discussions with market participants. Failure to file, errors in the filings, and questions generated by an analysis of the filings all supply regulators with a basis for further action.
Level 2 - Auditing of Data
Reporting is the foundation upon which financial regulation rests. But it is not enough on its own. Even a painstaking analysis of reports is insufficient for effective oversight. The regulator needs a means to verify the accuracy of the data.
Periodic audits are necessary to confirm the accuracy of information. A key element of any regulatory program is determining the timing, scope and elements of audits. They should be conducted regularly, cover an appropriate range of topics and be calibrated to the risks of the covered entity.
Level 3 - Review of Risk Management Procedures
The complexity of financial transactions has increased in recent decades. There has been a corresponding increase in the difficulty of identifying and measuring risk. Even a comprehensive reporting program coupled with a well-designed audit program may be insufficient to identify, measure and mitigate the risks faced by a financial institution.
Regulators have used a variety of techniques to address this issue. One method is to hold frequent meetings with senior management of the regulated entity. Another is to station regulatory staff on-site at the regulated entity. A third is to impose internal control procedures. A fourth is to require regulated entities to obtain third party reviews such as credit ratings or validations of risk management models.
Level 4 - Testing of Risk Management Procedures
The potential shortcoming of the techniques used at Level 3 is the lack of independent verification of data and testing of assumptions by the regulator itself. Level 4 entails the use of proactive techniques by which the regulator conducts independent assessments of the risks posed by a regulated entity’s business.
The relationship of Level 4 (Testing of Risk Management Procedures) to Level 3 (Review of Risk Management Procedures) is similar to the relationship of Level 2 (Auditing of Data) to Level 1 (Review of Data). While Level 1 and Level 3 involve review and analysis of information provided by the regulated entity, Level 2 and Level 4 involve attempts to test the accuracy of that information.
The CFTC uses Level 4 techniques in its ongoing oversight of cleared futures and swaps markets. It uses information gathered at Levels 1 to 3 in combination with tools that it has obtained or developed internally to measure risks and evaluate whether these risks are being appropriately managed.
As noted, the CFTC receives daily position and margin data in cleared futures and swaps markets as well as monthly capital and customer fund information. The CFTC has an internal labeling system that allows it to identify and to aggregate the positions of market participants across futures and swaps, and across clearing firms and DCOs.
The risk surveillance program contains four primary components:
- identifying traders, clearing members and DCOs at risk;
- estimating the magnitude of the risk;
- comparing the risk to the available financial resources; and
- assessing the risk management practices of the traders, clearing members and DCOs.
CFTC staff take actual current positions, select extreme but plausible changes in market prices, market volatility and instrument correlations, and calculate the potential losses under those conditions. These potential losses are then compared to the margin on deposit and other financial resources available to cover the losses, including the capital of the clearing firm and the guaranty fund of the DCO.
CFTC staff discusses its methodologies and results with individual traders, clearing firms, and DCOs. Staff shares its analysis and asks specific risk questions. Thus, in evaluating potential risks to regulated entities, staff gathers information not only from those entities but also from a wide range of market participants.
A recent example of the use of Level 4 methods is the systemic stress test exercise conducted by CFTC staff in 2016.
There have been instances where staff modified its techniques as a result of these discussions, thus improving the program going forward. There also have been instances where regulated entities increased the financial resources available to cover potential losses as a result of the discussions, thus reducing risk to the system. In this way, the program tests risk at all layers of the system—at the trader, at the clearing member, and at the clearinghouse.
A regulator must design its Level 4 techniques, of course, to fit the circumstances of the institutions, products, and markets being regulated. For example, products traded over-the-counter can raise different issues about valuation and liquidity than products traded on exchanges. But the key point remains that the regulator should be able to conduct an independent assessment of risk under a variety of scenarios.
A recent example of the use of Level 4 methods is the systemic stress test exercise conducted by CFTC staff in 2016. The exercise covered the 15 largest clearing firms at each of the five largest DCOs. CFTC staff designed and executed 11 different scenarios in order to evaluate the ability of DCOs and clearing members to withstand extreme but plausible price and volatility changes across a range of futures, options and swaps products.
Applicability to the Financial Crisis
Although a review of the financial crisis of 2008-2009 may provide some instances of problems at Level 1 (e.g., the swap positions of firms like AIG were not routinely reported), and at Level 2 (e.g., the activities of the Madoff company were not captured by audits), the more pervasive problem seems to have been an excessive reliance on Level 3 techniques and insufficient Level 4 analysis. Financial institutions took on large positions in various instruments that do not seem to have been subjected to independent analysis by regulators of the risks that they posed. For example, in some cases, regulators may have relied too much on credit rating agencies and third party providers of risk model validation.
Perhaps if regulators had conducted independent stress tests of some of the instruments, they might have identified problems earlier. They might then have required firms to increase the amount of capital or margin supporting the positions or to reduce the size of positions before such large losses had accumulated.
Conclusion
Since the financial crisis, many steps have been taken to improve the oversight of the financial services industry by strengthening existing requirements in some areas and by bringing more products and entities within the ambit of regulatory authorities. Perhaps the next step is to employ Level 4 techniques more widely. In order to decrease the likelihood that financial institutions will take on imprudent risks, regulators should have an independent ability to identify and measure risk.
