Management of Model Risk

Banking Supervision Department
Jerusalem, August 21, 2024
Circular Number 2792-06
Attn:
Banking Corporations and Payment Service Providers with Prudential Importance Re: Management of Model Risk (Proper Conduct of Banking Business Directive 369) Introduction

1. This new Directive deals with model-risk management. Models are widely used in all aspects of
   banking corporations’ activity. Alongside the many advantages of using models in business decision-making and risk management, model use carries risks: decision-making based on “erroneous” models and misuse of “correct” models. This Directive describes the main aspects of effective model-risk management.
2. The Directive is based mainly on US regulators’ April 2011 “Supervisory Guidance on Model
   Risk Management” and replaces the Supervisor of Banks’ letter of October 10, 2010 (see Section 13 below). The outlook expressed in the new Directive is that model risk should be managed similarly to other types of risks, including the responsibility of the Board of Directors and senior management to supervise, control, and manage the risk, and the involvement of the three lines of defense in managing the risk. The Directive also reflects, inter alia, a broader conceptualization of the process of model validation.
3. The Banking Supervision Department is aware of the growing use of Artificial Intelligence
   models.2 The use of AI models has advantages but these models may expose banking corporations to new risks or amplify existing ones. Everything stated in this Directive also applies to models that include the use of, or are based on, Artificial Intelligence (hereinafter—“AI models”). When this Directive is applied to AI models, the unique characteristics of models of this type must be addressed.
4. Regulation in this context has not been accompanied by the issuance of an RIA report under the
   Principles of Regulation Law, 5782-2021 (hereinafter—“the Law”) due to the exemption established in Section 34(c)(4), it being based on accepted rules in countries with significant markets. Said regulation will not be subject to ex-post review under Section 36 of the Law unless the accepted rules in the countries on which the regulation is based are revised.
5. After consulting with the Advisory Committee on Banking Business Affairs and obtaining the
   Governor’s approval, I have decided to establish this new Directive as specified below:

1 SR Letter 11-7 Attachment.
2 The term “Artificial Intelligence” is variously defined. For the time being, the Banking Supervision Department leaves the definition to the discretion of the banking corporation, provided the corporation bases itself on definitions that rest on appropriate professional ground.

Main Provisions of the Directive
6. General (Chapter A)
a. The chapter includes an explanation of the background of the Directive and its scope: the Directive applies to all models, as the term “model” is defined in the Directive, including AI models. b. Banking corporations shall apply the Directive in accordance with the complexity and importance of the models that they are using and also in accordance with the type, complexity, and riskiness of each specific model.
c. The Directive shall apply to each banking corporation and each corporation that it controls,
and to payment service providers with prudential importance and corporations that they control. Foreign banks shall apply the Directive in respect of domestic models that the parent bank does not use and to models of the parent bank that undergo major local adjustments. d. The section contains definitions of main terms in the Directive.
7. Model and Model Risks (Chapter B)
a. The term model denotes a quantitative method that applies statistical, economic, financial, or mathematical theories, techniques, and assumptions to process input data into output. The
chapter includes a description of the components and characteristics of a model.
b. Model risk is the potential for negative consequences resulting from decisions based on incorrect or misused outputs and reports. The realization of model risk can lead to financial loss, incorrect business and strategic decisions, or reputational damage.
c. Model risks should be managed similarly to other types of risks. A banking corporation shall
trace the sources of the risk, assess their intensity and scope, and determine ways of coping with the risk. The methods and intensity of model-risk management shall be tailored to the materiality of the model. Model risk is managed at three levels: corporate governance, policy, and controls; model development, application, and use; and model validation. Explanatory Notes The chapter establishes the framework for the Directive, it includes a description of what a model is, and delineates the inherent risks of the use of models. The Directive establishes the underlying principle that model risk should be managed like any other risk. A banking corporation shall manage the risk both at the individual model level and at the aggregate level of all models. Model risks should be identified, monitored, evaluated, and minimized. The following chapters of the Directive describe the tools that should be used for model-risk management.
8. Corporate Governance, Policies, and Controls (Chapter C)
a. Establishing and maintaining rigorous corporate governance, policies, and controls are very important for the effectiveness of model-risk management framework. Even if model development, implementation, use, and validation are appropriate, weaknesses in corporate governance will reduce the overall effectiveness of model-risk management. A strong framework of corporate governance provides clear support and structure for those involved in managing the risks through policies that define relevant risk management actions,

procedures that implement these policies, allocation of resources and mechanisms to assess whether the policies and procedures are implemented as defined. The scope and complexity of corporate governance in the banking corporation shall be adapted to the scope and complexity of model use in the corporation. b. The components of corporate governance in the context of model-risk management include the following:

1. Duties of the Board of Directors.
2. Duties of senior management.
3. Risk appetite, strategy, risk-management policy (including group policy as the case may
   be), and an effective framework of model-risk management and procedures.
4. Definition of duties and responsibilities, including the owner of a model, control of the
   model, and assurance of compliance with policies and procedures.
5. The role of internal audit.
6. Appropriateness of relying on third parties in risk-management processes.
7. Management of the inventory of models in use.
8. Adequate documentation.
9. Model-validation reports.
10. The connection between a new product and model-risk management.
11. Detecting a model.
    Explanatory notes
    The corporate-governance mechanisms that should be in place and applied for model-risk management are described in this chapter. Some are identical to the requirements pertaining to any other risk; others are adjusted to model risks. Corporate governance is a layer in the appropriate management of model risks; appropriate development and validation processes are not enough. To eliminate doubt, the responsibility for having appropriate model-risk management in place belongs to the highest echelons at the banking corporation (as it would with any other risk). The Directive includes, inter alia, requirements relating to appropriate incentives as part of model-risk management. Appropriate incentives should support the attainment of the desired outcome and may include, for example, remuneration, performance metrics, reporting channels, or positioning within the organization.

9. Model Development, Application, and Use (Chapter D)
   The model-risk management framework should include stringent processes of development and application that are carried out from a position of knowledge and consistency with the way the model is used, the purpose of the model, and the banking corporation’s policy. The chapter includes emphasis on stringent development and application processes including:
   a. Appropriate developers' knowledge, experience, and training. b. Clear definition of goals.
   c. Documentation coupled with explanatory material.
   d. Rigorous data quality and suitability. e. Performance of checks in the course of development.

f. Adequate and systematic adjustments, if any, during model-development, along with proper documentation. g. Alignment of the model with the banking corporation’s information systems. h. (Informed) reliance on the use and long-term performance of the model in order to determine whether the model is functioning effectively.
i. The importance of the reports that the model produce.
j. Channeling the processes of development, application, and use of the model toward, inter alia, an understanding of uncertainty and inaccuracy in the model and how to take it into account appropriately. Explanatory Notes This chapter deals with risk management as the model is developed and applied and in the course of its use. It describes the processes and measures that should be invoked to ensure the adequacy of the model that is placed in action and how correct use of the model may be ensured.
10. Model Validation (Chapter E)
a. Model validation denotes the set of processes and actions meant to ensure that a model functions as expected in accordance with the business objectives and uses for which it was designed. Effective validation is one way of effectively challenging a model—a guiding principle in model risk management. Validation should include all components of the model including input, processing, and reportage. It should take place before use of the model begins and should continue regularly for as long as the model is used. b. The model validation process should be implemented irrespective of how the model is developed and used.
c. Model validation should include an appropriate surrounding environment: adequate
incentives; appropriate knowledge, skills, and experience of the validation team; appropriate influence or status of the validation team; and periodic review of each model to determine whether the model is functioning as intended and whether the existing validation actions are adequate, inter alia. d. An effective validation framework should include three core elements:

1. Evaluation of the conceptual soundness of the model, including evidence from the model
   development process;
2. Ongoing monitoring, including process control and benchmarking.
3. Outcomes analysis, including back-testing.
   Explanatory notes
   Model validation is a crucial level of model risk management. It is meant to minimize model risks by detecting model errors, correcting them, and ensuring correct model use. The chapter describes the measures that characterize appropriate validation. The conceptual soundness is assessed in order to reevaluate the quality of model design and construction (on the basis of documentation amassed in the development process). Model functioning and use are monitored regularly in reference to the goals that the model is meant to attain. Regular monitoring includes, inter alia, process verification, benchmarking,

testing the strength and stability of the model (including the research foundation of the model), and analysis of model overrides. Model outcomes analysis is a test based on comparing the outcomes of model use with those in practice, the choice of testing technique is model-dependent. Back-testing is one accepted technique of outcomes analysis. A familiar example of back-testing is the estimation of value at risk (VaR), in which actual profit and loss is compared with the distribution of losses that the model predicts. A significant deviation of performance from forecasts and unexplained volatility in profit and losses from trading activity may indicate that the hedging and pricing ratios are not properly measured by the approach given. Even as it measures the frequency of realized losses beyond a given VaR, at the tail end of the distribution, banking corporations shall use additional tests such as evaluation of anomalous outcomes and testing of actual outcomes against estimated VaRs measured for different levels of significance. As stated in the Directive, it is important to include indices that deliver forewarnings such as a jump in the withdrawal rate or a change in the standard deviation.
11. Supplier Model Validation (Sections 45 and 93)
a. Selection of an appropriate supplier model. b. Validation of the supplier model.
c. Contingency plans in the event of an unavailable or unsupported model.
Explanatory notes
Supplier models are widely used. The guidelines in these sections are given in addition to those in Proper Conduct of Banking Business Directive no. 359A, on “Outsourcing”. Banking corporations shall ensure that the model chosen is suited to its needs and shall integrate the supplier model into their risk-model-management framework. The guiding principle in respect of supplier models is that one should act according to the same principles that are applied to an “internal model,” although the processes may be different. Effect
12. This Directive shall go into effect one year from the date it was officially published (August 21,
2025, hereinafter: “the Beginning Date”); however— a. In respect of a single model that was in use on the eve of the gazetting of this Directive (hereinafter—“existing model”):

1. If it is a material model (as defined by the banking corporation)—within six months of
   the Beginning Date; however, instead of initial validation (insofar as such was not carried out), the banking corporation shall subject the model to comprehensive validation.
2. Other model—within eighteen months of the Beginning Date.
   b. If the banking corporation identifies an existing model after the date on which this Directive is gazetted (August 21, 2024)—it may complete the processes required under this Directive within one year of the date on which it is identified or within one year of the Beginning Date, whichever is later.

Other
13. Supervisor of Banks Letter 10LM0779, “Guideline on Model Validation,” dated October 17,
2010, is canceled from the Beginning Date onward.
File update
14. Attached are update pages for this Proper Conduct of Banking Business file. The update
instructions follow:
Remove page Insert page
--------- 369-1-28 [1] (8/24)
Respectfully,
Daniel Hahiashvili
Supervisor of Banks