Recommendation W on Model Risk Management in Banks

Recommendation W Page 1 of 44
Financial Supervision Authority
Recommendation W concerning model risk management in banks

Warsaw, July 2015

Recommendation W Page 2 of 44
Table of Contents
I. Introduction .............................................................................................................................. 3
II. Glossary of Terms ................................................................................................................. 6
III. List of Recommendations ......................................................................................................... 9
IV. Principles and Organization of the Model Risk Management Process ....................................... 11
Model Risk Management Policy .................................................................... 11
Role of the Management Board and Supervisory Board ................................................................................ 13
Organizational Solutions and Human Resources .......................................................... 14
Internal Regulations ............................................................................................. 15
Management Information System .................................................................................. 18
Role of Internal and External Audit ............................................................ 19
V. Model Risk Management Process ........................................................................... 21
Model Classification ................................................................................................. 21
Elements of the Model Risk Management Process ..................................................... 23
VI. Model Management .................................................................................................. 28
Model Register .......................................................................................................... 28
Phases of the Model Lifecycle .......................................................................................... 29
Data Quality .......................................................................................................... 34
Model Performance Quality ........................................................................................... 35
Escalation Process for Negative Model Performance Verification Results ...... 37
VII. Validation ...................................................................................................................... 39
Independence of the Validation Unit .............................................................................. 39
Scope of Validation ....................................................................................................... 40
Validation Techniques ................................................................................................... 43

Recommendation W Page 3 of 44
I. Introduction
This Recommendation is issued pursuant to Article 137(5) of the Banking Law Act of 29 August 1997 (Journal of Laws of 2015, item 128).
Taking into account, on the one hand, the increasing scope of model usage for internal purposes, and on the other hand, the limited scope of supervisory regulations defining standards for model risk management in a comprehensive and detailed manner, the intention behind issuing Recommendation W is, among other things, to define standards for the model risk management process, including the need to define the framework for this process, including model construction principles and assessment of their performance quality, while ensuring appropriate solutions within corporate governance. Recommendation W is issued primarily for the purpose of:

• clearly defining supervisory expectations regarding the model risk management process, including those arising from Article 85(1) of CRDIV1, indicating to banks the necessity of introducing a systematic approach to model risk management establishing standards applicable throughout the institution (including external models, including group models), compliance with which ensures that the risk of models used by a given bank is adjusted to the bank's tolerance for this type of risk,
• indicating to banks best practices for an effective model risk management process at all its stages and, consequently, improving the quality of model risk management and developing high market standards,
• adjusting the actions taken by banks regarding model risk management to the level of model risk (the principle of proportionality),
• reducing the degree of exposure of the banking sector to model risk and preparing banks for efficient taking of appropriate remedial and corrective actions in the event of the materialization of model risk in the future.
  It should be emphasized that improving the quality of model risk management, which is a natural consequence of faithfully adjusting the course of internal processes in the bank to the requirements formulated in the Recommendation, promotes the development of potential for the possible use by banks of internal methods2, especially thanks to:
• introduction of mechanisms improving the risk measurement process in the bank,
• adjustment of model management actions to their nature, scale, and degree of complexity,
• subjecting models to comprehensive validation processes,
• development of professional knowledge regarding model risk management in the bank,

1 i.e. Directive 2013/36/EU of the European Parliament and of the Council of 26 June 2013 on access to the activity of credit institutions and the prudential supervision of credit institutions and investment firms, amending Directive 2002/87/EC and repealing Directives 2006/48/EC and 2006/49/EC. 2 i.e. methods used, after prior obtaining permission from the supervisory authority, to calculate capital requirements for credit, market, and operational risk.

Recommendation W Page 4 of 44

• conducting actions aimed at improving the quality of data used by models.
  In view of the above, as well as having in mind the content of Recital 42 of the CRR Regulation3 and Article 77(1) and (3) of CRDIV, the Financial Supervision Authority expresses the expectation that as a result of the implementation of the Recommendation and the improvement of the quality of model risk management, banks will take actions aimed at seeking broader use of internal risk measurement models also within the calculation of capital requirements for regulatory purposes, in order to ensure greater consistency between the level of capital requirements and the level of the bank's risk. The document contains 17 recommendations, which have been divided into the following areas:
• principles and organization of the model risk management process,
• model risk management process,
• model management,
• validation.
  An important issue from the perspective of the efficiency of the model risk management process, in addition to regulating this process from a formal-organizational aspect, is the appropriate adjustment of actions taken by banks to the level of risk generated by models. The risk level of a given model is determined by two components: the significance of the model and the degree of exposure to model risk. The significance of the model should be primarily identified with the scope of model usage in the bank – the greater the share of a given model in processes and decisions implemented in the bank, the higher its significance4. On the other hand, the degree of exposure to model risk is influenced by the broadly understood quality of the model – from data quality, through methodological aspects related to its construction process and the correctness of its implementation into bank systems and processes, to the rational use of its results in processes and decisions implemented in the bank. For this reason, model risk management should mainly focus on active actions rationally reducing the degree of exposure of significant models to model risk, and consequently the level of model risk to sizes consistent with the bank's tolerance for this type of risk. The proper way to reduce the degree of exposure to model risk is the use of appropriate control mechanisms, aimed at reducing the level of residual risk (for models already operating) and inherent risk (for models in the development phase). This is achieved by introducing model management rules by the bank, which – if used effectively –

3 i.e. Regulation (EU) No 575/2013 of the European Parliament and of the Council of 26 June 2013 on prudential requirements for credit institutions and investment firms, amending Regulation (EU) No 648/2012.
4 In the case of banks where no significant models have been identified – which should be identified with a low level of exposure to model risk – expectations regarding the complexity of managing their risk are limited. This is reflected, among other things, by the lack of a requirement to determine the degree of exposure to risk and the risk level of these models and tolerance for model risk.

Recommendation W Page 5 of 44 strengthen the effectiveness of control mechanisms at all stages of the model lifecycle and contribute to the gradual improvement of model performance quality. Due to the complex and multi-faceted nature of activities related to model management, they have been included in a separate chapter in the Recommendation. At the same time, the existence of an independent validation unit in the bank should be considered a key element of this set of rules and a highly effective control mechanism – supervisory expectations regarding the conditions for the functioning of this unit in the bank are presented in the last chapter of the Recommendation. Taking into account the specificity of issues related to model risk management and differences in conditions, scale of activity, risk level of individual banks, and the role played by models in them, the way of implementing the recommendations and the goals indicated in them may vary. Therefore, the descriptions and comments included with individual recommendations should be treated as a set of best practices, which, however, should be applied with due regard to the principle of proportionality. This means that the application of these practices should depend, among other things, on how well they fit the specificity and risk level of the bank, the scale of model usage, as well as on the relationship between the costs of their implementation and the benefits arising from it. In particular, for selected recommendations, specific requirements have been defined that should be met in the case of a bank having significant models – this concerns especially the expectation regarding the application of a comprehensive approach to model risk management and the functioning of a validation unit in the bank. At the same time, the supervisor expects that decisions regarding the scope and manner of implementing the solutions indicated in the Recommendation should be preceded by an in-depth and documented analysis, supported by appropriate arguments. All banks should comply with the recommendations contained in this document, with the reservation that the supervisory expectation is that at least in banks with extensive and complex scale of activity, the model risk management process should be shaped in a way that minimizes the bank's dependence on solutions provided by external entities (including dominant entities) – i.e., mainly in terms of model construction and ongoing assurance of performance quality, taking into account the necessity of having a validation unit with resources characterized by high qualifications and competencies. Furthermore, in the case of cooperative banks, the supervisory expectation is that the banks they belong to should support the implementation of this Recommendation, taking into account the scale and specificity of the activity of a given cooperative bank, applying the principle of proportionality. The scale of activity and solutions used with respect to models should
determine the scope and degree of adopted solutions. However, the implementation process of these solutions in cooperative banks, despite the active role of the central bank, must not contradict the scope of duties and statutory responsibilities of the governing bodies of the affiliated cooperative banks defined in individual recommendations. The Financial Supervision Authority expects that Recommendation W concerning the principles of model risk management in banks will be implemented no later than June 30, 2016.

Recommendation W Page 6 of 44
II. Glossary of Terms
Model Lifecycle – the period during which a model goes through successive phases of its life, from the moment work on its development and implementation begins (i.e., conceptual phase, model construction and quality verification, technical and process implementation, model usage and periodic quality verification, introducing changes), until the model is withdrawn from use or replaced by a new model. Internal Data – all data used in the model management or model risk management process, the source of which are the bank's systems and databases. External Data – all data used in the model management or model risk management process that are not internal data. Model Diary – a set of information allowing to obtain key (from the perspective of model risk management) information about the model, organized in a way that allows third parties to fully reconstruct the history of actions related to a given model and its logic. Internal Model Stakeholders – organizational units, teams, committees, and bodies of the bank that have an impact on the course of the model risk management process or remain under its influence. Model Significance – a feature of the model determined by the bank taking into account at least the significance of the process in which the model is used and the role of the model in this process, as well as the size of the exposure covered by the model's action. Model Risk Categories – areas identified by the bank, specific and distinct, which influence the degree of the bank's exposure to model risk (e.g., divided into: inherent limitations of models; data risk; assumption/methodological risk; administration risk; interdependence risk). Model – a tool serving to prepare a limited (to the most significant dimensions) description of a selected aspect of reality (identifying and approximating relationships occurring in it on the basis of theory or empiricism), the use of which is associated with the risk of the bank incurring losses due to errors in the development, implementation, or use of such a tool. Group Model – an external model used by the bank, in the development of which an entity from the same capital group as the given bank played a significant role. Significant Model – a model that has been classified by the bank into a set of models that play an important role in key processes or decisions implemented in the bank, including due to the size of the exposure covered by the model's action. Non-Significant Model – a model that has not been classified by the bank into the set of significant models.

Recommendation W Page 7 of 44
Internal Model – a model used and developed by the given bank.
External Model – a model used by the bank, in the development of which an entity other than the given bank played a significant role.
Model Monitoring – verification of the effectiveness of the model's operation carried out by the bank unit responsible for the functioning of the model, usually based on statistical measures.
Model Significance Assessment – a measure reflecting the significance of the model, used to assign the model to at least two specified model significance classes (significant model/non-significant model).
Model Risk Level – an objectively assessed model risk level based on the bank's internal criteria, which is influenced by the significance of the model and the degree of its exposure to model risk.
Model Register – a compilation containing ordered – in a uniform format – the most significant, current information regarding all models used by the bank.
Recommendation – Recommendation W.
Recommendation D – Recommendation D concerning the management of information technology areas and cybersecurity environment in banks.
Inherent Model Risk – the level of model risk that exists before taken or potentially applicable control mechanisms.
Model Risk – potential loss that a bank may incur as a result of decisions that may have been largely based on data obtained using internal models5, due to errors in the development, implementation, or use of such models6.
Residual Model Risk – the level of risk that remains despite the application of control mechanisms and subsequent actions resulting from them.
Degree of Exposure to Model Risk – a measure reflecting the cumulative impact of all model risk categories identified by the bank on the correctness of generating a correct result by it in the production environment, used in processes or decisions implemented in the bank. Development Environment – the IT environment in which the model is built and tested.

5 In this context, these are all models used by the bank in internal processes, regardless of the source of the model (internal model or external model, including group model).
6 Definition of model risk according to its definition resulting from Article 4(1)(12) of the CRR Regulation.

Recommendation W Page 8 of 44
Production Environment – the IT environment in which the model is ultimately implemented and used operationally.
Tolerance for Model Risk – the permissible level of model risk accepted by the bank.
Model User – the organizational unit of the bank carrying out activities related to the direct handling and generation of model results or using its results in the performance of its assigned tasks.
Validation – assessment of the effectiveness of the model's operation carried out by a bank unit not associated with the model construction process and its use, usually in a more comprehensive manner than in monitoring, including, among other things: the appropriateness of the model's concept and assumptions to the process or decision mechanism in which the model is used, and the correctness of its construction and implementation from a substantive and formal perspective. Process Implementation of the Model – activities as a result of which the model, after being transferred to the production environment, can be used in processes and decisions implemented in the bank in accordance with its purpose; it includes, in particular, necessary modifications to processes and internal regulations and training of model users. Technical Implementation of the Model – a process in which the model (along with input and output components) is transferred from the development environment to the production environment while maintaining its integrity and functionality. Model Owner – the organizational unit of the bank directly responsible for the development of the model and ensuring appropriate quality of its operation. Principle of Proportionality – a principle according to which model risk management is carried out using resources appropriate to the level of model risk, which are necessary to achieve the intended goal.

Recommendation W Page 9 of 44
III. List of Recommendations
Principles and Organization of the Model Risk Management Process Recommendation 1 The bank should develop and implement a policy on model risk management, consistent with the internal risk management strategy.
Recommendation 2
The bank's Supervisory Board should supervise the functioning of the model risk management process, while the bank's Management Board should ensure its correct and efficient implementation.
Recommendation 3
Significant aspects of model functioning should be approved by the bank's Management Board or a dedicated committee.
Recommendation 4
Organizational solutions and human resources dedicated to the model risk management process should be adequate to the significance of models and allow for effective and timely implementation of actions in this process.
Recommendation 5
The bank should have formalized rules defining the roles and responsibilities of participants in the model risk management process and standards regarding model construction, implementation, use, performance verification, and documentation, as well as the process of preparing data for model construction and ongoing feeding of models. Recommendation 6 Reporting on models operating in the bank and significant actions taken within model risk management should be an integral part of the management information system. Recommendation 7 The model risk management process should be the subject of periodic, independent audits. Model Risk Management Process Recommendation 8 The bank should subject all models used by it to the model risk management process, regardless of the significance of the models, their sources of origin, and the type of data used by them.

Recommendation W Page 10 of 44
Recommendation 9
The model risk management process operating in the bank, integrated with the overall risk management system in the bank, should include actions related to the identification, estimation, control, monitoring, and reporting of model risk.
Model Management
Recommendation 10
The bank should have a model register, and with respect to each model, a model diary.
Recommendation 11
The bank should define model management rules in all phases of their lifecycle.
Recommendation 12
The bank should ensure that data used in all phases of the model lifecycle are of high quality.
Recommendation 13
The bank should ensure that the models used by it are of appropriate quality in each phase of their lifecycle, which requires that their use and effectiveness be subject to regular verification, with a frequency adapted to their specificity.
Recommendation 14
A transparent escalation process for negative model performance verification results should operate in the bank, ensuring efficient taking of appropriate remedial and corrective actions.
Validation
Recommendation 15
In a bank possessing significant models, an independent unit responsible for conducting their validation should operate.
Recommendation 16
The scope of activities carried out during validation and its frequency should be adapted to the specificity of the model and its risk level.
Recommendation 17
Both quantitative and qualitative techniques should be used in the validation process in a complementary manner to ensure comprehensive knowledge regarding the quality of model performance.

Recommendation W Page 11 of 44
IV. Principles and Organization of the Model Risk Management Process
Model Risk Management Policy
Recommendation 1
The bank should develop and implement a policy on model risk management, consistent with the internal risk management strategy.
Model risk, as a specific element of operational risk, should be recognized by banks that use models in their activities as one of the risks inextricably linked with conducting banking business. Consequently – analogously to other types of risk – model risk should be covered by the management process, based on formally established rules allowing for proper identification of model risk and its reliable assessment, as well as ensuring the functioning of appropriate control mechanisms and tools for active management of the degree of exposure to