Management of Model Risk

Supervisor of Banks: Proper Conduct of Banking Business [1] (8/24) Management of Model Risk Page 369-1
Table of Contents
Chapter A: General 369.2
Chapter B: Model and Model Risks 369.3
Chapter C: Corporate Governance, Policies, and Controls 369.6
Chapter D: Model Development, Implementation, and Use 369.12
Chapter E: Model Validation 369.15
Chapter F: Commencement and Transitional Provisions 369.22
Appendix—Examples of Applying the Principles of the Directive
Regarding Artificial Intelligence Models
369.24

Supervisor of Banks: Proper Conduct of Banking Business [1] (8/24) Management of Model Risk Page 369-2
Chapter A: General
Introduction

1. Banking corporations rely on quantitative analysis and models in most aspects of financial
   decision-making. They use models regularly for a wide range of activities, including credit underwriting, exposure assessment, instrument and position evaluation, risk measurement, managing and safeguarding customer assets, determining capital and reserves adequacy, and numerous other activities.
2. The extensive use of models in all aspects of banking corporations' activities reflects the
   contribution of models to business decision-making and risk management. However, models also have costs and their use involves risks. There is a direct cost to allocating resources for the proper development and implementation of models. There may also be indirect costs resulting from reliance on models, such as potential negative consequences (including financial loss) of decisions based on incorrect models or misuse of models. These consequences must be addressed through active model risk management.
3. This directive describes the main aspects of effective model risk management. Proper
   development, implementation, and use of models, as well as rigorous model validation, are essential components of model risk management. Model risk management also includes corporate governance and control mechanisms, such as supervision by the board of directors and senior management, policies and procedures, controls and compliance, as well as an appropriate organizational structure and incentive system.
4. The guidelines in this directive should be implemented by each banking corporation according to
   the complexity and extent of its use of models, its business activities, and its risk exposures, and subject to any law. Additionally, the guidelines in the directive regarding the management of specific model risks will be applied proportionately, according to the type, complexity, and risk of the model, as assessed by the banking corporation.
5. All the provisions of this directive also apply to models that include the use or reliance on artificial
   intelligence (hereinafter – "AI models"). The Banking Supervision Department is aware of the increasing use of AI models. The use of AI models has advantages in terms of efficiency and quality of financial service delivery, operational processes, and risk management. However, these models are liable to expose banking corporations to new risks or amplify existing risks. These risks include, among others, unintended bias or discrimination, which also entail reputational and compliance risks. A banking corporation is required to manage the risks of AI models based on the requirements of this directive, with an emphasis on aspects such as: reliability and fairness of the models in terms of bias or discrimination; accountability adapted to AI models taking into account the degree and manner of human involvement; design of controls in decision-making mechanisms based on these models; transparency, with an emphasis on the ability to explain the achieved results, comprehensive documentation of the various components of the model, and due disclosure so that the rights of those affected by the model to receive information, appeal the results, etc., are not diminished. For examples of applying these principles to AI models, see the
   appendix to the directive.

Supervisor of Banks: Proper Conduct of Banking Business [1] (8/24) Management of Model Risk Page 369-3
6. This directive does not refer to other tools of banking corporations that are not models, such as
calculations. However, it is expected that a banking corporation will apply appropriate control tools to them in line with their use. Application
7. This directive applies to the following corporations (hereinafter in this directive – "banking
corporation"):
a. A banking corporation, except for a joint service company. b. A corporation as noted in Sections 11(a)(3a), 11(a)(3b), and 11(b) of the Banking (Licensing) Law, which is controlled, directly or indirectly, by a banking corporation.
c. A payment service provider with prudential importance license holder as defined in Section
36i of the Banking (Licensing) Law and a corporation controlled by it. d. A branch of a foreign bank, only regarding local models that are not used by the parent bank (internal or of an external supplier) or models of the parent bank that have undergone significant local adaptations. Where a branch of a foreign bank uses models of the parent bank that have not undergone significant local adaptations, the model risk management of the parent bank can be relied on.
8. Definitions
“Model” – As defined in Section 9 and as determined in the banking corporation's policy. “Supplier Model” – A model that is entirely or partially a product of a supplier or another third party, including data, parameter values, and complete models. “Model Risk” – The potential for negative consequences resulting from decisions or actions (including reporting) based on incorrect model outputs or misuse of model outputs. “Model Outputs” – Forecasts, estimates, and information obtained from a model.
Chapter B: Model and Model Risks
Model
9. A "model" is a quantitative method that applies statistical, economic, financial, or mathematical
theories, techniques, and assumptions to process input data into output. A model consists of three components: the information component—input, which provides assumptions and data to the model; the processing component, which executes various operations on the input to turn it into useful information; and the reporting component—output, which is a presentation of the model's products. Models that meet this definition may be used, among other things, for analyzing business strategies, supporting business decision-making, identifying and measuring risks, assessing exposures, instruments, or positions, conducting stress tests, assessing capital adequacy, managing customer assets, measuring compliance with internal limits, maintaining the banking corporation's control framework, and meeting various public or regulatory reporting requirements.

Supervisor of Banks: Proper Conduct of Banking Business [1] (8/24) Management of Model Risk Page 369-4 Quantitative approaches where the input, partially or entirely, is qualitative or based on expert judgment are also included under the definition of a model.
10. Models are simplified representations of relationships between characteristics, values, and events
in the real world. Simplified representation is inevitable due to the inherent complexity of these relationships, but it is also intentional and aims to focus attention on certain aspects perceived as most important in applying a given model. Generally, the quality of a model can be measured in various ways: accuracy, discrimination ability, robustness, stability, and reliability, among others. No model is perfect, and the appropriate metrics for evaluating model quality and the effort required to improve model quality depend on the circumstances. For example, accuracy is relevant for models predicting future values, while discrimination ability is relevant for models ranking risks. In all cases, it is important to understand the model's capabilities and limitations in light of the simplifications and assumptions underlying it. Model Risks
11. Model risk is the potential for negative consequences resulting from decisions based on incorrect
or misused outputs and reports. The realization of model risk can lead to financial loss, incorrect business and strategic decisions, or reputational damage. Model risk arises mainly from two reasons:
a. The model may contain fundamental errors and produce inaccurate or incorrect outputs relative to the model's defined objectives and intended business uses. The mathematical calculation and quantification underlying each model typically involve the application of theory, selection of sample characteristics and numerical calculations, selection of input data and estimates, and implementation in information systems. Errors can occur at any stage, from design through development to implementation. Additionally, shortcuts, simplifications, or approximations used to address complex problems can compromise the integrity and reliability of the calculations. Finally, the quality of the model's outputs depends on the quality of the input data and assumptions. Errors in input data or incorrect assumptions will lead to inaccurate or incorrect outputs. b. The model may be used incorrectly or inappropriately. Even a well-founded model that produces accurate results consistent with its defined objectives can generate high model risk if its implementation or use is incorrect. Models are inherently simplifications of reality, and real-world events may indicate that the simplification is inadequate. This problem is even more significant when a model is used outside the environment for which it was designed. Banking corporations may do this intentionally when applying existing models to new products or markets, or unintentionally when market conditions or customer behavior change. Decision-makers need to understand the model's limitations to avoid inconsistent use with the model's original intent. Model limitations partly derive from various weaknesses in the model due to its shortcomings, approximations, and uncertainties. Additionally, limitations also arise from the assumptions underlying the model, which may restrict its applicability to a specific set of circumstances and situations. Model Risk Management

Supervisor of Banks: Proper Conduct of Banking Business [1] (8/24) Management of Model Risk Page 369-5
12. Model risks should be managed similarly to other types of risks and in accordance with the
principles set out in Proper Banking Management Directive no. 310 on "Risk Management". The banking corporation will identify the sources of risk, assess their intensity and scope, and determine ways to address the risk. a. A banking corporation shall manage the risk both at the individual model level and at the aggregate level of all models. b. Risk management will also include, in addition to identification and assessment, mitigation, monitoring, and reporting.
c. Risk assessment at the individual model level will be based, among other things, on the
type of model and its objectives, the level of complexity and uncertainty in the model, the materiality of the model for the corporation, connectivity to other models, the quality and completeness of input data, the model's capabilities and limitations, and the extent of subjective judgment in the model's development or use. d. A banking corporation shall document risk assessments periodically and adjust the risk management approach to the risk level of the individual model. e. Aggregate model risk management will include, at a minimum, identification and addressing the following factors: interaction and dependence between models; reliance on common assumptions, data, or methodologies, and any other factor that may adversely affect multiple models and their outputs simultaneously. Additionally, the aggregate impact of model risks on other risks to which the corporation is exposed (such as credit, market, and liquidity risks, operational risk, etc.) should be identified, monitored, and assessed. f. A guiding principle in managing model risks for a banking corporation is effective model challenge. Effective model challenge means critical analysis by objective and knowledgeable individuals who can identify the model's limitations and assumptions (including hidden assumptions) and make the necessary changes. The objective individuals should have appropriate incentives, professional competence, and influence. g. Incentives for effective model challenge are stronger when there is a clear separation between those performing the challenge and the model developers, and when the challenge is supported by appropriate compensation practices and organizational culture. Professional competence is key to effectiveness, as technical knowledge and modeling capabilities are essential for proper analysis and critique. The challenge may not be effective if there is no influence to ensure that actions are taken to address identified issues. Influence stems from a combination of explicit authority, status within the corporation, and commitment and support from higher levels of management in the banking corporation. h. Skills in model development and robust validation do not eliminate model risks, so the banking corporation should use additional tools to manage model risks. These additional tools include, among others, setting limits on model use, monitoring model performance, adjusting and updating models over time, and using other information and analysis tools.
i. Conservatism in input components, model design, or through explicit adjustments to its
outputs can be a tool for addressing model risk but is not a substitute for model improvement processes. Intentional adjustments made in the model's design, input,

Supervisor of Banks: Proper Conduct of Banking Business [1] (8/24) Management of Model Risk Page 369-6 and/or outputs for conservatism reasons require reporting and informing senior management and model users.
13. A banking corporation where models and their outputs significantly influence business decisions,
including decisions related to risk management and capital and liquidity planning, and where model failure could have particularly harmful effects on its financial condition, is required to implement a more comprehensive and rigorous model risk management framework.
14. Model risk management includes three layers: corporate governance, policies, and controls; model
development, implementation, and use; and model validation. This is detailed in Chapters C to E.
Chapter C: Corporate Governance, Policies, and Controls
General
15. Establishing and maintaining rigorous corporate governance, policies, and controls are very
important for the effectiveness of the model risk management framework. Even if model development, implementation, use, and validation are appropriate, weaknesses in corporate governance will reduce the overall effectiveness of model risk management. A strong corporate governance framework provides clear support and structure for those involved in risk management through policies that define relevant risk management actions, procedures that implement these policies, allocation of resources, and mechanisms to assess whether the policies and procedures are implemented as defined. The scope and complexity of corporate governance in the banking corporation will be adapted to the scope and complexity of model use in the corporation. Board of Directors and Senior Management
16. Corporate governance of model risk management is the responsibility of the highest levels in the
banking corporation.
17. The board of directors will outline the strategy for model risk management and determine the risk
appetite. The board will approve the model risk management policy formulated by senior management and discuss it at least once a year, among other things, to ensure that the banking corporation maintains appropriate practices.
18. Senior management will formulate and implement the policy on the subject, in accordance with
the strategy and risk appetite set by the board.
19. As part of their overall responsibility, the board of directors and senior management will establish
a robust framework for model risk management, also in accordance with the guidelines in Proper Banking Management Directive no. 310 – "Risk Management." Such a framework should be based on an understanding of risk at the individual model level and at the aggregate level. The framework should include standards for model development, implementation, use, and validation.
20. Senior management is required to implement an effective model risk management framework. An
effective framework will include:

Supervisor of Banks: Proper Conduct of Banking Business [1] (8/24) Management of Model Risk Page 369-7 a. A clear and consistent definition of "model" in line with this directive, to ensure consistency in model identification within the corporation and to meet the objectives of model risk management. b. Formulation of appropriate policies and procedures and mechanisms to ensure compliance with them.
c. Allocation of skilled personnel.
d. Supervision of model development and implementation. e. Evaluation of model results. f. Mechanisms to ensure effective model challenge. g. Review of findings from validation processes and internal audit findings and taking prompt corrective actions where required. h. Regular reporting to the board of directors on significant risks at the individual model level and at the aggregate level, as well as on compliance with the policy. Policies and Procedures
21. A banking corporation shall anchor the required actions for model risk management in a policy
document (hereinafter – "the policy") and detail them in appropriate procedures. Among other things:
a. The policy will align with the complexity of the banking corporation's business activities, organizational culture, and overall organizational structure, and be consistent with this directive. b. The policy will cover all aspects of model risk management, including "model" and "model risk" definitions, model risk assessment, accepted practices for model development, implementation, and use, appropriate actions for model validation, as well as governance and control over the model risk management process.
c. The policy will be updated as needed to ensure that model risk management practices
remain appropriate and up-to-date concerning changes in the market, products, and activities of the banking corporation, and are consistent with industry best practices. d. The policy will set requirements for conducting tests and analyses, encourage the development of accuracy targets for models, set thresholds for acceptable deviation levels, and define procedures for reviewing and addressing unacceptable deviations. e. The policy will include a description of the processes for selecting supplier models and continuing their use, including defining the functions involved in these decisions and their authorities. f. The policy will set priorities, scope, and frequency of validation actions. It will define the scope of validation to be performed on a model before its entry into the production environment and the scope of ongoing validation for models. Additionally, the policy will detail the requirements for validating supplier models. g. The policy and procedures will include requirements for preparing and maintaining detailed documentation of all aspects of the model risk management framework, including an inventory of models in use, results of modeling and validation processes, as well as issues identified and decisions made.

Supervisor of Banks: Proper Conduct of Banking Business [1] (8/24) Management of Model Risk Page 369-8 h. The policy and procedures will define the roles included in the model risk management framework, including developer, user, model owner, model approver, and model validator. Such a definition will include, at a minimum, a description of responsibilities, required expertise, authorities, reporting lines, and continuity mechanisms for fulfilling the role. Additionally, the controls and use of external resources for model development, validation, and compliance, and their integration into the model risk management framework will be defined. Group Model Risk Management Policy
22. A banking corporation heading a banking group will comply with the following requirements:
a. Establish a group model risk management policy (hereinafter – "group policy"). b. Supervise the implementation of the group policy in controlled entities and foreign branches.
c. The board of directors of the controlled entity will adopt the group policy as long as it
aligns with the interests of the controlled entity and subject to any law. d. Establish a mechanism whereby significant adjustments made by controlled entities or foreign branches to the group policy will be reported to it. Roles and Responsibilities
23. In general, roles within the model risk management framework can be divided into ownership,
control, and compliance. There are several ways in which a banking corporation can allocate responsibilities among these roles, but it is important that reporting lines and incentives are clear and established with consideration of potential conflicts of interest. Similar to any other risk, the business unit (in the context of the specific model) will bear the primary responsibility for managing the model risk it uses.
24. Ownership: The model owner in the banking corporation is responsible for the use and
performance of the model within the banking corporation's model risk management framework. The model owner is responsible for ensuring that the model's development, implementation, and use are appropriate. Additionally, the model owner must ensure that a model in use has undergone proper validation and approval processes, immediately identify changes in the model, and provide all necessary information for validation actions. Each model must have a designated model owner within the banking corporation. The model owner may be the model developer, user, or a management function, provided they meet the qualification and knowledge requirements. Joint ownership of a specific model by multiple entities is not allowed. The banking corporation must define the criteria for being a model owner and the authorities granted to them by virtue of their role.
25. Control: Model risks taken by the business unit should be controlled. The responsibilities for risk
control can be assigned to specific individuals, committees, or a combination of both. Responsibilities for model risk control include risk measurement, control of limits, and monitoring. Additional responsibilities include managing independent validation and review processes to ensure effective challenge. Appropriate resources should be allocated for model validation and

Supervisor of Banks: Proper Conduct of Banking Business [1] (8/24) Management of Model Risk Page 369-9 directing the scope of work and priorities. Issues and problems identified through validation and other controls should be reported by the risk control team (hereinafter—"control team") to relevant individuals and business model users within the banking corporation, including senior management, along with an action plan for correction. The control team will have the authority to restrict model use and monitor model use limits. The control team may allow temporary deviations from standard validation procedures, provided they are accompanied by other control mechanisms, such as setting timelines for completing validation work and imposing restrictions on model use.
26. Compliance: Model owners and the control team are required to comply with the model risk
management policy and procedures, so the banking corporation must establish and implement processes to ensure that these roles are performed effectively and in accordance with the policy and procedures. Documentation and tracking of model development, implementation, use, and validation actions should be conducted to create transparency regarding compliance with the policy and procedures. Internal Audit
27. The internal audit function will assess the overall effectiveness of the model risk management
framework in the corporation, including its ability to address both types of model risks, as mentioned in Section 11, at both the individual model level and the aggregate level. The following are highlights for the internal audit function's work regarding model risks:
a. Internal examination findings related to models will be documented and reported as required by Proper Banking Management Directive no. 307 – "Internal Audit Function." b. A banking corporation will ensure that the internal audit function has the appropriate skills to audit model risk management.
c. The internal audit will examine whether model risk management is comprehensive,
rigorous, and effective. For this purpose, the internal audit team must have sufficient expertise in relevant model concepts, as well as their use in specific business lines. d. The assessment of the overall model risk management framework will not be performed by any members of the internal audit team who perform certain validation actions, if any. e. As part of the model risk management assessment, the internal audit function will:
i. Ensure that policies and procedures meeting the requirements of this directive
exist and that model owners and the control team comply with these policies and procedures.
ii. Review documentation of model use and validation to examine whether
validations are performed on time and whether there are controls on models that adequately consider validation weaknesses.
iii. Assess the accuracy and completeness of the model inventory as defined in Section
30.
iv. Evaluate the processes for setting model use limits and monitoring them.
v. Determine whether model update processes are clearly documented and whether
they are performed as required.
vi. Check whether model owners and the control team meet the documentation and
risk reporting requirements.

Supervisor of Banks: Proper Conduct of Banking Business [1] (8/24) Management of Model Risk Page 369-10
vii. Conduct assessments of operational support systems and evaluate the reliability
of data used in models.
viii. Ensure that validation work is performed properly and that effective model
challenge is conducted. This includes assessing the objectivity, skills, and status within the organization of key individuals involved in the validation process to ensure they have the right incentives to identify and report deficiencies. The internal audit must review validation actions performed by both internal and external entities with the same rigor to ensure they are conducted in accordance with the requirements of this directive. External Entities
28. A banking corporation may engage external entities to assist in performing certain actions related
to the model risk management framework. These actions may include, for example, model validation and review, compliance roles, or other actions supporting the internal audit. These external entities may provide additional knowledge and a critical and effective challenge layer, which can improve the internal processes of model development and model risk management. The banking corporation will weigh the benefits of using external entities against the additional costs involved and the extra time required for external entities to understand the internal data, systems, and other relevant circumstances specific to the banking corporation.
29. Without derogating from Proper Banking Management Directive no. 359A—"Outsourcing"—in the
case of outsourcing certain actions related to the model risk management framework, the banking corporation must:
a. Ensure that the actions performed by the external entity and the scope of work are explicitly defined. b. Designate an internal individual capable of understanding and evaluating the results of validation or risk control actions performed by external entities. The internal entity is responsible for: ensuring that the agreed-upon work with the outsourcing function is completed; evaluating identified issues, tracking them, and ensuring they are addressed; and ensuring that the completed work is integrated into the overall model risk management framework of the banking corporation.
c. If the external entity is hired to perform only part of the validation or control work, the
banking corporation must coordinate between internal resources to complete the full required work. d. Maintain a continuity plan in case an external entity relied upon in the model risk management process is unavailable or does not perform the work properly. e. A banking corporation will not engage an external entity involved in model development for the validation of that model. Model Inventory
30. A banking corporation will maintain a comprehensive information system on all models in use, in
development within the banking corporation, or that were recently used and discontinued (hereinafter – "model inventory"). This is, among other things, to assess the banking corporation's model risk at the aggregate level. If each business line maintains a separate model inventory, the

Supervisor of Banks: Proper Conduct of Banking Business [1] (8/24) Management of Model Risk Page 369-11 banking corporation must designate a specific function responsible for maintaining the inventory of all models at the banking corporation level. In cases where a model has more than one variation, each variation of a model that the banking corporation decides requires separate validation, will be included as a separate model in the model inventory, including references to other variation.
31. The banking corporation's model inventory will be managed electronically, allowing for automatic
segmentation, retrieval, and controls.
32. The model inventory may include different levels of information, given the varying complexity and
importance of each model, and must include at least the following information for each of the banking corporation's models:
a. Description of the purpose, products, or business areas for which the model was designed, the actual or planned use of the model, and any restrictions on its use. b. Description of the type and source of input data used in the model, a list of variables underlying the model (which may include outputs of other models), and model outputs (detailed information may be included in the model's documentation, with references included).
c. Model approval date.
d. Whether the model as developed and approved is functioning properly, the date of the last update, and any deviations from the policy. e. Names of individuals or entities responsible for the various stages of the model's lifecycle:
model developers, model users, model owner, model approver, model validators. f. Date of the last validation performed and the planned date for the next validation process. g. Planned model lifespan. h. Additional model characteristics, such as: internal model or supplier model, inherent and residual risk level, etc. Documentation
33. The banking corporation shall maintain adequate documentation of the development and
validation of each model. The level of detail in the documentation of model development and validation should allow entities unfamiliar with the model to understand how the model works, its limitations, and the main assumptions underlying it, as well as independently access the databases used in the model and independently reproduce the model. Adequate documentation ensures continuity of actions, provides transparency regarding policy compliance, and helps track recommendations, responses, and deviations. The documentation serves developers, users, control and compliance teams, and the Banking Supervision Department.
34. Regarding Section 33, the banking corporation will define in procedures the required
documentation, access permissions to information, how information is communicated, checks for completeness and consistency of documentation, and the duration of documentation retention or its components. A banking corporation will use advanced information and knowledge management systems as much as possible to improve documentation.
35. The banking corporation will update the documentation regularly according to changes in the
model and its application environment.
36. The banking corporation will ensure that model developers perform adequate documentation and
that business lines or other decision-makers document information that led to the selection of a specific model and information about its validation. Additionally, a banking corporation will ensure

Supervisor of Banks: Proper Conduct of Banking Business [1] (8/24) Management of Model Risk Page 369-12 that other participants in model risk management actions continuously document their work, including ongoing monitoring processes, process controls, benchmarking, and output analysis.
37. A banking corporation will ensure that there are incentives for model developers and users to
maintain effective and complete model documentation.
38. In cases where a banking corporation uses a supplier model, it must ensure access to adequate
documentation from the supplier, allowing for appropriate model validation. Model Validation Reports
39. Validation reports will include references to the model aspects reviewed, highlight potential
deficiencies under various financial and economic conditions, and include references to the need for adjustments or other compensating controls.
40. Validation reports will include a clear executive summary, a concise description of the model's
purpose, and a summary of the model's outputs and validation results, including information on limitations and key assumptions.
41. The level of detail in validation reports will be adapted to the type of validation (initial, ongoing).
New Models and Products
42. When approving a new product, as defined in section 16 of Proper Banking Management Directive
no. 310 – "Risk Management," the banking corporation will examine the potential impacts of the new product on existing models and model risk management. Model Identification
43. The banking corporation will establish and implement active procedures and methods for
identifying models and including them in the model inventory to ensure proper identification, estimation, and management of model risks. This may be done through, for example, new product procedure, process reviews (internal control processes, risk surveys, etc.), setting responsibility for updating the model inventory list, conducting training on the subject to raise awareness, and more.
Chapter D: Model Development, Implementation, and Use
44. The model risk management framework should include rigorous development and
implementation processes carried out with knowledge and consistent with the model's intended use and the banking corporation's policy. Model development is not a simple or routine technical process. The experience and judgment of developers, as well as their technical knowledge, markedly impact the appropriate choice of input and processing components. In exercising such judgment, the training and experience of developers affect the model's risk level. Furthermore, modeling is often a multidisciplinary activity based on economics, finance, statistics, mathematics, and other fields. Models operate in real-world markets and events, so they must be adapted to specific applications and updated according to business uses. Subjective judgment is applied to a large extent at various stages of model development, implementation, and validation. Decision￾makers must recognize that subjective judgment increases the importance of rigorous and comprehensive model risk management processes.
45. When a banking corporation uses a supplier model, it must select the models and specific variables
suitable for its size, scope of activity, and business lines, and ensure that the chosen models are

Supervisor of Banks: Proper Conduct of Banking Business [1] (8/24) Management of Model Risk Page 369-13 appropriate for their intended purpose; in the case of using a supplier model, the provisions of
Section 93 apply.
Model Development and Implementation
46. An effective model development process begins with a clear definition of objectives to ensure it
aligns with its intended use. The model design, as well as the theory and logic underlying it, should be properly documented and based on professional publications and industry best practices.
47. A banking corporation must ensure that the process includes a detailed explanation of the model's
methodology and processing components, including the mathematical specification, as well as numerical techniques and approximations, with particular attention to advantages and limitations. Model developers will ensure that these components function as intended, align with the business purpose, are conceptually sound, and are mathematically and statistically correct. The modeling process will include comparisons with alternative theories and approaches.
48. The data and other information used for model development are crucial; a careful assessment of
data quality and relevance must be conducted and adequately documented. The banking corporation must ensure that developers can demonstrate that the data and information are suitable for the model and consistent with the underlying theory and chosen methodology. In cases where proxies are used, they must be identified, justified, and documented carefully and cautiously. If data and information do not represent the banking corporation's "portfolio" or other characteristics, or if assumptions are used to adjust the data and information, these factors must be tracked and analyzed appropriately so that users are aware of potential model limitations. This is especially important when dealing with external data and information (received from a supplier or external entity), particularly if they relate to new products, instruments, or activities.
49. Testing is an integral part of model development. Testing evaluates model components and overall
performance to decide whether the model performs as intended. In model testing, the banking corporation must, at a minimum:
a. Test the model's accuracy, robustness, and stability, and assess potential limitations and model behavior across a range of input data values. b. Evaluate the impact of assumptions and identify situations where model performance is poor or unreliable.
c. Conduct tests for various market condition scenarios, including scenarios outside the
forecast range. Tests should cover the full range of products or applications for which the model is intended. Extreme value data should also be used to identify the ranges where the model is effective. d. Assess the impact of model results on other models that rely on these results as input values. e. Ensure that testing actions cover the purpose, design, and execution of test plans, and include a summary of test results with interpretation and evaluation, as well as a detailed analysis of examples that can be learned from. f. Adequately document all testing actions and results.
50. The nature of tests and analyses will depend on the type of model and will be weighed against
different criteria depending on the context. For example, the appropriate statistical tests depend on distribution assumptions and the model's purpose. Moreover, in many cases, statistical tests cannot unequivocally reject incorrect hypotheses or accept correct hypotheses based on a sample.

Supervisor of Banks: Proper Conduct of Banking Business [1] (8/24) Management of Model Risk Page 369-14 Different tests have different strengths and weaknesses under different conditions. The banking corporation must apply a variety of tests to develop a robust model.
51. A banking corporation will ensure that the development of qualitative aspects of a model and
those subject to judgment are also robust. In some cases, as part of model development, a banking corporation may make adjustments to statistical results based on judgment or qualitative adjustments. These practices may be appropriate, but the banking corporation must ensure that all adjustments made as part of the development process are conducted properly and systematically and are well-documented.
52. Typically, models are embedded in larger information systems that manage the flow of data from
various sources into the model, handle data aggregation, and report model results. Model calculations must be coordinated with the capabilities and requirements of information systems. Model risk management requires significant investment in systems supporting models to ensure data and reporting reliability and integrity, along with control tools to ensure proper model implementation and use and efficient integration of models into systems. Model Use
53. Model use is a tool for a corporation to assess whether the model functions efficiently and to
evaluate model performance over time as conditions and model applications change. Additionally, the banking corporation must ensure that model users are knowledgeable internal individuals, with an interest in the models functioning well and reflecting economic and business reality. Model users can provide valuable business insights as part of the development process. Managers in business lines affected by model results may challenge the methods or assumptions underlying the models. Constructive challenge causes model developers to explain and justify the model's assumptions and design.
54. A banking corporation must carefully assess the nature of feedback from model users and the
motivations behind it and encourage constructive suggestions and criticisms from sources independent of the business line using the model. Examples of situations where model challenge by users may be insufficient include: when the model does not significantly impact users' work, when required model changes are perceived as having negative effects on the business line or are generally considered expensive or complex, when the challenge focuses on aspects of the models with the most direct impact on business performance measurement or compensation (in this case, asymmetric challenge may occur as users are less likely to challenge results that benefit them).
55. A banking corporation must ensure that reports used for business decision-making are clear and
comprehensive. Reports providing a range of model results for input data under different scenarios and assumptions, may enable decision-makers to gain important indications about the model's accuracy, robustness, and stability, as well as information on its limitations.
56. Dealing with uncertainty or inaccuracy in the model. Models are imperfect representations of
reality; all models have some degree of uncertainty and inaccuracy. A banking corporation will channel the development, implementation, and use of models to achieve an understanding of model uncertainty and inaccuracy and how to account for them appropriately. Examples of assessing and quantifying uncertainty and inaccuracy that may be used by the banking corporation include: assessing the potential impact of unobserved factors or those not fully included in the model; using confidence intervals around a model's estimate (i.e., using a range of results rather than a single point estimate); qualitative assessment of model uncertainty and inaccuracy. An example of a means to account for model uncertainty is adjusting input or calculations to produce

Supervisor of Banks: Proper Conduct of Banking Business [1] (8/24) Management of Model Risk Page 369-15 more "severe" or "negative" results for conservatism reasons; additional conservatism measures may include making adjustments to model results based on judgment, reducing reliance on model results, or using the model only in conjunction with other models or approaches. For conservatism in the model, see also Section 12(i).
57. Despite the provisions of Section 56, a banking corporation shall avoid extensive use of a
conservative approach or making conservative adjustments and additions as a way to deal with model risk or as an alternative to investing effort in improving models. Using conservatism may affect models in an incorrect, unclear, or inconsistent manner over time and may cause model users to devalue its results. When conservatism is used in a model, the banking corporation must justify and validate the conservatism attributed to the model results. Examples of justifying or validating conservatism in a model include using sensitivity analysis or stress tests. Conservatism can also be achieved by allocating additional capital buffers against potential losses due to model risk.
58. A banking corporation will objectively assess model risk and the costs and benefits attributed to
the model through a structured model validation process, in addition to the proper development, implementation, and use processes described above.
Chapter E: Model Validation
59. Model validation is a set of processes and actions designed to ensure that models function as
expected, according to the objectives and business uses for which they were designed. Effective validation helps ensure that models are robust. It also identifies potential model limitations and assumptions and assesses their possible impact. As with other aspects of effective challenge, model validation should be conducted by a team with appropriate incentives, skills, and influence.
60. All model components – including input, processing, and reporting – should be subject to
validation. This applies to both models developed within the banking corporation and models purchased or developed by external suppliers. The level of rigor and sophistication of validation processes in the banking corporation should align with the overall use of models, the complexity and materiality of the models, and the scope and complexity of the banking corporation's activities.
61. The banking corporation must ensure that the validation process is conducted independently of
model development and use. Validation should be performed, as much as possible, by individuals not responsible for model development or use and not affected by the determination of whether the model is valid. However, independence is not an end in itself but helps ensure that incentives align with the goals of model validation. Independence may be supported by separating reporting lines, but it should also be evaluated based on actions and results. There may be situations where it is more efficient for model developers or users to perform part of the validation work. In this case, the banking corporation must ensure that the validation work is subject to critical review by an independent entity, who will perform additional actions to ensure proper validation. The critical review will include, among other things, assessing the scope and clarity of documentation, issues identified by objective entities, and actions taken by management to address identified issues.
62. A banking corporation will create appropriate incentives for proper validation through
compensation methods and performance evaluation standards directly related to the quality of model validation and the level of critical review and objectivity. Additionally, the banking corporation will ensure that its organizational culture supports objective thinking and encourages skepticism and challenge of decisions.

Supervisor of Banks: Proper Conduct of Banking Business [1] (8/24) Management of Model Risk Page 369-16
63. A banking corporation will ensure that the validation team has the necessary knowledge, skills, and
expertise, and, as required, a high level of technical expertise and familiarity with the business line using the model and the model's intended use. In doing so, the banking corporation will avoid relying on the model developer as an objective or sole source to establish the model's quality.
64. A banking corporation will ensure that the validation team has explicit authority to challenge
developers and users and escalate its findings, including problems and deficiencies. Additionally, the banking corporation will ensure that the person or unit to whom the team reports has sufficient influence or status within the banking corporation to ensure that issues and deficiencies receive appropriate attention and within a reasonable timeframe. Influence or status may be reflected in reporting lines, role, rank, or allocation of responsibilities, and, if necessary, in actual cases where models or their use were improved following validation.
65. The scope and rigor of pre-use validation actions will align with the potential risk associated
with model use. A banking corporation shall ensure that:
a. A model is not used if significant deficiencies are identified during the validation process, or it will allow model use only in exceptional circumstances and under restrictions until the identified issues are resolved. A banking corporation will reject a model if the deficiencies identified are too severe to handle. b. If it was not possible to perform all the required validation actions before starting to use the model due to a lack of data or other limitations, the banking corporation will ensure that this fact is documented and reported to users, senior management, and other relevant entities. In those exceptional cases where it was not possible to perform all the validation actions, the banking corporation will reduce the uncertainty regarding the quality of the model's results through other compensating controls. This is particularly relevant for new models and the use of existing models for new applications.
66. The banking corporation will perform ongoing validation actions for every model in use. This is
to monitor known model limitations and identify new ones. The banking corporation will use ongoing validation actions to ensure that changes in markets, products, exposures, activities, customers, or business practices do not create new limitations for models. For example, an ongoing validation process will ensure that changes in underwriting processes are incorporated into the model in time. Another example is using ongoing validation during periods of favorable economic and financial conditions when risk and potential loss estimates may be overly optimistic, and available information may not fully reflect conditions in extreme scenarios.
67. The banking corporation will conduct a periodic review of every model, at least once a year and
more frequently if necessary, to determine whether the model operates as intended and whether existing validation actions are sufficient. The review results can confirm previous validation work, indicate required updates to previous validation actions, or indicate the need for additional validation actions. Significant changes in models should also be subject to validation. Best practice is to ensure that all models undergo a full validation process as described below, on a regular and appropriate cycle, with up-to-date documentation of all actions (in addition to the initial validation as mentioned in Section 65 above). However, in rare cases (e.g., when dealing with a simple and narrow model with a small impact and low model risk), the banking corporation may set a periodic review frequency of once every two years. Such a determination will be accompanied by documentation of the reasons for the reduced frequency and approvals from senior management.
68. The banking corporation will ensure that validation is effective to help reduce model risks by
identifying model errors, performing corrective actions, and ensuring proper use. Effective

Supervisor of Banks: Proper Conduct of Banking Business [1] (8/24) Management of Model Risk Page 369-17 validation should provide the banking corporation with an assessment of a given model's reliability, based on the assumptions, theory, and methods underlying the model, help the banking corporation identify deterioration in model performance over time, and set thresholds for acceptable levels of errors through an analysis of the distribution of results around expected or predicted values. If validation results consistently fall outside the acceptance range, the banking corporation should stop using the model or redevelop it. Key Components of Proper Validation Process
69. An effective validation framework should include three core components:
a. Assessment of Conceptual Soundness, including evidence from the model development process. b. Ongoing Monitoring, including process verification and benchmarking.
c. Outcomes Analysis, including back-testing.
First Component—Assessment of Conceptual Soundness
70. This component includes evaluating the quality of the model's design and construction. It involves
reviewing documentation and empirical evidence supporting the methods used and the variables chosen for the model. The documentation and reviews should provide an understanding of the model's limitations and the assumptions on which it is based. Validation should ensure that full disclosure is given about the judgment exercised in the model's design and construction and that the judgment is consistent with published research and industry best practices. Evidence from the development process should be reviewed before the model is put into use and as part of the ongoing validation process, especially when a significant change is made to the model.
71. The banking corporation will conduct a proper development process that includes creating
documentation with evidence supporting all choices made in model development, including the theoretical basis, main assumptions, data, and specific mathematical calculations. As part of the model validation process, the banking corporation will ensure that these aspects are subject to critical analysis, both by assessing the quality and scope of the evidence from the development process and by conducting additional analyses and tests as needed. This includes comparing alternative theories and approaches, evaluating the main assumptions and variable selection by analyzing their impact on model outputs, and focusing on any potential limitations. Additionally, the banking corporation will assess the relevance of the data used in model construction to ensure they represent the banking corporation's "portfolio" or market conditions well, depending on the model type. This assessment is particularly important when using external data or when the model is used for new products or activities.
72. The banking corporation will use sensitivity analyses in model development and validation, as
appropriate for the specific model, to test the impact of small changes in input values and parameters on model results, to ensure they are within the expected range, and to identify conditions where the model may be unstable or inaccurate. Sensitivity analyses and tests that may be used by the banking corporation include, among others, checking the model's sensitivity to small changes in input values (large and unexpected changes in results due to small changes in input values may indicate model instability); making changes to several input values simultaneously as part of the sensitivity analysis to reveal unexpected interactions, especially when

Supervisor of Banks: Proper Conduct of Banking Business [1] (8/24) Management of Model Risk Page 369-18 these interactions are complex and non-intuitive; conducting stress tests on the model to examine its performance over a wide range of input values and parameters, including extreme values, to ensure the model is robust.
73. The banking corporation will have clear guidelines for using the results of sensitivity analyses and
additional quantitative tests. If the tests indicate that the model may be inaccurate or unstable under certain circumstances, management is responsible for considering changing certain model features, reducing reliance on its outputs, limiting its use, or developing a new approach.
74. The banking corporation will evaluate qualitative information and judgment used in the model
development process, including the logic, considerations, and types of information used. This is to establish the model's conceptual soundness and determine the appropriate conditions for its use. The validation process should ensure that qualitative assessments based on judgment are conducted properly and systematically, well-supported, and documented. Second Component—Ongoing Monitoring
75. The second core component of the validation process is ongoing monitoring. Such monitoring is
intended to ensure that the model is implemented correctly and that its use and performance are as intended.
76. Ongoing monitoring is necessary to assess whether changes in products, exposures, activities,
customers, or market conditions require adjustments, redevelopment, or replacement of the model, and to ensure that any extension of the model beyond its original scope is valid. As part of ongoing monitoring, known model limitations identified during development should be assessed on a continuous basis. Monitoring will begin when a model is first implemented in the production environment for actual business use. Monitoring should be conducted periodically at a frequency appropriate to the model's nature, the availability of new data or new modeling approaches, and the level of risk. The banking corporation will define a plan for the ongoing review and evaluation of model performance and establish procedures for addressing any issues that arise. Such a plan will include process verification and benchmarking.
77. Process verification includes checking that all model components function as intended. As part
of process verification, it should be ensured that internal and external input data continue to be accurate, complete, and consistent with the model's purpose and design, and that they are of the highest possible quality. Computer code for model implementation will be subject to strict quality control and change control procedures to ensure that the code is correct, cannot be altered except by authorized entities, and that all changes are documented and auditable. The banking corporation will ensure that system integration receives special attention, for example, when model processing components draw data from multiple sources, process large amounts of information, and then feed multiple databases and reporting systems. This also applies to user-developed applications, such as spreadsheets or ad-hoc database programs for generating quantitative estimates. Since the content or composition of information in such applications may change over time, system updates should be made to reflect changes in data or their use. Additionally, the banking corporation will review reports generated based on model outputs to ensure that the reports are accurate, complete, and informative, and that they include appropriate indicators of model performance and limitations.
78. The banking corporation is required to perform many of the tests conducted as part of model
development also as part of ongoing monitoring and on a regular basis, allowing for the integration of additional available information. Additionally, it should examine whether new

Supervisor of Banks: Proper Conduct of Banking Business [1] (8/24) Management of Model Risk Page 369-19 empirical evidence or theoretical research indicates a need to adjust or even replace the original methods. The banking corporation will conduct an analysis of the completeness and applicability of internal and external information sources, including information provided by external suppliers, on a regular basis.
79. The banking corporation shall periodically repeat sensitivity analyses and other tests of model
robustness and stability. If a model operates correctly only within certain ranges of input values, market conditions, or other factors, the banking corporation should monitor it to identify situations where there is a deviation from these ranges.
80. Ongoing monitoring should include an analysis of model overrides accompanied by proper
documentation. Almost every model will have cases where its outputs are ignored, altered, or replaced based on the judgment and expertise of model users. However, such overrides may indicate that, to some extent, the model is not functioning as intended or has effective limitations. The banking corporation is required to assess the reasons for such overrides, and to track and analyze their performance. When a model has many overrides, or if the override process consistently improves model performance, the banking corporation should update or redevelop the model. The banking corporation will establish procedures for model overrides, including, among other things, requirements for: appointing an approving entity, documenting the override, its approval, and its reasons, and setting limits on the extent of overrides. An override may occur not only regarding model outputs but also regarding other parts of the model, such as input data. Every override should be well-explained and documented and subject to review by an independent entity.
81. Benchmarking is a comparison between the input and output of a specific model and estimates
based on alternative internal or external models (e.g., against credit bureau models, securities and derivative instrument pricing models). The banking corporation can perform benchmarking during the model development process and as part of its ongoing monitoring. To ensure proper comparison, models used for benchmarking should be rigorous, and the comparative data should be accurate and complete.
82. If benchmarking reveals gaps between the model's results and the comparative data, the banking
corporation will investigate the source and extent of the gaps and examine whether they are within an expected or acceptable range, depending on the nature of the comparison. The banking corporation should consider whether the results of this analysis indicate that changes to the model are necessary; conversely, even if there is a good match between the model's results and the comparative data, this is evidence supporting the model, but it should also be approached with caution to avoid complacency. Third Component—Outcomes Analysis
83. The third core component of the validation process is outcomes analysis, i.e., comparing the
model's results with actual outcomes. The exact nature of the comparison depends on the model's objectives and may include evaluating the accuracy of estimates or forecasts, assessing ranking ability, or other appropriate tests. In all cases, such comparison helps assess model performance by determining the expected ranges for results, considering the model's objectives, and evaluating the reasons for discrepancies between expected and actual results.
84. Outcomes analysis typically relies on statistical tests or other quantitative measures. It may also
include expert opinions to examine the intuition explaining the results and ensure they are reasonable. When the model itself relies on expert opinions, quantitative analysis of the outcomes

Supervisor of Banks: Proper Conduct of Banking Business [1] (8/24) Management of Model Risk Page 369-20 helps assess the quality of the opinions. Outcomes analysis should be conducted on an ongoing basis to examine whether the model continues to operate according to its intended objectives and business uses.
85. If outcomes analysis indicates poor performance, the banking corporation should take steps to
make the necessary corrections to the model.
86. The banking corporation should conduct a variety of quantitative tests, qualitative tests, and
analytical techniques as part of model outcomes analysis. The choice of testing technique will be based on the model's methodology, complexity, data availability, and the model's potential risk to the banking corporation. The banking corporation's outcomes analysis should include a variety of tests, as each test has its weaknesses. For example, some tests are suitable for assessing the model's ranking ability or classifying observations into segments based on relative relationships, while other tests are suitable for assessing the absolute accuracy of the forecast. The tests should be tailored to each situation and model, as not all tests are effective or possible in all circumstances.
87. Models undergo adjustments regularly to account for new data, new techniques, or due to
deterioration in model performance. Therefore, the banking corporation should examine both the original model's forecasts and the adjusted model's forecasts against actual outcomes as part of outcomes analysis. If the adjusted model's performance is not better than the original model's performance, the banking corporation should consider making further changes and possibly redesigning the entire model before the adjusted model replaces the original.
88. A banking corporation will use back-testing in outcomes analysis as much as possible. Back-testing
involves comparing actual outcomes with the model's forecasts for a period not used for model development, using observations that match the forecast horizon or the time window on which the model operates. Typically, the comparison is made using predicted ranges or statistical confidence intervals around the model's forecasts. If actual outcomes fall outside these ranges, the banking corporation will analyze the gaps and investigate the main causes of the gaps in terms of scope or frequency, to determine whether the gaps result from omitting significant factors from the model, errors in other aspects of model characterization, such as variable interactions or assumptions about linearity, or whether they are random and considered acceptable model performance. Analyzing model fit based on the development sample (in-sample) and model performance based on data not used for development (out-of-sample) are important components of model development, but the banking corporation should not consider these tests as back￾testing.
89. A banking corporation should establish and document both the choice of back-testing tests and
the interpretation of test results. Analyzing back-testing results is not a mechanical and simple process, as the goal is to test the model, not specific predicted values.
90. A banking corporation will also conduct back-testing for models with a long forecast horizon, but
given the time required to accumulate data for back-testing, the testing should be supplemented by making forecasts for shorter periods. A banking corporation will include in outcomes analysis early warning metrics to measure performance shortly after the model is put into use, as well as an analysis of performance trends over time. The banking corporation should not consider these tools as a substitute for back-testing, which should be conducted over longer time frames, but rather as very important complementary means.
91. A banking corporation will make adjustments to the model, calibrate it, or redevelop it when model
outcomes analysis and other validation process components reveal significant errors or inaccuracies in model development or indicate that results consistently fall outside the acceptance

Supervisor of Banks: Proper Conduct of Banking Business [1] (8/24) Management of Model Risk Page 369-21 ranges set by the banking corporation. Adjustments and calibration should be made conservatively and undergo independent review.
92. A banking corporation that makes significant changes to a model's structure or technique or
redevelops a model will perform validation actions to the appropriate extent and rigor before implementation. In cases where the ability to use key validation tools, such as back-testing or sensitivity analysis, is limited due to various reasons such as lack of data or observations on prices, the banking corporation should place special emphasis on model limitations when assessing its suitability for use and ensure that senior management is aware of the model's limitations when using it for decision-making. This applies both at the individual model level and at the aggregate level of all models. Supplier Model Validation
93. The extensive use of supplier and third-party products, including data, parameter values, and
complete models, presents unique challenges in validation and other model risk management actions, as the modeling expertise is external to the user, and some components are not owned by the banking corporation. Without derogating from Proper Banking Management Directive no. 359A – "Outsourcing":
a. The banking corporation will integrate supplier products into its model risk management framework, using the same principles applied to models developed by the banking corporation, even if the process may differ. b. The banking corporation will ensure that proper processes exist for selecting a supplier model. The banking corporation will require the supplier to provide evidence from the development process explaining the product's components, design, and intended use, to determine whether the model is suitable for the banking corporation's products, exposures, and risks. The banking corporation will obtain from the supplier the results of appropriate tests indicating that the product operates as expected, clear and detailed information on model limitations and assumptions, and cases where model use may be problematic.
c. The banking corporation will expect the supplier to conduct ongoing performance
monitoring and outcomes analysis as described in this directive, with continuous disclosure to the banking corporation, and to make necessary changes and updates to the model over time. d. The banking corporation is required to validate its use of supplier products. Systematic validation processes are important to understand the supplier's product, its capabilities, applicability, and limitations. This detailed understanding is essential for performing basic controls on the banking corporation's activities. Through validation, the banking corporation should gain substantial internal knowledge in case the supplier or the banking corporation terminates their relationship for any reason, or if the supplier ceases operations. e. If an external model does not allow full access to the software code and implementation details, the banking corporation will rely more on sensitivity analysis and benchmarking and apply compensating controls.

Supervisor of Banks: Proper Conduct of Banking Business [1] (8/24) Management of Model Risk Page 369-22 f. If adjustments are made to a supplier model to adapt it to the banking corporation's specific circumstances, the banking corporation's decisions regarding model adjustments will be documented and justified and reviewed as part of the validation process. g. If a supplier provides input data or assumptions, or they are used to develop models, the banking corporation will assess the relevance of those data or assumptions to the banking corporation. h. The banking corporation will obtain information about the data used to develop the model and assess the extent to which they represent its situation. Additionally, the banking corporation will conduct ongoing monitoring and outcomes analysis of supplier model performance using its own outputs or other appropriate tools.
i. The banking corporation will have contingency plans for cases where the supplier model
is no longer available or the supplier can no longer support it. j. If model suppliers provide the banking corporation with independent validation or approval reports for those external models, the banking corporation will require these validation reports to present the model aspects reviewed, highlighting potential deficiencies over a range of financial and economic conditions (as applicable), and determine whether adjustments or other compensating controls are needed. The banking corporation will not rely solely on these reports. The banking corporation's management should understand any limitations imposed on the validator when assessing the processes and code used in the model.
Chapter F: Commencement and Transition Provisions
94. The directive will take effect one year from its publication date (21.8.25, hereinafter – "effective
date"); however – a. For an individual model in use on the eve of the directive's publication (hereinafter – "existing model"):
i. If the model is a significant model (according to the banking corporation's
definitions) – within six months from the effective date; however, instead of initial validation (if not performed), the banking corporation will conduct a comprehensive validation for this model.
ii. Other models – within 18 months from the effective date.
95. If a banking corporation identifies an existing model after the publication date of this directive
(21.8.24) – it may complete all the required processes according to this directive within one year from the date of identification or within one year from the effective date, whichever is later. Updates Updates to the Proper Banking Management Directives File Circular 06 number Version Details Date 2792 1 Original directive August 21, 2024

Supervisor of Banks: Proper Conduct of Banking Business [1] (8/24) Management of Model Risk Page 369-23

Supervisor of Banks: Proper Conduct of Banking Business [1] (8/24) Management of Model Risk Page 369-24
Appendix: Examples of Applying the Principles of the Directive Regarding Artificial Intelligence
Models
Responsibility – The internal organizational structure should be adapted to the fact that human involvement, including in decision-making, is limited or nonexistent. In particular, the responsibilities of all relevant role holders and the overall responsibility of the board of directors and senior management should be adapted. Ethics – The use of artificial intelligence may lead to undesirable outcomes, such as customer exploitation or prohibited discrimination. One of the means that can be used is to adapt the bank's ethical code in regards to concerns arising from the use of such models. It is expected that the ethical code will define and address, at a minimum, issues related to discrimination (permitted and prohibited), privacy protection, and the nature of the information the bank is allowed to use. Input – The nature of the checks and controls regarding input data may specifically refer to the processes of transferring information between systems in terms of completeness, field definition alignment, and so on, unlike traditional models where the emphasis may be on "establishing checks." Processing – The processing procedures performed in AI models can be complex and intricate compared to processing procedures in traditional models. The bank must ensure that there is an understanding of the processing procedures and that these procedures are robust and explainable. Additionally, it is essential to document the development processes, processing procedures, and features of AI models. In particular, the decision-making mechanisms designed for the model should be fully and comprehensively documented, and the databases and input data used in the development and design of the model, as well as in its validation, should be preserved and documented. Output – Unlike traditional models where a human factor in the bank receives the model output regularly and can highlight errors and inconsistencies, in AI models where human involvement is reduced, it is necessary to ensure that appropriate controls exist so that failures in the adequacy of the model results are highlighted and reviewed.