Derive Valuable Insights from the RPO: Ask Effective Questions and Ensure Proper Documentation

ANALYSIS REPORT
Contents
Summary 2

1. Background and Purpose of the Report 4
2. Observations on the Design of the RPO 5
   2.1 Improvement: Scope of Inquiry Greatly Expanded 5
   2.2 Do Not Present Unsuitable Questions to Respondents Regarding the Risk-Return Trade-off 5
   2.3 Warning: Beware of Over-Questioning Risk Capacity 6
   2.4 Points of Attention When Combining Methods 6
3. Observations on the FRAME Criteria 7
   3.1 'Errable': Detection of 'non-understanders' works well, but follow-up steps can be improved 7
   3.2 'Rationalising': Limit irrational behavioral effects 7
   3.3 'Appropriate': Progress in personal pension context, but inflation adjustment sometimes unclear 8
   3.4 'Measurable': Relative risk aversion present, but explanation can sometimes be more thorough 8
   3.5 'Feasible': Do Not Make the Inquiry Unnecessarily Complex 8
4. Documentation of the RPO 10
   4.1 Documentation Often Incomplete or Unclear 10
   4.2 Support for Representativeness Sometimes Lacking 10
   4.3 Add Assumptions Regarding Pension Income to the RPO 11
   4.4 Documentation of Reflection 11
5. Inquiring About Preferences Regarding the Payout Phase 12
   5.1 Questions About the Payout Phase Aid Decision-Making 12
   5.2 'FRAME' Is Also Important When Asking About the Payout Phase 12
6. Inquiring About Preferences Regarding Sustainable Investing 13
   6.1 Incorporating Sustainability Within the RPO 13
   6.2 It Helps to Also Investigate Motivation 13
   6.3 How Much Do Participants Value Sustainable Investing? 14
   Appendix - Research Methodology 15
   DECEMBER 2024
   Derive Valuable Insights from the RPO: Ask Effective Questions and Ensure Proper Documentation

In Brief
In this report, the AFM shares its findings from its second investigation into RPOs conducted by pension administrators. The AFM sees broad-based progress on several of the 5 'FRAME criteria'. However, we also see areas where RPOs can be improved. We wish to contribute to the further development of the RPO and therefore share recommendations and practical examples. Our recommendations focus, among other things, on the trade-offs between risk and return presented to participants and on the design of the RPO. We also call for careful documentation of the conducted RPO and reflection on what worked and what did not.

RPO Report 2
ANALYSIS REPORT
Summary
Pension administrators (pension funds, insurers, and PPIs) manage funds on behalf of their participants. How these funds are managed and allocated largely determines what the participants' retirement income will look like. Characteristics of the pension scheme, such as investment policy, distribution rules, and the design of the payout phase, have a significant impact on the amount, uncertainty, and flexibility of that pension income. People may have different wishes and expectations regarding their pension income. Therefore, it is important to investigate how much risk participants, former participants, or pension beneficiaries (hereinafter collectively referred to as participants) want and are able to take regarding their pension income. Pension administrators map this out by conducting a Risk Preference Study (RPO). The outcome of the RPO is one of the building blocks for determining investment policy and can also be useful for aligning communication with participants.

In 2023, the Authority for the Financial Markets (AFM) published a guideline on the risk preference study. Subsequently, the AFM conducted an exploratory investigation into the RPOs of several funds and published a report on this. This year, we conducted research again, on a new group of funds. This report is the result. It is an addition to the previous publications and contains additional messages for further improving and sharpening the RPO.

Due to the new and open nature of the statutory RPO norms, we wish to continue monitoring developments and proactively share knowledge and express our expectations. We assume that pension administrators will take note of the messages and recommendations from our publications and incorporate them into their RPOs.

Compared to the RPOs the AFM investigated last year, we see the most progress in three of the five 'FRAME criteria', namely measurable, appropriate, and errable. For the criteria feasible and rationalising, we have included some points of attention in Chapter 3.

• Measurable: In all RPOs conducted after July 1, 2023, the pension administrator has calculated relative risk aversion. This helps in translating the RPO into risk attitude and in comparing RPOs with each other.
• Appropriate: The RPO questionnaires align better with the pension context of participants. All investigated RPOs assume net pension amounts including AOW (state pension), and adjust the displayed amounts to the participant's age, income, and status. However, some pension administrators could explicitly state whether the displayed pension amounts have been adjusted for inflation.
• Errable: Most RPOs include questions to detect whether participants understand the questionnaire. Pension administrators must consider in advance how they will handle the results of participants who do not seem to understand the inquiry.

Another clear improvement is that almost all investigated RPOs this time have a sufficiently broad range to measure both high and low risk aversion and can demonstrate this via relative risk aversion or the large spread in investment mixes behind the choice options presented.

In our RPO research, we also looked at questions regarding participants' preferences on two special areas: sustainable investing and the design of the payout phase. We also asked about the experiences of pension administrators with the questions posed on these topics. In Chapters 5 and 6, we have included our perspective on these topics.

RPO Report 3
ANALYSIS REPORT
Based on our research, we also arrive at a number of recommendations in Chapters 2, 3, and 4. We summarize the four most important ones here.

Do Not Present Unsuitable Questions to Respondents Regarding the Risk-Return Trade-off As in the earlier exploration, we see in this investigation RPOs with questions where the displayed pension amounts show a higher expected return, but no or hardly any increase in risk. These questions do not provide useful insight into the risk tolerance of participants. We therefore expect that pension administrators do not present such questions to participants.

Pension administrators are free to determine how they arrive at suitable trade-offs to present to participants. The personalized pension amounts serve as support and help participants place the consequences of the different options in their own pension context. Carefully look at the pension amounts that different participants see and check whether the question you pose forces a trade-off between risk and return. If not, adjustments may be needed to the assumptions underlying the pension amounts.

Think in Advance About Combining Answers
When drafting the RPO, pension administrators combine quantitative and qualitative questions to participants. It is necessary to consider in advance how answers to different types of questions can be logically combined.

It must be transparent how combined results were arrived at. This ensures that successive RPOs are better comparable, prevents important information from being lost, and helps in justifying choices made in translating to risk attitude.

Do Not Make the Inquiry Unnecessarily Complex
Pay extra attention to how accessible and workable the questionnaire is for participants when drafting the RPO. This increases the chance of participants completing the questionnaire fully, and thus leads to higher reliability of the results. A high dropout rate, inconsistent answers, or long completion times can indicate an inaccessible or complicated questionnaire.

• Avoid questions where participants must look up or estimate financial information themselves, as well as questions about gross amounts or amounts excluding AOW. Many participants also find it difficult to understand percentages or technical terms well.
• Try to gain useful insights with as few questions as possible. Long questionnaires lead to higher dropout rates. Consider in advance whether you can draw conclusions from answers to a question and whether the question is effective for determining risk capacity, the design of the payout phase, or a sustainable investment policy. In the risk capacity section, for example, we saw very many qualitative questions about the participant's financial vulnerability, which seemed to be used only to a limited extent in translating the RPO into policy.

Document the RPO Well
The AFM expects pension administrators to carefully document all parts (including assumptions) of the RPO. This is essential for ensuring quality and reliability. You want to be able to compare results and conclusions of future RPOs well with the current one. Good documentation also contributes to improving the RPO. For this reason, also document which parts of the RPO you did not use in determining risk capacity/tolerance and which improvement points you have identified for future RPOs.

RPO Report 4
ANALYSIS REPORT

1. Background and Purpose of the Report
   Background
   In 2022, the AFM published the guideline on risk preference studies (the Guideline). In this guideline, we outline a number of conditions that the RPO must meet, without strictly prescribing exactly how the RPO must be conducted. Additionally, the guideline contains practical tips and examples. In addition to the guideline, the AFM published the report 'Exploration of Risk Preference Studies' (the Exploration) in June 2023 as a conclusion of an exploratory investigation into 12 RPOs. In general, we then concluded that the 12 pension administrators involved had seriously engaged with the RPO. At that time, we provided concrete observations and improvement points to all 12 investigated pension administrators. Through the report, we shared insights from the investigation using concrete examples.

The 2023 investigation had an exploratory character because it took place before the entry into force of the new pension legislation. Thanks to the definitive legislation and the aforementioned AFM publications, it has been clear to pension administrators since July 1, 2023, on which criteria the AFM supervises the RPO. Our new investigation in 2024, on which this report is based, therefore has a testing character.

Purpose of the Report
This report serves to conclude the investigation conducted in the second and third quarters of 2024. The purpose of this report is to inform the pension sector of our findings. Where possible, we have included good examples, without these being traceable to individual pension administrators. We thank those involved in this investigation for their open and constructive attitude and their efforts to continuously improve the RPO.

The AFM expects all pension administrators to be aware of the observations in this report and our earlier publications on the RPO. Due to the recurring nature of the RPO, which is conducted at least once every five years, this also applies to pension administrators who have already conducted an RPO. In future supervisory investigations, the AFM will select pension administrators in a risk-based manner to assess the extent to which their RPOs comply with laws and regulations.

Statutory Framework for RPO and Risk Attitude
For pension administrators executing a defined contribution scheme in the new pension system, it is mandatory under Article 52b, second paragraph of the Pension Act to use an RPO when determining risk attitude.

Article 14v of the Decision on Implementation of the Pension Act and Mandatory Occupational Pension Scheme Act (BuPw) sets further conditions for the RPO. Among other things, it states that the RPO investigates to what extent a group of participants, former participants, or pension beneficiaries is willing to bear investment risks with regard to their objectives. And that the RPO also takes into account the extent to which this group can bear investment risks.

The statutory framework and its interpretation by the AFM are described in detail in paragraph 2 of the guideline.

RPO Report 5
ANALYSIS REPORT
2. Observations on the Design of the RPO
Pension administrators have the freedom to make choices in the design of the RPO, within statutory requirements and set criteria, that fit their specific situation and participant population. The RPO often consists of a combination of elements and methods, for example, qualitative and quantitative questions about participants' risk tolerance. But it also includes questions about their risk capacity and financial situation. The AFM expects pension administrators to make visible and substantiate which elements they include in the RPO and how these parts help to map participants' risk tolerance and risk capacity. In this chapter, we discuss some choices and some good and bad examples.

2.1 Improvement: Scope of Inquiry Greatly Expanded
For a good picture of the risk preferences of all participants, the scope of the inquiry must be broad enough to measure both high and low risk aversion. In our previous report, we signaled that in several RPOs the scope was strongly limited, both at the top and bottom. We observed in those RPOs that many outcomes lay on the upper or lower boundary of the choice space. This suggests that many participants actually want to take more or less risk than indicated. In the current investigation, we see a clear improvement in this regard.

Almost all investigated RPOs this time have sufficient scope. They can also demonstrate this via the minimum and maximum 'relative risk aversion' they have calculated or based on the large spread in investment mixes behind the choice options presented to participants.

2.2 Do Not Present Unsuitable Questions to Respondents Regarding the Risk-Return Trade-off
The goal of the RPO is, among other things, to determine how much extra risk participants are willing to take in exchange for a higher expected return. Therefore, participants must make trade-offs between risk and return in the RPO.

As in the earlier exploration, we see in this investigation RPOs with questions where the displayed pension amounts show a higher expected return, but no or hardly any increase in risk. These questions do not provide useful insight into the risk tolerance of participants. We therefore expect that pension administrators do not present such questions to participants.

Unsuitable questions, where higher expected return is not or hardly accompanied by an increase in risk, seem to occur mainly with longer investment horizons. This may be a consequence of using scenario sets that do not work well in the context of the RPO. Pension administrators are free to determine how they arrive at suitable trade-offs between which participants can choose. Look at the pension amounts that different participants see and check whether the trade-off you present forces a choice between risk and expected return.

Good Practice
Example from the investigation to prevent a variant with a higher expected return from not being accompanied by an increase in risk:
For a clearly visible risk-return trade-off, the annual stock return is capped at <x>% in scenario calculations for the youngest age cohorts.

RPO Report 6
ANALYSIS REPORT
Showing realistic pension amounts helps participants place the consequences of different variants in their own pension context. However, we emphasize that the ability of an RPO to identify participants' risk preferences in a pure manner is more important than showing the most realistic possible picture of the amount and security of the pension income. If necessary, explicitly state in the RPO that these are example amounts.

2.3 Warning: Beware of Over-Questioning Risk Capacity
Risk capacity (the extent to which a participant can bear risks) is needed to translate the estimated risk tolerance into investment policy. The AFM finds it important that information about the AOW, the participant's age, and the pension built up with the administrator is included in this. This information does not always have to be asked by the pension administrator. Analyzing data from the administration or making well-founded assumptions can also be part of an RPO.

Usually, additional questions are asked in the RPO about the financial situation of participants, such as about the pension income respondents expect to need, the ability to cover shortfalls in pension income, their current financial reserve, or their investment knowledge and experience. Of course, these questions can yield useful insights for determining risk attitude and investment policy. We call, in the context of feasibility, to critically look at the balance between the number of questions about the financial situation and the insights this is expected to yield. RPOs with many such questions usually also had a high dropout rate. It is therefore advisable to explicitly state in the research design to what extent these questions contribute to translating the RPO into policy.

2.4 Points of Attention When Combining Methods
If, as a pension administrator, you choose a combination of methods and elements in your RPO, consider in advance in the research design how you will combine the answers to posed questions. This also forces you to think about the utility of questions. It also helps determine whether you can formulate a question better qualitatively or quantitatively.

To verify whether each part of the RPO actually measures what you intend (validity), it is important to present the results of each part separately in the research report – i.e., not only in combination with other parts. For reproducibility as well, it is important that it is transparent how the RPO results came about, especially if these results play a prominent role in communication about the RPO to the fund board.

Example Where Validity Is Important
In our investigation, we see questions aimed at investigating which policy fits the preferences of most participants. For example, by asking how participants feel about taking more or less risk than they initially chose. This is relevant for pension schemes with one collective investment policy or with a collective payout phase.

With such additional questions, it is important to record in advance how the results will be used. Only then can it be tested whether the part works as intended. Especially if the results of the part impact policy, we expect its validity to be tested.

RPO Report 7
ANALYSIS REPORT
3. Observations on the FRAME Criteria
When assessing RPO inquiries, the AFM uses the FRAME criteria. These stand for: feasible (uitvoerbaar), rationalising (rationaliserend), appropriate (passend), measurable (meetbaar), and errable (feilbaar). This chapter contains a brief overview of the findings per FRAME criterion, supported by examples.

3.1 'Errable': Detection of 'non-understanders' works well, but follow-up steps can be improved
An effective research method identifies participants who do not understand the questions well. For this, you must determine and record in advance how you will detect these participants and how you will handle their answers. Detecting 'non-understanders' can be done, for example, by allowing internally inconsistent answers or by posing a control question. You can also analyze whether certain groups of participants are overrepresented among the non-understanders. In that case, determine whether it is responsible to exclude their answers or if you can still correct their answers. In this way, you can ensure that the RPO is as representative as possible.

Most pension administrators in our investigation take inconsistent answers into account. However, it is often not exactly clear in which cases the inconsistent answers are still included in further analyses, or if the participant is considered a 'non-understander'. By writing this explicitly, the researcher and the pension administrator can have a good discussion about the reliability of the results, but also about possible signals coming from inconsistent answers and the size of the 'non-understander' group.

Good Practice
Example from the investigation:
After going through the choice sequence, participants are presented with the first choice from the choice sequence again. Participants who then give an inconsistent answer are excluded from the analysis for determining risk aversion.

3.2 'Rationalising': Limit irrational behavioral effects
For properly determining participants' risk tolerance, it is important to limit the influence of context-dependent, 'irrational' behavioral aspects.

In several RPOs, three scenarios are used to clarify the variation in future pension payouts: a disappointing scenario, an expected scenario, and a favorable scenario. But in some inquiries, it is unclear what probability each of these scenarios has. Often, it is only indicated that more investment risk brings a chance of both a higher and a lower pension. Without insight into the probability distribution, it can be difficult for participants to weigh the variants well.

In some RPOs, participants are presented with an option where all presented outcomes are lower than with the alternative option (see also paragraph 2.2). In practice, it sometimes happens that participants systematically choose the theoretically inferior option even in such a situation. This could be due to the visualization of the outcomes, by

RPO Report 8
ANALYSIS REPORT a wrong interpretation of the presented probabilities, by the order of questions, or by other subtly guiding elements in the questionnaire. For noticeable deviations from rational choices, we expect a reflection on this. When participants collectively choose the illogical option, this may indicate an underlying problem that you must try to identify through additional analyses.

People tend to weigh losses more heavily than gains of comparable size. This so-called 'loss