Consumer testing SFDR in the Netherlands - Autoriteit Financiële Markten

Consumer testing on financial product disclosures in pre￾contractual disclosures under the Sustainable Finance Disclosure Regulation June 2023 Authors:
Louise Nell, Zoë du Chattel, Annelot Wismans

Contents
Contents.............................................................................................................................................. 2
Introduction ........................................................................................................................................ 3
Executive Summary ............................................................................................................................ 4
Objectives ........................................................................................................................................... 5
Methodology....................................................................................................................................... 5
Detailed results................................................................................................................................... 7
Dashboard ........................................................................................................................................... 7
Decarbonisation section.................................................................................................................... 10
Asset allocation ................................................................................................................................. 12
Taxonomy .......................................................................................................................................... 14
Recommendations ............................................................................................................................ 16

Introduction
On 28 April 2022 the Commission mandated the European Supervisory Authorities (ESAs) to propose amendments to the Regulation (EU) 2019/2088 on sustainability-related disclosures in the financial services sector (hereafter the Sustainable Finance Disclosure Regulation, or SFDR) Delegated Regulation including to add decarbonisation targets in financial product’s disclosure templates. The ESAs considered it necessary to propose additional amendments to the financial product disclosures in order to improve the readability of these documents. In order to gather feedback from consumers on these draft proposals, the ESAs are conducting a consumer testing on the draft proposals in four of Member States (Poland, France, Netherlands and Italy) in cooperation with National Competent Authorities and academic institutions forming part of the ESAs stakeholder groups. The financial product disclosure templates required under the SFDR Delegated Regulation are part of existing pre-contractual and periodic product disclosure requirements under different EU sectoral legislation 1 . The SFDR empowers the ESAs to develop draft RTS, not only to specify the details of the “content” of the information to be disclosed, but also the details of the “presentation”. This report describes the consumer research conducted between April and May 2023 in the Netherlands by the Netherlands Authority for the Financial Markets (AFM) on the updated SFDR template for a product promoting environmental or social characteristics, with regards to the disclosure of the dashboard, decarbonization disclosures, asset allocation and Taxonomy disclosure. See Articles 6(3) and 11(2) of the SFDR as regards the existing sectoral disclosures. This includes, for example, product disclosures provided by AIFMs, UCITS management companies, insurance undertakings, IORPs, PEPP providers etc.

Executive Summary
This research was conducted by the Netherlands Authority for the Financial Markets (AFM) in the Netherlands. The primary objective of this study was to gain insights into the elements of the SFDR￾template that ought to be preserved and those that require enhancement focusing on document understandability and visual appeal. The sample of this study was a representative group of 239 Dutch people aged 18 or above who already invest or who consider investing. The ‘Tag-it' tool was used to collect data. This online tool allows participants to tag aspects in the SFDR document that they like and dislike (using green, orange, and red tags) and comment why they (dis)like this specific aspect. Participants also completed a content related questionnaire to test whether they understood the document. In this way both qualitative and quantitative insights were acquired. Summary of Results

• Understanding of the document is low. Although it can never be expected that all readers
  understand everything they read, the results of the knowledge questions are below par. For 6 out of 10 questions, less than 50% of the participants give the correct answer. The highest amount of correct answers is 67%.
• The aim and context of the document are not clear. Throughout the document people are
  confused about its purpose. Why should they read this document? How does it help them? They often assume the document has a marketing purpose. Participants also consider the document as being a ‘stamp of approval’ that the product is sustainable, because of the existence of the document. They showed surprise when encountering seemingly contrasting information. Moreover, it seems that readers need more context how they can assess information.
• Text and concepts are difficult to understand. The language level is too high for most readers and
  a lot of jargon is used (e.g. precontractual, taxonomy, asset allocation). Also, participants noted that text was too vague (e.g., ‘What are sustainability criteria?’). Specifically, the concept of Taxonomy is too complex. There was a general low understanding of its relationship with sustainable investments. This was underlined by the low percentage of participants (33%) who answered the question on the relationship between sustainable and taxonomy investments correctly.
• Participants note that the design (both textual and visual) of the document is not attractive and
  should be improved. This relates amongst others to mixing different fonts and font sizes, the use of colours, the design of visualisations and messy lay-out. Related to this, participants note there is too much text in the document. This makes it unappealing to read the document. Participants suggest the use of bullet points or visualizations when text is long.

Objectives
This report brings together the work for the European Supervisory Authorities (ESAs) carried out over from March to June 2023 as part of the consumer testing of the revised SFDR disclosures. Following a behavioural approach, this report provides evidence to support the ESAs in forming a view on the format and content of updated templates for disclosures for financial products disclosing under article 8 and 9 of the SFDR, which will be delivered as draft Regulatory Technical Standards to the European Commission in 2023. The results outline how information should be presented in the SFDR financial product templates, because deemed to be essential or useful by consumers, and how such templates should be designed to be user-friendly. The purpose of this study was to test the consumer experience of the revised SFDR pre-contractual disclosure, with focus on the ‘dashboard’ that presents summary/key information and subsequent
section on decarbonization objectives, as well as additionally specific disclosure on asset allocation and
on whether environmentally sustainable investments made are Taxonomy-aligned. This research was conducted by the Netherlands Authority for the Financial Markets (AFM) in the Netherlands. The primary objective of this study was to gain insights into the elements of the SFDR￾template that should be preserved and those that require improvement. In this study, we focused on understandability and visual appeal of the document. Specifically, we aimed to answer the following questions:

• How do (potential) investors generally evaluate the document when it comes
  comprehensibility, clarity, usefulness, and aspects such as language use and design?
• To what extent does the target audience understand the information provided (text, icons,
  graphs)?
• What does the target audience find unclear? What raises questions?
• What are the areas for improvement in the document?
  Methodology
  In this study, the following parts of the SFDR-template were studied:
• The dashboard (part 1)
• Decarbonisation (part 2)
• Asset allocation
  o Excluding visualization (part 3A) o Including visualization (part 3B)
• Taxonomy disclosure (part 4)
  For this study we made use of the online ‘Tag-it’ tool from research agency Ruigrok to collect qualitative insights on the document and a knowledge questionnaire to collect quantitative data and formally test the level of understanding.

At the start of the study, participants had to report their gender, age, education level, whether they currently (intend to) invest and to which degree they are interested in whether products and services consider their impact on the environment, people and/or the society. Tag-it tool Tag-it is an online tool which allows participants to place comments on a document to indicate the aspects that they like and dislike. After placing a tag, the participant inserts a comment explaining why they (dis)like the specific aspect. The tags can be compared to digital ‘post-its’ and can be placed anywhere on the document. Participants can make use of red (negative), orange (neutral) and green (positive) tags. Subsequently ten multiple-choice questions related to the content of the document were asked to test understanding. Sample The sample of this study was a representative group of 239 Dutch people aged 18 or above who (consider to) invest. The sample was approached using online consumerpanel Panelclix. To make sure the sample is representative, quotation was used. Weights were applied in reporting the quantitative data from the questionnaire, leading to a weighted sample of 250 people.

Detailed results
Fragment 1: Dashboard
First impression (Q1)
Many comments were related to the fact that context is missing and that the aim of the document is unclear. Participants reported that it is unclear how and why they should use this document. This is examplified by the following comments: ‘I don’t know how to interpret this information’, ‘A lot of details, but no context is given’, and ‘Percentages are meaningless, how should I read them?’. Moreover, people saw the document as a ‘stamp of approval’ and expected that the product is sustainable, because the document is about sustainability (e.g., ‘Is it actually sustainable or not??”). There were multiple remarks about the use of difficult words. Specifically related to the title (‘Pre￾contractual’) and the use of the word ‘Taxonomy’. The explanation on the left side of the document is appreciated, but was considered being (far too) vague. This is shown by comments such as ‘What is the definition of sustainable?’, ‘What’s this about? What are the criteria?’, ‘Generic’, and ‘What is contribute to?’. Finally, the lay-out of the dashboard is considered outdated and messy. Perception on implication of green colouring (Q2) Participants were positive about the use of coloursin general. However, the green colour is interpreted as indicating that the product is sustainable. This is confusing to some, for example indicated by the comment: ‘The green color does not match with the characteristics of the product’.

Feedback on the “first box” which includes the disclaimer that this product has some sustainabilty characteristics, but does not have sustainable investment as its objective This part mainly received positive comments. Participants thought the disclaimer was honest, upfront and clear (e.g., ‘this is an honest statement’). The transparency is appreciated. Nevertheless, judging by later comments, this message was not perceived by or was unclear to a large group of participants who reported being surprised by information that indicated that the product does not have sustainable investment as its objective. Feedback on the relation between Taxonomy investments and sustainable investments (Q3 and 4) This aspect received many remarks in the Tag-it tool. Participants reported that they thought both the information and the visualization were too difficult to understand. The concept of Taxonomy is too complex, even after reading the explanantion in the box on the left side. The relationship between Taxonomy investments and sustainable investments was not understood by many. Table 1 below shows that only 33% of participants correctly answered the question relating to this link. There was a better understanding of the total share of sustainable investments, as reported in Table 2 (62% answered correctly).
Table 1. Reported answers (in %) to question “What is the relationship between sustainable and
taxonomy investments?” (n=250)
Answer options
Sustainable investments are part of taxonomy investments 20% Taxonomy investments are part of sustainable investments 33% Sustainable investments have nothing to do with taxonomy investments 15% I don’t know 33%
Table 2. Reported answers (in %) to question “What is the total share of sustainable investments?”
(n=250)
Answer options
7% 14%
5% 62%
2% 12%
I don’t know 12%
Feedback on whether it is clear that the disclosure indicates commitments to invest (Q5) As can be seen in Table 3, a little more than half (53%) of the participants understood that the disclosure indicates a promise to make the investments.
Table 3. Reported answers (in %) to question “Are the investments mentioned in the text actual
investments, or is it a promise to make these investments?” (n=250)

Answer options
Actual investments 27%
Promise to make these investments 53%
I don’t know 20%
Feedback on decarbonisation icon (Q7)
None of the respondents placed a ‘tag’ to give feedback on this icon.
CO2
Proposed decarbonisation icon

Fragment 2: Decarbonisation section
Overall feedback (Q7 and Q8 on whether the information is understandable or too detailed) As can be seen from the tags placed on the document in the image above, the orange textbox received quite some negative comments. This mainly related to the fact that the orange textbox is considered contradictory to the rest of the text. Some examples of comments are: ‘This piece of text leads to confusion because it contradicts what is said in the rest of text.’ and ‘By formulating this so firmly, the effect of this product is negated. Surely it does contribute to this goal?’. The ‘answers’ to the questions in the document are considered being too long and difficult (e.g., ‘Difficult explanation for a layman’, ‘A lot of details and specific language’ and ‘Lots of text to tell that the goal is to reduce greenhouse gas’). Participants also commented on the fact that the answer that financial firms did not pick was still shown in the text. They considered this confusing and unprofessional. The lay-out of this part was considered boring and grey (‘A little more colour would have been nice’). Mixed comments were given about the table on GHG reduction targets. Participants liked that it provided a clear overview. However, it was reported that it is too complex to understand. Participants do not know how to interpret the numbers. Many comments were given related to the complexity of tCO2-eq/€M. The explanation of GHG in the upper textbox received many positive comments. It was considered clear and useful. Understanding of what is the GHG reduction target for the whole product by 2030 (Q9) As shown in Table 4, the understanding of the GHG reduction target for the whole product by 2030 is relatively good with 67% of the participants providing the correct answer.

Table 4. Reported answers (in %) to question “What is the GHG reduction target for the whole
product by 2030?” (n=250)
Answer options
Reduce emissions by 50% 67%
Reduce emissions by 15% 10%
Reduce emissions by 70% 10%
I don’t know 13%
Understanding of how the GHG emission reduction target is achieved (Q10) As shown in Table 5, there is limited understanding of how this reduction target will be achieved. Only 30% of the participants provided the correct answer to this question.
Table 5. Reported answers (in %) to question “How will the greenhouse gas emission reduction
target be achieved?” (n=250)
Answer options
By allowing companies in which investments are made to reduce their emissions 11% By investing less in companies that emit a lot of greenhouse gases 26% A combination of the top two answers 30% By offsetting greenhouse gas emissions with carbon credit 10% I don’t know 23%

Fragment 3: Asset allocation
In this part, we randomly allocated the participants to two groups. The first group (3A) saw the document on asset allocation only including textual explanation. The other group (3B) saw the document that, besides textual explanation, also included a visualization of this information. General feedback Overall, the language that is used is too difficult and descriptions are complex (e.g., ‘Even after rereading several times, I still don’t understand this text’ and ‘what is taxonomy?’). There were some remarks about the term asset allocation, which was not a known term to everyone. There were multiple negative comments on the length of the text and the way information is provided. In the version without the figure, suggestions are made to use a diagram or table to present information. Participants in both versions suggest to use bullet points. Participants are positive about providing a link to the website where more information can be found. From existing research, we know that these kinds of references are useful to prevent information overload. A condition is that links clearly show what information they refer to, and that they are as specific as possible. In the version including the figure, positive comments are given related to the lay-out of the page which combines both text and visualization. Participants are positive about the use of the diagram, though the diagram needs be improved. People report that it does not look well-designed and that the colors used are not accesible for people that are color blind. Finally, there are questions about the definitions used and what the category ‘other’ entails. Understanding of figures (Q11 and Q12 - investments that promote E/S characteristics but are not sustainable and investments that are not promoting E/S characteristics)
Table 6 below show that only 39% of the participants correctly understood what the percentage of
investments are that promote the E/S characteristics but are not sustainable. Strikingly, this percentage was higher in the group that saw the document excluding the visual (3A: 48% correct)

compared to the group that saw the same text including the visual (3B: 31% correct). This difference between the two groups is significant at 5% level. Although participants reported liking the visualization – or even requested a visualization in group 3A – this shows that it does not add to a better understanding and even seems to lower it. We suspect that this problem has to do with information overload: the amount of information then becomes unmanageable and leads, for example, to people no longer reading information (sufficiently) and making it more difficult for them to distinguish between main and side issues.
Table 7 shows that 56% of particpants correctly identified the percentage of investments that are not
promoting E/S characteristics. This percentage is almost identical between the two groups (3A: 55% correct; 3B: 56% correct).
Table 6. Reported answers (in %) to question “What is the percentage of investments that promote
environmental or social characteristics and are not sustainable investments?” (n=250) Answer options Total (n=250) 3A – excl. visual (n=113) 3B – incl. visual (n=137) 65% 39% 48% a 31% a 70% 26% 23% 29% 5% 17% 13% 21% I don’t know 18% 16% 19% a Signficant difference between the two groups using a two-sided test and a 5% significance level.
Table 7. Reported answers (in %) to question “What is the percentage of investments that are not
promoting environmental or social characteristics?” (n=250) Answer options Total (n=250) 3A – excl. visual (n=113) 3B – incl. visual (n=137) 10% 10% 6% 12% 20% 11% 9% 12% 30% 56% 55% 56% I don’t know 24% 29% 20%

Fragment 4: Taxonomy
Feedback on the footnote (Q13)
As we know from existing research, people often skip texts included in frames or footnotes (also known as ‘banner blindness’). The people who do read the footnote, consider the text to be ‘vague', 'unclear', 'hard to read' and ‘too long’. The assumption that the footnote is read nor understood in many cases, is confirmed by Table 8, which shows that only 41% of the participants answers the question about the information in the footnote correctly.
Table 8. Reported answers (in %) to question “Based on the above text, which statement is correct?”
(n=250)
Answer options
Fossil gas and nuclear activities can never be in line with the EU taxonomy 8% Fossil gas and nuclear activities can be compliant with the taxonomy as long as they do not harm the objectives of the taxonomy 22% Fossil gas and nuclear activities can be compliant with the taxonomy as long as they do not harm the objectives of the taxonomy and contribute to mitigating climate change 41% I don’t know 28%

Feedback on whether the product makes Taxonomy-aligned investments (Q14) based on the text Overall, people find the information in the text hard to understand and (especially the text above the diagrams) unclear (e.g. 'Very unclear text', ‘A lot of blablabla but no clarity’ and ‘I don’t get what is said here’). The fact that people find the information complicated is also shown in table 9. Only 30% of the participants answer the question correctly. Almost as many readers (29%) indicate that they do not know the answer. Also, a lot of jargon is used in the textboxes in the left, such as share of revenue, turnover, capital expenditure, operational activities, renewable power, low-carbon fuels etc. These words are not part of readers' everyday vocabulary, making it more difficult for them to read and understand the text. There is also confusion about how the concepts in the textboxes relate to the information in the main text.
Table 9. Reported answers (in %) to question “Does this product make EU Taxonomy investments?”
(n=250)
Answer options
Yes, but not in fossil gas or nuclear energy 27% Yes, including in fossil gas or nuclear energy 14% No 30% I don't know 29% Feedback on the graph – including understanding on whether the product makes Taxonomy-aligned investments (Q15) The diagrams are not clear to everyone and uninformative in this particular case (where the percentage of taxonomy-alighed investments is 0%) (e.g. ‘this diagram does not provide much information', 'Unclear diagram' and 'Useless diagram since the percentage is zero, does not add anything'.) This has consequences for their comprehensibility, as the knowledge question shows: only 31% of the respondents gives the correct answer. Almost 70% either interprets the diagrams incorrectly or does not know how to interpret the information.
Table 10. Reported answers (in %) to question “Based on the graphs (above), please select the
appropriate option” (n=250)
Answer options
There are no taxonomy-aligned investments 31%
There are some taxonomy-aligned investments and no taxonomy￾aligned investments in fossil gas or nuclear energy 37% I don’t know 33%

Recommendations

1. Make the title of the document more informative and easier to understand. Also, add a clarifying
   sentence to give readers context on how and why they should read the document
2. Hire a copywriter that can rewrite the text from a non-expert perspective.
3. Give financial institutions tools when filling in the template (e.g. maximum text length, use of
   words).
4. Give readers cues or tools to assess information in context of their own situation.
5. Use an experienced designer with knowledge of how readers use documents to improve both
   lay-out and design.
6. While the use of visualizations is valued, our results show that it does not guarantee a deeper
   understanding. It is crucial to ensure that visualizations are well-designed, capable of conveying information independently, and thoroughly tested for their comprehensibility.