The assessment of risk is fundamental to advisers and planners in their quest to ensure that clients benefit from effective and appropriate portfolio construction and asset allocation. This is the second and final part of his series for IFA Magazine on the important topic of risk-profiling, Keith Robertson continues to challenge adviser thinking when it comes to the process of deciding what constitutes “suitable” asset allocation.
Probably the biggest regulatory and legal risk for financial advisers today is their use of Risk Profiling-cum-Asset Allocation (RiPAA), which has become ubiquitous and the overwhelmingly predominant investment process.
Most risk-profiling questionnaires are designed to force respondents to provide answers which will tend towards the mean – a ‘moderate’ portfolio – at the end of the RiPAA process. The most critical questions are framed to deliberately mislead and divert attention from reality. There is usually a composure question along the lines of, “If markets weakened, would you be willing to accept a loss of 0% / 5% / 10% / 20% / >20%?” Most people opt for 10%, because they realise markets do go down and they can imagine 10% doesn’t seem too bad, but more than 20% would be upsetting. Many questions are designed so most people will pick the median number, resulting in most clients clustering round the mean; this is not accidental. Pairs of questions answered in a contradictory way will balance each other and the algorithm will revert to the mean, not looking any deeper for causes of the contradictions. Total scores will tend heavily towards the three moderate portfolios, flanking the median, in the asset allocation phase, avoiding regulatory notice which may look for extremes in the distribution. Practitioners who use questionnaires regularly will be aware of this clustering phenomenon.
It depends…
The composure question above is just one example of framing a question to distract and mislead. The honest answer about willingness to accept loss, indeed the honest answer to most key questions, would start with, “Well, it all depends . . .” For example, given the range of options from 0% to >20%, honest (and revealing) answers which clients may wish to make could include:
- “It all depends on whether you will impose a stop-loss at that point. If you can’t, what’s the point of the question? How could I know if a 10% fall was just the beginning of a much more serious market collapse?”
- “If my portfolio has just put on 60% in the last three years, and you can persuade me that any fall is only a technical reaction, with all the fundamentals still in place for the bull market to continue, then yes, I could cope with a 10% fall. However, if I heard on the news that an epidemic of virulent bird flu had been declared across Asia, or that China had sunk a Japanese warship in the Spratly Islands, I would not wish to wait for a 1% fall; I should want out immediately.”
The structure of questionnaires cannot accept ‘open answers’ because that generates more variables than naïve algorithms can cope with. It is no accident that all providers use ‘closed questions’, where one from a small number of pre-determined answers must always be accepted, in default of which the algorithm will normally score in the middle.
The framing of questions is grossly prejudicial to clients: consider the composure question and its range of answers from 0% to >20%. As explained, clients typically tick the middle answer. Historically, the worst top-to-bottom loss in US markets has been 89% and >90% in emerging markets. So, if the question were framed, as it logically should be to reflect what has happened in real life markets, “would you be willing to accept a loss of 0% / 25% / 45% / 65% / >85%?” there is no possibility clients would cluster around the median 45%; there would be a massive weighting towards zero loss.
Similarly, if the question contained additional historical information, the responses would be radically more conservative. For example, explaining that markets have quite often fallen 30%-40% in one year and continued to lose money into a second or beyond, a 10% loss just might be acceptable if it ended there, but if it were made clear the bottom of a bear market is totally unpredictable, again one would expect clients to dash for safety at zero loss. Can advisers explain to clients how and why the asset allocation is altered according to answers to composure questions?
The appliance of science?
Giving clients this sort of realistic information is bad news for RiPAA product manufacturers. The whole purpose of risk-profiling is not to inform, educate, or actually uncover clients’ deepest concerns about the risk of loss or the depth of regret likely to be felt, far less to protect them from loss, but to come up with a result that can be spun as ‘scientific’ and looks impressive. Ultimately, the purpose of RiPAA tools is simply to get clients invested with a decent weighting to risky assets, and give an illusion of ‘scientific’ certainty. It’s about marketing, stupid!
Don’t be seduced by other ‘scientific’ claims. Some questionnaires claim ‘psychometric’ validity, implying they are a cut above others. The psychometric tag refers to the measurement of the questionnaire scores to confirm that the range fits a normal (Gaussian) distribution and reflects the general population. This is done by doing the tests on many people (the normative sample), analysing the distribution of the range of scores to confirm it fits a bell curve and calculating a standard deviation. The fact that a provider’s scores have been normed and statistically validated tells one nothing about the validity and reliability of the underlying questions themselves or the variables being tested. If the questions are not fit for purpose to measure what is claimed to be measured, the outputs are still likely to be rubbish for practical purposes – albeit statistically validated rubbish.
Some RiPAA tools claim to measure psychological traits which prove to be stable over time and through market cycles, shown by numerous clients repeatedly completing the same questions over several years. The fact that pretty much the same results come out every time is said to demonstrate the ‘scientific’ strength and validity of these tools. Yet there is no doubt that, during market crashes and resulting loss, people’s feelings around risk change dramatically, and they wish they had had less exposure to it. So why measure a trait that never changes? It is absurd to knowingly measure something that has no diagnostic value, can never alert either client or adviser to possible danger ahead, or uncover fears about loss which will become apparent only after it is too late.
Painting-by-numbers approach
Given the implied promise of RiPAA to measure a client’s ATR and use that information to generate a portfolio specifically tailored to meet his ‘suitability’ requirements, one might reasonably expect a rational process whereby an investment manager looks at the questionnaire answers, detecting the client’s key concerns about things going wrong, and incorporates that information to create a bespoke asset allocation. The opposite happens. Nothing is directly created, but a sort of painting-by-numbers approach is used to push clients into one of a series of ready-made model portfolios (usually near the middle range), using a process which has no basis in finance, economic or mathematical theory.
A set of pre-determined portfolios is laid out to form an ‘efficient frontier’ on a graph, with the x-axis measuring subjectively-selected historical volatility, and the y-axis showing mean return, both in the same units – probably annualised percentages. Incorporating client questionnaire data is technically impossible. ATR, risk tolerance, risk appetite etc., have no units of measurement; at best, questionnaire results can only be an ordinal number, indicating a position within a range. Ordinal numbers (e.g. 5th floor, 1st in a race, 19th hole) cannot be used mathematically: they cannot be added, divided, multiplied, plugged into an equation or mathematical model.
Nevertheless, that is what happens. The x-axis changes from volatility to ‘RISK’ and, no matter what attitude or trait was supposed to have been measured, that metric is deemed to actually be the level of volatility that suits the client. Nobody knows how the x-axis is now calibrated: where does ‘risk-averse’ start on the axis, and where does ‘risk-seeking’ end? Does anyone care? A line is drawn vertically from wherever the score has been overlaid on the x-axis, and where it hits the efficient frontier is deemed to be the suitable portfolio. This painting-by-numbers process is totally illusory and fake.
RiPAA uses average values for return and variance. An asset allocation today will be pretty much the same as six months or six years ago. Using only volatility as ‘risk’ and the assumption that risk and return are essentially stable and fixed, the process is blind to the fact that assets’ riskiness changes dramatically over time. Equities were hugely more risky in March 2008 than March 2009, at which point they were rather low risk over the long-term. The sole determinant of long-term return is the entry point into the asset price cycle. Buying equities today will guarantee below-average future long-term real returns.
Finally, the client is required to sign-off on his risk-profile and the portfolio derived from it, without knowing what the outcome is likely to be, and almost certainly without having understood how RiPAA is supposed to work or having its flaws and weaknesses explained. It is believed that the client’s signature can stand as prima facie evidence against him in the event of his making a future claim. If he does not sign, the intermediary will not proceed. This is risk of an entirely novel sort, one feels.
Business risk
RiPAA brings risk to everyone involved, particularly intermediaries and their clients. The FCA Handbook (PRIN and COBS sections) requires advisers
- To treat customers fairly;
- Act in clients’ best interests;
- Ensure all information is fair, clear and not misleading;
- Ensure all advice meets stringent suitability tests.
The Regulator’s Final Guidance on Suitability [March 2011] does not give any RiPAA system a clean bill of health; the FCA is equivocal and ambivalent. But advisers must understand the tools they use and explain them to their clients. Beware: the Regulator probably does not fully understand RiPAA tools either but, in real life, you can be certain they will always be either right or not wrong when it comes to enforcement.
If that’s not enough, read the Consumer Rights Act 2015. If you say something that the client believes to be true just because you have said it (maybe telling him you can measure his ATR) and something goes wrong, you and your PI insurer may well be liable.
If you don’t believe in fairies, don’t believe in RiPAA systems unless your provider can prove to you that the risk-profiling questionnaire measures something real, definable, and precisely measurable, and can explain how a ‘suitable’ asset allocation is derived.
To read part one of this series, click here
About Keith Robertson
Keith Robertson spent over twenty years as a practising fee-charging financial planner and investment manager, and was one of the highest qualified practitioners in the UK. Active in both the CISI and PFS-CII, he reviewed and co-authored professional exam course-books, was London’s inaugural Chartered Champion, and continues to sit on CISI masters’ level exam panels and forum committees. Since 2016 he has focused on consultancy and training, particularly investment strategy and risk. He is a fierce critic of current orthodox systematised and quantitative approaches to investment, concerned that advisers are not taught critical thinking to analyse its weaknesses, leaving clients in no position to understand the risks they are being exposed to.
