Reframing Performance into Risk Management
A conversation with Raphaël Douady.
Raphaël is a French mathematician and economist, with more than 20 years of experience in the banking industry and 35 years in research in pure and applied mathematics. He is affiliated with Paris 1-Panthéon- Sorbonne University and the French National Centre for Scientific Research. From 2015 to 2018, Raphaël held the Frey Family Endowed Chair of Quantitative Finance at Stony Brook University in New York, and was, prior to that, Academic Director of the Laboratory of Excellence on Financial Regulation in Paris. Raphaël co-founded Riskdata and Datacore, two fin-tech firms. He is also a director at Praxis, a prominent Franco-American think tank. Advising top executives in banking and industrial firms, Raphaël is renowned around the world for his pedagogy and his ability to tackle the most complex questions with a systemic approach.
Tusker Club - To what extent can rankings help leaders make better decisions?
Raphaël Douady - Let me start by saying that the worst that can happen to a ranking is to be followed. By that, I mean one should not mistake packaged information for actual performance. A ranking reflects a set of choices made to make sense of a complex problem. Its value is descriptive, not normative. Until you care about it.
By giving too much importance to rankings, the system works towards its own failure. It is only natural to develop strategies to outsmart them, which doesn’t drive innovation but compliance. Even if you were to factor some diversity or originality criteria into the ranking, the dynamic would be the same. Think of affirmative action!
A parallel with the Darwinian theory of evolution can help us understand this phenomenon. The survival of the fittest requires adaptation of species, not of their individual representatives or particular interest groups if you draw the comparison beyond biology. Otherwise stated, the performance impetus of Darwinism is collaborative, which rankings don’t incentivise.
TC - How can artificial intelligence help?
RD - In principle, the more you train AI, the better the quality of its insights. Indeed, in the financial industry, as in many others, the trend is first to ask companies to disclose a large amount of information, only to assess their practices against it later.
For instance, the US Securities and Exchange Commission asks financial institutions to provide comprehensive logs of their trading activity to keep them in check. Practically, it means that so long as they comply with the regulation, financial institutions are not responsible for the systemic risk that they may induce. Essentially, they are giving organisations agency without responsibility—a perverse effect of a system mistaking transparency for accountability.
What’s more, information processing has a cost. So, the regulators’ taste for information disclosure favours more prominent players with enough resources to devote to compliance. The European Union is a case in point across sectors, from finance to sustainability.
As much as AI helps, it is not sufficient to drive policy or decision-making, and the risk is to subsume a political discussion into a technical one. Coming back to Darwin, one can consider that human intelligence has been trained for thousands of years, balancing limited cognitive abilities with instinct. And that feeling that sometimes ‘something is not quite right’ is the first step towards responsibility.
TC - How can decision-makers work around those issues?
RD - AI highlights the fact that the value of an insight is not based on the quantity of data processed, but on the relevance of the information considered. To that challenge, there is a mathematical and a political answer.
In my work with the financial industry, I have developed the theory of non-linear poly-models. Instead of processing millions of data points, it focuses on a few key variables, which are indicators of systemic risk. The question is, which variables will be affected by systemic shocks? Contrary to rankings, the implications are far-reaching, especially regarding market prediction and, particularly, crisis prediction.
Such an approach has political consequences. From an economic perspective, it challenges the rent of economic players whose profit is directly linked to compliance and rankings (think of auditors). From a more abstract perspective, it requires reframing stability into resilience, which is an evolution of the social contract. The public management of the recent pandemic provides examples aplenty.
Hence, there is a need to discuss rankings design and governance regularly (an example is the recurring discussions on the relevance of GDP as a measure of countries’ wealth). Who’s responsibility it is to do so is unclear and may explain why reflections on GDP remain inconclusive. In fact, this was a role traditionally bestowed on academia. But as an editor of several academic publications, I can attest that in a world where you ‘publish or perish’, fashionable arguments and cronyism can sometimes trump merit.
The value of an insight is not based on the quantity of data processed, but on the relevance of the information considered.
TC - What makes the governance of rankings so tricky?
RD - Rankings are a response to an organisation’s need to achieve scale and to an individual’s need to apprehend the complexity of our world. Rankings are only detrimental to the extent that they make it easier to give in to fake impressions of homogeneity, stability, and fairness at the expense of context and diversity.
Balancing this natural temptation can be a mathematical and cultural struggle, as it requires the ability to conceive non-linearity and to accept contradiction. Another challenge stems from the fact that anyone in a position of power has the incentive to maintain it. As rankings drive resourcing, the battle is deemed to be unfair and makes collaboration more difficult.
That may be the real challenge in dealing with performance measurement: one cannot distinguish the technical questions from the sociological, political, and philosophical ones.
TC - What advice would you give to decision-makers for dealing with rankings?
RD - Let me start by saying: don’t hate the player, hate the game! Rankings are a reality for any leader, from classrooms to boardrooms. For decision-makers to make the ‘best’ use of rankings, I would suggest reflecting on three questions that have to do with their relationship to norms, alterity, and risk:
Considering that rankings are an indication of what the system accepts and values (a normative balance of power), what can you learn about the level of resistance you may face in achieving your vision?
Considering that rankings create their audience, what level of diversity in opinions and approaches are you willing to accept to manage risk?
Considering that rankings don’t inform you of your blind spots, where do you expect to lag behind, and why?
In other words, rankings can help decision-makers reframe an issue of performance into one of risk management. Far from being a defensive move, they will be better equipped to deal with uncertainty. It also means that whomever you want out of the room is the one you need to pull a seat for: that’s the challenge that rankings pose.
Interview by Baptiste Raymond - 08/2023.