What changes are machine learning (ML) approaches bringing?

Machine learning is a popular term today, and one that is bandied about with increasing fervour, but is a topic that is difficult to quantify and one whose impact is hard to measure.

 For the majority of financial analysts working in the industry today, the basis for their approach is manual analysis of data however, the rapid rise of machine learning is driving a new generation of analysts supported by greater and greater machine computation.

While there are many ground-breaking uses of machine learning around, there are of course plenty of snakeoil applications as well and cutting through this segment is increasingly difficult as the majority is hidden by proprietary technology and opaque, blackbox operation.

Sentiment, Risk and Document Analysis

Despite drawbacks in oversight, leading banks and financial services companies are already deploying ML technology en masse, in a bid to streamline their processes, optimise portfolios, decrease risk and underwrite loans amongst other things.

Examples of successful applications with the analysis sector remain limited by the ability of operators in all cases. The ability to understand the sentiment of the data is vital for nearly all true analysis work and as such

Examples of areas where some success has been seen, include Portfolio Management – Robo-Advisors, Fraud Detection, Risk analysis, Document Analysis and Sentiment Analysis.

Of these it is the latter three which are likely to be the most important in future. Sentiment analysis will increasingly influence trading in the future, through analysis of enormous volumes of unstructured data like videos and video transcriptions, photos, audio files, social media posts, presentations, webpages, articles, blogs, and business documents to determine the market sentiment.

Risk analysis will become increasingly important as the complexities of fragmented global markets make the task of identifying and understanding the operation of next-generation business models in depth a task that relies heavily on big data.

For these reasons, ML will certainly be vital to the future development of quantitative finance than stochastic calculus will, but increasingly it will be seen as a tool to support, rather than a replace, traditional methods.

Hiring trends

At the moment, lots of people are hired for “machine learning” the way Wall Street treats any claimed technical skill: hire a bunch of people and put them in IT, pay them well but don’t treat them well, keep them around if they produce useful code, maybe let a few of them near decision makers and perhaps a few will prove to be good traders or developers of good products; figure most of them will be gone in a few years, or unrecognizable as machine learning specialists.

There are some serious efforts to develop useful machine learning financial applications, but not that many. These places hire people who have worked on objectively successful machine learning projects, not people with top degrees or who won prestigious machine learning competitions. The general view seems to be that technical machine learning education is cheap to buy, but that some people seem to be able to lead successful projects, and others do not.

So as a resume line item, “stochastic calculus” commands respect, “machine learning” is only a notch above the flavor-of-the-month hobby. But true ability in stochastic calculus is rare, and not very useful. True ability in machine learning is probably more common, and extremely useful.

How does ML expertise compare to traditional approaches?

Stochastic calculus was never an important practical skill. For one thing, almost no one can really do it. We only know the solutions to a few types of stochastic differential equations. What people call “stochastic calculus” consists mainly of recognizing when you can recast a problem as one of the known solutions, and how to fake it when you can’t. For another thing, not many places are interested in genuine financial research.

Nevertheless, stochastic calculus is hard, and an easily measured skill. It was always used more as an IQ test than something needed for the job. It is still respected on that basis. And since it’s central to the historical development of theoretical quantitative finance, anyone claiming financial quant skills who can’t solve textbook stochastic calculus problems is assumed to be unable to do it. I also think it’s worth learning because it does stretch your brain.