Differential machine learning (ML) extends supervised learning, with models
trained on examples of not only inputs and labels, but also differentials of
labels to inputs.
Differential ML is applicable in all situations where high quality first
order derivatives wrt training inputs are available. In the context of
financial Derivatives risk management, pathwise differentials are efficiently
computed with automatic adjoint differentiation (AAD). Differential ML,
combined with AAD, provides extremely effective pricing and risk
approximations. We can produce fast pricing analytics in models too complex for
closed form solutions, extract the risk factors of complex transactions and
trading books, and effectively compute risk management metrics like reports
across a large number of scenarios, backtesting and simulation of hedge
strategies, or capital regulations.
The article focuses on differential deep learning (DL), arguably the
strongest application. Standard DL trains neural networks (NN) on punctual
examples, whereas differential DL teaches them the shape of the target
function, resulting in vastly improved performance, illustrated with a number
of numerical examples, both idealized and real world. In the online appendices,
we apply differential learning to other ML models, like classic regression or
principal component analysis (PCA), with equally remarkable results.
This paper is meant to be read in conjunction with its companion GitHub repo
https://github.com/differential-machine-learning, where we posted a TensorFlow
implementation, tested on Google Colab, along with examples from the article
and additional ones. We also posted appendices covering many practical
implementation details not covered in the paper, mathematical proofs,
application to ML models besides neural networks and extensions necessary for a
reliable implementation in production.