Bayesian Index Tracking: optimisation by sampling

About the Project

Index trackers (ETFs, index funds, index mutual funds) aim to replicate the return of a reference equity index while keeping implementation frictions—notably turnover and trading costs—under control. Index tracking is a passive investment approach that frequently beats their actively managed counterparts over the long term. In practice, full index-replication can be costly or operationally infeasible; consequently, pure tracking with a sparse basket is common for segregated mandates, often subject to constraints such as a full-investment budget and, in some mandates, long-only weights. A large body of work formulates index tracking as an optimisation problem with convex losses and penalties, or as mixed-integer programs that directly control support size and transactions. These approaches typically deliver a single allocation weight vector and tune regularisation hyperparameters by heuristics or cross-validation. Such tuning requires manual interventions and is often subject to uncertainty.

This project develops novel Bayesian methodology with the aim to return a posterior over weights, tuning parameters, and the residual variance that drives tracking error. The aim is decision-grade uncertainty quantification (UQ) and principled data-driven parameter selection. Hence, the project will develop automatic portfolio rebalancing driven by UQ analysis, based on the posterior distribution of the allocation weights.

We will model the tracking residual returns—index minus portfolio—as a noisy outcome with unknown variance, regressing the index returns on the returns of the individual assets. Portfolio weights live in a constrained space reflecting business rules: non-negativity or signed positions, leverage or budget limits, optional sector exposures, and explicit cardinality. Sparsity comes from imposing hierarchical priors that prefer few active names: either spike-and-slab families (selection indicators plus a continuous “slab” for active weights) or continuous shrinkage alternatives such as horseshoe or Dirichlet-Laplace. Tuning parameters are not fixed numbers in this approach: global and local shrinkage scales, inclusion probabilities, tracking-error and turnover multipliers, and the residual variance all carry their own priors. The project is expected to result in a coherent hierarchy that links portfolio structure, transaction costs, prior beliefs, and a data-driven narrative for model selection.

Inference combines two ingredients. First, state-of-the-art MCMC samplers, which are designed to handle non-smooth penalties and hard constraints. These samplers perform gradient-informed random walks while enforcing feasibility at every step. These take advantage of recent advances and contribute to convex nonsmooth optimisation and numerical analysis of stochastic differential equations. Second, Gibbs or Metropolis updates for the hyperparameters: residual variance, global shrinkage, local scales, and selection indicators.

Alongside expert-set priors, the project proposes to also use machine learning techniques to learn parts of the prior and penalty structure from data in an interpretable way. Examples include mapping liquidity and volatility features to a prior preference for sparsity, learning sector-level shrinkage from historical co-movement, or learning a turnover multiplier that depends on spread and market depth. The plan is to start with simple parametric maps and progress to neural networks with monotonicity or convexity constraints so that learned priors remain stable and defensible. Expert ranges can be encoded as hyperpriors, allowing the data to refine but not overrule domain knowledge.

Funding Notes

This project is for self-funded and externally funded students only.

References

MPS - Research Cluster (Mathematical and Statistical Modelling)
https://sheffield.ac.uk/mps/research/maths/mathematical-and-statistical-modelling