Machine Learning for Asset Management

Question: can a model learn a relationship that remains useful on observations it has not seen?

The advanced portfolio lessons examined how estimated inputs affect an investment decision. Machine learning extends this work by learning relationships from data and evaluating their behavior on later observations. A good fit to historical returns is only the beginning: we must define the target, establish what information was available, and compare predictions with a meaningful baseline.

Course status: this overview and preparation guide establish the next learning sequence. Chapters 3.1–3.5 are planned; their executable lessons will be added in that order.

Before You Begin

You should be comfortable with pandas tables and date indexes, NumPy arrays, regression coefficients, and the distinction between expected and realized returns. Review:

• Style and Factor Exposures for regression, residuals, and the difference between exposures and actual holdings.
• Covariance Matrix Estimation for sampling uncertainty and the effect of structural assumptions.
• Expected-Return Shrinkage for the trade-off between noisy estimates and a structured target.
• The Portfolio Capstone for comparing decisions under common constraints, dates, and costs.

You do not need prior machine-learning experience. The first lesson will introduce features, targets, fitting, and prediction before using a model-selection library.

The Planned Learning Sequence

3.1 — Fundamentals of Machine Learning

Distinguish supervised learning, where training examples have a target, from unsupervised learning, where the aim is to describe structure. Construct a simple prediction problem, compare it with a baseline, and separate training, validation, and final evaluation. Explore why a more flexible model can fit past observations better while predicting later observations worse.

Experiment: change model complexity on a fixed teaching dataset and compare training and validation error using ECharts. Explain the result before selecting a model.

3.2 — Robust Estimation of Factor Models

Return to the factor regression from Chapter 2.1. Study correlated predictors, coefficient instability, LASSO and Elastic Net, and the role of regularization. Fit preprocessing within each training window and choose the penalty through chronological validation before forecasting the held-out period.

Experiment: vary regularization strength and inspect coefficient paths, prediction error, and the difference between an explanatory regression using contemporaneous factors and a forecast using available predictors.

3.3 — Learning Dependence for Diversification

Use clustering and graphical models to examine relationships among assets. Distinguish ordinary correlation from conditional dependence represented through a precision matrix. Study how a sparse estimated network changes the description of diversification and how stable those relationships are across samples.

Experiment: compare a correlation heatmap with a sparse network, then adjust sparsity. A graph edge represents an estimated statistical relationship, not proof of economic causation.

3.4 — Regime Analysis and Scenario-Based Decisions

Investigate trend filtering, regime identification, regime-dependent parameters, and scenario simulation. Use the resulting assumptions in an allocation problem. Identify whether a regime label describes the full historical sample or could have been computed at the decision date.

Experiment: change a filtering or scenario assumption and inspect regime overlays and simulated portfolio outcomes. Keep retrospective regime descriptions separate from prospective trading rules.

3.5 — Recessions, Crash Regimes, and Feature Selection

Define an event and forecast horizon, prepare predictors, and compare classifiers such as logistic regression and tree-based models. Evaluate rare events using confusion matrices, ROC AUC, and metrics sensitive to class imbalance. Separate feature selection, threshold selection, and final evaluation.

Experiment: adjust a probability threshold and explain the trade-off between missed events and false alarms. A classifier that predicts the majority class can have high accuracy while failing the investment question.

Establish the Information Timeline

Suppose the decision is made after month $t$ ends and the target is next month's excess return:

$$y_t=r_{t+1}^e,{\hat{y}}_t=f_{\hat{\theta }}({\mathbf{x}}_t).$$

The feature vector ${\mathbf{x}}_t$ must contain only information available at that decision. A variable labeled with month $t$ is not necessarily available then: an economic release may arrive later or be revised. Training pairs must also have fully observed targets when the model is fitted.

Consider 120 consecutive, fully observed feature–target pairs for a first offline exercise:

Block	Pairs	Purpose
Training	1–72	Fit preprocessing and model parameters
Validation	73–96	Compare candidate settings against a baseline
Final test	97–120	Evaluate the selected procedure once

After selecting the procedure, refit it on the first 96 pairs before evaluating the last 24. The block sizes are a teaching choice, not a universal recommendation. For an actual forecast, check the label's availability at every fit date; overlapping multi-period targets may require gaps between blocks. Randomly shuffling observations can allow later information to influence an earlier prediction.

Scaling, missing-value treatment, feature selection, and hyperparameter tuning all belong inside this timeline. A scaler fitted to the full dataset has already used information from the final test block, even if the prediction model itself only saw training rows.

Prepare a Small Research Specification

Before writing a forecasting cell, answer these questions:

1. Decision: what action could the prediction inform, and when is that action taken?
2. Target: what is predicted, in which units, and over what horizon?
3. Information: when did each feature and each training label become available?
4. Baseline: what simple procedure must the model improve upon?
5. Evaluation: which statistic answers the prediction question, and which additional calculation connects it to an investment decision?

For a return forecast, a training-window mean is a possible baseline. Lower forecast error does not automatically imply higher net portfolio returns: constraints, turnover, and trading costs intervene between a prediction and an outcome.

Check your reasoning: a model predicts next month's return using next month's realized market factor. Is this a forecast? No: it is an ex-post explanation unless the factor itself was forecast using information available at the original decision date. Compare this with the contemporaneous regressions in Chapter 2.1.

How the Lessons Will Work

Each lesson will begin with a financial question and a hand-checkable example, followed by editable browser-run Python and ECharts calculated from the same results. ipywidgets controls will support focused experiments such as changing a penalty or classification threshold. Examples will use documented data snapshots and explicit observation and availability dates.

After this sequence, Alternative Data for Asset Management will examine how transactions, documents, filings, and media become model inputs.

Syllabus Reference

This sequence follows the five modules currently listed in EDHEC's Python and Machine Learning for Asset Management. The explanations, examples, and interactive presentation will be developed for this website.