Data and signals

Market data — prices, volumes, order books — is universal and heavily mined. Fundamental data covers financial statements and estimates. Alternative data includes satellite images, web traffic, card transactions, shipping records and text from filings and news. Each requires cleaning and has its own biases.

Data must reflect what was known at the moment a trade would have been placed. Restated earnings, survivorship (only including companies that still exist) and late-arriving data all create look-ahead bias that makes backtests look better than reality.

A signal converts raw data into a forecast, usually a score ranking assets cross-sectionally. Standard steps: normalise (z-score), neutralise to unwanted exposures like sector, and combine with other signals.

Measure the information coefficient (correlation between signal and future return), its stability over time, turnover, and returns of quantile portfolios. A good signal shows a monotonic pattern: top decile beats second decile and so on down.

Noise is abundant and real effects are small. With enough data, random patterns look significant. Researchers insist on an economic rationale — a reason the effect should exist and persist — before trusting a statistical result.