Stephen Stigler’s The History of Statistics is not for the faint of heart. It’s a deep exploration of the origins of statistics as a form of mathematical inquiry, from astronomy and geodesy in the late 18th century to social science in the 19th century. It assumes you’re familiar with modern proofs of results like the weak law of large numbers and the normal approximation to the binomial distribution. It’s an intellectual challenge, and I’m loving it.

As I’ve been working through Stigler’s book, I’ve been taking the time to learn (or relearn) derivations that are unfamiliar to me. One of those is Legendre’s original exposition of the method of least squares. I’m comfortable with modern approaches to find the best-fitting curve to a set of points, but there were parts of Legendre’s formulation that felt foreign to me.

In order to shore up my own understanding and connect it to Legendre’s exposition, I worked through a number of resources, which I share below:

Finally, there is some debate about who developed the method first, Gauss or Legendre. Legendre published first in 1805, but Gauss claimed he had discovered the method by 1795. Stephen Stigler has a fascinating essay on this priority dispute.