$\newcommand{\R}{\mathbb R}\newcommand{\e}{\varepsilon}\newcommand{\0}{\mathbf 0}\newcommand{\E}{\operatorname{E}}\newcommand{\Var}{\operatorname{Var}}$Consider a linear regression model $y = X\beta+\e$ with $X\in\R^{n\times p}$ known and full column rank. Under common…
$\newcommand{\X}{\mathcal X}\newcommand{\x}{\mathbf x}\newcommand{\0}{\mathbf 0}\newcommand{\one}{\mathbf 1}\newcommand{\y}{\mathbf y}\newcommand{\z}{\mathbf z}$In this post I’m going to look at distributions over polynomials using Gaussian processes.…
$\newcommand{\e}{\varepsilon}$$\newcommand{\Var}{\operatorname{Var}}$$\newcommand{\x}{\mathbf x}$$\newcommand{\one}{\mathbf 1}$$\newcommand{\0}{\mathbf 0}$In my previous post I stopped right before working out the REML objective function so I’ll start…
$\newcommand{\Var}{\operatorname{Var}}$$\newcommand{\Cov}{\operatorname{Cov}}$$\newcommand{\one}{\mathbf 1}$$\newcommand{\0}{\mathbf 0}$$\newcommand{\e}{\varepsilon}$$\newcommand{\E}{\operatorname{E}}$In this post I’m going to discuss leverage scores in linear regression. In particular, I’ll show how the…
$\newcommand{\e}{\varepsilon}$$\newcommand{\Var}{\operatorname{Var}}$$\newcommand{\x}{\mathbf x}$$\newcommand{\one}{\mathbf 1}$$\newcommand{\0}{\mathbf 0}$In this post I will derive the point estimates for mixed models. I’ll focus on interpretation so…
$\newcommand{\R}{\mathbb R}$$\newcommand{\y}{\mathbf y}$$\newcommand{\thb}{\tilde{\hat\beta}}$$\newcommand{\L}{\Lambda}$$\newcommand{\tr}{\operatorname{tr}}$In this post I’ll look at updating a linear regression with new data. One place where this can…
$\newcommand{\R}{\mathbb R}$$\newcommand{\e}{\varepsilon}$$\newcommand{\H}{\mathcal H}$$\newcommand{\0}{\mathbf 0}$In this post I’m going to prove that the squared exponential kernel is positive definite. I’ll finish…
$\newcommand{\logit}{\operatorname{logit}}$$\newcommand{\y}{\mathbf y}$$\newcommand{\one}{\mathbf 1}$$\newcommand{\e}{\varepsilon}$$\newcommand{\0}{\mathbf 0}$$\newcommand{\Var}{\operatorname{Var}}$$\newcommand{\E}{\operatorname{E}}$In this post I’ll look at the gradient of the loss in logistic regression and explore a…
$\newcommand{\e}{\varepsilon}$$\newcommand{\one}{\mathbf 1}$Suppose I have collected data $X \in \mathbb R^{n\times p}$ ($n \geq p$ and $X$ is full column rank…
$\newcommand{\X}{\mathcal X}$$\newcommand{\dl}{\,\text d\lambda}$$\newcommand{\dx}{\,\text dx}$$\newcommand{\one}{\mathbf 1}$I’m going to take a look at functional linear regression, where I’ll find the best linear…