Blog on Data Science Mathematics. Statistics. Algorithms.
Posts with the tag Regression:

Unbiasedness and Consistency of the Regression Coefficients

Here we consider the basic asymptotic properties of the coefficient estimators in a simple linear regression. Many readers when first acquainted with the subject stay under the expression that this is a rather tedious and technical matter. In what follows, we would like to show that when approached from the right perspective, the subject becomes rather intuitive and clear. Derivation of the estimators We consider a simple linear regression model $$ Y = \beta_0 + \beta_1 X + \epsilon, $$ where $X$ and $Y$ are random variables, $\beta_0$, $\beta_1$ are constants and $\epsilon$ is the random error. We assume that the observations $(x_1,y_1), \dots, (x_n,y_n)$ are independently and identically distributed. In the ordinary least squares (OLS) method, we want to minimize the mean squared error