8.1.1 Definitions and Formulas

Simple linear regression

This is a process for systematically determining the equation of the best-fit line to a given set of (x,y) data. The regression equation is determined by a process called least squares regression and results in a formula to compute the slope and y-intercept of the line that will minimize the ”total squared error” of the line. Based on some theoretical calculations with calculus, you can show that the slope, B, of a regression line is given by

where corr(X,Y ) represents the correlation of the variable X with Y and the σ represent the standard deviations of the X and Y variables. Once you have the slope, the y-intercept is easy to find: A = Y - BX, where X and Y are the means of the X and Y variables.

Proportional

Two quantities are proportional when a specific amount of change in one of the quantities results in a certain amount of change in the other quantity given by a fixed multiplicative factor. In mathematical terms, the phrase ”the change in y is proportional to the change in x” can be written as Δy ∝ Δx. This means that Δy = kΔx for some constant k that is independent of y and x.

Coefficient

A coefficient is a fixed (or constant) number in an algebraic model. For example, linear equations have two coefficients: the slope and y-intercept. Coefficients are sometimes called constants or parameters. In regression output, a table of coefficients is produced. It is up to you to combine these correctly into the model equation. In the examples below, you will see how to do this.

Constant

In most regression output, the y-intercept of the regression line is labeled ”constant” in the table of coefficients. More generally, a constant is any value in a formula that is fixed, like the number 2 in the linear relationship y = 2x + 5. Sometimes constants are called parameters.

Explanatory Variable

This is the variable (or variables, in later chapters) used to explain the results of the model. In simple regression, the x-variable is the explanatory variable. As you can guess, this is just another name for the independent variable.

Response Variable

This is the variable that responds to the independent (or explanatory) variable. Thus, it is really the y-variable or dependent variable.