Linear Regression Calculator Documentation

Definition of Linear Regression

It is a type of statistical method used to find a linear relationship between two variables—independent and dependent. This method is also called "simple linear regression." There is also multiple linear regression, which finds the linear relationship between more than one independent variable and one dependent variable.

The best-fitting line is obtained by minimizing the sum of the squared errors, which are the differences between the predicted and the actual values.

Linear Regression Equation

The equation of the regression line is:

$$y = a + b \cdot x$$

where:

• $a$ - intercept
• $b$ - slope
• $x$ - independent variable
• $y$ - dependent variable

You can use our tool as a linear regression calculator by choosing "Linear fit" from the dropdown menu ("Choose fit method"). The calculator will automatically determine the regression equation with coefficients.

How to Calculate Linear Regression?

The regression coefficient $b$ is given by:

$$b = \frac{SS_{xy}}{SS_{xx}} = \frac{\sum_{i=1}^{n}(x_i - \bar{x})(y_i - \bar{y})}{\sum_{i=1}^{n}(x_i - \bar{x})^2}$$

and $a$ is calculated as:

$$a = \bar{y} - b \cdot \bar{x}$$

Where Linear Regression Analysis is Used?

Linear regression analysis is a fundamental technique in statistics and has a wide range of applications across various fields such as:

• Economy
• Social sciences
• Data science
• Sport analytics
• Engineering

Linear regression can also help analyze phenomena in environmental science or the manufacturing industry.

Least Squares Method

The least squares method is a mathematical procedure for finding the best-fitting curve to a given set of points. It is based on minimizing the sum of the squares of the residuals of the points from the curve (SSE - Sum of Squared Error). It is defined as:

$$SSE = \sum_{i=1}^{n} \left( y_i - \hat{y}_i \right)^2$$

where:

• $y_i$ - value of $y$ at point $i$
• $\hat{y}_i$ - predicted value of $y$ at point $i$

The function SSE is minimized to find the best fit line.

The condition for SSE to be a minimum is:

$$\frac{\partial SSE}{\partial a_i} = 0$$

for $i = 1,...,n$.

Measures of Goodness

R-Squared

The coefficient of determination, also denoted $R^2$, is a statistical measure that shows how well the regression model fits the data. It represents the proportion of variance in the dependent variable that can be explained by the independent variable. The formula for calculating $R^2$ is:

$$R^2 = 1 - \frac{\text{Unexplained Variation}}{\text{Total Variation}}$$

Typically, $R^2$ is in the range of 0 and 1. A higher $R^2$ value (closer to one) implies that more variation is explained by the model. An $R^2$ of 1 indicates that the model predictions perfectly fit the observed data. The coefficient of determination is the square of the correlation coefficient.

Sum of Squared Errors (SSE)

The sum of squared errors is defined as:

$$SSE = \sum_{i=1}^{n} \left( y_i - \hat{y}_i \right)^2$$