The linear regression problem attemps to fit the coefficients of a linear functions to describe the provided datapoints. This function can be defined for more than one indepedent variable and the equations are typically written in matrix form. For a generic point with output $y_i$ then there are $p$ different input variables and each of them is associated to a specific $\beta$ coefficient and then there is also the coefficient
$\beta_0$ not associated to any variable because this is the intercept. The equation for a generic $i$ data point is:
\begin{equation}
y_i=\beta_0+\beta_{1}x_{i1}+\beta_{2}x_{i2}+\cdots+\beta_{p}x_{ip}+\epsilon_i
\end{equation}
Click the button below for an introduction on linear regression.
Click this other button to see how to code linear regression from scratch in Python.
This article provides an interactive code for the Linear Regression written in Javascript. The code can be modified and it runs by clicking the button "Run Code".
n=100; //Number of samples
x_dim=1; //Features dimension
x_min=0; //Min value for features
x_max=10; //Max value for features
coeffs=[[3],[8]]; //Coefficients for linear function
sigma_noise=5; //Standard deviation for gaussian noise
//Sample Generation
x_sample=math.random([n,x_dim],x_min,x_max);
//Features including the bias
x=math.concat(math.ones([n,1]),x_sample);
//Generate Linear Output with Noise
noise=ai_lib.generate_normal_sample(n,0,sigma_noise);
y=math.add(math.multiply(x,coeffs),noise);
//Estimation of coefficients
inverse_matrix=math.inv(math.multiply(math.transpose(x),x));
coeff_est=math.multiply(math.multiply(inverse_matrix,math.transpose(x)),y);
//Estimated linear function using the estimated coefficients
y_est=math.multiply(x,coeff_est)
//Settings for the plot
var original_sample = {
x: math.transpose(x_sample)[0],
y: math.transpose(y)[0],
mode: 'markers',
type: 'scatter',
name: 'Samples'
};
var estimated_function = {
x: math.transpose(x_sample)[0],
y: math.transpose(y_est)[0],
mode: 'lines',
type: 'scatter',
name: 'Estimated Function'
};
var layout = {
xaxis: {
range: [math.min(x_sample)-1, math.max(x_sample)+1],
autorange: false
},
yaxis: {
range: [math.min(y)-30, math.max(y)+30],
autorange: false
},
legend: {
x: 1,
y: 0.5
},
margin: { t: 0 }
}
//Print the values of the estimated coefficients
document.getElementById("text").innerHTML='The values of the estimated coefficient are:'+math.round(coeff_est,3);
//Plot of the original samples and the estimated linear function
var data = [original_sample,estimated_function];
PLOT = document.getElementById('plot');
Plotly.newPlot( PLOT, data,layout );