The Polynomial Regression problem attempts to fit datapoints with a polynomial function by finding optimal coefficients.
In the general case, the output $y_i$ is described by the p-th degree polynomial shown in Equation 1.
\begin{equation}
y_i=\beta_0+\beta_{1}x_{i}+\beta_{2}x_{i}^2+\cdots+\beta_{p}x_{i}^p+\epsilon_i
\end{equation}
There are $p$ different polynomial terms and each of them is associated to a specific $\beta$ coefficient and then there is also the coefficient
$\beta_0$ not associated to any variable because this is the intercept. The $\epsilon$ term takes into account all the factors and noise not included in the inputs.
Click the button below for an introduction on polynomial regression.
Click this other button to see how to code polynomial regression from scratch in Python.
This article provides an interactive code for the Polynomial Regression written in Javascript. The code can be modified and it runs by clicking the button "Run Code".
n=300; //Number of samples
pol_order=3; //Polynomial order
x_min=-5; //Min value for features
x_max=5; //Max value for features
coeff_range=1 //Max absolute coefficient value
sigma_noise=10; //Standard deviation for gaussian noise
//Sample Generation
x_sample=math.random([n,1],x_min,x_max);
x=x_sample;
var i;
for (i = 2; i <= pol_order; i++) {
x=math.concat(x,math.dotPow(x_sample,i));
}
//Generate Coefficients
coeffs=math.random([pol_order+1,1],-coeff_range,coeff_range)
//Features including the bias
x=math.concat(math.ones([n,1]),x);
//Generate Polynomial Output with Noise
noise=ai_lib.generate_normal_sample(n,0,sigma_noise);
y=math.add(math.multiply(x,coeffs),noise);
//Estimation of coefficients
coeff_est=ai_lib.ols(x,y);
//Estimated linear function using the estimated coefficients
y_est=math.multiply(x,coeff_est)
//Settings for the plot
var original_sample = {
x: math.transpose(x_sample)[0],
y: math.transpose(y)[0],
mode: 'markers',
type: 'scatter',
name: 'Samples'
};
var estimated_function = {
x: math.transpose(x_sample)[0],
y: math.transpose(y_est)[0],
mode: 'markers',
type: 'scatter',
name: 'Estimated Function'
};
var layout = {
xaxis: {
range: [math.min(x_sample)-1, math.max(x_sample)+1],
autorange: false
},
yaxis: {
range: [math.min(y)-30, math.max(y)+30],
autorange: false
},
legend: {
x: 1,
y: 0.5
},
margin: { t: 0 }
}
//Print the values of the estimated coefficients
document.getElementById("text").innerHTML='The values of the estimated coefficient are:'+math.round(coeff_est,3);
//Plot of the original samples and the estimated linear function
var data = [original_sample,estimated_function];
PLOT = document.getElementById('plot');
Plotly.newPlot( PLOT, data,layout );