Mean Square Error Implementation and gfit2 function in matlab

most likely when you create a regression model you want to evaluate the model to estimate the googdness of your fit. Most common way of doing that is to calculate the mean square error. here is the implementation of that in Matlab :

 X = randn(256,256);  
   Xapp = randn(256,256);  
   D = abs(X-Xapp).^2;  
   MSE = sum(D(:))/numel(X);  

There is also a function by Richard Crozier that offer various methods to evaluate the goodness of fit. you can find it in here:

http://www.mathworks.co.uk/matlabcentral/fileexchange/22020-goodness-of-fit--modified-/content/gfit2.m

3D Curve Fitting in Matlab(Linear and Higher Order Polynomial)

To fit a curve onto a set of points, we can use ordinary least-squares regression. There is a solution pageby MathWorks describing the process.

As an example, let's start with some random data:

% some 3d points
data = mvnrnd([0 0 0], [1 -0.5 0.8; -0.5 1.1 0; 0.8 0 1], 50);

As @BasSwinckels showed, by constructing the desired design matrix, you can use mldivide or pinvto solve the overdetermined system expressed as Ax=b:

% best-fit plane
C = [data(:,1) data(:,2) ones(size(data,1),1)] \ data(:,3);    % coefficients

% evaluate it on a regular grid covering the domain of the data
[xx,yy] = meshgrid(-3:.5:3, -3:.5:3);
zz = C(1)*xx + C(2)*yy + C(3);

% or expressed using matrix/vector product
%zz = reshape([xx(:) yy(:) ones(numel(xx),1)] * C, size(xx));

Next we visualize the result:

% plot points and surface
figure('Renderer','opengl')
line(data(:,1), data(:,2), data(:,3), 'LineStyle','none', ...
    'Marker','.', 'MarkerSize',25, 'Color','r')
surface(xx, yy, zz, ...
    'FaceColor','interp', 'EdgeColor','b', 'FaceAlpha',0.2)
grid on; axis tight equal;
view(9,9);
xlabel x; ylabel y; zlabel z;
colormap(cool(64))

---------------------------------------------------------------------------------

Higher-order Polynomial Fitting:

As was mentioned, we can get higher-order polynomial fitting by adding more terms to the independent variables matrix (the A in Ax=b).

Say we want to fit a quadratic model with constant, linear, interaction, and squared terms (1, x, y, xy, x^2, y^2). We can do this manually:

% best-fit quadratic curve
C = [ones(50,1) data(:,1:2) prod(data(:,1:2),2) data(:,1:2).^2] \ data(:,3);
zz = [ones(numel(xx),1) xx(:) yy(:) xx(:).*yy(:) xx(:).^2 yy(:).^2] * C;
zz = reshape(zz, size(xx));

There is also a helper function x2fx in the Statistics Toolbox that helps in building the design matrix for a couple of model orders:

C = x2fx(data(:,1:2), 'quadratic') \ data(:,3);
zz = x2fx([xx(:) yy(:)], 'quadratic') * C;
zz = reshape(zz, size(xx));

Finally there is an excellent function polyfitn on the File Exchange by John D'Errico that allows you to specify all kinds of polynomial orders and terms involved:

model = polyfitn(data(:,1:2), data(:,3), 2);
zz = polyvaln(model, [xx(:) yy(:)]);
zz = reshape(zz, size(xx));

Plot CDF in Python

perhaps the most easy way of plotting the cumilative distribution function in python:


import numpy as np
import statsmodels.api as sm # recommended import according to the docs
import matplotlib.pyplot as plt

sample = np.random.uniform(0, 1, 50)
sample=[1,2,2,3,2,3,3,3,3,3,3,4,4,4,4,4,60,3,3,3,10]
ecdf = sm.distributions.ECDF(sample)

dsum = sum(sample);
normalized_data = sample/dsum;

x = np.linspace(min(sample), max(sample),10000)
print x
y = ecdf(x)
plt.step(x, y)

plt.show()


Here is the plot:

Fitting data with SciPy using curve_fit

Suppose that you have a data set consisting of Throughput vs Latency data for your experiment. We’ll start by importing the needed libraries and defining a fitting function. in this example we define a quadratic function. you may define your own function:

from scipy.optimize import curve_fit
def fitFunc(x, a, b, c):
    return a*(x**2)+b*x + c

now we create some points:

y=[25.6, 31.21, 36.82, 42.43, 44.67, 46.91, 48.04, 49.15, 51.2] 
x=[26.0, 41.0, 50.0, 62.5, 69.0, 74.0, 75.050000000000182, 109.0, 98.5]

The scipy.optimize module contains a least squares curve fit routine that requires as input a user-defined fitting function (in our case fitFunc ), the x-axis data (in our case, t) and the y-axis data (in our case, noisy). The curve_fit routine returns an array of fit parameters, and a matrix of covariance data.

tt = np.linspace(min(x),max(x),100)

x=numpy.asarray(x)

y=numpy.asarray(y)

fitParams, fitCovariances = curve_fit(fitFunc, x, y)

print fitParams

print fitCovariances

plt.ylabel('Throughput (%)', fontsize = 16)

plt.xlabel('Latency' , fontsize = 16)

'''plot the real data'''

plt.plot(t,noisy,'b+',markersize=19)   

'''plot the best curving fit'''

plt.plot(tt, fitFunc(tt, fitParams[0], fitParams[1], fitParams[2]))

output:
[-0.00420386  0.88363413  3.96364659]
[[  4.78028309e-07  -6.52166801e-05   1.92964390e-03]
 [ -6.52166801e-05   9.24484129e-03  -2.86615224e-01]
 [  1.92964390e-03  -2.86615224e-01   9.57342339e+00]]

result:

Linrear Regression in Python using Scipy with plot

Here is a linear regression example using the scipy.stats library.

from scipy import stats

import numpy



x = [5.05, 6.75, 3.21, 2.66]

y = [1.65, 26.5, -5.93, 7.96]

plot(x,y,'b+',markersize=19,label='Real Data')



gradient, intercept, r_value, p_value, std_err = stats.linregress(x,y)

print "Gradient and intercept", gradient, intercept



xx= numpy.linspace(-5,10,100)

z=gradient*xx+intercept

plt.plot(xx,z,'r',label='Linear Regression Function')

plt.legend(loc=2)

Here is the result of running the above code:

fitting 3d data

Imagine you have 2 variables and you want to create a regression model to predict the 3rd variables. for that you have two options:
1) use the surface fitting toolbox in matlab. enter the following command in the consol:
sftool
once it is launched, you would be able to provide your first and second variables as the factos for ceating the model and the 3rd variable as the predictor. You have also some options to fit different degree of polynomial and see how you model would be.


2) use the command line function to create the above model:
following command create a 3rd order polynomial fitting model using the data y1 and y2:
sf = fit([y1,y2],y3,'poly33')
this will plot your model and your training data:
plot(sf, [y1,y2], y3)

if you want to exclude a range of data at the time of creating model do this:
sf = fit([y1,y2],y3,'poly33','Exclude', y2 < 5)