finding a trend line where x axis semilog and y axis linear and trend line follow the eqn y=a*exp(b*x)

can you help to find a trend line and the coefficient values for semilox plot

where x- axis == semilog == time axis;

and y-axis == linear == depth axis;

and trend line follow the exponential eqn y = a*exp(b*x)

[Arvind]

Sorry! , it's my mistake the data file I uploaded had the wrong entries so I uploaded new file with original data

Answer 1

[Image Analyst]

See attached demo for fitting data to exponentials. Adapt as needed.

[Image Analyst]

As we've both said, your attached data is bad. Depth is a constant -- all the same value. Please attach the correct data file. This one is not it!

Answer 2

[Star Strider]

AA straight horizonta line would be the best model, since ‘Depth’ does not change at all over time —

T1 = readtable('data_mat_code.xlsx')

T1 = 914×2 table
     time     Depth 
    ______    ______

    208.57    72.183
    208.85    72.183
     209.1    72.183
    209.24    72.183
    209.28    72.183
    209.22    72.183
    209.03    72.183
     208.7    72.183
    208.23    72.183
    207.62    72.183
     206.8    72.183
    206.07    72.183
    205.82    72.183
       206    72.183
    206.33    72.183
    206.43    72.183

time = T1.time;
Depth = T1.Depth;
VN = T1.Properties.VariableNames;

DepthStats = [min(Depth); max(Depth); std(Depth)]

DepthStats = 3×1

72.1831 72.1831 0.0000

fcn = @(b,x) b(1).*exp(b(2).*x);

mdl = fitnlm(time,Depth,fcn,rand(2,1))

mdl = 
Nonlinear regression model:
    y ~ b1*exp(b2*x)

Estimated Coefficients:
           Estimate         SE          tStat         pValue   
          __________    __________    __________    ___________

    b1        72.183    8.2362e-10    8.7641e+10              0
    b2    2.0514e-12    5.7373e-14        35.756    1.0175e-175


Number of observations: 914, Error degrees of freedom: 912
Root Mean Squared Error: 6.87e-10
R-Squared: -4.75e+07,  Adjusted R-Squared -4.76e+07
F-statistic vs. constant model: 0, p-value = 1

[y,yci] = predict(mdl, time);

figure
hp1 = semilogx(time, Depth, '.', 'DisplayName','Data');
hold on
hp2 = plot(time, y, '-r', 'DisplayName','Regression');
hp3 = plot(time, yci, '--r', 'DisplayName','95% Confidence Intervals');
hold off
grid
xlabel(VN{1})
ylabel(VN{2})
legend([hp1,hp2,hp3(1)], 'Location','best')

Perhaps this will work better on a different data set.

.

[Arvind]

at the end this type of fit comes as output; blue line is fit line and black solid line DT(ms/ft)

[Image Analyst]

Please give us code to generate that plot from your data, or include the correct data, because I get this:

tbl = readtable('data_mat_code.xlsx')

tbl = 914×2 table
     time     Depth 
    ______    ______

    208.57    72.183
    208.85    72.183
     209.1    72.183
    209.24    72.183
    209.28    72.183
    209.22    72.183
    209.03    72.183
     208.7    72.183
    208.23    72.183
    207.62    72.183
     206.8    72.183
    206.07    72.183
    205.82    72.183
       206    72.183
    206.33    72.183
    206.43    72.183

t = tbl.time;
depth = tbl.Depth;
% Times are all scrambled.  Sort them
[t, sortOrder] = sort(t);
% Sort depth the same way
depth = depth(sortOrder);
min(depth)

ans = 72.1831

max(depth)

ans = 72.1831

if max(depth) == min(depth)
    fprintf('Min and max are the same value!!!\n')
end

Min and max are the same value!!!

plot(t, depth, 'b-');
grid on;
xlabel('Time');
ylabel('Depth')

[Star Strider]

@Arvind — That plot does not descibe the data in the file.

[Arvind]

The figure is an example picture that I have provided, I do not have the code.

The code I made is manual fitted and it is a straight line.

%% but it's not right

time = T1.time;
depth = T1.depth;
figure('Name','NCT')
plot(time,depth, 'b-', 'LineWidth', 1);
coeff = polyfit(time,depth, 1);
xFit = 180:2:206;
yFit = linspace(900, 70,length(xFit));
hold on
plot(xFit, yFit, ['r','-'], 'LineWidth', 1);
xlabel('Sonic Transit time');
ylabel('Depth');
set(gca,'XScale','log','XAxisLocation','top','YAxisLocation','left','ydir','reverse');
grid on
title('Normal compaction trend plot')

[Star Strider]

I have the same problem with this file — ‘Depth’ is constant —

figure
imshow(imread('nct.jpg'))

T1 = readtable('data_mat_code (1).xlsx')

T1 = 914×2 table
     time     Depth 
    ______    ______

    208.57    72.183
    208.85    72.183
     209.1    72.183
    209.24    72.183
    209.28    72.183
    209.22    72.183
    209.03    72.183
     208.7    72.183
    208.23    72.183
    207.62    72.183
     206.8    72.183
    206.07    72.183
    205.82    72.183
       206    72.183
    206.33    72.183
    206.43    72.183

time = T1.time;
Depth = T1.Depth;
VN = T1.Properties.VariableNames;

DepthStats = [min(Depth); max(Depth); max(Depth)-min(Depth); std(Depth)]

DepthStats = 4×1

72.1831 72.1831 0 0.0000

fcn = @(b,x) b(1).*exp(b(2).*x);

mdl = fitnlm(time,Depth,fcn,rand(2,1))

Warning: Rank deficient, rank = 1, tol = 2.761527e+41.

Warning: The Jacobian at the solution is ill-conditioned, and some model parameters may not be estimated well (they are not identifiable). Use caution in making predictions.

mdl = 
Nonlinear regression model:
    y ~ b1*exp(b2*x)

Estimated Coefficients:
           Estimate         SE          tStat         pValue   
          __________    __________    __________    ___________

    b1    6.4027e-26     2.119e-27        30.216    1.4172e-139
    b2       0.59046    3.1039e-40    1.9023e+39              0


Number of observations: 914, Error degrees of freedom: 913
Root Mean Squared Error: 2.88e+27
R-Squared: -8.36e+80,  Adjusted R-Squared -8.36e+80
F-statistic vs. zero model: 0, p-value = 1

[y,yci] = predict(mdl, time);

figure
hp1 = semilogx(time, Depth, '.', 'DisplayName','Data');
hold on
hp2 = plot(time, y, '-r', 'DisplayName','Regression');
hp3 = plot(time, yci, '--r', 'DisplayName','95% Confidence Intervals');
hold off
grid
xlabel(VN{1})
ylabel(VN{2})
legend([hp1,hp2,hp3(1)], 'Location','best')

It would be preferable to have a file where ‘Depth’ varies wias a function of ‘time’.

.

[Star Strider]

The correct data are definitely an improvement.

The problem remains that the model does not fit them well. It would be extremely helpful to know what the parameters are that fit the data in the illustration, since nonlinear models are quite sensitive to the initial parameter estimates. (I use ga here to see if it could estimate an acceptable set of initial parameters, since it is usually robust, however even it is having problems, although the results appear to be reasonable.)

T1 = readtable('data_mat_code.xlsx', 'VariableNamingRule','preserve')

T1 = 1549×3 table
    depth     depth_1    transit time
    ______    _______    ____________

    0.0558    0.18307        213.4   
    0.2082    0.68307       213.74   
    0.3606     1.1831       213.71   
     0.513     1.6831       213.21   
    0.6654     2.1831       213.05   
    0.8178     2.6831       213.38   
    0.9702     3.1831       213.72   
    1.1226     3.6831       214.91   
     1.275     4.1831       216.66   
    1.4274     4.6831       216.73   
    1.5798     5.1831       215.62   
    1.7322     5.6831       214.53   
    1.8846     6.1831        213.2   
     2.037     6.6831        211.9   
    2.1894     7.1831       211.19   
    2.3418     7.6831       211.25   

time = T1.('transit time');
Depth = T1.depth;
Depth_1 = T1.depth_1;
VN = T1.Properties.VariableNames;

DepthStats = [min(Depth); max(Depth); max(Depth)-min(Depth); std(Depth)]

DepthStats = 4×1

0.0558 235.9710 235.9152 68.1688

Depth_1Stats = [min(Depth_1); max(Depth_1); max(Depth_1)-min(Depth_1); std(Depth_1)]

Depth_1Stats = 4×1

0.1831 774.1831 774.0000 223.6511

fcn = @(b,x) b(1).*exp(b(2).*x);

ftns = @(b) norm(Depth - fcn(b,time));
[B1,fv1,exitflag1,output1] = ga(ftns,2);

Optimization terminated: maximum number of generations exceeded.

B1

B1 = 1×2

203.2323 -0.0022

fv1

fv1 = 2.6977e+03

output1.generations

ans = 200

mdl1 = fitnlm(time,Depth,fcn,B1)

Warning: Some columns of the Jacobian are effectively zero at the solution, indicating that the model is insensitive to some of its parameters. That may be because those parameters are not present in the model, or otherwise do not affect the predicted values. It may also be due to numerical underflow in the model function, which can sometimes be avoided by choosing better initial parameter values, or by rescaling or recentering. Parameter estimates may be unreliable.

mdl1 = 
Nonlinear regression model:
    y ~ b1*exp(b2*x)

Estimated Coefficients:
           Estimate         SE          tStat       pValue
          __________    __________    __________    ______

    b1    1.8762e+07    1.5549e-14    1.2066e+21      0   
    b2     -0.059993    5.7701e-05       -1039.7      0   


Number of observations: 1549, Error degrees of freedom: 1548
Root Mean Squared Error: 55.8
R-Squared: 0.33,  Adjusted R-Squared 0.33
F-statistic vs. zero model: 7.7e+03, p-value = 0

[y,yci] = predict(mdl1, time);

figure
hp1 = semilogx(time, Depth, '.', 'DisplayName','Data');
hold on
hp2 = plot(time, y, '-r', 'DisplayName','Regression');
hp3 = plot(time, yci, '--r', 'DisplayName','95% Confidence Intervals');
hold off
grid
xlabel(VN{3})
ylabel(VN{1})
legend([hp1,hp2,hp3(1)], 'Location','best')

ftns = @(b) norm(Depth_1 - fcn(b,time));
[B2,fv2,exitflag2,output2] = ga(ftns,2);

Optimization terminated: maximum number of generations exceeded.

B2

B2 = 1×2

455.2062 -0.0008

fv2

fv2 = 8.7584e+03

output2.generations

ans = 200

mdl2 = fitnlm(time,Depth_1,fcn,B2)

Warning: Some columns of the Jacobian are effectively zero at the solution, indicating that the model is insensitive to some of its parameters. That may be because those parameters are not present in the model, or otherwise do not affect the predicted values. It may also be due to numerical underflow in the model function, which can sometimes be avoided by choosing better initial parameter values, or by rescaling or recentering. Parameter estimates may be unreliable.

mdl2 = 
Nonlinear regression model:
    y ~ b1*exp(b2*x)

Estimated Coefficients:
           Estimate         SE          tStat       pValue
          __________    __________    __________    ______

    b1    6.1555e+07    4.7394e-15    1.2988e+22      0   
    b2     -0.059993    5.7701e-05       -1039.7      0   


Number of observations: 1549, Error degrees of freedom: 1548
Root Mean Squared Error: 183
R-Squared: 0.33,  Adjusted R-Squared 0.33
F-statistic vs. zero model: 7.7e+03, p-value = 0

[y,yci] = predict(mdl2, time);

figure
hp1 = semilogx(time, Depth_1, '.', 'DisplayName','Data');
hold on
hp2 = plot(time, y, '-r', 'DisplayName','Regression');
hp3 = plot(time, yci, '--r', 'DisplayName','95% Confidence Intervals');
hold off
grid
xlabel(VN{3})
ylabel(strrep(VN{2},'_','\_'))
legend([hp1,hp2,hp3(1)], 'Location','best')

This is the best that ga and I can do with these data.

.

[Image Analyst]

In his desired/sample plot it looks like he does not fit ALL the data, only the part that is roughly a line or exponential. Therefore I think you should only include depths below 510 and use another name for time since it's a built-in function

times = T1.('transit time');
Depth = T1.depth;
Depth_1 = T1.depth_1;
% Define "good" data as where Depth_1 is less than 510.
mask = Depth_1 < 510;
% Extract "good" part of data that we want to use in the fit.
Depth_1 = Depth_1(mask);
times = times(mask);

[Arvind]

@Image Analyst Yes,You are right, I want to fit the line only on those data which have linearity or bets trend and the trend line what I want upto depth<510 , but also want to that trend line plot on all data

@Star Strider Thanks for your work

[Star Strider]

My pleasure!

For the depths < 510, try these —

T1 = readtable('data_mat_code.xlsx', 'VariableNamingRule','preserve')

T1 = 1549×3 table
    depth     depth_1    transit time
    ______    _______    ____________

    0.0558    0.18307        213.4   
    0.2082    0.68307       213.74   
    0.3606     1.1831       213.71   
     0.513     1.6831       213.21   
    0.6654     2.1831       213.05   
    0.8178     2.6831       213.38   
    0.9702     3.1831       213.72   
    1.1226     3.6831       214.91   
     1.275     4.1831       216.66   
    1.4274     4.6831       216.73   
    1.5798     5.1831       215.62   
    1.7322     5.6831       214.53   
    1.8846     6.1831        213.2   
     2.037     6.6831        211.9   
    2.1894     7.1831       211.19   
    2.3418     7.6831       211.25   

time = T1.('transit time');
Depth = T1.depth;
Depth_1 = T1.depth_1;
Lv1 = Depth < 510;
Lv2 = Depth_1 < 510;
VN = T1.Properties.VariableNames;

DepthStats = [min(Depth); max(Depth); max(Depth)-min(Depth); std(Depth)]

DepthStats = 4×1

0.0558 235.9710 235.9152 68.1688

Depth_1Stats = [min(Depth_1); max(Depth_1); max(Depth_1)-min(Depth_1); std(Depth_1)]

Depth_1Stats = 4×1

0.1831 774.1831 774.0000 223.6511

fcn = @(b,x) b(1).*exp(b(2).*x);

% ftns = @(b) norm(Depth - fcn(b,time));
% [B1,fv1,exitflag1,output1] = ga(ftns,2);
% B1
% fv1
% output1.generations
% 
% mdl1 = fitnlm(time,Depth,fcn,B1)
% [y,yci] = predict(mdl1, time);
% 
% figure
% hp1 = semilogx(time, Depth, '.', 'DisplayName','Data');
% hold on
% hp2 = plot(time, y, '-r', 'DisplayName','Regression');
% hp3 = plot(time, yci, '--r', 'DisplayName','95% Confidence Intervals');
% hold off
% grid
% title('All Data')
% xlabel(VN{3})
% ylabel(VN{1})
% legend([hp1,hp2,hp3(1)], 'Location','best')
% 
% ftns = @(b) norm(Depth_1 - fcn(b,time));
% [B2,fv2,exitflag2,output2] = ga(ftns,2);
% B2
% fv2
% output2.generations
% 
% mdl2 = fitnlm(time,Depth_1,fcn,B2)
% [y,yci] = predict(mdl2, time);
% 
% figure
% hp1 = semilogx(time, Depth_1, '.', 'DisplayName','Data');
% hold on
% hp2 = plot(time, y, '-r', 'DisplayName','Regression');
% hp3 = plot(time, yci, '--r', 'DisplayName','95% Confidence Intervals');
% hold off
% grid
% title('All Data')
% xlabel(VN{3})
% ylabel(strrep(VN{2},'_','\_'))
% legend([hp1,hp2,hp3(1)], 'Location','best')

ftns = @(b) norm(Depth(Lv1) - fcn(b,time(Lv1)));
[B3,fv3,exitflag3,output3] = ga(ftns,2);

Optimization terminated: average change in the fitness value less than options.FunctionTolerance.

B3

B3 = 1×2

10.2959 0.0117

fv3

fv3 = 2.8890e+03

output3.generations

ans = 114

mdl3 = fitnlm(time(Lv1),Depth(Lv1),fcn,B3)

Warning: Some columns of the Jacobian are effectively zero at the solution, indicating that the model is insensitive to some of its parameters. That may be because those parameters are not present in the model, or otherwise do not affect the predicted values. It may also be due to numerical underflow in the model function, which can sometimes be avoided by choosing better initial parameter values, or by rescaling or recentering. Parameter estimates may be unreliable.

mdl3 = 
Nonlinear regression model:
    y ~ b1*exp(b2*x)

Estimated Coefficients:
           Estimate         SE          tStat       pValue
          __________    __________    __________    ______

    b1    1.8762e+07    1.5549e-14    1.2066e+21      0   
    b2     -0.059993    5.7701e-05       -1039.7      0   


Number of observations: 1549, Error degrees of freedom: 1548
Root Mean Squared Error: 55.8
R-Squared: 0.33,  Adjusted R-Squared 0.33
F-statistic vs. zero model: 7.7e+03, p-value = 0

[y,yci] = predict(mdl3, time(Lv1));

figure
hp1 = semilogx(time(Lv1), Depth(Lv1), '.', 'DisplayName','Data');
hold on
hp2 = plot(time(Lv1), y, '-r', 'DisplayName','Regression');
hp3 = plot(time(Lv1), yci, '--r', 'DisplayName','95% Confidence Intervals');
hold off
grid
xlabel(VN{3})
ylabel(VN{1})
title('Depth < 510')
legend([hp1,hp2,hp3(1)], 'Location','best')

ftns = @(b) norm(Depth_1(Lv2) - fcn(b,time(Lv2)));
[B4,fv4,exitflag4,output4] = ga(ftns,2);

Optimization terminated: average change in the fitness value less than options.FunctionTolerance.

B4

B4 = 1×2

10.3339 0.0156

fv4

fv4 = 5.4074e+03

output4.generations

ans = 93

mdl4 = fitnlm(time(Lv2),Depth_1(Lv2),fcn,B4)

Warning: Rank deficient, rank = 1, tol = 4.103161e-07.

Warning: Some columns of the Jacobian are effectively zero at the solution, indicating that the model is insensitive to some of its parameters. That may be because those parameters are not present in the model, or otherwise do not affect the predicted values. It may also be due to numerical underflow in the model function, which can sometimes be avoided by choosing better initial parameter values, or by rescaling or recentering. Parameter estimates may be unreliable.

mdl4 = 
Nonlinear regression model:
    y ~ b1*exp(b2*x)

Estimated Coefficients:
           Estimate         SE          tStat       pValue
          __________    __________    __________    ______

    b1    1.1436e+09     1.378e-16    8.2993e+24      0   
    b2     -0.076496    3.0998e-05       -2467.8      0   


Number of observations: 1020, Error degrees of freedom: 1019
Root Mean Squared Error: 56
R-Squared: 0.855,  Adjusted R-Squared 0.855
F-statistic vs. zero model: 2.71e+04, p-value = 0

[y,yci] = predict(mdl4, time(Lv2));

figure
hp1 = semilogx(time(Lv2), Depth_1(Lv2), '.', 'DisplayName','Data');
hold on
hp2 = plot(time(Lv2), y, '-r', 'DisplayName','Regression');
hp3 = plot(time(Lv2), yci, '--r', 'DisplayName','95% Confidence Intervals');
hold off
grid
title('Depth\_1 < 510')
xlabel(VN{3})
ylabel(strrep(VN{2},'_','\_'))
legend([hp1,hp2,hp3(1)], 'Location','best')

That is the best I can do with these data.

(The ‘All Data’ regressions are in the earlier series, and since it is very difficult to get all of these working in the same run, I am only calculating and plotting the ‘Depth < 510’ and ‘Depth_1 < 510’ regressions here.

.