Derive a p value for a logistic curve fit.

Is it possible to derive a p value for a logistic curve fit from the goodness of fit stats?

Answer 1

[Drew]

If you are looking for p-values, consider using fitglm from the SMLT toolbox. Here is an example from the doc:

https://www.mathworks.com/help/stats/fitglm.html#bt0dghf-2

load hospital
dsa = hospital;
% Specify the model using a formula that allows up to two-way interactions \
% between the variables age, weight, and sex. Smoker is the response variable.
modelspec = 'Smoker ~ Age*Weight*Sex - Age:Weight:Sex';
% Fit a logistic binomial model.
mdl = fitglm(dsa,modelspec,'Distribution','binomial')

mdl = 
Generalized linear regression model:
    logit(Smoker) ~ 1 + Sex*Age + Sex*Weight + Age*Weight
    Distribution = Binomial

Estimated Coefficients:
                        Estimate         SE         tStat      pValue 
                       ___________    _________    ________    _______

    (Intercept)            -6.0492       19.749     -0.3063    0.75938
    Sex_Male               -2.2859       12.424    -0.18399    0.85402
    Age                    0.11691      0.50977     0.22934    0.81861
    Weight                0.031109      0.15208     0.20455    0.83792
    Sex_Male:Age          0.020734      0.20681     0.10025    0.92014
    Sex_Male:Weight        0.01216     0.053168     0.22871     0.8191
    Age:Weight         -0.00071959    0.0038964    -0.18468    0.85348


100 observations, 93 error degrees of freedom
Dispersion: 1
Chi^2-statistic vs. constant model: 5.07, p-value = 0.535

If you prefer an app interface, fitglm models can be built using the Classification Learner app. https://www.mathworks.com/help/stats/classificationlearner-app.html

If you are using the "fit" function from the curve fitting toolbox, here are some thoughts from GenAI about how to go about deriving p-value/s. (I recommend verifying/checking the GenAI output below, I didn't check the details)

1. What the Curve Fitting Toolbox Provides

When you fit a logistic curve (e.g., 'a/(1+exp(-b*(x-c)))') using fit, MATLAB gives you:

• Parameter estimates (a, b, c)
• Confidence intervals for parameters
• Residuals
• Goodness-of-fit stats: SSE, R-square, Adjusted R-square, RMSE

However, it does not directly provide p-values for the fit or for the parameters.

2. Can You Get a p-value for the Overall Fit?

Not directly from the toolbox output. The Curve Fitting Toolbox is designed primarily for nonlinear regression and does not implement hypothesis testing (like overall model p-values) as in linear or logistic regression.

• R-square and RMSE measure fit quality, but do not have associated p-values.
• Parameter confidence intervals can give you an idea of significance, but not a formal p-value.

3. What About Parameter p-values?

You can estimate parameter p-values using the parameter estimates and their standard errors:

• MATLAB does not provide standard errors or p-values for nonlinear fits directly in the Curve Fitting Toolbox.
• You’d have to compute the standard errors from the Jacobians (which fit can output if you request it), then compute t-statistics and p-values manually.

4. Goodness-of-fit p-value (Chi-square)

If you want a goodness-of-fit p-value (e.g., "does my logistic curve fit the data adequately?"), you can perform a Chi-square goodness-of-fit test. This requires:

• Binning your data (if appropriate)
• Calculating observed vs. expected counts
• Using chi2gof or manual calculation

This is not done automatically by the curve fitting toolbox.

5. Likelihood Ratio Test?

To do a likelihood ratio test (as in logistic regression), you need:

• The log-likelihood of your model and of a null model
• This is not output by the Curve Fitting Toolbox for nonlinear fits

You’d need to fit models using MATLAB’s Statistics and Machine Learning Toolbox (e.g., fitglm for logistic regression), which does provide p-values for coefficients and model fit.

What Should You Do?

• If you need formal p-values:Use fitglm or mnrfit for logistic regression (if your response is binary).
• If you want to stick with nonlinear curve fit:
• You can estimate parameter significance from confidence intervals.
• For overall fit, consider a custom Chi-square test or bootstrap resampling.

[Anthony]

Thanks for extensive answer. I'm a real novice in this, but hope to improve. This gives me some good avenues to explore. I will download the SMLT toolbox. It seems to have the sort of tools I am looking for.