Main Content

kfoldMargin

Classification margins for cross-validated classification model

    Description

    example

    M = kfoldMargin(CVMdl) returns classification margins obtained by the cross-validated classification model CVMdl. For every fold, kfoldMargin computes classification margins for validation-fold observations using a classifier trained on training-fold observations. CVMdl.X and CVMdl.Y contain both sets of observations.

    M = kfoldMargin(CVMdl,'IncludeInteractions',includeInteractions) specifies whether to include interaction terms in computations. This syntax applies only to generalized additive models.

    Examples

    collapse all

    Find the k-fold margins for an ensemble that classifies the ionosphere data.

    Load the ionosphere data set.

    load ionosphere

    Create a template tree stump.

    t = templateTree('MaxNumSplits',1);

    Train a classification ensemble of decision trees. Specify t as the weak learner.

    Mdl = fitcensemble(X,Y,'Method','AdaBoostM1','Learners',t);

    Cross-validate the classifier using 10-fold cross-validation.

    cvens = crossval(Mdl);

    Compute the k-fold margins. Display summary statistics for the margins.

    m = kfoldMargin(cvens);
    marginStats = table(min(m),mean(m),max(m),...
        'VariableNames',{'Min','Mean','Max'})
    marginStats=1×3 table
          Min       Mean      Max  
        _______    ______    ______
    
        -11.312    7.3236    23.517
    
    

    Input Arguments

    collapse all

    Cross-validated partitioned classifier, specified as a ClassificationPartitionedModel, ClassificationPartitionedEnsemble, or ClassificationPartitionedGAM object. You can create the object in two ways:

    • Pass a trained classification model listed in the following table to its crossval object function.

    • Train a classification model using a function listed in the following table and specify one of the cross-validation name-value arguments for the function.

    Flag to include interaction terms of the model, specified as true or false. This argument is valid only for a generalized additive model (GAM). That is, you can specify this argument only when CVMdl is ClassificationPartitionedGAM.

    The default value is true if the models in CVMdl (CVMdl.Trained) contain interaction terms. The value must be false if the models do not contain interaction terms.

    Data Types: logical

    Output Arguments

    collapse all

    Classification margins, returned as a numeric vector. M is an n-by-1 vector, where each row is the margin of the corresponding observation and n is the number of observations. (n is size(CVMdl.X,1) when observations are in rows.)

    If you use a holdout validation technique to create CVMdl (that is, if CVMdl.KFold is 1), then M has NaN values for training-fold observations.

    More About

    collapse all

    Classification Margin

    The classification margin for binary classification is, for each observation, the difference between the classification score for the true class and the classification score for the false class. The classification margin for multiclass classification is the difference between the classification score for the true class and the maximal score for the false classes.

    If the margins are on the same scale (that is, the score values are based on the same score transformation), then they serve as a classification confidence measure. Among multiple classifiers, those that yield greater margins are better.

    Algorithms

    kfoldMargin computes classification margins as described in the corresponding margin object function. For a model-specific description, see the appropriate margin function reference page in the following table.

    Model Typemargin Function
    Discriminant analysis classifiermargin
    Ensemble classifiermargin
    Generalized additive model classifiermargin
    k-nearest neighbor classifiermargin
    Naive Bayes classifiermargin
    Neural network classifiermargin
    Support vector machine classifiermargin
    Binary decision tree for multiclass classificationmargin

    Extended Capabilities

    Introduced in R2011a