Calculate map image classification efficacy (MICE) and other metrics using columns/vectors of reference and predicted classes
Arguments
- reference
column/vector of reference labels as factor data type.
- prediction
column/vector of predicted labels as factor data type.
- mappings
names of classes (if not provided, factor levels are used).
- multiclass
TRUE or FALSE. If TRUE, treats classification as multiclass. If FALSE, treats classification as binary. Default is TRUE.
- positiveIndex
index for positive case for binary classification. Ignored for multiclass classification. Default is 1 or first factor level.
Value
multiclass or binary assessment metrics in a list object. See details for description of generated metrics.
Details
For multiclass classification, returns a list object with the following items: $Mappings = class names; $confusionMatrix = confusion matrix where columns represent the reference data and rows represent the classification result; $referenceCounts = count of samples in each reference class; $predictionCounts = count of predictions in each class; $overallAccuracy = overall accuracy; $MICE = map image classification efficacy; $usersAccuracies = class-level user's accuracies (1 - commission error); $CTBICEs = classification-total-based image classification efficacies (adjusted user's accuracies); $producersAccuracies = class-level producer's accuracies (1 - omission error); $RTBICEs = reference-total-based image classification efficacies (adjusted producer's accuracies); $F1Scores = class-level harmonic mean of user's and producer's accuracies; $F1Efficacies = F1-score efficacies; $macroPA = class-aggregated, macro-averaged producer's accuracy; $macroRTBICE = class-aggregated, macro-averaged reference-total-based image classification efficacy; $macroUA = class-aggregated, macro-averaged user's accuracy; $macroCTBICE = class-aggregated, macro-averaged classification-total-based image classification efficacy; $macroF1 = class-aggregated, macro-averaged F1-score; $macroF1Efficacy = class-aggregated, macro-averaged F1 efficacy;
For binary classification, returns a list object with the following items: $Mappings = class names; $confusionMatrix = confusion matrix where columns represent the reference data and rows represent the classification result; $referenceCounts = count of samples in each reference class; $predictionCounts = count of predictions in each class; $postiveCase = name or mapping for the positive case; $overallAccuracy = overall accuracy; $MICE = map image classification efficacy; $Precision = precision (1 - commission error relative to positive case); $precisionEfficacy = precision efficacy; $NPV = negative predictive value (1 - commission error relative to negative case); $npvEfficacy = negative predictive value efficacy; $Recall = recall (1 - omission error relative to positive case); $recallEfficacy = recall efficacy; $specificity = specificity (1 - omission error relative to negative case); $specificityEfficacy = specificity efficacy; $f1Score = harmonic mean of precision and recall; $f1Efficacy = F1-score efficacy;
Examples
#Multiclass example
data(mcData)
mice(mcData$ref,
mcData$pred,
mappings=c("Barren", "Forest", "Impervious", "Low
Vegetation", "Mixed Dev", "Water"),
multiclass=TRUE)
#> $Mappings
#> [1] "Barren" "Forest" "Impervious" "Low\nVegetation"
#> [5] "Mixed Dev" "Water"
#>
#> $confusionMatrix
#> Reference
#> Predicted Barren Forest Impervious Low\nVegetation Mixed Dev Water
#> Barren 75 7 59 46 1 6
#> Forest 13 20585 62 617 142 21
#> Impervious 10 8 196 33 22 12
#> Low\nVegetation 63 138 34 2413 84 1
#> Mixed Dev 1 64 75 72 270 2
#> Water 1 5 0 1 1 158
#>
#> $referenceCounts
#> Barren Forest Impervious Low\nVegetation Mixed Dev
#> 163 20807 426 3182 520
#> Water
#> 200
#>
#> $predictionCounts
#> Barren Forest Impervious Low\nVegetation Mixed Dev
#> 194 21440 281 2733 484
#> Water
#> 166
#>
#> $overallAccuracy
#> [1] 0.9367144
#>
#> $MICE
#> [1] 0.7937788
#>
#> $usersAccuracies
#> Barren Forest Impervious Low\nVegetation Mixed Dev
#> 0.3865979 0.9601213 0.6975089 0.8829125 0.5578512
#> Water
#> 0.9518072
#>
#> $CTBICEs
#> Barren Forest Impervious Low\nVegetation Mixed Dev
#> 0.3826138 0.7753487 0.6923248 0.8660647 0.5485675
#> Water
#> 0.9514226
#>
#> $producersAccuracies
#> Barren Forest Impervious Low\nVegetation Mixed Dev
#> 0.4601227 0.9893305 0.4600939 0.7583281 0.5192308
#> Water
#> 0.7900000
#>
#> $RTBICEs
#> Barren Forest Impervious Low\nVegetation Mixed Dev
#> 0.4566161 0.9398949 0.4508410 0.7235537 0.5091362
#> Water
#> 0.7883244
#>
#> $f1Scores
#> Barren Forest Impervious Low\nVegetation Mixed Dev
#> 0.4201680 0.9745071 0.5544554 0.8158918 0.5378486
#> Water
#> 0.8633879
#>
#> $f1Efficacies
#> Barren Forest Impervious Low\nVegetation Mixed Dev
#> 0.4163522 0.8497292 0.5460772 0.7884211 0.5281168
#> Water
#> 0.8622284
#>
#> $macroPA
#> [1] 0.662851
#>
#> $macroRTBUCE
#> [1] 0.6447277
#>
#> $macroUA
#> [1] 0.7394665
#>
#> $macroCTBICE
#> [1] 0.7027237
#>
#> $macroF1
#> [1] 0.6990658
#>
#> $macroF1Efficacy
#> [1] 0.6724776
#>
#Binary example
data(biData)
mice(biData$ref,
biData$pred,
mappings = c("Mined", "Not Mined"),
multiclass=FALSE,
positiveIndex=1)
#> $Mappings
#> [1] "Mined" "Not Mined"
#>
#> $confusionMatrix
#> Reference
#> Predicted Mined Not Mined
#> Mined 158 2
#> Not Mined 20 4820
#>
#> $referenceCounts
#> Mined Not Mined
#> 178 4822
#>
#> $predictionCounts
#> Mined Not Mined
#> 160 4840
#>
#> $positiveCase
#> [1] "Mined"
#>
#> $overallAccuracy
#> [1] 0.9956
#>
#> $mice
#> [1] 0.9359024
#>
#> $Precision
#> [1] 0.9874999
#>
#> $precisionEfficacy
#> [1] 0.9870384
#>
#> $NPV
#> [1] 0.9958678
#>
#> $npvEfficacy
#> [1] 0.8838934
#>
#> $Recall
#> [1] 0.8876404
#>
#> $recallEfficacy
#> [1] 0.8834915
#>
#> $Specificity
#> [1] 0.9995852
#>
#> $specificityEfficicacy
#> [1] 0.9883459
#>
#> $f1Score
#> [1] 0.9349112
#>
#> $f1ScoreEfficacy
#> [1] 0.9323989
#>