Statistically compare two models using a paired t-test and bootstrap samples of the assessment results

Usage

miceCompare(ref, result1, result2, reps, frac)

Arguments

ref: column of reference labels as factor data type.
result1: column of predicted labels as factor data type (first result to compare).
result2: column of predicted labels as factor data type (second result to compare).
reps: number of bootstrap replicates to use. Default is 200.
frac: proportion of samples to include in each bootstrap sample. Default is 0.7.

Value

paired t-test results including t-statistic, degrees of freedom, p-value, 95% confidence interval, and mean difference

Examples

data(compareData)
compareResult <- miceCompare(ref=compareData$ref,
result1=compareData$rfPred,
result2=compareData$dtPred,
reps=1000,
frac=.7)
print(compareResult)
#> 
#> 	Paired t-test
#> 
#> data:  resultsDF$mice1 and resultsDF$mice2
#> t = 257.71, df = 999, p-value < 2.2e-16
#> alternative hypothesis: true mean difference is not equal to 0
#> 95 percent confidence interval:
#>  0.1067377 0.1083757
#> sample estimates:
#> mean difference 
#>       0.1075567 
#>