Reliability study designer

Calculation of the number of required subjects and replications in a reliability assessment

You are now following this Submission

File name : 'Shoukri.m'. This file can be used
for calculating the required number of subjects
at a desired kappa value for reliability
assessment.
Five input values : 'alfa', 'omega', 'pi', 'rho' and 'kappa'
Three output value : 'k', 'n' and 'N'
alfa : significance level (value between 0 and 1)
omega : desired interval of kappa (value between 0 and 1)
pi : probability of positive rating (value between 0 and 1)
rho : within-subject interclass correlation
kappa : desired coefficient of agreement (value between 0 and 1)

k : number of subjects (value larger than 1)
n : number of replication (value equal to or larger than 1)
N : number of allocations (value larger than 1)
Figure : plot of the number of subjects versus the number of replications
Input syntax : Shoukri(alfa,omega,pi,kappa)
NOTE: No default values are set
REFERENCES
[1] : Shoukri, M. M. (2010). Measures of interobserver agreement and reliability: CRC Press.
[2] : Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 159-174.
[3] : Shoukri, M., Asyali, M. & Walter, S. (2003). Issues of cost and efficiency in the design of reliability studies. biometrics. 59(4): p. 1107-1112.
[4] : Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and psychological measurement. 20(1): p. 37-46

Developed by Joris Meurs BASc (2015)

Cite As

Joris Meurs (2026). Reliability study designer (https://ch.mathworks.com/matlabcentral/fileexchange/52990-reliability-study-designer), MATLAB Central File Exchange. Retrieved .

Categories

Find more on Behavior and Psychophysics in Help Center and MATLAB Answers

General Information

MATLAB Release Compatibility

  • Compatible with any release

Platform Compatibility

  • Windows
  • macOS
  • Linux
Version Published Release Notes Action
2.3.0.0

Updated required products

2.2.0.0

Added picture

2.1.0.0

Change of title

2.0.0.0

1. Plot of number of subjects vs. number of replications added
2. Number of input and output arguments changed

1.1.0.0

Title name is changed

1.0.0.0