pcd2 {irtQ} | R Documentation |
Pseudo-count D2 method
Description
This function calculates the Pseudo-count D^{2}
statistic
to evaluate item parameter drift, as described by Cappaert et al. (2018) and
Stone (2000). The Pseudo-count D^{2}
statistic is designed to detect
item parameter drift efficiently without requiring item recalibration, making
it especially valuable in computerized adaptive testing (CAT) environments.
This method compares observed and expected response frequencies across
quadrature points, which represent latent ability levels. The expected
frequencies are computed using the posterior distribution of each examinee's
ability (Stone, 2000), providing a robust and sensitive measure of item
parameter drift, ensuring the stability and accuracy of the test over time.
Usage
pcd2(
x,
data,
D = 1,
item.skip = NULL,
missing = NA,
Quadrature = c(49, 6),
weights = NULL,
group.mean = 0,
group.var = 1,
crit.val = NULL,
min.resp = NULL,
purify = FALSE,
max.iter = 10,
verbose = TRUE
)
Arguments
x |
A data frame containing item metadata (e.g., item parameters, number
of categories, IRT model types, etc.). See |
data |
A matrix of examinees' item responses corresponding to the items
specified in the |
D |
A scaling constant used in IRT models to make the logistic function closely approximate the normal ogive function. A value of 1.7 is commonly used for this purpose. Default is 1. |
item.skip |
A numeric vector of item indices to exclude from IPD analysis.
If |
missing |
A value indicating missing responses in the data set. Default
is |
Quadrature |
A numeric vector of length two:
|
weights |
A two-column matrix or data frame containing the quadrature
points (in the first column) and their corresponding weights (in the second
column) for the latent variable prior distribution. If not If |
group.mean |
A numeric value specifying the mean of the latent variable
prior distribution when |
group.var |
A positive numeric value specifying the variance of the
latent variable prior distribution when |
crit.val |
A critical value applied in hypothesis testing using
the Pseudo-count |
min.resp |
A positive integer specifying the minimum required number of
responses for each evaluated item. Defaults to |
purify |
Logical. Indicates whether to apply a purification procedure.
Default is |
max.iter |
A positive integer specifying the maximum number of
iterations allowed for the purification process. Default is |
verbose |
Logical. If |
Details
The Pseudo-count D^{2}
statistic quantifies item parameter drift (IPD) by
computing the weighted squared differences between the observed and expected
response frequencies for each score category across ability levels. The expected
frequencies are determined using the posterior distribution of each examinee's
ability (Stone, 2000).
The Pseudo-count D^{2}
statistic is calculated as:
Pseudo-count D^{2} = \sum_{k=1}^{Q} \left( \frac{r_{0k} + r_{1k}}{N}\right)
\left( \frac{r_{1k}}{r_{0k} + r_{1k}} - E_{1k} \right)^2
where r_{0k}
and r_{1k}
are the pseudo-counts for the incorrect
and correct responses at each ability level k
, E_{1k}
is the
expected proportion of correct responses at each ability level k
,
calculated using item parameters from the item bank, and N
is the total
count of examinees who received each item.
Critical Value (crit.val
):
The crit.val
argument specifies the threshold used to flag an item as
exhibiting potential parameter drift. If an item's Pseudo-count D^{2}
value exceeds this threshold, it is identified as a drifted item. If
crit.val = NULL
, the function reports the raw statistic without flagging.
Minimum Response Count (min.resp
):
The min.resp
argument sets a lower bound on the number of responses required
for an item to be included in the analysis. Items with fewer responses than
min.resp
are automatically excluded by replacing all their responses
with NA
. This avoids unreliable estimates based on small sample sizes.
Purification Procedure:
Although Cappaert et al. (2018) did not incorporate purification into their method,
pcd2()
implements an optional iterative purification process similar
to Lim et al. (2022). When purify = TRUE
and a crit.val
is provided:
The procedure begins by identifying items flagged for drift using the initial Pseudo-count
D^{2}
statistics.In each subsequent iteration, the item with the highest flagged Pseudo-count
D^{2}
value is removed from the item set, and the statistics are recalculated using only the remaining items.The process continues until no additional items are flagged or the number of iterations reaches
max.iter
.All flagged items and statistics are saved, and convergence status is reported.
This process ensures that drift detection is not distorted by already-flagged items, improving the robustness of the results.
Value
This function returns a list containing four main components:
no_purify |
A list containing the results of Pseudo-count
|
purify |
A logical value indicating whether the iterative purification
procedure was applied ( |
with_purify |
A list containing the results of Pseudo-count
|
crit.val |
A numeric value indicating the critical threshold used to flag
items for parameter drift. If not specified by the user, this will be |
Author(s)
Hwanggyu Lim hglim83@gmail.com
References
Cappaert, K. J., Wen, Y., & Chang, Y. F. (2018). Evaluating CAT-adjusted approaches for suspected item parameter drift detection. Measurement: Interdisciplinary Research and Perspectives, 16(4), 226-238.
Stone, C. A. (2000). Monte Carlo based null distribution for an alternative goodness-of-fit test statistic in IRT models. Journal of educational measurement, 37(1), 58-75.
Examples
## Example 1: No critical value specified
## Compute the Pseudo-count D² statistics for dichotomous items
## Import the "-prm.txt" output file generated by flexMIRT
flex_sam <- system.file("extdata", "flexmirt_sample-prm.txt", package = "irtQ")
# Extract metadata for the first 30 3PLM items
x <- bring.flexmirt(file = flex_sam, "par")$Group1$full_df[1:30, 1:6]
# Generate abilities for 500 examinees from N(0, 1)
set.seed(25)
score <- rnorm(500, mean = 0, sd = 1)
# Simulate response data using the item metadata and ability values
data <- simdat(x = x, theta = score, D = 1)
# Compute the Pseudo-count D² statistics (no purification applied)
ps_d2 <- pcd2(x = x, data = data)
print(ps_d2)
## Example 2: Applying a critical value with purification
# Compute the Pseudo-count D² statistics with purification enabled
ps_d2_puri <- pcd2(x = x, data = data, crit.val = 0.002, purify = TRUE)
print(ps_d2_puri)