calc_type_metrics {qtkit} | R Documentation |
Calculate Frequency and Dispersion Metrics for Text Types
Description
Calculates various frequency and dispersion metrics for types (terms/tokens) in tokenized text data. Provides a comprehensive analysis of how types are distributed across documents in a corpus.
Usage
calc_type_metrics(data, type, document, frequency = NULL, dispersion = NULL)
Arguments
data |
Data frame. Contains the tokenized text data with document IDs and types/terms. |
type |
Symbol. Column in |
document |
Symbol. Column in |
frequency |
Character vector. Frequency metrics to calculate: - NULL (default): Returns only type counts - 'all': All available metrics - 'rf': Relative frequency - 'orf': Observed relative frequency (per 100) |
dispersion |
Character vector. Dispersion metrics to calculate: - NULL (default): Returns only type counts - 'all': All available metrics - 'df': Document frequency - 'idf': Inverse document frequency - 'dp': Gries' deviation of proportions |
Details
The function creates a term-document matrix internally and calculates the requested metrics. Frequency metrics show how often types occur, while dispersion metrics show how evenly they are distributed across documents.
The 'dp' metric (Gries' Deviation of Proportions) ranges from 0 (perfectly even distribution) to 1 (completely clumped distribution).
Value
Data frame containing requested metrics:
type: Unique types from input data
n: Raw frequency count
rf: Relative frequency (if requested)
orf: Observed relative frequency per 100 (if requested)
df: Document frequency (if requested)
idf: Inverse document frequency (if requested)
dp: Deviation of proportions (if requested)
References
Gries, Stefan Th. (2023). Statistical Methods in Corpus Linguistics. In Readings in Corpus Linguistics: A Teaching and Research Guide for Scholars in Nigeria and Beyond, pp. 78-114.
Examples
data_path <- system.file("extdata", "types_data.rds", package = "qtkit")
df <- readRDS(data_path)
calc_type_metrics(
data = df,
type = letter,
document = doc_id,
frequency = c("rf", "orf"),
dispersion = "dp"
)