precision {genridge} | R Documentation |
Measures of Precision and Shrinkage for Ridge Regression
Description
The goal of precision
is to allow you to study the relationship between shrinkage of ridge
regression coefficients and their precision directly by calculating measures of each.
Three measures of (inverse) precision based on the “size” of the
covariance matrix of the parameters are calculated. Let V_k \equiv \text{Var}(\mathbf{\beta}_k)
be the covariance matrix for a given ridge constant, and let \lambda_i , i= 1,
\dots p
be its eigenvalues. Then the variance (= 1/precision) measures are:
-
"det"
:\log | V_k | = \log \prod \lambda
(withdet.fun = "log"
, the default) or|V_k|^{1/p} =(\prod \lambda)^{1/p}
(withdet.fun = "root"
) measures the linearized volume of the covariance ellipsoid and corresponds conceptually to Wilks' Lambda criterion -
"trace"
:\text{trace}( V_k ) = \sum \lambda
corresponds conceptually to Pillai's trace criterion -
"max.eig"
:\lambda_1 = \max (\lambda)
corresponds to Roy's largest root criterion.
Two measures of shrinkage are also calculated:
-
norm.beta
: the root mean square of the coefficient vector\lVert\mathbf{\beta}_k \rVert
, normalized to a maximum of 1.0 ifnormalize == TRUE
(the default). -
norm.diff
: the root mean square of the difference from the OLS estimate\lVert \mathbf{\beta}_{\text{OLS}} - \mathbf{\beta}_k \rVert
. This measure is inversely related tonorm.beta
A plot method, plot.precision
facilitates making graphs of these quantities.
Usage
precision(object, det.fun, normalize, ...)
Arguments
object |
An object of class |
det.fun |
Function to be applied to the determinants of the covariance
matrices, one of |
normalize |
If |
... |
Other arguments (currently unused) |
Value
An object of class c("precision", "data.frame")
with the following columns:
lambda |
The ridge constant |
df |
The equivalent effective degrees of freedom |
det |
The |
trace |
The trace of the covariance matrix |
max.eig |
Maximum eigen value of the covariance matrix |
norm.beta |
The root mean square of the estimated coefficients, possibly normalized |
norm.diff |
The root mean square of the difference between the OLS solution
( |
Note
Models fit by lm
and ridge
use a different scaling for
the predictors, so the results of precision
for an lm
model
will not correspond to those for ridge
with ridge constant = 0.
Author(s)
Michael Friendly
See Also
Examples
longley.y <- longley[, "Employed"]
longley.X <- data.matrix(longley[, c(2:6,1)])
lambda <- c(0, 0.005, 0.01, 0.02, 0.04, 0.08)
lridge <- ridge(longley.y, longley.X, lambda=lambda)
# same, using formula interface
lridge <- ridge(Employed ~ GNP + Unemployed + Armed.Forces + Population + Year + GNP.deflator,
data=longley, lambda=lambda)
clr <- c("black", rainbow(length(lambda)-1, start=.6, end=.1))
coef(lridge)
(pdat <- precision(lridge))
# plot log |Var(b)| vs. length(beta)
with(pdat, {
plot(norm.beta, det, type="b",
cex.lab=1.25, pch=16, cex=1.5, col=clr, lwd=2,
xlab='shrinkage: ||b|| / max(||b||)',
ylab='variance: log |Var(b)|')
text(norm.beta, det, lambda, cex=1.25, pos=c(rep(2,length(lambda)-1),4))
text(min(norm.beta), max(det), "Variance vs. Shrinkage", cex=1.5, pos=4)
})
# plot trace[Var(b)] vs. length(beta)
with(pdat, {
plot(norm.beta, trace, type="b",
cex.lab=1.25, pch=16, cex=1.5, col=clr, lwd=2,
xlab='shrinkage: ||b|| / max(||b||)',
ylab='variance: trace [Var(b)]')
text(norm.beta, trace, lambda, cex=1.25, pos=c(2, rep(4,length(lambda)-1)))
# text(min(norm.beta), max(det), "Variance vs. Shrinkage", cex=1.5, pos=4)
})