#####Original Paper: Smith,J.A.,& Sivo,S.A.(2012). Predicting continued use of online teacher professional development and the influence of social presence and sociability. British Journal of Educational Technology, 43(6), 871-882. doi:10.1111/j.1467-8535.2011.01223.x
This document provides a simple example analysis of a path analysis dataset, a survey of teachers enrolled in a statewide online reading course. The study examines how a Technology Acceptance Model (TAM) could predict teachers’ intentions to continue using e-learning for professional development based on perceived ease of use and usefulness, as well as examine mediating influences of social presence and sociability in e-learning professional development.
The dataset has six manifest variables: Perceived Usefulness (PU), Perceived Ease of Use (PEU), Teachers’ Reading Knowledge Assessment gains (Gains), Social Presence (SP), Sociability (SOC), and Continuance Intention (CI).
In addition to the SEMsens package, this vignette also
makes use of lavaan.
Here, we reproduce the the correlation matrix found in the article.
First we create the lower diagonal and then convert to a covariance
matrix and label the variables with getCov() from
lavaan.
#Set a correlation matrix
lower = '
1.00
0.68 1.00
0.54 0.55 1.00
0.65 0.63 0.67 1.00
0.33 0.37 0.68 0.54 1.00
-0.01 0.00 0.03 -0.04 0.07 1.00'
#convert to full covariance matrix, using function from lavaan
full = getCov(lower, sds= c(4.61,5.37,7.25,3.44,8.91,8.80),
names = c("PU", "PEU", "SP", "CI","SOC","Gains"))We next set up the path model from the article, using
lavaan model syntax with sem function. Through
this code, we can get the result of (standardized) path coefficients and
model fit indices. Standardized coefficient and model fit of this test
almost exactly reproduces the results of the original paper (Smith &
Sivo,2012). Slight differences are a result of using different
statistical software (R or LISREL).
# Original model
lav_model <- 'SP~SOC
Gains~SP
PU~SP+PEU
PEU~SP
CI~SP+PU+PEU+SOC
Gains ~~ 0*CI
'
# Fit the original model with sem function
modelFit <- sem(lav_model, sample.nobs=517, sample.cov=full, fixed.x=TRUE, std.lv=TRUE)
summary(modelFit, standardized = TRUE) #look at Std.all## lavaan 0.6-19 ended normally after 9 iterations
##
## Estimator ML
## Optimization method NLMINB
## Number of model parameters 14
##
## Number of observations 517
##
## Model Test User Model:
##
## Test statistic 9.567
## Degrees of freedom 6
## P-value (Chi-square) 0.144
##
## Parameter Estimates:
##
## Standard errors Standard
## Information Expected
## Information saturated (h1) model Structured
##
## Regressions:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## SP ~
## SOC 0.553 0.026 21.087 0.000 0.553 0.680
## Gains ~
## SP 0.036 0.053 0.682 0.495 0.036 0.030
## PU ~
## SP 0.151 0.024 6.404 0.000 0.151 0.238
## PEU 0.471 0.032 14.775 0.000 0.471 0.549
## PEU ~
## SP 0.407 0.027 14.974 0.000 0.407 0.550
## CI ~
## SP 0.128 0.020 6.283 0.000 0.128 0.268
## PU 0.226 0.029 7.746 0.000 0.226 0.302
## PEU 0.134 0.025 5.318 0.000 0.134 0.209
## SOC 0.069 0.015 4.769 0.000 0.069 0.179
##
## Covariances:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## .Gains ~~
## .CI 0.000 0.000 0.000
##
## Variances:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## .SP 28.203 1.754 16.078 0.000 28.203 0.538
## .Gains 77.221 4.803 16.078 0.000 77.221 0.999
## .PU 10.565 0.657 16.078 0.000 10.565 0.498
## .PEU 20.075 1.249 16.078 0.000 20.075 0.697
## .CI 4.651 0.289 16.078 0.000 4.651 0.392## npar fmin chisq
## 14.000 0.009 9.567
## df pvalue baseline.chisq
## 6.000 0.144 1359.242
## baseline.df baseline.pvalue cfi
## 15.000 0.000 0.997
## tli nnfi rfi
## 0.993 0.993 0.982
## nfi pnfi ifi
## 0.993 0.397 0.997
## rni logl unrestricted.logl
## 0.997 -7436.961 -7432.177
## aic bic ntotal
## 14901.922 14961.394 517.000
## bic2 rmsea rmsea.ci.lower
## 14916.955 0.034 0.000
## rmsea.ci.upper rmsea.ci.level rmsea.pvalue
## 0.072 0.900 0.711
## rmsea.close.h0 rmsea.notclose.pvalue rmsea.notclose.h0
## 0.050 0.021 0.080
## rmr rmr_nomean srmr
## 1.038 1.038 0.020
## srmr_bentler srmr_bentler_nomean crmr
## 0.020 0.020 0.024
## crmr_nomean srmr_mplus srmr_mplus_nomean
## 0.024 0.020 0.020
## cn_05 cn_01 gfi
## 681.482 909.558 0.994
## agfi pgfi mfi
## 0.978 0.284 0.997
## ecvi
## 0.073We can get same results by using
lavannify,lavaan and the
standardizedsolution functions. These are all in the
lavaan package and present more focused results for
standardized path coefficients and their standard error and p-values.
Depending on users’ research questions, it is possible to select results
for individual pathways in the model.
smith_original <- lavaan::lavaanify(model = lav_model, auto = TRUE, model.type = "sem", fixed.x = TRUE)
smith_original <- lavaan::lavaan(model = smith_original, sample.cov = full, sample.nobs = 517)
smith_original_par <- lavaan::standardizedSolution(smith_original, type = "std.all")
smith_original_par #4th row and 7th column of table : smith_original_par[1:4,1:7]## lhs op rhs est.std se z pvalue ci.lower ci.upper
## 1 SP ~ SOC 0.680 0.021 32.801 0.000 0.639 0.721
## 2 Gains ~ SP 0.030 0.044 0.683 0.495 -0.056 0.116
## 3 PU ~ SP 0.238 0.037 6.492 0.000 0.166 0.310
## 4 PU ~ PEU 0.549 0.033 16.453 0.000 0.484 0.615
## 5 PEU ~ SP 0.550 0.030 18.226 0.000 0.491 0.609
## 6 CI ~ SP 0.268 0.042 6.343 0.000 0.186 0.351
## 7 CI ~ PU 0.302 0.038 7.852 0.000 0.227 0.378
## 8 CI ~ PEU 0.209 0.039 5.347 0.000 0.132 0.286
## 9 CI ~ SOC 0.179 0.037 4.804 0.000 0.106 0.252
## 10 Gains ~~ CI 0.000 0.000 NA NA 0.000 0.000
## 11 SP ~~ SP 0.538 0.028 19.068 0.000 0.482 0.593
## 12 Gains ~~ Gains 0.999 0.003 378.979 0.000 0.994 1.004
## 13 PU ~~ PU 0.498 0.031 16.195 0.000 0.438 0.558
## 14 PEU ~~ PEU 0.697 0.033 21.013 0.000 0.632 0.763
## 15 CI ~~ CI 0.392 0.026 15.201 0.000 0.342 0.443
## 16 SOC ~~ SOC 1.000 0.000 NA NA 1.000 1.000After checking the original path model, we then create the sensitivity model using a Phantom Variable. A phantom variable is modeled with paths to all other variables to see the trajectory of estimates in the original model affected by specification of the Phantom variable. As shown in the code below, the phantom variable follows the normal distribution which has mean of zero and variance of one.
# Sensitivity model, with sensitivity parameters for all variables
sens_model <- 'SP~SOC
Gains ~ SP
PU ~ SP+PEU
PEU ~ SP
CI ~ SP+PU+PEU+SOC
Gains ~~ 0*CI
SP ~ phantom1*phantom
Gains ~ phantom2*phantom
PU ~ phantom3*phantom
PEU ~ phantom4*phantom
CI ~ phantom5*phantom
SOC ~ phantom6*phantom
phantom =~ 0 #mean of zero
phantom ~~ 1*phantom # variance of one'Based on the specified sens_model, we can run the
sensitivity analysis through sa.aco() function in
SEMsens package. Note that we run with the parameters
k = 5 and max.iter = 20 for a simple
illustration. The default values for these parameters are
k = 50 and max.iter = 1000. For the other
options, see the paper or vignette of SEMsens package (https://cran.r-project.org/package=SEMsens).
smith_example <- sa.aco(
sample.cov = full,
sample.nobs = 517,
model = lav_model,
sens.model = sens_model,
opt.fun = 1,
paths = c(1:9),
max.iter = 20,
k = 5)## Number of tried evaluations is 1.
## Number of converged evaluations is 1.
## Number of tried evaluations is 2.
## Number of converged evaluations is 2.
## Number of tried evaluations is 3.
## Number of converged evaluations is 3.
## Number of tried evaluations is 4.
## Number of converged evaluations is 4.
## Number of tried evaluations is 5.
## Number of converged evaluations is 5.We can get the sensitivity analysis results after 5 iterations. The sens.tables function helps us to summarize of sensitivity analysis. In the smith_tables results, the sens.summary table contains estimates and p-values for each path in the original model information suggested in Step 1. It also provides the minimum, mean and maximum path estimates during sensitivity analysis.
## model.est model.pvalue mean.est.sens min.est.sens max.est.sens
## Gains~SP 0.03000001 4.947480e-01 0.03078806 0.02735571 0.03562662
## CI~SOC 0.17912705 1.553405e-06 0.17525972 0.14255763 0.20699081
## CI~PEU 0.20913951 8.949240e-08 0.20793924 0.16071079 0.24031659
## PU~SP 0.23799283 8.462719e-11 0.23152339 0.22321878 0.24179787
## CI~SP 0.26848140 2.255842e-10 0.27010031 0.22550080 0.32256848
## CI~PU 0.30228503 3.996803e-15 0.31350603 0.26999750 0.39231625
## PEU~SP 0.55000000 0.000000e+00 0.54972379 0.54023885 0.56146020
## PU~PEU 0.54910394 0.000000e+00 0.55356443 0.53608380 0.57792956
## SP~SOC 0.68000000 0.000000e+00 0.68091945 0.67523652 0.68849064The result of phan.paths suggests the minimum, mean and maximum value of sensitivity parameters which were formed in the relationship between phantom variable and each variables in the path model during the iteration of Ant Colony Optimization (ACO).
## mean.phan min.phan max.phan
## PEU~phantom -0.032628286 -0.16322067 0.1521077
## SOC~phantom 0.001431526 -0.08418588 0.1103978
## SP~phantom 0.002450675 -0.06472979 0.1227683
## Gains~phantom 0.003651090 -0.02906461 0.0630220
## CI~phantom 0.017121906 -0.17295276 0.2147725
## PU~phantom 0.036459926 -0.21132376 0.1887766The table of phan.min indicates the sensitivity parameters for each path that led to smallest size of path estimates during the iteration process of ACO.
## SP~phantom Gains~phantom PU~phantom PEU~phantom CI~phantom
## SP~SOC 0.05161516 -0.02637793 0.11714527 -0.0008575855 0.12632124
## Gains~SP 0.12276830 0.03659312 0.05607790 -0.1632206676 -0.17060386
## PU~SP 0.05161516 -0.02637793 0.11714527 -0.0008575855 0.12632124
## PU~PEU -0.06426463 -0.02591714 -0.21132376 -0.0438425624 0.21477253
## PEU~SP -0.03313566 0.06302200 0.18877664 -0.1073283518 0.08807237
## CI~SP -0.06472979 -0.02906461 0.03162358 0.1521077372 -0.17295276
## CI~PU -0.03313566 0.06302200 0.18877664 -0.1073283518 0.08807237
## CI~PEU 0.12276830 0.03659312 0.05607790 -0.1632206676 -0.17060386
## CI~SOC 0.12276830 0.03659312 0.05607790 -0.1632206676 -0.17060386
## SOC~phantom
## SP~SOC 0.11039776
## Gains~SP -0.07264936
## PU~SP 0.11039776
## PU~PEU -0.03585607
## PEU~SP -0.08418588
## CI~SP 0.08945118
## CI~PU -0.08418588
## CI~PEU -0.07264936
## CI~SOC -0.07264936Similar to phan.min case, phan.max table provides the sensitivity parameters for each path that resulted in the largest size of path estimates during the process of ACO.
## SP~phantom Gains~phantom PU~phantom PEU~phantom CI~phantom
## SP~SOC 0.12276830 0.03659312 0.05607790 -0.16322067 -0.17060386
## Gains~SP -0.03313566 0.06302200 0.18877664 -0.10732835 0.08807237
## PU~SP -0.06472979 -0.02906461 0.03162358 0.15210774 -0.17295276
## PU~PEU -0.03313566 0.06302200 0.18877664 -0.10732835 0.08807237
## PEU~SP 0.12276830 0.03659312 0.05607790 -0.16322067 -0.17060386
## CI~SP 0.12276830 0.03659312 0.05607790 -0.16322067 -0.17060386
## CI~PU -0.06426463 -0.02591714 -0.21132376 -0.04384256 0.21477253
## CI~PEU -0.03313566 0.06302200 0.18877664 -0.10732835 0.08807237
## CI~SOC -0.06472979 -0.02906461 0.03162358 0.15210774 -0.17295276
## SOC~phantom
## SP~SOC -0.07264936
## Gains~SP -0.08418588
## PU~SP 0.08945118
## PU~PEU -0.08418588
## PEU~SP -0.07264936
## CI~SP -0.07264936
## CI~PU -0.03585607
## CI~PEU -0.08418588
## CI~SOC 0.08945118The final p.paths table covers not only the p-values
of original model’s path estimates at the first column (default
significance level: 0.05) but the final p-value of each path estimates
that reverse the null-hypothesis decision of original path estimates.
From the third column of table, sensitivity parameters are suggested
that leads to the change of p-value. An NA result in
the table occurs if there is no change in p-value and meaningful
sensitivity parameters that changed p-value in the sa.aco
function.
## p.value p.changed SP~phantom Gains~phantom PU~phantom PEU~phantom
## SP~SOC 0.000000e+00 NA NA NA NA NA
## Gains~SP 4.947480e-01 NA NA NA NA NA
## PU~SP 8.462719e-11 NA NA NA NA NA
## PU~PEU 0.000000e+00 NA NA NA NA NA
## PEU~SP 0.000000e+00 NA NA NA NA NA
## CI~SP 2.255842e-10 NA NA NA NA NA
## CI~PU 3.996803e-15 NA NA NA NA NA
## CI~PEU 8.949240e-08 NA NA NA NA NA
## CI~SOC 1.553405e-06 NA NA NA NA NA
## CI~phantom SOC~phantom
## SP~SOC NA NA
## Gains~SP NA NA
## PU~SP NA NA
## PU~PEU NA NA
## PEU~SP NA NA
## CI~SP NA NA
## CI~PU NA NA
## CI~PEU NA NA
## CI~SOC NA NALeite, W., Shen, Z., Marcoulides, K., Fish, C., & Harring, J. (2022). Using ant colony optimization for sensitivity analysis in structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal, 29 (1), 47-56.