Generalized ridge estimators adapted in structural equation models

Multicollinearity is detected via regression models, where independent variables are strongly correlated. Since they entail linear relations between observed or latent variables, the structural equation models (SEM) are subject to the multicollinearity effect, whose numerous consequences include the singularity between the inverse matrices used in estimation methods. Given to this behavior, it is natural to understand that the suitability of these estimators to structural equation models show the same features, either in the simulation results that validate the estimators in different multicollinearity degrees, or in their application to real data. Due to the multicollinearity overview arose from the fact that the matrices inversion is impracticable, the usage of numerical procedures demanded by the maximum likelihood methods leads to numerical singularity problems. An alternative could be the use of the Partial Least Squares (PLS) method, however, it is demanded that the observed variables are built by assuming a positive correlation with the latent variable. Thus, theoretically, it is expected that the load signals are positive, however, there are no restrictions to these signals in the algorithms used in the PLS method. This fact implies in corrective areas, such as the observed variables removal or new formulations of the theoretical model. In view of this problem, this paper aimed to propose adaptations of six generalized ridge estimators as alternative methods to estimate SEM parameters. The conclusion is that the evaluated estimators presented the same performance in terms of accuracy, precision while considering the scenarios represented by model without specification error and model with specification error, different levels of multicollinearity and sample sizes.


Introduction
The existence of a strong correlation between the variables involved in the estimation of parameters of a model characterizes the multicollinearity problem, whose main consequence is the high estimates of standard coefficients and errors, compromising conclusions related to statistical inference (Mori & Suzuki, 2018). In view of this problem, numerous alternatives to detect and solve this problem are reported in literature. Tarka (2018), in a review article, mentions that despite the potentially severe consequences for statistical inference, the issue of multicollinearity, as well as the impact of omitting variables, has been little studied about the effects on the analysis of structural equation models (SEM) . Corroborating this statement, some studies on this problem are addressed. Yang and Yuan (2019) mention that the presence of multicollinearity involves obtaining operations of approximate or badly conditioned inverse matrices, causing a problem of numerical nature related to obtaining the estimates of maximum likelihood of a SEM.
Due to numerical convergence problems, Can, Schoot, and Hox (2015) proposed a study that includes the parameter estimates in multilevel structural equations models (MLSEM), obtained by the maximum likelihood and Bayesian methods specifying different degrees of correlation between and among the levels, so that, given a value above 0.80, multicollinearity was detected.
When considering the maximum likelihood approach, the solutions obtained as a result of numerical non-convergence were called as inadmissible solutions. In this way, both methods were compared, having as reference the Bayesian procedure, in which, all solutions were admissible. In view of the above, it was observed that the effect of multicollinearity has a greater impact on the estimates of the intraclass correlation coefficients in both methods, however, a greater amount of inadmissible solutions was found when multicollinearity was specified between the levels. Leeflang (2011) mentions that, in general, the effect of multicollinearity observed in a SEM model, refers to high correlations between the latent exogenous variables and, as a consequence, specifically the validation of these models becomes complex. Can et al. (2015) point out that the effect of multicollinearity generates a correlation matrix between poorly conditioned predictors so that there is no single mathematical solution to estimate the model's coefficients, referring to the model identifiability problems.
Lan and Maguire (2012) emphasize that the interpretation of direct and indirect effects must be made with caution when the variables are multicolinear since these effects express a cause-and-effect relationship.
Among the solutions addressed and feasible to be applied, given the presence of multicollinearity in SEM, ridge modeling has been little explored. Nyrhinen and Leskinen (2014) studied two methodological procedures involving the ridge trace, namely method A and B. Method A consisted of assigning a constant k in all elements on the diagonal of the model correlation matrix. Method B characterized the assignment of this constant to the corresponding elements of the endogenous and exogenous variables in the model correlation matrix. In this context, in simulation studies, the authors concluded that in both methods, the coefficient estimates were the same; however, method B produced lower standard errors. Yuan, Wu, and Bentler (2011) showed by means of empirical results that the Ridge procedure for SEM with ordinal data presents a better convergence rate, lower bias, lower mean squared error and better general model evaluation than the widely used procedure of maximum likelihood.
Given that the formation of an SEM model involves linear relations between independent and dependent variables, either observed or latent, with or without measuring errors (Marsh, Morin, Parker, & Kaur, 2014;Larina, 2015;Neelaveni & Manimaran, 2016), the adaptation of different ridge estimators becomes applicable, for example, the generalized ridge regression likelihood and other alternatives proposed by Kibria (2003).
Based on the assumption that the variables involved in structural equation modeling present different correlation levels, it is reasonable to assume the existence of multicollinearity. Given this motivation, this study aimed to incorporate the ridge estimators listed in Table 1 into structural equation modeling, as well as to evaluate them regarding the properties of accuracy and precision by means of the Monte Carlo simulation. Finally, an application to real data is presented, providing the script of the function used in the application and simulation of the parameter estimates.

Material and methods
The methodology proposed for the adaptation of ridge estimators in structural equation modeling is described in the following stages: i) Estimators of generalized ridge regression and alternatives; ii) specification of the structural equation model; iii) adaptation of ridge estimators to the structural equation model and iv) scenarios and parametric values used in the Monte Carlo procedure to validate the generalized ridge estimators in structural equation models.

Estimators of generalized ridge regression and its alternatives
Defining the linear regression model (Equation 1). (1) where: is the vector of independent observations, is the parametric vector of regression coefficients to be estimated, is a known matrix of explanatory variables and is the vector of errors, with each component being . Supposing the existence of an orthogonal matrix , by means of decomposition , where is the diagonal matrix of eigenvalues of matrix , model (1) is rewritten in canonical form as Equation 2. (2) where: and . Note that depends on , according to Kibria (2003) recommendations , the mean square error (MSE) will be minimized when is considered as the normalized eigenvector, corresponding to the highest eigenvalue of matrix , respecting the restriction . With these specifications, the generalized ridge estimators are obtained by Equation 3.
( 3) where: ; ; equals the number of independent variables involved in the model, and is the estimator of minimum squares of . Estimators and are obtained, respectively by the inverse transformation of and . Kibria (2003) mentions that an estimated value of Equation 4, which minimizes the mean square error (MSE) Equation 5 of is defined by: (4) error variance is estimated by the residual mean square; is the eigenvalue of matrix and the element of . Following this methodology, several estimators were proposed in the literature by different authors, as illustrated by the summary described in Table 1. is a partitioned matrix, containing the coefficients that correlate the endogenous factors and relate the exogenous factors to the endogenous factors;

Specification of the structural equation model
represents a partitioned vector of endogenous latent variables and exogenous latent variables and is an error vector. As a matrix, the system is defined in Equation 7.
Consider subsystem Equation 8 referring to the exogenous variables.
corresponds to the observed exogenous variables vector; is a matrix of regression coefficients that relates the exogenous factors to each of observable variable assigned to measure them; represents a vector of exogenous latent variables and is a vector of measurement errors in X. Following these specifications, the matrix representation is given in Equation 9. (9)

Adaptation of ridge estimators to the structural equation model
Following the structural model given in Equation 6, the adaptation of ridge estimators is initially made by obtaining an orthogonal matrix D, by means of0 the decomposition , where is the matrix containing the eigenvalues of . Therefore, the structural model was rewritten in canonical form as Equation 10: where: and . In relation to the measurement model (8) where: and refer to the estimations of error variances of Equation 10 and 11; the estimation of ; and the estimation of . Therefore, following these specifications, the generalized ridge estimators listed in Table 1 and adapted to structural equation modeling, following the methodology proposed herein, are described in Table 2.
Concerning the measurement model related to the endogenous variables, the adaptations of the generalized ridge estimators are made as with the measurement model in described in this study.

Scenarios and parametric values used in the Monte Carlo procedure to validate the generalized ridge estimators in structural equation models
After defining the generalized ridge estimators (Table 2), two thousand ( ) Monte Carlo simulations were used, based on the structural equation model, with the parametric values specified as illustrated in Figure 1.
Mantaining the usual assumptions of the structural model, where the expectations of error vectors and latent variables equal zero, and (i = 1, 2, 3) are not correlated; (j = 1, 2, 3, 4) are not correlated to , and ; and (i = 1, 2, 3) are nor correlated to , and . The indicators of exogenous latent variables will be considered multicollinear in different levels, being generated by Monte Carlo simulations, according to the procedure proposed by Pereira, Milani, and Cirillo (2014) [Equation 20].
where: , is specified so that the correlation between both explanatory variables is given by ; is the sample size; and is the number of observed exogenous variables to be generated. Thus, the multicollinearity level between variables, controlled and classified in levels, is: weak ( ), moderate ( and ) and strong ( and ), for sample sizes evaluated in , and , therefore, a identifiability of the model due to the number of parameters required as a function of sample size (Cirillo & Barroso, 2017) for each estimator (Table 2).
For all estimators, and for each configuration between sample size and multicollinearity level, the mean square error Equation 18 and 19 was computed considering the mean estimations obtained in Monte Carlo iterations. For this purpose, a function will be built and implemented in the software R (R Core Team, 2018).

Accuracy and precision of generalized ridge estimators considering a structural equation model without specification error
According to the methodology, in view of the evaluated scenarios (Section iii), given the approaches of estimations of mean square errors due to Monte Carlo oscillations, the results described in Table 3 show that all estimators were accurate and precise for all levels of multicollinearity evaluated. Diamantopoulos, Riefler and Roth (2008) mention that the presence of multicollinear variables in models with formative indicators may induce the researcher to exclude insignificant indicators, thus altering the construct definition. Therefore, considering that results are similar, generalized ridge estimators can be recommended as an alternative method for estimation of the model parameters in relation to their competitors, once their accuracy and precision have been confirmed through the low values of the mean square error. Concerning the effect of sample size, the accuracy and precision of the generalized ridge estimators (Table 3) were in accordance with the studies conducted by Cassel, Hackl, and Westlund (1999) by using the PLS estimation method, where it was concluded that the bias estimations were not affected by the increase in sample size. However, comparing to the results obtained by Jung (2013), where the ridge regression incorporated to the square minimum method in two stages was considered, the authors concluded that, for small sample sizes, the estimations obtained by a ridge method were more stable and precise, but associated with the greatest biases. Given that the residues were generated by assuming normality, there are statistical evidences to state that accuracy and precision are reached by generalized ridge estimators. The equality between estimations of mean square errors was certainly influenced by the fact that residues were generated by a symmetric distribution, in this case, normal distribution. This statement is confirmed through studies obtained by Cirillo and Barroso (2012) by considering robust estimators LMS and LTS for the same model (Figure 1), but generated with symmetric and asymmetric errors. The results related to biases were quite discrepant. In relation to the accuracy of estimations, for all sample sizes, the LMS method showed a trend of overestimating parametric values; and the LTS method, a trend of underestimating them.
Given the same scenarios evaluated by the Monte Carlo simulation, the results described in Table 4 showed that, by considering the specification error, omitting and simultaneously, mean square error estimations were accurate and precise, for all generalized ridge estimators with small oscillations due to the Monte Carlo error. It is important to note that the results obtained for these estimators are coherent with studies conducted by Maydeu-Olivares et al. (2019), who compared a structural model without specification error to a model considering the omission of two causal relations simultaneously. In this context, the authors concluded that there are no significant differences between models with or without specification error in relation to the adjustment quality.

Application to real data
Based on data related to the profile description of coffee consumers in relation to brand and quality, a questionnaire was applied using a 5-point Likert scale involving demographic and economic questions in a sample of individuals (Table 5). As these are categorical and ordinal questions, the authors chose to transform the data into a continuous scale between and . For such, the transformation given in Equation 21 was used, referring to the response to the question.
where: and are, respectively, the lowest and highest responses. Aiming to identify latent variables and , an exploratory factorial analysis was performed, using the data transformed in continuous scale (Equation 21), justifying the Pearson correlation matrix used in the analysis. Results are listed below in Table 6. The selection of the variables to be used in constructing each construct was determined according to the variables that had higher estimations of factorial loads (Table 6). Therefore, the set of equations constituting the structural model is represented by Equation 22.
The results in Table 7 validate the model adjustment because they have low values for the mean square error of generalized ridge estimators in comparison to square minimum estimations.
The generalized ridge estimators showed inferior results to those of the MSE obtained by the square minimum method. Another important result is the proximity of estimations obtained in each method, corroborating the results obtained in simulation. Subsequently, the estimations of the model parameters for each evaluated method are described in Table 8, along with the coefficient of determination .
Considering the estimates of model parameters, the results in Table 8 showed that all g eneralized regression ridge methods have similar estimations, including results related to the mean square error (Table 6). Therefore, there is statistical evidence to state that the inferential procedures, which can be applied to model adjustment, will provide similar results to those of the estimators evaluated in this study. Table 5. Categorical Questions and observed variables (X) used in the research.

2-Family income
Wage income levels: 1; 2; 3; 4; 5. Importance assigned Questions Description of categories 3-How important do you attribute the brand when choosing coffee?
Degree of importance: 1; 2; 3; 4; 5 4-The coffee quality depends more on the roaster firm and not the farmers.
Degree of importance of producer: 1; 2; 3; 4; 5 6-The region where coffee is produced interferes with the quality.
Degree of importance to region: 1; 2; 3; 4; 5 7-When I buy coffee, I look for information on the production region.
Degree of importance to region: 1; 2; 3; 4 ; 5 8 -I prefer coffee with identification of the origin, even if they are more expensive. Degree of importance for the preference: 1; 2; 3; 4 ; 5 9-I prefer coffee of superior quality, even if they are more expensive. Degree of importance for the price: 1; 2; 3; 4; 5

Conclusion
Our results support the conclusion that the generalized ridge estimators adapted to structural equation models can be applied to real situations, including problems involving a strong multicollinearity between the observed variables. The generalized ridge estimators showed the same performance in relation to accuracy, precision, model specification error, multicollinearity level and sample size.
Regarding the situations in which the model was generated considering the specification error of the latent variables, the ridge estimators presented accurate and precise results, showing robustness in relation to the omission of the latent variables.