Book Description

With increasing appearances of high dimensional data over the past decades, variable selections through likelihood penalization remains a popular yet challenging research area in statistics. Ridge and Lasso, the two of the most popular penalized regression methods, served as the foundation of regularization technique and motivated several extensions to accommodate various circumstances, mostly through frequentist models. These two regularization problems can also be solved by their Bayesian counterparts, via putting proper priors on the regression parameters and then followed by Gibbs sampling. Compared to the frequentist version, the Bayesian framework enables easier interpretation and more straightforward inference on the parameters, based on the posterior distributional results. In general, however, the Bayesian approaches do not provide sparse estimates for the regression coefficients. In this thesis, an innovative Bayesian variable selection method via a benchmark variable in conjunction with a modified BIC is proposed under the framework of linear regression models as the first attempt, to promote both model sparsity and accuracy. The motivation of introducing such a benchmark is discussed, and the statistical properties regarding its role in the model are demonstrated. In short, it serves as a criterion to measure the importance of each variable based on the posterior inference of the corresponding coefficients, and only the most important variables providing the minimal modified BIC value are included. The Bayesian approach via a benchmark is extended to accommodate linear models with covariates exhibiting group structures. An iterative algorithm is implemented to identify both important groups and important variables within the selected groups. What's more, the method is further developed and modified to select variables for generalized linear models, by taking advantage of the normal approximation on the likelihood function. Simulation studies are carried out to assess and compare the performances among the proposed approaches and other state-of-art methods for each of the above three scenarios. The numerical results consistently illustrate our Bayesian variable selection approaches tend to select exactly the true variables or groups, while producing comparable prediction errors as other methods. Besides the numerical work, several real data sets are analyzed by these methods and the corresponding performances are further compared. The variable selection results by our approach are intuitively appealing or consistent with existing literatures in general.