Mulivariate Reduced Rank Regression in Non-Gaussian Contexts, Using Copulas


Book Description

We propose a new procedure to perform Reduced Rank Regression (RRR) in non-Gaussian contexts, based on Multivariate Dispersion Models. Reduced-Rank Multivariate Dispersion Models (RR-MDM) generalise RRR to a very large class of distributions, which include continuous distributions like the normal, Gamma, Inverse Gaussian, and discrete distributions like the Poisson and the binomial. A multivariate distribution is created with the help of the Gaussian copula and stimation is performed using maximum likelihood. We show how this method can be amended to deal with the case of discrete data. We perform Monte Carlo simulations and show that our estimator is more efficient than the traditional Gaussian RRR. In the framework of MDM's we introduce a procedure analogous to canonical correlations, which takes into account the distribution of the data.




Sparse Multivariate Reduced-Rank Regression with Covariance Estimation


Book Description

Multivariate multiple linear regression is multiple linear regression, but with multiple responses. Standard approaches assume that observations from different subjects are uncorrelated and so estimates of the regression parameters can be obtained through separate univariate regressions, regardless of whether the responses are correlated within subjects. There are three main extensions to the simplest model. The first assumes a low rank structure on the coefficient matrix that arises from a latent factor model linking predictors to responses. The second reduces the number of parameters through variable selection. The third allows for correlations between response variables in the low rank model. Chen and Huang propose a new model that falls under the reduced-rank regression framework, employs variable selection, and estimates correlations among error terms. This project reviews their model, describes its implementation, and reports the results of a simulation study evaluating its performance. The project concludes with ideas for further research.







Topics in Reduced Rank Regression


Book Description







On Integrative Reduced-rank Models and Applications


Book Description

The emerging of multi-view data, or multiple datasets collected from different sources measuring distinct but interrelated sets of characteristics on the same set of subjects, brings much complexity to the data analyses. Due to the view-specific characteristics and the interrelationship of multi-view data, integrative statistical methodologies are demanded. The reduced-rank structure is useful for extracting the complex dependence structure, as it achieves dimension reduction in coefficient matrix estimation and admits an appealing latent factor interpretation. We propose two approaches for integrative multivariate regression analyses incorporating certain reduced-rank structure, motivated by two kinds of multi-view data. We first consider the data with multi-view covariates, together with certain phenotype/outcome variables. Essential task is how to integratively extract the possibly low dimensional association structure among the sets of covariates when utilizing it to build a good predictive model. The proposed canonical variate regression (CVR) bridges the gap between canonical correlation analysis (CCA) and reduced-rank regression (RRR) by examining the interrelationship between multiple sets of features under the supervision from the responses. The non-convex optimization problem is solved by an alternating direction method of multipliers (ADMM) based algorithm. Simulation and two genetic study examples are presented. We also consider the data with multi-view responses, in which the mixed-type response variables are interrelated but have different distributions with missing values. The proposed mixed-response reduced-rank regression (mRRR) characterizes the joint dependence structure of responses by assuming a low-rank structure of the coefficient matrix. An efficient computation algorithms is developed and guaranteed to converge. The non-asymptotic bound of nature parameter estimation with rank constraint is also explored. Numerical examples including simulation and a longitudinal study of aging (LSOA) are presented. Limitations of proposed methods and directions of future work are summarized in the discussion chapter.




Multivariate Reduced-Rank Regression


Book Description

In the area of multivariate analysis, there are two broad themes that have emerged over time. The analysis typically involves exploring the variations in a set of interrelated variables or investigating the simultaneous relation ships between two or more sets of variables. In either case, the themes involve explicit modeling of the relationships or dimension-reduction of the sets of variables. The multivariate regression methodology and its variants are the preferred tools for the parametric modeling and descriptive tools such as principal components or canonical correlations are the tools used for addressing the dimension-reduction issues. Both act as complementary to each other and data analysts typically want to make use of these tools for a thorough analysis of multivariate data. A technique that combines the two broad themes in a natural fashion is the method of reduced-rank regres sion. This method starts with the classical multivariate regression model framework but recognizes the possibility for the reduction in the number of parameters through a restrietion on the rank of the regression coefficient matrix. This feature is attractive because regression methods, whether they are in the context of a single response variable or in the context of several response variables, are popular statistical tools. The technique of reduced rank regression and its encompassing features are the primary focus of this book. The book develops the method of reduced-rank regression starting from the classical multivariate linear regression model.




Integrative Multivariate Learning Via Composite Low-Rank Decompositions


Book Description

We develop novel composite low-rank methods to achieve integrative learning in multivariate linear regression, where both the multivariate responses and predictors can be of high dimensionality and in different data forms. We first consider a regression with multi-view feature sets where only a few views are relevant to prediction and the predictors within each relevant view contribute to the prediction collectively rather than sparsely. To tackle this problem, we propose an integrative reduced-rank regression (iRRR) where each view has its own low-rank coefficient matrix, to conduct view selection and within-view latent feature extraction in a supervised fashion. In addition, to assess the significance of each view in iRRR model, we propose a scaled approach for model estimation and develop a hypothesis testing procedure through de-biasing. Next, to facilitate integrative multi-view learning with grouped sub-compositional predictors, we incorporate the view-specific low-rank structure into a newly proposed multivariate log-contrast model to enable sub-composition selection and latent principal compositional factor extraction. Finally, we propose a nested reduced-rank regression (NRRR) approach to relate multivariate functional responses and predictors. The nested low-rank structure is imposed on the functional regression surfaces to simultaneously identify latent principal functional responses/predictors and control the complexity and smoothness of the association between them. Efficient computational algorithms are developed for these methods, and their theoretical properties are investigated. We apply the proposed methods to multiple applications including the longitudinal study of aging, the preterm infant study and the electricity demand prediction.