Integrative Multivariate Learning Via Composite Low-Rank Decompositions


Book Description

We develop novel composite low-rank methods to achieve integrative learning in multivariate linear regression, where both the multivariate responses and predictors can be of high dimensionality and in different data forms. We first consider a regression with multi-view feature sets where only a few views are relevant to prediction and the predictors within each relevant view contribute to the prediction collectively rather than sparsely. To tackle this problem, we propose an integrative reduced-rank regression (iRRR) where each view has its own low-rank coefficient matrix, to conduct view selection and within-view latent feature extraction in a supervised fashion. In addition, to assess the significance of each view in iRRR model, we propose a scaled approach for model estimation and develop a hypothesis testing procedure through de-biasing. Next, to facilitate integrative multi-view learning with grouped sub-compositional predictors, we incorporate the view-specific low-rank structure into a newly proposed multivariate log-contrast model to enable sub-composition selection and latent principal compositional factor extraction. Finally, we propose a nested reduced-rank regression (NRRR) approach to relate multivariate functional responses and predictors. The nested low-rank structure is imposed on the functional regression surfaces to simultaneously identify latent principal functional responses/predictors and control the complexity and smoothness of the association between them. Efficient computational algorithms are developed for these methods, and their theoretical properties are investigated. We apply the proposed methods to multiple applications including the longitudinal study of aging, the preterm infant study and the electricity demand prediction.




Handbook of Robust Low-Rank and Sparse Matrix Decomposition


Book Description

Handbook of Robust Low-Rank and Sparse Matrix Decomposition: Applications in Image and Video Processing shows you how robust subspace learning and tracking by decomposition into low-rank and sparse matrices provide a suitable framework for computer vision applications. Incorporating both existing and new ideas, the book conveniently gives you one-stop access to a number of different decompositions, algorithms, implementations, and benchmarking techniques. Divided into five parts, the book begins with an overall introduction to robust principal component analysis (PCA) via decomposition into low-rank and sparse matrices. The second part addresses robust matrix factorization/completion problems while the third part focuses on robust online subspace estimation, learning, and tracking. Covering applications in image and video processing, the fourth part discusses image analysis, image denoising, motion saliency detection, video coding, key frame extraction, and hyperspectral video processing. The final part presents resources and applications in background/foreground separation for video surveillance. With contributions from leading teams around the world, this handbook provides a complete overview of the concepts, theories, algorithms, and applications related to robust low-rank and sparse matrix decompositions. It is designed for researchers, developers, and graduate students in computer vision, image and video processing, real-time architecture, machine learning, and data mining.




On Integrative Reduced-rank Models and Applications


Book Description

The emerging of multi-view data, or multiple datasets collected from different sources measuring distinct but interrelated sets of characteristics on the same set of subjects, brings much complexity to the data analyses. Due to the view-specific characteristics and the interrelationship of multi-view data, integrative statistical methodologies are demanded. The reduced-rank structure is useful for extracting the complex dependence structure, as it achieves dimension reduction in coefficient matrix estimation and admits an appealing latent factor interpretation. We propose two approaches for integrative multivariate regression analyses incorporating certain reduced-rank structure, motivated by two kinds of multi-view data. We first consider the data with multi-view covariates, together with certain phenotype/outcome variables. Essential task is how to integratively extract the possibly low dimensional association structure among the sets of covariates when utilizing it to build a good predictive model. The proposed canonical variate regression (CVR) bridges the gap between canonical correlation analysis (CCA) and reduced-rank regression (RRR) by examining the interrelationship between multiple sets of features under the supervision from the responses. The non-convex optimization problem is solved by an alternating direction method of multipliers (ADMM) based algorithm. Simulation and two genetic study examples are presented. We also consider the data with multi-view responses, in which the mixed-type response variables are interrelated but have different distributions with missing values. The proposed mixed-response reduced-rank regression (mRRR) characterizes the joint dependence structure of responses by assuming a low-rank structure of the coefficient matrix. An efficient computation algorithms is developed and guaranteed to converge. The non-asymptotic bound of nature parameter estimation with rank constraint is also explored. Numerical examples including simulation and a longitudinal study of aging (LSOA) are presented. Limitations of proposed methods and directions of future work are summarized in the discussion chapter.




Multivariate Reduced-Rank Regression


Book Description

In the area of multivariate analysis, there are two broad themes that have emerged over time. The analysis typically involves exploring the variations in a set of interrelated variables or investigating the simultaneous relation ships between two or more sets of variables. In either case, the themes involve explicit modeling of the relationships or dimension-reduction of the sets of variables. The multivariate regression methodology and its variants are the preferred tools for the parametric modeling and descriptive tools such as principal components or canonical correlations are the tools used for addressing the dimension-reduction issues. Both act as complementary to each other and data analysts typically want to make use of these tools for a thorough analysis of multivariate data. A technique that combines the two broad themes in a natural fashion is the method of reduced-rank regres sion. This method starts with the classical multivariate regression model framework but recognizes the possibility for the reduction in the number of parameters through a restrietion on the rank of the regression coefficient matrix. This feature is attractive because regression methods, whether they are in the context of a single response variable or in the context of several response variables, are popular statistical tools. The technique of reduced rank regression and its encompassing features are the primary focus of this book. The book develops the method of reduced-rank regression starting from the classical multivariate linear regression model.




Mathematics for Machine Learning


Book Description

The fundamental mathematical tools needed to understand machine learning include linear algebra, analytic geometry, matrix decompositions, vector calculus, optimization, probability and statistics. These topics are traditionally taught in disparate courses, making it hard for data science or computer science students, or professionals, to efficiently learn the mathematics. This self-contained textbook bridges the gap between mathematical and machine learning texts, introducing the mathematical concepts with a minimum of prerequisites. It uses these concepts to derive four central machine learning methods: linear regression, principal component analysis, Gaussian mixture models and support vector machines. For students and others with a mathematical background, these derivations provide a starting point to machine learning texts. For those learning the mathematics for the first time, the methods help build intuition and practical experience with applying mathematical concepts. Every chapter includes worked examples and exercises to test understanding. Programming tutorials are offered on the book's web site.




Handbook of Variational Methods for Nonlinear Geometric Data


Book Description

This book covers different, current research directions in the context of variational methods for non-linear geometric data. Each chapter is authored by leading experts in the respective discipline and provides an introduction, an overview and a description of the current state of the art. Non-linear geometric data arises in various applications in science and engineering. Examples of nonlinear data spaces are diverse and include, for instance, nonlinear spaces of matrices, spaces of curves, shapes as well as manifolds of probability measures. Applications can be found in biology, medicine, product engineering, geography and computer vision for instance. Variational methods on the other hand have evolved to being amongst the most powerful tools for applied mathematics. They involve techniques from various branches of mathematics such as statistics, modeling, optimization, numerical mathematics and analysis. The vast majority of research on variational methods, however, is focused on data in linear spaces. Variational methods for non-linear data is currently an emerging research topic. As a result, and since such methods involve various branches of mathematics, there is a plethora of different, recent approaches dealing with different aspects of variational methods for nonlinear geometric data. Research results are rather scattered and appear in journals of different mathematical communities. The main purpose of the book is to account for that by providing, for the first time, a comprehensive collection of different research directions and existing approaches in this context. It is organized in a way that leading researchers from the different fields provide an introductory overview of recent research directions in their respective discipline. As such, the book is a unique reference work for both newcomers in the field of variational methods for non-linear geometric data, as well as for established experts that aim at to exploit new research directions or collaborations. Chapter 9 of this book is available open access under a CC BY 4.0 license at link.springer.com.




Modern Multivariate Statistical Techniques


Book Description

This is the first book on multivariate analysis to look at large data sets which describes the state of the art in analyzing such data. Material such as database management systems is included that has never appeared in statistics books before.




Image and Graphics


Book Description

This three-volume set LNCS 12888, 12898, and 12890 constitutes the refereed conference proceedings of the 11th International Conference on Image and Graphics, ICIG 2021, held in Haikou, China, in August 2021.* The 198 full papers presented were selected from 421 submissions and focus on advances of theory, techniques and algorithms as well as innovative technologies of image, video and graphics processing and fostering innovation, entrepreneurship, and networking. *The conference was postponed due to the COVID-19 pandemic.




Big Data Analysis and Artificial Intelligence for Medical Sciences


Book Description

Big Data Analysis and Artificial Intelligence for Medical Sciences Overview of the current state of the art on the use of artificial intelligence in medicine and biology Big Data Analysis and Artificial Intelligence for Medical Sciences demonstrates the efforts made in the fields of Computational Biology and medical sciences to design and implement robust, accurate, and efficient computer algorithms for modeling the behavior of complex biological systems much faster than using traditional modeling approaches based solely on theory. With chapters written by international experts in the field of medical and biological research, Big Data Analysis and Artificial Intelligence for Medical Sciences includes information on: Studies conducted by the authors which are the result of years of interdisciplinary collaborations with clinicians, computer scientists, mathematicians, and engineers Differences between traditional computational approaches to data processing (those of mathematical biology) versus the experiment-data-theory-model-validation cycle Existing approaches to the use of big data in the healthcare industry, such as through IBM’s Watson Oncology, Microsoft’s Hanover, and Google’s DeepMind Difficulties in the field that have arisen as a result of technological changes, and potential future directions these changes may take A timely and up-to-date resource on the integration of artificial intelligence in medicine and biology, Big Data Analysis and Artificial Intelligence for Medical Sciences is of great benefit not only to professional scholars, but also MSc or PhD program students eager to explore advancement in the field.




Spectral Learning on Matrices and Tensors


Book Description

This book provides a theoretical and practical introduction to designing and deploying spectral learning on both matrices and tensors. It is of interest for all students, researchers and practitioners working on modern day machine learning problems.