Learning in Non-Stationary Environments


Book Description

Recent decades have seen rapid advances in automatization processes, supported by modern machines and computers. The result is significant increases in system complexity and state changes, information sources, the need for faster data handling and the integration of environmental influences. Intelligent systems, equipped with a taxonomy of data-driven system identification and machine learning algorithms, can handle these problems partially. Conventional learning algorithms in a batch off-line setting fail whenever dynamic changes of the process appear due to non-stationary environments and external influences. Learning in Non-Stationary Environments: Methods and Applications offers a wide-ranging, comprehensive review of recent developments and important methodologies in the field. The coverage focuses on dynamic learning in unsupervised problems, dynamic learning in supervised classification and dynamic learning in supervised regression problems. A later section is dedicated to applications in which dynamic learning methods serve as keystones for achieving models with high accuracy. Rather than rely on a mathematical theorem/proof style, the editors highlight numerous figures, tables, examples and applications, together with their explanations. This approach offers a useful basis for further investigation and fresh ideas and motivates and inspires newcomers to explore this promising and still emerging field of research.







Machine Learning in Non-Stationary Environments


Book Description

Theory, algorithms, and applications of machine learning techniques to overcome “covariate shift” non-stationarity. As the power of computing has grown over the past few decades, the field of machine learning has advanced rapidly in both theory and practice. Machine learning methods are usually based on the assumption that the data generation mechanism does not change over time. Yet real-world applications of machine learning, including image recognition, natural language processing, speech recognition, robot control, and bioinformatics, often violate this common assumption. Dealing with non-stationarity is one of modern machine learning's greatest challenges. This book focuses on a specific non-stationary environment known as covariate shift, in which the distributions of inputs (queries) change but the conditional distribution of outputs (answers) is unchanged, and presents machine learning theory, algorithms, and applications to overcome this variety of non-stationarity. After reviewing the state-of-the-art research in the field, the authors discuss topics that include learning under covariate shift, model selection, importance estimation, and active learning. They describe such real world applications of covariate shift adaption as brain-computer interface, speaker identification, and age prediction from facial images. With this book, they aim to encourage future research in machine learning, statistics, and engineering that strives to create truly autonomous learning machines able to learn under non-stationarity.




Machine Learning in Non-stationary Environments


Book Description

Dealing with non-stationarity is one of modem machine learning's greatest challenges. This book focuses on a specific non-stationary environment known as covariate shift, in which the distributions of inputs (queries) change but the conditional distribution of outputs (answers) is unchanged, and presents machine learning theory, algorithms, and applications to overcome this variety of non-stationarity.







Adapting Machine Learning to Non-stationary Environments


Book Description

Machine learning stimulates a broad range of computational methods that exploit experience, which typically takes the form of electronic data, to make profitable decisions or accurate predictions. To date, the machine learning models have been applied to extensive application domains across diverse fields, including but not limited to computer vision [1, 2, 3], natural language processing [4, 5, 6], robotic control [7, 8], and cyber security [9, 10, 11].










Reinforcement Learning in Non-stationary Environments


Book Description

How should an agent act in the face of uncertainty on the evolution of its environment?In this dissertation, we give a Reinforcement Learning perspective on the resolution of nonstationaryproblems. The question is seen from three different aspects. First, we study theplanning vs. re-planning trade-off of tree search algorithms in stationary Markov DecisionProcesses. We propose a method to lower the computational requirements of such an algorithmwhile keeping theoretical guarantees on the performance. Secondly, we study thecase of environments evolving gradually over time. This hypothesis is expressed through amathematical framework called Lipschitz Non-Stationary Markov Decision Processes. Wederive a risk averse planning algorithm provably converging to the minimax policy in thissetting. Thirdly, we consider abrupt temporal evolution in the setting of lifelong ReinforcementLearning. We propose a non-negative transfer method based on the theoretical study ofthe optimal Q-function's Lipschitz continuity with respect to the task space. The approachallows to accelerate learning in new tasks. Overall, this dissertation proposes answers to thequestion of solving Non-Stationary Markov Decision Processes under three different settings.




Machine Learning in Non-Stationary Environments


Book Description

Theory, algorithms, and applications of machine learning techniques to overcome "covariate shift" non-stationarity.