Output Feedback Reinforcement Learning Control for Linear Systems


Book Description

This monograph explores the analysis and design of model-free optimal control systems based on reinforcement learning (RL) theory, presenting new methods that overcome recent challenges faced by RL. New developments in the design of sensor data efficient RL algorithms are demonstrated that not only reduce the requirement of sensors by means of output feedback, but also ensure optimality and stability guarantees. A variety of practical challenges are considered, including disturbance rejection, control constraints, and communication delays. Ideas from game theory are incorporated to solve output feedback disturbance rejection problems, and the concepts of low gain feedback control are employed to develop RL controllers that achieve global stability under control constraints. Output Feedback Reinforcement Learning Control for Linear Systems will be a valuable reference for graduate students, control theorists working on optimal control systems, engineers, and applied mathematicians.




Parallel Algorithms for Optimal Control of Large Scale Linear Systems


Book Description

Parallel Algorithms for Optimal Control of Large Scale Linear Systems is a comprehensive presentation for both linear and bilinear systems. The parallel algorithms presented in this book are applicable to a wider class of practical systems than those served by traditional methods for large scale singularly perturbed and weakly coupled systems based on the power-series expansion methods. It is intended for scientists and advance graduate students in electrical engineering and computer science who deal with parallel algorithms and control systems, especially large scale systems. The material presented is both comprehensive and unique.




Optimal Control


Book Description

Unique in scope, Optimal Control: Weakly Coupled Systems and Applications provides complete coverage of modern linear, bilinear, and nonlinear optimal control algorithms for both continuous-time and discrete-time weakly coupled systems, using deterministic as well as stochastic formulations. This book presents numerous applications to real world systems from various industries, including aerospace, and discusses the design of subsystem-level optimal filters. Organized into independent chapters for easy access to the material, this text also contains several case studies, examples, exercises, computer assignments, and formulations of research problems to help instructors and students.




Stability of Time-Delay Systems


Book Description

This book is a self-contained presentation of the background and progress of the study of time-delay systems, a subject with broad applications to a number of areas.




Feedback Control Theory


Book Description

An excellent introduction to feedback control system design, this book offers a theoretical approach that captures the essential issues and can be applied to a wide range of practical problems. Its explorations of recent developments in the field emphasize the relationship of new procedures to classical control theory, with a focus on single input and output systems that keeps concepts accessible to students with limited backgrounds. The text is geared toward a single-semester senior course or a graduate-level class for students of electrical engineering. The opening chapters constitute a basic treatment of feedback design. Topics include a detailed formulation of the control design program, the fundamental issue of performance/stability robustness tradeoff, and the graphical design technique of loopshaping. Subsequent chapters extend the discussion of the loopshaping technique and connect it with notions of optimality. Concluding chapters examine controller design via optimization, offering a mathematical approach that is useful for multivariable systems.




Control and Dynamic Systems V16


Book Description

Control and Dynamic Systems: Advances in Theory and Application, Volume 16 is concerned with applied dynamic systems control techniques. It describes various techniques for system modeling, which apply to several systems issues. This book presents a comprehensive treatment of powerful algorithmic techniques for solving dynamic-system optimization problems. It also describes approaches for systems model that apply to system issues such as time delays. The remaining chapters of this book explore the simulation of large closed-loop systems and optimization of low-order feedback controllers for discrete-time systems. Researchers who wish to broaden their understanding of dynamic systems control techniques will find this book invaluable.




Optimal Control


Book Description

This new, updated edition of Optimal Control reflects major changes that have occurred in the field in recent years and presents, in a clear and direct way, the fundamentals of optimal control theory. It covers the major topics involving measurement, principles of optimality, dynamic programming, variational methods, Kalman filtering, and other solution techniques. To give the reader a sense of the problems that can arise in a hands-on project, the authors have included new material on optimal output feedback control, a technique used in the aerospace industry. Also included are two new chapters on robust control to provide background in this rapidly growing area of interest. Relations to classical control theory are emphasized throughout the text, and a root-locus approach to steady-state controller design is included. A chapter on optimal control of polynomial systems is designed to give the reader sufficient background for further study in the field of adaptive control. The authors demonstrate through numerous examples that computer simulations of optimal controllers are easy to implement and help give the reader an intuitive feel for the equations. To help build the reader's confidence in understanding the theory and its practical applications, the authors have provided many opportunities throughout the book for writing simple programs. Optimal Control will also serve as an invaluable reference for control engineers in the industry. It offers numerous tables that make it easy to find the equations needed to implement optimal controllers for practical applications. All simulations have been performed using MATLAB and relevant Toolboxes. Optimal Control assumes a background in the state-variable representation of systems; because matrix manipulations are the basic mathematical vehicle of the book, a short review is included in the appendix. A lucid introductory text and an invaluable reference, Optimal Control will serve as a complete tool for the professional engineer and advanced student alike. As a superb introductory text and an indispensable reference, this new edition of Optimal Control will serve the needs of both the professional engineer and the advanced student in mechanical, electrical, and aerospace engineering. Its coverage encompasses all the fundamental topics as well as the major changes of recent years, including output-feedback design and robust design. An abundance of computer simulations using MATLAB and relevant Toolboxes is included to give the reader the actual experience of applying the theory to real-world situations. Major topics covered include: Static Optimization Optimal Control of Discrete-Time Systems Optimal Control of Continuous-Time Systems The Tracking Problem and Other LQR Extensions Final-Time-Free and Constrained Input Control Dynamic Programming Optimal Control for Polynomial Systems Output Feedback and Structured Control Robustness and Multivariable Frequency-Domain Techniques




Robust Control System Design


Book Description

Robust Control System Design: Advanced State Space Techniques, Second Edition expands upon a groundbreaking and combinatorial approach to state space control system design that fully realizes the critical loop transfer function and robustness properties of state/generalized state feedback control. This edition offers many new examples and exercises to illustrate and clarify new design concepts, approaches, and procedures while highlighting the fact that state/generalized state feedback control can improve system performance and robustness more effectively than other forms of control. Revised and expanded throughout, the second edition presents an improved eigenstructure assignment design method that enhances system performance and robustness more directly and effectively and allows for adjustment of design formulations based on design testing and simulation. The author proposes the systematic controller order adjustment for the tradeoff between performance and robustness based on the complete unification of the state feedback control and static output feedback control. The book also utilizes a more accurate robust stability measure to guide control designs.




Optimization Techniques And Applications: International Conference (In 2 Volumes)


Book Description

With the advent of powerful computers and novel mathematical programming techniques, the multidisciplinary field of optimization has advanced to the stage that quite complicated systems can be addressed. The conference was organized to provide a platform for the exchanging of new ideas and information and for identifying areas for future research. The contributions covered both theoretical techniques and a rich variety of case studies to which optimization can be usefully applied.




Investigation and Appreciation of Optimal Output Feedback. Volume 1: A Convergent Algorithm for the Stochastic Infinite-time Discrete Optimal Output Feedback Problem


Book Description

The stochastic, infinite time, discrete output feedback problem for time invariant linear systems is examined. Two sets of sufficient conditions for the existence of a stable, globally optimal solution are presented. An expression for the total change in the cost function due to a change in the feedback gain is obtained. This expression is used to show that a sequence of gains can be obtained by an algorithm, so that the corresponding cost sequence is monotonically decreasing and the corresponding sequence of the cost gradient converges to zero. The algorithm is guaranteed to obtain a critical point of the cost function. The computational steps necessary to implement the algorithm on a computer are presented. The results are applied to a digital outer loop flight control problem. The numerical results for this 13th order problem indicate a rate of convergence considerably faster than two other algorithms used for comparison.