Reinforcement Learning, second edition


Book Description

The significantly expanded and updated new edition of a widely used text on reinforcement learning, one of the most active research areas in artificial intelligence. Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives while interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the field's key ideas and algorithms. This second edition has been significantly expanded and updated, presenting new topics and updating coverage of other topics. Like the first edition, this second edition focuses on core online learning algorithms, with the more mathematical material set off in shaded boxes. Part I covers as much of reinforcement learning as possible without going beyond the tabular case for which exact solutions can be found. Many algorithms presented in this part are new to the second edition, including UCB, Expected Sarsa, and Double Learning. Part II extends these ideas to function approximation, with new sections on such topics as artificial neural networks and the Fourier basis, and offers expanded treatment of off-policy learning and policy-gradient methods. Part III has new chapters on reinforcement learning's relationships to psychology and neuroscience, as well as an updated case-studies chapter including AlphaGo and AlphaGo Zero, Atari game playing, and IBM Watson's wagering strategy. The final chapter discusses the future societal impacts of reinforcement learning.




Hands-On Reinforcement Learning for Games


Book Description

Explore reinforcement learning (RL) techniques to build cutting-edge games using Python libraries such as PyTorch, OpenAI Gym, and TensorFlow Key FeaturesGet to grips with the different reinforcement and DRL algorithms for game developmentLearn how to implement components such as artificial agents, map and level generation, and audio generationGain insights into cutting-edge RL research and understand how it is similar to artificial general researchBook Description With the increased presence of AI in the gaming industry, developers are challenged to create highly responsive and adaptive games by integrating artificial intelligence into their projects. This book is your guide to learning how various reinforcement learning techniques and algorithms play an important role in game development with Python. Starting with the basics, this book will help you build a strong foundation in reinforcement learning for game development. Each chapter will assist you in implementing different reinforcement learning techniques, such as Markov decision processes (MDPs), Q-learning, actor-critic methods, SARSA, and deterministic policy gradient algorithms, to build logical self-learning agents. Learning these techniques will enhance your game development skills and add a variety of features to improve your game agent’s productivity. As you advance, you’ll understand how deep reinforcement learning (DRL) techniques can be used to devise strategies to help agents learn from their actions and build engaging games. By the end of this book, you’ll be ready to apply reinforcement learning techniques to build a variety of projects and contribute to open source applications. What you will learnUnderstand how deep learning can be integrated into an RL agentExplore basic to advanced algorithms commonly used in game developmentBuild agents that can learn and solve problems in all types of environmentsTrain a Deep Q-Network (DQN) agent to solve the CartPole balancing problemDevelop game AI agents by understanding the mechanism behind complex AIIntegrate all the concepts learned into new projects or gaming agentsWho this book is for If you’re a game developer looking to implement AI techniques to build next-generation games from scratch, this book is for you. Machine learning and deep learning practitioners, and RL researchers who want to understand how to use self-learning agents in the game domain will also find this book useful. Knowledge of game development and Python programming experience are required.




Cognitive Analytics and Reinforcement Learning


Book Description

COGNITIVE ANALYTICS AND REINFORCEMENT LEARNING The combination of cognitive analytics and reinforcement learning is a transformational force in the field of modern technological breakthroughs, reshaping the decision-making, problem-solving, and innovation landscape; this book offers an examination of the profound overlap between these two fields and illuminates its significant consequences for business, academia, and research. Cognitive analytics and reinforcement learning are pivotal branches of artificial intelligence. They have garnered increased attention in the research field and industry domain on how humans perceive, interpret, and respond to information. Cognitive science allows us to understand data, mimic human cognitive processes, and make informed decisions to identify patterns and adapt to dynamic situations. The process enhances the capabilities of various applications. Readers will uncover the latest advancements in AI and machine learning, gaining valuable insights into how these technologies are revolutionizing various industries, including transforming healthcare by enabling smarter diagnosis and treatment decisions, enhancing the efficiency of smart cities through dynamic decision control, optimizing debt collection strategies, predicting optimal moves in complex scenarios like chess, and much more. With a focus on bridging the gap between theory and practice, this book serves as an invaluable resource for researchers and industry professionals seeking to leverage cognitive analytics and reinforcement learning to drive innovation and solve complex problems. The book’s real strength lies in bridging the gap between theoretical knowledge and practical implementation. It offers a rich tapestry of use cases and examples. Whether you are a student looking to gain a deeper understanding of these cutting-edge technologies, an AI practitioner seeking innovative solutions for your projects, or an industry leader interested in the strategic applications of AI, this book offers a treasure trove of insights and knowledge to help you navigate the complex and exciting world of cognitive analytics and reinforcement learning. Audience The book caters to a diverse audience that spans academic researchers, AI practitioners, data scientists, industry leaders, tech enthusiasts, and educators who associate with artificial intelligence, data analytics, and cognitive sciences.




Deep Reinforcement Learning Hands-On


Book Description

This practical guide will teach you how deep learning (DL) can be used to solve complex real-world problems. Key Features Explore deep reinforcement learning (RL), from the first principles to the latest algorithms Evaluate high-profile RL methods, including value iteration, deep Q-networks, policy gradients, TRPO, PPO, DDPG, D4PG, evolution strategies and genetic algorithms Keep up with the very latest industry developments, including AI-driven chatbots Book Description Recent developments in reinforcement learning (RL), combined with deep learning (DL), have seen unprecedented progress made towards training agents to solve complex problems in a human-like way. Google’s use of algorithms to play and defeat the well-known Atari arcade games has propelled the field to prominence, and researchers are generating new ideas at a rapid pace. Deep Reinforcement Learning Hands-On is a comprehensive guide to the very latest DL tools and their limitations. You will evaluate methods including Cross-entropy and policy gradients, before applying them to real-world environments. Take on both the Atari set of virtual games and family favorites such as Connect4. The book provides an introduction to the basics of RL, giving you the know-how to code intelligent learning agents to take on a formidable array of practical tasks. Discover how to implement Q-learning on ‘grid world’ environments, teach your agent to buy and trade stocks, and find out how natural language models are driving the boom in chatbots. What you will learn Understand the DL context of RL and implement complex DL models Learn the foundation of RL: Markov decision processes Evaluate RL methods including Cross-entropy, DQN, Actor-Critic, TRPO, PPO, DDPG, D4PG and others Discover how to deal with discrete and continuous action spaces in various environments Defeat Atari arcade games using the value iteration method Create your own OpenAI Gym environment to train a stock trading agent Teach your agent to play Connect4 using AlphaGo Zero Explore the very latest deep RL research on topics including AI-driven chatbots Who this book is for Some fluency in Python is assumed. Basic deep learning (DL) approaches should be familiar to readers and some practical experience in DL will be helpful. This book is an introduction to deep reinforcement learning (RL) and requires no background in RL.




Deep Reinforcement Learning in Action


Book Description

Summary Humans learn best from feedback—we are encouraged to take actions that lead to positive results while deterred by decisions with negative consequences. This reinforcement process can be applied to computer programs allowing them to solve more complex problems that classical programming cannot. Deep Reinforcement Learning in Action teaches you the fundamental concepts and terminology of deep reinforcement learning, along with the practical skills and techniques you’ll need to implement it into your own projects. Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications. About the technology Deep reinforcement learning AI systems rapidly adapt to new environments, a vast improvement over standard neural networks. A DRL agent learns like people do, taking in raw data such as sensor input and refining its responses and predictions through trial and error. About the book Deep Reinforcement Learning in Action teaches you how to program AI agents that adapt and improve based on direct feedback from their environment. In this example-rich tutorial, you’ll master foundational and advanced DRL techniques by taking on interesting challenges like navigating a maze and playing video games. Along the way, you’ll work with core algorithms, including deep Q-networks and policy gradients, along with industry-standard tools like PyTorch and OpenAI Gym. What's inside Building and training DRL networks The most popular DRL algorithms for learning and problem solving Evolutionary algorithms for curiosity and multi-agent learning All examples available as Jupyter Notebooks About the reader For readers with intermediate skills in Python and deep learning. About the author Alexander Zai is a machine learning engineer at Amazon AI. Brandon Brown is a machine learning and data analysis blogger. Table of Contents PART 1 - FOUNDATIONS 1. What is reinforcement learning? 2. Modeling reinforcement learning problems: Markov decision processes 3. Predicting the best states and actions: Deep Q-networks 4. Learning to pick the best policy: Policy gradient methods 5. Tackling more complex problems with actor-critic methods PART 2 - ABOVE AND BEYOND 6. Alternative optimization methods: Evolutionary algorithms 7. Distributional DQN: Getting the full story 8.Curiosity-driven exploration 9. Multi-agent reinforcement learning 10. Interpretable reinforcement learning: Attention and relational models 11. In conclusion: A review and roadmap




Deep Reinforcement Learning


Book Description

This book starts by presenting the basics of reinforcement learning using highly intuitive and easy-to-understand examples and applications, and then introduces the cutting-edge research advances that make reinforcement learning capable of out-performing most state-of-art systems, and even humans in a number of applications. The book not only equips readers with an understanding of multiple advanced and innovative algorithms, but also prepares them to implement systems such as those created by Google Deep Mind in actual code. This book is intended for readers who want to both understand and apply advanced concepts in a field that combines the best of two worlds – deep learning and reinforcement learning – to tap the potential of ‘advanced artificial intelligence’ for creating real-world applications and game-winning algorithms.




Advancing Our Understanding of Deep Reinforcement Learning with Community-driven Insights


Book Description

Simulated environments have been essential to advancing the field of artificial intelligence, providing vast amounts of synthetic data that tests novel approaches safely and efficiently. This has most often taken the form of games, ranging from simple board games to modern multiplayer strategy games. These games served as a good starting point, but Danny Lange (Unity Technologies) reveals an opportunity to push the state of the art in AI research to the next level. United introduced the Obstacle Tower, a high-visual-fidelity, 3-D, third-person, procedurally generated game environment purpose built to test a deep reinforcement learning-trained agent's vision, control, planning, and generalization abilities. Over the past year, Unity invited researchers and developers to try to solve the tower with the intention of sharing those insights with the broader community. Prerequisite knowledge A basic knowledge of machine learning and AI What you'll learn See how you can use what Unity learned from hosting the challenges to engage the broader community to advance AI research Find out how participants fared as they attempted to solve the tower, what that taught Unity, and what's next for Obstacle Tower as it continues pushing advances in deep reinforcement learning Learn how observing emergent behaviors of multiple AI agents in a simulated virtual environment can lead to the most optimal designs and real-world practices This session is from the 2019 O'Reilly Artificial Intelligence Conference in San Jose, CA.




Deep Reinforcement Learning Hands-On


Book Description

New edition of the bestselling guide to deep reinforcement learning and how it's used to solve complex real-world problems. Revised and expanded to include multi-agent methods, discrete optimization, RL in robotics, advanced exploration techniques, and more Key Features Second edition of the bestselling introduction to deep reinforcement learning, expanded with six new chapters Learn advanced exploration techniques including noisy networks, pseudo-count, and network distillation methods Apply RL methods to cheap hardware robotics platforms Book Description Deep Reinforcement Learning Hands-On, Second Edition is an updated and expanded version of the bestselling guide to the very latest reinforcement learning (RL) tools and techniques. It provides you with an introduction to the fundamentals of RL, along with the hands-on ability to code intelligent learning agents to perform a range of practical tasks. With six new chapters devoted to a variety of up-to-the-minute developments in RL, including discrete optimization (solving the Rubik's Cube), multi-agent methods, Microsoft's TextWorld environment, advanced exploration techniques, and more, you will come away from this book with a deep understanding of the latest innovations in this emerging field. In addition, you will gain actionable insights into such topic areas as deep Q-networks, policy gradient methods, continuous control problems, and highly scalable, non-gradient methods. You will also discover how to build a real hardware robot trained with RL for less than $100 and solve the Pong environment in just 30 minutes of training using step-by-step code optimization. In short, Deep Reinforcement Learning Hands-On, Second Edition, is your companion to navigating the exciting complexities of RL as it helps you attain experience and knowledge through real-world examples. What you will learn Understand the deep learning context of RL and implement complex deep learning models Evaluate RL methods including cross-entropy, DQN, actor-critic, TRPO, PPO, DDPG, D4PG, and others Build a practical hardware robot trained with RL methods for less than $100 Discover Microsoft's TextWorld environment, which is an interactive fiction games platform Use discrete optimization in RL to solve a Rubik's Cube Teach your agent to play Connect 4 using AlphaGo Zero Explore the very latest deep RL research on topics including AI chatbots Discover advanced exploration techniques, including noisy networks and network distillation techniques Who this book is for Some fluency in Python is assumed. Sound understanding of the fundamentals of deep learning will be helpful. This book is an introduction to deep RL and requires no background in RL