Computational Learning Theory and Natural Learning Systems: Intersections between theory and experiment


Book Description

Annotation These original contributions converge on an exciting and fruitful intersection of three historically distinct areas of learning research: computational learning theory, neural networks, and symbolic machine learning. Bridging theory and practice, computer science and psychology, they consider general issues in learning systems that could provide constraints for theory and at the same time interpret theoretical results in the context of experiments with actual learning systems. In all, nineteen chapters address questions such as, What is a natural system? How should learning systems gain from prior knowledge? If prior knowledge is important, how can we quantify how important? What makes a learning problem hard? How are neural networks and symbolic machine learning approaches similar? Is there a fundamental difference in the kind of task a neural network can easily solve as opposed to those a symbolic algorithm can easily solve? Stephen J. Hanson heads the Learning Systems Department at Siemens Corporate Research and is a Visiting Member of the Research Staff and Research Collaborator at the Cognitive Science Laboratory at Princeton University. George A. Drastal is Senior Research Scientist at Siemens Corporate Research. Ronald J. Rivest is Professor of Computer Science and Associate Director of the Laboratory for Computer Science at the Massachusetts Institute of Technology.




Computational Learning Theory and Natural Learning Systems: Making learning systems practical


Book Description

This is the fourth and final volume of papers from a series of workshops called "Computational Learning Theory and Ǹatural' Learning Systems." The purpose of the workshops was to explore the emerging intersection of theoretical learning research and natural learning systems. The workshops drew researchers from three historically distinct styles of learning research: computational learning theory, neural networks, and machine learning (a subfield of AI). Volume I of the series introduces the general focus of the workshops. Volume II looks at specific areas of interaction between theory and experiment. Volumes III and IV focus on key areas of learning systems that have developed recently. Volume III looks at the problem of "Selecting Good Models." The present volume, Volume IV, looks at ways of "Making Learning Systems Practical." The editors divide the twenty-one contributions into four sections. The first three cover critical problem areas: 1) scaling up from small problems to realistic ones with large input dimensions, 2) increasing efficiency and robustness of learning methods, and 3) developing strategies to obtain good generalization from limited or small data samples. The fourth section discusses examples of real-world learning systems. Contributors : Klaus Abraham-Fuchs, Yasuhiro Akiba, Hussein Almuallim, Arunava Banerjee, Sanjay Bhansali, Alvis Brazma, Gustavo Deco, David Garvin, Zoubin Ghahramani, Mostefa Golea, Russell Greiner, Mehdi T. Harandi, John G. Harris, Haym Hirsh, Michael I. Jordan, Shigeo Kaneda, Marjorie Klenin, Pat Langley, Yong Liu, Patrick M. Murphy, Ralph Neuneier, E.M. Oblow, Dragan Obradovic, Michael J. Pazzani, Barak A. Pearlmutter, Nageswara S.V. Rao, Peter Rayner, Stephanie Sage, Martin F. Schlang, Bernd Schurmann, Dale Schuurmans, Leon Shklar, V. Sundareswaran, Geoffrey Towell, Johann Uebler, Lucia M. Vaina, Takefumi Yamazaki, Anthony M. Zador.




Systems that Learn


Book Description

This introduction to the concepts and techniques of formal learning theory is based on a number-theoretical approach to learning and uses the tools of recursive function theory to understand how learners come to an accurate view of reality.




Boosting


Book Description

An accessible introduction and essential reference for an approach to machine learning that creates highly accurate prediction rules by combining many weak and inaccurate ones. Boosting is an approach to machine learning based on the idea of creating a highly accurate predictor by combining many weak and inaccurate “rules of thumb.” A remarkably rich theory has evolved around boosting, with connections to a range of topics, including statistics, game theory, convex optimization, and information geometry. Boosting algorithms have also enjoyed practical success in such fields as biology, vision, and speech processing. At various times in its history, boosting has been perceived as mysterious, controversial, even paradoxical. This book, written by the inventors of the method, brings together, organizes, simplifies, and substantially extends two decades of research on boosting, presenting both theory and applications in a way that is accessible to readers from diverse backgrounds while also providing an authoritative reference for advanced researchers. With its introductory treatment of all material and its inclusion of exercises in every chapter, the book is appropriate for course use as well. The book begins with a general introduction to machine learning algorithms and their analysis; then explores the core theory of boosting, especially its ability to generalize; examines some of the myriad other theoretical viewpoints that help to explain and understand boosting; provides practical extensions of boosting for more complex learning problems; and finally presents a number of advanced theoretical topics. Numerous applications and practical illustrations are offered throughout.




Computational Learning Theory and Natural Learning Systems


Book Description

As with Volume I, this second volume represents a synthesis of issues in three historically distinct areas of learning research: computational learning theory, neural network research, and symbolic machine learning. While the first volume provided a forum for building a science of computational learning across fields, this volume attempts to define plausible areas of joint research: the contributions are concerned with finding constraints for theory while at the same time interpreting theoretic results in the context of experiments with actual learning systems. Subsequent volumes will focus on areas identified as research opportunities.Computational learning theory, neural networks, and AI machine learning appear to be disparate fields; in fact they have the same goal: to build a machine or program that can learn from its environment. Accordingly, many of the papers in this volume deal with the problem of learning from examples. In particular, they are intended to encourage discussion between those trying to build learning algorithms (for instance, algorithms addressed by learning theoretic analyses are quite different from those used by neural network or machine-learning researchers) and those trying to analyze them.The first section provides theoretical explanations for the learning systems addressed, the second section focuses on issues in model selection and inductive bias, the third section presents new learning algorithms, the fourth section explores the dynamics of learning in feedforward neural networks, and the final section focuses on the application of learning algorithms.A Bradford Book




Learning Theory and Kernel Machines


Book Description

This book constitutes the joint refereed proceedings of the 16th Annual Conference on Computational Learning Theory, COLT 2003, and the 7th Kernel Workshop, Kernel 2003, held in Washington, DC in August 2003. The 47 revised full papers presented together with 5 invited contributions and 8 open problem statements were carefully reviewed and selected from 92 submissions. The papers are organized in topical sections on kernel machines, statistical learning theory, online learning, other approaches, and inductive inference learning.




Computational Learning Theory and Natural Learning Systems: Selecting good models


Book Description

Volume I of the series introduces the general focus of the workshops. Volume II looks at specific areas of interaction between theory and experiment. Volumes III and IV focus on key areas of learning systems that have developed recently. Volume III looks at the problem of "Selecting Good Models." The present volume, Volume IV, looks at ways of "Making Learning Systems Practical." The editors divide the twenty-one contributions into four sections. The first three cover critical problem areas: 1) scaling up from small problems to realistic ones with large input dimensions, 2) increasing efficiency and robustness of learning methods, and 3) developing strategies to obtain good generalization from limited or small data samples. The fourth section discusses examples of real-world learning systems.




The Nature of Statistical Learning Theory


Book Description

The aim of this book is to discuss the fundamental ideas which lie behind the statistical theory of learning and generalization. It considers learning as a general problem of function estimation based on empirical data. Omitting proofs and technical details, the author concentrates on discussing the main results of learning theory and their connections to fundamental problems in statistics. This second edition contains three new chapters devoted to further development of the learning theory and SVM techniques. Written in a readable and concise style, the book is intended for statisticians, mathematicians, physicists, and computer scientists.




Probably Approximately Correct


Book Description

Presenting a theory of the theoryless, a computer scientist provides a model of how effective behavior can be learned even in a world as complex as our own, shedding new light on human nature.