Markov Processes from K. Itô's Perspective


Book Description

Kiyosi Itô's greatest contribution to probability theory may be his introduction of stochastic differential equations to explain the Kolmogorov-Feller theory of Markov processes. Starting with the geometric ideas that guided him, this book gives an account of Itô's program. The modern theory of Markov processes was initiated by A. N. Kolmogorov. However, Kolmogorov's approach was too analytic to reveal the probabilistic foundations on which it rests. In particular, it hides the central role played by the simplest Markov processes: those with independent, identically distributed increments. To remedy this defect, Itô interpreted Kolmogorov's famous forward equation as an equation that describes the integral curve of a vector field on the space of probability measures. Thus, in order to show how Itô's thinking leads to his theory of stochastic integral equations, Stroock begins with an account of integral curves on the space of probability measures and then arrives at stochastic integral equations when he moves to a pathspace setting. In the first half of the book, everything is done in the context of general independent increment processes and without explicit use of Itô's stochastic integral calculus. In the second half, the author provides a systematic development of Itô's theory of stochastic integration: first for Brownian motion and then for continuous martingales. The final chapter presents Stratonovich's variation on Itô's theme and ends with an application to the characterization of the paths on which a diffusion is supported. The book should be accessible to readers who have mastered the essentials of modern probability theory and should provide such readers with a reasonably thorough introduction to continuous-time, stochastic processes.




Labelled Markov Processes


Book Description

Labelled Markov processes are probabilistic versions of labelled transition systems with continuous state spaces. The book covers basic probability and measure theory on continuous state spaces and then develops the theory of LMPs.




Introduction to Stochastic Models


Book Description

Newly revised by the author, this undergraduate-level text introduces the mathematical theory of probability and stochastic processes. Using both computer simulations and mathematical models of random events, it comprises numerous applications to the physical and biological sciences, engineering, and computer science. Subjects include sample spaces, probabilities distributions and expectations of random variables, conditional expectations, Markov chains, and the Poisson process. Additional topics encompass continuous-time stochastic processes, birth and death processes, steady-state probabilities, general queuing systems, and renewal processes. Each section features worked examples, and exercises appear at the end of each chapter, with numerical solutions at the back of the book. Suggestions for further reading in stochastic processes, simulation, and various applications also appear at the end.




Markov Chains and Stochastic Stability


Book Description

New up-to-date edition of this influential classic on Markov chains in general state spaces. Proofs are rigorous and concise, the range of applications is broad and knowledgeable, and key ideas are accessible to practitioners with limited mathematical background. New commentary by Sean Meyn, including updated references, reflects developments since 1996.




Nonlinear Markov Processes and Kinetic Equations


Book Description

A nonlinear Markov evolution is a dynamical system generated by a measure-valued ordinary differential equation with the specific feature of preserving positivity. This feature distinguishes it from general vector-valued differential equations and yields a natural link with probability, both in interpreting results and in the tools of analysis. This brilliant book, the first devoted to the area, develops this interplay between probability and analysis. After systematically presenting both analytic and probabilistic techniques, the author uses probability to obtain deeper insight into nonlinear dynamics, and analysis to tackle difficult problems in the description of random and chaotic behavior. The book addresses the most fundamental questions in the theory of nonlinear Markov processes: existence, uniqueness, constructions, approximation schemes, regularity, law of large numbers and probabilistic interpretations. Its careful exposition makes the book accessible to researchers and graduate students in stochastic and functional analysis with applications to mathematical physics and systems biology.




Continuous Time Markov Processes


Book Description

Markov processes are among the most important stochastic processes for both theory and applications. This book develops the general theory of these processes, and applies this theory to various special examples.




Perspectives of Systems Informatics


Book Description

This book contains thoroughly refereed and revised papers from the 7th International Andrei Ershov Memorial Conference on Perspectives of System Informatics, PSI 2009, held in Akademgorodok, Novosibirsk, Russia, in June 2009. The 26 revised full papers and 4 revised short papers presented were carefully reviewed and selected from 67 submissions. The volume also contains 5 invited papers covering a range of hot topics in system informatics. The papers address all current aspects of theoretical computer science, programming methodology, and new information technologies, which are among the most important contributions of system informatics.




Essentials of Stochastic Processes


Book Description

Building upon the previous editions, this textbook is a first course in stochastic processes taken by undergraduate and graduate students (MS and PhD students from math, statistics, economics, computer science, engineering, and finance departments) who have had a course in probability theory. It covers Markov chains in discrete and continuous time, Poisson processes, renewal processes, martingales, and option pricing. One can only learn a subject by seeing it in action, so there are a large number of examples and more than 300 carefully chosen exercises to deepen the reader’s understanding. Drawing from teaching experience and student feedback, there are many new examples and problems with solutions that use TI-83 to eliminate the tedious details of solving linear equations by hand, and the collection of exercises is much improved, with many more biological examples. Originally included in previous editions, material too advanced for this first course in stochastic processes has been eliminated while treatment of other topics useful for applications has been expanded. In addition, the ordering of topics has been improved; for example, the difficult subject of martingales is delayed until its usefulness can be applied in the treatment of mathematical finance.




Pseudo Differential Operators & Markov Processes: Markov processes and applications


Book Description

This work covers two topics in detail: Fourier analysis, with emphasis on positivity and also on some function spaces and multiplier theorems; and one-parameter operator semigroups with emphasis on Feller semigroups and Lp-sub-Markovian semigroups. In addition, Dirichlet forms are treated.




IoT and WSN based Smart Cities: A Machine Learning Perspective


Book Description

This book provides an investigative approach to how machine learning is helping to maintain and secure smart cities, including principal uses such as smart monitoring, privacy, reliability, and public protection. The authors cover important areas and issues around implementation roadblocks, ideas, and opportunities in smart city development. The authors also include new algorithms, architectures and platforms that can accelerate the growth of smart city concepts and applications. Moreover, this book provides details on specific applications and case studies related to smart city infrastructures, big data management, and prediction techniques using machine learning.