Vivek S. Borkar Libri

This simple, compact toolkit for designing and analyzing stochastic approximation algorithms requires only a basic understanding of probability and differential equations. Although powerful, these algorithms have applications in control and communications engineering, artificial intelligence and economic modeling. Unique topics include finite-time behavior, multiple timescales and asynchronous implementation. There is a useful plethora of applications, each with concrete examples from engineering and economics. Notably it covers variants of stochastic gradient-based optimization schemes, fixed-point solvers, which are commonplace in learning algorithms for approximate dynamic programming, and some models of collective behavior.

Focusing on stochastic approximation algorithms, this advanced text offers a thorough exploration through the lens of ordinary differential equations (ODE). The second edition enhances its coverage of classical convergence analysis while incorporating recent advancements like concentration bounds and stability tests. It also addresses distributed and asynchronous schemes, multiple time scales, and general noise models. With a category-wise exposition of significant applications, it serves as an essential resource for graduate students and professionals in probability, statistics, engineering, economics, and machine learning.

Focusing on stochastic approximation algorithms, this advanced text offers a thorough exploration using the ordinary differential equation (ODE) approach. The second edition enhances the classical convergence analysis and incorporates recent advancements like concentration bounds, stability tests, and distributed schemes. It also addresses multiple time scales and general noise models, making it a valuable resource for graduate students in various disciplines as well as researchers and practitioners seeking a comprehensive reference on the subject.

This book presents a selection of topics from probability theory. Essentially, the topics chosen are those that are likely to be the most useful to someone planning to pursue research in the modern theory of stochastic processes. The prospective reader is assumed to have good mathematical maturity. In particular, he should have prior exposure to basic probability theory at the level of, say, K.L. Chung's 'Elementary probability theory with stochastic processes' (Springer-Verlag, 1974) and real and functional analysis at the level of Royden's 'Real analysis' (Macmillan, 1968). The first chapter is a rapid overview of the basics. Each subsequent chapter deals with a separate topic in detail. There is clearly some selection involved and therefore many omissions, but that cannot be helped in a book of this size. The style is deliberately terse to enforce active learning. Thus several tidbits of deduction are left to the reader as labelled exercises in the main text of each chapter. In addition, there are supplementary exercises at the end. In the preface to his classic text on probability ('Probability', Addison Wesley, 1968), Leo Breiman speaks of the right and left hands of probability.