As artificial intelligence (AI) is applied to more complex problems and a wider set of applications, the ability to take advantage of the computational power of distributed and parallel hardware architectures and to match these architec­ tures with the inherent distributed aspects of applications (spatial, functional, or temporal) has become an important research issue. Out of these research concerns, an AI subdiscipline called distributed problem solving has emerged. Distributed problem-solving systems are broadly defined as loosely-coupled, distributed networks of semi-autonomous problem-solving agents that perform sophisticated problem solving and cooperatively interact to solve problems. N odes operate asynchronously and in parallel with limited internode commu­ nication. Limited internode communication stems from either inherent band­ width limitations of the communication medium or from the high computa­ tional cost of packaging and assimilating information to be sent and received among agents. Structuring network problem solving to deal with consequences oflimited communication-the lack of a global view and the possibility that the individual agents may not have all the information necessary to accurately and completely solve their subproblems-is one of the major focuses of distributed problem-solving research. It is this focus that also is one of the important dis­ tinguishing characteristics of distributed problem-solving research that sets it apart from previous research in AI.

The word stochastic is jargon for random. A stochastic process is a system which evolves in time while undergoing chance fluctuations. We can describe such a system by defining a family of random variables, {Xt}' where Xt measures, at time t, the aspect of the system which is of interest. For example, X t might be the number of customers in a queue at time t. As time passes, customers will arrive and leave, and so the value of Xt will change. At any time t, Xt takes one of the values 0, 1,2, … ; and t can be any value in a subset of ( - 00, 00), the infinite past to the infinite future. If we observe the queue continuously, and customers arrive one at a time to be served by a single server, then, when a customer arrives, the value of Xt' the queue size, increases by one, and when a customer departs after being served, X t decreases by one (Figure 1.1).