Explorations in Parallel Distributed Processing - Macintosh version: A Handbook of Models, Programs, and Exercises

Preface

Uses of This Handbook

We have produced this handbook primarily to help others who wish to undertake their own explorations in parallel distributed processing. The handbook can be used in a self -guided program of exploration. We provide tutorials on the theoretical background of all the models considered, as well as simple exercises to help the reader develop a feeling for the basic concepts of PDP and for the simulation models that embody these principles.
The material covered comes largely from the two vol urnes on parallel dis - tributed processing that grew out of our work with the PDP research group at UCSD (McClelland, Rumelhart, & “the PDP Research group, 1986; Rumelhart, McClelland, & the PDP Research Group, 1986). This represents fairly broad coverage, since many prominent workers in the field contributed to these volumes and many of the models built upon the pioneering work of others.
The range of models covered should be sufficiently broad to let this volume serve as a primer in PDP or connectionist modeling. There are some gaps, but we have found that once students have gained some hands - on experience with a subset of models in the field, there is considerable positive transfer, and they are able to assimilate written material about other models better once they have had this experience.
The handbook should prove suitable for courses that serve as a first introduction to PDP or connectionist modeling. We have used this material successfully in courses for juniors, seniors, and graduate students who have had no prior experience in the area. We hope that the coverage of basic theoretical material provided in each chapter will help to free instructors to devote some lectures and discussions to the ideas and motivations behind the models and to introduce material we have not covered here. We have given answers to the questions posed in the exercises to encourage students to use the questions to test their own understanding of the material and to help anyone who uses the handbook for self-study purposes to stay on track. We have also tried to make it possible for students to explore appplications of the various models to problems they pose for themselves. In some cases, it may even be feasible for students with programming experience to extend the software and explore variants of the models that we have not implemented or even to develop new models of their own. However, our experience has been that most students do not have the time, within a course that lasts a single quarter or semester, to do too much extension of the programs themselves; this is often better left for a follow-up, independent study course in a later term. An additional goal of the handbook is to make our simulation programs available to researchers who already have some specific research interest in PDP. Over the years we have frequently been asked for copies of our programs by researchers who wished to try our simulation models on their own problems or to modify and extend the models. Judging from our own experience, the software provided with this handbook will prove useful in meeting many of these needs; we find that the bp program, in particular, has become a major workhorse that has been adapted by us and many of our graduate students for use in a number of ongoing research projects. We have retained copyright to the text and accompanying software, but encourage the use and modification of the software for educational and research purposes. We reserve all rights to market the text and software, both source and object code.