Neural Networks and Intellect: Using Model-Based Concepts

Rating:
Summary: mathematical models and philosophy of AI
Review: Artificial intelligence research goes back to the 1940s as do the first developments in electronic computers. Perlovsky traces the history of AI in a thoughtful and scholarly manner, emphasizing his philosophy and his own generalization of the theory which he calls Modeling Field Theory (MFT). He also traces the study of intelligence back to the Greek philosophers beginning with Plato and Aristotle some 2300 years ago.

The book however, provides more than just a philosophy for artificial intelligence. It mixes in some very important mathematics from the disciplines of engineering, statistics and computer science. Mathematical techniques and models have been particularly useful in the solution to problems in classification, clustering, pattern recognition, rule-based expert system development, multiple-target tracking, orbit determination and Kalman filtering, and time series prediction.

Perlovsky, over the course of his career, has had a great deal of involvement in the development of this research in both his consulting work and his work at Nichols Research Corporation. I know a lot about this because, in 1980 I began working at the Aerospace Corporation in El Segundo California as a MTS (statistician). I worked on statistical problems including Kalman filtering, image processing, image recognition, orbit determination, rule-based expert systems, multiple-target tracking and target discrimination. When I moved over to manage the tracking and discrimination algorithm development for the Air Force's Space Surveillance and Tracking System I became familar with the work being done for us by contractors that included Nichols Research Corporation (NRC). I became very familar with the work of Perlovsky and his colleagues at Nichols Research who supported us from the Newport Beach, Colorado Springs and Boston offices of the company on the multiple-target tracking algorithms and the target classification algorithms. I found the work to be so interesting and of such high quality that I joined NRC in 1988.

Perlovsky sees artifical intelligence as a very practical discipline and believes that computing machines can do a good job of at least mimicking human intelligence through the use of a priori knowledge (as rules preprogrammed into the computer or a priori probability distributions) along with experience (collected data from observational or statistical designed studies) combined using algorithms (Bayes theorem, adaptive neural networks) based on the mathematical foundation of uncertainty incorporated through probability theory and/or fuzzy set theory.

In my experience, I have found rule-based expert systems to be one of the major successful developments in the field of artificial intelligence. At the heart of these systems lies the tools of mathematics and statistics, including the discriminant or classification algorithms based on multivariate Gaussian models (linear and quadratic classifiers) and the nonparametric classification algorithms (kernel discriminant algorithms and classification tree algorithms). Also, patterns can be discovered by computers through the use of clustering algorithms based on Gaussian mixture models or nonparametric techniques like nearest neighbor rules.

The Bayesian approach to statistical analysis has been useful in many areas including the Kalman filter. In Kalman filtering prior knowledge plus current data is used to update the estimate of the current state and for the prediction of the future state of a dynamic system using a simple recursive algorithm that is easily updated in the computer. Many of these developments are well characterized and developed from first principles in this text.

Perlovsky emphasizes his own work including the MLANS system which is a neural network system that incorporates important statistical ideas such as maximum likelihood, the Cramer-Rao inequality and statistical efficiency along with the neural network architecture.

Some of this work was developed by Perlovsky under an Army contract that was coincidentally managed by my brother Julian. I have always viewed this research as being successful because it applied appropriate statistical models to the real problems. I think the crucial aspects of this work are the appropriate use of the Bayesian paradigm and teh indentification of appropriate models for construction of the likelihood equations. The fundamental and well established tools of probability and statistics are the keys. In his proposals, Leonid also included ideas from fuzzy set theory and embedded his methods in an artificial neural network framework. I always thought that these modern theories (fuzzy set theory and neural networks) were gimmicks to get military funding. This may not have been a fair assessment on my part as a careful reading of this book indicates that Perlovsky honestly views these tools as important.

There are subtleties to concepts such as fuzzy set theory. Although I do not yet see its value as a substitute for measure theoretic probability theory for characterizing human uncertainty, it is possible that I just haven't thought hard enough about it. Maybe a continued reading and rereading of Perlovsky's book will help me.

This is a very interesting and unique book on artificial intelligence from a perspective that is quite different from what one find in the standard books written by computer scientists (who often do not have the deep understanding of probability and statistics that Perlovsky possesses).

Rating:
Summary: mathematical models and philosophy of AI
Review: Artificial intelligence research goes back to the 1940s as do the first developments in electronic computers. Perlovsky traces the history of AI in a thoughtful and scholarly manner, emphasizing his philosophy and his own generalization of the theory which he calls Modeling Field Theory (MFT). He also traces the study of intelligence back to the Greek philosophers beginning with Plato and Aristotle some 2300 years ago.

The book however, provides more than just a philosophy for artificial intelligence. It mixes in some very important mathematics from the disciplines of engineering, statistics and computer science. Mathematical techniques and models have been particularly useful in the solution to problems in classification, clustering, pattern recognition, rule-based expert system development, multiple-target tracking, orbit determination and Kalman filtering, and time series prediction.

Perlovsky, over the course of his career, has had a great deal of involvement in the development of this research in both his consulting work and his work at Nichols Research Corporation. I know a lot about this because, in 1980 I began working at the Aerospace Corporation in El Segundo California as a MTS (statistician). I worked on statistical problems including Kalman filtering, image processing, image recognition, orbit determination, rule-based expert systems, multiple-target tracking and target discrimination. When I moved over to manage the tracking and discrimination algorithm development for the Air Force's Space Surveillance and Tracking System I became familar with the work being done for us by contractors that included Nichols Research Corporation (NRC). I became very familar with the work of Perlovsky and his colleagues at Nichols Research who supported us from the Newport Beach, Colorado Springs and Boston offices of the company on the multiple-target tracking algorithms and the target classification algorithms. I found the work to be so interesting and of such high quality that I joined NRC in 1988.

Perlovsky sees artifical intelligence as a very practical discipline and believes that computing machines can do a good job of at least mimicking human intelligence through the use of a priori knowledge (as rules preprogrammed into the computer or a priori probability distributions) along with experience (collected data from observational or statistical designed studies) combined using algorithms (Bayes theorem, adaptive neural networks) based on the mathematical foundation of uncertainty incorporated through probability theory and/or fuzzy set theory.

In my experience, I have found rule-based expert systems to be one of the major successful developments in the field of artificial intelligence. At the heart of these systems lies the tools of mathematics and statistics, including the discriminant or classification algorithms based on multivariate Gaussian models (linear and quadratic classifiers) and the nonparametric classification algorithms (kernel discriminant algorithms and classification tree algorithms). Also, patterns can be discovered by computers through the use of clustering algorithms based on Gaussian mixture models or nonparametric techniques like nearest neighbor rules.

The Bayesian approach to statistical analysis has been useful in many areas including the Kalman filter. In Kalman filtering prior knowledge plus current data is used to update the estimate of the current state and for the prediction of the future state of a dynamic system using a simple recursive algorithm that is easily updated in the computer. Many of these developments are well characterized and developed from first principles in this text.

Perlovsky emphasizes his own work including the MLANS system which is a neural network system that incorporates important statistical ideas such as maximum likelihood, the Cramer-Rao inequality and statistical efficiency along with the neural network architecture.

Some of this work was developed by Perlovsky under an Army contract that was coincidentally managed by my brother Julian. I have always viewed this research as being successful because it applied appropriate statistical models to the real problems. I think the crucial aspects of this work are the appropriate use of the Bayesian paradigm and teh indentification of appropriate models for construction of the likelihood equations. The fundamental and well established tools of probability and statistics are the keys. In his proposals, Leonid also included ideas from fuzzy set theory and embedded his methods in an artificial neural network framework. I always thought that these modern theories (fuzzy set theory and neural networks) were gimmicks to get military funding. This may not have been a fair assessment on my part as a careful reading of this book indicates that Perlovsky honestly views these tools as important.

There are subtleties to concepts such as fuzzy set theory. Although I do not yet see its value as a substitute for measure theoretic probability theory for characterizing human uncertainty, it is possible that I just haven't thought hard enough about it. Maybe a continued reading and rereading of Perlovsky's book will help me.

This is a very interesting and unique book on artificial intelligence from a perspective that is quite different from what one find in the standard books written by computer scientists (who often do not have the deep understanding of probability and statistics that Perlovsky possesses).

Rating:
Summary: Excellent new ideas...
Review: Perlovsky has published a large number of papers exploring limitations associated with today's main approaches to AI. This book captures the essence of those papers with the addition of developing and expanding Perlovsky's philosophy of AI.

Perlovsky takes the reader through the development of AI from its behaviouristic beginnings through Minsky's "revolution" of purely symbolic AI. He then goes through some more recent methods of combining both methods with some adaptive learning techniques. For all cases Perlovsky clearly demonstrates the inherent limitations of all methods through analytic means.

He then presents his view on a possible way forward though adaptive networks employing fuzzy logic, illustrated with some examples such as work done on SAR image analysis. Throughout the book he provides many examples and certainly this will make an excellent advanced textbook for the field of AI.

I am particularly impressed by his good overview and development of his philosophical views. After years of books by people like the Churchland's, Chalmers, Searle and Hofstadter this is finally a great example of someone who is not afraid to cut through the fluff and expose the real problem to further progress. This should be required reading for anyone looking into philosophy, in particular the philosophy of mind and science. The references are very valuable and Perlovsky and done an excellent job of listing many.

That said, there are a few points I would suggest for the next edition of this book. First, I find it odd that Perlovsky seems unaware of Chaitin's work in algorithmic information theory, particularly his book "The Limits of Mathematics". Considering Perlovsky's many references to Godel and Turing this is a glaring omission. Also, he is missing a reference to Wilson's excellent "Spikes, Decisions, and Actions : The Dynamical Foundations of Neurosciences", possibly because it is also a very recent publication. These three books belong together!

Perlovsky also needs to answer the issues posed by Susan Haack ("Deviant Logic, Fuzzy Logic : Beyond the Formalism") regarding the viability of fuzzy logic and better-define the methods that are the "correct" interpretation when applied to neural networks. I would hope to see an expansion on this theme in the future.

There are some irritating small problems, the most major being the spelling of "Plank" throughout the text; someone's spellcheck was acting up I guess...The index is a bit thin as well.

An excellent book and, as stated in the other reveiw, "must" reading for anyone interested in the future of AI. Perlovsky, Chaitin, Prigogine, Wilson and James H. Austin ("Zen and the Brain") are to be applauded for breaking with the current strangling hold of ancient thought.

Rating:
Summary: Philosophy of Intelligence and Intelligence of Philosophy
Review: The book is a fascinating review of concepts in philosophy, semiotics, and mathematics of intelligence; it is interesting for a wide audience, not just for mathematicians. The author relates mathematics and philosophy and finds detailed and specific connections, suggesting that the evolution of mathematics of intelligence during the last fifty years paralleled the evolution of philosophy of mind beginning from Plato. The author develops mathematical techniques for semiotics, which reconciles the differences among concepts in classical semiotics. He argues that *symbol* as understood in general culture is a process relating conscious and unconscious, and he proposes modifications in semiotic terminology needed to bring in correspondence varying usages of this term. A most intriguing is a chapter that connects the mathematics to Kantian theory; an ability for perceiving beauty, concludes the author, is associated with learning. A useful feature of the book is the proposed course outlines for mathematically prepared students and for non-mathematicians.

Rating:
Summary: Philosophy of Intelligence and Intelligence of Philosophy
Review: The book is a fascinating review of concepts in philosophy, semiotics, and mathematics of intelligence; it is interesting for a wide audience, not just for mathematicians. The author relates mathematics and philosophy and finds detailed and specific connections, suggesting that the evolution of mathematics of intelligence during the last fifty years paralleled the evolution of philosophy of mind beginning from Plato. The author develops mathematical techniques for semiotics, which reconciles the differences among concepts in classical semiotics. He argues that *symbol* as understood in general culture is a process relating conscious and unconscious, and he proposes modifications in semiotic terminology needed to bring in correspondence varying usages of this term. A most intriguing is a chapter that connects the mathematics to Kantian theory; an ability for perceiving beauty, concludes the author, is associated with learning. A useful feature of the book is the proposed course outlines for mathematically prepared students and for non-mathematicians.

Rating:
Summary: Review of Neural Networks and Intellect
Review: While this book is not mainstream and would not be a good place to try to begin to learn AI it is thought provoking and presents a number of new ideas. Where Rod Brooks seaks to eliminate internal models Perlovsky makes them a part of his definition of intelligence. I rather liked the philosophical slant of the book. Perlovsky expects to overcome complexity using fuzzy logic. Looked at from Allen Newell's multiple levels or bands perspective fuzzy logic is an extension of classical logic and in any real computer is implemented by a lower level (underlying level) of ordinary Boolean logic. It would seem to me that different granularities at each level would accomplish the same thing. Rules in first order logic can represent subsymbolic entities just as easily as they represent macroscopic objects and actions. Perlovsky presents an interesting MLANS architecture but gives rather less detail on a hierachical version. The computational complexity of these recurrent networks looks rather high to me. Perlovsky's book is a good challenge to those who have already studied some traditional AI and connectionist literature. He criticizes nearest neighbor (and CBR) systems but it seens to me that they offer some very fast parallel implementations such as Kanerva's distributed memory and Stanfill and Waltz's memory-based reasoning.

Rating:
Summary: Review of Neural Networks and Intellect
Review: While this book is not mainstream and would not be a good place to try to begin to learn AI it is thought provoking and presents a number of new ideas. Where Rod Brooks seaks to eliminate internal models Perlovsky makes them a part of his definition of intelligence. I rather liked the philosophical slant of the book. Perlovsky expects to overcome complexity using fuzzy logic. Looked at from Allen Newell's multiple levels or bands perspective fuzzy logic is an extension of classical logic and in any real computer is implemented by a lower level (underlying level) of ordinary Boolean logic. It would seem to me that different granularities at each level would accomplish the same thing. Rules in first order logic can represent subsymbolic entities just as easily as they represent macroscopic objects and actions. Perlovsky presents an interesting MLANS architecture but gives rather less detail on a hierachical version. The computational complexity of these recurrent networks looks rather high to me. Perlovsky's book is a good challenge to those who have already studied some traditional AI and connectionist literature. He criticizes nearest neighbor (and CBR) systems but it seens to me that they offer some very fast parallel implementations such as Kanerva's distributed memory and Stanfill and Waltz's memory-based reasoning.