Computational Intelligence : An Introduction

Rating:
Summary: A fine overview
Review: The book is divided into four parts. The first part is about neural networks and is covered in much more depth than other parts. The author touches upon every major neural network paradigm and covers fundamental learning techniques. What bothers me about the first part is following:
- it is too theoretical
- the chapter about neural networks performance is too fat, although practical (but it would be better to be more practical about general NN techniques), so i lost interest in it when reading.
The second part about genetic algorithms is interesting and concise, but techniques like cultural evolution and coevolution are flabby explained. Few pages more could be spared.
The third part about swarm intelligence covers two techniques, namely ant colonies and particle optimization. Since there is not much literature about these young techniques, i find them quite well explained.
The last part about fuzzy logic was a bit disappointing, because it seemed like the author was in a rush to finish the book. Too shallow to be of any practical use (but still as a reference).
All in all is the book a good overview with lots of references on more specialized literature. I don't know about any other book to cover such diverse fields in one volume, and therefore recommend it for readers, who do not intend to go into deep study of things.

Rating:
Summary: A good start
Review: This book is suitable for undergraduates and first-year graduate students in computer science who are interested in getting an overview of what is and has been done in a field that used to be classified as "artificial intelligence". The new designation as "computational intelligence" was done so as to distance it from other activities in artificial intelligence that are deemed somewhat exotic, such as the efforts to build autonomous thinking machines. The book can be read very quickly, due to the sparse nature of the paragraph organization, and the author puts exercises at the end of each chapter to test the reader's insight and problem solving ability.

The author divides his book into five parts, the first one being an introduction and history, and the next four being on neural networks, evolutionary computation, swarm intelligence, and fuzzy systems. The third part is relatively new on the scene in computational intelligence, and so its inclusion in this book is welcomed.

Neural networks have been extensively used in the last four decades in many different fields, such as financial engineering, bioinformatics, physics, and network analysis. The author devotes six chapters to their elucidation, and he discusses the most important types of artificial neural networks and the different learning schemes employed by them. Detailed derivations and pseudocode are given for these, particularly for back-propagation, and the "LeapFrog" and "conjugate gradient" approaches to optimization. Most interesting is the inclusion of a "swarm optimization" technique. Although brief, the author includes pseudocode and references for further reading.

The author includes a discussion of product unit neural networks, which are sometimes not treated in other books on neural networks. In this type of neural network, the neurons compute the weighted product of the input signals, and the author includes a "distortion factor" that allows the dynamical shaping of the activiation function during the training phase. Only a cursory discussion of what it actually means for a neural network to learn is given, but references for further reading are included. The author also includes a nice discussion of performance issues in the use of neural networks. Readers who eventually use them in practice will find that neural networks can be computationally intensive. An interesting inclusion in this discussion is that of the VC-dimension, and how it relates to the total number of weights in a 1-hidden layer network, and thus of the number of training examples needed. The reader is asked to investigate the role of the Huber function in making a neural network more robust to outliers. The author cautions the reader that the use of neural networks should not be taken lightly and great care is need to insure optimal performance.

In part 3 the author treats genetic and evolutionary programming, evolutionary strategies, differential and cultural evolution, and co-evolution. The use of these techniques has dramatically increased in the last decade, especially in optimization problems. The author points out the role of finding the right chromosome representation in the design of an evolutionary algorithm, this serving also as a kind of classification of the different approaches: binary strings for genetic algorithms, tree representations for genetic programming, and real numbers for evolutionary programming. The author compares briefly the use of evolutionary computing versus using techniques from classical optimization theory, with the role played by the "no-free-lunch" theorem. The author holds that evolutionary algorithms are superior for discontinuous, nondifferentiable, multimodal, and noisy problems.

Of personal interest to me, an application of genetic algorithms to routing optimization is discussed in this part. The author also briefly mentions "niching" as a method to locate multiple minima and asks the reader to develop a genetic algorithm to train a feed-forward neural network as an exercise. For readers who have a knowledge of LISP the discussion on genetic programming will seem much more natural. Evolutionary programming, which differs from genetic algorithms and genetic programming with its omission of crossover and its emphasis on simulating adaptive behavior in evolution, and not genetic models is given ample treatment, along with some useful pseudocode. An elementary application to function optimization is given. The discussion on evolutionary strategies is a fine one since the ones in the literature may be too difficult mathematically for a reader at this level. The discussions on differential and cultural evolution, and coevolution are too brief however (only a few pages devoted to each).

Of great interest to me was the part on swarm intelligence, as my knowledge of this field is very meager. It has taken on importance recently in research circles and so I was very interested in seeing if could be used successfully in practical optimization problems. Having just begun to get my feet wet in swarm intelligence algorithms, this part was a good help in giving me more insight into the use of them. This book is one of the few that covers swarm intelligence, most of the literature still being found in online preprints and published journal articles. The author calls one approach "particle swarm optimization" (PSO), and he explains how the different social interaction schemes serve to classify the algorithm, going by the namnes of star, ring, and wheels topology. He also discusses various modifications to PSO that improve its convergence and to increase its diversity. Another approach is based on the social interaction of ants, and is called appropriately "ant colony optimization". This approach has been applied with some success to route assignment in packet networks, and has been shown to be competetive with vector-distance and link-state shortest path algorithms. It remains to be seen if such an approach will be implemented in real-world network topologies, dominated as they are currently with legacy hardware. The author also gives brief discussions on some of its other applications, such as robotics, and the quadratic assignment, job-scheduling, and graph coloring problems.

I did not read the part on fuzzy systems so I will omit its review.