3 Approaches to machine intelligence: Classic AI, Simple Neural Networks, and Biological Neural Networks

Jeff Hawkins & Donna Dubinsky (Numenta), Classic AI:

For many problems, researchers concluded that a computer had to have access to large amounts of knowledge in order to be “smart”. Thus they introduced “expert systems”, computer programs combined with rules provided by domain experts to solve problems, such as medical diagnoses, by asking a series of questions. If the disease was not properly diagnosed, the expert adds additional questions/rules to narrow the diagnosis. A Classic AI system is highly tuned for a specific problem.

When classical AI kept failing, researchers turned to artificial neural networks (aka ANNs), though the neurons are only marginally like real ones. Hence Simple Neural Network:

Instead, the emphasis of ANNs moved from biological realism to the desire to learn from data without human supervision. Consequently, the big advantage of Simple Neural Networks over Classic AI is that they learn from data and don’t require an expert to provide rules. Today ANNs are part of a broader category called “machine learning” which includes other mathematical and statistical techniques. Machine learning techniques, including ANNs, look at large bodies of data, extract statistics, and classify the results. [...] they don’t work well when there is limited data for training, and they don’t handle problems where the patterns in the data are constantly changing. Essentially, the Simple Neural Network approach is a sophisticated mathematical technique that finds patterns in large, static data sets.

And so we arrive at Biological Neural Networks:

For example we know the brain represents information using sparse distributed representations (SDRs), which are essential for semantic generalization and creativity. We are confident that all truly intelligent machines will be based on SDRs. SDRs are not something that can be added to existing machine learning techniques; they are more like a foundation upon which everything else depends. Other essential attributes include that memory is primarily a sequences of patterns, that behavior is an essential part of all learning, and that learning must be continuous. In addition, we now know that biological neurons are far more sophisticated than the simple neurons used in the Simple Neural Network approach — and the differences matter.