Neural Symbolic Learning Systems: Neural Networks for Normative Reasoning

In this section we are going to discuss about the Neural-Symbolic Integration approach. According to Minsky [13], both symbolic and connectionist intelligent computational systems have virtues and deficiencies that should be correspondingly exploited and compensated in order to maximize the capabilities of both.
To describe what is a Neural-Symbolic Integration approach we can begin by seeing what it aims to do. This approach follows Minsky’s thought by integrating the two different approaches, exploiting in this way the capabilities of both. The symbolic approach makes use of complex symbols and processing mechanisms to produce an intelligent behavior, instead the connectionist approach believes that the brain is the key of the intelligence, so it aims to model it and reproduce its functions using mathematical models.
A symbolic approach possess the capabilities to have an inference process that can be explained through an automatic theorem proving and also possesses a powerful declarative language to describe the knowledge and the rules that will be used for the reasoning. On the other hand a symbolic approach is affected by the curse of dimensionality and the knowledge acquisition bottleneck that is the problem that affects the learning capability of a symbolic approach, relative to the ability to increase the knowledge possessed by the system due to the difficulty of automatically produce new rules to be used to define an intelligent behavior.
A connectionist approach instead has the advantages of a massive parallelism for the computation, the ability to learn and increase its capabilities from instances and the capacity to generalize. These last two capabilities means that a connectionist system, given enough instances is able to learn the general patterns that the instances have in common. This kind of approach also has some drawbacks, as discussed in [8] it is not trivial how to bind the knowledge embedded in a neural networks to the processing of its components, this is the reason why they are considered a black box approach.
Those different approaches can be merged together in order to exploit the advantages of both and using them to overcome the deficiencies that emerge if considering them alone. The main idea is to use a connectionist approach as the executioner of the task, in this way the massive parallelism computation of this kind of approach can be used. In addition by using neural networks as task executioners it is possible to exploit their better learning capabilities than the one possessed by symbolic approaches, as described in details by [19] and in addition they are more tolerant to noisy data. On the other hand, a symbolic approach can be used to overcome the lacks of neural networks by using its capabilities to better define the knowledge and using it for the interactions between the task executioner and the environment.