Neural networks have greatly improved our ability to model human behavior and learning by simulating neuronal functioning and organization. However, they are not able to reproduce the rich repertoire of behaviors that humans use to solve complex problems. Our work incorporates hierarchical behavioral models of how humans learn and solve problems into the design of neural networks. Using Commons' Model of Hierarchical Complexity (1998), we have designed stacked neural networks that by their structure and function parallel the behavioral learning process in humans.
Commons' model is based on cognitive developmental stages. Task actions performed at increasingly higher stages are more complex than task actions performed at immediately preceding stages. Movement to a higher stage occurs when the brain combines, orders, and transforms the behaviors at its current stage. Thus, learning proceeds by stages. In our design of hierarchical stacked neural networks, we parallel this learning process by ordering neural networks in the same way as the developmental learning sequence. The behaviors performed by a network in the stack correspond to a stage and are not dictated by task logic or architectural limitations.
In designing these systems, the task to be performed is first analyzed to determine the sequence of behaviors needed to perform the task and the highest stage of development required to perform task actions. The number of neural networks in the stack is determined by the number of stages of development identified in this analysis. Each network in the stack represents a stage of development with higher stage networks performing the increasingly complex behaviors found at higher stages.