|
Over the past quarter century, concepts and theory derived from neural networks (NNs) have featured prominently
in the literature of pattern recognition. Implementationally, classical NNs based on the linear inner product can present
performance challenges due to the use of multiplication operations. In contrast, NNs having nonlinear kernels based on
Lattice Associative Memories (LAM) theory tend to concentrate primarily on addition and maximum/minimum
operations. More generally, the emergence of LAM-based NNs, with their superior information storage capacity, fast
convergence and training due to relatively lower computational cost, as well as noise-tolerant classification has extended
the capabilities of neural networks far beyond the limited applications potential of classical NNs.
This paper explores theory and algorithmic approaches for the efficient computation of LAM-based neural
networks, in particular lattice neural nets and dendritic lattice associative memories. Of particular interest are massively
parallel architectures such as multicore CPUs and graphics processing units (GPUs). Originally developed for video
gaming applications, GPUs hold the promise of high computational throughput without compromising numerical
accuracy. Unfortunately, currently-available GPU architectures tend to have idiosyncratic memory hierarchies that can
produce unacceptably high data movement latencies for relatively simple operations, unless careful design of theory and
algorithms is employed. Advantageously, some GPUs (e.g., the Nvidia Fermi GPU) are optimized for efficient
streaming computation (e.g., concurrent multiply and add operations). As a result, the linear or nonlinear inner product
structures of NNs are inherently suited to multicore GPU computational capabilities. In this paper, the authors' recent
research in lattice associative memories and their implementation on multicores is overviewed, with results that show
utility for a wide variety of pattern classification applications using classical NNs or lattice-based NNs. Dataflow
diagrams are presented in terms of a parameterized model of data burden and LAM partitioning.
|
