List Of Authors |
Editorial Preface |
Neural Network Models for Pattern Recognition and Associative Memory / Chapter 1: |
Preface |
Abstract |
Introduction / 1: |
The McCulloch-Pitts Neuron / 2: |
Adaptive Filter Formalism / 3: |
Logical Calculus and Invariant Patterns / 4: |
Perceptrons and Back-Coupled Error Correction / 5: |
Adaline and Madaline / 6: |
Multi-Level Perceptrons: Early Back Propagation / 7: |
Later Back Propagation / 8: |
Hebbian Learning / 9: |
The Learning Matrix / 10: |
Linear Associative Memory (LAM) / 11: |
Real-Time Models and Embedding Fields / 12: |
Instars and Outstars / 13: |
Additive and Shunting Activation Equations / 14: |
Learning Equations / 15: |
Learning Space-Time Patterns: The Avalanche / 16: |
Adaptive Coding and Category Formation / 17: |
Shunting Competitive Networks / 18: |
Competitive Learning / 19: |
Computational Maps / 20: |
Instability of Computational Maps / 21: |
Adaptive Resonance Theory (ART) / 22: |
ART for Associative Memory / 23: |
Cognitron and Neocognitron / 24: |
Simulated Annealing / 25: |
Conclusion / 26: |
Bibliography |
References |
Nonlinear Neural Networks: Principles, Mechanisms, and Architectures / Chapter 2: |
Reference |
Interdisciplinary Studies during the Nineteenth Century: Helmholtz, Maxwell, and Mach |
The Schism between Physics and Psychology |
The Nonlinear, Nonlocal, and Nonstationary Phenomena of Mind and Brain |
Color Theory / A: |
Top-Down Learning, Expectation, and Matching / B: |
The Nature of an Enduring Synthesis |
Sources of Neural Network Research: Binary, Linear, Continuous-Nonlinear |
Binary |
Linear |
Continuous-Nonlinear / C: |
Nonlinear Feedback between Fast Distributed STM Processing and Slow Associative LTM Processing |
Principles, Mechanisms, and Architectures |
Content-Addressable Memory Storage: A General STM Model and Liapunov Method |
Other Liapunov Methods |
Testing the Global Consistency of Decisions in Competitive Systems |
Stable Production Strategies for a Competitive Market |
Sensitive Variable-Load Parallel Processing by Shunting Cooperative-Competitive Networks: Automa... |
Physiological Interpretation of Shunting Mechanisms as a Membrane Equation |
Sigmoid Feedback, Contrast Enhancement, and Short Term Memory Storage by Shunting Feedback Netwo... |
Competitive Learning Models |
Stable Self-Organization of Pattern Recognition Codes |
Internally Regulated Learning and Performance in Neural Models of Sensory-Motor Control: Adaptiv... |
External Error Signals for Learning Adaptive Movement Gains: Push-Pull Opponent Processing |
Match-Invariants: Internally Regulated Learning of an Invariant Self-Regulating Target Position ... |
Presynaptic Competition for Long Term Memory: Self-Regulating Competitive Learning |
Neural Pattern Discrimination / Chapter 3: |
Connection with Learning Theory |
Local Temporal Discrimination |
Choices between Incompatible Behaviors: "Majority Rule" in Non-Recurrent "Interference Patterns" |
Local Temporal Generalization: Variable Velocities of Motor Performance |
Why are Sensory Pathways in Different Modalities Anatomically Different if Universal Discriminato... |
Unselective Filtering of Spatial Patterns by Excitatory Networks |
Two Stages of Non-Recurrent Inhibition for Pattern Discrimination |
Specific versus Nonspecific Inhibitory Interneurons, Inhibition at the Axon Hillock, Presynaptic ... |
Pattern Normalization in Type I Networks |
High-Band Filters |
Discrimination of Space-Time Patterns |
Velocity and Orientation Detectors |
Alternative Mechanisms of Pattern Normalization: Saturating Potentials in an On-Off Field, or Lo... |
Proof of Proposition 1 / Appendix A: |
Proof of Lemma 2 / Appendix B: |
Proof of Theorem 1 / Appendix C: |
Proof of Lemma 3 / Appendix D: |
Proof of Proposition 3 / Appendix E: |
Proof of Corollary 1 / Appendix F: |
Proof of Proposition 4 / Appendix G: |
Neural Expectation: Cerebellar and Retinal Analogs of Cells Fired by Learnable or Unlearned Pattern Classes / Chapter 4: |
Neural Expectation: Cerebellar And Retinal Analogs of Cells Fired by Learnable or Unlearned Pattern ... |
Theoretical Review |
Retinal Analog of R Cells |
A Learnable Preset Mechanism: Subtractive Case |
Cerebellar Analogs of U Cells |
A Learnable Present Mechanism: Multiplicative Case |
Self-Organization of Orientation Sensitive Cells in the Striate Cortex / Chapter 5: |
Self-Organization of Orientation Sensitive Cells In The Striate Cortex |
The Model / I: |
The Elements / a: |
The Wiring of the Model / b: |
The Afferent Organization / c: |
The Learning Principle / d: |
The Function of the Model / III: |
Basic Equations of Evolution |
Specification of Details |
The Procedure of the Numerical Calculations |
The Results without Learning |
The Results after a Learning Phase / e: |
The Effect of Non-Standard Stimuli |
The Sensitivity to Nonspecific Input / g: |
Redundancy of Information Storage / h: |
Discussion / IV: |
Structure of the Model and Generalizations |
Comparison with Experiments |
Adaptive Pattern Classification and Universal Recoding, I: Parallel Development and Coding of Neural Feature Detectors / Chapter 6: |
Adaptive Pattern Classification And Universal Recoding, I: Parallel Development And Coding Of Neural... |
The Tuning Process |
Ritualistic Pattern Classification |
Shunts versus Additive Interactions as Mechanisms of Pattern Classification |
What Do Retinal Amacrine Cells Do? |
Arousal as a Tuning Mechanism |
Arousal as a Search Mechanism |
Development of an STM Code |
Appendix |
The "Neural" Phonetic Typewriter by / Chapter 7: |
The ''Neural" Phonetic Typewriter |
Why is speech recognition difficult? |
The promise of neural computers |
Acoustic preprocessing |
Vector quantization |
The neural network |
Shortcut learning algorithm |
Phonotopic maps |
Postprocessing in symbolic form |
Hardware implementations and performance |
Counterpropagation Networks / Chapter 8: |
Counterpropagation Network |
CPN Error Analysis |
CPN Variants and Evolutes |
Adaptive Pattern Classification and Universal Recoding, Ii: Feedback, Expectation, Olfaction, and Illusions / V: |
Adaptive Pattern Classification And Universal Recoding, Ii: Feedback, Expectation, Olfaction, Illusi... |
Adaptive Resonance: Stable Coding and Reset of STM |
Adaptive Resonance in Reinforcement, Motivation, and Attention |
Search and Lock Mechanism |
Olfactory Coding and Learned Expectation |
Modulation of Nonspecific Arousal by a Learned Expectation Mechanism |
Universal Recoding |
Search |
Slow Noradrenergic Transmitter Accumulation-Depletion as a Search Mechanism |
Spatial Frequency Adaptation |
Afterimages |
A Massively Parallel Architecture for A Self-Organizing Neural Pattern Recognition Machine / Chapter 10: |
Introduction: Self-Organization of Neural Recognition Codes |
Self-Scaling Computational Units, Self-Adjusting Memory Search, Direct Access, and Attentional Vi... |
Bottom-Up Adaptive Filtering and Contrast-Enhancement in Short Term Memory |
Top-Down Template Matching and Stabilization of Code Learning |
Interactions between Attentional and Orienting Subsystems: STM Reset and Search |
Attentional Gain Control and Attentional Priming |
Matching: The 2/3 Rule |
Code Instability and Code Stability |
Using Context to Distinguish Signal from Noise in Patterns of Variable Complexity |
Vigilance Level Tunes Categorical Coarseness: Disconfirming Feedback |
Rapid Classification of an Arbitrary Type Font |
Network Equations: Interactions between Short Term Memory and Long Term Memory Patterns |
Direct Access to Subset and Superset Patterns |
Weber Law Rule and Associative Decay Rule for Bottom-Up LTM Traces |
Template Learning Rule and Associative Decay Rule for Top-Down LTM Traces |
Direct Access to Nodes Coding Perfectly Learned Patterns |
Initial Strengths of LTM Traces |
Summary of the Model |
Order of Search and Stable Choices in Short-Term Memory |
Stable Category Learning |
Critical Feature Patterns and Prototypes |
Direct Access after Learning Self-Stabilizes |
Order of Search: Mathematical Analysis |
Order of Search: Computer Simulations |
Biasing the Network towards Uncommitted Nodes |
Computer Simulation of Self-Scaling Computational Units: Weighing the Evidence |
Concluding Remarks: Self-Stabilization and Unitization within Associative Networks / 27: |
Variations on Adaptive Resonance / Chapter 11: |
Background |
Adaptive Thresholding |
Learning with Iterative Short-Time Pattern Presentations |
Continuous ARC Operation |
Conclusions |
ART 2: Self-Organization of Stable Category Recognition Codes for Analog Input Patterns / Chapter 12: |
Art 2: Self-Organization of Stable Category Recognition Codes for Analog Input Patterns |
Adaptive Resonance Architectures |
ART 1: Binary Input Patterns |
ART 2: Analog Input Patterns |
ART 2 Design Principles |
ART 2 STM Equations: F1 |
ART 2 STM Equations: F2 |
ART 2 LTM Equations |
ART 2 Reset Equations: The Orienting Subsystem |
The Match-Reset Tradeoff: Choice of Top-down Initial LTM Values |
Learning Increases Mismatch Sensitivity and Confirms Category Choice |
Choosing a New Category: Bottom-up LTM Initial Values |
The Stability-Plasticity Tradeoff |
Alternative ART 2 Architectures |
Adaptive Bidirectional Associative Memories / Chapter 13: |
Introduction: Storing Data Pairs in Associative Memory Matrices |
Discrete Bidirectional Associative Memory (BAM) Stability |
BAM Correlation Encoding |
Continuous BAMs |
Adaptive BAMs |
ART 3: Hierarchical Search Using Chemical Transmitters in Self-Organizing Pattern Recognition Architectures / Chapter 14: |
Art 3: Hierarchical Search Using Chemical Transmitters in Self-Organizing Pattern Recognition Archit... |
Introduction: Distributed Search of ART Network Hierarchies |
An ART Search Cycle |
ART 2: Three-Layer Competitive Fields |
ART Bidirectional Hierarchies and Homology of Fields |
ART Cascade |
Search in an ART Hierarchy |
A New Role for Chemical Transmitters in ART Search |
Equations for Transmitter Production, Release, and Inactivation |
Alternative ART 3 Systems |
Transmitter Release Rate |
System Dynamics at Input Onset: An Approximately Linear Filter |
System Dynamics after Intrafield Feedback: Amplification of Transmitter Release by Postsynaptic ... |
System Dynamics during Reset: Inactivation of Bound Transmitter Channels |
Parametric Robustness of the Search Process |
Summary of System Dynamics during a Mismatch-Reset Cycle |
Automatic STM Reset by Real-Time Input Sequences |
Reinforcement Feedback |
Notation for Hierarchies |
Trade-Off between Weight Size and Pattern Match |
ART 3 Simulations: Mismatch Reset and Input Reset of STM Choices |
Search Time Invariance at Different Vigilance Values |
Reinforcement Reset |
Input Hysteresis Simulation |
Distributed Code Simulation |
Alternative ART 3 Model Simulation |
Simulation Equations |
Artmap: Supervised Real-Time Learning and Classification of Nonstationary Data by a Self-organizing Neural Network / Chapter 15: |
Artmap: Supervised Real-Time Learning and Classification of Nonstatationary Data by a Self-Organizin... |
The ARTMAP system |
ARTMAP simulations: Distinguishing edible and poisonous mushrooms |
ART modules ARTa and ARTb |
The Map Field |
Simulation Algorithms |
Neuronal Activity as A Shaping factor in The Self Organization of Neuron Assemblies / Chapter 16: |
Neuronal Activity As A Shaping Factor in The Self-Organization of Neuron Assemblies |
A Teleological Argument |
Evidence for a Central Control of Local Hebbian Modifications |
The Gating Mechanism |
Functional Implications of Developmental Plasticity |
Probing Cognitive Processes Through The Structure of Event-Related Potentials: An Experimental and Theoretical Analysis / Chapter 17: |
Probing Cognitive Processes Through The Structure of Event-Related Potentials During Learning: An Ex... |
Attentional Subsystem and Orienting Subsystem |
STM Reset and Search |
Matching via the 2/3 Rule |
ERP Components |
Experimental Paradigm |
Experimental Results: ERP Profiles |
Comparison of ERP Profiles with Adaptive Resonance Theory Mechanisms |
Conclusion: The Relationship of Learning to ERPs |
Unitization, Automaticity, Temploral Order, and Word Recongnition / Chapter 18: |
Unitization, Automaticity, Temporal Order, and Word Recognition |
The Word Length Effect |
Unitization and Psychological Progress |
The Temporal Chunking Problem |
All Letters are Sublists |
Expectancy Learning and Priming |
The McClelland and Rumelhart Model |
The Schneider and Shiffrin Model |
Parallel Processing and Unlimited Capacity |
The Functional Unit of Cognitive Processing: Not Spreading Activation |
Capacity versus Matching |
Adaptive Filter: The Processing Bridge between Sublist Masking and Temporal Order Information ov... |
The LTM Invariance Principle: Temporal Order Information without a Serial Buffer |
Spatial Frequency Analysis of Temporal Order Information |
Speech Perception and Production by a Self-Organizing Neural Network / Chapter 19: |
Speech Perception and Production By A Self-Organizing Neural Network |
The Learning of Language Units |
Low Stages of Processing: Circular Reactions and the Emerging Auditory and Motor Codes |
The Vector Integration to Endpoint Model |
Self-Stabilization of Imitation via Motor-to-Auditory Priming |
Higher Stages of Processing: Context-Sensitive Chunking and Unitization of the Emerging Auditory ... |
Masking Fields |
Neural Dynamics of Adaptive Timing and Temporal Discrimination During Associative Learning / Chapter 20: |
Introduction: Timing the Expected Delay of a Goal Object in a Spatially Distributed and Nonstatio... |
Timing the Balance between Exploration for Novel Rewards and Consummation of Expected Rewards |
Distinguishing Expected Nonoccurrences from Unexpected Nonoccurrences: Inhibiting the Negative Co... |
Spectral Timing Model / Part I: |
Spectral Timing Model: An Application of Gated Dipole Theory |
Spectral Timing Equations |
The Activation Spectrum |
The Habituation Spectrum |
The Gated Signal Spectrum |
Temporally Selective Associative Learning / D: |
The Doubly Gated Signal Spectrum / E: |
The Output Signal / F: |
Effect of Increasing ISI and US Intensity |
Comparison with Nictitating Membrane Conditioning Data |
Inverted U in Learning as a Function of ISI |
Multiple Timing Peaks |
Effect of Increasing US Duration |
Effect of Increasing CS Intensity |
Timed Gating of Read-Out From the Orienting Subsystem / Part II: |
Locating the Timing Circuit within a Self-Organizing Sensory-Cogni-tive and Cognitive-Reinforcem... |
Cognitive-Reinforcement Circuit |
The Gated Dipole Opponent Process |
Adaptive Timing as Spectral Conditioned Reinforcer Learning |
Timed Inhibition of the Orienting Subsystem by Drive Representations |
Timed Activation of the Hippocampus and the Contingent Negative Variation |
Effect of CS Intensity on Timed Motor Behavior |
Spatial Coding of Stimulus Intensity by a PTS Shift Map |
Effect of Drugs on Timed Motor Behavior |
Concluding Remarks: Timing Paradox and Multiple Types of Timing Circuits |
Author Index |
Subject Index |