Neural Networks
Basic Chapters
Mathematical Notation for Equations and Figures
Basic Concepts
Language
Weight Matrices
Layer Notation
Figure and Equation Examples
Mathematics and Code Equivalents
Neural Network Design Book
Acknowledgments
Introduction
Getting Started
Basic Chapters
Help and Installation
What's New in Version 4.0
Control System Applications
Graphical User Interface
New Training Functions
Design of General Linear Networks
Improved Early Stopping
Generalization and Speed Benchmarks
Demonstration of a Sample Training Session
Neural Network Applications
Applications in this Toolbox
Business Applications
Aerospace
Automotive
Banking
Credit Card Activity Checking
Defense
Electronics
Entertainment
Financial
Industrial
Insurance
Manufacturing
Medical
Oil and Gas
Robotics
Speech
Securities
Telecommunications
Transportation
Summary
Neuron Model and Network Architectures
Neuron Model
Simple Neuron
Transfer Functions
Neuron with Vector Input
Network Architectures
A Layer of Neurons
Multiple Layers of Neurons
Data Structures
Simulation With Concurrent Inputs in a Static Network
Simulation With Sequential Inputs in a Dynamic Network
Simulation With Concurrent Inputs in a Dynamic Network
Training Styles
Incremental Training (of Adaptive and Other Networks)
Batch Training
Summary
Figures and Equations
Perceptrons
Introduction
Important Perceptron Functions
Neuron Model
Perceptron Architecture
Creating a Perceptron (newp)
Simulation (sim)
Initialization (init)
Learning Rules
Perceptron Learning Rule (learnp)
Training (train)
Limitations and Cautions
Outliers and the Normalized Perceptron Rule
Graphical User Interface
Introduction to the GUI
Create a Perceptron Network (nntool)
Train the Perceptron
Export Perceptron Results to Workspace
Clear Network/Data Window
Importing from the Command Line
Save a Variable to a File and Load It Later
Summary
Figures and Equations
New Functions
Linear Filters
Introduction
Neuron Model
Network Architecture
Creating a Linear Neuron (newlin)
Mean Square Error
Linear System Design (newlind)
Linear Networks with Delays
Tapped Delay Line
Linear Filter
LMS Algorithm (learnwh)
Linear Classification (train)
Limitations and Cautions
Overdetermined Systems
Underdetermined Systems
Linearly Dependent Vectors
Too Large a Learning Rate
Summary
Figures and Equations
New Functions
Backpropagation
Overview
Fundamentals
Architecture
Simulation (sim)
Training
Faster Training
Variable Learning Rate (traingda, traingdx)
Resilient Backpropagation (trainrp)
Conjugate Gradient Algorithms
Line Search Routines
Quasi-Newton Algorithms
Levenberg-Marquardt (trainlm)
Reduced Memory Levenberg-Marquardt (trainlm)
Speed and Memory Comparison
Summary
Improving Generalization
Regularization
Early Stopping
Summary and Discussion
Preprocessing and Postprocessing
Min and Max (premnmx, postmnmx, tramnmx)
Mean and Stand. Dev. (prestd, poststd, trastd)
Principal Component Analysis (prepca, trapca)
Post-Training Analysis (postreg)
Sample Training Session
Limitations and Cautions
Summary
Control Systems
Introduction
NN Predictive Control
System Identification
Predictive Control
Using the NN Predictive Controller Block
NARMA-L2 (Feedback Linearization) Control
Identification of the NARMA-L2 Model
NARMA-L2 Controller
Using the NARMA-L2 Controller Block
Model Reference Control
Using the Model Reference Controller Block
Importing and Exporting
Importing and Exporting Networks
Importing and Exporting Training Data
Summary
Radial Basis Networks
Introduction
Important Radial Basis Functions
Radial Basis Functions
Neuron Model
Network Architecture
Exact Design (newrbe)
More Efficient Design (newrb)
Demonstrations
Generalized Regression Networks
Network Architecture
Design (newgrnn)
Probabilistic Neural Networks
Network Architecture
Design (newpnn)
Summary
Figures
New Functions
Self-Organizing and Learn. Vector Quant. Nets
Introduction
Important Self-Organizing and LVQ Functions
Competitive Learning
Architecture
Creating a Competitive Neural Network (newc)
Kohonen Learning Rule (learnk)
Bias Learning Rule (learncon)
Training
Graphical Example
Self-Organizing Maps
Topologies (gridtop, hextop, randtop)
Distance Funct. (dist, linkdist, mandist, boxdist)
Architecture
Creating a Self Organizing MAP Neural Network (newsom)
Training (learnsom)
Examples
Learning Vector Quantization Networks
Architecture
Creating an LVQ Network (newlvq)
LVQ1 Learning Rule(learnlv1)
Training
Supplemental LVQ2.1 Learning Rule (learnlv2)
Summary and Conclusions
Self-Organizing Maps
Learning Vector Quantizaton Networks
Figures
New Functions
Recurrent Networks
Introduction
Important Recurrent Network Functions
Elman Networks
Architecture
Creating an Elman Network (newelm)
Training an Elman Network
Hopfield Network
Fundamentals
Architecture
Design (newhop)
Summary
Figures
New Functions
Adaptive Filters and Adaptive Training
Introduction
Important Adaptive Functions
Linear Neuron Model
Adaptive Linear Network Architecture
Single ADALINE (newlin)
Mean Square Error
LMS Algorithm (learnwh)
Adaptive Filtering (adapt)
Tapped Delay Line
Adaptive Filter
Adaptive Filter Example
Prediction Example
Noise Cancellation Example
Multiple Neuron Adaptive Filters
Summary
Figures and Equations
New Functions
Applications
Introduction
Application Scripts
Applin1: Linear Design
Problem Definition
Network Design
Network Testing
Thoughts and Conclusions
Applin2: Adaptive Prediction
Problem Definition
Network Initialization
Network Training
Network Testing
Thoughts and Conclusions
Appelm1: Amplitude Detection
Problem Definition
Network Initialization
Network Training
Network Testing
Network Generalization
Improving Performance
Appcr1: Character Recognition
Problem Statement
Neural Network
System Performance
Summary
Advanced Topics
Custom Networks
Custom Network
Network Definition
Network Behavior
Additional Toolbox Functions
Initialization Functions
Transfer Functions
Learning Functions
Custom Functions
Simulation Functions
Initialization Functions
Learning Functions
Self-Organizing Map Functions
Network Object Reference
Network Properties
Architecture
Subobject Structures
Functions
Parameters
Weight and Bias Values
Other
Subobject Properties
Inputs
Layers
Outputs
Targets
Biases
Input Weights
Layer Weights
Reference
Functions -- By Category
Functions by Network Type
Functions by Class
Transfer Functions
Transfer Function Graphs
Reference Headings
Functions -- Alphabetical List
Glossary
Bibliography
Demonstrations and Applications
Tables of Demonstrations and Applications
Chapter 2: Neuron Model and Network Architectures
Chapter 3: Perceptrons
Chapter 4: Linear Filters
Chapter 5: Backpropagation
Chapter 7: Radial Basis Networks
Chapter 8: Self-Organizing and Learn. Vector Quant. Nets
Chapter 9: Recurrent Networks
Chapter 10: Adaptive Networks
Chapter 11: Applications
Simulink
Block Set
Transfer Function Blocks
Net Input Blocks
Weight Blocks
Control Systems Blocks
Block Generation
Example
Exercises
Code Notes
Dimensions
Variables
Utility Function Variables
Functions
Code Efficiency
Argument Checking
Printable Documentation (PDF)
Product Page (Web)