Neural Network Toolbox

Preface

Neural Networks

Basic Chapters

Mathematical Notation for Equations and Figures

Basic Concepts

Language

Weight Matrices

Layer Notation

Figure and Equation Examples

Mathematics and Code Equivalents

Neural Network Design Book

Acknowledgments

Introduction

Getting Started

Basic Chapters

Help and Installation

What's New in Version 4.0

Control System Applications

Graphical User Interface

New Training Functions

Design of General Linear Networks

Improved Early Stopping

Generalization and Speed Benchmarks

Demonstration of a Sample Training Session

Neural Network Applications

Applications in this Toolbox

Business Applications

Aerospace

Automotive

Banking

Credit Card Activity Checking

Defense

Electronics

Entertainment

Financial

Industrial

Insurance

Manufacturing

Medical

Oil and Gas

Robotics

Speech

Securities

Telecommunications

Transportation

Summary

Neuron Model and Network Architectures

Neuron Model

Simple Neuron

Transfer Functions

Neuron with Vector Input

Network Architectures

A Layer of Neurons

Multiple Layers of Neurons

Data Structures

Simulation With Concurrent Inputs in a Static Network

Simulation With Sequential Inputs in a Dynamic Network

Simulation With Concurrent Inputs in a Dynamic Network

Training Styles

Incremental Training (of Adaptive and Other Networks)

Batch Training

Summary

Figures and Equations

Perceptrons

Introduction

Important Perceptron Functions

Neuron Model

Perceptron Architecture

Creating a Perceptron (newp)

Simulation (sim)

Initialization (init)

Learning Rules

Perceptron Learning Rule (learnp)

Training (train)

Limitations and Cautions

Outliers and the Normalized Perceptron Rule

Graphical User Interface

Introduction to the GUI

Create a Perceptron Network (nntool)

Train the Perceptron

Export Perceptron Results to Workspace

Clear Network/Data Window

Importing from the Command Line

Save a Variable to a File and Load It Later

Summary

Figures and Equations

New Functions

Linear Filters

Introduction

Neuron Model

Network Architecture

Creating a Linear Neuron (newlin)

Mean Square Error

Linear System Design (newlind)

Linear Networks with Delays

Tapped Delay Line

Linear Filter

LMS Algorithm (learnwh)

Linear Classification (train)

Limitations and Cautions

Overdetermined Systems

Underdetermined Systems

Linearly Dependent Vectors

Too Large a Learning Rate

Summary

Figures and Equations

New Functions

Backpropagation

Overview

Fundamentals

Architecture

Simulation (sim)

Training

Faster Training

Variable Learning Rate (traingda, traingdx)

Resilient Backpropagation (trainrp)

Conjugate Gradient Algorithms

Line Search Routines

Quasi-Newton Algorithms

Levenberg-Marquardt (trainlm)

Reduced Memory Levenberg-Marquardt (trainlm)

Speed and Memory Comparison

Summary

Improving Generalization

Regularization

Early Stopping

Summary and Discussion

Preprocessing and Postprocessing

Min and Max (premnmx, postmnmx, tramnmx)

Mean and Stand. Dev. (prestd, poststd, trastd)

Principal Component Analysis (prepca, trapca)

Post-Training Analysis (postreg)

Sample Training Session

Limitations and Cautions

Summary

Control Systems

Introduction

NN Predictive Control

System Identification

Predictive Control

Using the NN Predictive Controller Block

NARMA-L2 (Feedback Linearization) Control

Identification of the NARMA-L2 Model

NARMA-L2 Controller

Using the NARMA-L2 Controller Block

Model Reference Control

Using the Model Reference Controller Block

Importing and Exporting

Importing and Exporting Networks

Importing and Exporting Training Data

Summary

Radial Basis Networks

Introduction

Important Radial Basis Functions

Radial Basis Functions

Neuron Model

Network Architecture

Exact Design (newrbe)

More Efficient Design (newrb)

Demonstrations

Generalized Regression Networks

Network Architecture

Design (newgrnn)

Probabilistic Neural Networks

Network Architecture

Design (newpnn)

Summary

Figures

New Functions

Self-Organizing and
Learn. Vector Quant. Nets

Introduction

Important Self-Organizing and LVQ Functions

Competitive Learning

Architecture

Creating a Competitive Neural Network (newc)

Kohonen Learning Rule (learnk)

Bias Learning Rule (learncon)

Training

Graphical Example

Self-Organizing Maps

Topologies (gridtop, hextop, randtop)

Distance Funct. (dist, linkdist, mandist, boxdist)

Architecture

Creating a Self Organizing MAP Neural Network (newsom)

Training (learnsom)

Examples

Learning Vector Quantization Networks

Architecture

Creating an LVQ Network (newlvq)

LVQ1 Learning Rule(learnlv1)

Training

Supplemental LVQ2.1 Learning Rule (learnlv2)

Summary and Conclusions

Self-Organizing Maps

Learning Vector Quantizaton Networks

Figures

New Functions

Recurrent Networks

Introduction

Important Recurrent Network Functions

Elman Networks

Architecture

Creating an Elman Network (newelm)

Training an Elman Network

Hopfield Network

Fundamentals

Architecture

Design (newhop)

Summary

Figures

New Functions

Adaptive Filters and
Adaptive Training

Introduction

Important Adaptive Functions

Linear Neuron Model

Adaptive Linear Network Architecture

Single ADALINE (newlin)

Mean Square Error

LMS Algorithm (learnwh)

Adaptive Filtering (adapt)

Tapped Delay Line

Adaptive Filter

Adaptive Filter Example

Prediction Example

Noise Cancellation Example

Multiple Neuron Adaptive Filters

Summary

Figures and Equations

New Functions

Applications

Introduction

Application Scripts

Applin1: Linear Design

Problem Definition

Network Design

Network Testing

Thoughts and Conclusions

Applin2: Adaptive Prediction

Problem Definition

Network Initialization

Network Training

Network Testing

Thoughts and Conclusions

Appelm1: Amplitude Detection

Problem Definition

Network Initialization

Network Training

Network Testing

Network Generalization

Improving Performance

Appcr1: Character Recognition

Problem Statement

Neural Network

System Performance

Summary

Advanced Topics

Custom Networks

Custom Network

Network Definition

Network Behavior

Additional Toolbox Functions

Initialization Functions

Transfer Functions

Learning Functions

Custom Functions

Simulation Functions

Initialization Functions

Learning Functions

Self-Organizing Map Functions

Network Object Reference

Network Properties

Architecture

Subobject Structures

Functions

Parameters

Weight and Bias Values

Other

Subobject Properties

Inputs

Layers

Outputs

Targets

Biases

Input Weights

Layer Weights

Reference

Functions -- By Category

Functions by Network Type

Functions by Class

Transfer Functions

Transfer Function Graphs

Reference Headings

Functions -- Alphabetical List

Glossary

Bibliography

Demonstrations and
Applications

Tables of Demonstrations and Applications

Chapter 2: Neuron Model and Network Architectures

Chapter 3: Perceptrons

Chapter 4: Linear Filters

Chapter 5: Backpropagation

Chapter 7: Radial Basis Networks

Chapter 8: Self-Organizing and Learn. Vector Quant. Nets

Chapter 9: Recurrent Networks

Chapter 10: Adaptive Networks

Chapter 11: Applications

Simulink

Block Set

Transfer Function Blocks

Net Input Blocks

Weight Blocks

Control Systems Blocks

Block Generation

Example

Exercises

Code Notes

Dimensions

Variables

Utility Function Variables

Functions

Code Efficiency

Argument Checking


 Getting Started