Neural Network Toolbox

learnwh

Widrow-Hoff weight/bias learning function

Syntax

• [dW,LS] = learnwh(W,P,Z,N,A,T,E,gW,gA,D,LP,LS)
  [db,LS] = learnwh(b,ones(1,Q),Z,N,A,T,E,gW,gA,D,LP,LS)
  info = learnwh(code)
  

Description

learnwh is the Widrow-Hoff weight/bias learning function, and is also known as the delta or least mean squared (LMS) rule.

learnwh(W,P,Z,N,A,T,E,gW,gA,D,LP,LS) takes several inputs,

• W - S x R weight matrix (or b, and S x 1 bias vector).

  P - R x Q input vectors (or ones(1,Q)).

  Z - S x Q weighted input vectors.

  N - S x Q net input vectors.

  A - S x Q output vectors.

  T - S x Q layer target vectors.

  E - S x Q layer error vectors.

  gW - S x R weight gradient with respect to performance.

  gA - S x Q output gradient with respect to performance.

  D - S x S neuron distances.

  LP - Learning parameters, none, LP = [].

  LS - Learning state, initially should be = [].

and returns,

• dW - S x R weight (or bias) change matrix.

  LS - New learning state.

Learning occurs according to learnwh's learning parameter shown here with its default value.

• LP.lr - 0.01 - Learning rate.

learnwh(code) returns useful information for each code string:

• 'pnames' - Names of learning parameters.

  'pdefaults' - Default learning parameters.

  'needg' - Returns 1 if this function uses gW or gA.

Examples

Here we define a random input P and error E to a layer with a two-element input and three neurons. We also define the learning rate LR learning parameter.

• p = rand(2,1);
  e = rand(3,1);
  lp.lr = 0.5;
  

Since learnwh only needs these values to calculate a weight change (see algorithm below), we will use them to do so.

• dW = learnwh([],p,[],[],[],[],e,[],[],[],lp,[])
  

Network Use

You can create a standard network that uses learnwh with newlin.

To prepare the weights and the bias of layer i of a custom network to learn with learnwh:

1. Set net.trainFcn to 'trainb'. net.trainParam will automatically become trainb's default parameters.
2. Set net.adaptFcn to 'trains'. net.adaptParam will automatically become trains's default parameters.
3. Set each net.inputWeights{i,j}.learnFcn to 'learnwh'. Set each net.layerWeights{i,j}.learnFcn to 'learnwh'. Set net.biases{i}.learnFcn to 'learnwh'.

Each weight and bias learning parameter property will automatically be set to learnwh's default parameters.

To train the network (or enable it to adapt):

1. Set net.trainParam (net.adaptParam) properties to desired values.
2. Call train(adapt).

See newlin for adaption and training examples.

Algorithm

learnwh calculates the weight change dW for a given neuron from the neuron's input P and error E, and the weight (or bias) learning rate LR, according to the Widrow-Hoff learning rule:

• dw = lr*e*pn'
  

See Also

newlin, adapt, train

References

Widrow, B., and M. E. Hoff, "Adaptive switching circuits," 1960 IRE WESCON Convention Record, New York IRE, pp. 96-104, 1960.

Widrow B. and S. D. Sterns, Adaptive Signal Processing, New York: Prentice-Hall, 1985.

learnsom

linkdist