Neural Network Toolbox

learnhd

Hebb with decay weight learning rule

Syntax

[dW,LS] = learnhd(W,P,Z,N,A,T,E,gW,gA,D,LP,LS)

info = learnhd(code)

Description

learnhd is the Hebb weight learning function.

learnhd(W,P,Z,N,A,T,E,gW,gA,D,LP,LS) takes several inputs,

• W - S x R weight matrix (or S x 1 bias vector).

  P - R x Q input vectors (or ones(1,Q)).

  Z - S x Q weighted input vectors.

  N - S x Q net input vectors.

  A - S x Q output vectors.

  T - S x Q layer target vectors.

  E - S x Q layer error vectors.

  gW - S x R gradient with respect to performance.

  gA - S x Q output gradient with respect to performance.

  D  - S x S neuron distances.

  LP - Learning parameters, none, LP = [].

  LS - Learning state, initially should be = [].

and returns,

• dW - S x R weight (or bias) change matrix.

  LS - New learning state.

Learning occurs according to learnhd's learning parameters shown here with default values.

• LP.dr - 0.01 - Decay rate.

  LP.lr - 0.1 - Learning rate.

learnhd(code) returns useful information for each code string:

• 'pnames' - Names of learning parameters.

  'pdefaults' - Default learning parameters.

  'needg' - Returns 1 if this function uses gW or gA.

Examples

Here we define a random input P, output A, and weights W for a layer with a two-element input and three neurons. We also define the decay and learning rates.

• p = rand(2,1);
  a = rand(3,1);
  w = rand(3,2);
  lp.dr = 0.05;
  lp.lr = 0.5;
  

Since learnhd only needs these values to calculate a weight change (see algorithm below), we will use them to do so.

• dW = learnhd(w,p,[],[],a,[],[],[],[],[],lp,[])
  

Network Use

To prepare the weights and the bias of layer i of a custom network to learn with learnhd:

1. Set net.trainFcn to 'trainr'. (net.trainParam will automatically become trainr's default parameters.)
2. Set net.adaptFcn to 'trains'. (net.adaptParam will automatically become trains's default parameters.)
3. Set each net.inputWeights{i,j}.learnFcn to 'learnhd'. Set each net.layerWeights{i,j}.learnFcn to 'learnhd'. (Each weight learning parameter property will automatically be set to learnhd's default parameters.)

To train the network (or enable it to adapt):

1. Set net.trainParam (net.adaptParam) properties to desired values.
2. Call train (adapt).

Algorithm

learnhd calculates the weight change dW for a given neuron from the neuron's input P, output A, decay rate DR, and learning rate LR according to the Hebb with decay learning rule:

• dw =  lr*a*p' - dr*w
  

See Also

• learnh, adapt, train
  

learnh

learnis