/* --
   Memo to rnnHira
   Jun Tani Aug29, 2007
   -- */

[1] Basic of RNNPB
RNNPB is a recurrent neural network with parametric biases.
The coventional RNN can lean only a single type of dynamic saquence.
Please refer to the attached paper of mine.
The RNNPB can learn multiple types of dynamic pattern sequences by
differenciating the PB vector for each sequence to be learned.
In other words, PB plays a role of parameters to bifurcate dynamic
characteristics of RNN.
Figures in RNNPB-Verona07.gif shows the architecture of the RNNPB.
This correspons to exact implementation in rnnHira interms of number
of units in the net.
In (a), OutN: number of output units, InN: number of inputunits, PB:
number of PB vector, ContN: number of context units and HiddenN:
number of hidden units.

The learning is conducted by using multiple teaching sequences.
The objective of learning is to regenerate each teaching sequence by
determining a common synaptic weight matrics and specific PB vector
for each sequence. This is done by the iterative comptation of BPTT.
See (a).

The generation can be done after the error convergence through
training. With specifying the PB vector and initial input state,
corresponding sequence is generated.
It is done by the closed-loop mode. See (b)

In the case of robot implementation, see (c).

[2] RNNPB program rnnHira
% Makefile to compile this program is Makefile.bsd
% The following shows an example transaction with the program with
% some comments (started by %).

% In order to start training of multiple sequences, we need to prepare
% two files in tmp directory. 
% Those are train.pat and train.ft
% train.pat contains 3 seqs as an example.
% The file starts with 2 2 which means that input unit number and
% output one are 2 and 2.
% Each seq consists of multiple steps which are formatted as
% step In[0] In[1] Out[0] Out[1]
% Actually In[0] In[1] is the current input states and Out[0] and
% Out[1] are their next step prediction.
% Each sequence is deliminated by negative number like -1 -2 -3 after
% 20 or 30 steps.
% train.ft is parameter file which you don't need to modify. 
% But you might modify the learning rate epsi0 which is currently set
 0.02. If you make it larger, faster to converge but becomes unstable
 in learning.

% To start program in unix => ./rnnHira
% The program starts!!
 /* ------------------------------------------ 
      The open loop learnig of RNN 2002 summer
                Jun Tani a 
    ------------------------------------------ */

 InN 2 ContN 2 OutN 2 HiddenN 10 TrainSeq+0 134729728 TrainSeq+1 134729944 
 train gene robo 

% Let's start to train sequences in train.pat.

train

ReadPattern inN 2 outN 2 MaxTrainSeq3 net->trainSeqS 134746112 AllocTch134750720 
seq_i0 steps27 AllocTch134824448 
seq_i1 steps26 AllocTch134852608 
seq_i2 steps72 net->trainSeqS->Tch134750720 TrainSeq.Tch134750720 
=>MaxTrainSeq 3

w21134706176 b2134700672 LeNet: InN2 OutN2 maxSteps27 
% you are asked to start synaptic weight of prior-saved one 
% or from randomized one. Here we start from randomized one by
% putting 0.
ReadWeight? 1,0 0

% Ask you learning iteration. Maybe 4000 steps here.
nepochs? 4000

% Now training iteration start.
--
Skip here
--
 PB PB[seq0}  0.70  0.56 PB[seq1}  0.67  0.54 PB[seq2}  0.19  0.41 
time=3800 tss 0.000389 inputRatio0.100000 epsi0.020000 
iniCiU
 PB PB[seq0}  0.70  0.56 PB[seq1}  0.67  0.54 PB[seq2}  0.19  0.41 
time=3900 tss 0.000394 inputRatio0.100000 epsi0.020000 
iniCiU
 PB PB[seq0}  0.70  0.56 PB[seq1}  0.67  0.54 PB[seq2}  0.18  0.41 
% Meanwhile, you see the plots during the training process.
% You see pairs of OutTch (teaching seq pattern) and out
(self-generated pattern)
% You see 3 seqs of them each of which is associated with the PB
% vector shown under each seq.

% after 4000 steps training, it asks you where to save the result.
directory ? => aug29La0
% Here, results are saved in data/aug29La0

mkdir data/aug29La0 

% Now, let the network self-generate the sequences
 train gene robo 
gene

% You specify the weight files to read.
directory to read weights ? => aug29La0

 Read weight from data/aug29La0/aug29La0.wt 
TestMaxSteps? (should be less than 1000!!) 200
AllocTch134791168 
 alloc seq[0] 
bug0 
bug1 
% You specify the initial step state of sequence to be generated.
% input initial input vector of 2 dimension 
0.2 0.8
% You specify the parametric biases vector for the seq to be
% generated. We put 0.28 0.21 which correspond to the third cyclic pattern.
% input the PB vector of 2 dimension 
0.28 0.21
% Now we see the plot as like gene0.gif

% We conitue to generate one more pattern
continue:1 

continue:1 1

TestMaxSteps? (should be less than 1000!!) 100
AllocTch135155712 
 alloc seq[1] 
bug0 
bug1 
 input initial input vector of 2 dimension 
0.2 0.8

% Now we set PB as arbitrary....
 input the PB vector of 2 dimension 
0.64 0.68
% Now we see the plot as like gene1.gif
continue:1 0
% Now we save the result of generation in data/aug29La0G
directory ? => aug29La0G

mkdir data/aug29La0G 
 train gene robo 

% Back to unix
Cont-C
% The program is terminated.

-------------------------------
% The analysis of the data saved might be done using excel.
% We see data saved for learning results.
ls data/aug29La0                              
aug29La0.ft   aug29La0.wt     l1aug29La0.seq  rnn99.h
aug29La0.pat  l0aug29La0.seq  l2aug29La0.seq

% aug29La0.wt is synaptic weight file.
% l0aug29La0.seq is the training result of 0th training seq pattern
% and l?aug29La0.seq is ?th seq pattern

% Similarly, we can see the generation results saved.
ls data/aug29La0G                             
aug29La0G.ft   aug29La0G.wt     l1aug29La0G.seq
aug29La0G.pat  l0aug29La0G.seq  rnn99.h
% Because we attempted to generate 2 seqs, l0aug29La0G.seq and
% l1aug29La0G.seq are the results.

[3] Comments for implementation to mobile robot developed in Verona
The pattern generation with the robot can be done with "robo" in the
main command by initiating subroutin of Robo().
But, you need to develop the robot interface.
Only, you need to change is L697 of rnn99.c where you see:

	if (mode==RoboGene) {
	  /* This part has to be made in Verona!! 
		 The current sensor state and motor state
		 has to be put into in[InN] in the folowing function */
	  //GetCurrentSensorMotorState(in,InN); 
	  for (i=0; i<InN; ++i) *(inTeach+i) = *(in+i);
	}
	SingleForwardAct(in,ci,hid,out,outU,co,para,fs);
	if (mode==RoboGene) {
	  /* The predicted out vector should be sent to robot 
		 as next motor command */
	//SendNextSensorMotorState(out,OutN); 
	}

In this routin, you need to call a function of
GetCurrentSensorMotorState(in,InN)
by developing this function.
This function is going to put the current sensory-motor state in robot
into "in" vector of RNNPB.

Then, one step forward activation by SingleForwardAct(), you need to
call SendNextSensorMotorState(out,OutN) which is supposed to put
next prediction of sensory-motor state in "out" to robot.

Please not that here "in" is sensory-motor state at current step and
out is its prediction in next step.
Their dimension is the same.
In the current implementation, InN=OutN=2.
This means that one sensory scalar and one motor scalar.
If you increase this to 2 dimensional sensory vectror and 2
dimensional motor vector, you need to modify "DataDimG" defined in
rnn99.h as 4 (currently defined as 2)
In this case, the first 2 dimension of "in" or "out" correspond to
sensory state and nect 2 dimension does to motor state as like left
and right motor speed.
It is noted that sensory and motor values has to be converted to range
from 0.0 to 1.0.

Finally, how to get teaching pattern is obtained is explained.
"Robo" command can be used not only for generating robot action but
also for getting teaching data for training.
Under "Robo" when the robot is manually controlled by remote controller
if all sensory-motor state is scanned, all the sequence data is
recorded by GetCurrentSensorMotorState(in,InN) as have been described.
When "Robo" is finished, all the teaching sequence data is saved in a
file "****.pat" where *** is specified directory name under "data"
directory.
This ****.pat is copied into tmp/train.pat and it can be trained by
"train" command as have been described.





