Professional Basis of AI Backprop Hypertext Documentation

Copyright (c) 1990-97 by Donald R. Tveter

This is not the public domain version.

The Format Command (f)

There are several ways to format the printing of patterns and nodes in the canvas area and output patterns, numbers weights and other values. In all versions there is the typed f command that allows you to set one or more values on one line as in:

f b+ ir oc wB
In all versions you can type in the capital letter F and it will list the current values of each parameter. In the W95 version these are listed in a menu from which you can select items to make changes. In W95 you can also get to this menu by selecting the F entry in the Alt-C menu.

Input Patterns

The programs are able to read pattern values in two different formats, real and compressed. In most cases the patterns are given using real numbers separated by blanks but when the inputs are just 0 or 1 it is convenient to use the compressed format that does not require blanks between the digits.

Reading Real Input Patterns

Real numbers follow the C language notation and must be separated by a space. The letters `h', `i', `o', `H' and `X' that are used in patterns for recurrent networks are also allowed. There is also an `x' character, with an initial default value of 0.5. The value of `x' can be changed, for example to make `x' -1 use:

f x -1
Real input format is now the default but if you use the other format (a compressed binary format) you can re-set the format to real with:
f ir

Reading Compressed Input Patterns

The other format is the compressed format, a format consisting of 1s, 0s, the letter `x' for the unknown and the `h', `i', `o', `X' and `H' characters used in recurrent networks. In compressed format each value is one character and it is not necessary to have blanks between the characters. For example, in compressed format the patterns for xor could be written out in either of the following ways:

101      10 1
000      00 0
011      01 1
110      11 0

The second example is preferable because it makes it easier to see the input and the output patterns.

To change to compressed format use:

f ic

Output of Patterns

Output format is controlled with the `f' command as in: f or * output node values using real (f) format f oc * output node values using compressed format f oa * output node values using analog compressed format f os * output scaled real values f oe * output values with e notation The first sets the output to real numbers. The second sets the output to be compressed mode where the value printed will be a `1' when the unit value is greater than 1.0 - tolerance, a `^' when the value is above 0.5 but less than 1.0 - tolerance, a `v' when the value is less than 0.5 but greater than the tolerance. Below the tolerance value a `0' is printed. The tolerance can be changed using the `t' command (not a part of the format command). For example, to make all values greater than 0.8 print as `1' and all values less than 0.2 print as `0' use: t 0.2 Of course this same tolerance value is also used to check to see if all the patterns have converged. The third output format is meant to give "analog compressed" output. In this format a `c' is printed when a value is close enough to its target value. Otherwise, if the answer is close to 1, a `1' is printed, if the answer is close to 0, a `0' is printed, if the answer is above the target but not close to 1, a `^' is printed and if the answer is below the target but not close to 0, a `v' is printed. This output format is designed for problems where the output is a real number, as for instance, when the problem is to make a network learn sin(x). The format, "s" outputs real numbers and scales them up to the original range, the inverse operation of what the scale program does. For more, see the section on the scale program. The format "e" writes out node values using exponential notation with four places to the right of the decimal point. Breaking up the Output Values In the compressed formats the default is to print a blank after every 10 values. This can be altered using the `B' (for inserting breaks) option within the format ('f') command. The use for this command is to separate output values into logical groups to make the output more readable. For instance, you may have 24 output units where it makes sense to insert blanks after the 4th, 7th and 19th positions. To do this, specify: f B 4 7 19 Then for example the output will look like: 1 10^0 10^ ^000v00000v0 01000 (0.17577) 2 1010 01v 0^0000v00000 ^1000 (0.16341) 3 0101 10^ 00^00v00000v 00001 (0.16887) 4 0100 0^0 000^00000v00 00^00 (0.19880) The break option allows up to 20 break positions to be specified. The default output format is the real format with 10 numbers per line. For the output of real values the option specifies when to print a carriage return rather than when to print a blank. Pattern Formats There are two different types of problems that back-propagation can handle, the general type of problem where every output unit can take on an arbitrary value and the classification type of problem where the goal is to turn on output unit i and turn off all the other output units when the pattern is of class i. The xor problem is an example of the general type of problem. For an example of a classification problem, suppose you have a number of data points scattered about through two-dimensional space and you have to classify the points as either class 1, class 2 or class 3. For a pattern of class 1 you can always set up the output: "1 0 0", for class 2: "0 1 0" and for class 3: "0 0 1", however doing the translation to bit patterns can be annoying so another notation can be used. Instead of specifying the bit patterns you can set the pattern format option to classification (as opposed to the default value of general) like so: f pc and then the program will read data in the form: 1.33 3.61 1 * shorthand for 1 0 0 0.42 -2.30 2 * shorthand for 0 1 0 -0.31 4.30 3 * shorthand for 0 0 1 and translate it to the bit string form. To switch to the general form use "f pg". Another benefit of the classification format is that when the program outputs a status line it will also include the percentage of correct patterns based on the maximum value rather than just on tolerance. The values of 0 and 1 for output units are the default values and there is probably no reason to use any other values however there is a capability to change these default values to other ones. For instance with the tanh function where the output values run from -1 to 1 you may want the target values to be -1 and 1. To set the targets to -1 and 1 use: f t -1 1 Note that the output values are set when the patterns are read in so if you change the targets after reading in the patterns you must read in the patterns again. Controlling Summaries When the program is learning patterns you normally want to have it print out the status of the learning process at regular intervals. The default is to print out a one-line summary of how learning is going and this is set by using "f s+". However if you want to customize exactly what is printed out and you don't want the standard summary, use "f s-". To see how to customize the output of various values see the examples given in the "i" command. A third option is to set the summary flag to r to get statistics for recurrent classifications (see below). Recurrent Classification Network Flags In most applications you present the network with a single input pattern and you get a single output pattern, however if you have a series of inputs over time and you use a recurrent network and you want the network to classify the type of the input then you are more interested in getting answers based on the series of patterns than on each individual pattern. To get this type of result you must tell the program how many "minor patterns" there are for each "major pattern". The command to set this value to 128 for example is "f r 128". To undo this set the number of minor patterns to 1 with "f r 1". To get the network to print out results based on a series of patterns you must also set the summary flag to r by: "f sr". To undo this use "f s+" to get the regular statistics based on minor patterns or use "f s-" to suppress the summary altogether. Skipping the "running . . ." Message Normally whenever you run more training iterations the message, "running . . ." prints out to reassure you that something is in fact being done, however this can also be annoying at times. To get rid of this message use "f R-" and to bring it back use "f R+". Ringing the Bell To ring the bell when the learning has been completed use "f b+" and to turn off the bell use "f b-". Echoing Input When you are reading commands from a file it is sometimes worthwhile (especially when the program finds a mistake in the input) to see those commands echoed on the screen. To do this, use "f e+" and to turn off the echoing, use "f e-". Paging To set the page size to some value, say, 25, use "f P 25" or to skip paging use "f P 0". Making a Copy of Your Session To make a copy of what appears on the screen use "f c+" to start writing to the file "copy" and "f c-" to stop writing to this file. Ending the session automatically closes this file as well. If you use "f c+" when there is already a copy being made then the current file is closed and a new one is opened with the name copy and a numeric extension, for example, copy.2, copy.3 and so on. Up-To-Date Statistics During the ith pass thru the network the program will collect statistics on how many patterns are correct and how much error there is. It does this so that it will know when to stop the training. But it gets these numbers BEFORE the weights are changed in the ith pass. In the case of periodic update methods (the periodic, delta-bar-delta, quickprop and supersab) this is not much of a problem. If the off by 1 flag is off ("f O-") there is another forward pass done whenever the statistics are printed out so you get up to data statistics anyway. If the off by 1 flag is on ("f O+") you get the string "-1" after the number of iterations is printed on the summary line. Getting the statistics in the off by 1 form is harmless and it saves a little CPU time. When the network converges the "-1" flag will not be shown. Benchmarking results will be accurate. However with the continuous update methods the weights are changed after each pattern and this skews the statistics gathered by the training process by quite a lot. To get an accurate assessment of how well the training is going when results are printed on the summary line you either need to have the off by 1 flag set to "f O+" or you need to set the up to date statistics flag by: "f u+". The default is to leave this flag off: "f u-". Furthermore, if you are training to get an accurate assessment of how many iterations it takes to learn the training set you need to set "f u+" (NOT JUST "f O+"!). The "u+" setting makes a check after every complete pass through the training set. The "f O+" setting only makes a check when it is time to print the status line. Saving Weights Weights can be saved in several formats. First the r format saves the weights in an ASCII file, second the R format saves the weights and other weight parameters in an ASCII file, third, b saves the weights in a binary format and finally B saves the weights and other weight parameters in a binary format. The commands are: f w r f w R f w b f w B In addition the program can count the number of weight files written and assign each one a unique name. If the name of the weights files is "weights", the weights will be saved in files named "weights.1", "weights.2", "weights.3" and so on. This numbering capability is turned on with: f W+ and turned off with: f W-