At this stage we're completely done with the C code. It's been written, integrated into Yazoo, compiled, and that the whole thing works, at least in the author's hands. Let's now try our hand at writing a Yazoo script to `wrap' around our C routine.
NN.zoo
neural_network :: {
neurons_num :: ulong
inputs_num :: outputs_num
weights[*][*] :: double
activity[*] :: double
init :: {
hidden_neurons_num :: ulong
args_copy :: { ins :: outs :: hiddens :: ulong }
if trap( args_copy = args ) /= passed
print("usage: myNN.init(inputs, outputs, hidden neurons)\n")
return 1
endif
{ inputs_num, outputs_num } = { args_copy[1], args_copy[2] }
neurons_num = inputs_num + outputs_num + args_copy.hiddens + 1
activity[^neurons_num]
weights[^0][^0] | to speed up the resize
weights[^neurons_num][^neurons_num]
return 0
}
run :: {
args_num :: rtrn :: ulong
inputs[1] :: outputs[0] :: step_size :: learning_rate :: double
inputs[1] = 1 // the 'bias' input
code
args_num = top(args)
if trap(
inputs[^inputs_num + 1]
inputs[2, inputs_num+1] = args[1][*]
if args_num == 4
outputs[^outputs_num]
outputs[1, outputs_num] = args[2][*]
{ step_size, learning_rate } = { args[3], args[4] }
elseif args_num == 2
step_size = args[2]
else, throw(1)
endif
) /= passed
print("usage: myNN.run(input, step_size OR ",
"input, output, step_size, learning_rate)\n")
return 1
end if
if args_num == 2
rtrn = call("RunNetwork", weights, activity, inputs, step_size)
else
rtrn = call("RunNetwork", weights, activity, inputs, step_size, outputs, learning_rate)
end if
if rtrn == 1, print("run() did not converge; try lowering step size?\n"), endif
return 0
}
init(0, 0, 0)
}
We should save NN.zoo in same directory that Yazoo, start.zoo and user.zoo reside in. (The start.zoo script provides the default command prompt; user.zoo pre-loads the command prompt with a number of useful routines which are listed in the reference chapter.) Once our own script is in working order we'll be able to apply our network to something useful.
To begin with, we need to make sure Yazoo is running in interactive mode -- i.e. there should be a command prompt. (The only reason it might not run interactively is that we may have run Yazoo with an extra argument. If you just type `yazoo' or `./yazoo' then by default start.zoo's interactive prompt will pop up.) In order to use our new neural_network class from the interactive prompt, we will need to execute the NN.zoo script via the run() function.
> run("NN.zoo")
run(): Syntax error in file "NN.zoo" on line 33: unknown command
inputs[1] = 1 // the 'bias' input
^
Hmm.. we're obviously not finished with NN.zoo yet. Fortunately this is a type-I error (not at all a technical term): Yazoo wasn't able to even read our script and it flagged the line where it got confused. It is the scripting equivalent of a compile-time error. In fact, Yazoo even calls its initial preprocessing `compiling' even though it doesn't generate machine code. Type-I errors are usually the easiest problems to fix. We can see straight away what went wrong: we accidentally wrote a C-style comment `//' in place of a Yazoo comment `|'. Note that Yazoo somewhat missed the mark: the first thing it thought to complain about was the single-quote, which has no meaning it its language, rather than what looked to it like a double-division sign.
We make the straightforward fix to NN.zoo:
inputs[1] = 1 | the 'bias' input
and try running the script again.
> run("NN.zoo")
Error in file "NN.zoo": member 'outputs_num' not defined
3: inputs_num :: outputs_num
We have made progress: NN.zoo succeeded in `compiling'. Yazoo then tried running the script, made it part way (though it did print out a strange usage: message), but then impaled itself on line 3 and duly printed a complaint. Notice the different format of the error message compared with the first -- in particular Yazoo was only able to flag the line at which the problem occurred, not any specific word or symbol.
Runtime errors, which is the second type of bug, are sometimes more difficult to debug than type-I errors since they more often result from problems upstream. Fortunately, that is not the case here. With a little knowledge of the scripting language we would see that we tried to define inputs_num to be a variable of type outputs_num, but the latter has not been defined yet. What we meant to do was to define both of type ulong, so we need to make the following correction to line 3:
inputs_num :: outputs_num :: ulong
[ When we run the script again on an older version of Yazoo, we get a different error, this time at an earlier point in the code than our last error.
> run("NN.zoo")
Error in file "NN.zoo": type mismatch
1: neural_network :: {
Why didn't we get this error last time? The reason is that a type mismatch error happens when we try to redefine an existing object to a new type. Technically the definition of neural_network did change since we corrected the last error, but unfortunately Yazoo would have thrown this error even if we hadn't since it doesn't bother to compare the contents of the old curly braces with the new. The current version has a smarter run() function that will automatically deletes old variables, but under older versions we must manually remove neural_network by adding the following before the definition of neural_network (i.e. before line 1):
trap(remove neural_network)
Note that we only have to remove the outer neural_network class, since all of its own members will then disappear. If writing trap(remove ...) gets tiresome.. upgrade! ]
Let's re-run the script.
> run("NN.zoo")
usage: myNN.init(inputs, outputs, hidden neurons)
Well, we managed not to get an official Yazoo error. Though it's puzzling why we keep getting this usage statement, which is supposed to indicate that the init() routine was invoked with bad arguments. The strange thing is that we never tried initializing a neural network. This is evidently a type-III bug, the most insidious kind: it doesn't cause a compile-time or runtime error, but it makes the program behave oddly. Type-III errors are best debugged from the command line.
Let's see if init() works when we try to invoke it normally. Try
> neural_network.init(3, 4, 5)
>
So far so good(?). There should now be 13 neurons in our network (including the `bias' neuron).
> sprint(neural_network.activity)
{ }
So our init() call was a dud -- nothing happened. Did it even attempt to run? We might try put a trace statement -- say, print("A") -- somewhere in the coding section of the init() function. Hmm.. what coding section? And so we discover that -- aha! -- we forgot the code marker, which explains all of our problems. Add it after the second line of init(); the function should now begin:
hidden_neurons_num :: ulong
args_copy :: { ins :: outs :: hiddens :: ulong }
code
if trap( args_copy = args ) /= passed
Having made this change, we can go back and try:
> run("NN.zoo"), neural_network.init(3, 4, 5)
> sprint(neural_network.activity)
{ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 }
Which is exactly what we were hoping to see: an array of neurons, initialized to a resting state and ready to start processing.
Last update: July 28, 2013