What is Zambesi?
Zambesi is an experimental upgrade for the Frosh Artificial
Intelligence in Legend of the Greasepole. I implemented it as part of a project for CISC 864, a Neural Networks course.
Zambesi replaces the complicated logic the Frosh use to make
decisions with a Reinforcement Learning Network. This network consists of
neuron-like units that accept a series of inputs and generate an output that suggests what
the Frosh should do next.
The inputs Zambesi uses are simple: the Frosh's distance from
the Greasepole, whether or not they're being squished or stepped on, and how high up
they've climbed. This information is fed into the network of neurons, which in turn
outputs one of five behaviors: climb up, jump down, hold fast, walk towards the pole, or
waddle across the pyramid.
The process is called "Reinforcement Learning"
because it improves the network's performance by adjusting its neural weights. A
similar process governs learning in the human brain.
The algorithm used, called the associative reward-penalty
rule, strengthens the neural connections that result in success, and weakens the ones
that lead the frosh astray. "Success" and "failure" are measured
by a "critic" that observes the pole climb from a distance and provides an
evaluation of how well the Frosh are doing. The critic doesn't assign
"credit" (or blame) to any particular frosh; instead, it averages it
over everyone.
Imagine what would happen if pre-trained Frosh could return
year after year to climb the pole! Zambesi makes this possible, by letting us save
the neural weights as a sort of "genotype" to pass down information from one
Frosh to another. This genotyping is
a rudimentary version of the "reproduction" that Cyberlife's "Creatures" perform.
Why Call it "Zambesi?"
Mrs. Zambesi, a "pepperpot" lady in a classic Monty Python skit,
ordered a new strap-on brain from Curry's (a department store). The model she chose
was called the "Roadster." She even had to undergo a sort of reinforcement
learning technique while her brain was being fitted! The Python gang pronounced it
"Zam-Bee-Zee."
Where can I find out more?
A larger document about Zambesi is available here. You can also read about the first round of
testing here.
Can I try Zambesi?
You sure can! I've released a version that gives you a
general idea of how the Frosh look when they're under the influence of Zambesi. They
will save their weights between games, so that at least for the first few games, the Frosh
should improve from one game to the next.
Don't worry - it won't affect your original installation of
Legend of the Greasepole. Just install the original version first, upgrade to the
J-section version (if you'd like) and then install Zambesi. The original game will
work just the same, and you'll have the option to check out Zambesi. If I get the
chance to play with Zambesi some more, I'll release additional versions.
As the testing results suggest, the Frosh don't improve
indefinitely; in fact, they end up pretty volatile. I mean, if you had to
climb the greasepole 250 times, wouldn't you end up a little shaky? You can delete
the contents of the "weights" directory (*.wgt) if you want to
"reset" the frosh brains.
Click here to go to the
downloads section and get Zambesi!
Barto, Andrew G., "Reinforcement Learning" in The
Handbook of Brain Theory and Neural Networks (Michael A. Arbib, Ed.) pp 804-809.
Barto, Andrew G., "Reinforcement Learning in Motor
Control" in The Handbook of Brain Theory and Neural Networks (Michael A. Arbib, Ed.)
pp 809-813.
Burke, Robert C., The
Legend of the Greasepole Technical Documentation.
Cyberlife Web site, http://www.cyberlife.co.uk.
Haykin, Simon, Neural Networks: A Comprehensive Foundation
(1999) 2nd Ed.
Nolfi, Stefano and Domenico Parisi,
"‘Genotypes’ for Neural Networks" in The Handbook of Brain Theory and
Neural Networks (Michael A. Arbib, Ed.) pp 431-434.
