"DePo & TiPo" : Neural Net Models Found with Far Greater Evolvabilities

The CAM-Brain Project is based on the idea of evolving neural net circuit modules directly in hardware at hardware speeds (in a few seconds each), assembling 10,000s of such modules into a gigabyte of RAM, and interconnecting them according to the designs of human BAs (brain architects), to make artificial brains. The brain building machine that evolves the individual modules, is then used to update the signaling of the interconnected artificail brain in real time.

This approach will only work if the quality of the evolution of the modules is adequate (i.e. their "evolvabilities" are adequate).

Since the modules evolve directly in hardware, the constraints of state-of-the-art (1996) programmable/evolvable chips forced the neural net model implemented in the chips (Xilinx's XC6460) to be very simple, e.g. 1 bit neural signaling. The neural net model we used was 3D cellular automata based, called CoDi.ps, CoDi.pdf. The bit stream output was convoluted with a digitized bell-shaped function, to convert a digital signal to a (quasi)analog signal, whose wave form could be compared to some target wave form. Fig. 1 shows the quality of the evolution. We were not impressed by the results, but were not too worried, because we knew that the programmable chips could be reprogrammed with newer better models.

But the bankruptcy of Starlab killed the CAM-Brain Project, forcing us to start over on a second generation machine, which we call BM2 (Brain-building Machine, 2nd generation). As a prerequisite for building the BM2, my research team at USU needed to find more evolvable neural net models (i.e. with higher evolvabilities).

We tried a model we called DePo (delayed pointer) which definitely improved results. This model was then modified to TiPo which gave spectacular results. In fact, the evolved output curves were so close to the target curves, that it was difficult to tell there were two curves. See Fig.2. An experiment was performed to see whether reducing the number of bits used in neural weights and signals to 4, would unduly affect the evolvability. Fig. 3 shows the result. A 4 bit neural net model is probably implementable in the latest generation (2002) of programmable chips, which benefit from 6 years of Moore's law, compared with the chips of the CBM (1996).

Ongoing research in my Brain Builder Group is experimenting with more evolvable neural net models, and putting them into programmable/evolvable hardware (using Xilinx's Virtex family of chips). Once we have shown that it is possible to evolve successfully 10s of interconnected modules, we will push for funding to scale up to 10,000s of modules to build our first (the world's first) artificial brain.




Fig. 1 CoDi Neural Net Model Evolvabililty (ballpark but not accurate).




Fig. 2 TiPo Neural Net Model Evolvability
Overlap of the evolved and target curves is so perfect that they can only be distinguished at the upper tips.




Fig. 3 TiPo with only 4 bit weights and signals

This 4 bit TiPo model is probably implementable in Xilinx's Virtex 2 6000 (6 million logic gates) chip that our group has (June 2002). Stay tuned.