So after a fierce battle with my own neurons, I am ready to release part II of my Processing series: “Neural Network! Huah! What is it good for? (Sing it again, now.)”
This example implements a multi-layered neural network that learns via “back propogation.” It’s specifically trained to solve XOR. In other words, there are two inputs and the desired result is input1 XOR input2.
0,1 –> 1
1,0 –> 1
0,0 –> 0
1,1 –> 0
The structure looks something like this:
However, I think there might be a flaw in my back propogation learning algorithm. For whatever reason, with the above neural structure, I can only successfully train my network (starting with random connection weights between -1 and 1) approximately 60% of the time. For the other 40%, the network gets stuck and can’t find the proper solution. If I add two more neurons to the hidden layer, like so. . .
. . . it trains flawlessly, finding a reasonable solution space after a few thousand training iterations 100% of the time (or at least as far as I can reasonably test.) What am I missing?
Anyway, a more involved tutorial about the theory, concepts, algorithms, and code behind neural networks is forthcoming. . . at some point. . . after I invent that machine that makes time that is . .
If you are downloading the source, note that the code for the nn.jar package is contained in /xor/code/src/nn. Because I’m using a large number of classes in the design of the network, I didn’t want to restrict myself to Processing tabs. Update (2/08/10): New download link: http://www.shiffman.net/teaching/nature/nn/