Karsten Schmidt’s Nokia Friends and Elie Zananiri’s Big Screams.

Both of these projects involve an underlying skeleton (Nokia Friends uses toxiclibs verlet physics and Big Screams uses Box2D) that serves as the structure for a cute, cuddly, jiggly creature drawn on top. In my efforts to expand the nature of code materials this semester, I’ve released a new example that demonstrates this technique. It constructs a skeleton of out Box2D bodies and distance joints and draws a curvy polygon with eyes and mouth on top.

Source: BlobSkeleton.zip
(Requires pbox2d: https://github.com/shiffman/PBox2D

void draw() {
  saveFrame("output/frames####.png");
}

This will create a folder called “output” in your sketch folder and for each frame of draw(), it will write a file — frames0001.png, frames0002.png, etc. The ‘#’ sign tells processing to auto-number the images.

There are a couple additional tricks included in this new example. First, it uses a boolean variable to turn rendering on and off and cycles the boolean in keyPressed() allowing recording to be stopped and started by pressing ‘r’.

void draw() {
  // If we are recording call saveFrame!
  if (recording) {
    saveFrame("output/frames####.png");
  }
}
 
void keyPressed() {  
  // If we press r, start or stop recording!
  if (key == 'r' || key == 'R') {
    recording = !recording;
  }
}

In addition, if you draw anything after saveFrame() it won’t actually appear in the output, but will be seen on screen. This is useful for debugging information that you don’t want included in the render.

void draw() {
 
  // Draw lots of stuff to be recorded!
 
  if (recording) {
    saveFrame("output/frames####.png");
  }
 
  // Draw our debugging stuff that won't be recorded!
}

Until 2.0 is released, I recommend you use 3rd party software to take the image sequence and turn it into a movie. The old Quicktime 7 will do the trick, as well as any number of video production applications like final cut, after effects, iMovie, etc. The nice thing about using a PNG sequence is that the images are uncompressed, but aren’t as large as say TIFFs. I don’t recommend saving JPGs because then you will likely be compressing twice (once when saving the image, once when exporting the movie file).

Source: SavingFrames.zip

This first example expands on the existing Recursive Tree example that comes with Processing.

Chapter 8: Recursion and Fractals

The recursive tree fractal is a nice example of a scenario in which adding a little bit of randomness can make the tree look more natural. Take a look outside and you’ll notice that branch lengths and angles vary from branch to branch, not to mention the fact that branches don’t all have exactly the same number of smaller branches. First, let’s see what happens when we simply vary the angle and length. This is a pretty easy one, given that we can just ask Processing for a random number each time we draw the tree.

void branch(float len) {	
  // Start by picking a random angle for each branch
  float theta = random(0,PI/3);  
 
  line(0, 0, 0, -len);
  translate(0, -len);
  len *= 0.66;
  if (len > 2) {
    pushMatrix();    
    rotate(theta);   
    branch(len);
    popMatrix();     
    pushMatrix();
    rotate(-theta);
    branch(len);
    popMatrix();
  }
}

In the above function, we always call branch() twice. But why not pick a random number of branches and call branch() that number of times?

void branch(float len) {	
 
  line(0, 0, 0, -len);
  translate(0, -len);
 
  if (len > 2) {
 
    // Call branch() a random number of times
    int n = int(random(1,4));		 
    for (int i = 0; i < n; i++) {	
 
      // Each branch gets its own random angle
      float theta = random(-PI/2, PI/2); 
      pushMatrix();     
      rotate(theta);
      branch(h);
      popMatrix();
    }
  }
}

The example below takes the above a few steps further. It uses Perlin noise to generate the angles, as well as animate them. In addition, it draws each branch with a thickness according to its level and sometimes shrinks a branch by a factor of two to vary where the levels begin.

TreeStochasticNoise.zip

Next up an excerpt from the Genetic Algorithm chapter.

Chapter 9: Evolution and Code

In 2009, Jer Thorp released a great genetic algorithms example on his blog entitled “Smart Rockets.” Jer points out that NASA uses evolutionary computing techniques to solve all sorts of problems, from satellite antenna design to rocket firing patterns. This inspired him to create a Flash demonstration of evolving rockets. Here is a description of the scenario:

A population of rockets launches from the bottom of the screen with the goal of hitting a target at the top of the screen (with obstacles blocking a straight line path).

Each rocket is equipped with five thrusters of variable strength and direction. The thrusters don’t fire all at once and continuously; rather, they fire one at a time in a custom sequence. In this example, we’re going to evolve our own simplified Smart Rockets, inspired by Jer Thorp’s. You can leave implementing some of Jer’s additional advanced features as an exercise.

Our rockets will have only one thruster, and this thruster will be able to fire in any direction with any strength in every single frame of animation. This isn’t particularly realistic, but it will make building out the framework a little easier. (We can always make the rocket and its thrusters more advanced and realistic later.)

Source: SmartRockets.zip

And here’s a short excerpt from the beginning of the chapter on neural networks, as well as the example that closes out the chapter demonstrating how to visualize the flow of information through a network.

Chapter 10: The Brain

Computer scientists have long been inspired by the human brain. In 1943, Warren S. McCulloch, a neuroscientist, and Walter Pitts, a logician, developed the first conceptual model of an artificial neural network. In their paper, “A logical calculus of the ideas imminent in nervous activity,” they describe the concept of a neuron, a single cell living in a network of cells that receives inputs, processes those inputs, and generates an output.

Their work, and the work of many scientists and researchers that followed, was not meant to accurately describe how the biological brain works. Rather, an artificial neural network (which we will now simply refer to as a “neural network”) was designed as a computational model based on the brain that can solve certain kinds of problems.

It’s probably pretty obvious to you that there are certain problems that are incredibly simple for a computer to solve, but difficult for you. Take the square root of 964,324, for example. A quick line of code produces the value 982, a number Processing computed in less than a millisecond. There are, on the other hand, problems that are incredibly simple for you or me to solve, but not so easy for a computer. Show any toddler a picture of a kitten or puppy and they’ll be able to tell you very quickly which one is which. Say hello and shake my hand one morning and you should be able to pick me out of a crowd of people the next day. But need a machine to perform one of these tasks? People have already spent careers researching and implementing complex solutions.

The most common application of neural networks in computing today is to perform one of these easy-for-a-human, difficult-for-a-machine” tasks, often referred to as pattern classification. Applications range from optical character recognition (turning printed or handwritten scans into digital text) to facial recognition. We don’t have the time or need to use some of these more elaborate artificial intelligence algorithms here, but if you are interested in researching neural networks, I’d recommend the books Artificial Intelligence: A Modern Approach by Stuart J. Russell and Peter Norvig and AI for Game Developers by David M. Bourg and Glenn Seemann.

In this chapter, we’ll instead begin with a conceptual overview of the properties and features of neural networks and build the simplest example possible of one (a network that consists of a singular neuron). Afterwards, we’ll examine strategies for building a “Brain” object that can be inserted into our Vehicle class and used to determine steering. Finally, we’ll also look at techniques for visualizing and animating a network of neurons.

Source: NetworkAnimation.zip ]]> http://www.shiffman.net/2011/12/27/night-7-nature-of-code-excerpts/feed/ 2 http://www.shiffman.net/2011/12/26/night-6-image-sequence-object-with-variable-speed/ http://www.shiffman.net/2011/12/26/night-6-image-sequence-object-with-variable-speed/#comments Mon, 26 Dec 2011 16:00:19 +0000 Daniel http://www.shiffman.net/?p=1036 Continue reading →]]> I have an example from Learning Processing which demonstrates how to package a “pre-made” animation (i.e. sequence of images) into an object in Processing so that it can be duplicated many times on screen. For tonight’s example, I’m going to make a new version that improves a few key points.

First, in the original example the the image files are loaded in the class itself. This is problematic. Sure, if you make one object then you are loading files from the hard drive once. However, if you make many objects, then you are loading the same images over and over again which is totally unnecessary (and can cause problems like using too much memory, stuttering if objects are made during draw(), taking too long to start up, etc.).

We can fix this by loading an array of images in setup() and passing it to the object.

Animation a;
 
void setup() {
  // Load the image sequence first!
  PImage[] seq = new PImage[40];
  for (int i = 0; i < seq.length; i++) {
    seq[i] = loadImage("stick/stick"+nf(i+1,2)+".png"); 
  }
 
  // Now when you make the animation object, you pass it the image array!
  a = new Animation(seq);  
}

The class then receives the array in the constructor and passes it to its own array.

class Animation {
  // The array of images
  PImage[] images;
 
  Animation(PImage[] images_) {
    images = images_;
  }

This way (as you'll see in the example) if we make an array of objects, each one uses the same array of images (which we loaded only once). Another feature of this improvement is that the Animation object is more generic, and can be created with any arbitrary array of images.

The original example demonstrated how to have the sequences start at different images so that they didn't all appear to be perfectly in sync. However, the question I usually get is instead: "How can the sequences play back at variable speeds?"

The original example used an integer to keep track of the current "frame" of the animation.

int index = 0;
 
void next() {
  index = (index + 1) % images.length;
}

Here, you see that we move one spot in the array each frame, and the animation is then shown at the frame rate of our sketch. So in theory, you could change the above to "index = index + 2" to, say, double the speed. A more flexible way to vary the rate of the animation, however, is to use a float for the index in the array, i.e.

float index = 0;
float speed = random(1,5);
 
void next() {
  // Move the index forward in the animation sequence
  index += speed;
  // If we are at the end, go back to the beginning
  if (index >= images.length) {
    // We could just say index = 0
    // but this is slightly more accurate
    index -= images.length;
  } 
}

Of course, you can't actually use this float when you go to look up an image in the array -- indices must be integers! So we simply convert it to an int when the time comes to draw the image.

void display() {
  int imageIndex = int(index);
  image(images[imageIndex], x, y);
}

Here is the example.

Download source: AnimationExample.zip

(This following is excerpted from the tutorial).

You are likely familiar with the idea of writing a program that follows a specific sequence of steps — setup() first then draw() over and over and over again! A Thread is also a series of steps with a beginning, a middle, and an end. A Processing sketch is a single thread, often referred to as the “Animation” thread. Other threads sequences, however, can run independently of the main “Processing” sketch. In fact, you can launch any number of threads at one time and they will all run concurrently.

Processing does this all the time, whenever you write an event callback, such as serialEvent(), or captureEvent(), etc. these functions are triggered by a different thread running behind the scenes, and they alert Processing whenever they have something to report. This is useful whenever you need to perform a task that takes too long and would slow down the main animation’s frame rate, such as grabbing data from the network (XML, database, etc.) If a separate thread gets stuck or has an error, the entire program won’t grind to a halt, since the error only stops that individual thread. To create independent, asynchronous threads, you can use the thread() function built into Processing.

void setup() {
  size(200,200);
  // This will tell someFunction() to execute now as a separate thread
  thread("someFunction");
}
 
void draw() {
 
}
 
void someFunction() {
  // This function will run as a thread when called via
  // thread("someFunction") as it was in setup!
}

The thread() function receives a String as an argument. The String should match the name of the function you want to run as a thread. While using the thread() function is a very simple way of getting an independent thread, it is somewhat limited. Being able to make a thread object is a great deal more powerful, and this can be done by extending java’s Thread class.

For more about how to do that, take a look at the full tutorial.

Following is an example draws a loading bar in the “animation” thread that reports progress on another thread(). This is a nice demonstration, however, it would not be necessary in a sketch where you wanted to load data in the background and hide this from the user, allowing the draw() loop to simply continue.

Threads.zip

Imagine for a moment, you have an ArrayList of PVectors called “vertices.” When the user clicks, the mouse you could add that mouse location to that ArrayList.

void mousePressed() {
  vertices.add(new PVector(mouseX,mouseY));
}

You could then draw that list as a polygon using beginShape() and endShape().

void draw() {
  beginShape();
  for (PVector v : vertices) {
    vertex(v.x, v.y);
  } 
  endShape(CLOSE);
}

But depending on how the user set the vertices, you might end up with:

when what you really want is the following:

One solution for solving this problem is to always sort all of the vertices according to their relative angle from the center. Let’s say you calculate the center of the polygon as the average location of all vertices.

  PVector centroid = new PVector();
  for (PVector v : vertices) {
    centroid.add(v);
  } 
  centroid.div(vertices.size());

You can then make a vector that points from the center to each vertex and get its direction (using PVector’s heading2D() method).

  for (PVector v : vertices) {
    PVector dir = PVector.sub(v, centroid);
    float a = dir.heading2D() + PI;
  }

Note how we add PI to the angle. The heading2D() function will return an angle between -PI and PI and it’ll be easier to sort if we just have an angle between 0 and TWO_PI. One way to sort an ArrayList is called a Selection Sort. In the example below, you’ll find a variation of the selection sort. The code iterates through the ArrayList, finds the element with the highest angle, removes that element and sticks it at the end of a new ArrayList. It does this again and again until the original ArrayList is empty. The result is a new ArrayList in sorted order.

Following is that example which implements all of the above into a class. If you are looking for an exercise, try allowing the user to move or delete existing vertices. Also, how you would make a system of these polygons so that the user can draw more than one?

Source: PolygonVertexSorting.zip

• Pattern — a compiled representation of a regular expression.
• Matcher — an engine that performs match operations on a character sequence (or String) by interpreting a Pattern.

An example of Pattern and Matcher in Java (which you can write directly into Processing) looks like the following:

String inputtext = "This is a test of regular expressions.";  // Input text
String regex = "test";              // The regular expression to match
Pattern p = Pattern.compile(regex); // Making a Pattern object with the regex 
Matcher m = p.matcher(inputtext);   // Matching that regex in the input string
if (m.find()) {
  System.out.println(m.group());     // Here's the match!
} else {
  System.out.println("No match!");   // No match!
}

Of course, in most cases, you want to do something more sophisticated where you iterate over many matches.

// Regex that matches double words
// Ugh, look at all these double back slashes!!
String regex = "\\\\b(\\\\w+)\\\\b\\\\W+\\\\1";   
 
Pattern p = Pattern.compile(regex);     // Compile Regex
Matcher m = p.matcher(content);         // Create Matcher
while (m.find()) {
  System.out.println(m.group());
}

Processing provides some regex helper functions that wrap all of this Java Pattern/Matcher stuff. They are match() and matchAll().

The match() function is used to apply a regular expression to a piece of text, and return matching groups (elements found inside parentheses) as a String array. If there is no match, the function will return null. If no groups are specified in the regular expression, but the sequence matches, an array of length one (with the matched text as the first element of the array) will be returned.

Here’s an example (this is straight from the reference page).

String s = "Inside a tag, you will find <tag>content</tag>.";
String[] m = match(s, "<tag>(.*?)</tag>");
println("Found '" + m[1] + "' inside the tag.");
// Prints to the console:
// "Found 'content' inside the tag."

The matchAll() function is at first a bit confusing because it returns a two dimensional array. But if you look right back to how match() works, it’s pretty simple. match() assumes you want just one match, and gives you an array, a list of all the groups for that single match. matchAll() assumes you want all the matches, so it gives you a bunch of those arrays, one for every match. What’s an array of an array? A two dimensional array! The first dimension is the match itself, and the second dimension is the group for that match, i.e.

String s = "Some tags! <tag>one</tag> <tag>two</tag> <tag>three</tag>.";
 
// Match this regular expression
String[][] m = matchAll(s, "<tag>(.*?)</tag>");
// Loop through all the matches     
for (int i = 0; i < m.length; i++) {
  // Print out group 1 for each match                
  println("Found '" + m[i][1] + "' inside a tag."); 
}

This new example uses a regex that matches anything inside an HTML href tag and draws it the screen.

Regex.zip ]]> http://www.shiffman.net/2011/12/22/night-3-regular-expressions-in-processing/feed/ 1 http://www.shiffman.net/2011/12/21/night-2-fade-sound-in-and-out-using-minim/ http://www.shiffman.net/2011/12/21/night-2-fade-sound-in-and-out-using-minim/#comments Thu, 22 Dec 2011 03:37:36 +0000 Daniel http://www.shiffman.net/?p=978 Continue reading →]]> I made this example earlier in the semester after seeing countless projects with the following functionality: turning a sound on when a sensor reading reaches a given level, turning the sound off when a sensor reading falls below a certain level. Most the projects used Minim for sound playback and pause() and play() to start and stop a sound, along with rewind() to send the sound back to the beginning. While this does work, I find a more effective strategy is to fade a sound in and out using shiftGain().

Now this is a fairly simple problem if you can pinpoint the moment a sound should fade in. For example, let’s say you want it to happen when the mouse is clicked.

void mousePressed() {
  player.shiftGain(-80, 13, 1000); 
}

The above line of code will fade the sound over 1,000 milliseconds (i.e. 1 second) from a decibel level of -80 (inaudible) to 13 (some vaguely loud level.)

If you put this code in draw(), however, you’ve got a problem.

void draw() {
  if (sensorVal > threshold) {
    player.shiftGain(-80, 13, 1000); 
  }
}

You can’t call shiftGain() over and over again! You want this to happen just once, the moment the sensor value reaches that threshold. Introducing a boolean is a quick way to solve this problem. If you set the boolean to true when the sound is fading and only call shiftGain() when the boolean is set to false, you’ve now got only one call to the function.

boolean fade = false;
 
void draw() {
  if (sensorVal > threshold && !fade) {
    player.shiftGain(-80, 13, 1000); 
    fade = true;
  }
}

The question remains: when does fade get set back to false? One likely solution is to reset the boolean when the sensor value falls below the threshold, i.e.

  if (sensorVal > threshold && !fade) {
    player.shiftGain(-80, 13, 1000); 
    fade = true;
  } else if (sensorVal < threshold) {
    fade = false;
  }

Following is an example that does this with a double threshold (fading up above a value and fading down below a value). In addition, it offers some other improvements (such as having the sound fade from its current gain level). The mouseX location is the stand-in for a sensor value.

Source: SoundFade.zip

First, you need to make sure you translate to the center of your window.

  translate(width/2, height/2);

Then you can use the scale() function to scale the world according to a percentage (i.e. 2.0 is 200%, 0.5 is 50%). We’ll use a variable called zoom.

  float zoom = 1.5;  // 150%
  scale(zoom);

Then we can translate additionally to pan according to some offset.

  float offsetX = 100;  // Some arbitrary offset
  float offsetY = 0;
  translate(offsetX,offsetY);

Here is an example (running in processing.js) that allows the user to pan around a design by dragging the mouse, and zoom in and out using key presses.

Source: PanZoom2D.zip

I have completely failed to remember I have a blog. Well, I suppose it wasn’t a failure of memory, rather it just fell out of my routine to keep things up-to-date. When you have one of these jobs at an academic institution, one thing they require you to do every so often is put together a big pile of paper that provides evidence you are contributing member of society. The way I’ve always managed to do that in the past is by scanning through all my old posts to find out if I have in fact done anything. According to this blog, I have not existed since April 2011. I’m going to try to remedy that. The first thing I’m going to do is post a list of things I’ve been up to in the last seven or eight months (this is for me and anyone reading should really just stop reading). The second thing I’m going to do (starting tonight) is simultaneously attempt to revive this blog and celebrate the festival of lights by posting eight nights of Processing examples. I’ve made a few new ones for the upcoming Processing 2.0 release as well as helped out students on a number of different problems over the course of the fall semester. So hopefully this can be a place where I can document and share some of these examples in the hopes that they will help others.

So first, the part everyone should feel free to ignore. . .

• April 2011. I visited my good friend Chris Ault at The College of New Jersey and gave a presentation about the Nature of Code and Most Pixels Ever and saw several demos of student work.
• June 2011. I attended Seb Lee-Delisle’s Creative JS course in NYC. I made this: fibonacci particles
• June 17-20 2011. I participated in the Processing 2.0 summit. Ben Fry, Casey Reas, Andres Colubri, Jer Thorp, Patrick Hebron, and I worked together to define the 2.0 release. (Thanks to Casey for the photos below.)




• Summer 2011. I participated in Google Summer of Code as a mentor for Processing.
• September 2011. Launched the “Magic Book” project with Rune Madsen and Miguel Bermudez at ITP (stay tuned for more about this).
• October 2011. I attended IndieCade in Los Angeles and participated in a panel discussion with Zach Gage, Robert Hodgin, and Casey Reas, and Daniel Shiffman. We discussed projects which use code as a generator of form, and explore the various ways in which procedurally generated elements can contribute to the mechanics, dynamics, and/or aesthetics of video games. (video of the panel).
• October 2011. Attended Art && Code 3D and was proud to be Greg Borenstein’s helper for his workshop Using the Kinect with Processing. Also really enjoyed Stephen Howell’s Scratch + Kinect workshop.
• November 2011. Began development on a CakeMix Processing library. Source on github.
• Fall 2011. Technical edited Greg Borenstein’s upcoming book: Making Things See.
• December 2011. Gave a presentation at the Neuberger Museum of Art as part of the New Media Lecture Series
• December 2011. Completing work on Processing 2.0. New examples, new tutorials. Stay tuned for further updates

.

Next up, 8 Processing examples!