Coming soon to a Processing library folder near you, I’m pleased to announce a new project I’m developing at ITP with Chris Kairalla, tentatively titled “Most Pixels Ever.”
“Most Pixels Ever” (not to be confused with “Best Pixels Ever”) is an open source Java framework for spanning real-time graphics applets/applications across multiple screens. The above video is a quick demonstration of the first prototype. Three client applications on three Mac Pros connect to six 32 inch LCD displays (each Mac has a dual video card, but this could just have easily work with 6 client machines). One of the Macs is also running a server application. The server tells each client about the master pixel dimensions of all the screens combined (here 8160×768). The client keeps track of its own location dimensions (say 2720×768) as well as its location with in master dimensions (say 5040,0). The server keeps everyone in line, making sure that frames are rendered in sync.
In theory, the system is scalable to whatever your network will allow. If you want to run 100 LCD displays off of 100 mac minis, you can (and I really hope you do.)
We’ve got quite a bit of testing and tweaking to do before we release the library to the public along with documentation and instructions. So stay tuned!
Thanks to Caleb Clark for the video.
You might be interested in Chromium… : http://chromium.sourceforge.net/ – its quite a nice implementation for things like this.
Does your system send the OpenGL scene data to all of the clients? Meaning, if on monitor ‘A’, there is a sphere moving left, does monitor ‘D’ have the sphere off of its clipping area ?
Oh God!
Last time i asked you whether I would make multiple screen applet. Then you made it!!!!!!!
Waiting for the library!!
Ah, what an uncanny moment. Almost serendipitous. I only just started looking into something like this a few *minutes* ago and — voilà , it pops up on Processingblogs. Are you reading my mind?
It’s not quite clear what you are actually communicating from the server to the clients. Is the server figuring out all the vertexes, etc, and the clients doing the actual pixel rendering? Or is it more pixel-based (which is appears to be, based on your post) in which case the server would seem to have to do most of the heavy lifting as it would have to generate the pixel array and send it over the network (maybe this is why you talk about network speed?). But I don’t see how you could get to 100 machines in that case.
I know there are other techniques than these two, and it will probably all become clear when you post the library. But I am curious already. That said, I can wait, especially since you read my mind so quickly!
The current system is incredibly simplistic. The server communicates nothing to the clients other than “go, go, go, go, go, go. . .” so that each frame is rendered precisely in sync. Each client runs an identical application, with one small caveat — each client knows about its local dimensions within a theoretical larger display and therefore renders the graphical elements translated based on its position within that larger world.
We have also implemented the ability for clients to send messages to each other (so that a mouse on one machine can act as a mouse for the entire system for example). In addition, the server can also broadcast information (such as sensor input, etc.) to all of the clients.
The biggest limitation stems from the fact that all the clients do all of the processing (no pun intended?) for everything. This makes converting a single screen applet into a multi-screen one really quick and painless, but limits how much the system can handle. We are trying to keep it generic and simple. . .but will be continually adding features as we move forward. . .
Ok, that’s pretty clear.
I was asking (I should probably reveal myself here) because I’ve been working different techinical configurations for a 6-screen cube projection. If you know anything about my work, I think it’ll be pretty obvious why I’m doing that. So that’s what brings me to your solution as a possibility. I’m also looking at some other multi-screen solutions, but this sounds pretty similar to a solution we came up with in my Atelier: using OSC to keep everyone synchronized, with each client doing their own thing. But I wasn’t happy with that from an architectural point-of-view.
The major problem with your solution is that it has to be coded specifically for it: i.e. the sketch has to made somewhat generic and a lot of care has to be made to make variables sharable, or for some main program to do all the positioning/collision calculations (or least the major ones) and then farm out the actual drawings to the clients.
At one point I had thought of creating a protoco on top of OSC that would allow objects to move “from” machine to machine and temporarily be “shared” across two sketches. I though that might be pretty cool because then the behavior wouldn’t have to be controlled by a central brain. A more abstract layer could then be placed on top of those individual objects, but would architecturally be two distinct layers. Are you going in this direction?
That said, the best thing is having a common-library. That’s a good idea as it will at least standardize significantly the process. Especially if you put the M. Shiffman stamp on it
This is exciting. There are some very practical applications of this such as large scale multi-screen sigange.
I do really wonder about the performance since each machine is doing all the work. It is easy to overload even a really, really fast box with seemingly simple tasks when you’re working at resolutions measured in the multiple thousands of pixels.
Regardless, I am eagerly awaiting the release.
The performance is definitely an issue. However, although each client is doing the computation for all the elements in the system, each client is only asked to do the rendering for its own portion. So performance-wise, it runs super-fast with certain types of applications — e.g. our early hello world bouncing ball demos. It’s no big deal for a client to compute the position of 100 bouncing balls (whether they are on a little screen or big screen) and render the ones that appear in its section.
Now, let’s say you wanted to do something grid-based where the grid is 16,000 by 800 pixels. You’ll have a problem asking each client to compute the entire grid. Or if you wanted to have 100 screens and 1,000,000 bouncing balls. We’re thinking along these lines too and will be experimenting with different solutions. . .
Daniel, this rocks and am glad this feature is soon available to Processing. Even if there’re performance issues, it’s a great tool for prototyping design ideas for digital environments. I’ve written a similar albeit more specific framework for a 28 screen responsive Shockwave3D retail environment. The MUS server in that system not only provided the heartbeat for the clients but also kept track of persistent data and events clients were generating themselves (3D geometry of realtime generated plants) and then distributed these to neighbouring machines (since plants can grow from one screen to the other) based on some configuration files.
My biggest issue in that project was to synch up all machines and to compensate for the slight discrepancies of system clocks and fluctuating network lag. What was working fine for 4 machines in our test rig, didn’t initially quite easily scale up in concept to the 28 machine setup in the shop. Am very interested in finding out how you solved the issue in your library. Mine involved normalizing everything to Unix timestamps and a constant re-adjusting of client’s local time offsets… Not easy considering there’s no built-in unixtime support in Lingo.
You ARE the man!
[00:00:30] good effort! weldone!
Hello Dan. For OpenGL apps, are you sharing world-view variables across the system so that 3d views can be utilized, or is this primarily a 2d tool for the time being? I’m imagining how useful this could be in giving volume to an installation space. Say, for example, you have eight screens which you position into an octagonal structure. If you could set up an openGL scene where you tell each screen to zoom out a certain amount from the origin and rotate 45(* monitor number) degrees, then you could give volume to an openGL scene.
Not much in terms of shared computing, since everyone would more than likely be viewing the same information, but it would be from different views.
Again, this is more of a question, I can’t tell from the videos if this is strictly 2d or not. Looks HOT regardless, especially when I was able to sneak a peek in person on the floor last week.
Currently, the system only works for 2D. However, it can (and will be) extended for 3D environments. The issue is positioning the point of view / camera properly so that the perspective matches from screen to screen for a 3D environment. glfrustum to the rescue!
[00:00:00] whats the green thing going right across the room?
[00:00:00] hold on… maybe thats just my strange video plugin. i guess no one else can see the green thing, right?
That’s amazing !
I can’t wait for a 3D version.
Without meaning to be unpleasant, is it possible to add a value to sync the server about the thickness of the monitors’s edges (according to the frameRate() value) ?
Because, i’ve seen a little TC advance on the sketch’s motion when it is passing from a screen to another.
Dear Daniel,
I’ve seen the MPE video is very nice, and congratulation for this project thati like to try it.
How can i get the open source Java framework for your MPE project , did you make the processing Library?
Charles
It’s not available yet, but we hope to be able release a working version soon!
That looks fantastic. I can’t wait to try it in processing — this is a real service to the community. Thank you.
[00:00:30] Brilliant stuff.
hi, i’m a student of Douglas in Aix-en-Provence, France.
I’m going to use mpe for a projection into a cupola. we use 4 video projectors.
Are there any extensions of mpe which permit simple transformations of each screen…to handle vernier adjustments? thank you.