X.5 Quartz Composer¶
2983 words on Mac OS X 10.5 Leopard
Quartz Composer is one of the newer graphics technologies in Mac OS X which was introduced in version X.4. It is quite powerful and has the limitation that it requires somewhat modern graphics hardware to actually do its job. Or at least to do it well. But apart from that it is quite amazing.
It is amazing because it gives people graphics tools and effects that are relatively easy to use, rather powerful and reasonably efficient. And people programming for Mac OS X.4 and higher can use all of that goodness right in their applications. Be it for silly graphics effects or as a small building block in bigger projects which even lets plugins use spiffy effects without requiring much effort, let alone programming.
While the Quartz Composer technology is available directly from Cocoa, it is more attractive to use the Quartz Composer application which comes with Apple’s developer tools to explore the technology and create your own compositions. While this somehow worked in X.4, there were a number of issues with Quartz Composer that somewhat spoiled the experience. The great thing in X.5 is that most, if not all, of both the Quartz Composer back-end and its editor application has been refined and many of these problems have been solved.
Patches
Quartz Compositions are made up of entities known as patches which can provide or process data or images. Quartz Composer always came with a number of useful patches. And reading through the release notes of X.5’s Quartz Composer suggests that not just the underlying graphics engine has been updated significantly but many of the existing patches saw updates as well. My favourite improvements are these:
- The Movie Loader patch now has an ‘Asynchronous Mode’ setting with the side effect that films can now be played back with sound and also via streaming. Consequently, the patch now also accepts internet URLs for the ‘Movie Location’ rather than just local file paths.
- The Image With String patch now actually works on Macs with the cheapo Intel graphics chips. Text just used to come out of that patch in a blurry fashion in X.4. Which I thought shed a rather embarrassing light on Apple’s skills at ‘crafting’ hardware and software that go along well. The problem wasn’t (and most likely will never be) fixed in Mac OS X.4, so if you want to use text in Quartz Composer with predictable results, you’ll have to require the use of X.5.
- The Mathematical Expression patch is a rather simple new patch that simplifies doing computations in Quartz Composer. Previously the Math patch would do the same but it was so clumsy to use that you either ran away screaming or just used the JavaScript patch instead to do your computations. Now the Mathematical Expression patch took that place. And it’s as simple as it should be. You type in the expression with variables and the patch automatically creates the needed inputs. Simple as that.
- Finally, there is also a Smooth patch now. If some of your data inputs come as integers, say, and you want to use them as parameters for rendering an image, the resulting animation most likely will be ugly and jumpy. We had that problem with our Sudden Motion Sensor patch. With the updated Quartz Composer you can just throw the Smooth patch in and things will become much more pleasant to look at.
An amusing sidenote is that the ‘Affine Transform’ patch has been renamed to ‘Image Transform’. While I like my affine transforms, the patch should have had that name from the get-go. And that isn’t to say that these are the only improvements. Just the ones I benefitted most from already.
Quartz Composer also comes with plenty of new patches. While some of them are ‘just’ useful technical additions, other add new features with a single click. The Apple Remote patch, say, which lets you easily add interactivity to your composition. Other new patches give you more convenient tools for drawing. While yet other ones like Grep or Watcher simplify the processing you may want to do. Finally, patches like the Network Synchronizer and its one way cousins or the GLSL patch look like they’ll open up a whole new can of worms which will hopefully give us new and mind-blowing compositions in the long run.
I don’t know much about graphics programming. But to me it looks like particularly that GLSL patch which lets you put in your own simple code for computing textures in the graphics card should enable people to really step beyond what was possible with Quartz Composer so far.
Editor
The Quartz Composer application which is installed with the developer tools has been greatly overhauled. The greatest improvement is that Undo support has been added. With most Cocoa applications which handle non-trivial data having Undo support, I always expected Quartz Composer to have that feature as well. Having good Undo support elsewhere means you get sloppy work habits and are tempted to try more things because you know that Command-Z will bail you out if necessary. Just that it didn’t in the old Quartz Composer. This made using Quartz Composer somewhat painful for editing and possibly prevented quite a bit of experimentation from happening (both in my experience and from what I heard from other Quartz Composer users).
Luckily the Quartz Composer application in X.5 solves the Undo issue and I predict progress in Quartz Compositions because of that. The application is full of other improvements as well. Ranging from a zoom feature for the now supposedly unlimited work area to (I think somewhat unreliable or at least overly cautious) compatibility testing features for old MacOS versions or bit-counts (64 bit performance doesn’t seem particularly convincing at this stage), to some new ideas for making the settings of patches more easily accessible right from the editor window by toggling the ‘Patch Paramters’ (Command-T). All much better than before, but still not perfect. Even with a MacBook size screen, it’s impossible to get a good view of both your workspace, the rendered composition and the settings, but at least there is progress over the previous version..
Another really nice feature is that Quartz Composer now offers commands to insert splitters into the already connected inputs and outputs of paths via the contextual menu of patches you work on in your composition. While this improvement will sound like esoteric gibberish to people who haven’t worked with the Quartz Composer application before, everybody else will sigh in relief for the number of fiddly mouse operations this command can save them.
Compatibility
I have to admit that I don’t know much about the internals of the Quartz Composer technology and its file format (well, it’s a property list). However, from using it, the impression is that it has been quite well designed. Even with compatibility in mind. Consider the following case: The Movie Loader patch has been updated to support asynchronous playback in X.5. And that feature won’t make it to X.4. So what do you do if you have a composition you made in X.4 and which would benefit from asynchronous playback when it’s available? Will you make a copy of that composition and create a new version that can use the new feature in X.5?
Or will you hope to live in the best of all worlds where you hope that you’ll just be able to turn on the new feature in X.5, save the file and the file will work just as it did before in X.4 but using asynchronous playback in X.5? Yeah, right, sounds pretty absurd to even expect that given the usual file format problems that programmers like to create. But it’s exactly what you can do in Quartz Composer.
I made that nice experience when doing ASCII Projektor. Of course I was rather keen on the asynchronous playback feature so I played around with it in X.5 and the file continued working in X.4 without forgetting the X.5-only setting in the saved file. I assume that this kind of compatibility (within reasonable limits) isn’t that hard to achieve from a technical point of view. But it certainly takes a bit of forward thinking and carful work by the people designing it. And in the case of Quartz Composer it looks like people did a good job. Cheers.
Talking of ASCII Projektor – it also gains the feature to accept URLs instead of file paths when running in X.5. So if you have a QuickTime file that is streamed, ASCII Projektor can convert that right away as well. In fact, if you have Perian installed and have a tool at hand to give you URLs for YouTube films, you could use this to play YouTube videos through ASCII Projektor right away. Does anybody know about a reasonably reliable Cocoa class (or Sven and Mac compatible C-function) that looks up the FLV file’s URL from a YouTube URL? With that I could make ASCII Projektor conveniently YouTube compatible in a future version.
All that said, backwards compatibility isn’t perfect. For example when you have a Composition created in X.4 and insert the JavaScript patch from X.5 into it, you seem to get zero results only. You’ll have to copy your whole composition to a fresh file to get things to work – which I found rather non-obvious.
Plugins
Another shortcoming of Quartz Composer’s initial installment was the difficulty of extending it. It just seemed natural to have a plugin architecture for new patches. And, indeed, an archaic plugin architecture was uncovered by clever people and a number of people used it to create their own patches which had to be ‘sneaked’ into the right folder, with ‘right’ being a moving target. Our own contribution to this was the ‘Quartz Composer in Sudden Motion’ plugin that makes the current acceleration values of a portable’s Sudden Motion Sensor available to Quartz Composer.
With X.5 Apple finally gave green light for plugins. It is now documented how they are to be implemented and that they should live in a ‘Library/Graphics/Quartz Composer Plug-Ins’ folder to be automatically used by the system. Unfortunately the plugin format has been changed from the old one. And thus an update for our Motion Sensor patch was due.
In fact, unlike in earlier OS X releases where Apple made creating the various types of plugins for their software unnecessarily hard by leaving it to you to set up an XCode project correctly, put all the right methods in there and gather the essential keys for your Info.plist file from different places in the documentation, this time they did a much better job: XCode ships with ‘stationery’ projects for pretty much any plugin type. These projects come with property lists that are properly filled, with UUIDs being present where they need be and with the essential methods pre-existing in the automatically created files, complete with a bit of explanation in comments. I think this makes a huge difference (and may be one of the best improvements in the developer tools). Because, all of a sudden, when having that twitch to ‘just’ try and do something, you can actually do the thing you want to do rather than wasting ages in a fight against documentation and the system’s bureaucracy.
When using Objective-C 2 ‘properties’ for the inputs and outputs of a custom patch as is suggested by Apple’s documentation, it seems advisable to declare these properties in an order that’s reversed from how you want them to show up on screen. Odd.
Examples
If you are interested in Quartz Composer – and even if you aren’t – I urge you to look at the /Developer/Examples/Quartz Composer folder on your drive (if you have Apple’s developer tools installed). It’s full with actual examples. Even non-trivial ones. That’s another great step towards making things easier and even fun to learn. I always wondered whether it shouldn’t be possible to write an image kernel for quick creation of fractals. But I never managed to do that. Either because of my lack of programming skills or because – as far as I can tell – image kernels won’t let you run loops.
But looking at the examples for the new GLSL patch in the Compositions/GLSL folder I found Mandelbrot.qtz which does just that. And it runs a loop. Kind of cool. Twitter lovers may want to look for the ‘Twitterverse’ in the examples for XML compositions (that one doesn’t work right out of the box, though).
Furthermore, compositions can be made available to the system and its applications by placing them in a folder named Compositions inside a Library folder. The one inside /System is full of compositions and you’ll even find a composition called ASCII Art in there which is similar to our ASCII Projektor in spirit.
I guess this is obvious (haha!) but I feel compelled to once more mention how irritating in general the concept of localised file names is. What good does it do to display the name of the folder Compositions as Kompositionen if the user’s preferred language is German? Particularly if that folder resides inside a folder whose name is the English Library even though other applications localise that word as Bibliothek or just thek. And how should I, as a user name the Compositions folder in my personal Library folder if it doesn’t exist yet? With the English or the German (or some other language’s) name? And how will the system display the name of that folder later on? And how can anyone even pretend that all this makes sense, restores a sense of childlike wonder or makes things easier to use or discover?
System-wide Compositions
In fact, those Compositions folders are more than just for examples. They provide compositions which all Quartz Composer using applications can easily access and use. For example you will find the video effects from X.5’s iChat or from PhotoBooth in the /System/Library/Compositions folder.
Even better, Quartz Composer now knows so-called Protocols which give information about the the tasks a given composition is capable of. And the Editor → Edit Protocol Conformance… menu item in the Quartz Composer application gives you easy access to setting up conformance to a protocol which will cause the relevant input and output patches to be created. These named inputs and outputs of a composition define the protocol conformance.
And using the ready-made settings you are very quickly on your way to creating a screen saver, an image filter, a transition effect or – most coolly – an iTunes visualiser. Quartz Composer just looked like the perfect tool for creating iTunes visualisers from day one. And finally it’s time to play in that area has come without needing iVisualise which stepped in in the meantime. Just save your newly created Quartz Composition conforming to the music visualiser protocol in a Compositions folder and iTunes will offer it in its Visualisations menu the next time it is launched (yeah, I guess we’d all prefer instant recognition for newly added compositions).
The same is true for iChat. Want to make a new fun effect for your video chats? Just create a Quartz Composition conforming to the Image Filter protocol, save it in a Compositions folder and see it appear before (and disarrange) the User Backdrop effects. Unfortunately PhotoBooth isn’t as well behaved.
Bonus
And as if all that weren’t enough, Apple also throw in the Quartz Composer Visualizer application (and source code). As the icon suggests, this is a nifty little tool that lets you play Quartz Compositions across several screens. This is done by enlarging the composition to cover the area of all your screens and splitting it up accordingly.
I tried this with my MacBook when an external screen was attached to it and the results showed both the potential and the limitations of this: The limitations are clearly that on the one hand the MacBook with its non existent graphics chips certainly isn’t good hardware for sophisticated filling of two screens with smoothly animated 3D graphics. The other limitation being that when you are using two screens for a composition that’s made for the aspect ratio of a single screen things are going to be cut off on the sides or at the top and bottom depending on how your screens are arranged. The potential is of course to imagine running, say, four displays on a sufficiently powerful MacPro and imagining what can happen there.
And there is much more potential: The Quartz Composer Visualizer application can also be run on several networked Mac simultaneously with one being the server and the others displaying things. I didn’t have the opportunity to try this out, but it sounds like it opens the door to even more goodness.
When I first saw that Apple advertised their iMacs in their stores by placing three of them in different sizes next to each other making them present a continuous animation, I did wonder how that worked. And I assumed they’d have some clever infrastructure behind it, hoping it might be Quartz Composer based. And whether or not this application is the thing they used back then, it could certainly do the same job. Way cool.
But use with care! It crash-restarted my MacBook twice while I used it. Which isn’t exactly what I expect from a modern Unixy OS. There wasn’t even a proper kernel panic.
[Screenshot showing the preview window of Quartz Composer Visualizer which displays one of the fantastic Futurismo Zugakousaku compositions.]
Summary
Quartz Composer has been significantly improved for X.5. Not just in a few points but pretty much everyhwere. It’s still not a technology for the unwashed masses, I guess, but being so easy to implement in applications, that more and more developers are going to take advantage of the technology.
Enjoy an additional X.5 only Quartz Composition, more notes and even more notes.
