Backups

Backups are a sore topic. On the one hand, thinking about them is a painful and dull endeavor. On the other hand, not having a backup when shit hits the proverbial fan can be a disaster. The most important thing on a computer are your data. It’s not the machine itself. It’s not the operating system. It’s your files you will be worried about. Your work, your collected resources and work environment, your e-mail of the last decade or two, your photos and possibly your music collection or the playlists you created from it. Many of these things cannot be re-created if they were lost others would demand a massive effort to re-create which you wouldn’t undertake because it exceeds the ‘worth’ of the data in question.

Unfortunately many people aren’t aware of all this. They work with their data every day and spread it across machines and systems without much care. Parts of their e-mail will be at some web mail provider, other parts of their e-mail are still at their old web mail provider – each provider having a different state and subset of their address book. Phone numbers are stored in the mobile phone only. And documents reside in the folder the application creating them suggested as a default. The documents created after 2005, that is, because back then old computer with all the older documents on it was stolen…

Often people don’t worry too much about this, simply because they aren’t obsessive compulsive. But everybody seems to have weak points. Once the half finished dissertation or the wedding photos go missing because of a broken hard drive, careless operation or theft, panic strikes. And the count of people who become aware of backups increases.

Unfortunately, making backups is a bit of a hassle and thus of the few people who have a ‘backup strategy’ even fewer actually stick to it for a long time. It’s tedious, it’s boring and there is no day-to-day benefit from it as long as nothing goes wrong. Without ease-of-use and automation, backups don’t happen.

Once you start thinking about it you will discover many potential points of failure for a backup solution. Things you need are:

• Awareness of the need for backups.
• A mechanism that is easy and foolproof to set up.
• Automatically scheduled running of the backup that works reliably.
• The ability to read and recover your backup when your data are lost.
• Bonus: The ability to restore your data in a quick and foolproof way to a working condition. By Murphy’s Law you are going to be in a stressful tight-schedule situation when you need to do so.

None of these points is trivial to achieve. And – optimistically speaking – Time Machine is a brave attempt to address all these problems: Its mere existence raises awareness about backups. It creates and maintains backups automatically. It does so in a relatively simple way. And it makes restoring data easy.

Apple and Backups

All that said, Time Machine is also a product by Apple. And – as far as I can tell – Apple never went out of their way to protect their users’ data. The quality of their applications’ backwards compatibility varies widely (the OS X Address Book will export contacts [up to the photos associated to them] to the standard VCARD format, OS X Mail has no export feature at all, Mac OS X.5 doesn’t voluntarily play System 7 sound files, …). If Apple were self-confident enough to worry about their users’ data more than about locking their users in to peddle more machines, I’d certainly start off with a more positive impression.

While Apple’s software isn’t generally destructive, they have had their run of installers deleting users’ files and they also had the .mac Backup application gaining a reputation for not being something you’d want to entrust your data to. There were reports about people being not being able to restore data from backups (i.e. failure at its single most crucial function) a few years back. That seeds a bad reputation which is deadly for a backup tool – particularly for one by a tight lipped company which will neither admit errors nor document the changes they make in updates.

And from a good friend of mine I heard a possibly worse report about .mac Backup. He had a hardware failure of his hard drive. Which he wasn’t too concerned about because he had run the .mac Backup application all the time. He got a new drive and restored his data. Just to find that all his photos and iTMS purchases were there. But, the most essential data, his documents, were missing. Panic ensued and a large amount of money had to be spent for hard drive recovery to rescue his thesis.

Idiot! I hear you scream. Sure, he should have checked the recoverability of his data. But keep in mind that .mac customers are exactly the people more technical minded folks consider as ‘idiots’. If Apple tells them they have a backup and essential parts are missing at the end of the day, it doesn’t exactly build trust. It wasn’t possible to reconstruct how exactly that situation came to be. But even if he turned off backing up documents himself, the software should have told him about the implications. Just as it warns you before erasing a drive.

As a consequence of all this Apple will have to earn the trust needed for Time Machine as a viable backup software. And while a lack of actual experience with the technology as well as initial software problems still dominate the reports at this stage, I think they have a reasonable chance at succeeding in the long run.

Which I wish they do. Simply because it will help people’s data survive.

Background activity

There are two aspects to how Time Machine works: The user interface and the background activity needed to do the actual magic. Both of them are probably of equal importance as only a good user interface will entice people to create backups of their data and enable them to recover data which would be lost otherwise. But without the behind-the-scenes work nothing is going to happen. To keep up the good spirit, I will dig into the more obscure topic of background activity first before bitching about the user interface. This first part will be both lengthy and slightly technical. If you can’t take that skip it.

Ideas about what a backup is vary a bit. The most obvious – and very widespread – interpretation of the word is simply making a ‘clone’ of your hard drive which you update from time to time. Obviously that’s better than nothing but once you work with, or think about, backups, you soon realise that it’s really not enough. At least not if you want to be able to do more than just restore your system in the case of mass deletion or a hardware failure. Say, some data corruption happens to a file, but you don’t realise it at the time. You’ll run your clone ‘backup’ that evening and your only existing copy of the file’s previous version will be lost. Ouch! Suffer from this once and soon you’ll join the people who consider a backup to be an initial copy of all your data plus fresh copies of all files that have been changed since the previous backup run. Time Machine provides just that and I shall try to give a rough sketch of how that works (note for the overly optimistic: I have neither written Time Machine myself nor seen its code, so I can only go from observation of Mac OS X.5 and the documentation).

First, let’s look at the data storage: As things are, Time Machine requires a HFS Extended file system to work. This has been the default format for Mac hard drives for a long time, so it’s not a big deal. But if you don’t have a Mac OS Extended volume (but a drive shared by another Mac OS X.5 computer via Apple’s File Sharing or a drive in a different format), things will work as well as Time Machine creates a disk image in HFS Extended format and saves your data into that if necessary. In what follows I will just assume we are dealing with a HFS Extended volume.

At the top level of that volume Time Machine creates a folder named ‘Backups.backupdb’. And inside that folder there is a subfolder with the computer’s name – meaning you can back up different computers to the same Time Machine volume. Inside that folder there is a number of folders whose names are time stamps. These folders include the data of the drives Time Machine backed up at that time.

Hard links

Unlike many other backup solutions Time Machine doesn’t store backups in a single file backup database which may have an unknown format and may make it hard to extract data from it without the software that created it. Instead, Time Machine uses ordinary folders and files. That may be a bit less efficient space-wise but has the advantage of being simple as well as browsable on any machine that’s able to read HFS Extended volumes without Time Machine at hand.

Time Machine’s trick to avoid duplicates of unchanged files in its archives is to use hard links. Hard links are a very old and very basic concept. Yet they can be a bit mind boggling. When looking at how a file system works, you end up discovering that there are two things: The hierarchical folder structure and the files themselves. A folder essentially contains nothing but a bunch of references to the files themselves rather than the actual files: You can move even a huge file from one folder to another on the same volume in a split second because all that happens is that a reference to the file is inserted in the destination folder and it is removed from the source folder. The data themselves remain in the same place throughout.

In fact, that reference inserted to the folder is just a hard link. The new thing is just that once you have several such hard links to the same file, the file is listed several times in the file system’s folders but the actual data only exist exactly once. And that’s exactly what Time Machine uses.

Just to get a better feeling for this, let me answer a few questions I wished you had asked:

What is the difference between hard links and the ‘symbolic links’ Unixy people normally use?

The difference is that a symbolic link points to the path the file is located at. That is, it only points to the entry in the folder hierarchy rather than the ‘actual’ file. [Perhaps the image below or the next question clarifies this.]

Uh, what? You’re talking gibberish.

Just imagine the following situation: You have a file and a symbolic link to it. Then you delete the original file. As a result the symbolic link will point to nothing. And then you move another file to the same path that the original file was at. The symbolic link will then open that file. None of that could happen with a hard link. It always points to the same ‘actual’ file. Essentially it ‘is’ the file.

If ‘hard links’ are so great, why aren’t we using them all the time?

While they are great for many purposes, hard links also have a number of drawbacks. Most notably they can only point to things on the same volume.

How do Mac aliases figure in this game?

Mac aliases are more like symbolic links as they are separate files containing data about the location of the file they reference. However, aliases can contain information about the location of the ‘actual’ file and thus be much better at finding files even after they have been moved than symbolic links storing a path reference can hope to be.

One way in which Time Machine goes beyond what is usually done is that it makes extensive use of hard links for folders. These are usually avoided as they require very careful handling in situations when multiply hard linked folders are nested within another. But the feature is essential for Time Machine so it can just use a single hard link as a reference for a folder and everything it contains instead of having to create actual folders for everything and hard links for every single file in each snapshot.

Having multiple hard links to a folder turns out to be problematic once you start thinking about circular references and how deleting files should work: currently you can only remove a folder from the file system if it is empty; if you have multiple hard links to the folder it wouldn’t seem unreasonable to remove one of them without the folder being empty. But then imagine the only remaining link to that folder being from an item that is contained in the folder itself: You wouldn’t be able to reach that folder from the file system hierarchy anymore! In fact, the System’s ln tool doesn’t let you create hard links to folders yourself.

Perhaps a little demonstration will be a good way to finish this off. You can play around with the relevant commands in the Terminal without much effort, creating files, creating hard links with the ln command as well as soft links with the ln -s command. And keeping a close eye on the output of the file listing command ls -l. Its output contains a mysterious number after the initial permissions (e.g. drw-r--r--) bit. That number is exactly the number of ‘hard’ links to the file in question. Looking at these a bit you quickly realise that most files have a number of 1 and a folder has a number of at least 2 [the parent folder pointing to it and the folder pointing to itself – the ‘.’ item in the listing of the folder. Every additional item inside the folder will increase that number by 1].

[kalle:~] ssp% mkdir linktest
[kalle:~] ssp% cd linktest
[kalle:~/linktest] ssp% echo "test" > originalfile
[kalle:~/linktest] ssp% ls -la
total 8
drwxr-xr-x   3 ssp  ssp   102 17 Feb 20:25 .
drwxr-x---@ 41 ssp  ssp  1394 17 Feb 20:25 ..
-rw-r--r--   1 ssp  ssp     5 17 Feb 20:25 originalfile

Up to here we created a file originalfile containing the string test and looked at the folder contents. There is a link count of 1 for the file and of 3 for the folder. Next, create both a hard link and a symbolic link to that file:

[kalle:~/linktest] ssp% ln originalfile hardlink
[kalle:~/linktest] ssp% ln -s originalfile softlink
[kalle:~/linktest] ssp% ls -la
total 24
drwxr-xr-x   5 ssp  ssp   170 17 Feb 20:25 .
drwxr-x---@ 41 ssp  ssp  1394 17 Feb 20:25 ..
-rw-r--r--   2 ssp  ssp     5 17 Feb 20:25 hardlink
-rw-r--r--   2 ssp  ssp     5 17 Feb 20:25 originalfile
lrwxr-xr-x   1 ssp  ssp    12 17 Feb 20:25 softlink -> originalfile

And observe how the numbers change: The file with the hard link to it has an increased link count of 2. Finally, we delete the original file and convince ourselves that it survived thanks to the other hard link to it:

[kalle:~/linktest] ssp% rm originalfile
[kalle:~/linktest] ssp% ls -la
total 16
drwxr-xr-x   4 ssp  ssp   136 17 Feb 20:36 .
drwxr-x---@ 41 ssp  ssp  1394 17 Feb 20:25 ..
-rw-r--r--   1 ssp  ssp     5 17 Feb 20:25 hardlink
lrwxr-xr-x   1 ssp  ssp    12 17 Feb 20:25 softlink -> originalfile
[kalle:~/linktest] ssp% cat hardlink
test

Once such hard links come into play, you lose the sense of which one is the ‘original’ and which one is ‘just being pointed to’ – as both of them are the original. And thus, when browsing your Time Machine folders on the backup drive, all files will look like proper files and not carry the little superimposed arrow icon which Mac OS X normally uses to indicate aliases or symbolic links. But with the Terminal commands above you can hope to get an idea of what is going on.

When peeking around your backup you can learn a few things about it if the numbers are interpreted correctly. For example for files at the top level of your startup volume the number tells you in how many backups the file has been included. E.g.

[kalle:~] ssp% ls -l  /Volumes/Snowman/Backups.backupdb/Kalle/Latest/Kalle/ | grep mach
-r--r--r--@ 43 root  admin    616052  6 Nov 06:40 mach.sym
-rw-r--r--@ 30 root  wheel  10272820  6 Feb 01:15 mach_kernel

Suggests that the ‘mach_kernel’ file has been replaced 30 backup runs ago. The same heuristic doesn’t work as easily for folders as you’ll have to subtract the number of items in them before knowing the number of hard links around. Also be careful when examining folders lower down in the file system. You will frequently see link counts of 1 there. That happens when no item in that folder or in any folder contained by it has been changed. Then Time Machine only created a hard link for the enclosing folder and thus every file inside it has only been backed up and hard linked to once.

Extended Attributes

But there’s more to see for those who like poking around. Apple introduced extended attributes in Mac OS X.4 and Time Machine uses them a lot, both inside the backup copy and on your usual files. Mac OS X.5 comes with the xattr command line tool for reading and writing them.

Poking around the ‘Backups.backupbd’ folder you will find these attributes:

Machine Name – a top level folder with the name of the machine backed up, e.g. ‘Kalle’

• com.apple.backupd.BackupMachineAddress — containing a string of the Ethernet port’s MAC address on my MacBook and that of the Airport card on a MacBook Air .

Timestamp – a folder with the date of the backup it contains as its name, e.g. ‘2008-02-15-000458’

• com.apple.backup.SnapshotNumber – a string representing the index of the backup, increasing by one in each iteration.
• com.apple.backup.SnapshotVersion – I have only seen the string ‘1’ here.
• com.apple.backupd.SnapshotCompletionDate – sounds clear, though I haven’t tried figuring out the number format yet.
• com.apple.backupd.SnapshotStartDate – dito.
• com.apple.backupd.SnapshotState – gives the string ‘4’ for all snapshots I have.
• com.apple.backupd.SnapshotType – gives the string ‘3’ in the time range when snapshots are daily, ‘2’ in the time range when they are hourly (with a little glitch where daily and hourly meet) and ‘1’ for the latest snapshot. I’m guessing a bit here, to be honest.

Volume-Name

• com.apple.backupd.SnapshotVolumeFSEventStoreUUID – as the name suggests the UUID of the current fsevents store (see below); this matches the string in the file /Volumes/Volume-Name/.fseventsd/fseventsd-uuid.
• com.apple.backupd.SnapshotVolumeLastFSEventID – the ID of the last event reflected in the snapshot, not sure how to find that one in the files.
• com.apple.backupd.SnapshotVolumeUUID - the UUID of the volume in question, matching the one you get with diskutil info /Volumes/Volume-Name.
• com.apple.backupd.VolumeIsCaseSensitive – naturally 0 on sane volumes.
• com.apple.metadata:_kTimeMachineNewestSnapshot – a binary property list containing a Date as its root object. The date is that of the newest Snapshot containing the file or 1.1.4001 if the file is still present in the newest snapshot.
• com.apple.metadata:_kTimeMachineOldestSnapshot – a binary property list containing the date of the oldest Snapshot which contains (or once contained) the file.

Further subfolders

• com.apple.metadata:_kTimeMachineNewestSnapshot – as above.
• com.apple.metadata:_kTimeMachineOldestSnapshot – as above.

With all these extended attributes in place, Time Machine’s archive is more than just a copy of the files. It also contains guides to match the backup up with local data and to see where the previous and next versions of a file are stored.

Time Machine also uses extended attributes on your working volume to mark files and folders as excluded from the backup.

Exclusion

As Time Machine currently works, exclusion is a vital concept for it. The software defaults to backing up a complete volume and folders can be excluded from that to control the backup a little. Say, to ensure that huge but frequently changing files – like disk images from a virtual machine or a large database – don’t clutter up the backup too much. There are various ways files can be excluded from the backup. (Don’t read too much into the order of that list as I haven’t checked which of these items takes precendence in case of conflicts. That might be an interesting thing to do.)

StdExclusions.plist in CoreService’s ‘backupd.bundle’

This file lists many paths which are excluded from backups, both globally and in user folders. These are mostly folders for temporary files or files which can otherwise be recreated like the Spotlight index. As of Mac OS X.5.2 this file excludes the content of the folders /Volumes, /Network, /automount, /.vol, /tmp, /cores, /private/tmp, /private/Network, /private/tftpboot, /private/var/automount, /private/var/log, /private/var/folders, /private/var/log/apache2, /private/var/log/cups, /private/var/log/fax, /private/var/log/ppp, /private/var/log/sa, /private/var/log/samba, /private/var/log/uucp, /private/var/run, /private/var/spool, /private/var/tmp, /private/var/vm, /private/var/db/dhcpclient, /private/var/db/fseventsd, /Library/Caches, /Library/Logs, /System/Library/Caches, /System/Library/Extensions/Caches, the files and folders /.Spotlight-V100, /.Trashes, /.fseventsd, /.hotfiles.btree, /Backups.backupdb, /Desktop DB, /Desktop DF, /Network/Servers, /Previous Systems, /Users/Shared/SC Info, /Users/Guest, /dev, /home, /net, /private/var/db/Spotlight, /private/var/db/Spotlight-V100 and for each user account the files and folders Library/Application Support/MobileSync, Library/Application Support/SyncServices, Library/Caches, Library/Logs, Library/Mail/Envelope Index, Library/Mail/AvailableFeeds, Library/Mirrors, Library/PubSub/Database, Library/PubSub/Downloads, Library/PubSub/Feeds, Library/Safari/Icons.db, Library/Safari/HistoryIndex.sk. I knew you’d find that interesting.

com.apple.TimeMachine.plist in global preferences

This property list contains two items of interest: ExcludeByPath which seems to accumulate items like ‘Library/Calendars/Calendar Cache’ or ‘Library/Mail/Envelope Index’ for users that used them (why isn’t the first one included in the previous list? Why is the second one in both?) and SkipPaths which contains a list of all items the user dragged to the exclude list in Time Machine system preference pane.

extended attributes

Applications can also mark files as not-to-be backed up. Apple’s Backup Core provides an API for that. The effect that seems to have is adding an extended attribute “:EA:com_apple_backup_excludeItem” to the file with the content “com.apple.backupd”. A Spotlight query for com_apple_backup_excludeItem = "com.apple.backupd" should reveal all files excluded from backups in this way. Currently it seems to be mainly used for the iTunes Music Library.xml file.

While there certainly is a point to many of the exclusions made by Time Machine, I do find them slightly problematic. Particularly as many people will not know they exist and it is non-obvious how to find out which files will not be backed up. Even more so, as applications can easily remove your documents from future backups by accident (or, maliciously, on purpose). It’s a shame that there isn’t an easily accessible list of files which aren’t backed up. I guess the sufficiently paranoid may want to keep an eye on the property lists and attributes mentioned above.

A point that is rather sore as well for me is the fact that Caches folders are completely excluded from backups. By Spotlight’s poor design the Caches folder contains a Metadata folder which applications have to store their data in if they want it to be found in the index. Those metadata files will not be in a backup. As a consequence restoring your system from a backup will leave you with an incomplete Spotlight index until you run all the applications which stored data in the Caches/Metadata folder and make sure they re-create them. Ultimately this is a Spotlight issue, I think, but with Time Machine being made by the same company, they should have had an eye on it.

Spotlight

Time Machine also ties in with Spotlight. Items in the backup are indexed by the Spotlight engine to help you find stuff when needing it. The index for the Time Machine backup is stored separately from the remaining index of the volume, indicating that Spotlight has become a bit more flexible in this regard. I suppose separating things in this way may also improve performance for everyday searches in the non-Time Machine content of the volume.

An interesting aspect is that extended attributes beginning with com.apple.metadata can also be found in the Spotlight index. In particular, this gives you a way to look for all files added to the backup in a certain time range. So if you sometimes wonder ‘what the heck caused my last backup run to take so long and to contain that much data, Spotlight might be helpful.

If Spotlight’s graphical user interface weren’t a complete POS, this could even be fun. But currently it’s a bit painful to do. And given Apple’s past speed of improving the Spotlight UI, I won’t expect considerable improvements there before 2050. But still, we can attempt to do it. Just be sure to take this with a lump of salt. I have seen loads of strange behaviour when testing this. ‘Strange’ behaviour included the Spotlight server process mds crashing and happily starting a complete re-index after being automatically restarted by the System. It also included recently backed-up files not being in the Spotlight index and thus not appearing in the results for these searches. All that seemed a bit random, so while in principle possible, I wouldn’t rate these steps as reliable but more as an interesting exploration at this stage.

1. In the Finder, navigate to your machine folder (‘Kalle’ in my case) inside the ‘Backups.backupdb’ folder.
2. Use the Find command
3. Switch the scope of the search to the current folder This switch will ensure that the Spotlight index for the backup is used instead of the general one.
4. Add a ‘Raw Query’ search criterion (still localised as Reine Daten in German – whatever that is supposed to tell me).
5. Enter a raw search query string like _kTimeMachineOldestSnapshot > $time.now(-3600) into the field in the Finder window. [This didn’t work when I initially tested it and the text used to contain a more complicated way to create a query. Thanks to tinbert for pointing out that this does work after all and encouraging me to give it another try.]
5. Determine the number of seconds between an hour ago and the beginning of 2001. Unless you have something like an NSDate handy this will suck. Luckily thanks to Cocoa’s new ‘scripting bridges’ in Mac OS X.5, you tend to have an NSDate handy. So I invested two lines of Python to get this number [note that (a) I don’t know any python, (b) probably some smartass can do the same with less resources in bash or ed or whatever and (c) Google Calculator apparently can’t evaluate ‘seconds since 1.1.2001’]

   #!/usr/bin/env python
   from Foundation import NSDate
   print NSDate.timeIntervalSinceReferenceDate() - 3600
   

   [Download a version for the current time or one for an hour ago if you’re too lazy to copy and paste.]
6. Enter a raw search query string like _kTimeMachineOldestSnapshot > 225450653.661 into the field in the Finder window. Where the lengthy number has to be replaced by the number you determined in the previous step. This is yet another good opportunity to curse the people who thoughtfully ‘designed’ the Finder to not extend the width of that query text field to grow with the window’s width.

The Finder lets you save elaborate search queries like this one as a ‘smart folder’ for easy accessibility later on.

In case you want to edit it at a later stage you need a Finder window with a toolbar and an action/cogwheel menu in there. That menu contains a command for editing the search query. Thanks to Dirk and S for pointing this out.

If you are happy using the Terminal, running a command like

mdfind -onlyin /Volumes/Snowman/Backups.backupdb/Kalle/ '_kTimeMachineOldestSnapshot >  $time.now(-3600)'

may be more convenient in practice.

fseventsd

One thing that hasn’t been looked at so far is how the System knows which files need backing up. The traditional way of just scanning all the folders and subfolders on the attached drives would be quite an undertaking with hundreds of thousands of files hiding on a normal Mac OS X startup drive these days. Particularly if you want to update your backups hourly.

But luckily Apple gave the System the ability to just know when a file has been changed in Mac OS X.4 where Spotlight needed it to know which files require re-indexing after being modified. This approach has now been extended in a way that the fseventsd background process monitors all file changes on the system and. It notes the folders containing files which were changed on a volume in a file inside the .fsevents folder at the top level of that volume. A more detailed description of how that works can be found in the Ars Technica X.5 review.

For this to work reliably it is crucial that the fseventsd logs are complete as otherwise files could accidentally be omitted from a backup. And thus if you ‘improperly’ remove a drive, say, because of the system crashing or being too daft to properly unmount it, fseventsd creates a fresh catalogue of changes the next time you use that drive. In the logs that will look like this:

23.02.08 14:47:38 fseventsd[45] event logs in /Volumes/Bender/.fseventsd
    out of sync with volume.  destroying old logs. (2394 0 2433) 
23.02.08 14:47:38 fseventsd[45] log dir: /Volumes/Bender/.fseventsd
    getting new uuid: 6AF6C56A-3E96-4D99-89F6-BB60256E0C97 

backupd

Finally, there is backupd, the workhorse of Time Machine. It uses the information collected by fseventsd to zone in on the files which have been changed since the previous backup. If that isn’t possible because there has been a problem with fseventsd, that will be recognised [see that the fsevents store gets a new UUID in the log message above and that the extended metadata for the previous backup include the UUID of the fsevents store used] and kicks off a ‘deep traversal’ of the file system. In that case it will go through all files to ensure it is aware of all changes. In the logs that’ll look like this:

23.02.08 14:48:08 /System/Library/CoreServices/backupd[26227] 
    Event store UUIDs don't match for volume: Bender 
23.02.08 14:48:10 /System/Library/CoreServices/backupd[26227] 
    Node requires deep traversal:/Volumes/Bender 
    reason:kFSEDBEventFlagMustScanSubDirs| 
    kFSEDBEventFlagReasonEventDBUntrustable| 

Once the list of files needing backup has been collected, the process will copy them over to the Time Machine volume, creating all the metadata described above. And of course making sure there is enough space to accomplish the task before starting it. It also purges old backups when more space is needed on the backup volume as well as those which aren’t needed anymore (that is, hourly backups which are older than a day as well as daily backups older than a month).

To do its job particularly well, backupd will check for any files that may have changed during a lengthy backup run and add those files to the backup at the very end.

Weakness

All this sounds fine and dandy. But you really have to keep in mind how it works: fseventsd only notes which folders have been changed. Then backupd scans them for files which have changed. And backupd doesn’t make a big effort when doing that job. Two obvious tests for its cleverness are failed with ease.

The first test is renaming. Renaming a file only changes its name in the folder but not its contents. Thus – keeping in mind all the hubub about hard links above – I’d expect Time Machine to create a hard link with the new name to the same old file in the backup when running the next time after renaming the file. So let’s try just that. Create the file

[kalle:~] ssp% mkdir /Volumes/Bender/renametest
[kalle:~] ssp% touch /Volumes/Bender/renametest/file

Run the backup and have a quick look at it:

[kalle:~] ssp% ls -l /Volumes/Snowman/Backups.backupdb/Kalle/Latest/Bender/renametest/
total 0
-rw-r--r--@ 1 ssp  admin  0 23 Feb 15:07 file

The file exists, now rename it

[kalle:~] ssp% mv /Volumes/Bender/renametest/file /Volumes/Bender/renametest/file-2

Run a backup again and check once more:

[kalle:~] ssp% ls -l /Volumes/Snowman/Backups.backupdb/Kalle/Latest/Bender/renametest/
total 0
-rw-r--r--@ 1 ssp  admin  0 23 Feb 15:07 file-2

The file with the new name exists in the backup. But its link count is just 1. Thus it is a new file. The harm done by this varies. For a tiny file like this one it’s hardly worth mentioning. For a big video project where renaming may mean a wastage of many gigabytes of storage space just because it has been renamed, that may be more painful.

What is potentially even more harmful is the following behaviour: Time Machine considers two files to be identical if their path and dates and size are the same. I.e. it relies exclusively on metadata to determine whether or not a file changed, which ironically means it checks everything but the only relevant information. Luckily this isn’t too much of a problem in most practical situations, but it still strikes me as ironic. Having something like a hash of the file’s content could improve this a lot.

And although the last changed date of a file is supposed to change when that file is edited, that isn’t necessarily the case. (In fact, with the shambles that are metadata on computers the ability of GraphicConverter to not modify the last changed date of a file when saving it is convenient as it will not cause converted versions of an image to have a date that differs from the original file’s.)

And thus I can create a file (I used a 1 by 1 gif image for a simple test), back it up, edit it (with that file not changing its size and dates in the process), run the backup again and have only the first version of my file in the backup. If my hard drive breaks now, the changes to that file will be lost.

This may seem like a tiny point and a situation that shouldn’t happen. I think it’s good to have a clear sense of what the words we use are. And in the way Time Machine uses the word ‘changed’ it means that a file that has been moved to a different location has been ‘changed’ and that a file whose content has been replaced by other content of the same size in a way which didn’t modify its other metadata is not ‘changed’. As my usual understanding of the word ‘changed’ differs from that, it’s worth noticing and keeping in mind to avoid disappointment.

User Interface

Not last but least there is the graphical user interface for Time Machine. Like many recent GUI ‘innovations’ by Apple it leaves me with a ‘WTF What were they thinking?!’ expression:

While not the biggest Bauhaus fan, I am a ‘form follows function’ type. And that concept certainly didn’t play a role when ‘designing’ the GUI for Time Machine. There are so many things wrong with it that I don’t know where to start or how to make sense of it. So I’ll just give a semi-random list of points to think about.

• Shouldn’t tasteless Sci-Fi backgrounds be an exclusive feature of Linux distributions?
• If you need to have moving stars in the background image of your backup application, can’t I expect them to move smoothly no matter what I do? Particularly if I’m using a ‘supercomputer’? With an OS that has been ‘crafted’ to perfectly match the hardware? [This can be worked around using the _FXShowBackgroundAnimation hidden preference of the Finder.]
• Wouldn’t it be helpful to have a real timeline on the side? One with a meaningful scale?
• Wouldn’t it be helpful to see which files changed in a particular snapshot? (Rumours are Apple thought about that, but obviously they didn’t ship it. The Spotlight comments above point at how doing that could be reasonably easy.)
• Wouldn’t it be helpful to be able to see the development of a folder or file rather than just having a 3D arrangement of older versions of the window behind the front one, all of which are covered up so much that they are completely useless?
• If you start Time Machine from a lean Finder window without the sidebar, is there any chance that the user doing it may want to use the Find command? (The only workaround I currently see for that is quitting Time Machine turning on the sidebar in the Finder window and turning Time Machine on again – no kidding!)
• I am not a fan of contextual menus, but shouldn’t those work in the Time Machine environment as well when it pretends to be just like the Finder?
• Is a feature like the remove from backup command really ‘discoverable’ when hidden away in a cog wheel menu that may or may not be visible on screen.
• If you try to give users the illusion that they really are working with exactly the same window they fiddled with in the Finder while using Time Machine, wouldn’t it be a good idea to actually do this in a way that scroll bar positions remain constant as well?
• … and in a way that the Show Package Contents command works?
• If you need a full screen UI for restoring and you even hide the Dock when showing it, wouldn’t it be great if the ‘shadows’ of F11-Exposé went away as well?

… and so on. When playing with just the Finder’s Time Machine interface you quickly get the idea that some Sci-Fi geek had an UI idea which they then implemented but never really tried out. If you are using a large sidebared Finder windows in column mode, it will work all right for recovering a single file. In any other situation things quickly appear a bit broken and not exactly well planned and thought-through.

Other aspects of the UI such as the navigation arrows could also be better in that what happens when you click them would be predictable. Right now clicking the backwards arrow will slide things around but I have no idea how far they will slide before seeing it. Even if it slides just ‘the right’ distance, why not indicate that in the UI beforehand by omitting the superfluous layers or greying them out? It’s not really clear to me anyway what all those Finder windows actually represent on screen. Usually they do not represent the state of the file system at the time the window in question represents. They just seem to be a copy of the frontmost window and will only be updated when they come to the front themselves.

Time Machine also comes with two UI elements: An icon which appears in the Dock by default in X.5 (Apple’s tendency to clutter your Dock with stuff starts reminding me of the Windows ‘Start Menu’ and Desktop which are littered by every installer – Apple just have a better monopoly to get more exclusive spamming opportunities for themselves) and a menu bar item which was added in the X.5.2 update. It lets you run a backup and launch the Time Machine UI. And it rotates like hell, the wrong way round, while files are being backed up. (The icons the Finder displays in its sidebar next to the icon of the Time Machine volume rotate the right way round, btw.)

Restoring Files

The most important thing about a backup is that you can get your files back if you need them. As a Time Machine backup is nothing but a folder with files in it, you always have the ability to just browse into the Backups.backupdb folder in the Finder and get the file you want.

Or you can use the ‘official’ starry sky Time Machine GUI described above and grab the files from there if you have the nerves for that. You will even be asked whether you want to use the backup file to replace the newer version of itself or whether you’d like to have both files next to each other. With the files I recovered so far, both methods worked fine.

Both these methods use the Finder for copying the files. And the Finder will strip the extended attributes relevant to the backup from them when doing that. Which is probably a good idea. Copying the files using the cp command leaves the extended attributes in place and you’ll probably want to remove them afterwards.

The Mac OS X.5 installer also offers an option to restore the system from a Time Machine backup rather than setting up or migrating accounts. That’s a very appealing option if the whole hard drive breaks. I haven’t tried it myself yet but reports I read so far seem to be mixed. While no harm is done to the backup data, doing the full restore doesn’t seem to be terribly reliable yet. It may be a good idea to keep those cloned drives handy for a while if you need to be able to recover quickly.

Other applications

Another feature of Time Machine is that applications are encouraged to integrate with it. Some Apple applications like the Address Book, Mail or iPhoto offer such integration (and applications like iTunes, iCal or Safari lack it). But I fear that many people won’t even notice that as you have to invoke Time Machine while the application is active to get the Time Machine interface for that application. And there isn’t an out-of-the-box way to invoke Time Machine for most of those applications if you haven’t cluttered your Dock or menu bar with its icons. Only iPhoto seems to have an explicit menu item for accessing backups at this stage.

I haven’t really needed this so far, so I can’t say terribly much about it. The idea behind it seems to be a laudable one. As many applications store their data in an opaque format which consists of many little files and perhaps some database keeping them together, it’s not feasible to do fine-grained restores for these applications. You can either revert the full database to a previous state – undoing many other, wanted, changes in the same move or you can’t undo anything.

Letting the application access its old database and restore the little bits and pieces from it is a sweet idea. But I wonder how many applications will offer that. I am sure that doing so may be quite a bit of effort. And doing that in a correct way for the rare situations when a user both needs it and knows the feature is there, seems to be a lot to ask.

As things are, the integration with Apple’s non-Finder applications isn’t first class anyway. Not only is half of the date at the bottom of the screen covered frequently covered by a button on my ‘small’ MacBook screen (a problem so obvious that it suggests nobody at Apple ever bothered to think about this properly or to try it out on such a machine), the general behaviour also seems to be quite erratic (iPhoto windows showing up empty at first and only filling eventually after I scrolled back and forth a bit, iTunes playback skipping while the computer made that ‘effort’, the lack of displaying actual differences in the Address Book, to give a few impressions I collected when trying things out for a few minutes).

The integration with applications also makes me slightly uneasy. The beauty of standard Time Machine backups is that they are a-file-for-a-file. It’s easy to understand what goes on there. And recovering a file from the backup is essentially a copy operation. The more ‘clever’ behaviour applications show here means that the transparency of what is going on is lost. I will have to hope that the software does the right thing.

Preferences

Time Machine also comes with its own preference pane in System Preferences. It features a huge logo with the name beneath it in Myriad which doesn’t really belong to OS X. It also features a huge ‘slider’ On/Off switch that is in no way a standard UI element and in no way behaves like the real world slider it’s supposed to represent. It works like a checkbox, so why isn’t it a checkbox?

With space being so generously used for the ‘pretties’, little remains for actual preferences. You mainly get a display of the current backup status, some information about how backups work and two odd looking buttons which in open sheets containing the actual settings. Just get that: There are a mere five settings they let you make for Time Machine and you need a whole preference pane and two sheets for them.

In fact those sheets are quite anaemic. They are tiny and all they let you do is select the backup volume and exclude stuff from backups. The excluded items from various volumes are then displayed alphabetically in a single list with no sense of their location. So if you excluded a folder named ‘Pictures’, good luck finding out which one it actually is. You’ll have to hover over the line in the list to get a ‘tooltip’ with a Unix style path to the folder in question. Very user friendly and Mac like!

And that’s before I get into real bitching mode, my friends! Let’s say you have a hard drive and a single folder on that hard drive which you want to include in your backup. If you naïvely assumed that this should be a ‘super easy’ single drag-and-drop operation in Apple’s OS, you’d be wrong. As far as I can tell you are expected to add all other folders on the disk in question to the exclude list and of course update its state whenever you add another folder. Excellent!

Security

Security seems a topic which they mostly ignored when designing Time Machine. If you consider your data to be private, you can be screwed in two ways by Time Machine.

If you really are more concerned about your data remaining private than about it being unaccessible, you are probably using FileVault on your machine. It stores your home folder in an encrypted disk image (these started off with a reputation for easily being irreparably broken in system crashes or so – I heard the format improved since, but never really checked that). And you get plenty of inconvenience for that – such as your home folder not being available via file sharing and possibly a bit of a speed hit. Oh – and you will only get very limited Time Machine goodies as well. Because, rather than saving the files in your account, the computer will only make a backup copy of your complete encrypted disk image if you are using File Vault. To make that a bit more efficient, Apple introduced a new disk image format, the ‘bundle image’ which splits its contents up in several ‘slices’. Meaning that Time Machine only has to back up the slices which changed rather than the complete image. This isn’t exactly neat but – assuming that they got the encryption right – it will keep your data encrypted even in the backup. But your only option for restoring will be to restore the whole image in one go, rather than the individual files contained in it.

The other aspect is encryption of backups. Usually backup software offers it as an option. And it makes a lot of sense to me. Even if you say that someone has to break into your home to steal your data and that you don’t need local encryption because of that, it seems entirely possible that you would (or rather, should) store your backup somewhere else. Put differently: There will be another copy of all your data now which you rarely need to access. Thus the inconvenience of the encryption plays less of a role and it would be nice to have.

Unfortunately Time Machine doesn’t offer to encrypt backups. Which is a shame – it will use disk images anyway and encrypted disk images do exist, so it seems like this would be an easy thing to do. Of course that extra level of encryption would require a bit of careful design about where the password for the backup image is stored on a shared computer. And it could lead to agony if someone ran their backup for years and ended up needing it without remembering the password. It’d still be a nice option.

Once you start thinking about network backups – possibly storing your data on servers of some third party on the internet – which hopefully becomes an option in the future, encryption of backups would become pretty much mandatory. Call me paranoid, but at least I wouldn’t want all my data to be stored in clear text on their machines.

What Else

Even though this text is both wordy (for sure) and rich in information (hopefully), it still doesn’t present a full coverage of Time Machine. Mostly because I am running a single machine in a single environment which means that I am not able to know about the full range of usage cases, the opportunities, the conceptual problems, the practical problems. All I can go by is word of mouth. Where word of mouth isn’t even that but just what I picked up from the garbage dump known as the internet.

The impression I get from this – my limited experience spanning a few months and the reports I read – is the following: Time Machine is worth having because it does make an additional copy of your data and is unlikely to accidentally destroy that additional copy.

On the other hand there are still plenty of practical issues at this stage. Ranging from the – not necessarily Time Machine intrinsic – fact that OS X.5’s support for external drives could be much better performance-wise and even seems kernel panic prone (from my own experience). Then there’s the sub-par UI, both for setting up the backup (no way tell Time Machine ‘include this specific folder in future backups’) and for recovering files (which is both tacky and giving much less information than would be helpful and possible). I’d say that technically those UI points aren’t biggies. And perhaps we can hope for someone givng us a better UI for it. With Apple’s personality being a bit too far on the self-consumed and narcissistic side to admit they shipped something less than ideal or – gasp! – poorly designed, I wouldn’t hold my breath while waiting for them to improve things.

And then there are future questions, most of which were touched somewhere above: Quotas trying to fairly split backup space among several users? Dealing with small changes to large files (frankly, I consider Apple’s idea of simply splitting them up – as demonstrated for bundle disk images – to be a cheap hack rather than a good solution)? The apparent problems with applications like Aperture (no first hand experience)? Getting the ability to easily revert system updates you didn’t like with Time Machine? The numerous glitches which have been reported by people trying to do full restores (i.e. your data are saved but recovering them may not be totally fool proof). The whole issue of encryption. And in a next step file systems (there will always been someone mentioning ZFS which isn’t a reality yet but sounds like it might be a good match for Time Machine although it may require a number of changes to the concept). And in yet another step, networked file systems for backups to another location (with encryption being a must-have feature there). Which leads to more questions about which file server volumes Apple support (currently OS X.5 computers and Time Capsule devices by default and give or take a hidden preference for everybody who likes doing unsupported things with their backups). This is far less than they initially promised and it’s usually said technical rather than marketing issues cause the scrapping of that feature. Of course with Apple always achieving all their goals we will never learn what the real reason was. And while them limiting the features of their software just to shift more Time Capsule hardware would be a shame and really do injustice to a topic as important as backups, it wouldn’t seem completely along the lines of Apple’s ‘personality’ to do exactly this.

It appears that Apple’s mid-March 2008 Airport/Time Machine update at least enables volumes connected via their own Airport Extreme base stations to be used for backups.

Fin

So here we are – I hope you enjoyed the trip. Time Machine looks like a great step forward for preserving data. But we will have to keep an eye on Apple whether they are serious about it or not.

Want some Time Machine goodness?
[Buy at amazon .com, .uk, .de]

NAME

pictd — general helper tool

SYNOPSIS

pictd

DESCRIPTION

pictd is a helper tool for the Cocoa frameworks.

There are no configuration options to pictd. Users should not run pictd manually.

Strings like Unable to register connection for PICT renderer. or _NSPICTRenderer in the tool suggest that it is related to handling old-school PICT images. But it’s not really clear to me what causes the tool to be launched.

Just being curious…

I know that nobody asked me, and thus I will have to do that myself. During the interview I answered each of the following questions with a firm no

• Should the machine be sluggish just because Spotlight wants to re-index an external drive for whichever reason?
• Should Time Machine need an hour to backup 100MB to another drive at the same time?
• Should copying files be slooow while that is going on?
• Should typing in TextEdit stall for many seconds at a time just because I am copying files to an external USB drive and burning a DVD with data from another external USB drive?
• Should the mouse cursor stop moving for a full second just because the machine is busy updating its Spotlight index and I dare to use Quartz Composer at the same time?
• Have I been a bit optimistic about OS X.5’s ability to use arbitrarily high process IDs? (They seem to cycle round somewhere in the 100000 range.)
• Should my Airport connection break down a few times an hour?
• And generally be unreliable as it has been for well over a year? (Which I am fairly sure by now is an Apple software problem as I am seeing the same problem on another machine and yet non-Apple machines can use the network just fine.)
• Should the machine magically increase the current PID by a few thousand each hour without me doing anything to justify that?
• Should it be impossible for me to find out what is happening there? (Some process which quickly launches and then dies I suppose, but how can spot it if it’s short-lived?)
• Should Safari stall for 20 seconds just because I turn off ‘Private Browsing’?
• Should I need ‘Private Browsing’ mode at all?
• Should iTunes stall for 15-30 seconds every other time I try to interact with it? (This makes using it almost impossible, particularly for the obsessive-compulsive types who need to fix typos in metadata when they spot them.)
• Should I really be required to close all my applications and restart (which takes around 20 minutes until everything is up again) just to work around these shortcomings temporarily?
• And do I need to do this in the firm knowledge that the wireless networking problems will persist after that?

Unfortunatlely Mac OS X.5.2’s opinion to each of these question is yes. And, yes, compared to all these problems, my remarks about the menu bar icon for Time Machine rotating too obviously and in the wrong direction are utterly irrelevant.

Aaargh!

But before doing that let me just mention the newest nuisance they introduced with Mac OS X.5.2: A menu bar icon for Time Machine. That in itself is a good idea as it means you have the option of kicking out the space wasting Time Machine icon from the Dock. – And replace it by a space wasting one in the menu bar, that is. (Note that, as far as I can tell, there is no way to fully use Time Machine without one of these space wasting gadgets.)

But the icon itself is friggin’ silly. Not only does it rotate and distract you whenever the magic Time Machine does its wonders (thus ruining the whole beauty of Time Machine’s transparent operation), but the damn thing rotates the wrong way round! It rewinds the clock, thus giving the impression that, well, it’s rewinding your data to some previous state, i.e. erasing your recent work. Obviously I wouldn’t even expect Apple to be negligent enough to let such a bug slip through, but my first impression when seeing this was a shock of horror because the damn machine was deleting my data. I was about to shut down / kill / unplug things before I calmed down.

I’m afraid I remain a believer in boring design. No such excitement for me, please.

I suppose now we can try to turn this into a sport to figure out an expression which fools that cleverness…

While that is clever and suggests the patch makes an effort, I do wonder whether it actually is a good thing. It keeps you from trying out things by mathematically ‘commenting out’ variables in the equation that is being computed. If you do so, the corresponding inputs vanish. And the connections hooked up to those inputs vanish with them. Which means that restoring the previous state will require extra effort. Which in turn makes it less likely that you’ll try things out.

Another thing are loops. Some kinds of loops are easily done in Quartz Composer. With the Replicate in Space patch, for example. But other things just seem rather hard to get right and you quickly reach a complexity where you wish you had tried programming a custom patch with exactly the feature you want to begin with rather than fiddling with Quartz Composer’s built-in patches.

One thing I tried to do was the following: I wanted to monospace a string. That is, I wanted to throw in a String, and, as an output get an image where the string is written in an arbitrary font with the space used being the same for each character. In a way that should have been easy. But things didn’t quite work out as I wanted them to and I ended up with a monstrous composition for a limited length string instead. As a bonus it even contains a feedback loop (I have to draw all characters to find out which one is the widest and then I’ll have to draw all characters centred on that width). The composition seems to do the job but it’s just an awful ‘solution’:

When looking at compositions like this one you start thinking whether you couldn’t try to create compositions just for the æsthetics of their components…

Timelines

A great new patch which I didn’t see in my initial X.5 Quartz Composer review was the Timelines patch. It lets you define and edit a number of functions depending on the time graphically. While this seems odd at first, it can really take the pain out of trying to orchestrate some animation on screen where you want to have more than one thing that changes with time. Certainly beats writing a bit of JavaScript that spits out the correct numbers in dependence of the time.

As you can see in the screenshot, it’s fairly easy to orchestrate the time development of several values. You can even do non-linear developments using standard vector graphics editing techniques with tangent directions. The only ‘but’ here is that currently the editor still seems to be a big buggy and the Quartz Composer application can be easily crashed by dragging anchor points around too wildly. I recommend frequent saving of the file while using it.

Randomness

Quartz Composer’s random number generator always seems to begin with the same number. I wonder how random it actually is.

Quartz Composer Visualizer

I mentioned the cool Quartz Composer Visualizer application earlier on. It can be used to display Quartz Compositions across several tiled screens – even if they are attached to different machines. At least in its standard setup the Compositions are rendered separately on the different machines which the application controls. And each of them creates its own random numbers, thus making things inconsistent across screens for my example. No biggie, I suppose, but keeping me from using my composition with the Visualizer without extra effort.

Unfortunately, Time Machine and me were off to a bad start. Even trying to create the initial backup spectacularly failed on my machine and led to kernel panics time after time. Those panics pointed to some USB problem and somehow managed to screw up the Time Machine backup in a way that the software couldn’t recover from. I started to develop a number of theories about what exactly caused that problem but things couldn’t really be narrowed down. Something fishy with USB seemed to be the cause of the problem (with the panic log pointing right there) but that’s all I could tell. Whether it’s my MacBook, the cable, the hard drive enclosure or the hard drive itself that OS X.5’s buggy USB drivers trip over remains unclear to me.

After a few rounds of that, the machine finally managed to create its initial backup. I have no idea why and what the difference to my previous setup was but I just accepted that and things worked. After more than six weeks of no problems, OS X.5 decided to switch to its nasty side again, though, for the new year. Instead of wishing me a happy new one, the machine showed me a Time Machine error message about backing up having failed.

I didn’t mind much at that stage and assumed that the next Time Machine run would rectify things. Which it didn’t. Instead, I kept getting the same message hour after hour until I started being rather weary of that. The Time Machine preferences showed this:

and clicking the promising red information button in there, failed to provide any helpful information or even a way to fix the problem, it just displayed a rather pointless sheet containing no new information at all:

My first intuition was that perhaps the backup volume had run full and Time Machine wasn’t that good at clearing out old backups to gain extra space yet. Sure, there were still 5GB free on the volume in question and the amount to be backed up couldn’t have been more than a few 100MB. But the numbers I have seen Time Machine guess always seem to be a bit on the high side – which probably is a good way of playing things safely.

So I first moved some GB to another drive in the hope that this would magically solve the problem. Which it didn’t. As a last resort I decided to take a look at the logs. What I saw – hour after hour – typically looked like this:

Jan  1 00:04:19 Kalle /System/Library/CoreServices/backupd[52657]: Backup requested by automatic scheduler
Jan  1 00:04:19 Kalle /System/Library/CoreServices/backupd[52657]: Starting standard backup
Jan  1 00:04:19 Kalle /System/Library/CoreServices/backupd[52657]: Backing up to: /Volumes/Snowman/Backups.backupdb
Jan  1 00:06:26 Kalle /System/Library/CoreServices/backupd[52657]: No pre-backup thinning needed: 3.45 GB requested (including padding), 4.94 GB available
Jan  1 00:06:30 Kalle mds[52525]: (/Volumes/Snowman/.Spotlight-V100/Store-V1/Stores/6F049E25-9A76-49D6-ADC2-9C2478199143)(Error) IndexSDB in map_update:/Volumes/Snowman/.Spotlight-V100/Store-V1/Stores/6F049E25-9A76-49D6-ADC2-9C2478199143/store.db : ERR: map_update: page offset doesn't match! 0x1191 != 0x1189
Jan  1 00:06:30 Kalle mds[52525]: /SourceCache/Spotlight/Spotlight-398.1/core-db/sdb.c:1029: failed assertion 'dbme->pgnum == pgnum'   
Jan  1 00:06:32 Kalle ReportCrash[52661]: Formulating crash report for process mds[52525]
Jan  1 00:06:33 Kalle com.apple.launchd[1] (0x1109e0.mdworker[52531]): Exited: Terminated
Jan  1 00:06:35 Kalle com.apple.launchd[1] (com.apple.metadata.mds[52525]): Exited abnormally: Bus error
Jan  1 00:06:35 Kalle ReportCrash[52661]: Saved crashreport to /Library/Logs/CrashReporter/mds_2008-01-01-000631_Kalle.crash using uid: 0 gid: 0, euid: 0 egid: 0
Jan  1 00:06:36 Kalle mds[52662]: (/Volumes/Snowman/.Spotlight-V100/Store-V1/Stores/6F049E25-9A76-49D6-ADC2-9C2478199143)(Error) IndexCI in ContentIndexOpenBulk:Unclean shutdown of /Volumes/Snowman/.Spotlight-V100/Store-V1/Stores/6F049E25-9A76-49D6-ADC2-9C2478199143/live.0.; needs recovery
Jan  1 00:06:36 Kalle mds[52662]: (/Volumes/Snowman/.Spotlight-V100/Store-V1/Stores/50572971-97D1-41D8-9A8D-3A9A929EB339)(Error) IndexCI in ContentIndexOpenBulk:Unclean shutdown of /Volumes/Snowman/.Spotlight-V100/Store-V1/Stores/50572971-97D1-41D8-9A8D-3A9A929EB339/0.; needs recovery
Jan  1 00:06:37 Kalle /System/Library/CoreServices/backupd[52657]: MDBackupIndexFile returned -1134 for: /Applications, /Volumes/Snowman/Backups.backupdb/Kalle/2007-12-31-190422.inProgress/C52FE0AD-F4A1-4A55-8654-7A78144150E9/Kalle/Applications
Jan  1 00:06:37 Kalle /System/Library/CoreServices/backupd[52657]: Aborting backup because MDBackupIndexFile failed
Jan  1 00:06:37 Kalle /System/Library/CoreServices/backupd[52657]: Canceling backup.
Jan  1 00:06:37 Kalle /System/Library/CoreServices/backupd[52657]: Copied 113 files (0 bytes) from volume Kalle.
Jan  1 00:06:37 Kalle /System/Library/CoreServices/backupd[52657]: Copy stage failed with error:11
Jan  1 00:06:38 Kalle /System/Library/CoreServices/backupd[52657]: Backup failed with error: 11

While these messages weren’t quite as clear as I had hoped for, they point in the direction of Spotlight. Not only does Spotlight’s background mds process run into an error, it actually crashes over this with one of its threads ending up in a state like this:

Thread 15 Crashed:
0   com.apple.spotlight.index       0x00233580 _dead_beef + 11
1   com.apple.spotlight.index       0x00150ff1 map_update + 689
2   com.apple.spotlight.index       0x001577e1 page_insert_obj + 657
3   com.apple.spotlight.index       0x0015cfa2 db_store_obj + 546
4   com.apple.spotlight.index       0x00175b41 si_writeBackDBO + 412
5   com.apple.spotlight.index       0x0016d683 si_writeBackAndIndex + 1851
6   com.apple.spotlight.index       0x0016f23e setAttributes + 6676
7   com.apple.spotlight.index       0x001879bd si_work_queue_process + 637
8   com.apple.spotlight.index       0x00187cab si_scheduler_once + 419
9   com.apple.spotlight.index       0x00187cab si_scheduler_once + 419
10  com.apple.spotlight.index       0x00188036 si_scheduler_run_waiting_timeout + 581
11  com.apple.spotlight.index       0x00174a9e runLoop + 77
12  com.apple.spotlight.index       0x00174b8a setattr_runLoop + 68
13  libSystem.B.dylib               0x9296c075 _pthread_start + 321
14  libSystem.B.dylib               0x9296bf32 thread_start + 34

This left me non-amused. As Spotlight was totally and unrecoverably broken for me while running X.4, X.5’s improved Spotlight was a big deal for me. And mds crashes along with failed backups were pretty much the last thing I wanted to see in this brave new X.5 world.

My first attempt to solve the problem was to reindex the volume using the sudo mdutil -E /Volumes/Snowman command. As I also store loads of huge PDF files on that volume, doing this took ages. And it didn’t seem to solve the problem. Still having the possibly vain hope that I could fix things myself, I decided to have a look at the Spotlight index itself and possibly trash it (the results of which for some reason seem to differ a bit from what the mdutil command does).

Going to the interesting folder in question I found this:

[kalle:~] ssp% sudo ls /Volumes/Snowman/.Spotlight-V100/Store-V1/
Stores             VolumeConfig.plist

Of which the VolumeConfig.plist file was interesting:

[kalle:~] ssp% sudo cat /Volumes/Snowman/.Spotlight-V100/Store-V1/VolumeConfig.plist 
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>50572971-97D1-41D8-9A8D-3A9A929EB339</key>
    <dict>
        <key>partialPath</key>
        <string></string>
        <key>policySearch</key>
        <integer>3</integer>
    </dict>
    <key>6F049E25-9A76-49D6-ADC2-9C2478199143</key>
    <dict>
        <key>partialPath</key>
        <string>Backups.backupdb</string>
        <key>policySearch</key>
        <integer>3</integer>
    </dict>
</dict>
</plist>

While it is clear that the Time Machine backup has to be indexed by Spotlight to be searchable, this looks like Spotlight was actually refined a bit more carefully and can now create several indices per volume. In this case one for the volume in general and another one for the Backups.backupdb folder which contains the Time Machine backups. The UUID the index for the Backups.backupdb folder has is the one that also occurred in the log messages I had. So I decided to turn off Time Machine, delete the …/Stores/6F049E25-9A76-49D6-ADC2-9C2478199143/ folder, use the mdimport command to re-index the backup and turn Time Machine on again.

The re-indexing took quite a while. And the backup that eventually started included almost 10GB of files – of which I am not quite sure which they were (who can write a script that traverses the backup sessions and just lists the files which were freshly backed up in that session?) – but at least I seem to have a working backup system again at this moment. Let’s see how long this lasts.

Unfortunately actual information about solving Time Machine problems is still scarce in Google. In case you had similar problems, I hope this could help you. But in any case take care to not mess too badly with your data in case they are important. If you have a better understanding of what is going on, please leave a comment below or send a message.

Bonus Geek Material

Unfortunately I also ran into another kernel panic while trying to save my backup. The system was busy re-indexing, doing some further copying and playing a film. I also launched EyeTV. Shortly after that I got a kernel panic which points to EyeTV as the culprit:

panic(cpu 0 caller 0x001A7BED): Kernel trap at 0x0041e1bb, type 14=page fault, registers:
CR0: 0x8001003b, CR2: 0x0000000c, CR3: 0x00d4d000, CR4: 0x00000660
EAX: 0x00000000, EBX: 0x03d58d80, ECX: 0x03aab998, EDX: 0x36dde96c
CR2: 0x0000000c, EBP: 0x36dde8e8, ESI: 0x03d58d88, EDI: 0x03d58d90
EFL: 0x00010206, EIP: 0x0041e1bb, CS:  0x00000008, DS:  0x08d80010
Error code: 0x00000002

Backtrace, Format - Frame : Return Address (4 potential args on stack) 
0x36dde6d8 : 0x12b0e1 (0x455670 0x36dde70c 0x133238 0x0) 
0x36dde728 : 0x1a7bed (0x45ea20 0x41e1bb 0xe 0x45e1d4) 
0x36dde808 : 0x19e517 (0x36dde820 0x0 0x36dde8e8 0x41e1bb) 
0x36dde818 : 0x41e1bb (0xe 0x48 0x10 0x10) 
0x36dde8e8 : 0x41da73 (0x3d58d80 0x36dde96c 0x0 0x0) 
0x36dde938 : 0x41e3f4 (0x3c240c0 0x41e15e 0x36dde96c 0x0) 
0x36dde988 : 0x34597ff1 (0x3d58d80 0x0 0x36dde9a8 0x1a136f) 
0x36dde9f8 : 0x345ab616 (0x3c3b800 0x36ddead0 0x36ddea74 0x7) 
0x36ddea48 : 0x3459f3d3 (0x3c3b800 0x36ddead0 0x36ddea74 0x7) 
0x36ddea98 : 0x345aeca7 (0xc89d5c0 0x36ddead0 0x1388 0x0) 
0x36ddeaf8 : 0x345aea06 (0x4aba400 0x0 0xc0 0x0) 
0x36ddeb78 : 0x439b71 (0x4aba400 0x0 0x36ddec20 0x2) 
0x36ddebd8 : 0x4379b0 (0x4aba400 0xd 0x36ddec20 0x345bb1a4) 
0x36ddecf8 : 0x18d037 (0x4aba400 0xd 0xbbbda68 0x9) 
0x36ddfdb8 : 0x12d165 (0xbbbda40 0xc11e5a0 0x0 0x0) 
0x36ddfdf8 : 0x126247 (0xbbbda00 0x0 0x9c 0x36ddfee4) 
    Backtrace continues...
      Kernel loadable modules in backtrace (with dependencies):
         com.apple.iokit.IOUSBFamily(3.0.5)@0x34595000->0x345bcfff

BSD process name corresponding to current thread: EyeTV

Mac OS version:
9B18

Kernel version:
Darwin Kernel Version 9.1.0: Wed Oct 31 17:46:22 PDT 2007; root:xnu-1228.0.2~1/RELEASE_I386
System model name: MacBook2,1 (Mac-F4208CAA)

However, I thought that the beginning of that stack trace looks very similar to the panics I had in November, even though the version number of Apple’s IOUSBFamily stuff seems to have increased slightly since:

Sat Nov 17 18:51:07 2007
panic(cpu 1 caller 0x001A7BED): Kernel trap at 0x0041e1bb, type 14=page fault, registers:
CR0: 0x8001003b, CR2: 0x0000000c, CR3: 0x01138000, CR4: 0x00000660
EAX: 0x00000000, EBX: 0x03f50280, ECX: 0x0061e216, EDX: 0x344fbbdc
CR2: 0x0000000c, EBP: 0x344fbb58, ESI: 0x03f50288, EDI: 0x03f50290
EFL: 0x00010206, EIP: 0x0041e1bb, CS:  0x00000008, DS:  0x00000010
Error code: 0x00000002

Backtrace, Format - Frame : Return Address (4 potential args on stack) 
0x344fb948 : 0x12b0e1 (0x455670 0x344fb97c 0x133238 0x0) 
0x344fb998 : 0x1a7bed (0x45ea20 0x41e1bb 0xe 0x45e1d4) 
0x344fba78 : 0x19e517 (0x344fba90 0x216 0x344fbb58 0x41e1bb) 
0x344fba88 : 0x41e1bb (0xe 0x48 0x344f0010 0xba20010) 
0x344fbb58 : 0x41da73 (0x3f50280 0x344fbbdc 0x0 0x0) 
0x344fbba8 : 0x41e3f4 (0x3f54940 0x41e15e 0x344fbbdc 0x0) 
0x344fbbf8 : 0x612cfb (0x3f50280 0x0 0x1000 0x3eed800) 
0x344fbc68 : 0x6173dc (0x3eed800 0xb9ada00 0x9 0x406ac4c) 
0x344fbca8 : 0xa03135 (0x406ac40 0xb9ada00 0x7530 0x7530) 
0x344fbd08 : 0xa033b6 (0x4049200 0x40492e4 0x4 0x3f5db00) 
0x344fbd48 : 0x612350 (0x4049200 0x40492e4 0x0 0x0) 
0x344fbda8 : 0x9e5d85 (0x3eed800 0x3f5ec00 0x0 0x0) 
0x344fbe48 : 0x9e6724 (0x3eed800 0x3efd7b0 0x0 0x0) 
0x344fbec8 : 0x9e6779 (0x3eed800 0x0 0x0 0x408ea0) 
0x344fbee8 : 0x9df998 (0x3eed800 0x0 0x344fbf18 0x9dfa7c) 
0x344fbf18 : 0x41d149 (0x3eed800 0x0 0x1 0x19ccc1) 
    Backtrace continues...
      Kernel loadable modules in backtrace (with dependencies):
         com.apple.iokit.IOUSBMassStorageClass(2.0.0)@0xa00000->0xa09fff
            dependency: com.apple.iokit.IOSCSIArchitectureModelFamily(2.0.0)@0x89a000
            dependency: com.apple.iokit.IOUSBFamily(3.0.3)@0x60d000
            dependency: com.apple.iokit.IOStorageFamily(1.5.2)@0x8b2000
         com.apple.driver.AppleUSBEHCI(3.0.3)@0x9dc000->0x9edfff
            dependency: com.apple.iokit.IOPCIFamily(2.4)@0x63c000
            dependency: com.apple.iokit.IOUSBFamily(3.0.3)@0x60d000
         com.apple.iokit.IOUSBFamily(3.0.3)@0x60d000->0x634fff

BSD process name corresponding to current thread: kernel_task

Mac OS version:
9B18

Kernel version:
Darwin Kernel Version 9.1.0: Wed Oct 31 17:46:22 PDT 2007; root:xnu-1228.0.2~1/RELEASE_I386
System model name: MacBook2,1 (Mac-F4208CAA)

Not that this is terribly meaningful to me. I’m just wondering whether both of these problems have the same roots. And if they do, who is to blame. As I used to get the crashes without EyeTV running, my bet would be on OS X.

Looking into those files reveals some kind of stack trace which, strangely, seems to include all applications on the system rather than just the application which spun. Doesn’t that rise privacy issues of some kind if you are submitting the reports using the Apple’s Spin Reporter tool that comes up once a log has been generated (just as the crash reporter does). Similarly, doesn’t it raise questions about whether you should submit such a hang log to the developer of the application. I suppose that it might give them a reasonably good idea about where the hang happened in their code so it could be very helpful. But they don’t really need to see all other applications on my system, do they?

Could the problem I am seeing be related to the fact that all processes are considered UNKNOWN on my system and the sampling would be limited to just the affected process if the system managed to figure out its name (which, ironically, it does know and write into the log itself)? Does everybody have an UNKNOWN subfolder only rather than subfolders with the application names? And if the latter is the case, how can things be fixed for me?

In this entry I will discuss a number of iChat’s new features along with their abilities and weaknesses.

Screen Sharing

You know the situation: A parent phones and tells you that their computer ‘can’t print anymore’. Which could mean anything from a fuse having blown to some subtle software problem. Once you ruled out the cabling from being the problem you will have to start navigating the parent in question through the computer’s UI remotely. While doing that you will invariably learn that indications such as ‘the first button’ aren’t as clear as they may seem and you will also learn that having parents read screen contents to you over the phone is extremely inefficient and nerve-wrecking for both parties.

If you have been there and done that, it is immediately clear that screen sharing is an absolute killer feature to have. You won’t need those nerve wrecking phone calls anymore but your parent – if running a X.5 system – just needs to invite you to share their screen using the new button at the bottom of each of iChat’s contact list windows:

Or you could just invite yourself to use their screen. Depending on whether you invited someone to access your screen or try to invite yourself to sharing theirs the invitation’s recipient will see a matching window on their screen. A simple one if you invite them to access your screen:

And a more detailed and poorly localised one will appear if access to the screen has been requested from a potentially untrusted party. Duly it will note the dangers of what is going to happen and give you the option to bail out to a text only (but strangely not voice) answer.

Once the invitation has been accepted, the screen sharing connection will be established. You get a typical iChat window for connections that are being established while that happens. It sports a green version of what we should consider the screen sharing icon and doesn’t really tell you anything about the progress of what it does until the connection works or the attempt failed.

If you keep in mind that the complete contents of the screen need to be transferred for screen sharing, it is clear that it can take a moment for the image to be transferred across the network when the connection is established. Particularly so if you are on a slow network link, part of which is consumed by the simultaneous audio connection which iChat opens.

Later on only the parts of the screen which have changed will be transferred and that will be substantially faster than the initial transfer. However, you will not necessarily get a real-time experience from this – which is worth keeping in mind when using screen sharing. For example you can open a menu which will immediately display on your own screen but take a few seconds to be transferred over the network to the person looking at your screen via iChat. If you want to talk your parent through some steps you should make sure they see the menu before mentioning it as confusion ensues otherwise.

In case you have bandwidth limitations which mean that screen content transfer has quite a delay you will quickly learn to avoid the aspects of the Mac UI which change a lot of the screen. Invoking the Dashboard does, for example, as does any other large-area operation such as moving windows, opening long menus, or hovering over your Dock that has icon zooming enabled. On the up-side: If you have a fast connection, things will be rather smooth and convenient. And fast doesn’t mean crazy-fast here, but merely reasonably fast DSL.

To indicate that screen sharing is running, iChat displays a notice across the screen for a few seconds after the connection has been established. The rest of the time iChat’s menu bar item is exchanged for a red screen sharing icon every other second on the sharer’s side. This is likely all the feedback you will get about screen sharing being active and it may be even less if you don’t use iChat’s menu bar item.

While screen sharing is as exciting as the stuff you do with it, Apple also threw in a few nice extras. For example, screens are automatically letterboxed or scaled to fit the recipients screen size. Furthermore, multi-screen setups are shared without any additional effort as well.

The guest of the screen sharing session has a floating window with a miniature version of his own screen inside the shared screen and clicking that window will temporarily toggle the display, making his own screen large and usable again and putting the shared screen into the little window. Even better, the little window is drag and droppable for files as well as spring loaded. So you can switch to your own desktop, grab a file there, drag it atop the little window of the remote screen, wait a moment, enjoy the animation of the remote screen coming up on your machine once more and then drop the file on the remote desktop. The actual file transfer seems to use iChat’s normal file transfer mechanism and files can easily be moved back and forth in this way to work with them. The following screenshot displays a state in the transition between the local and the remote screen – they are swapped in a nice 3D animation which isn’t really captured that well in the still shot.

That’s pretty neat. And using the feature a few times will teach you that it’s a good idea to tell the person whose screen you are sharing about what you are doing because moving your mouse will move their mouse as well. Not just to make them feel safe and comfortable but also because they don’t see the little window with your own screen and thus they will see the mouse move to some location on their own screen for what looks like no good reason and they may even try to do something with the mouse, while you are trying to ‘escape’.

When it works, iChat screen sharing is an extremely useful feature. And I can see it as the best major new feature of Mac OS X.5 for many people. The only thing they left out is the ability to do bi-directional screen sharing. I.e. the ability to toggle whose screen is shared while a connection is running.

Sidenote: Better screen sharing

If you only have limited bandwidth and the lags you are seeing with iChat’s screen sharing are too long to tolerate or if iChat has problems establishing the connection, OS X.5 comes with an even better solution. At least if you are able to open some ports on your router. The alternative solution means that the person sharing their screen has to enable screen sharing in their machine’s Sharing system preference pane. Using the setting where anybody can ask for permission to share the screen worked well for me and spares the effort of sharing passwords. For this to work, the person sharing their screen has to make sure the relevant port for VNC connections (5900) is forwarded from their router to the computer itself – which limits the target audience for this method.

Once that is done you can easily invite the person you want to use your screen by sending them an URL of the form vnc://your.ip.addre.ss. Clicking that URL will launch the Screen Sharing application (which hides in the /System/Library/CoreServices folder and seems to have the code name Liaison) on the recipient’s X.5 machine and establish the connection to yours. My impression is that connections established this way are more responsive than those established via iChat. It also frees you to use an ordinary phone for the audio connection which frees up some bandwidth and comes without the hassles, poorer sound quality and possible delays of iChat based telephony.

Further advantages for advanced users are that you can significantly tweak the settings of the Screen Sharing application to suit your needs and speeds if you are so inclined.

iChat Theater

Another new feature of X.5’s iChat is what Apple advertise as iChat Theater. The feature lets you add a document to an existing audio or video connection (actual features may vary depending on the speed and OS version of the systems involved as can be looked up on one of Apple’s more byzantine web pages). What seems to happen there is that iChat uses Quick Look to create a preview of the document in question and then feeds that preview into a video chat. Which, in particular, means that you can also ‘share’ a document to someone using an older version of iChat via iChat Theater and that you do have features like live scrolling or video playback.

Unfortunately that’s about the best thing there is about actually using iChat Theater. Generally the image quality is very poor. Because of the trick they use by piping this through a video connection even reading normal sized text in a document is pretty much impossible. At the end of the day this will work for photos or films where fine details are not essential at all. But whenever a proper screen sharing connection along with sufficient bandwidth is an option you will be better off using that.

To me iChat Theater looks like something that started as a sweet hack. It has the further advantage that it is less scary than screen sharing and it has a bit of background infrastructure behind it which lets third party applications pipe their own output into an iChat Theater connection. That’s is rather cool idea. However my impression so far was that the image quality just doesn’t cut it. It’s too poor to actually use it for anything beyond showing a few rough photos. While this may improve with faster network connections, the very fact that the image transfer seems to come through a video stream whose size is constrained by iChat’s usual size limits, really makes it hard to see how things are going to improve significantly unless Apple significantly re-work iChat Theater’s underpinnings and sacrifice the backwards compatibility of the feature.

File Transfers

File transfers have also seen improvements in X.5. Apart from being neatly integrated with the screen sharing feature, file transfers are now easier to use when in a video chat as well. All you need to do is drag the file you want to send onto the video chat window. The window will then get an overlay with two parts which let you choose whether you want to transfer the file in question or whether you’d prefer using iChat Theater to just share it via the video stream. While this is somewhat non-standard UI, it seems to do just the right thing in just the right place without getting in your way otherwise. (It’s just impossible to discover for people who don’t happen to drag a file above the chat window, those will need to continue dropping files on the contact’s entry in the contact list or using some menu commands.)

File transfers themselves now come in a file transfer window – much like Safari’s – in which they are collected. That’s probably a good thing – formally. But to me it just looks like an extra window littering my screen. Files transferred via instant messenging usually are a fire and forget (or fire and give up, if the other party is using Adium) affair rather than something that needs careful managing.

Actually using the window quickly reveals that it has a number of shortcomings. For example a button to resume or restart a cancelled or failed transfer is sorely missing. As is an awareness of the window about the files it represents: As the window seems to store transfers from all old sessions until you manually delete them, some of the older files may have been deleted in the meantime. Yet, iChat still displays a clickable magnifying glass icon for those files. Clicking it brings the Finder to the front but – obviously – doesn’t reveal the file in question. About the only good feature of the file transfer window is that you can drag the transferred files right from the window itself by dragging the icon from the list – a feature that I’d warmly welcome in Safari’s downloads window as well.

A further subtle but useful improvement of file transfers is that when both the sender and recipient are using X.5, you can now send folders and bundles in file transfers. In earlier iChat versions you had to zip these things up before being able to transfer them which made things unnecessarily complicated. That said, it appears that this feature is still very new and iChat isn’t quite up to the job of properly judging a file’s type yet.

Multichat window

A great UI improvement of iChat 4 is that its text chat window now supports several chats at the same time. While previous versions of iChat littered your screen with a new window for ever single chat you had open, all these chats can live in a single window now once you clicked the correct check box in the Messages preferences.

Apple smartly decided against tabs at the top of the window in this case as you could hardly fit mor than three of those above a typical chat window and you’d furthermore lose the contact’s images in reasonably sized tab implementation. Instead, Apple went for a list at the side of the window which lists all the open chats with their images, names, and – neat feature – a speech bubble indicating whether they sent another message since you last looked at the chat in question. The first bit of the message is visible right there in the speech bubble. Very useful in my opinion.

Archiving

iChat has long had the feature to automatically save protocols of your chats in a designated folder. That’s useful for later reference. Unfortunately iChat’s strategy for this hasn’t been particularly clever so far: Essentially it just dumped a file per chat into your iChat protocols folder. Which quickly led to hundreds, if not thousands, of files with long names containing the name of the contact along with the date and time spelled out residing in that folder.

These file names depended on the state of your address book database as well as your preferred language at the time of their creation, which may be considered unfortunate but at least kept all files of chats to the same person neatly together in most cases (that failed if you hadn’t all IM contacts of a person added to your address book from the beginning, so this strategy is at best an ad-hoc heuristic).

Generally having thousands of files in a single folder could be considered unfortunate as well on the Mac. Not just because of the Finder but these days also because of Time Machine. If I understand things correctly, Time Machine has to create a hard link for each unchanged file in a folder whose have changed. As a consequence your backup gets thousands of extra hard links because of this. Bang!

X.5’s iChat tries to improve things for the saved files a little, but its solution seems equally thoughtless and ad-hoc: Each day it creates a folder with the name of the current day. Rather stupidly the format for that date is dd.mm.yyyy which prevents you from having these folders in a helpful order when sorted by name. And then the chats which started during the day are saved inside the folder using a name which still depends on the state of your address book, your iChat name display setting as well as your current language. So that’s hardly an improvement and I am far from impressed.

I suppose we wouldn’t need to worry about this if OS X.5 provided good facilities for searching for communication protocols on your machine. But currently that’s not the case. The only way you can easily search for a chat is to type some terms into the Spotlight menu. Searches with their default settings in Finder windows will not find iChat protocols and, likewise, searches in the Mail application won’t find iChat protocols either while iChat itself doesn’t even provide a search feature to begin with.

To me that’s a big thing and I think instead of integrating gratuitous crap like stationery, RSS or appointments into Mail they should have given it access to your instant messenging protocols. These are remarkably similar to e-mail in the form of content they represent. And it’s not rare these days that you receive a question by e-mail and answer in a quick IM later on (or vice versa) which means that Mail and iChat conversations should really be treated as the same thing and should be searched in one go. Unfortunately doing that is a rather laborious process in the Mac OS X’s current state.

Tidbits

There have been numerous smaller improvements of iChat as well.

Status save on quit

Quite possibly the best ‘little’ feature of iChat: Quitting the application now saves the current state of your chats and your message window will be re-opened in exactly the same state it had before when the application is re-launched. (Unfortunately the feature is buggy when iChat crashes: It will re-open all relevant chats when restarting but not display any of their texts. The saved chat protocols are not affected by this – which raises the question what exactly iChat does to implement that feature.)

Fast User Switching

iChat has been aware of Fast User Switching for a while. But it only offered options to log you out or set your status to inactive when switching to a different user. Finally, iChat can also do nothing when switching the user.

Animated Contact Images

You can have animated images for contacts now (possibly created using the new Photo Booth feature). And thankfully you can also tell iChat to not animate the images of your contacts. Which may be good for your sanity.

Auto log-in

iChat now has the option to let the iChat menu connect to the IM services automatically when you log in without further clicks or needing to launching the iChat application. Nice.

Insecurity

iChat still offers encrypted connections exclusively to people who pony up for their .mac offer. Even with certificates on your system which can and will be used by Apple’s Mail, iChat will not offer encryption features to non .mac users. Which I consider a bad thing. It means that Apple don’t want me to communicate privately with iChat.

Google Talk support

Just like in Mail, Apple added explicit knowledge about GMail Jabber accounts to iChat. Which means you can set these up without needing to worry about the technical details. Nice.

I shall spare myself comments on iChat 4’s even larger collection of graphical smileys and head to the unpleasant part of this review next. It’s the part about iChat’s ability to actually establish connections, about comparing it to Skype, about its UI, and about its Bluetooth capabilities.

Reliability

The great thing about a telephone is that it very rarely fails. As long as you have a proper phone it will ‘just work’ as long as it is plugged into the phone line and not overloaded with complicated crap. Your house doesn’t even need to have electricity for it to do its job. In comparison to that, iChat’s AV capabilities have always been on the limited and fragile side as you will need a working computer and internet connection to begin with and you can only communicate with people who have the same. Which narrows things down significantly. But at least iChat AV ‘just worked’ when I used it the first time more than four years ago. And its reliability, while not perfect, seemed decent in those days.

For reasons I don’t understand iChat’s ability to establish audio and video connections has become worse over time. I realise that iChat may have to jump through a number of hoops to establish such connections in a world of NATed mini networks connected to the internet. But you know what? First and formost, I don’t care. And secondly, iChat seemed to be able to handle the very same situation reasonably well in earlier versions.

The bottom line for me is that I frequently get connection errors when trying to start audio or video connections. In fact, I’d go so far that I say iChat’s connection error sheet is one of the UI elements in the application which I have become most familiar with.