• Root (271)
  • digest (139)
    • debate (1)
    • economy (12)
    • education (5)
    • freedom (7)
    • freespeech (18)
    • idiots (9)
    • media (26)
    • music (1)
    • politics (29)
    • religion (1)
    • society (2)
    • stumbleupon (6)
    • tech-sci (3)
  • fun (22)
    • videos (7)
  • ideas (6)
    • epistemology (5)
  • meta (6)
  • misc (11)
  • news (36)
    • media (5)
    • politics (25)
  • personal (12)
    • florin (5)
    • lupo (6)
  • series (4)
    • complexity (1)
    • ponerology (3)
  • tech-sci (35)
    • comp (4)
    • medicine (28)
    • psychology (1)
    • quantum (1)

RSS feed

GPG/PGP pubkeys

How NOT to defrag a MacOS X partition

author: www-data

As most of you who know me probably already know, here's how my computer is set up: 35 GB for a MacOS X partition, 25GB for a Linux partition, roughly 15 GB for an exchange data partition. Well... that changed. I just trashed MacOS X trying to defrag the partition.

The MacOS X (non)fragmentation myth

"But HFS+ doesn't need any defrag, MacOS X does everything automagically" is bullshit. HFS+ partitions do fragment, period. What Mac OS X does is reducing the file fragmentation, if (a) a file is less than 20GB in size, (b) the system has been up for more than 3 min and (c) the file is accessed. Else, it does not. Oh, and MacOS X does not do anything about disk fragmentation. BTW, the difference between the one and the other is that when a file is fragmented, then the file is scattered all around the disk, instead of being in one continuous piece. When a disk is fragmented, then the disk more or less like a piece of swiss cheese: there are files all over the disk. Even if every one of the individual files may be defragmented, i.e. in one piece each, they are still badly placed. The problem is then, that although your df -hm says you have 10 GB of disk space left, the biggest free disk portion is merely few 100 MB...

But back on topic: MacOS X disks not needing defragmentation is a myth. The true core of the myth is that, due to the automagical semi-defragmentation of MacOS X, it takes quite a while until the fragmentation of the disk becomes an issue -- much longer than in the case of a FAT or NTFS system. But sooner or later, even on a HFS+ partition, fragmentation will become an issue. This is mostly the case when the drive is filled up to 70-80%, as lots of people on the net reported. (Apple's answer to that is "buy a larger HDD" :). Or, as in my case, if you wish to shrink your MacOS X partition to create some more space, a fragmented disk can also effectively veto that.

The fragmentation issue

I (used to) have a MacBook with 4 partitions:

• 200 MB EFI partition (needed by "them" :)
• 35 GB HFS+ MacOS X partition
• 25 GB ext3 Linux partition
• 15 GB HFS Linux/MacOS X data partition

The 4th partition I typically mounted to ~/shared/ from both MacOS X and Linux. I had pretty much everything I work on (projects, diploma thesis, docs, dotfiles) on that ~/shared/ partition. I symlinked dotfiles and dotdirs (like .gnupg/ and .muttrc) to their correct locations in either operating system... The idea was to be able to do my work from whichever OS I booted. And it worked quite well, with few limitations (like having to be careful with permissions, and having to boot MacOS X to fsck the HFS partition whenever something went wrong.)

The partition table is a GPT/MBR hybrid (google that one up for yourself, I'm not going to explain what it means right now...). Few weeks ago I stumbled upon a blog entry stating that a GPT/MBR hybrid with more than 4 primary partitions was possible. And: yes, it is :-) The whole point of the "MBR" part in "GPT/MBR hybrid" is for the BIOS-emulation-based boot loader of whatever OS you're trying to boot (Linux+Grub, in my case) to be able to actually load the OS from that partition. But this boils down to: you can have as many GPT partitions starting from 5 on as you wish to... you just cannot boot any BIOS-based OS from them :-) But since Linux starting with more recent versions (2.6.28 for sure, 2.6.26 maybe) can read GPT partition tables directly, I can use the whole beauty of the GPT table at will to store data on partitions beyond the magic limit of 4.

And all I wanted was a swap partition for linux. So, here was the deal:

• shrink the MacOS X partition by 5 GB (30 GB remaining)
• create a 5 GB swap partition (number 5) in the newly freed space
• load linux -- like used to -- from MBR partition 3, and then, after booting, activate swap GPT partition no. 5

The command

should have done the trick. But instead it stopped with an error: "No space left on device". And that was it: the point where things started to turn for the worst. It's like one of those moments, where you know you should stop whatever you're doing, or else... Well, I didn't, so here's the "or else":

I looked up the error for a while, and leadned a lot about MacOS X and HFS+ fragmentation. Ultimately, I found two tools: iDefrag and ShowVolumeFragmentation. The former is a commercial (as in "costs money" :-) defragmentation tool, the latter is a shareware/freeware tool based on hfsdebug. From the ShowVolumeFragmentation tool I learned that from approximately ~7 GB of free space, more than 5 were in pieces smaller than 10 MB, and about 40% of my complete free space was in chunks smaller than 1 MB. I had no piece of disk larger than 100 MB...

Well, Apple fanboys, how's that for an "MacOS X automagically defrags your disk"? (Why're there so many clueless morons with an oppinion out on the streets nowadays?... )

So, despite the existence of a perfectly working (albeit somewhat expensive) defragmentation tool for HFS+, I did what any geek with a root password would do when defragmentation time has come: booted from external disk, tar gzip-ed the whole MacOS X tree into safety, and reformated the partition.

Biiiig mistake.

For whatever reason (I don't know and I don't really care to know -- I assume it has something to do with funny meta-data of MacOS X), after re-populating the newly formatted partition, the OS didn't come up any more. It hung at trying to boot some basic tool (mDNS* stuff), and then eventually gave me a paniced system root shell without a login name -- just enough to for me to type "reboot".

So, folks, this is how not to defrag a MacOS X HFS+ partition. Instead, go buy iDefrag. As about me...

Sticking with linux

The good part about this story is the timing: prior to trashing my MacOS X install, I had just finished installing my Linux on the 'mac in such a way that everything worked practically out of the box: WiFi with WPA2 encryption (with the ath9k driver of the 2.6.28 kernel, using the knetworkmanager), accelerated graphics (works with debian lenny out of the box), suspend to RAM (need to rmmod/insmod appletouch with 2.6.26, works flawlessly with 2.6.28), Huaiwei USB UMTS modem... :-) The only reason I went through the MacOS X defrag hack in the first place was to be able to have a swap partition for s2disk, remember?

Well, eventually I booted the Linux environment using the MacBook builtin boot menu (works great without rEFIt, too ;). I partitioned the disk as I initially planed to.

[NOTE on partitioning: I used parted for the first time on this occasion. At first, I was somewhat confused about not being able to specify disk sectors at which the partition to begin and to end... until I figured that parted will do a gread job in guessing the correct disk sectors if one tells it just roughly the start/end positions in GB measures. I must say, disk partitioning has come a long way lately... Guess that means I must be getting old now... :-o ]

So, now I'm back to linux. To be honest: it feels like home again :)

I never was really far away anyway, I used almost exclusively free software tools like Gimp, xpdf, gv, Inkscape in MacOS X anyway. But a native linux is just... the real thing ;)

And guess what: linux's UI is faster/snappier than MacOS X by a noticeable amount, I'd say roughly 3-5%. s2disk still doesn't work. But I'm not going to fix it right now, maybe in a few days. I'll write about it when I do, and I'll also write about my git-managed home directory under linux the next days. Stay tuned :)

2009-02-08 14:42 | www-datarootshell.ro | [/tech-sci] | permanent link