Filesystems and Disks

Introduction

The information in this section is intended to give a nontechnical reader some idea about the workings of hard disks and the ways in which computers store files. You don’t need to read this section to use iDefrag, but it may help to understand what the program is showing you and why you might want to defragment your disk in the first place.

About your hard disk

Everyone is familiar with the notion that a hard disk can store data in files, but quite how this is achieved is, for most users, largely a mystery. Indeed, many people these days probably don’t even give it a second thought.

The explanation that follows is talking about magnetic disks; the details for other storage devices, including tapes, optical discs, solid-state storage devices and so on are different. A magnetic disk, whether a floppy disk or a hard disk, consists of

• one or more circular platters (the disk media) coated in a magnetic medium,
• one or more disk heads, which are able to replay and record a signal on the magnetic surface of the platter,
• a spindle motor, which is responsible for spinning the platters at high speed (typically for a hard disk 5,400 rpm or higher), and
• some kind of acruator that can move the disk heads in and out (this might be a stepper motor for floppy disks, or more typically on hard disks a voice coil).

The inside of a typical hard disk might look something like the following:

(In practice there might be two heads per platter, one for each surface, the platters and armature would be much closer together, and the disk heads would be very small indeed.)

To read data from the disk, the disk drive must position its heads at the correct distance from the spindle, then wait for the data to pass under the head for the platter on which the data is stored. The location of the head relative to the spindle is defined by the cylinder number. A combination of a cylinder number and a head number defines a circular track on a particular side of a particular platter. Each track is further divided up into sectors, each of which can usually hold 512 bytes of data, and so the combination of a cylinder number, a head number and a sector number identifies a unique sector on the disk’s media.

Selecting a head is done electronically and takes essentially no time at all. The time spent waiting for a particular sector to pass under the disk head obviously depends on the rate of rotation of the drive and the size of the sectors; this is one reason why hard disks that spin at a higher rate are preferable, though the high rate of rotation also means that individual bits of data pass under the head faster, resulting in a higher rate of data transfer.

Changing cylinder is comparatively expensive, taking of the order of several milliseconds on a typical modern drive. When moving the head, a disk takes the following steps:

1. Use the actuator to move to approximately the correct location. Typical head actuators can reliably move small distances, but are less accurate for longer movements.
2. Wait until a special index mark moves under the disk head. This tells the disk which cylinder the head is over.
3. If the head is not over the correct cylinder, repeat from step 1.

Typically each movement will be smaller than the last until the head settles over the correct cylinder.

Older disks typically expect the host computer to provide them with coordinates for data in Cylinder Head Sector or CHS form. Unfortunately this scheme has a number of downsides, notably that different disks will have different numbers of cylinders and heads, and may even choose to have different numbers of sectors on each of their tracks; the arrangement of cylinders, heads and sectors for a given disk is referred to as its geometry. Using real CHS values also prevents the disk from easily reassigning sectors where it discovers that the magnetic media is damaged, and instead the computer’s software must deal with this problem itself.

Worse, as disks grew in size, it became apparent that the fields reserved for cylinder, head and sector numbers, both in software and in hardware, were too small to cope. As a result, disks started to lie to the host computer about their geometry, both for compatibility with the PC BIOS and so that they could start to manage bad sectors automatically themselves.

Modern disks are not generally used with CHS values; instead, sectors are given a number starting from zero or one. This is often referred to as a Linear Block Address or LBA. When accessed in this way, the disk manufacturer is free to choose the locations of each sector on the physical media in any way they please so as to optimise the performance of their drive mechanism; normally the assignment of blocks is done in such a way as to guarantee good performance if the blocks are accessed in LBA order.

Filesystems, volumes and partitions

So we know now that disks can be regarded as a numbered array of sectors, each of which can store 512 bytes of data. There are disks where the sector size differs from that, but almost all hard disks used today stick with (or pretend to stick with) these 512 byte blocks.

If that was all that we had, life would be very tedious indeed. Even on a 1.44MB floppy disk, there are two heads (corresponding to the two sides of the disk), eighty cylinders and each track holds eighteen sectors, giving a total sector count of 2 × 80 × 18 = 2,880 sectors. Keeping track of which of them are in use and where your files are on the disk would be a real pain.

A modern 500GB hard disk, on the other hand, has nigh on a billion sectors and can store hundreds of thousands or even millions of files. How are we supposed to locate that photo that we saved last week? You’d need a building full of filing cabinets!

Fortunately this is exactly the kind of tedious job that computers are good at, and the pieces of software that they used to keep track of all of your precious data are called filesystem drivers. Apple’s HFS+ (aka Mac OS Extended) is one example, and you have probably also heard of Microsoft’s FAT and NTFS filesystems as well.

Different filesystems use different approaches to store the information about where your files are on the disk, about the sectors that are or are not currently in use, and so on, but what they have in common is that given a disk formatted for use with that file system, and the name of a file on that disk, they can work out which sectors hold the data for that file and return the data to the program that is attempting to access the file.

It is important at this point to note that the word filesystem is used interchangeably by some people to mean several different things; sometimes they mean the computer code responsible for locating your files on the disk; other times they mean a particular formatted disk; and on still other occasions they mean the abstract design of the data structures themselves.

This is all very confusing, and so Apple very sensibly tends to use the following words:

filesystem

The abstract design, including data structures and algorithms; the information presented in the filesystem specification — essentially an instruction book that tells you how the data is stored.

filesystem driver

The computer program that implements the abstract design, thereby allowing your Mac to read a disk formatted according to the filesystem specifications.

volume

The contents of a particular disk, formatted according to the filesystem specifications.

In addition, rather than the whole disk being used to hold a single volume, disks can be divided into pieces called partitions, each of which might contain a separate volume.

SimpleFS - a worked example

To help understand how all of this works together, let’s take a look at an idealised filesystem, which we’ll call SimpleFS.

SimpleFS version 1 supports storage of files in contiguous chunks on disk. We’ll imagine that we have a very small disk with sixteen blocks in total, and we’ll allocate one block to hold a list of the files on the disk, with another to hold information on which blocks are in use.

We’ll skirt over exactly how this information is stored in these reserved blocks, as it doesn’t matter for the purposes of our explanation.

Our disk, before we store any files, looks like this:

0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15
File List	Alloc Table

So, let’s save a file to our disk; our file is 2,450 bytes in length, so it takes up 2,450 ÷ 512 = 4.79 blocks. SimpleFS only lets us allocate entire blocks, so we’ll use 5 blocks to store this file — call it “File A”. We’ll also save “File B”, 1,300 bytes long (3 blocks), and “File C”, 1,000 bytes long (2 blocks). Our disk now looks like this:

0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15
File List	Alloc Table	File A					File B			File C

Easy so far, right?

Next, let’s delete “File B” from our disk. We now have

0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15
File List	Alloc Table	File A								File C

Imagine we now wish to save a new file, “File D”, and that this new file is 2,400 bytes in length (5 blocks). We have seven blocks free, but the largest contiguous space is only four blocks long! That’s no good — we can’t save our new file, even though we have 50% of the usable blocks available.

Let’s imagine for a moment that we can somehow upgrade our disk to a new SimpleFS 2.0 that allows us to use all of these blocks to store our data. We still want to save “File D”, which takes up five blocks, so we store the first three in blocks 7, 8 and 9, and the remainder in blocks 12 and 13. The result is as follows:

0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15
File List	Alloc Table	File A					File D (Part 1)			File C		File D (Part 2)

At first glance, this is great! We can use every block on the disk if we want.

Unfortunately, we’ve created a problem for ourselves. Let’s assume that the disk we’ve formatted for use with SimpleFS is very old and slow. For sake of argument, let’s give it a single platter with only a single side, so we only have one head, and let’s give it four cylinders, with each track holding four sectors.

Here’s what our disk looks like, with the locations of our files superimposed

For the next part, you can follow what is going on by starting at the block spot in the right hand diagram above, and tracing the line as it goes clockwise around the disk.

Assume we want to read “File D” from the disk; we first have to go and find it in the file list, which means reading block 0. The file list tells us where “File D” is on the disk, so the next sector we want to read is block 7, which on our disk is on track 1; we’re currently on track 0, so we move the head out one track. But we aren’t in the right place to read block 7, so we have to wait for the disk to revolve until the head is over block 7 before we can read it.

Having read block 7, we now need block 8; this is on track 2, and our disk head is over track 1, so we move the head outwards again. Yet again, we aren’t in the right place to read block 8 (we’ve missed the beginning of it), so we need to wait for the disk to rotate again.

(If you’re following along at this point, switch to the grey line.)

We can not read blocks 8 and 9 one after the other, without moving the disk head. The next block, though, is block 12, and that’s on the outer track, track 3, so we yet again need to move the head out one track, and since yet again we find ourselves in the wrong place, we need to wait for the disk to rotate.

Finally, we read blocks 12 and 13 one after the other, and we’re done.

During this process the disk has revolved three and a half times, and we’ve had to move the head three times, not counting any initial movement to get to the starting point on track 0.

Let’s say this disk is very slow indeed, takes half a second to move the head, and rotates at 20rpm. We’ve spent a total of 3 × 0.5 + 3.5 ÷ 20 × 60 = 12 seconds reading this file.

Now let’s consider what might happen if we swapped the positions of the end of “File D” with “File C”, giving

0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15
File List	Alloc Table	File A					File D					File C

This one change puts the final blocks of “File D” on the same track as the previous two blocks, so we can read 8, 9, 10 and 11 one after another without moving the disk head and without waiting for the disk to rorate. We save one head movement and one-and-a-half rotations, and only spend 2 × 0.5 + 2 ÷ 20 × 60 = 7 seconds reading the file.

That’s a 42% time saving, just for moving the blocks of “File D” so that they’re next to one another on the disk!

As you can hopefully see from this heavily simplified example, allowing the filesystem to fragment files is a trade-off. Without it, you might not be able to use all the space on your disk, but on the other hand, a fragmented file might take a log longer to read or write than it would have if the file was in a single piece.

Clearly, therefore, what you don’t want is for a file to be fragmented when it didn’t need to be.