STRUCTURE

Containers and Volumes

ApFS is structured in a single container that can contain multiple ApFS volumes. Also a container is the primary object for storing data. It needs to be > 512 Mb to contain more than one volume, > 1024Mb to contain more than 2 volumes and so on. This image shows an overview of the ApFS structure:

ApFS

Each element of this structure (except for the allocation file) starts with a 32 byte block header, which contains some general information about the block. Afterwards the body of the structure is following. The following types exist:

  • 0x01: Container Superblock
  • 0x02: Node
  • 0x05: Space manager
  • 0x07: Allocation Info File
  • 0x11Unknown
  • 0x0B: B-Tree
  • 0x0C: Checkpoint
  • 0x0D: Volume Superblock

Containers are usually exactly the same as the GUID Partition Table (GPT) entries. They have their own crash protection and disk space allocation scheme. Each container contains one or more volumes or file systems, each of which has its own namespace, that is, a set of files and directories.

ApFS does not directly support software RAID, but it can be used with Apple RAID volumes to support Striping (RAID 0), Mirroring (RAID 1), and Concatenation (JBOD).

With a 64-bit index, ApFS volumes will support up to 9 quintillion (1018) files.

The new file system uses nanoseconds to set timestamps. In HFS+ timestamps were set to the nearest second. This will reduce the number of failures in data transfer and other file operations.

ApFS has a built-in encryption system and uses AES-XTS or AES-CBC systems, depending on the device. The user can use several encryption keys to ensure data security even in the case of “physical compromise” of the medium.

This is not a complete list of the innovations that ApFS will bring. The new file system will be released for widespread use next year, and by that time, developers need to prepare to transfer the utilities to the new file system.

Partitions formatted in ApFS are not recognized by OS X 10.11 Yosemite and earlier versions of the operating system.

Block header

Each filesystem structure in ApFS starts with a block header. This header starts with a checksum for the whole block. Other informations in the header include the copy-on-write version of the block, the block id and the block type.

pos

size

type

id

0

8

uint64

checksum

8

8

uint64

block_id

16

8

uint64

version

24

2

uint16

block_type

26

2

uint16

flags

28

4

uint32

padding

Container Superblock

The CS (container superblock) is the entry point to the filesystem. Because of the structure with containers and flexible volumes, allocation needs to be handled on container level. The CS contains information on the blocksize, the number of blocks and pointers to the space manager for this task. Additionally the block IDs of all volumes are stored in the superblock. To map block IDs to block offsets a pointer to a block map B-tree is stored. This B-tree contains entries for each volume with its ID and offset. The CS is the highest level in the file system.

pos

size

type

id

0

4

byte

magic “NXSB”

4

4

uint32

blocksize

8

8

uint64

totalblocks

40

16

byte

guid

56

8

uint64

next_free_block_id

64

8

uint64

next_version

104

4

uint32

previous_containersuperblock_block

120

8

uint64

spaceman_id

128

8

uint64

block_map_block

136

8

uint64

unknown_id

144

4

uint32

padding2

148

4

uint32

apfs_count

152

8

uint64

offset_apfs (repeat apfs_count times)

Main superblock in block 0x00:

ApFS

Volume Superblock

A volume superblock exists for each volume in the file system. It contains the name of the volume, ID and a timestamp. Similarly to the container superblock it contains a pointer to a block map which maps block IDs to block offsets. Additionally a pointer to the root directory which is stored as a node is stored in the volume superblock.

pos

size

type

id

0

4

byte

magic “APSB”

96

8

uint64

block_map

104

8

uint64

root_dir_id

112

8

uint64

pointer3

120

8

uint64

pointer4

208

16

byte

guid

224

8

uint64

time1

272

8

uint64

time2

672

8

str(ASCII)

name

Checkpoint

A checkpoint is a historical state of the container. Each checkpoint is initialized with a Container Superblock  and the current state is usually the last Container Superblock in the Container Superblock collection. The Container Superblock in the current state is the one the Main Superblock originates from. A checkpoint involves both the container and volume metadata. Restore points and snapshots are similar each other. The main difference between a checkpoint and a snapshot is the user ability to restore the file system from stored snapshots using the file system API.

Checkpoint Superblock Descriptor

This block contains information about metadata structures in APFS and is the preceding block to the CSB (except the MSB). There is always a CSBD for each CSB. Forensically, the most important information in this block is the location of the Bitmap Structure (BMS), the former allocation file in HFS+.

Checkpoint superblock descriptor:

ApFS

Bitmap Structures

Records of used and unused blocks. There is only one bitmap system that covers the whole container and is common to all volumes in the file system. From HFS+ we are familiar with the allocation file, however ApFS uses a collection of blocks to store the Bitmap Structures (BMS).

In ApFS the bitmap structures are common to all volumes in the container. Each volume has a quote of the blocks in the container but the blocks are not in dedicated areas. The BMS are referred to from the CSBD, that has information about the topmost level in the BMS, the Bitmap Descriptor (BMD). Picture below shows the basic structure of the complete BMS. The levels reflects the hierarchy where we have the BMD on the top setting the boundaries. At the bottom we have the Bitmap Blocks (BMB) that keep track of the blocks in the container. One byte in the BMB keeps track of eight blocks where each bit provides the allocation status. Each bit is the status of a single block.

ApFS

Tables

Tables are used in structures such as the catalog and extent B-trees, Volume lists and the Object-ID map. 8 distinct table types have been observed so far... To fully understand ApFS it is critical to understand the structures and roles of the tables. Without interpreting the tables correctly further interpretation of the file system is almost impossible. Tables used in ApFS are small single block "databases" with slightly different purposes in the file system structures. The table type field is composed of 2 bytes located at block with offset 0x20 directly after the node header. Table types are from 0 to 7. The next 2 bytes provide the table level from 0 upwards. So far, levels 0 to 3 have been observed, but we can expect to see even larger tables' depths in large containers filled with millions of files. We have only tested volumes with up to 220,000 files and that requires four levels (0-3) in the B-Tree. A level two table will have records referring to an underlying level 1 table. Level 0 tables refer to blocks which contain file metadata often in an underlying table of level 0.

The table types are different in structure but the 24-byte table header is consistent across all table types. The following picture provides a sample table header structure:

ApFS

And following table describes the meaning of the fields in this structure:

ApFS

A common layout of the different tables is shown here:

ApFS

Not all of the elements on the picture are used in all the tables. It shows a complete block with the block/node header at the top. The remainder of the block forms the table.

Immediately after the table header is the record index. There are 2 types. One with 2 values with only offset in keys and data section each of Uint16. The other use 4 Uint16-values with offset and length for both key and data sections. The table record index has information about key and data records in the table. Another distinction between the table types are their use of footers.

Table types 1, 3, 5 and 7 use a 0x28 byte footer at the end of the block. In these tables all data offsets are relative to offset 0xFD8 and the footer contains different values specific to the table type. The other table types have no footer and all references to the data section content are relative to the end of the block. In B-trees with several layers we have table types 1, 3, 5 or7 at the top-most level as these have a footer. The footer seems to be used to store information about the complete B-Tree. One of the values in the footer is the total number of records in the whole B-Tree structure.

The 8 table types (0-7) have a lot in common and we will focus on this first. Then we provide a short description of each table type. The table definition commences at offset 0x20 in the block with table type, number of rows, size of key section and gap between key and data section. After the table setup the table row and column definitions are described from offset 0x38 in the block. The table contains a header, record definitions, key and data sections. Certain table types also have a footer. The header begins at offset 0x20 in the block and is 0x18 bytes in length. This table type header starts with a 16-bit value which represents the table type. This is then followed by two bytes representing the level in the B-Tree at which the table is used. The two subsequent bytes represent the number of rows in the table. The length of the record definition scan be found at 0x2A followed by an Uint16 which records the length of the key section. This is followed by the gap between the key and data section. The table footer is always 0x28 bytes and always occupies the end of the block. Table indices are of 4 or 8 bytes each. On 8 byte indices the two first Uint16 are the offset and length of the key record. The next two Uint16 are the offset and length of the data record in the table. Tables with 4 byte indices have two Uint16 values which is the offset to the key and data record. The data length in the two records are predefined. In tables with a footer the offset to the data record is relative to the start of the footer (0x28). And for the other table types it is relative to the end of the block.

Key offsets are relative to the start of the key section.

Most of the values regarding table header and footer are clear at least to read the type of table. Offset 0x18 in the footer (offset 0xFF in a 4 Kb block) is the number of records in the table and all underlying tables (if this is a table with level higher than 0 in offset 0x22). Offset 0x20 in the footer is the next record number in the table.

Table TYPE 0

Table TYPE 0 has been observed in the B-Tree Catalog structure (in B-Tree level) between leaf nodes and the root node. The values Unknown 3-6 appear to be the key offset and length. And the data offset and length of the next available record. If there are no free index records the offsets are set to 0xFFFF and length of 0x00.

The records in the table are four Uint16 values. The first 2 are the offset and length of value in key section and the next are the offset and value of the content in the data section.

An example of table TYPE 0 could be Catalog Node ID and named key in the key section and Object ID in data section. This table does not have a footer.

Table TYPE 1

Table TYPE 1 has a footer and the table index contains 4 16-bit values where the first 2 values are the offset to the record in the key section and the length of the record. The next 2 values provide the offset to the record in the data section and the length of this record. This table is frequently observed in both the BTCS (B-Tree Catalog Structure) and the Extents B-Tree for the top-level node. Example values are Parent ID and a key name (file/folder name in BTCS and block start number in the Extent B-Tree) in the key section and an Object ID when used as root-node in the BTCS or a block-number when used in the Extent B-Tree. Examples of this table are provided below (The B-tree Catalog root node, BTRN) when used in the Extents B-Tree:

ApFS

Table TYPE 2

Initially this table is identical to the previous one but has no footer. This table type is very frequently encountered in leaf nodes in the BTCS where the key section is often observed with either a Parent ID and key name or CNID and data type in the data section.

Table TYPE 3

This table is equal to the previous one. The table index is the same as table TYPE 1. Typical values depend on the structure they are used in. In the BTCS and the Extents B-Tree this table is often used as top level node in small volumes  where the root node serves both as root node and leaf node. In such example of use the key record might be Parent ID. And the named key and the data record might be file metadata with large variations in size.

Other typical records could be Object ID and object type in the key record. With extent information for files in the data record. Table TYPE 3 has a footer. An example of this table used in the Extents B-tree is shown below:

ApFS

Table TYPE 4

Table TYPE 4 diverges somewhat from the previous ones. The table has no footer and the table index only has 2 values per record, the offset to the record in the key section and then 1 for the data section. The length of the content is fixed with 16 bytes in the key section and 8 bytes in the data section. Offsets in the data section are relative to the end of the block.

Table TYPE 5

Table TYPE 5 is very similar to TYPE 4. The only difference is that this type has a footer and all offsets to data are from offset- 0x28 (beginning of the footer). The records in the key section are 16 bytes and 8 bytes in the data section. This table type is mostly observed at top level nodes in the BTCS and larger containers with multilevel B-Tree‘s.

Table TYPE 6

Table TYPE 6 is very similar to TYPE 4. The table index has only the offset to content in the key and data section and not the length. The lengths are predefined. Each record is 16 bytes in both the key and data sections. There is no footer for this type of table. This type of table is often observed in the leaf nodes in the BTCS. Typical key section content includes Object ID and Volume Checkpoint Superblock ID  while the data section typically records the size of the data and a block number.

Table TYPE 7

This table type is very similar to TYPE 6. The only difference is the footer that contains similar information to that described for table TYPE 1. This table type is observed in a broad range of structures and is often encountered in the top most levels of multilayer structures or in single layer structures such as the Volume declarations. An example of this type is shown below:

ApFS

Tables' summary

The following table (artifacts) shows the basic properties of the different table types:

ApFS

One of the most important blocks in the B-Tree Catalog Structure is the root node which is the highest level in the folder structure. This node utilizes search keys of variable length. One of the improved features within ApFS is Fast Directory Searching (FDS). One of the values that is tightly connected with this feature is the count of all records in the tree structure located in the table footer. In the B-Tree catalog structure the root node has only 2 options in the selection of table to use since both of these have footers. This is also according to the observations in many ApFS containers investigated. Table TYPE 3 acting as a root node is only observed in small containers with few files where the root node is also an index and leaf node. In the B-Tree Object Map  only table TYPE 5 is used for the root node except in the case of very small structures where TYPE 7 may be encountered. The interpretation of the tables show that tables TYPE 0 and 2 have the same artifacts. The same is observed between tables TYPE 1 and 3. These tables appear to have a different purpose depending on which structure they are in.

Snapshots

Snapshots - readonly "snapshots" of the file system in the volume. The operating system can use snapshots for a more efficient backup procedure. Finally, Time Machine will work fine (fast). With ApFS support, for Time Machine instant images, you no longer need to save several full copies of the file to your disk - it can simply track specific changes. For example, if you are editing a PowerPoint presentation, changing a single slide using the old HFS+ means saving two copies of the file in which your new changes are recorded, and one in case you want to return. Now it can simply save the source file plus recording the differences between the source file and any updated versions, performing the same task in much less space. As with the improvements in Fusion Drive, the information takes up less space on the SSD, that is, less data is written to the disk, which ultimately will increase the life of your drive, and for such an important thing as Time Machine it is extremely important.

Of course ApFS is significantly inferior in its capabilities to the 128-bit ZFS, which is supported by Linux, FreeBSD and other free OSes but on the part of Apple this is a step in the right direction.

It is strange that the preliminary documentation does not mention the compression function that HFS+ BTW supports…

As mentioned above Apple tried to port ZFS to OS X for a long time. Later OpenZFS was implemented for OS X (O3X) and MacZFX.

Nodes

Nodes are flexible containers that are used for storing different kinds of entries. They can be part of a B-tree or exist on their own. Nodes can either contain flexible or fixed sized entries. A node starts with a list of pointers to the entry keys and entry records. This way for each entry the node contains an entry header at the beginning of the node, an entry key in the middle of the node and an entry record at the end of the node.

ApFS

pos

size

type

id

0

4

uint32

alignment

4

4

uint32

entry_count

10

2

uint16

head_size

16

8

entry

meta_entry

24

entry

entries (repeat entry_count times)

Node header structure:

Offset

Field

Data type

Comments

0

8

10

18

1A

1C

1E

Checksum

ID

Checkpoint ID

Unknown

Unknown

Unknown

Unknown

Uint64

Uint64

Uint64

Uint16

Uint16

Uint16

Uint16

Fletchers Checksum Algorithm

Object-ID or Block#

 

Possible level in B-Tree

All observations shows value 0x4000 Flag?

Often seen value 0x0b, 0x0e and 0x0f

Space manager

The Space Manager (sometimes called spaceman) is used to manage allocated blocks in the ApFS container. Stores the number of free blocks and a pointer to the allocation info files.

pos

size

type

id

0

4

uint32

blocksize

16

8

uint64

totalblocks

40

8

uint64

freeblocks

144

8

uint64

prev_allocationinfofile_block

352

8

uint64

allocationinfofile_block

Allocation Info File

The allocation info file works as a missing header for the allocation file. The allocation files length, version and the offset of the allocation file are stored here.

pos

size

type

id

4

4

uint32

alloc_file_length

8

4

uint32

alloc_file_version

24

4

uint32

total_blocks

28

4

uint32

free_blocks

32

4

uint32

allocationfile_block

File and folder B-Tree

Records all files and folders in the volume. It performs the same role as the catalog file in HFS+.

Extents B-Tree

A separate B-Tree of all extents per volume. Extents are references to file content with information about where the data content starts and the length in blocks. A file with some content will have at least one extent. A fragmented file will have multiple extents. The extent B-Tree is a separate structure. In each file record extents are defined per file in the file/folder B-Tree. This separate extent structure is part of the snapshot feature.

64-bit inodes (index descriptors)

64-bit inodes significantly increase the namespace compared to 32-bit identifiers in HFS+. The ApFS 64-bit file system supports more than 9 quintillion files on each volume. That should be enough for everyone as Bill Gates said. smile!

< Previous | Content | Next >