= New Technologies File System (NTFS)
Analysis of NTFS

:toc:
:toclevels: 4

:numbered!:
[abstract]
== Summary

NTFS is the primary file system for Microsoft Windows versions that are based
on Windows NT. This specification is based on publicly availble work on the
format and was enhanced by analyzing test data.

This document is intended as a working document for the NTFS format.

[preface]
== Document information

[cols="1,5"]
|===
| Author(s): | Joachim Metz <joachim.metz@gmail.com>
| Abstract: | This document contains information about the New Technologies File System (NTFS)
| Classification: | Public
| Keywords: | New Technologies File System, NTFS
|===

[preface]
== License

....
Copyright (C) 2009-2019, Joachim Metz <joachim.metz@gmail.com>.
Permission is granted to copy, distribute and/or modify this document under the
terms of the GNU Free Documentation License, Version 1.3 or any later version
published by the Free Software Foundation; with no Invariant Sections, no
Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included
in the section entitled "GNU Free Documentation License".
....

[preface]
== Revision history

[cols="1,1,1,5",options="header"]
|===
| Version | Author | Date | Comments
| 0.0.1 | J.B. Metz | August 2009 +
September 2010 +
October 2010 +
November 2010 +
December 2010 | Initial version, based on earlier notes.
| 0.0.2 | J.B. Metz | March 2011 | Additional information about multi data run MFT.
| 0.0.3 | J.B. Metz | May 2012 | Additional information.
| 0.0.4 | J.B. Metz | February 2014 | Additional information.
| 0.0.5 | J.B. Metz | July 2014 +
August 2014 | Textual changes and additional information.
| 0.0.6 | J.B. Metz | March 2015 | Additional information. +
Switched to asccidoc format.
| 0.0.7 | J.B. Metz | May 2015 | Additional information.
| 0.0.8 | J.B. Metz | July 2015 | Textual changes.
| 0.0.9 | J.B. Metz | August 2015 | Additional information.
| 0.0.10 | J.B. Metz | August 2015 | Additional information regarding index and security descriptors.
| 0.0.11 | J.B. Metz | January 2016 | Additional information about common alternate data stream.
| 0.0.12 | J.B. Metz | January 2016 | Additional information about extended attributes.
| 0.0.13 | J.B. Metz | March 2016 | Small changes.
| 0.0.14 | J.B. Metz | April 2016 | Small changes.
| 0.0.15 | J.B. Metz | June 2018 | Corrected typos.
| 0.0.16 | J.B. Metz | January 2019 | Changes to format.
|===

:numbered:
== Overview

NTFS is the primary file system for Microsoft Windows versions that are based
on Windows NT.

[yellow-background]*TODO describe relation to OS2 HPFS*

[cols="1,5",options="header"]
|===
| Characteristics | Description
| Byte order | little-endian
| Date and time values | FILETIME in UTC
| Character strings | ASCII strings are Single Byte Character (SBC) or Multi Byte Character (MBC) string stored with a codepage. Sometimes referred to as ANSI string representation. +
Though technically maybe incorrect, this document will use term (extended) ASCII string. +
Unicode strings are stored in UTF-16 little-endian without the byte order mark (BOM).
|===

=== Versions

There are multiple version of NTFS.

[cols="1,5",options="header"]
|===
| NTFS version | Remarks
| 1.0 | Introduced in Windows NT 3.1
| 1.1 | Introduced in Windows NT 3.5
| 1.2 | Introduced in Windows NT 3.51
| 3.0 | Introduced in Windows 2000
| 3.1 | Introduced in Windows XP
|===

[yellow-background]*Need test images for version 1.0 and 1.1*

[NOTE]
The versions mentioned above are the version as used by NTFS. Another common
versioning schema uses the Windows version, e.g. NTFS 5.0 is the version of
NTFS used on Windows XP which is version 3.1 in schema mentioned above.

=== Test version

The following version of programs were used to test the information within this
document:

* Windows NT4 (version 1.2)
* Windows 2000 (version 3.0)
* Windows XP SP3 (version 3.1)
* Windows 2003 (version 3.1)
* Windows Vista (version 3.1)
* Windows 2008 (version 3.1)
* Windows 7 (version 3.1)
* Windows 8 (version 3.1)
* Windows 10 (version 3.1)
* NTFS-3G

== Terminology

=== Cluster

NTFS refers to it file system blocks as clusters. Note that these are not the
same as the physical clusters of a harddisk. For clarity these are referred to
as cluster blocks. In other sources they are also referred to as logical
clusters which are numbered globally (or absolute).

Typically the cluster block is 8 sectors (8 x 512 = 4096 bytes) of size.

=== Virtual cluster

The term virtual cluster refers to cluster blocks which are numbered locally
(or relative).

=== Long and short (file) name

In Windows terminology the name of a file (or directory) can either be short or
long. The short name is an equivalent of the filename in the (DOS) 8.3 format.
The long name is actual the (full) name of the file. The term long refers to
the aspect that the name is longer than the short variant. Because most
documentation refer to the (full) name as the long name, for clarity sake so
will this document.

== The volume

Everything on an NTFS volume is a file. There are two types of files:

* files that contain volume and file system metadata (referred to as metadata files);
* files that contain data (referred to as files).

=== The metadata files

NTFS uses the Master File Table (MFT) to store information about files and
directories. The MFT entries reference the different volume and file system
metadata. There are several predefined metadata files.

The following metadata files are predefined and use a fixed MFT entry index.

[cols="1,1,5",options="header"]
|===
| MFT entry index | Filename | Description
| 0 | $MFT | Master File Table
| 1 | $MFTMirr | Back up of the first 4 entries of the Master File Table
| 2 | $LogFile | Metadata transaction journal
| 3 | $Volume | Volume information
| 4 | $AttrDef | MFT entry attribute definitions
| 5 | .  | Root directory
| 6 | $Bitmap | Cluster block allocation bitmap
| 7 | $Boot | Boot record (or boot code)
| 8 | $BadClus | Bad clusters
| 9 | $Quota | Quota information +
Last used in NTFS version 1.2
| 9 | $Secure | Security and access control information +
Introduced in NTFS version 3.0
| 10 | $UpCase | Table of uppercase characters used for ensuring case insensitivity in Windows and DOS namespaces.
| 11 | $Extend | A directory containing extended metadata files
| 12-15 | | Reserved +
Marked as in use but empty
| 16-23 | | Unused +
Marked as unused
3+| _As of NTFS version 3.0_
| 24 | $Extend\$Quota | Quota information +
Was MFT entry 9 in Windows NT 4
| 25 | $Extend\$ObjId | Unique file identifiers for distributed link tracking
| 26 | $Extend\$Reparse | Backreferences to reparse points
3+| _As of Windows Vista_ ([yellow-background]*or server 2003?*) +
_Transactional NTFS metadata (See section: <<transactional_ntfs,Transactional NTFS (TxF)>>)_
| 27 | $Extend\$RmMetadata | Resource manager metadata directory
| 28 | $Extend\$RmMetadata\$Repair | Repair information
| 29 | $Extend\$RmMetadata\$TxfLog | Transactional NTFS (TxF) log metadata directory
| 30 | $Extend\$RmMetadata\$Txf | Transactional NTFS (TxF) metadata directory
| 31 | $Extend\$RmMetadata\$TxfLog\$Tops | TxF Old Page Stream (TOPS) file +
Used to store data that has been overwritten inside a currently active transaction
| 32 | $Extend\$RmMetadata\$TxfLog\$TxfLog.blf | Transactional NTFS (TxF) base log metadata file
3+| _Common_
| | ... | A file or directory
|===

The following metadata files are predefined, however the MFT entry index is
commonly used but not fixed.

[cols="1,1,5",options="header"]
|===
| MFT entry index | Filename | Description
| | $Extend\$UsnJrnl | USN change journal +
See section: <<usn_change_journal,USN change journal>> +
[yellow-background]*Has this file been added in Windows XP SP3? Otherwise what are reasons for it to not be present?*
|===

== The volume header

The volume header is stored at the start of the volume (in the $Boot metadata
file) and contains:

* the volume signature
* the BIOS parameter block
* the boot loader

The volume header is 512 bytes of size and consists of:

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | 3 | | Boot entry point +
Often contains: +
eb52   jmp 0x52 +
90       nop +
This is a jump instruction to the bootcode at offset 84 followed by a no-operation.
| 3 | 4 | "NTFS\x20\x20\x20\x20" | File system signature +
(Also known as OEM and/or dummy identifier) +
4+| _DOS version 2.0 BIOS parameter block (BPB)_
| 11 | 2 | | Bytes per sector
| 13 | 1 | | Sectors per cluster block +
[yellow-background]*Values seen 1, 2, 8*
| 14 | 2 | 0x00 | Reserved Sectors +
not used by NTFS `[POLLARD06]` and must be 0 `[MSDN]`
| 16 | 1 | 0x00 | Number of File Allocation Tables (FATs) +
not used by NTFS `[POLLARD06]` and must be 0 `[MSDN]`
| 17 | 2 | 0 | Root directory entries +
not used by NTFS `[POLLARD06]` and must be 0 `[MSDN]`
| 19 | 2 | | Total number of sectors (16-bit) +
[yellow-background]*Used if the total of number of sectors fits in 16-bit?*
| 21 | 1 | | Media descriptor +
See section: <<media_descriptor,Media descriptor>>
| 22 | 2 | 0x00 | Sectors Per File Allocation Table (FAT) +
not used by NTFS `[POLLARD06]` and must be 0 `[MSDN]`
4+| _DOS version 3.4 BIOS parameter block (BPB)_
| 24 | 2 | 0x3f | Sectors per track +
Not used by NTFS `[MSDN]`
| 26 | 2 | 0xff | Number of heads +
Not used by NTFS `[MSDN]`
| 28 | 4 | 0x3f | Number of hidden sectors +
Not used by NTFS `[MSDN]`
| 32 | 4 | 0x00 | Total number of sectors (32-bit) +
[yellow-background]*Used if the total of number of sectors fits in 32-bit?* +
Not used by NTFS must be 0 `[MSDN]`
4+| _NTFS version 8.0 BIOS parameter block (BPB) or extended BPB_ +
_Introduced in Windows NT version 3.1_
| 36 | 1 | 0x80 | [yellow-background]*Unknown (Disc unit number)* +
Not used by NTFS `[MSDN]`
| 37 | 1 | 0x00 | [yellow-background]*Unknown (Flags)* +
Not used by NTFS `[MSDN]`
| 38 | 1 | 0x80 | [yellow-background]*Unknown (BPB version signature byte)* +
Not used by NTFS `[MSDN]`
| 39 | 1 | 0x00 | [yellow-background]*Unknown (Reserved)* +
Not used by NTFS `[MSDN]`
| 40 | 8 | | Total number of sectors (64-bit)
| 48 | 8 | | Master File Table (MFT) cluster block number
| 56 | 8 | | Mirror MFT cluster block number
| 64 | 1 | | MFT entry size +
See below.
| 65 | 3 | | [yellow-background]*Unknown* +
Not used by NTFS `[MSDN]`
| 68 | 1 | | Index entry size +
See below.
| 69 | 3 | | [yellow-background]*Unknown* +
Not used by NTFS `[MSDN]`
| 72 | 8 | | NTFS volume serial number +
See below.
| 80 | 4 | 0x00 | Checksum +
not used by NTFS `[POLLARD06]`, `[MSDN]`
4+|
| 84 | 426 | | Bootcode +
[yellow-background]*What is the exact end of the bootcode and are there no trailing values?*
| 510 | 2 | 0x55 0xaa | Sector signature
|===

Both the MFT and index entry sizes are defined as following:

* Values 0 to 127 represent sizes of 0 to 127 cluster blocks.
* Values 128 to 255 represent sizes of `2^(256-n)` bytes; or `2^(-n)` if considered as a signed byte.
* Other values are not considered valid `[POLLARD06]`.

The cluster block size can be determined as following:
....
cluster block size = bytes per sector x sectors per cluster block
....

Values available in Windows are:

* 512
* 1024
* 2048
* 4096
* 8192
* 16K (16384)
* 32K (32768)
* 64K (65536)

The MFT offset can be determined as following:
....
MFT offset = volume header offset
           + ( MFT cluster block number x Cluster block size )
....

Note that the lower 32-bit part of the NTFS volume serial number is the WINAPI
volume serial number. E.g. compare the output of:
....
fsutil fsinfo volumeinfo C:
fsutil fsinfo ntfsinfo C:
....

Often the volume will be smaller than the underlying partition. A (nearly
identical) backup of the volume header is stored in last sector of cluster
block, that follows the last cluster block of the volume. Often this is the 512
bytes after the last sector of the volume, but not necessarily. The backup
volume header is not included in the volume size.

=== BitLocker Drive Encryption (BDE)

BitLocker Drive Encryption (BDE) uses the file system signature: "-FVE-FS-".
Where FVE is an abbreviation of Full Volume Encryption.

The data structures of BDE on Windows Vista and 7 differ.

A Windows Vista BDE volume starts with:
....
eb 52 90 2d 46 56 45 26 46 53 2d
....

A Windows 7 BDE volume starts with:
....
eb 58 90 2d 46 56 45 26 46 53 2d
....

BDE is largely a stand-alone but has some integration with NTFS. For more
information about BDE see https://github.com/libyal/libbde/blob/master/documentation/BitLocker%20Drive%20Encryption%20(BDE)%20format.asciidoc[[LIBBDE\]].

=== Volume Shadow Snapshots (VSS)

Volume Shadow Snapshots (VSS) uses the GUID
3808876b-c176-4e48-b7ae-04046e6cc752 (stored in little-endian) to identify its
data. VSS is largely a stand-alone but has some integration with NTFS.

For more information about VSS see https://github.com/libyal/libvshadow/blob/master/documentation/Volume%20Shadow%20Snapshot%20(VSS)%20format.asciidoc[[LIBVSHADOW\]].

=== [[media_descriptor]]Media descriptor

[cols="1,1,5",options="header"]
|===
| Bit(s) | Identifier | Description
| 0 | | Sides: +
0 => single-sided +
1 => double-sided
| 1 | | Track size: +
0 => 9 sectors per track +
1 => 8 sectors per track
| 2 | | Density: +
0 => 80 tracks +
1 => 40 tracks
| 3 | | Type: +
0 => Fixed disc +
1 => Removable disc
| 4 – 7 | | Always set to 1
|===

=== The boot loader

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 512 | | | [yellow-background]*Windows NT (boot) loader* +
[yellow-background]*NTLDR/BOOTMGR*
|===

== The Master File Table (MFT)

The MFT consist of an array of MFT entries. The offset of the MFT table can be
found in the volume header and the size of the MFT is defined by the MFT entry
of the $MFT metadata file.

[NOTE]
The MFT can consists of multiple data ranges, defined by the data runs in the
$MFT metadata file.

=== MFT entry

Although the size of a MFT entry is defined in the volume header is commonly
1024 bytes of size and consists of:

* The MFT entry header
* The fix-up values
* An array of MFT attribute values
* Padding, which should contain 0-byte values

[NOTE]
The MFT entry can be filled entirely with 0-byte values. Seen in Windows XP
for MFT entry indexes 16 - 23.

==== MFT entry header

The MFT entry header (FILE_RECORD_SEGMENT_HEADER) is 42 or 48 bytes of size
and consists of:

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
4+| _MULTI_SECTOR_HEADER_
| 0 | 4 | "BAAD" +
"FILE" | Signature
| 4 | 2 | | The fix-up values offset +
Contains an offset relative from the start of the MFT entry +
According to `[MSDN]` this value is the update sequence array offset
| 6 | 2 | | The number of fix-up values +
According to `[MSDN]` this value is the update sequence array size.
4+|
| 8 | 8 | | Metadata transaction journal sequence number +
Contains a $LogFile Sequence Number (LSN)
| 16 | 2 | | Sequence (number)
| 18 | 2 | | Reference (link) count
| 20 | 2 | | Attributes offset (or first attribute offset) +
Contains an offset relative from the start of the MFT entry
| 22 | 2 | | Entry flags +
See section: <<mft_entry_flags,MFT entry flags>>
| 24 | 4 | | Used entry size +
Contains the number of bytes of the MFT entry that are in use
| 28 | 4 | | Total entry size +
Contains the number of bytes of the MFT entry
[yellow-background]*Could this be used to store data larger than 1024 - header continuously?*
| 32 | 8 | | Base record file reference +
See section: <<file_reference,The file reference>>
| 40 | 2 | | First available attribute identifier
4+| _Version 3.0_
| 42 | 2 | | [yellow-background]*Unknown (wfixupPattern)*
| 44 | 4 | | [yellow-background]*Unknown*
4+| _Version 3.1_
| 42 | 2 | | [yellow-background]*Unknown (wfixupPattern)*
| 44 | 4 | | The index
|===

The base record file reference indicates if the MFT entry is used to store
additional attributes for another MFT entry, e.g. for attribute list attributes.

According to `[MSDN]` the sequence number is incremented each time that a file
record segment is freed; it is 0 if the segment is not used.

`[MSDN]` states that the update sequence array must end before the last USHORT
value in the first sector. It also claims the update sequence array size value
contains the number of bytes. It seems to be more likely to the number of words.

The "BAAD" signature presumably indicates a bad MFT entry. `[LINUXNTFS]` states
that during chkdsk, if NTFS finds a multi-sector item where the multi-sector
header does not match the values at the end of the sector, it marks the item as
"BAAD" and fill it with 0-byte values except for a fix-up value at the end of
the first sector. The "BAAD" signature has been seen to be used on Windows NT4
and XP.

In NT4 (version 1.2) the MFT entry is 42 bytes in size and the fix-up values
are stored at offset 42. This is likely where the field name wfixupPattern
originates from.

==== Notes

....
Live MFT header:
00000000: 02 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ........ ........
00000010: 00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ........ ........
00000020: 00 00 00 00 00 00 00 00  00 00                     ........ ..

00000000: 03 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ........ ........
00000010: 00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ........ ........
00000020: 00 00 00 00 00 00 00 00  00 00                     ........ ..
....

=== [[mft_entry_flags]]MFT entry flags

[cols="1,1,5",options="header"]
|===
| Value | Identifier | Description
| 0x0001 | FILE_RECORD_SEGMENT_IN_USE | In use
| 0x0002 | FILE_NAME_INDEX_PRESENT | Has file name (or $I30) index +
When this flag is present the file entry is a directory (or contains sub file entries)
| 0x0004 | | [yellow-background]*Unknown (set for $ObjId, $Quota, $Reparse, $UsnJrnl)*
| 0x0008 | | [yellow-background]*Unknown (set for $ObjId, $Quota, $Reparse. $Secure)*
|===

=== [[file_reference]]The file reference

The file reference (FILE_REFERENCE or MFT_SEGMENT_REFERENCE) is 8 bytes of size
and consists of:

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | 6 | | MFT entry index +
Note that the index value in the MFT entry is only 32-bit of size.
| 6 | 2 | | Sequence number
|===

=== [[fix_up_values]]The fix-up values

The fix-up values are variable of size and consists of:

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | 2 | | Fix-up placeholder value
| 2 | 2 x number of fix-up values | | Fix-up (original) value array
|===

On disk the last 2 bytes in each sector is replaced by the fix-up placeholder
value. The original value is stored in the corresponding fix-up (original)
value array entry.

[NOTE]
There can be more fix-up values than the amount of sectors in the data.

See `[CARRIER05]` and/or `[RUSSON05]` for examples on applying the fix-up values.

=== MFT attribute

The MFT attribute consist of:

* the attribute header
* the attribute resident or non-resident data
* the attribute name
* [yellow-background]*unknown data likely alignment padding (4-byte alignment)*
* the attribute data runs or data
* alignment padding (8-byte alignment), can contain remnant data

==== MFT attribute header

The MFT attribute header (ATTRIBUTE_RECORD_HEADER) is 16 bytes of size and
consists of:

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | 4 | | Attribute type (or type code) +
See section: <<attribute_types,The attribute types>>
| 4 | 4 | | Size (or record length) +
The size of the attribute including the 8 bytes of the attribute type and size
| 8 | 1 | | Non-resident flag (or form code) +
0 => RESIDENT_FORM +
1 => NONRESIDENT_FORM
| 9 | 1 | | Name size (or name length) +
Contains the number of characters with-out the end-of-string character
| 10 | 2 | | Name offset +
Contains an offset relative from the start of the MFT entry
| 12 | 2 | | Attribute data flags +
See section: <<mft_attribute_data_flags,MFT attribute data flags>>
| 14 | 2 | | Attribute identifier (or instance) +
[yellow-background]*An unique identifier to distinguish between attributes that contain segmented data.*
|===

===== Notes

[yellow-background]*Size (or record length) upper 2 bytes overloaded or remnant data?*
....
type					: 0x000000a0 ($INDEX_ALLOCATION)
size					: 458832 (0x70050)
non resident flag			: 0x01
name size				: 4
name offset				: 64
data flags				: 0x0000

identifier				: 4
....

===== [[mft_attribute_data_flags]]MFT attribute data flags

[cols="1,1,5",options="header"]
|===
| Value | Identifier | Description
| 0x0001 | | Is compressed
| 0x00ff | ATTRIBUTE_FLAG_COMPRESSION_MASK |
| | |
| 0x4000 | ATTRIBUTE_FLAG_ENCRYPTED | Is encrypted
| 0x8000 | ATTRIBUTE_FLAG_SPARSE | Is sparse
|===

[yellow-background]*Does 0x0001 indicate the LZNT1 compression method?
Do other values indicate other compression values?*

==== Resident MFT attribute

The resident MFT attribute data is present when the non-resident flag is not
set (0). The resident data is 8 bytes in size and consists of:

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | 4 | | Data size (or value length)
| 4 | 2 | | Data offset (or value size) +
Contains an offset relative from the start of the MFT attribute
| 6 | 1 | | Indexed flag +
[yellow-background]*Only the lower bit is used, do the other bits have any significance?*
| 7 | 1 | 0x00 | Padding +
Contains an empty byte
|===

[yellow-background]*What meaning has ATTRIBUTE_FLAG_COMPRESSION_MASK in
$INDEX_ROOT attribute? The attribute data is uncompressed.*

Seen on Windows 10 (version 3.1)
....
type                                 : 0x00000090 ($INDEX_ROOT)
size                                 : 88
non resident flag                    : 0x00
name size                            : 4
name offset                          : 24
data flags                           : 0x0001
        Is compressed
....

==== Non-resident MFT attribute

The non-resident MFT attribute data is present when the non-resident flag is
set (1). The non-resident data is 48 or 56 bytes in size and consists of:

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | 8 | | First (or lowest) Virtual Cluster Number (VCN) of the data
| 8 | 8 | | Last (or highest) Virtual Cluster Number (VCN) of the data +
[yellow-background]*Seen this value to be -1 in combination with data size of 0*
| 16 | 2 | | Data runs offset (or mappings pairs offset) +
Contains an offset relative from the start of the MFT attribute
| 18 | 2 | | Compression unit size +
Contains the compression unit size as `2^(n)` number of cluster blocks. +
This value is used for compressed data in the data runs. +
[yellow-background]*A value of 0 indicates the attribute data is uncompressed.?* +
[yellow-background]*Seen on XP, compressed MFT attribute data with compression unit size of 0.* +
[yellow-background]*So it looks more the default compression unit size (16 cluster blocks) should be used.*
| 20 | 4 | | Padding +
Contains zero-bytes
| 24 | 8 | | Allocated data size (or allocated length) +
Contains the allocated data size in number of bytes. +
This value is not valid if the first VCN is nonzero.
| 32 | 8 | | Data size (or file size) +
Contains the data size in number of bytes. +
This value is not valid if the first VCN is nonzero.
| 40 | 8 | | Valid data size (or valid data length) +
Contains the valid data size in number of bytes.
This value is not valid if the first VCN is nonzero. +
4+| _If compression unit size > 0_
| 48 | 8 | | Total allocated size +
Contains the total allocated size in number of cluster blocks.
|===

[NOTE]
The total size of the data runs should be larger or equal to the data size.

[NOTE]
Windows will fill data ranges beyond the valid data size with 0-byte values.
The data size remains unchanged. This applies to compressed and uncompressed
data. If the first VCN is zero a valid data size of 0 represents a file
entirely filled with 0-byte values.

==== Attribute name

The attribute name is variable of size and consists of:

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | ... | | Name +
Contains an UTF-16 little-endian without the end-of-string character
|===

==== Data runs

The data runs are stored in a variable size (data) runlist. This runlist
consists of runlist elements.

A runlist element is variable of size and consists of:

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0.0  | 4 bits | | Number of cluster blocks value size +
Contains the number of bytes used to store the data run size
| 0.4 | 4 bits | | Cluster block number value size +
Contains the number of bytes used to store the data run size
| 1 | Size value size | | Data run number of cluster blocks +
Contains the number of cluster blocks
| ... | Cluster block number value size | | Data run cluster block number +
See below.
|===

The data run cluster block number is a singed value, where the MSB is the
singed bit, e.g. if the data run cluster block contains \'dbc8' it corresponds
to the 64-bit value 0xffffffffffffdbc8.

The first data run offset contains the absolute cluster block number where
successive data run offsets are relative to the last data run offset.

[NOTE]
The cluster block number byte size is the first nibble when reading the byte
stream, but here it is represented as the upper nibble of the first byte.

The last runlist element is an empty value size tuple; in other words a 0 byte.

[yellow-background]*Does a data run with a "number of cluster blocks value size"
of 0 represent this as well?*

The MFT attribute data flag (ATTRIBUTE_FLAG_SPARSE) indicates if the data
stream is sparse or not.

A sparse data run has a "cluster block number value size" 0, representing there
is no offset (cluster block number). A sparse data run should be filled with
0-byte values.

[NOTE]
Compressed files also define sparse data runs without setting the sparse flag.

[yellow-background]*TODO: what about data runs with a cluster block number
value size of 0 but without the necessary flags? Seen in ADS: $BadClus:$Bad.
Assuming for now the data run is sparse.*

The MFT attribute data flags (0x00ff) indicate if the data stream is compressed
or not. The currently known value for LZNT1 compression is 1.

[yellow-background]*TODO: have a look at Windows 10 compression see if they use
a different value.*

The data is stored in compression unit blocks. A compression unit typically
consists of 16 cluster blocks. However the actual value is stored in the
non-resident MFT attribute. See <<compression,Compression>> for more
information on how to determine which data runs store the compressed and which
do not.

[NOTE]
Compression is supported upto NTFS file systems with a cluster block size of
4096 bytes or less.

The compression is specified on a pre attribute basis. Where an attribute
chain can consists of attribute with compressed and uncompressed attribute
data. [yellow-background]*Note that it is unknown if mixing compressed and
uncompressed attributes is supported by the Windows implementation.*

According to `[RUSSON05]` the size of the runlist is rounded up to the next
multitude of 4 bytes. The size of the trailing data can be even larger than 3
and are not always zero-bytes.

See `[CARRIER05]` and/or `[RUSSON05]` for examples on reading the runlist.

== The attributes

=== [[attribute_types]]The attribute types

Technically the attribute types are stored in the `$AttrDef` metadata file.
Also see section: <<attribute_definitions,The attribute definitions>>

[cols="1,1,5",options="header"]
|===
| Value | Identifier | Description
| 0x00000000 | | Unused
| 0x00000010 | $STANDARD_INFORMATION | Standard information
| 0x00000020 | $ATTRIBUTE_LIST | Attributes list
| 0x00000030 | $FILE_NAME | The file or directory name
| 0x00000040 | $VOLUME_VERSION | Volume version +
Removed in NTFS version 3.0
| 0x00000040 | $OBJECT_ID | Object identifier +
Introduced in NTFS version 3.0
| 0x00000050 | $SECURITY_DESCRIPTOR | Security descriptor
| 0x00000060 | $VOLUME_NAME | Volume name
| 0x00000070 | $VOLUME_INFORMATION | Volume information
| 0x00000080 | $DATA | Data stream
| 0x00000090 | $INDEX_ROOT | Index root
| 0x000000a0 | $INDEX_ALLOCATION | Index allocation
| 0x000000b0 | $BITMAP | Bitmap
| 0x000000c0 | $SYMBOLIC_LINK | Symbolic link +
Removed in NTFS version 3.0
| 0x000000c0 | $REPARSE_POINT | Reparse point +
Introduced in NTFS version 3.0
| 0x000000d0 | $EA_INFORMATION | (HPFS) extended attribute information
| 0x000000e0 | $EA | (HPFS) extended attribute
| 0x000000f0 | $PROPERTY_SET | Property set +
Removed in NTFS version 3.0
| 0x00000100 | $LOGGED_UTILITY_STREAM | Logged utility stream +
Introduced in NTFS version 3.0
| | |
| 0x00001000 | | First user defined attribute
| | |
| 0xffffffff | | End of attributes marker
|===

=== The standard information attribute

The standard information attribute ($STANDARD_INFORMATION) contains the basic
file entry metadata. It is stored as a resident MFT attribute.

The standard information data (STANDARD_INFORMATION) is either 48 or 72 bytes
of size and consists of:

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | 8 | | Creation date and time +
Contains a FILETIME
| 8 | 8 | | Last modification date and time +
(Also referred to as last written date and time) +
Contains a FILETIME
| 16 | 8 | | MFT entry last modification date and time +
Contains a FILETIME
| 24 | 8 | | Last access date and time +
Contains a FILETIME
| 32 | 4 | | File attribute flags +
See section: <<file_attribute_flags,File attribute flags>>
| 36 | 4 | | Maximum number of versions +
[yellow-background]*What does it contain and what is it used for?*
| 40 | 4 | | Version number +
[yellow-background]*What does it contain and what is it used for?*
| 44 | 4 | | Class identifier +
[yellow-background]*What does it contain and what is it used for?*
4+| _Introduced in NTFS version 3.0_
| 48 | 4 | | Owner identifier +
[yellow-background]*What does it contain and what is it used for?*
| 52 | 4 | | Security descriptor identifier +
Contains the entry number in the security ID index ($Secure:$SII) +
See section: <<access_control,Access Control>>
| 56 | 8 | | Quota charged +
[yellow-background]*What does it contain and what is it used for?*
| 64 | 8 | | Update Sequence Number (USN) +
[yellow-background]*What does it contain and what is it used for?*
|===

=== The attribute list attribute

The attribute list attribute ($ATTRIBUTE_LIST) is a list of attributes in an
MFT entry. The attributes stored in the list are placeholders for other
attributes. Some of these attributes could not be stored in the MFT entry due
to space limitations. The attribute list attribute can be stored as either a
resident (for a small amount of data) and non-resident MFT attribute.

The attribute list data contains an array of attribute list entries.

Note that MFT entry 0 also can contain an attribute list.
[yellow-background]*Does NTFS allow to store listed attributed beyond the first
data run?*

==== The attribute list entry

The attribute list entry consists of:

* the attribute list entry header
* the the attribute name
* alignment padding (8-byte alignment), can contain remnant data

===== The attribute list entry header

The attribute list entry header (ATTRIBUTE_LIST_ENTRY) is 26 bytes of size and consists of:

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | 4 | | Attribute type (or type code) +
See section: <<attribute_types,The attribute types>>
| 4 | 2 | | Size (or record length) +
The size of the attribute including the 6 bytes of the attribute type and size
| 6 | 1 | | Name size (or name length) +
Contains the number of characters with-out the end-of-string character
| 7 | 1 | | Name offset +
Contains an offset relative from the start of the attribute list entry
| 8 | 8 | | Data first (or lowest) VCN
| 16 | 8 | | File reference (or segment reference) +
The file reference to the MFT entry that contains (part of) the attribute data +
See section: <<file_reference,The file reference>>
| 24 | 2 | | Attribute identifier +
[yellow-background]*An unique identifier to distinguish between attributes that contain segmented data.*
|===

The data first VCN is used when the attribute data is stored in multiple MFT
entries. The attribute list contains an attribute list entry for every MFT
entry. The corresponding MFT entry will contain an MFT attribute containing the
attribute data. See `[CARRIER05]` pages 365 and 366 for more information.

===== Attribute name

The attribute name is variable of size and consists of:

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | ... | | Name +
Contains an UTF-16 little-endian without the end-of-string character
|===

=== [[file_name_attribute]]The file name attribute

The file name attribute ($FILE_NAME) contains the basic file system
information, like the parent file entry, MAC times and filename. It is stored
as a resident MFT attribute.

The file name data (FILE_NAME) is variable of size and consists of:

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | 8 | | Parent file reference +
See section: <<file_reference,The file reference>>
| 8 | 8 | | Creation date and time +
Contains a FILETIME
| 16 | 8 | | Last modification date and time +
(Also referred to as last written date and time) +
Contains a FILETIME
| 24 | 8 | | MFT entry last modification date and time +
Contains a FILETIME
| 32 | 8 | | Last access date and time +
Contains a FILETIME
| 40 | 8 | | Allocated (or reserved) file size +
See below.
| 48 | 8 | | File size +
See below.
| 56 | 4 | | File attribute flags +
See section: <<file_attribute_flags,File attribute flags>>
| 60 | 4 | | Extended data +
See below.
| 64 | 1 | | Name string size +
Contains the number of characters with-out the end-of-string character
| 65 | 1 | | Namespace of the name string +
| 66 | ... | | Name string
Contains an UTF-16 little-endian without an end-of-string character
|===

The extended data contains:

* the reparse point tag (see section <<reparse_point_rage,Reparse point tag>>) if the reparse point file attribute flag (FILE_ATTRIBUTE_REPARSE_POINT) is set;
* [yellow-background]*the extended attribute data size.*

The allocated file size and file size values do not always contain accurate
values when stored in a MFT attribute, see `[CARRIER05]` page 363 for more
information. [CARRIER05] also states that the file size values are accurate
when 'used in a directory index' (stored in an index value), however this seems
to be true for most files but not for all. At least the $MFT and $MFTMirr
metadata file directory entries on a Windows Vista NTFS volume were found to
contain the same value as the corresponding MFT entries, which were not equal
to the size of the data stream.

An MFT attribute can contain multiple file name attributes, e.g. for a separate
(long) name and short name.

In several cases on a Vista NTFS volume the MFT entry contained both a DOS &
Windows and POSIX namespace name. However the directory entry index ($I30) of
the parent directory only contained the DOS & Windows name.

In case of a hard link the MFT entry will contain additional file name
attributes with the parent file reference of each hard link.

==== Namespace

[cols="1,1,5",options="header"]
|===
| Value | Identifier | Description
| 0 | POSIX | Case sensitive character set that consists of all Unicode characters except for: +
\0 (zero character), +
/ (forward slash). +
 +
The : (colon) is valid for NTFS but not for Windows. +
| 1 | FILE_NAME_NTFS (or WINDOWS) | A case insensitive sub set of the POSIX character set that consists of all Unicode characters except for: +
" * / : < > ? \ \| +
 +
Note that names cannot end with a . (dot) or ' ' (space).
| 2 | FILE_NAME_DOS (or DOS) | A case insensitive sub set of the WINDOWS character set that consists of all upper case ASCII characters except for: +
" * + , / : ; < = > ? \ +
 +
Note the name must follow the 8.3 format.
| 3 | DOS_WINDOWS | Both the DOS and WINDOWS names are identical +
[yellow-background]*Which is the same as the DOS character set, with the exception that lower case is used as well.*
|===


[NOTE]
The Windows API function CreateFile allows to create case sensitive file names when the flag FILE_FLAG_POSIX_SEMANTICS is set.

==== Long to short name conversion

Basically the conversion from a long name to short name boils down to the
approach mentioned below. Note that it differs from the approach mentioned in
`[RUSSON05]` and `[MSSUPPORT]`, in regard of the third case to make the short name
unique.

In the long name:

* ignore Unicode characters beyond the first 8-bit (extended ASCII)
* ignore control characters and spaces (character < 0x20)
* ignore non-allowed characters (" * + , / : ; < = > ? \)
* ignore dots except the last one (extension) and one at the start of the name
* make all letters upper case

Make the name unique:

1. use the characters 1 to 6 add ~1 and if the long name has an extension add the a dot and its first 3 letters
2. if the name already exists try \~2 up to ~9
3. if the name already exists use [yellow-background]*some 16-bit hexadecimal value* for characters 3 to 6 with ~1

[yellow-background]*`[MSDN]` Generates the next four letters of the short file
name by mathematically manipulating the remaining letters of the long file
name.*

[yellow-background]*Note: behavior dependent on fsutil?*

case 1: "Program Files" becomes "PROGRA\~1" or " ~PLAYMOVIE.REG" becomes "\~PLAYM~1.REG"

case 2: "Program Data", in the same directory as "Program Files", becomes "PROGRA~2"

case 3:
"x86_microsoft-windows-r..ry-editor.resources_31bf3856ad364e35_6.0.6000.16386_en-us_f89a7b0005d42fd4",
in a directory with a lot of filenames starting with "x86_microsoft", becomes "X8FCA6~1.163"

=== The volume version attribute

The volume version attribute ($VOLUME_VERSION) contains [yellow-background]*TODO*

[yellow-background]*Need a pre NTFS 3.0 volume with this attribute. $AttrDef
indicates the attribute to be 8 bytes of size.*

=== The object identifier attribute

The object identifier attribute ($OBJECT_ID) contains distributed link tracker
properties. It is stored as a resident MFT attribute.

The object identifier data is either 16 or 64 bytes of size and consists of:

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | 16 | | Droid file identifier +
Contains a GUID
| 16 | 16 | | Birth droid volume identifier +
Contains a GUID
| 32 | 16 | | Birth droid file identifier +
Contains a GUID
| 48 | 16 | | Birth droid domain identifier +
Contains a GUID
|===

Droid in this context refers to CDomainRelativeObjId.

=== The security descriptor attribute

[yellow-background]*TODO: does this override any value in $Secure:$SDS?*

The security descriptor attribute ($SECURITY_DESCRIPTOR) contains a Windows NT
security descriptor. It can be stored as either a resident (for a small amount
of data) and non-resident MFT attribute.

See: https://github.com/libyal/libfwnt/blob/master/documentation/Security%20Descriptor.asciidoc[[LIBFWNT\]]

=== The volume name attribute

The volume name attribute ($VOLUME_NAME) contains the name of the volume. It is
stored as a resident MFT attribute.

The volume name data is variable of size and consists of:

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | ... | | Name string +
Contains an UTF-16 little-endian without an end-of-string character
|===

The volume name attribute is used in the $Volume metadata file MFT entry.

=== The volume information attribute

The volume information attribute ($VOLUME_INFORMATION) contains the name of the
volume. It is stored as a resident MFT attribute.

The volume information data is 12 bytes of size and consists of:

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | 8 | | [yellow-background]*Unknown (empty value?)*
| 8 | 1 | | Major version number
| 9 | 1 | | Minor version number
| 10 | 2 | | Volume flags
|===

The volume information attribute is used in the $Volume metadata file MFT entry.

==== Volume flags

[cols="1,1,5",options="header"]
|===
| Value | Identifier | Description
| 0x0001 | | [yellow-background]*Is dirty*
| 0x0002 | | [yellow-background]*Re-size journal (LogFile)*
| 0x0004 | | [yellow-background]*Upgrade on next mount*
| 0x0008 | | [yellow-background]*Mounted on Windows NT 4*
| 0x0010 | | [yellow-background]*Delete USN underway*
| 0x0020 | | [yellow-background]*Repair object identifiers*
| | |
| 0x8000 | | [yellow-background]*Modified by chkdsk*
|===

=== The data stream attribute

The data stream attribute ($DATA) contains the file data. It can be stored as
either a resident (for a small amount of data) and non-resident MFT attribute.

Also note that multiple data attributes for the same data stream can be used in
the attribute list to define different parts of the data stream data. The first
data stream attribute will contain the size of the entire data stream data.
Other data stream attributes should have a size of 0. Also see:
<<attribute_chains,Attribute chains>>.

=== The index root attribute

The index root attribute ($INDEX_ROOT) contains the root of the index tree. It
is stored as a resident MFT attribute.

See section: <<index,The index>> and <<index_root,The index root>>.

=== The index allocation attribute

The index allocation attribute ($INDEX_ALLOCATION) contains an array of index
entries. It is stored as a non-resident MFT attribute.

Note that the index allocation attribute itself does not define which attribute
type it contains in the index value data. For this information it needs the
corresponding index root attribute.

Also note that multiple index allocation attributes for the same index can be
used in the attribute list to define different parts of the index allocation
data. The first index allocation attribute will contain the size of the entire
index allocation data. Other index allocation attributes should have a size of
0. Also see: <<attribute_chains,Attribute chains>>.

See section: <<index,The index>>.

=== The bitmap attribute

The bitmap attribute ($BITMAP) contains the allocation bitmap. It can be stored
as either a resident (for a small amount of data) and non-resident MFT
attribute.

It is used to maintain information about which entry is used and which is not.
Every bit in the bitmap represents an entry. The index is stored byte-wise with
the LSB of the byte corresponds to the first allocation element; the allocation
element can represent several things, see below.

The allocation element is allocated if the corresponding bit contains 1 or
unallocated if 0.

It is known to be used in:

* the MFT (nameless), where an allocation element represents a MFT entry;
* indexes ($I##), where an allocation element represents an index entry.

=== The symbolic link attribute

The symbolic link attribute ($SYMBOLIC_LINK) contains [yellow-background]*TODO*

[yellow-background]*Need a pre NTFS 3.0 volume with this attribute. $AttrDef
indicates the attribute is variable of size.*

=== The reparse point attribute

The reparse point attribute ($REPARSE_POINT) contains information about a file
system-level link. It is stored as a resident MFT attribute.

See section: <<reparse_point,The reparse point>>.

=== The (HPFS) extended attribute information

The (HPFS) extended attribute information ($EA_INFORMATION) contains
information about the extended attribute ($EA).

The extended attribute information data is 8 bytes of size and consists of:

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | 2 | | Size of an extended attribute entry
| 2 | 2 | | Number of extended attributes which have NEED_EA set +
[yellow-background]*TODO: determine what this flag is used for*
| 4 | 4 | | Size of the extended attribute ($EA) data
|===

E.g.
....
00000000: 08 00 00 00 18 00 00 00                            ........
....

=== The (HPFS) extended attribute

The (HPFS) extended attribute ($EA) contains the extended attribute data.

The extended attribute data is variable of size and consists of:

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | 4 | | Offset to next extended attribute entry +
The offset is relative from the start of the extended attribute data
| 4 | 1 | | Flags +
0x80 => NEED_EA (Need EA) flag
| 5 | 1 | | Number of characters of the extended attribute name
| 6 | 2 | | Value data size
| 8 | ... | | The extended attribute name +
Contains an ASCII string +
[yellow-background]*TODO: is this value 16-bit aligned?*
| ... | ... | | Value data
| ... | ... | | [yellow-background]*TODO: unknown trailing data*
|===

E.g.
....
00000000: 18 00 00 00 00 09 04 00  2e 55 4e 49 58 41 54 54   ........ .UNIXATT
00000010: 52 00 b6 01 00 00 03 87                            R.......
....

==== UNITATTR extended attribute value data

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | 4 | | [yellow-background]*TODO: is this an equivalent of st_mode?*
|===

=== The property set attribute

The property set attribute ($PROPERTY_SET) contains [yellow-background]*TODO*

[yellow-background]*Need a pre NTFS 3.0 volume with this attribute. $AttrDef
indicates does not seem to always define this attribute.*

=== The logged utility stream attribute

[yellow-background]*attribute type for storing additional data for the files and directories*

[yellow-background]*resident, known to cause problems when non-resident on Windows Vista*

[cols="1,1,5",options="header"]
|===
| Value | Identifier | Description
| $EFS | | Encrypted NTFS (EFS)
| $TXF_DATA | | Transactional NTFS (TxF)
|===

[yellow-background]*TODO add text*

=== Attribute chains

Multiple attributes can make up a single attribute, e.g. the attributes:

1. $INDEX_ALLOCATION ($I30) VCN: 0
2. $INDEX_ALLOCATION ($I30) VCN: 596

The first attribute will contain the size of the data defined by all the
attributes. Other attributes should have a size of 0.

It is assumed that the attributes in a chain must be continuous and defined
in-order.

== [[attribute_types]]The attribute types

The attribute types are stored in the `$AttrDef` metadata file.

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | 128 | | Attribute name +
Contains an UTF-16 little-endian [yellow-background]*with the end-of-string character?* +
[yellow-background]*The unused bytes are filled with 0-byte values*
| 128 | 4 | | Attribute type (or type code)
| 132 | 8 | | [yellow-background]*Unknown (empty values?)*
| 140 | 4 | | [yellow-background]*Unknown (flags?)*
Seen: 0x40, 0x42, 0x80
| 144 | 8 | | [yellow-background]*Unknown (minimum attribute size?)*
| 152 | 8 | | [yellow-background]*Unknown (maximum attribute size?)* +
Seen: -1 (no maximum?), 48
|===

....
00000000  24 00 53 00 54 00 41 00  4e 00 44 00 41 00 52 00  |$.S.T.A.N.D.A.R.|
00000010  44 00 5f 00 49 00 4e 00  46 00 4f 00 52 00 4d 00  |D._.I.N.F.O.R.M.|
00000020  41 00 54 00 49 00 4f 00  4e 00 00 00 00 00 00 00  |A.T.I.O.N.......|
00000030  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000080  10 00 00 00 00 00 00 00  00 00 00 00 40 00 00 00  |............@...|
00000090  30 00 00 00 00 00 00 00  30 00 00 00 00 00 00 00  |0.......0.......|
000000a0  24 00 41 00 54 00 54 00  52 00 49 00 42 00 55 00  |$.A.T.T.R.I.B.U.|
000000b0  54 00 45 00 5f 00 4c 00  49 00 53 00 54 00 00 00  |T.E._.L.I.S.T...|
000000c0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000120  20 00 00 00 00 00 00 00  00 00 00 00 80 00 00 00  | ...............|
00000130  00 00 00 00 00 00 00 00  ff ff ff ff ff ff ff ff  |................|
00000140  24 00 46 00 49 00 4c 00  45 00 5f 00 4e 00 41 00  |$.F.I.L.E._.N.A.|
....

== [[index]]The index

The index structures are used for various purposes one of which are the
directory entries.

The root of the index is stored in index root. The index root attribute defines
which type of attribute is stored in the index and the root index node.

If the index is too large part of the index is stored in an index allocation
attribute with the same attribute name. The index allocation attribute defines
a data stream which contains index entries. Each index entry contains an index
node.

See `[CARRIER05]` page 378 for an illustration how the index root and index
allocation attribute relate.

An index consists of a tree, where both the branch and index leaf nodes contain
the actual data. E.g. in case of a directory entries index, any node that
contains index value data make up for the directory entries.

The index value data in a branch node signifies the upper bound of the values
in the that specific branch. E.g. if directory entries index branch node
contains the name 'textfile.txt' all names in that index branch are smaller
than 'textfile.txt'. Note the actual sorting order is dependent on the
collation type defined in the index root attribute.

The index allocation attribute is accompanied by a bitmap attribute with the
corresponding attribute name. The bitmap attribute defines the allocation of
virtual cluster blocks within the index allocation attribute data stream.

[NOTE]
The index allocation attribute can be present even though it is not used.

=== Common used indexes

Indexes commonly used by NTFS are:

[cols="1,1,5",options="header"]
|===
| Value | Identifier | Description
| $I30 | | Directory entries (used by directories)
| $SDH | | Security descriptor hashes (used by $Secure)
| $SII | | Security descriptor identifiers (used by $Secure)
| $O | | Object identifiers (used by $ObjId)
| $O | | Owner identifiers (used by $Quota)
| $Q | | Quotas (used by $Quota)
| $R | | Reparse points (used by $Reparse)
|===

=== [[index_root]]The index root

The index root consists of:

* index root header
* index node header
* an array of index values

==== The index root header

The index root header is 16 bytes of size and consists of:

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | 4 | | Attribute type +
Contains the type of the indexed attribute or 0 if none
| 4 | 4 | | Collation type +
Contains a value to indicate the ordering of the index entries +
See section: <<collection_type,Collation type>>
| 8 | 4 | | Index entry size
| 12 | 4 | | Index entry number of cluster blocks
|===

Note that `[CARRIER05]` and `[RUSSON05]` state that the last 3 bytes are unused
(alignment padding). However it is highly probably that the last value is
32-bit of size.

===== Version 1.2 Notes

In NT4 (version 1.2)

* the index entry size does not have to match the index entry size in the volume header. The correct size seems to be the value in the index root header.

==== [[collation_type]]Collation type

[cols="1,1,5",options="header"]
|===
| Value | Identifier | Description
| 0x00000000 | COLLATION_BINARY | Binary +
The first byte is most significant
| 0x00000001 | COLLATION_FILENAME | Unicode strings case-insensitive
| 0x00000002 | COLLATION_UNICODE_STRING | Unicode strings case-sensitive +
Upper case letters should come first
| | |
| 0x00000010 | COLLATION_NTOFS_ULONG | Unsigned 32-bit little-endian integer
| 0x00000011 | COLLATION_NTOFS_SID | NT security identifier (SID)
| 0x00000012 | COLLATION_NTOFS_SECURITY_HASH | Security hash first, then NT security identifier
| 0x00000013 | COLLATION_NTOFS_ULONGS | An array of unsigned 32-bit little-endian integer values
|===

=== The index entry

The index entry consists of:

* the index entry header
* the index node header
* the fix-up values
* alignment padding (8-byte alignment), contains zero-bytes
* an array of index values

==== The index entry header

The index entry header is 32 bytes of size and consists of:

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | 4 | "INDX" | Signature
| 4 | 2 | | The fix-up values offset +
Contains an offset relative from the start of the index entry header.
| 6 | 2 | | The number of fix-up values
| 8 | 8 | | Metadata transaction journal sequence number +
Contains a $LogFile Sequence Number (LSN)
| 16 | 8 | | Virtual Cluster Number (VCN) of the index entry
|===

[NOTE]
There can be more fix-up value than supported by the index entry data size.

=== The index node header

The index node header is 16 bytes of size and consists of:

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | 4 | | Index values offset +
The offset is relative from the start of the index node header
| 4 | 4 | | Index node size +
The value includes the size of the index node header. See below.
| 8 | 4 | | Allocated index node size +
The value includes the size of the index node header
| 12 | 4 | | Index node flags +
See section: <<index_node_flags,The index node flags>>
|===

Note that `[RUSSON05]` states that the last 3 bytes are unused (alignment
padding), while `[CARRIER05]` states that the last value is 32-bit of size.
Here we assume that the index node flags are a 32‑bit value.

In an index entry (index allocation attribute) the index node size includes the
size of the fix-up values and the alignment padding following it.

The remainder of the index node contains remnant data and/or zero-byte values.

==== [[index_node_flags]]The index node flags

[cols="1,1,5",options="header"]
|===
| Value | Identifier | Description
| 0x00000001 | | Has index allocation attribute +
Used in an index root attribute to indicate the presence of an index allocation attribute which contains the index values
|===

=== The index value

The index value is variable of size and consists of:

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | 8 | | File reference +
See section: <<file_reference,The file reference>>
| 8 | 2 | | Index value size
| 10 | 2 | | Index key data size
| 12 | 4 | | Index value flags
4+| _If index key data size > 0_
| 16 | ... | | Index key data
| ... | ... | | Index value data
4+| _If index value flag 0x00000001 (has sub node) is set_
| ... | 8 | | Sub node Virtual Cluster Number (VCN)
|===

The index values appear to be stored 8 byte aligned.

[NOTE]
Some sources define the index value flags as a 16-bit value followed by
2 bytes of padding.

==== The index value flags

[cols="1,1,5",options="header"]
|===
| Value | Identifier | Description
| 0x00000001 | | Has sub node +
If set the index value contains a sub node Virtual Cluster Number (VCN)
| 0x00000002 | | Is last +
If set the index value is the last in the index values array
|===

=== Index key and value data

==== Directory entry index value

The MFT attribute name of the directory entry index is: $I30.

The directory entry index value contains a file name attribute in the index
key data. See section: <<file_name_attribute,The file name attribute>>.

The index value data seems to contain remnant data.

[NOTE]
Both the short and long names of the same file have a separate index value. The
short name uses the DOS namespace and the long name the WINDOWS namespace.

[yellow-background]*Index values with a single name use either the POSIX or
DOS_WINDOWS namespace?*

A hard link to a file in the same directory will also have a separate index
value.

[yellow-background]*Does the hard link always has POSIX namespace?*

==== Security descriptor hash index value

The MFT attribute name of the security descriptor hash index is: $SDH.
It appears to only to be used by the $Secure metadata file.

See section: <<security_descriptor_hash_index_value,The security descriptor hash index value>>

==== Security descriptor identifier index value

The MFT attribute name of the security descriptor identifier index is: $SII.
It appears to only to be used by the $Secure metadata file.

See section: <<security_descriptor_identifier_index_value,The security descriptor identifier index value>>

== [[compression]]Compression

Typically NTFS compression groups 16 cluster blocks together. This group of 16
cluster blocks also named a compression unit is either \'compressed' or
uncompressed data. The term compressed is quoted here because, as you will see
below, the group of cluster blocks can also contain uncompressed data. A group
of cluster blocks is \'compressed' when it is compressed size is smaller than
its uncompressed data size.

[NOTE]
The actual compression unit size is stored in the non-resident MFT attribute.

Within a group of cluster blocks each of the 16 blocks is \'compressed'
individually see <<compression_block_based_storage,Block based storage>>.
The maximum uncompressed data size is always the cluster size (in most cases
4096).

The data runs in the $DATA stream define cluster block ranges. A simple example:
....
21 02 35 52
....

This data run defines 2 data blocks starting at block number 21045 followed by
14 sparse blocks. The total number of blocks is 16 which is the size of the
compression unit. The data is stored compressed in the first 2 blocks and the
14 sparse blocks are only there to make sure the data runs add up to the
compression unit size. They do not define actual sparse data.

Another example:
....
21 40 37 52
....

This data run defines 64 data blocks starting at block number 21047. Since
this data run is larger than the compression unit size the data is stored
uncompressed.

If the data run was e.g. 60 data blocks followed by 4 sparse blocks the first 3
compression units (blocks 1 to 48) would be uncompressed and the last
compression unit (blocks 49 to 64) would be compressed.

Also "sparse data" and "sparse compression unit" data runs can be mixed. If in
the previous example the 60 data blocks would be followed by 20 sparse blocks
the last compression unit (blocks 65 to 80) would be sparse.

[NOTE]
A compression unit can consists of multiple compressed data runs, e.g. 1 data
block followed by 4 data blocks followed by 11 sparse blocks.

[NOTE]
The sparse blocks data run can be stored in a subsequent attribute in an
attribute chain.

[yellow-background]*Does the sparse flag needs to be set for sparse compressed
files?*

[yellow-background]*Is resident data is always uncompressed?*

Also see `[RUSSON05]` for more detailed examples.

=== [[compression_block_based_storage]]Block based storage

NTFS compression stores the \'compressed' data in blocks. Each block has a 2
byte block header.

The block is variable of size and consists of:

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | 2 | | Block size
| 2 | (compressed data size) | |
|===

The upper 4 bits of the block size are used as flags.

[cols="1,5",options="header"]
|===
| Bit(s) | Description
| 0 ‑ 11 | Compressed data size
| 12 - 14 | [yellow-background]*Unknown flags*
| 15 | Data is compressed
|===

=== LZNT1 compression method

The LZNT1 compression method is based on LZ77 compression.

Every compression block consists of tagged compression groups. A tagged group
consist of 8 values (not bytes) preceded by a tag byte:
....
tag A B C D E F G H
....

The LSB of the tag byte represents the first value in the group, the MSB the
last

* a tag bit of 0 indicates an uncompressed byte;
* a tag bit of 1 indicates compressed data using a little-endian 16-bit (2-byte) compression tuple (meaning combination of two values).

The compression tuple contains an offset (back reference) and a size value.

Where the size is the actual size minus 3. Use the following calculation to
correct the size value in the tuple.
....
size = size + 3
....

And the offset a positive representation of a back reference minus 1. Use the
following calculation to correct the offset value in the tuple.
....
offset = -1 * ( offset + 1 )
....

The compression tuple uses a dynamic amount of bits to store the offset and
size values.

The calculation of the amount of bits used for the offset and size values is as
following:

* at the uncompressed data block offset 0, the size is stored in the least significant 12 bits of size and the offset 4 bits
* the larger the uncompressed data block offset, the larger the amount of bits are used for the offset value and the smaller the amount of bits for the size .

The following calculation is used to determine the amount of bits to store the offset and size values.
....
compression_tuple_size_offset_shift = 12;
compression_tuple_size_mask         = 0xfff;

for( iterator = uncompressed_data_block_offset - 1;
     iterator >= 0x10;
     iterator >>= 1 )
{
	/* bit shift for the offset value */
	compression_tuple_size_offset_shift--;

	/* bit mask for size value */
	compression_tuple_size_mask >>= 1;
}
....

The tuple is uncompressed by copying the byte at the offset in the uncompressed
data to the end of the uncompressed data. This is repeated for the size value
of the tuple.

[NOTE]
The offset value itself does not change, the offset remains fixed relative to
the end of the uncompressed data. However this means that for every increment
of the size value the offset refers to another byte in the uncompressed data.
Consider the following example.

==== Example

Consider the following tagged compression group:
....
0x02 0x20 0xfc 0x0f
....

The tag byte consists of:
....
0x02 => 00000010b
....

This means that the 2nd and 3rd values contain a 16-bit compression tuple.
....
0x0ffc
....

Because this compression tuple is near the start of the uncompressed data the
offset shift is 12 and the size mask is 0x0fff.
....
offset:	0x0ffc >> 12    => -1 * ( 0 + 1 ) => -1
size:	0x0ffc & 0x0fff => 4092 + 3       => 4095
....

The algorithm starts with an uncompressed value of 0x20 which represents the
space character (ASCII). This value is added to the uncompressed data. Next the
algorithm reads the compression tuple and determines the offset and size
values. The offset refers to the previous space value in the uncompressed data
and add this to uncompressed data. And so on. Note that the offset remains
referring to the last value in the uncompressed data. In the end we end up with
a block of 4096 spaces.

Now consider the following uncompressed data:
....
#include <ntfs.h>\n
#include <stdio.h>\n
....

Note that the \n is the string representation of the newline character (ASCII:
0x0a)

This is logically compressed to:
....
#include <ntfs.h>\n(-18,10)stdio(-17,4)
....

In the example above the tuples are represented by (offset,size).

The first part of this is data stored with tag bytes looks like:
....
00000000b '#' 'i' 'n' 'c' 'l' 'u' 'd' 'e'
00000000b ' ' '<' 'n' 't' 'f' 's' '.' 'h'
00000100b '>' '\n' 0x07 0x88 's' 't' 'd' 'i' 'o'
00000001b 0x01 0x80
....

And as a hexdump:
....
00000000  00 23 69 6e 63 6c 75 64  65 00 20 3c 6e 74 66 73  |.#include. <ntfs|
00000010  2e 68 04 3e 0a 07 88 3c  73 74 64 69 01 01 80     |.h.>...stdio... |
....

For the first tuple the offset shift is 11 and the size mask is 0x07ff. The
tuple consists of:
....
offset:	0x8807 >> 11    => -1 * ( 17 + 1 ) => -18
size:	0x8807 & 0x07ff =>  7 + 3          => 10
....

This tuples refer to:
....
(-18,10) => #include <
....

== [[reparse_point]]The reparse point

The reparse point is used to create file system-level links. Reparse data is stored in the reparse point attribute.

The reparse point data (REPARSE_DATA_BUFFER) is variable of size and consists of:

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | 4 | | Reparse point tag
| 4 | 2 | | Reparse data size
| 6 | 2 | 0 | Reserved
| 8 | ... | | Reparse data
|===

[yellow-background]*What about the GUID mentioned in `[RUSSON05]` in third
party reparse points.*

[yellow-background]*`[MSDN]` ReparseGuid: A 16-byte GUID that uniquely
identifies the owner of the reparse point. Reparse pointGUIDs are assigned by
the implementer of a file system, the file system filter driver, or the
minifilter driver. The implementer must generate one GUID to use with their
assigned reparse point tag, and must always use this GUID as the ReparseGuid
for that tag.*

=== [[reparse_point_tag]]Reparse point tag

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0.0  | 16 bits | | Type
| 2.0  | 12 bits | | Reserved
| 3.4 | 4 bits | | Flags
|===

==== Predefined reparse point tag values

Predefined reparse point tag values according to `[MSDN]`:

[cols="1,1,5",options="header"]
|===
| Value | Identifier | Description
| 0x00000000 | IO_REPARSE_TAG_RESERVED_ZERO | Reserved
| 0x00000001 | IO_REPARSE_TAG_RESERVED_ONE | Reserved
| | |
| 0x80000005 | IO_REPARSE_TAG_DRIVER_EXTENDER | Used by Home server drive extender
| 0x80000006 | IO_REPARSE_TAG_HSM2 | Used by Hierarchical Storage Manager Product
| 0x80000007 | IO_REPARSE_TAG_SIS | Used by single-instance storage (SIS) filter driver
| | |
| 0x8000000a | IO_REPARSE_TAG_DFS | Used by the Distributed File System (DFS)
| 0x8000000b | IO_REPARSE_TAG_FILTER_MANAGER | Used by filter manager test harness
| | |
| 0x80000012 | IO_REPARSE_TAG_DFSR | Used by the Distributed File System (DFS)
| | |
| 0xa0000003 | IO_REPARSE_TAG_MOUNT_POINT | Junction or mount point
| | |
| 0xa000000c | IO_REPARSE_TAG_SYMLINK | Symbolic link
| | |
| 0xc0000004 | IO_REPARSE_TAG_HSM | Used by Hierarchical Storage Manager Product
|===

==== Notes

[yellow-background]*single-instance storage (SIS): An NTFS feature that
implements links with the semantics of copies for files stored on an
NTFSvolume. SIS uses copy-on-close to implement the copy semantics of its
links.*

....
Is this documentation wrong or are these alternative values?

Flag	Description
0x68000005	NSS
0x68000006	NSS recover
0x68000007	SIS
0x68000008	DFS
0x88000003	Mount point
0xA8000004	HSM
0xE8000000	Symbolic link
....

==== Reparse point tag flags

[cols="1,1,5",options="header"]
|===
| Value | Identifier | Description
| 0x1 | | Reserved according to `[MSDN]`
| 0x2 | | Is alias (Name surrogate bit) +
If this bit is set, the file or directory represents another named entity in the system.
| 0x4 | | Is high-latency media +
Reserved according to `[MSDN]` +
| 0x8 | | Is native (Microsoft-bit) +
[yellow-background]*Does this flag influence the reparse point GUID?*
|===

=== Junction or mount point reparse data

The junction or mount point reparse data is variable of size and consists of:

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | 2 | | Substitute name offset +
The offset is relative from the start of the reparse name data
| 2 | 2 | | Substitute name size +
Value in bytes, the size of the end-of-string character is not included
| 4 | 2 | | Print name offset +
The offset is relative from the start of the reparse name data
| 6 | 2 | | Print name size +
Value in bytes, the size of the end-of-string character is not included
4+| _Reparse name data_
| 8 | ... | | Substitute name +
Contains an UTF-16 little-endian [yellow-background]*with the end-of-string character?*
| ... | ... | | Print name +
Contains an UTF-16 little-endian [yellow-background]*with the end-of-string character?*
|===

[yellow-background]*TODO: what do byte values like 0x02 represent in the substitute name?*
....
00000010: 5c 00 3f 00 3f 00 02 00  43 00 3a 00 5c 00 55 00   \.?.?... C.:.\.U.
00000020: 73 00 65 00 72 00 73 00  5c 00 74 00 65 00 73 00   s.e.r.s. \.t.e.s.
00000030: 74 00 5c 00 44 00 6f 00  63 00 75 00 6d 00 65 00   t.\.D.o. c.u.m.e.
00000040: 6e 00 74 00 73 00 00 00                            n.t.s...
....

=== Symbolic link reparse data

The symbolic link reparse data is variable of size and consists of:

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | 2 | | Substitute name offset +
The offset is relative from the start of the reparse name data
| 2 | 2 | | Substitute name size +
Value in bytes, the size of the end-of-string character is not included
| 4 | 2 | | Print name offset +
The offset is relative from the start of the reparse name data
| 6 | 2 | | Print name size +
Value in bytes, the size of the end-of-string character is not included
| 8 | 4 | | Symbolic link flags
4+| _Reparse name data_
| 12 | ... | | Substitute name +
Contains an UTF-16 little-endian [yellow-background]*with the end-of-string character?*
| ... | ... | | Print name +
Contains an UTF-16 little-endian [yellow-background]*with the end-of-string character?*
|===

==== Symbolic link flags

[cols="1,1,5",options="header"]
|===
| Value | Identifier | Description
| 0x00000001 | SYMLINK_FLAG_RELATIVE | The substitute name is a path name relative to the directory containing the symbolic link.
|===

== The allocation bitmap

The metadata file $Bitmap contains the allocation bitmap.

Every bit in the allocation bitmap represents a block the size of the cluster
block, where the LSB is the first bit in a byte.

== [[access_control]]Access control

The $Secure metadata file contains the security descriptors used for access control.

[cols="1,1,5",options="header"]
|===
| Type | Name | Description
| Data | $SDS | Security descriptor data stream +
Contains all the Security descriptors on the volume
| Index | $SDH | Security descriptor hash index
| Index | $SII | Security descriptor identifier index +
Contains the mapping of the security descriptor identifier (in $STANDARD_INFORMATION) to the offset of the security descriptor data (in $Secure:$SDS)
|===

[yellow-background]*TODO add text*

[yellow-background]*In the $SII index do the index values contain a 32-bit checksum?*

=== Security descriptor hash ($SDH) index

==== [[security_descriptor_hash_index_value]]The security descriptor hash index value

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
4+| _Key data_
| 0 | 4 | | Security descriptor hash
| 4 | 4 | | Security descriptor identifier
4+| _Value data_
| 8 | 4 | | Security descriptor hash
| 12 | 4 | | Security descriptor identifier
| 16 | 8 | | Security descriptor data offset (in $SDS)
| 24 | 4 | | Security descriptor data size (in $SDS)
| 28 | 4 | | [yellow-background]*Alignment padding (8-byte alignment)* +
[yellow-background]*Contains string "I\x00I\x00" ?*
|===

=== Security descriptor identifier ($SII) index

==== [[security_descriptor_identifier_index_value]]The security descriptor identifier index value

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
4+| _Key data_
| 0 | 4 | | Security descriptor identifier
4+| _Value data_
| 4 | 4 | | Security descriptor hash +
[yellow-background]*TODO describe the hash algorithm*
| 8 | 4 | | Security descriptor identifier
| 12 | 8 | | Security descriptor data offset (in $SDS)
| 20 | 4 | | Security descriptor data size (in $SDS)
|===

=== Security descriptor ($SDS) data stream

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | 4 | | Security descriptor hash
| 4 | 4 | | Security descriptor identifier
| 12 | 8 | | Security descriptor data offset (in $SDS)
| 20 | 4 | | Security descriptor data size (in $SDS)
| 24 | ... | | Security descriptor data +
See: https://github.com/libyal/libfwnt/blob/master/documentation/Security%20Descriptor.asciidoc[[LIBFWNT\]]
| ... | ... | | 16-bit alignment padding.
|===

== The object identifiers

[yellow-background]*TODO add text*

=== $ObjID:$O

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
4+| _Key data_
| 0 | 16 | | File (or object) identifier +
Contains a GUID
4+| _Value data_
| 4 | 8 | | File reference +
See section: <<file_reference,The file reference>>
| 12 | 16 | | Birth droid volume identifier +
Contains a GUID
| 28 | 16 | | Birth droid file (or object) identifier +
Contains a GUID
| 44 | 16 | | Birth droid domain identifier +
Contains a GUID
|===

....
00000000  00 00 00 00 13 00 00 00  00 10 00 00 01 00 00 00  |................|
00000010  10 00 00 00 88 00 00 00  88 00 00 00 01 00 00 00  |................|
00000020  20 00 38 00 00 00 00 00  60 00 10 00 01 00 00 00  | .8.....`.......|

OBJECT_ID: 43ecee59-e2b3-11dc-ad7e-001c2582598f of root directory

00000030  59 ee ec 43 b3 e2 dc 11  ad 7e 00 1c 25 82 59 8f  |Y..C.....~..%.Y.|

MFT file reference
OBJECT_ID: e6a67b60-c0b5-4b53-b8fe-94470c83df89 of $Volume

00000040  05 00 00 00 00 00 05 00  60 7b a6 e6 b5 c0 53 4b  |........`{....SK|
00000050  b8 fe 94 47 0c 83 df 89  59 ee ec 43 b3 e2 dc 11  |...G....Y..C....|
00000060  ad 7e 00 1c 25 82 59 8f  00 00 00 00 00 00 00 00  |.~..%.Y.........|
00000070  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000080  00 00 00 00 00 00 00 00  18 00 00 00 03 00 00 00  |................|
00000090  01 00 00 00 00 00 00 00                           |........|
....

== [[log_file]]Metadata transaction journal (log file)

The metadata file $LogFile contains the metadata transaction journal.

[yellow-background]*TODO add text.*

* Log File Service restart page header
* <<fix_up_values,fix-up values>>

=== Log File service restart page header

The Log File service restart page header (LFS_RESTART_PAGE_HEADER) is 30 bytes
of size and consists of:

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
4+| _MULTI_SECTOR_HEADER_
| 0 | 4 | "CHKD" +
"RCRD" +
"RSTR" | Signature
| 4 | 2 | | The fix-up values offset +
Contains an offset relative from the start of the restart page header +
According to `[MSDN]` this value is the update sequence array offset
| 6 | 2 | | The number of fix-up values +
According to `[MSDN]` this value is the update sequence array size.
4+|
| 8 | 8 | | Checkdisk last LSN
| 16 | 4 | | System page size
| 20 | 4 | | Log page size
| 24 | 2 | | Restart offset
| 26 | 2 | | Minor format version
| 28 | 2 | | Major format version +
-1 => Beta Version +
0 => Transition +
1 => Update sequence support
|===

=== Notes

Log File Service record header (LFS_RECORD_HEADER)

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | 8 | | Metadata transaction journal sequence number +
Contains a $LogFile Sequence Number (LSN)
| 8 | 8 | | Previous metadata transaction journal sequence number +
Contains a $LogFile Sequence Number (LSN)
| 16 | 8 | | Undo next metadata transaction journal sequence number +
Contains a $LogFile Sequence Number (LSN)
|===

....
LFS_RECORD_HEADER {

    //
    //  The following field is the size of data area for this record.  The
    //  log record header will be padded if necessary to fill to a 64-bit
    //  boundary, so the client data will begin on a 64-bit boundary to
    //  insure that all of his data is 64-bit aligned.  The below value
    //  has not been padded to 64 bits however.
    //

    ULONG ClientDataLength;

    //
    //  Client ID.  This identifies the owner of this log record.  The owner
    //  is uniquely identified by his offset in the client array and the
    //  sequence number associated with that client record.
    //

    LFS_CLIENT_ID ClientId;

    //
    //  This the Log Record type.  This could be a commit protocol record,
    //  a client restart area or a client update record.
    //

    LFS_RECORD_TYPE RecordType;

    //
    //  Transaction ID.  This is used externally by a client (Transaction
    //  Manager) to group log file entries.
    //

    TRANSACTION_ID TransactionId;

    //
    //  Log record flags.
    //

    USHORT Flags;

    //
    //  Alignment field.
    //

    USHORT AlignWord;
....

== [[usn_change_journal]]USN change journal

The metadata file $Extend\$UsnJrnl contains the USN change journal. It
is a sparse file in which NTFS stores records of changes to files and
directories. Applications make use of the journal to respond to file and
directory changes as they occur, like e.g. the Windows File Replication Service
(FRS) and the Windows (Desktop) Search service.

The USN change journal consists of:

* the $UsnJrnl:$Max data stream, containing metadata like the maximum size of the journal
* the $UsnJrnl:$J data stream, containing the update (or change) entries. The $UsnJrnl:$J data stream is sparse.

=== USN change journal metadata

The USN change journal metadata is 32 bytes of size and consists of:

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | 8 | | Maximum size +
Contains the number of bytes
| 8 | 8 | | Allocation delta +
Contains the number of bytes
| 16 | 8 | | Update (USN) journal identifier +
Contains a FILETIME
| 24 | 8 | | [yellow-background]*Unknown (empty)*
|===

=== USN change journal entries

The $UsnJrnl:$J data stream consists of an array of USN change journal entries.
The USN change journal entries are stored on a per block-basis and 64-bit
aligned. Therefore the remainder of the block can contain 0-byte values.

[yellow-background]*TODO: The journal block size seems to be 4096 bytes, but
could this be defined by the index entry size? It does not match the cluster
block size.*

Once the stream reaches maximum size the earliest USN change journal entries
are removed from the stream and replaced with a sparse data run.

==== USN change journal entry

===== USN change journal entry version 2

The USN change journal entry version 2 (USN_RECORD or USN_RECORD_V2) is
variable of size and consists of:

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | 4 | | Entry (or record) size
| 4 | 2 | 0x0002 | Major version
| 6 | 2 | 0x0000 | Minor version
| 8 | 8 | | File reference
| 16 | 8 | | Parent file reference
| 24 | 8 | | Update sequence number (USN) +
Contains the file offset of the USN change journal entry which is used as a unique identifier
| 32 | 8 | | Update date and time +
Contains a FILETIME
| 40 | 4 | | Update reason flags +
See section: <<update_reason_flags,Update reason flags>>
| 44 | 4 | | Update source flags +
See section: <<update_source_flags,Update source flags>>
| 48 | 4 | | Security descriptor identifier +
Contains the entry number in the security ID index ($Secure:$SII) +
See section: <<access_control,Access Control>>
| 52 | 4 | | File attribute flags +
See section: <<file_attribute_flags,File attribute flags>>
| 56 | 2 | | Name size +
Contains the byte size of the name
| 58 | 2 | | Name offset +
The offset is relative from the start of the USN change journal entry
4+| _Common_
| 60 | (name size) | | Name
| ... | ... | 0x00 | Padding
|===

===== USN change journal entry version 3

[yellow-background]*TODO need a sample to confirm this version of the USN
record is actually used by NTFS. For now it is assumed that it is likely used
by ReFS.*

The USN change journal entry version 3 (USN_RECORD_V3) is variable of size and
consists of:

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | 4 | | Entry (or record) size
| 4 | 2 | 0x0003 | Major version
| 6 | 2 | 0x0000 | Minor version
| 8 | *16* | | File reference
| 24 | *16* | | Parent file reference
| 40 | 8 | | Update sequence number (USN) +
Contains the file offset of the USN change journal entry which is used as a unique identifier
| 48 | 8 | | Update date and time +
Contains a FILETIME
| 56 | 4 | | Update reason flags +
See section: <<update_reason_flags,Update reason flags>>
| 60 | 4 | | Update source flags +
See section: <<update_source_flags,Update source flags>>
| 64 | 4 | | Security descriptor identifier +
Contains the entry number in the security ID index ($Secure:$SII) +
See section: <<access_control,Access Control>>
| 68 | 4 | | File attribute flags +
See section: <<file_attribute_flags,File attribute flags>>
| 72 | 2 | | Name size +
Contains the byte size of the name
| 74 | 2 | | Name offset +
The offset is relative from the start of the USN change journal entry
4+| _Common_
| 78 | (name size) | | Name
| ... | ... | 0x00 | Padding
|===

The file reference has changed from a 64-bit to a 128-bit value which consists
of:

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | 8 | | MFT entry index +
Note that the index value in the MFT entry is only 32-bit of size.
| 8 | 8 | | Sequence number
|===

==== [[update_reason_flags]]Update reason flags

[cols="1,1,5",options="header"]
|===
| Value | Identifier | Description
| 0x00000001 | USN_REASON_DATA_OVERWRITE | The data in the file or directory is overwritten. +
[yellow-background]*The default (unnamed) $DATA attribute was overwritten*
| 0x00000002 | USN_REASON_DATA_EXTEND | The file or directory is extended +
[yellow-background]*The default (unnamed) $DATA attribute was extended*
| 0x00000004 | USN_REASON_DATA_TRUNCATION | The file or directory is truncated. +
[yellow-background]*The default (unnamed) $DATA attribute was truncated*
| | |
| 0x00000010 | USN_REASON_NAMED_DATA_OVERWRITE | One or more named data streams ($DATA attributes) of file were overwritten
| 0x00000020 | USN_REASON_NAMED_DATA_EXTEND | One or more named data streams ($DATA attributes) of file were extended
| 0x00000040 | USN_REASON_NAMED_DATA_TRUNCATION | One or more named data streams ($DATA attributes) of a file were truncated
| | |
| 0x00000100 | USN_REASON_FILE_CREATE | The file or directory was created
| 0x00000200 | USN_REASON_FILE_DELETE | The file or directory was deleted
| 0x00000400 | USN_REASON_EA_CHANGE | The extended attributes of the file were changed
| 0x00000800 | USN_REASON_SECURITY_CHANGE | The access rights (security descriptor) of a file or directory were changed
| 0x00001000 | USN_REASON_RENAME_OLD_NAME | The name changed +
The USN change journal entry contains the old name
| 0x00002000 | USN_REASON_RENAME_NEW_NAME | The name changed +
The USN change journal entry contains the new name
| 0x00004000 | USN_REASON_INDEXABLE_CHANGE | Content indexed status changed +
the file attribute: FILE_ATTRIBUTE_NOT_CONTENT_INDEXED was changed
| 0x00008000 | USN_REASON_BASIC_INFO_CHANGE | Basic file or directory attributes changed +
One or more file or directory attributes were changed e.g. read-only, hidden, system, archive, or sparse attribute, or one or more time stamps.
| 0x00010000 | USN_REASON_HARD_LINK_CHANGE | A hard link was created or deleted
| 0x00020000 | USN_REASON_COMPRESSION_CHANGE | The file or directory was compressed or decompressed
| 0x00040000 | USN_REASON_ENCRYPTION_CHANGE | The file or directory was encrypted or decrypted
| 0x00080000 | USN_REASON_OBJECT_ID_CHANGE | The object identifier of a file or directory was changed
| 0x00100000 | USN_REASON_REPARSE_POINT_CHANGE | The reparse point that in a file or directory was changed, or a reparse point was added to or deleted from a file or directory.
| 0x00200000 | USN_REASON_STREAM_CHANGE | A named data stream ($DATA attribute) is added to or removed from a file, or a named stream is renamed
| 0x00400000 | USN_REASON_TRANSACTED_CHANGE | [yellow-background]*Unknown found in TxF USN change journal entry list*
| | |
| 0x80000000 | USN_REASON_CLOSE | The file or directory was closed
|===

==== [[update_source_flags]]Update source flags

[cols="1,1,5",options="header"]
|===
| Value | Identifier | Description
| 0x00000001 | USN_SOURCE_DATA_MANAGEMENT | The operation added a private data stream to a file or directory. The modifications did not change the application data.
| 0x00000002 | USN_SOURCE_AUXILIARY_DATA | The operation was caused by the operating system. Although a write operation is performed on the item, the data was not changed.
| 0x00000004 | USN_SOURCE_REPLICATION_MANAGEMENT | The operation was caused by file replication
|===

== Alternate data streams

[cols="1,5",options="header"]
|===
| Data stream name | Description
| ♣BnhqlkugBim0elg1M1pt2tjdZe +
♣SummaryInformation +
{4c8cc155-6c1e-11d1-8e41-00c04fb9386d} | Used to store property sets +
Where ♣ is Unicode character U+2663 also known as black club
| {59828bbb-3f72-4c1b-a420-b51ad66eb5d3}.XPRESS | Used during remote differential compression
| AFP_AfpInfo +
AFP_Resource | Used to store Macintosh operating system property lists
| encryptable | Used to store attributes relating to thumbnails in the thumbnails database
| favicon | Used to store favorite icons for web pages.
| ms-properties | [yellow-background]*Unknown (Used to store property sets)*
| OECustomProperty | Used to store custom properties related to email files
| Zone.Identifier | Used to store URL security zones
|===

=== ms-properties

....
00000000  e9 00 00 00 8c 00 00 00  31 53 50 53 53 f1 ef fc  |........1SPSS...|
00000010  39 e8 f3 4c a9 e7 ea 22  83 20 94 b8 25 00 00 00  |9..L...". ..%...|
00000020  6f 00 00 00 00 1f 10 00  00 01 00 00 00 08 00 00  |o...............|
00000030  00 50 00 72 00 69 00 76  00 61 00 74 00 65 00 00  |.P.r.i.v.a.t.e..|
00000040  00 25 00 00 00 6c 00 00  00 00 1f 00 00 00 09 00  |.%...l..........|
00000050  00 00 53 00 6b 00 79 00  44 00 72 00 69 00 76 00  |..S.k.y.D.r.i.v.|
00000060  65 00 00 00 00 00 11 00  00 00 6d 00 00 00 00 13  |e.........m.....|
00000070  00 00 00 00 0d 04 00 15  00 00 00 6e 00 00 00 00  |...........n....|
00000080  15 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000090  59 00 00 00 31 53 50 53  d6 b9 f9 b2 c4 fe d5 4d  |Y...1SPS.......M|
000000a0  94 d7 89 57 48 8c 80 7b  3d 00 00 00 03 00 00 00  |...WH..{=.......|
000000b0  00 1f 00 00 00 15 00 00  00 37 00 46 00 30 00 39  |.........7.F.0.9|
000000c0  00 35 00 31 00 34 00 39  00 30 00 32 00 37 00 38  |.5.1.4.9.0.2.7.8|
000000d0  00 34 00 38 00 45 00 44  00 21 00 31 00 30 00 33  |.4.8.E.D.!.1.0.3|
000000e0  00 00 00 00 00 00 00 00  00 00 00 00 00           |.............|
000000ed
....

=== Zone.Identifier

....
00000000  5b 5a 6f 6e 65 54 72 61  6e 73 66 65 72 5d 0d 0a  |[ZoneTransfer]..|
00000010  5a 6f 6e 65 49 64 3d 33  0d 0a                    |ZoneId=3..|
0000001a
....

== [[transactional_ntfs]]Transactional NTFS (TxF)

As of Vista ([yellow-background]*or windows server 2003?*) Transactional NTFS
(TxF) was added.

In TxF the resource manager (RM) keeps track of transactional metadata and log
files. The TxF related metadata files are stored in the metadata directory:
....
$Extend\$RmMetadata
....

=== Resource manager repair information

The resource manager repair information metadata file:
$Extend\$RmMetadata\$Repair consists of the following data streams:

* the default (unnamed) data stream, [yellow-background]*purpose unknown*
* the $Config data stream, contains the resource manager repair configuration information

==== Resource manager repair configuration information

The $Repair:$Config data streams contains:

[yellow-background]*TODO*

....
00000000  01 00 00 00 01 00 00 00                           |........|
....

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | 4 | | [yellow-background]*Unknown*
| 4 | 4 | | [yellow-background]*Unknown*
|===

=== Transactional NTFS (TxF) metadata directory

The transactional NTFS (TxF) metadata directory: $Extend\$RmMetadata\$Txf is used to isolate files for delete or overwrite operations.

File format? All files seem to start with similar information

=== TxF Old Page Stream (TOPS) file

The TxF Old Page Stream (TOPS) file: $Extend\$RmMetadata\$TxfLog\$Tops consists of the following data streams:

* the default (unnamed) data stream, contains metadata about the resource manager, such as its GUID, its CLFS log policy, and the LSN at which recovery should start
* the $T data stream, contains the file data that is partially overwritten by a transaction as opposed to a full overwrite, which would move the file into the Transactional NTFS (TxF) metadata directory

==== TxF Old Page Stream (TOPS) metadata

The $Tops default (unnamed) data streams contains:

[yellow-background]*TODO*

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | 2 | 0x000a | [yellow-background]*Unknown*
| 2 | 2 | 0x0064 | [yellow-background]*Size of TOPS metadata*
| 4 | 4 | 0x0001 | [yellow-background]*Unknown* +
[yellow-background]*Number of resource managers/streams?*
| 8 | 16 | | Resource Manager (RM) identifier +
Contains a GUID
| 24 | 8 | | [yellow-background]*Unknown (empty)*
| 32 | 8 | | [yellow-background]*Base (or log start)* LSN of TxFLog stream
| 40 | 8 | | [yellow-background]*Unknown*
| 48 | 8 | | [yellow-background]*Last* flushed LSN of TxFLog stream
| 56 | 8 | | [yellow-background]*Unknown*
| 64 | 8 | | [yellow-background]*Unknown (empty)*
| 72 | 8 | | [yellow-background]*Restart LSN?*
| 80 | 20 | | [yellow-background]*Unknown*
|===

==== TxF Old Page Stream (TOPS) file data

The $Tops:$T data streams contains the file data that is partially overwritten
by a transaction. It consists of multiple pending transaction XML-documents.

[yellow-background]*Note that the start of each sector contains 0x0001, is this
a value indication the sector is empty? Or are there fix-up values stored
somewhere else?*

A pending transaction XML-document starts with an UTF-8 byte-order-mark. Is
roughly contains the following data:
....
<?xml version='1.0' encoding='utf-8'?>
<PendingTransaction Version="2.0" Identifier="...">
   <Transactions>
      <Transaction TransactionId="...">
      <Install Application="..., Culture=..., Version=..., PublicKeyToken=...,
                           ProcessorArchitecture=..., versionScope=..."
               RefGuid="..."
               RefIdentifier="..."
               RefExtra="..."/>
      ...
      </Transaction>
   </Transactions>
   <ChangeList>
      <Change Family="..., Culture=..., PublicKeyToken=...,
                     ProcessorArchitecture=..., versionScope=..."
              New="..."/>
      ...
   </ChangeList>
   <POQ>
      <BeginTransaction id="..."/>

      <CreateFile path="..."
                  fileAttribute="..."/>
      <DeleteFile path="..."/>
      <MoveFile source="..." destination="..."/>
      <HardlinkFile source="..." destination="..."/>
      <SetFileInformation path="..."
                          securityDescriptor="binary base64:..."
                          flags="..."/>

       <CreateKey path="..."/>
       <SetKeyValue path="..."
                    name="..."
                    type="..."
                    encoding="base64"
                    value="..."/>
      <DeleteKeyValue path="..."
                      name="..."/>

      ...
   </POQ>
   <InstallerQueue Length="...">
      <Action Installer="..."
              Mode="..."
              Phase="..."
              Family="..., Culture=..., PublicKeyToken=...,
                     ProcessorArchitecture=..., versionScope=..."
              Old="..."
              New="..."/>

      ...
   </InstallerQueue >
</PendingTransaction>
....

=== Transactional NTFS (TxF) Common Log File System (CLFS) files

TxF uses a Common Log File System (CLFS) log store and the logged utility
stream attribute named $TXF_DATA.

See `[RUSSNOVICH09]`, `[MSDN]` and `[LIBFSCLFS]` for more information about
CLFS.

The base log file (BLF) of the TxF log store is:
....
$Extend\$RmMetadata\$TxfLog\TxfLog.blf
....

Commonly the corresponding container files are:
....
$Extend\$RmMetadata\$TxfLog\TxfLogContainer00000000000000000001
$Extend\$RmMetadata\$TxfLog\TxfLogContainer00000000000000000002
....

TxF uses a multiplexed log store which contains two streams:

* the KtmLog stream used for Kernel Transaction Manager (KTM) metadata records
* TxfLog stream, which contains the TxF log records.

=== Transactional data logged utility stream attribute

The transactional data ($TXF_DATA) logged utility stream attribute is 56 bytes
of size and consist of:

[cols="1,1,1,5",options="header"]
|===
| Offset | Size | Value | Description
| 0 | 6 | | [yellow-background]*Unknown (remnant data)*
| 6 | 8 | | Resource manager root file reference +
Contains an NTFS file reference that refers to the MFT
| 14 | 8 | | [yellow-background]*USN index?*
| 22 | 8 | | File identifier (TxID) +
Contains a TxF file identifier
| 30 | 8 | | Data LSN +
Contains a CLFS LSN of file data transaction records
| 38 | 8 | | Metadata LSN +
Contains a CLFS LSN of file system metadata transaction records
| 46 | 8 | | Directory index LSN +
Contains a CLFS LSN of directory index transaction records
| 54 | 2 | | [yellow-background]*Flags?* +
[yellow-background]*Seen: 0x0000, 0x0002*
|===

[yellow-background]*Note there can be more than 1 per MTF entry*

== Windows definitions

=== [[file_attribute_flags]]File attribute flags

The file attribute flags consist of the following values:

[cols="1,1,5",options="header"]
|===
| Value | Identifier | Description
| 0x00000001 | FILE_ATTRIBUTE_READONLY | Is read-only
| 0x00000002 | FILE_ATTRIBUTE_HIDDEN | Is hidden
| 0x00000004 | FILE_ATTRIBUTE_SYSTEM | Is a system file or directory
| 0x00000008 | | Is a volume label +
[yellow-background]*Not used by NTFS*
| 0x00000010 | FILE_ATTRIBUTE_DIRECTORY | Is a directory +
[yellow-background]*Not used by NTFS*
| 0x00000020 | FILE_ATTRIBUTE_ARCHIVE | Should be archived
| 0x00000040 | FILE_ATTRIBUTE_DEVICE | Is a device +
[yellow-background]*Not used by NTFS*
| 0x00000080 | FILE_ATTRIBUTE_NORMAL | Is normal +
None of the other flags should be set
| 0x00000100 | FILE_ATTRIBUTE_TEMPORARY | Is temporary
| 0x00000200 | FILE_ATTRIBUTE_SPARSE_FILE | Is a sparse file
| 0x00000400 | FILE_ATTRIBUTE_REPARSE_POINT | Is a reparse point or symbolic link
| 0x00000800 | FILE_ATTRIBUTE_COMPRESSED | Is compressed
| 0x00001000 | FILE_ATTRIBUTE_OFFLINE | Is offline +
The data of the file is stored on an offline storage.
| 0x00002000 | FILE_ATTRIBUTE_NOT_CONTENT_INDEXED | Do not index content +
The content of the file or directory should not be indexed by the indexing service.
| 0x00004000 | FILE_ATTRIBUTE_ENCRYPTED | Is encrypted
| 0x00008000 | | [yellow-background]*Unknown (seen on Windows 95 FAT)*
| 0x00010000 | FILE_ATTRIBUTE_VIRTUAL | Is virtual
3+| _The following flags are mainly used in the file name attribute and sparsely in the standard information attribute;_ +
_it could be that they have a different meaning in both types of attributes or that the standard information flags are not updated._ +
_For now the latter is assumed._
| 0x10000000 | | [yellow-background]*Unknown* +
[yellow-background]*Is directory (or has $I30 index ?) used instead of 0x00000010 ? Seen $Extend directory without this flag*
| 0x20000000 | | [yellow-background]*Is index view* +
[yellow-background]*(copy from corresponding bit in MFT record)*
|===

== Notes

Signatures as indicated in `[LINUXNTFS]`:
....
        "HOLE" == ??? (NTFS 3.0+?)
....

=== NTFS reserved file names

[cols="1,1,5",options="header"]
|===
| MFT entry index | Filename | Description
| | \$Extend\$Config | Used for NTFS repair activity
| | \$Extend\$Delete | Delete file name
| | \$Extend\$Repair.log | Repair log name
| | \$Extend\$Tops |
| | \$Extend\$TxfLog | Transactional NTFS log
|===

=== File system flags

....
fsutil fsinfo volumeinfo C:
....

....
FILE_CASE_PRESERVED_NAMES
0x00000002

The specified volume supports preserved case of file names when it places a name on disk.

FILE_CASE_SENSITIVE_SEARCH
0x00000001

The specified volume supports case-sensitive file names.

FILE_FILE_COMPRESSION
0x00000010

The specified volume supports file-based compression.

FILE_NAMED_STREAMS
0x00040000

The specified volume supports named streams.

FILE_PERSISTENT_ACLS
0x00000008

The specified volume preserves and enforces access control lists (ACL). For example, the NTFS file system preserves and enforces ACLs, and the FAT file system does not.

FILE_READ_ONLY_VOLUME
0x00080000

The specified volume is read-only.

    Windows 2000:  This value is not supported.

FILE_SEQUENTIAL_WRITE_ONCE
0x00100000

The specified volume supports a single sequential write.

    Windows 2000:  This value is not supported.

FILE_SUPPORTS_ENCRYPTION
0x00020000

The specified volume supports the Encrypted File System (EFS). For more information, see File Encryption.

FILE_SUPPORTS_EXTENDED_ATTRIBUTES
0x00800000

The specified volume supports extended attributes. An extended attribute is a piece of application-specific metadata that an application can associate with a file and is not part of the file's data.

    Windows Server 2008, Windows Vista, Windows Server 2003, and Windows XP/2000:  This value is not supported until Windows Server 2008 R2 and Windows 7.

FILE_SUPPORTS_HARD_LINKS
0x00400000

The specified volume supports hard links. For more information, see Hard Links and Junctions.

    Windows Server 2008, Windows Vista, Windows Server 2003, and Windows XP/2000:  This value is not supported until Windows Server 2008 R2 and Windows 7.

FILE_SUPPORTS_OBJECT_IDS
0x00010000

The specified volume supports object identifiers.

FILE_SUPPORTS_OPEN_BY_FILE_ID
0x01000000

The file system supports open by FileID. For more information, see FILE_ID_BOTH_DIR_INFO.

    Windows Server 2008, Windows Vista, Windows Server 2003, and Windows XP/2000:  This value is not supported until Windows Server 2008 R2 and Windows 7.

FILE_SUPPORTS_REPARSE_POINTS
0x00000080

The specified volume supports re-parse points.

FILE_SUPPORTS_SPARSE_FILES
0x00000040

The specified volume supports sparse files.

FILE_SUPPORTS_TRANSACTIONS
0x00200000

The specified volume supports transactions. For more information, see About KTM.

    Windows 2000:  This value is not supported.

FILE_SUPPORTS_USN_JOURNAL
0x02000000

The specified volume supports update sequence number (USN) journals. For more information, see Change Journal Records.

    Windows Server 2008, Windows Vista, Windows Server 2003, and Windows XP/2000:  This value is not supported until Windows Server 2008 R2 and Windows 7.

FILE_UNICODE_ON_DISK
0x00000004

The specified volume supports Unicode in file names as they appear on disk.

FILE_VOLUME_IS_COMPRESSED
0x00008000

The specified volume is a compressed volume, for example, a DoubleSpace volume.

FILE_VOLUME_QUOTAS
0x00000020

The specified volume supports disk quotas.
....

=== $MFT metadata file only parsing

What file system metadata is missing:

* Attribute lists are stored outside the $MFT
* $I30 indexes are stored outside the $MFT
* How to map MFT entry to parent? what if parent MFT entry has changed?

:numbered!:
[appendix]
== References

`[SANDERSON02]`

[cols="1,5",options="header"]
|===
| Title | NTFS Compression – a forensic view
| Author(s) | Paul Sanderson
| Date | October 2002
| URL | http://www.sandersonforensics.co.uk/Files/NTFS%20compression%20white%20paper.pdf
|===

`[CARRIER05]`

[cols="1,5",options="header"]
|===
| Title | File System Forensic Analysis
| Author(s) | Brian Carrier
| Date | 2005
| ISBN-10 | 0-321-26817-2
|===

`[RUSSON05]`

[cols="1,5",options="header"]
|===
| Title | NTFS Documentation
| Author(s) | Richard Russon, Yuval Fiedel
| Date | 2005
| URL | http://linux-ntfs.org/
|===

`[POLLARD06]`

[cols="1,5",options="header"]
|===
| Title | All about BIOS parameter blocks
| Author(s) | Jonathan de Boyne Pollard
| URL | http://homepage.ntlworld.com./jonathan.deboynepollard/FGA/bios-parameter-block.html
|===

`[MEDEIROS08]`

[cols="1,5",options="header"]
|===
| Title | NTFS Forensics - A Programmers View of Raw Filesystem Data Extraction
| Author(s) | Jason Medeiros
| URL | http://grayscale-research.org/new/pdfs/NTFS%20forensics.pdf
|===

`[RUSSNOVICH09]`

[cols="1,5",options="header"]
|===
| Title | Windows Internals 5 - Covering Windows Server 2008 and Windows Vista
| Author(s) | Mark E. Russinovich and David A. Solomon
| Date | June 17, 2009
| ISBN-13 | 978-0735625303
|===

`[LIBBDE]`

[cols="1,5",options="header"]
|===
| Title | BitLocker Drive Encryption (BDE) format specification - Analysis of theBitLocker Drive Encryption (BDE) volume format
| Date | March 2011
| Author(s) | Joachim Metz
| URL | https://github.com/libyal/libbde/blob/master/documentation/BitLocker%20Drive%20Encryption%20(BDE)%20format.asciidoc
|===

`[LIBFSCLFS]`

[cols="1,5",options="header"]
|===
| Title | Common Log File System – Analysis of the Windows ARIES log system
| Date | November 2010
| Author(s) | Joachim Metz
| URL | https://googledrive.com/host/0B3fBvzttpiiSM1QwSmdTYjB0b0E/Common%20Log%20File%20System%20(CLFS).pdf
|===

`[LIBVSHADOW]`

[cols="1,5",options="header"]
|===
| Title | Volume Shadow Snapshot (VSS) - Analysis the Windows NT VSS format
| Date | March 2011
| Author(s) | Joachim Metz
| URL | https://github.com/libyal/libvshadow/blob/master/documentation/Volume%20Shadow%20Snapshot%20(VSS)%20format.asciidoc
|===

`[LINUXNTFS]`

[cols="1,5",options="header"]
|===
| Title | Linux-NTFS Project
| URL | http://inform.pucp.edu.pe/~inf232/Ntfs/ntfs_doc_v0.5/index.html
|===

`[MSSUPPORT]`

[cols="1,5",options="header"]
|===
| Title | Microsoft Support - How Windows Generates 8.3 File Names from Long File Names
| URL | http://support.microsoft.com/kb/142982/en-us
|===

`[MSDN]`

[cols="1,5",options="header"]
|===
| Title | Microsoft Developer Network
| URL | http://technet.microsoft.com/en-us/library/cc781134%28WS.10%29.aspx
|===

[cols="1,5",options="header"]
|===
| Subject | Master File Table
| URL | http://msdn.microsoft.com/en-us/library/bb470206(v=vs.85).aspx
|===

[cols="1,5",options="header"]
|===
| Subject | Reserved file names
| URL | http://msdn.microsoft.com/en-us/library/ff469234%28v=PROT.10%29.aspx
|===

[cols="1,5",options="header"]
|===
| Subject | NTFS attribute types
| URL | http://msdn.microsoft.com/en-us/library/ff469236%28PROT.10%29.aspx
|===

[cols="1,5",options="header"]
|===
| Subject | Reparse point
| URL | http://msdn.microsoft.com/en-us/library/aa365740%28VS.85%29.aspx +
http://msdn.microsoft.com/en-us/library/aa365511%28v=VS.85%29.aspx +
http://msdn.microsoft.com/en-us/library/dd541667%28PROT.13%29.aspx +
http://msdn.microsoft.com/en-us/library/cc232005%28v=PROT.13%29.aspx +
http://msdn.microsoft.com/en-us/library/cc232006%28v=PROT.13%29.aspx +
http://msdn.microsoft.com/en-us/library/cc232007%28v=PROT.13%29.aspx
|===

[cols="1,5",options="header"]
|===
| Subject | Understanding NTFS Compression
| URL | http://blogs.msdn.com/b/ntdebugging/archive/2008/05/20/understanding-ntfs-compression.aspx
|===

[cols="1,5",options="header"]
|===
| Subject | Update (or change) journal
| URL | http://msdn.microsoft.com/en-us/library/aa363798.aspx +
http://msdn.microsoft.com/en-us/library/aa363803%28VS.85%29.aspx +
http://msdn.microsoft.com/en-us/library/aa365722%28VS.85%29.aspx
|===

[cols="1,5",options="header"]
|===
| Subject | Known Alternate Stream Names
| URL | https://msdn.microsoft.com/en-us/library/dn365326.aspx
|===

[cols="1,5",options="header"]
|===
| Subject | transactional NTFS
| URL | http://msdn.microsoft.com/en-us/library/bb968806%28v=VS.85%29.aspx +
http://msdn.microsoft.com/en-us/library/bb986748%28VS.85%29.aspx +
http://msdn.microsoft.com/en-us/library/bb540368%28VS.85%29.aspx
|===

`[WIKI]`

[cols="1,5",options="header"]
|===
| URL | http://en.wikipedia.org/wiki/NTFS +
http://en.wikipedia.org/wiki/BIOS_parameter_block +
http://en.wikipedia.org/wiki/Transactional_NTFS
|===

`[WIN32PROG]`

[cols="1,5",options="header"]
|===
| Title | The Win32 Programming Tutorials For Fun – Appendix E. NTFS On-Disk Structure
| URL | http://www.installsetupconfig.com/win32programming/1996%20AppE_apnilife.pdf
|===

[appendix]
== GNU Free Documentation License

Version 1.3, 3 November 2008
Copyright © 2000, 2001, 2002, 2007, 2008 Free Software Foundation, Inc.
<http://fsf.org/>

Everyone is permitted to copy and distribute verbatim copies of this license
document, but changing it is not allowed.

=== 0. PREAMBLE

The purpose of this License is to make a manual, textbook, or other functional
and useful document "free" in the sense of freedom: to assure everyone the
effective freedom to copy and redistribute it, with or without modifying it,
either commercially or noncommercially. Secondarily, this License preserves for
the author and publisher a way to get credit for their work, while not being
considered responsible for modifications made by others.

This License is a kind of "copyleft", which means that derivative works of the
document must themselves be free in the same sense. It complements the GNU
General Public License, which is a copyleft license designed for free software.

We have designed this License in order to use it for manuals for free software,
because free software needs free documentation: a free program should come with
manuals providing the same freedoms that the software does. But this License is
not limited to software manuals; it can be used for any textual work,
regardless of subject matter or whether it is published as a printed book. We
recommend this License principally for works whose purpose is instruction or
reference.

=== 1. APPLICABILITY AND DEFINITIONS

This License applies to any manual or other work, in any medium, that contains
a notice placed by the copyright holder saying it can be distributed under the
terms of this License. Such a notice grants a world-wide, royalty-free license,
unlimited in duration, to use that work under the conditions stated herein. The
"Document", below, refers to any such manual or work. Any member of the public
is a licensee, and is addressed as "you". You accept the license if you copy,
modify or distribute the work in a way requiring permission under copyright law.

A "Modified Version" of the Document means any work containing the Document or
a portion of it, either copied verbatim, or with modifications and/or
translated into another language.

A "Secondary Section" is a named appendix or a front-matter section of the
Document that deals exclusively with the relationship of the publishers or
authors of the Document to the Document's overall subject (or to related
matters) and contains nothing that could fall directly within that overall
subject. (Thus, if the Document is in part a textbook of mathematics, a
Secondary Section may not explain any mathematics.) The relationship could be a
matter of historical connection with the subject or with related matters, or of
legal, commercial, philosophical, ethical or political position regarding them.

The "Invariant Sections" are certain Secondary Sections whose titles are
designated, as being those of Invariant Sections, in the notice that says that
the Document is released under this License. If a section does not fit the
above definition of Secondary then it is not allowed to be designated as
Invariant. The Document may contain zero Invariant Sections. If the Document
does not identify any Invariant Sections then there are none.

The "Cover Texts" are certain short passages of text that are listed, as
Front-Cover Texts or Back-Cover Texts, in the notice that says that the
Document is released under this License. A Front-Cover Text may be at most 5
words, and a Back-Cover Text may be at most 25 words.

A "Transparent" copy of the Document means a machine-readable copy, represented
in a format whose specification is available to the general public, that is
suitable for revising the document straightforwardly with generic text editors
or (for images composed of pixels) generic paint programs or (for drawings)
some widely available drawing editor, and that is suitable for input to text
formatters or for automatic translation to a variety of formats suitable for
input to text formatters. A copy made in an otherwise Transparent file format
whose markup, or absence of markup, has been arranged to thwart or discourage
subsequent modification by readers is not Transparent. An image format is not
Transparent if used for any substantial amount of text. A copy that is not
"Transparent" is called "Opaque".

Examples of suitable formats for Transparent copies include plain ASCII without
markup, Texinfo input format, LaTeX input format, SGML or XML using a publicly
available DTD, and standard-conforming simple HTML, PostScript or PDF designed
for human modification. Examples of transparent image formats include PNG, XCF
and JPG. Opaque formats include proprietary formats that can be read and edited
only by proprietary word processors, SGML or XML for which the DTD and/or
processing tools are not generally available, and the machine-generated HTML,
PostScript or PDF produced by some word processors for output purposes only.

The "Title Page" means, for a printed book, the title page itself, plus such
following pages as are needed to hold, legibly, the material this License
requires to appear in the title page. For works in formats which do not have
any title page as such, "Title Page" means the text near the most prominent
appearance of the work's title, preceding the beginning of the body of the text.

The "publisher" means any person or entity that distributes copies of the
Document to the public.

A section "Entitled XYZ" means a named subunit of the Document whose title
either is precisely XYZ or contains XYZ in parentheses following text that
translates XYZ in another language. (Here XYZ stands for a specific section
name mentioned below, such as "Acknowledgements", "Dedications",
"Endorsements", or "History".) To "Preserve the Title" of such a section when
you modify the Document means that it remains a section "Entitled XYZ"
according to this definition.

The Document may include Warranty Disclaimers next to the notice which states
that this License applies to the Document. These Warranty Disclaimers are
considered to be included by reference in this License, but only as regards
disclaiming warranties: any other implication that these Warranty Disclaimers
may have is void and has no effect on the meaning of this License.

=== 2. VERBATIM COPYING

You may copy and distribute the Document in any medium, either commercially or
noncommercially, provided that this License, the copyright notices, and the
license notice saying this License applies to the Document are reproduced in
all copies, and that you add no other conditions whatsoever to those of this
License. You may not use technical measures to obstruct or control the reading
or further copying of the copies you make or distribute. However, you may
accept compensation in exchange for copies. If you distribute a large enough
number of copies you must also follow the conditions in section 3.

You may also lend copies, under the same conditions stated above, and you may
publicly display copies.

=== 3. COPYING IN QUANTITY

If you publish printed copies (or copies in media that commonly have printed
covers) of the Document, numbering more than 100, and the Document's license
notice requires Cover Texts, you must enclose the copies in covers that carry,
clearly and legibly, all these Cover Texts: Front-Cover Texts on the front
cover, and Back-Cover Texts on the back cover. Both covers must also clearly
and legibly identify you as the publisher of these copies. The front cover must
present the full title with all words of the title equally prominent and
visible. You may add other material on the covers in addition. Copying with
changes limited to the covers, as long as they preserve the title of the
Document and satisfy these conditions, can be treated as verbatim copying in
other respects.

If the required texts for either cover are too voluminous to fit legibly, you
should put the first ones listed (as many as fit reasonably) on the actual
cover, and continue the rest onto adjacent pages.

If you publish or distribute Opaque copies of the Document numbering more than
100, you must either include a machine-readable Transparent copy along with
each Opaque copy, or state in or with each Opaque copy a computer-network
location from which the general network-using public has access to download
using public-standard network protocols a complete Transparent copy of the
Document, free of added material. If you use the latter option, you must take
reasonably prudent steps, when you begin distribution of Opaque copies in
quantity, to ensure that this Transparent copy will remain thus accessible at
the stated location until at least one year after the last time you distribute
an Opaque copy (directly or through your agents or retailers) of that edition
to the public.

It is requested, but not required, that you contact the authors of the Document
well before redistributing any large number of copies, to give them a chance to
provide you with an updated version of the Document.

=== 4. MODIFICATIONS

You may copy and distribute a Modified Version of the Document under the
conditions of sections 2 and 3 above, provided that you release the Modified
Version under precisely this License, with the Modified Version filling the
role of the Document, thus licensing distribution and modification of the
Modified Version to whoever possesses a copy of it. In addition, you must do
these things in the Modified Version:

A. Use in the Title Page (and on the covers, if any) a title distinct from that
of the Document, and from those of previous versions (which should, if there
were any, be listed in the History section of the Document). You may use the
same title as a previous version if the original publisher of that version
gives permission.

B. List on the Title Page, as authors, one or more persons or entities
responsible for authorship of the modifications in the Modified Version,
together with at least five of the principal authors of the Document (all of
its principal authors, if it has fewer than five), unless they release you from
this requirement.

C. State on the Title page the name of the publisher of the Modified Version,
as the publisher.

D. Preserve all the copyright notices of the Document.

E. Add an appropriate copyright notice for your modifications adjacent to the
other copyright notices.

F. Include, immediately after the copyright notices, a license notice giving
the public permission to use the Modified Version under the terms of this
License, in the form shown in the Addendum below.

G. Preserve in that license notice the full lists of Invariant Sections and
required Cover Texts given in the Document's license notice.

H. Include an unaltered copy of this License.

I. Preserve the section Entitled "History", Preserve its Title, and add to it
an item stating at least the title, year, new authors, and publisher of the
Modified Version as given on the Title Page. If there is no section Entitled
"History" in the Document, create one stating the title, year, authors, and
publisher of the Document as given on its Title Page, then add an item
describing the Modified Version as stated in the previous sentence.

J. Preserve the network location, if any, given in the Document for public
access to a Transparent copy of the Document, and likewise the network
locations given in the Document for previous versions it was based on. These
may be placed in the "History" section. You may omit a network location for a
work that was published at least four years before the Document itself, or if
the original publisher of the version it refers to gives permission.

K. For any section Entitled "Acknowledgements" or "Dedications", Preserve the
Title of the section, and preserve in the section all the substance and tone of
each of the contributor acknowledgements and/or dedications given therein.

L. Preserve all the Invariant Sections of the Document, unaltered in their text
and in their titles. Section numbers or the equivalent are not considered part
of the section titles.

M. Delete any section Entitled "Endorsements". Such a section may not be
included in the Modified Version.

N. Do not retitle any existing section to be Entitled "Endorsements" or to
conflict in title with any Invariant Section.

O. Preserve any Warranty Disclaimers.

If the Modified Version includes new front-matter sections or appendices that
qualify as Secondary Sections and contain no material copied from the Document,
you may at your option designate some or all of these sections as invariant. To
do this, add their titles to the list of Invariant Sections in the Modified
Version's license notice. These titles must be distinct from any other section
titles.

You may add a section Entitled "Endorsements", provided it contains nothing but
endorsements of your Modified Version by various parties—for example,
statements of peer review or that the text has been approved by an organization
as the authoritative definition of a standard.

You may add a passage of up to five words as a Front-Cover Text, and a passage
of up to 25 words as a Back-Cover Text, to the end of the list of Cover Texts
in the Modified Version. Only one passage of Front-Cover Text and one of
Back-Cover Text may be added by (or through arrangements made by) any one
entity. If the Document already includes a cover text for the same cover,
previously added by you or by arrangement made by the same entity you are
acting on behalf of, you may not add another; but you may replace the old one,
on explicit permission from the previous publisher that added the old one.

The author(s) and publisher(s) of the Document do not by this License give
permission to use their names for publicity for or to assert or imply
endorsement of any Modified Version.

=== 5. COMBINING DOCUMENTS

You may combine the Document with other documents released under this License,
under the terms defined in section 4 above for modified versions, provided that
you include in the combination all of the Invariant Sections of all of the
original documents, unmodified, and list them all as Invariant Sections of your
combined work in its license notice, and that you preserve all their Warranty
Disclaimers.

The combined work need only contain one copy of this License, and multiple
identical Invariant Sections may be replaced with a single copy. If there are
multiple Invariant Sections with the same name but different contents, make the
title of each such section unique by adding at the end of it, in parentheses,
the name of the original author or publisher of that section if known, or else
a unique number. Make the same adjustment to the section titles in the list of
Invariant Sections in the license notice of the combined work.

In the combination, you must combine any sections Entitled "History" in the
various original documents, forming one section Entitled "History"; likewise
combine any sections Entitled "Acknowledgements", and any sections Entitled
"Dedications". You must delete all sections Entitled "Endorsements".

=== 6. COLLECTIONS OF DOCUMENTS

You may make a collection consisting of the Document and other documents
released under this License, and replace the individual copies of this License
in the various documents with a single copy that is included in the collection,
provided that you follow the rules of this License for verbatim copying of each
of the documents in all other respects.

You may extract a single document from such a collection, and distribute it
individually under this License, provided you insert a copy of this License
into the extracted document, and follow this License in all other respects
regarding verbatim copying of that document.

=== 7. AGGREGATION WITH INDEPENDENT WORKS

A compilation of the Document or its derivatives with other separate and
independent documents or works, in or on a volume of a storage or distribution
medium, is called an "aggregate" if the copyright resulting from the
compilation is not used to limit the legal rights of the compilation's users
beyond what the individual works permit. When the Document is included in an
aggregate, this License does not apply to the other works in the aggregate
which are not themselves derivative works of the Document.

If the Cover Text requirement of section 3 is applicable to these copies of the
Document, then if the Document is less than one half of the entire aggregate,
the Document's Cover Texts may be placed on covers that bracket the Document
within the aggregate, or the electronic equivalent of covers if the Document is
in electronic form. Otherwise they must appear on printed covers that bracket
the whole aggregate.

=== 8. TRANSLATION

Translation is considered a kind of modification, so you may distribute
translations of the Document under the terms of section 4. Replacing Invariant
Sections with translations requires special permission from their copyright
holders, but you may include translations of some or all Invariant Sections in
addition to the original versions of these Invariant Sections. You may include
a translation of this License, and all the license notices in the Document, and
any Warranty Disclaimers, provided that you also include the original English
version of this License and the original versions of those notices and
disclaimers. In case of a disagreement between the translation and the original
version of this License or a notice or disclaimer, the original version will
prevail.

If a section in the Document is Entitled "Acknowledgements", "Dedications", or
"History", the requirement (section 4) to Preserve its Title (section 1) will
typically require changing the actual title.

=== 9. TERMINATION

You may not copy, modify, sublicense, or distribute the Document except as
expressly provided under this License. Any attempt otherwise to copy, modify,
sublicense, or distribute it is void, and will automatically terminate your
rights under this License.

However, if you cease all violation of this License, then your license from a
particular copyright holder is reinstated (a) provisionally, unless and until
the copyright holder explicitly and finally terminates your license, and (b)
permanently, if the copyright holder fails to notify you of the violation by
some reasonable means prior to 60 days after the cessation.

Moreover, your license from a particular copyright holder is reinstated
permanently if the copyright holder notifies you of the violation by some
reasonable means, this is the first time you have received notice of violation
of this License (for any work) from that copyright holder, and you cure the
violation prior to 30 days after your receipt of the notice.

Termination of your rights under this section does not terminate the licenses
of parties who have received copies or rights from you under this License. If
your rights have been terminated and not permanently reinstated, receipt of a
copy of some or all of the same material does not give you any rights to use it.

=== 10. FUTURE REVISIONS OF THIS LICENSE

The Free Software Foundation may publish new, revised versions of the GNU Free
Documentation License from time to time. Such new versions will be similar in
spirit to the present version, but may differ in detail to address new problems
or concerns. See http://www.gnu.org/copyleft/.

Each version of the License is given a distinguishing version number. If the
Document specifies that a particular numbered version of this License "or any
later version" applies to it, you have the option of following the terms and
conditions either of that specified version or of any later version that has
been published (not as a draft) by the Free Software Foundation. If the
Document does not specify a version number of this License, you may choose any
version ever published (not as a draft) by the Free Software Foundation. If the
Document specifies that a proxy can decide which future versions of this
License can be used, that proxy's public statement of acceptance of a version
permanently authorizes you to choose that version for the Document.

=== 11. RELICENSING

"Massive Multiauthor Collaboration Site" (or "MMC Site") means any World Wide
Web server that publishes copyrightable works and also provides prominent
facilities for anybody to edit those works. A public wiki that anybody can edit
is an example of such a server. A "Massive Multiauthor Collaboration" (or
"MMC") contained in the site means any set of copyrightable works thus
published on the MMC site.

"CC-BY-SA" means the Creative Commons Attribution-Share Alike 3.0 license
published by Creative Commons Corporation, a not-for-profit corporation with a
principal place of business in San Francisco, California, as well as future
copyleft versions of that license published by that same organization.

"Incorporate" means to publish or republish a Document, in whole or in part, as
part of another Document.

An MMC is "eligible for relicensing" if it is licensed under this License, and
if all works that were first published under this License somewhere other than
this MMC, and subsequently incorporated in whole or in part into the MMC, (1)
had no cover texts or invariant sections, and (2) were thus incorporated prior
to November 1, 2008.

The operator of an MMC Site may republish an MMC contained in the site under
CC-BY-SA on the same site at any time before August 1, 2009, provided the MMC
is eligible for relicensing.

