ntdll format modules¶
LZNT1 (0x0002)¶
ntcompress.ntdll.lznt1
¶
MS-XCA LZNT1 raw codec (COMPRESSION_FORMAT_LZNT1, 0x0002).
Standalone MS-XCA codec operating on a bare stream. An LZNT1 buffer is a series of independently decompressible chunks, each beginning with a 16-bit little-endian header: bit 15 is the compressed flag, bits [14:12] are a signature that MUST be 3, and bits [11:0] hold the total chunk size minus three bytes ([MS-XCA] §2.5.1.2). A header of 0x0000 is the optional End_of_buffer terminal. Compressed chunks hold flag groups: one flag byte followed by up to eight data elements, where a clear bit means a literal byte and a set bit means a 16-bit compressed word ([MS-XCA] §2.5.1.3). A compressed word packs a D-bit displacement (high bits) and an L-bit length (low bits) with D + L = 16; D grows with the amount of chunk-relative uncompressed data already processed ([MS-XCA] §2.5.1.4). Stored displacement is actual minus 1; stored length is actual minus 3 (minimum match 3).
Input is compressed in units of 4096 bytes per chunk ([MS-XCA] §2.5.3), and matches that overlap the write cursor MUST be copied front to back so a word may reference bytes it is itself producing.
Authority: [MS-XCA] §2.5 (algorithm details), §2.5.3 (processing rules),
§3.3 (worked example, used as the pinned test vector).
decompress(data: Buffer) -> bytes
¶
Decode a raw MS-XCA LZNT1 buffer to plaintext.
Walks the chunk sequence of [MS-XCA] §2.5.1.2: a 0x0000 header is the optional End_of_buffer terminal, an uncompressed chunk carries raw literal data, and a compressed chunk carries flag groups.
Raises:
| Type | Description |
|---|---|
DecompressionError
|
Bad signature, truncated chunk, or a match that reaches before the start of the output. |
compress(data: Buffer) -> bytes
¶
Encode plaintext into a raw MS-XCA LZNT1 buffer.
Consumes the input in 4096-byte units per [MS-XCA] §2.5.3, emitting one chunk each: a compressed chunk when the flag-group body is smaller than the raw data, otherwise an uncompressed chunk.
XPRESS / Plain LZ77 (0x0003)¶
ntcompress.ntdll.xpress
¶
MS-XCA Plain LZ77 (LZXPRESS) raw stream codec (COMPRESSION_FORMAT_XPRESS, 0x0003).
This is the canonical home for the raw Plain LZ77 codec. The ESE XPRESS (scheme 0x3) module imports from here and adds ESE framing on top.
Wire format ([MS-XCA] §2.3.4 encode / §2.4.4 decode):
- Literal/match flags are packed in 32-bit little-endian words, consumed MSB-first (1 = match, 0 = literal); the final partial word is padded with 1-bits.
- A match is a 16-bit little-endian token: high 13 bits = offset - 1 (so the largest offset is 8192), low 3 bits = length - 3. A low-bits value of 7 escapes to the extended-length ladder: a 4-bit nibble (two consecutive long matches share one byte, low nibble first), then if 15 an 8-bit byte, then if 255 a 16-bit word, then (v10.0) if that word is 0 a 32-bit dword.
- Match copies run one byte at a time because a match may overlap its own output (length > offset), [MS-XCA] §2.4.4.
Authority: [MS-XCA] §2.3 (compression), §2.4 (decompression), §3.1
(worked examples).
MIN_MATCH: Final = 3
module-attribute
¶
Smallest encodable match ([MS-XCA] §2.3.4 "length of at least 3").
MAX_OFFSET: Final = 8192
module-attribute
¶
Largest encodable match distance, 2^13 ([MS-XCA] §2.3.4).
MAX_MATCH_LENGTH: Final = 4294967295
module-attribute
¶
MatchLength is a ULONG, so 4,294,967,295 at most ([MS-XCA] §2.3.4).
decompress(data: Buffer, /, *, max_size: int | None = None) -> bytes
¶
Decode a raw MS-XCA Plain LZ77 stream to plaintext.
Direct port of [MS-XCA] §2.4.4: flag words are read 4 bytes at a time and consumed MSB-first; a 0 bit copies one literal byte, a 1 bit decodes a match.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
Buffer
|
The compressed stream. |
required |
max_size
|
int | None
|
Size of the caller's output buffer, when one is known. |
None
|
Raises:
| Type | Description |
|---|---|
DecompressionError
|
Any read outside the input, a match reaching before the
start of the output, or a write past |
compress(data: Buffer) -> bytes
¶
Encode plaintext into a raw MS-XCA Plain LZ77 stream.
Direct port of [MS-XCA] §2.3.4 with a greedy hash-chain match finder.
XPRESS_HUFF / LZ77+Huffman (0x0004)¶
ntcompress.ntdll.xpress_huff
¶
MS-XCA LZ77+Huffman raw codec (COMPRESSION_FORMAT_XPRESS_HUFF, 0x0004).
Standalone MS-XCA codec operating on a bare stream. Data is processed in 64 KiB blocks; each block begins with a 256-byte table giving the 4-bit code length of all 512 Huffman symbols (0-255 literals, 256 EOF, 257-511 matches), followed by the Huffman-coded literals and matches read as 16-bit little-endian chunks through a 32-bit register. The whole stream ends with the EOF symbol (256).
This targets [MS-XCA] v10.0: the match-length continuation is nibble -> byte ->
uint16 -> (v10.0) uint32, removing the old 65,538-byte cap.
Authority: [MS-XCA] §2.1 (compression), §2.2 (decompression), §3.2
(worked example).
BLOCK_SIZE: Final = 65536
module-attribute
¶
Each block decodes to at most 64 KiB of output ([MS-XCA] §2.1).
TABLE_SIZE: Final = 256
module-attribute
¶
The per-block Huffman code-length table is 512 symbols x 4 bits = 256 bytes (§2.1.4.3).
SYMBOL_COUNT: Final = 512
module-attribute
¶
Huffman alphabet size: 0-255 literals, 256 EOF, 257-511 matches (§2.1).
EOF_SYMBOL: Final = 256
module-attribute
¶
End-of-file marker symbol, encoded after the final block (§2.1).
MIN_MATCH: Final = 3
module-attribute
¶
Minimum match length; 3 is subtracted before length encoding (§3.2).
MAX_CODE_LENGTH: Final = 15
module-attribute
¶
A code length is stored in 4 bits, so no Huffman code may exceed 15 bits (§2.1.4.2).
MAX_OFFSET: Final = 65535
module-attribute
¶
Largest encodable match distance.
decompress(data: Buffer, /, *, max_size: int | None = None) -> bytes
¶
Decode a raw MS-XCA LZ77+Huffman stream to plaintext (per [MS-XCA] §2.2).
Processes 64 KiB blocks in order, each led by its own 256-byte Huffman table, until the EOF symbol (256) is decoded.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
Buffer
|
The raw LZ77+Huffman stream. |
required |
max_size
|
int | None
|
Optional ceiling on the decoded length. |
None
|
Raises:
| Type | Description |
|---|---|
DecompressionError
|
Any read past the end, malformed Huffman table, or output
exceeding |
compress(data: Buffer) -> bytes
¶
Encode plaintext into a raw MS-XCA LZ77+Huffman stream (per [MS-XCA] §2.1).
Runs a greedy LZ77 pass, segments the tokens into 64 KiB output blocks each with its own canonical Huffman table, and appends the EOF symbol after the final block.
Compact XPRESS9 (0x0005)¶
ntcompress.ntdll.xpress9
¶
Compact XPRESS9 codec (COMPRESSION_FORMAT_XPRESS9, 0x0005).
Undocumented ntdll.dll compression format introduced in Windows Server 2022
(Build 20348). Uses the same canonical-Huffman LZ77 engine as the ESE XPRESS9
codec (ntcompress.ese.xpress9) but with a streamlined 10-byte header instead
of XPRESS9's 32-byte block header.
Reverse-engineered from ntdll.dll Build 20348 (decompressor at RVA 0x111810,
Huffman builder at 0x114DA8, header parser at 0x115AB0). The header magic is
0xC039E510; the bitstream after the header is identical to XPRESS9's token
format: canonical Huffman short-symbol table (704 entries for window_log=24) for
literals (0-255) and match tokens (256+), a second long-length table (256 entries)
for escaped match lengths, and an LZ77 token stream with 4-entry MTF support.
On-disk layout:
bytes 0-3: magic 0xC039E510 (LE u32)
bytes 4-5: params (LE u16) -- bits 0-2: window_log index; bit 3: mode
bytes 6-9: control (LE u32) -- bits 0-27: payload bit count (comp_data + 32-bit CRC);
bit 29: compressed flag; bit 31: end-of-stream
bytes 10+: payload data (ceil(payload_bits / 8) bytes)
trailing: CRC-32C of the original plaintext (32 bits, inside the payload area)
MAGIC: Final = 3225019664
module-attribute
¶
Block magic -- distinct from XPRESS9's 0x4E86D72A.
HEADER_SIZE: Final = 10
module-attribute
¶
Fixed header size: 4 (magic) + 2 (params) + 4 (control).
decompress(data: Buffer, /, *, verify: bool = True) -> bytes
¶
Decompress a compact XPRESS9 stream (ntdll format 0x0005).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
Buffer
|
The compressed stream including the 10-byte header. |
required |
verify
|
bool
|
If True (default), verify the trailing CRC-32C against the decompressed plaintext. Set to False to skip verification. |
True
|
Returns:
| Type | Description |
|---|---|
bytes
|
The decompressed plaintext. |
Raises:
| Type | Description |
|---|---|
DecompressionError
|
The stream is corrupt or truncated. |
IntegrityError
|
CRC-32C mismatch (only when |
compress(data: Buffer) -> bytes
¶
Compress data into a compact XPRESS9 stream (ntdll format 0x0005).
Uses flat (stored) Huffman tables for simplicity. Produces valid streams that
RtlDecompressBufferEx(0x0005) accepts, but not necessarily byte-identical
to RtlCompressBuffer(0x0005) output.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
Buffer
|
The plaintext to compress. |
required |
Returns:
| Type | Description |
|---|---|
bytes
|
The compressed stream including header and CRC-32C trailer. |
XP10 / LZ4 block (0x0006)¶
ntcompress.ntdll.xp10
¶
XP10 raw LZ4 block codec (COMPRESSION_FORMAT_XP10, 0x0006).
Windows ntdll.dll exposes raw LZ4 block compression as CompressionFormatAndEngine
0x0006 (default engine) and 0x0106 (ENGINE_MAXIMUM). Available on Win11 / Server 2025
(Build 26100+).
The bitstream is the standard LZ4 block format -- the same codec used by ESE XPRESS10 (scheme 0x06), but without the 15-byte ESE header. The uncompressed size is not stored in the stream; the caller must supply it.
Thin wrapper over :mod:ntcompress.ese.lz4 block-level functions.
decompress(data: Buffer, /, uncompressed_size: int = 0) -> bytes
¶
Decompress a raw LZ4 block.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
Buffer
|
The compressed LZ4 block payload. |
required |
uncompressed_size
|
int
|
Expected decompressed size. Required because the LZ4 block format does not store the original size. |
0
|
Returns:
| Type | Description |
|---|---|
bytes
|
The decompressed bytes. |
compress(data: Buffer) -> bytes
¶
Compress data into a raw LZ4 block.
Produces byte-identical output to RtlCompressBuffer(0x0006) on Windows.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
Buffer
|
The plaintext to compress. |
required |
Returns:
| Type | Description |
|---|---|
bytes
|
The compressed LZ4 block. |
DEFLATE (0x0007)¶
ntcompress.ntdll.deflate
¶
Raw DEFLATE codec (ntdll format 0x0007, library extension Format.DEFLATE).
Windows ntdll.dll exposes raw DEFLATE as CompressionFormatAndEngine 0x0007
(default engine) and 0x0107 (ENGINE_MAXIMUM). The bitstream is RFC 1951 raw
deflate with no zlib or gzip wrapper. Available on Win11 / Server 2025
(Build 26100+).
Thin wrapper over Python's zlib module with wbits=-15 (raw deflate).
decompress(data: Buffer, /, *, max_size: int | None = None) -> bytes
¶
Decompress a raw DEFLATE stream (RFC 1951, no zlib/gzip wrapper).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
Buffer
|
The compressed DEFLATE bitstream. |
required |
max_size
|
int | None
|
Optional output size limit. If the decompressed output would
exceed this, a |
None
|
Returns:
| Type | Description |
|---|---|
bytes
|
The decompressed bytes. |
compress(data: Buffer, /, *, level: int = 1) -> bytes
¶
Compress data into a raw DEFLATE stream (RFC 1951, no wrapper).
The default level (1) matches Windows ntdll.dll RtlCompressBuffer(0x0007)
byte-for-byte. Level 7 matches ENGINE_MAXIMUM (0x0107).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
Buffer
|
The plaintext to compress. |
required |
level
|
int
|
Compression level 0-9 (default 1, matching ntdll default engine). |
1
|
Returns:
| Type | Description |
|---|---|
bytes
|
The compressed raw DEFLATE bitstream. |
ZLIB (0x0008)¶
ntcompress.ntdll.zlib
¶
ZLIB codec (ntdll format 0x0008, library extension Format.ZLIB).
Windows ntdll.dll exposes ZLIB as CompressionFormatAndEngine 0x0008
(default engine) and 0x0108 (ENGINE_MAXIMUM). The bitstream is RFC 1950 ZLIB
(DEFLATE with a 2-byte header and 4-byte Adler-32 trailer). Available on
Win11 / Server 2025 (Build 26100+).
Thin wrapper over Python's zlib module with default wbits=15.
decompress(data: Buffer) -> bytes
¶
Decompress a ZLIB stream (RFC 1950 header + DEFLATE + Adler-32).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
Buffer
|
The compressed ZLIB stream. |
required |
Returns:
| Type | Description |
|---|---|
bytes
|
The decompressed bytes. |
compress(data: Buffer, /, *, level: int = 1) -> bytes
¶
Compress data into a ZLIB stream (RFC 1950).
The default level (1) matches Windows ntdll.dll RtlCompressBuffer(0x0008)
byte-for-byte. Level 7 matches ENGINE_MAXIMUM (0x0108).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
Buffer
|
The plaintext to compress. |
required |
level
|
int
|
Compression level 0-9 (default 1, matching ntdll default engine). |
1
|
Returns:
| Type | Description |
|---|---|
bytes
|
The compressed ZLIB stream (2-byte header + DEFLATE + Adler-32). |