OpenSQLAnywhere - Page-Store Format Specification

Status: v0.1 draft. Covers the page-store layer only. Catalog-row decoding is deferred to v0.2.

This document describes the on-disk format of SAP SQL Anywhere 17 (build 2182, 2015 release) database files as it is actually observed on disk. Every non-trivial claim is tagged either observed (directly verified by reading raw bytes) or inferred (consistent with SAP's public documentation but not yet directly validated).

0. Conventions

• Multi-byte integers are denoted u16_LE / u32_LE etc., and are little-endian unless stated.
• Offsets are in bytes from start-of-file unless noted.
• Hex literals use the 0x prefix.

1. File shape

An SA17 page-store file is a sequence of fixed-size 4 KiB pages.

Fact	Value
Page size	4096 bytes (4 KiB)
File size	Exact multiple of page size
Page 0	Superblock (see §3)
Pages 1..N	Data / metadata / index / catalog pages

observed - 112 files, 456 521 pages, 100 % page-aligned.

2. Universal per-page invariants

Every page, including page 0, is subject to the following structural invariants.

2.1 CRC-32 footer at 0xFFC

observed (456 521 / 456 521 pages):

page[0xFFC..0x1000] == u32_LE( crc32( page[0x0000..0x0FFC] ) )

The CRC polynomial is the standard zlib / IEEE 802.3 CRC-32 (same as crc32fast::hash in Rust). The CRC is computed over the raw on-disk bytes - for obfuscated variants such as QBW this means the footer can be validated without peeling the obfuscation layer first.

2.2 Twelve-byte page trailer at 0xFF0..0xFFB

observed (456 409 non-zero pages, zero invariant failures):

Offset	Size	Field	Notes
0xFF0	1	flag_ff0	per-type variable byte
0xFF1	1	flag_ff1	almost always 0x00
0xFF2	1	page_type	ASCII letter; see §2.3
0xFF3	1	zero_ff3	always 0x00
0xFF4	4	lsn	u32_LE log sequence number; see §2.2a
0xFF8	4	zero_ff8	always four zero bytes
0xFFC	4	crc32_le	see §2.1

2.2a Page LSN at 0xFF4..0xFF8

observed: the four bytes at 0xFF4 are a little-endian u32 carrying the log sequence number of the page's last write, not two loose metadata bytes followed by six reserved zeros.

The field is genuinely 32 bits wide. It reads as reserved-zero above 0xFF5 only on files whose LSN has never exceeded 65 535, which is true of the entire original 112-file corpus - hence the earlier "six zero bytes" invariant. On larger files bytes 0xFF6..0xFF7 carry the high half, and treating them as reserved rejects valid pages.

Three independent in-file relationships support the reading (measured by an independent reader across three files, reported in opensqlany#7):

1. The maximum per-page value is exactly ten less than the u32 at superblock 0x08 (§3.1a):

   file	superblock 0x08	max page 0xFF4	difference
   sample A	35 800	35 790	10
   sample B	1 848 242	1 848 232	10
   production file	747 273 956	747 273 946	10

2. Superblock 0x0C, which a 64-bit file identifier would need, is zero in all three files.

3. Pages carrying the 0x20 case bit in their page-type byte (§2.3) have a substantially newer LSN distribution than pages without it.

Inferred: the constant offset of ten is a fixed reservation between the file's current LSN and the highest LSN actually stamped on a page, and the case-bit correlation in (3) marks pages written after a checkpoint. Neither is confirmed. What the reserved four bytes at 0xFF8 are for is still unknown.

2.3 Page-type alphabet

observed (corpus-wide frequency, pages 1..N):

Byte	ASCII	Count	%	Working name
0x45	'E'	242 460	53.1 %	extent / data page
0x41	'A'	208 783	45.7 %	allocation / free-space map
0x4D	'M'	3 159	0.69 %	map / metadata
0x48	'H'	550	0.12 %	header block
0x43	'C'	492	0.11 %	catalog
0x40	'@'	448	0.10 %	reserved / bootstrap
0x49	'I'	421	0.09 %	index
0x47	'G'	96	0.02 %	unknown

Names are working hypotheses consistent with SA's published storage design; the type-byte value itself is observed, the semantic interpretation is inferred.

observed on a QuickBooks Desktop Enterprise 24.0 file: the same letters can appear lowercase (e.g. 'e' 0x65 instead of 'E' 0x45) on otherwise-ordinary extent/data pages. Classification is case-insensitive; the raw byte (case included) is preserved for callers that want to distinguish the variant. What the case bit signals is not yet known - see §6.

3. Page-0 superblock

Page 0 is the only page with extensive plaintext structure. The first 64 bytes follow a fixed layout; the rest of the page contains a collation block (§3.3) and a rolling copyright fingerprint (§3.4).

3.1 Fixed header at 0x00..0x2F

observed invariants (112 files):

Offset	Size	Type	Name	Notes
0x00	6	zeros	reserved_0	always 00 00 00 00 00 00
0x06	1	u8 flag	flags_06	bitfield over base 0x09; observed 0x09, 0x49 (|0x40), 0x29 (|0x20)
0x07	1	zero	reserved_07	always 0x00
0x08	4	u32_LE	current_lsn	file's current LSN; see §3.1a
0x0C	4	zeros	reserved_0C	always 00 00 00 00
0x10	4	u32_LE	format_major	always 3
0x14	4	u32_LE	magic	always 0xDA7ABA5E
0x18	2	u16_LE	version_a	always 201 (0x00C9)
0x1A	2	u16_LE	version_b	always 12 (0x000C)
0x1C	4	u32_LE	page_count_hint	typically total_pages - 128
0x2D	3	const	const_2D	always 0D 04 00
0x30	16	zeros	reserved_30	always 16 zero bytes

3.1a Current LSN at 0x08

observed: the u32 at 0x08 is the file's current log sequence number, not the low half of a per-file identifier. It is unique per file in practice, which is what the identifier reading was based on, but it behaves as a write counter: it is exactly ten greater than the highest per-page LSN in every file measured (§2.2a), and the four bytes at 0x0C that a 64-bit identifier would occupy are always zero.

3.2 Page-count hint

observed: page_count_hint == total_pages - 128 in 112 / 112 files when total_pages >= 128. Inferred: the first 128 pages are reserved metadata and the hint counts data pages after them.

3.3 Collation / codepage block (~0x162..0x1FF)

observed substrings in every file in this region, in this order:

• 1252LATIN1 - CHAR collation
• windows-1252 - CHAR codepage label
• UCA - Unicode Collation Algorithm (NCHAR collation)
• UTF-8 - NCHAR codepage label

These are standard SA17 collation record strings and their invariant presence is one of the strongest pieces of evidence that the payload is a genuine SAP SQL Anywhere database image.

3.4 Engine fingerprint (0x400..0xFFC)

observed in 112 / 112 files: offsets 0x400..0xFFC contain the 38-byte ASCII string

"2182 SAP SE, Copyright (c)2015 17.0.4."

repeated as a rolling cycle. The final four bytes 0xFFC..0x1000 are the CRC-32 footer per §2.1. The fingerprint pins the engine to SAP SQL Anywhere 17.0.4 build 2182.

4. Pages 1..N - body layout

4.1 Trailer and CRC apply universally

Every page 1..N has the trailer (§2.2) and CRC (§2.1). No exceptions in the observed corpus.

4.2 Slotted-page directory (catalog and data pages)

Pages with real row data use the classic slotted-page layout:

• a header region at the start of the page,
• row bodies grown downward from near the page end,
• a descending array of u16_LE row-offset entries.

observed quirks:

• the slot array may start on an odd byte boundary,
• a 0x0000 sentinel word may immediately precede the first entry,
• deleted slots appear as interior zero words,
• the array bytes immediately before the array contain the minimum row offset and the slot count, though the exact field layout varies by page type.

observed worked examples on a Rock Castle sample:

Page	Type	Array start	Slots	Min offset
2	A	0x06B	39	0x076
11	E	0x071	124	0x1C2
340	E	0x09D	44 (2 deleted)	0x1F8

4.3 SYSTABLE row tag

observed: SYSTABLE rows carry an 8-byte invariant tag immediately before the table_name column:

05 00 00 00                    row marker
<table_id u32_LE> 00 00 00 00  32-bit id, zero-padded to 8 bytes
b1 0d 19 0d 00 00 00 00        fixed record tag
<name_len u8> <name ASCII>

The trailer after the name contains monotone sequences that correspond to SA object_id space, but the exact variable-field layout is not yet decoded. Deferred to v0.2.

4.4 Other page types

Index (I), allocation (A) bitmap, map (M), and header (H) page bodies are outside the scope of v0.1. Their trailers (§2.2) and CRC footers (§2.1) are validated; their bodies are opaque to this spec.

5. What is deliberately out of scope for v0.1

Item	Status
SYSCOLUMN, SYSINDEX, SYSUSER row formats	v0.2
Typed column decoding (DECIMAL, VARCHAR, DATE, ...)	v0.2+
first_page / primary_root pointers from SYSTABLE	v0.2
Multi-dbspace database (13-file) layout	later
Write support	out of scope indefinitely
Encryption (the SA strong-encryption option)	not encountered in corpus

6. Open questions

• Meaning of flags_06's variant bits (bit 6 0x40, bit 5 0x20 - see Superblock::flags_06_variant_bits). Known to be a bitfield over base 0x09 rather than an enum of magic values; what each bit signals is still unknown.
• Field layout of the SYSTABLE row trailer past the name column.
• Meaning of the prelude fields before the slot array (0x0404, 0x30C6, 0x0304, 0x022C, 0x0001, 0x05B4, ...).
• Interior structure of I, A, M, H pages.
• Meaning of the lowercase page-type variant (§2.3). Circumstantial evidence ties it to recency (pages carrying the lowercase/0x20 case bit show a newer LSN distribution than same-type pages without it, consistent with marking pages written after a checkpoint), but this isn't confirmed. See OpenSQLAnywhere#7 and OpenSQLAnywhere#8.
• ap.rs's per-sector step recovery uses histogram-peak search, but on at least one file step shows global structure instead: values repeat in pairs and shift by one sector per page (steps(pn+1)[1:] == steps(pn)[:7] on 723/1443 adjacent trusted page pairs in one sample). The per-block-pair index into that structure appears permuted and wasn't recovered - borrowing a neighbouring page's step sequence to decode a dense page failed 0/40 times. If the keystream is genuinely global, deriving it once would be faster and more robust than per-sector histogram recovery, but someone needs to find the index permutation first. See OpenSQLAnywhere#8 (finding 3) for the full evidence and the two ruled-out approaches.

7. References

• SAP SQL Anywhere 17.0.01 product documentation: https://help.sap.com/docs/SAP_SQL_Anywhere
• crc32fast Rust crate (IEEE 802.3 / zlib polynomial): https://crates.io/crates/crc32fast