_FUNCnnn.IDX

Binary scan index file. One file per function (e.g. _FUNC001.IDX, _FUNC002.IDX). Provides random access into the paired .DAT file.

Status: Mostly Decoded (Phase 4)

Core structure and all critical access fields are confirmed. A few instrument-state fields remain without a confirmed interpretation.

Variant A: 22-byte record (non-IMS / simple TOF-MS)

Observed in: PXD058812 (Waters QTOF), PXD041695 (QTof), and PXD081045 (Vion UNIFI export), and MTBLS701/MTBLS13770 (LCT Premier). Vion uses 8-byte DAT records; the LCT mass functions use 12-byte records.

Key facts:

• File size = N x 22 bytes (exact, no header)
• u32@0x00 = DAT byte offset - confirmed
• Scan 0-2 are often zero-data "blank" scans (12 bytes each in DAT)

Offset	Type	Confirmed	Description
0x00	u32	Yes	Byte offset into .DAT file
0x04	u32	Yes	Lower 24 bits = DAT record count; high byte = format marker (0x18, or 0x4c/0x08 in the public LCT mass/lock functions)
0x08	f32	Partial	Non-zero for data scans, 0 for blank scans. Correlates with scan signal level but no confirmed formula.
0x0C	f32	Yes	Retention time (minutes)
0x10	u16	Yes	Centroid peak count (0 for blank scans, 17-196 per data scan in corpus)
0x12	u16	No	Hardware register; varies over run (62496→34848 in corpus); purpose unknown
0x14	u16	No	Hardware register; slow drift (~38446-38468 in corpus); purpose unknown

Validated: 22 x 197 = 4334 bytes (molecular_mass_P15_01.raw), 22 x 426 = 9372 bytes (MS_fragmentation_P29_01.raw)

Field +0x04: Packed type-code and record count

Format: (marker << 24) | n_records.

The lower 24 bits count DAT records. PXD041695 has scans with more than 65,535 records, so reading only the lower 16 bits truncates them. The high byte is 0x18 in the other observed functions and 0x4c/0x08 in the public LCT mass/lock functions. DAT records may be 6, 8 or 12 bytes; consecutive DAT offsets establish the width. Those offsets also establish scan boundaries directly. At sizes divisible by both 22 and 30, these known markers and ordered DAT offsets disambiguate the two variants.

peak_count cannot be asserted for equality against the length returned by the current Encoding A decoder. In the corpus, a scan can contain thousands of non-zero 6-byte DAT records while the IDX field reports only tens or hundreds of centroid peaks (for example, 3,253 records and 47 peaks). The decoder emits the former after removing sentinel and zero-intensity records; it does not centroid, so its output count and peak_count measure different things and an equality check would reject valid corpus scans.

What does hold, and is what Reader::decode_scan checks (check_peak_count_sanity in reader.rs, which logs a warning when it fails): a centroid count can never exceed the raw decoded record count, since every centroid is built from at least one raw point. peak_count <= decoded_len is therefore a cheap decode sanity check - it catches a decode that produced implausibly few points for the peak count the index claims - without depending on the unimplemented centroiding relationship (Sigilweaver/OpenWRaw#24).

n_records = u32@0x04 & 0x00FFFFFF provides the record count when the next offset is unavailable. When the IDX length is divisible by both 22 and 30, the repeated 0x18 marker and monotonic offsets distinguish this variant from Variant B.

Variant B: 30-byte record (IMS / HDMS and non-IMS QTof)

Observed in:

• PXD066594 (WANG.raw, SYNAPT G2-Si, IMS)
• PXD068881 (CtpA, SYNAPT G2-Si, IMS)
• PXD075602 (DHPR_11257-1.raw, Xevo G2-XS QTof, non-IMS)

Key facts:

• File size = N x 30 bytes (exact, no header)
• DAT byte offset stored at +0x16 (NOT +0x00) - confirmed for all three datasets
• Scan sizes vary depending on ion density
• No record count: a scan runs from its offset to the next record's offset (the last scan runs to the end of the DAT)
• The index does not determine the DAT record layout. Behind this variant the corpus holds 8-byte Encoding D records and, in some Xevo G2-XS lock-mass functions, 12-byte Encoding E records; the width is judged from the data (see _FUNCnnn.DAT, "Choosing the record width")
• Both SYNAPT (mobility-capable) and Xevo QTof instruments use this variant

Offset	Type	Confirmed	Description
0x00	u32	No	Flags (always 0 in tested datasets)
0x04	u32	Yes	Hardware scan-duration counter; see note below
0x08	u32	Partial	Hardware event counter; scan-varying; see note below
0x0C	f32	Yes	Retention time (minutes)
0x10	u32	No	4-byte field; interpreted as f32 gives scan-varying values (~18577 in WANG) but some scans have non-physical f32 garbage; purpose unclear
0x14	u16	No	Hardware register; large values (35124-65208 in WANG); purpose unknown
0x16	u32	Yes	Byte offset into .DAT file, low 32 bits
0x1A	u32	Yes	Byte offset into .DAT file, high 32 bits (0 while the DAT is under 4 GiB)

The DAT offset is a 64-bit little-endian value at +0x16..+0x1E. Most DAT files are under 4 GiB, so the high word at +0x1A is usually 0. Functions with a larger DAT set it once the offset crosses 4 GiB: in PXD045625 Abu_190520_Sha11.raw, _FUNC001.IDX scan 2699 has low word 1,626,312 and high word 1 (offset 4,296,593,608). Three corpus functions (7.0 GB, 5.9 GB and 4.5 GB DAT files) have offsets past 4 GiB, covering 2,708 scans. Reading only the low word wraps the offset back to the start of the file.

Validated: sum of (IDX[i+1].offset - IDX[i].offset) for all i = DAT file size exactly. Offsets are non-decreasing and never exceed the DAT size in the corpus; the reader rejects a scan whose offsets go backwards or past the end of the DAT.

Field +0x04: Hardware tick counter

Observed range: approximately 402.6M-402.8M across all corpus instruments. Varies by +/-50K between consecutive scans (jitter in scan execution time).

Hypothesis: total 800-MHz reference-clock ticks elapsed during the scan. Predicted value = n_pushes x (pusher_period_us x 800):

Dataset	n_pushes	IDX+0x04 / n_pushes	Implied period at 800 MHz
DHPR	8299	48520	60.6 us
WANG	14389	27990	35.0 us
CtpA	4317	93299	116.6 us

DHPR result (60.6 us) is consistent with the FUNCTNS.INF-derived 60.25 us pusher period. WANG result (35.0 us) differs from the 69 us implied by tof_depth alone. CtpA (116.6 us) is anomalously long; may reflect a different acquisition mode.

The 800 MHz figure is inferred; the actual clock rate is not confirmed from available data.

Field +0x08: Scan counter (purpose unclear)

Observed range: approximately 1.17B-1.29B across corpus. Fluctuates by up to 7% from first to last scan of a run. The variation pattern across scans is consistent with a TIC chromatogram (rises and falls with elution) but the absolute value does not match the sum of intensity values from the DAT records by any simple factor. May encode raw TDC hit counts from the detector hardware before per-scan intensity processing.

Distinguishing Variant A from B

• Check file_size mod 22 == 0 (Variant A) or mod 30 == 0 (Variant B)
• Both should be mutually exclusive in practice
• Variant B is used by both SYNAPT and Xevo QTof instruments, and does not by itself indicate an ion mobility acquisition

Fields Under Investigation

• Variant A: exact formula for +0x08 f32 (confirmed non-zero for data scans; correlated with TIC but no clean factor)
• Variant A: +0x12 and +0x14 hardware register semantics
• Variant B: +0x04 clock-rate confirmation (800 MHz assumed)
• Variant B: +0x08 raw TDC hit count vs processed intensity relationship
• Variant B: +0x10 4-byte field type and semantics
• Whether a 32-byte variant exists for other instrument generations

Reference Sources

• Empirical hex analysis: re/src/analysis/inspect.py records
• Corpus samples:
  • PXD066594/WANG.raw (Variant B, SYNAPT G2-Si, 590 scans)
  • PXD068881/20220517_CtpA_1076_2h_1.raw (Variant B, SYNAPT G2-Si, 1138 scans)
  • PXD058812/molecular_mass_P15_01.raw (Variant A, QTOF non-IMS, 197 scans)
  • PXD058812/MS_fragmentation_P29_01.raw (Variant A, QTOF non-IMS, 426 scans)
  • PXD075602/DHPR_11257-1.raw (Variant B, Xevo G2-XS QTof non-IMS, 1150 scans)