DecodePhysicalRecord only checked length upper bound, not lower bound
(length > 0 per design §3.2 line 389). It also didn't validate the
fragment type field (RecInvalid=0 and types > RecLast are corruption
indicators per design §3.2 line 366-372).
Corrupt data with length=0 could pass CRC (payload is empty, CRC only
covers length+type bytes) and inject empty records into the fragment
collector. Invalid type values would only be caught at the collector
level, wrapped as TailCorruptionError, rather than rejected at the
parser level.
Changes:
- wal/record.go: DecodePhysicalRecord now rejects length=0 and
type ∉ {RecFull..RecLast} before payload copy and CRC check.
Checks are ordered to reject invalid records ASAP.
- wal/record_test.go: 5 tests:
- TestDecodePhysicalRecord_RejectZeroLength (H1)
- TestDecodePhysicalRecord_RejectInvalidType (H2, type=0)
- TestDecodePhysicalRecord_RejectUnknownType (H2, type>RecLast)
- TestDecodePhysicalRecord_ValidRecordsUnaffected (regression for
all 4 valid types)
- TestParseBlockWrapsInvalidRecordAsTailCorruption (integration:
ParseBlock wraps H1/H2 errors as TailCorruptionError)
Verified: all existing tests pass. Full suite green including
go test -race ./... .
Audit context: docs/audit-3.2.md H1+H2.
Per design §3.2 line 359, all WAL CRC computations must use crc32c
(Castagnoli polynomial 0x82F63B78). The previous code used crc32.IEEE
(Ethernet/PNG polynomial 0xEDB88320) in 5 places. While the system was
self-consistent (encode + decode both used IEEE), it diverged from the
design spec and lost SSE4.2 hardware acceleration (the native CRC32
instruction only supports Castagnoli).
Changes:
- wal/crc32c.go (new): package-level crc32cTable = crc32.MakeTable(
crc32.Castagnoli). Central definition prevents future drift.
- wal/header.go: 2 ChecksumIEEE calls replaced with crc32.Checksum(
data, crc32cTable). Comment updated to reference design §3.2 line 341, 359.
- wal/record.go: 2 ChecksumIEEE calls replaced.
- wal/block_writer_test.go: 1 ChecksumIEEE call in test helper replaced.
- wal/crc32c_test.go (new): 4 regression guards:
- TestCRC32CStandardVector: RFC 3720 fixed vector (crc32c("123456789")
= 0xE3069283).
- TestCRC32CEdistinctFromIEEE: confirms IEEE produces different value.
- TestHeaderCRCUsesCastagnoli: direct assertion on stored header CRC
(catches paired encode/decode reversion that round-trip tests miss).
- TestPhysicalRecordCRCUsesCastagnoli: same for physical record CRC.
BREAKING CHANGE: WAL files written before this fix (with IEEE CRC)
cannot be read after this fix (expects crc32c). Phase 1 has not been
released, so no real data migration is needed. Production users
post-release would need to drain + re-create the database.
Developers pulling this change should delete any local Phase-1 WAL
directories (`rm -rf <db-dir>/segment-*.wal`) before running the code;
old IEEE-encoded WALs will fail recovery on local dev machines.
Verified: all existing round-trip tests pass (encode + decode both use
crc32c, still self-consistent). Full suite green including
go test -race ./... .
Audit context: docs/audit-3.2.md C1.
- go.mod with github.com/dailz/go-kv, Go 1.26.3, testify
- config/config.go with WalConfig, Validate() with checked arithmetic
- errors.go with sentinel errors (ErrCommitUnknown, ErrWriteStopped, etc.)
- wal/constants.go with all WAL format constants and enums
- wal/header.go with WAL File Header encode/decode (CRC32 IEEE)
- wal/record.go with Physical Record codec, block boundary, SplitIntoRecords
- wal/entry.go with WAL Entry codec (varint keys/values, OpType, ValueKind)
- wal/sequence.go with SequenceManager (atomic, CAS, overflow-safe)
- manifest/manifest.go with MANIFEST stub (Load/Save atomic)
- manifest/current.go with CURRENT file (WriteCurrent/ReadCurrent)
- Comprehensive tests for all modules
- .golangci.yml configuration
dailz