go-kv/.omo/plans/fix-c4-non-last-segment-corruption.md

C4 修复方案：非尾段损坏必须硬错误

TL;DR

目标：让 WAL recovery 按 design §3.2 line 704 区分"尾段损坏（可截断）"和"中间段损坏（硬错误）"。当前 ReplaySegmentFile 不论在哪个 segment 都把 CollectingFragments / parse error 当 TailCorruption，导致中间段损坏时 Recover 错误地截断最后一段的有效数据。

交付：

• ReplaySegmentFile 加 isLastSegment bool 参数
• TailCorruptionError 加 SegmentPath string 字段（诊断 + 防御）
• 非尾段遇到 corruption → 硬错误（不再是 TailCorruptionError）
• RecoverFromSegments 传递 isLastSegment 给 ReplaySegmentFile
• Recover 用 tce.SegmentPath 做截断目标（防御性 fallback 到 segments[last]）
• ~5 个新测试覆盖单/多 segment 场景
• 单次 commit

预估工时：2-3 小时 风险：中。ReplaySegmentFile 签名变更是 breaking change（影响测试调用）

---

Context

Bug 摘要

wal/recovery.go:133-138 在 ReplaySegmentFile 末尾：

if collector.State() == FragmentCollecting {
    return nextSequence, &TailCorruptionError{...}
}

不区分 segment 是不是最后一个。同样地，ParseRecordsFromFile / ParseBlock 返回的 TailCorruptionError 也不带 segment 位置信息。

Recover 拿到 TailCorruptionError 后，总是对 segments[len(segments)-1] 调用 truncate（C5+H8 修复后的 wal/recover.go:60-71）。如果 corruption 实际发生在中间段：

• 真正损坏的中间段不动
• 最后一段（完全有效）被错误截掉
• 中间段之后、最后段之前的有效 batch 全部丢失

Oracle 验证（bg_ef425776）

C4 还有一个更严重的失败模式。wal/recover.go:61 总是对 segments[len(segments)-1] 调用 truncateSegment，但 RecoverFromSegments 的 TailCorruptionError 可能来自非尾段。结果：真正损坏的 segment 不动，最后一段的有效数据被错误截掉。

设计依据（§3.2 line 704）

尾部损坏通常来自进程崩溃或机器掉电时最后一次写入的 partial write。若后面还有需要恢复的 segment，前一个 segment 的尾部异常不能按尾部损坏截断，否则可能跳过已经持久化历史并破坏 sequence 连续性。中间损坏说明已持久化 WAL 文件被破坏，默认不静默跳过，不默认 repair。

错误分类表（line 697-704）明确：所有 4 类物理损坏（header 半写 / length 越界 / CRC 失败 / 非 0 padding）在"WAL 中间"位置都是硬错误。

修复后的行为矩阵

Corruption 位置	isLastSegment	行为
最后一段尾部	true	TailCorruptionError → Recover 截断（现有行为）
中间段任何位置	false	硬错误（新行为）→ Recover 返回错误 → DB.Open 失败
干净 WAL	N/A	正常 replay，无 corruption

协同：C5+H8 / C8 关系

• C5+H8 已修：截断路径用 findLastCompleteBatchEnd（H8）+ 4 步协议（C5）。C4 修复后，截断路径只在"最后一段尾部损坏"时触发，行为正确。
• C8 未修：segment_manager.go:64-66 把字节偏移当 startSequence。但 writeTestSegment 直接构造 segment，可以指定正确 startSequence，所以 C4 测试不依赖 C8。
• 多 segment recovery 在 C4 修复后行为正确（前提是 startSequence 正确，即 C8 也已修）。但 C4 本身的修复不依赖 C8。

---

执行计划

Phase A：代码改动（45-60 分钟）

A.1 扩展 TailCorruptionError 加 SegmentPath

文件：wal/record_parser.go

type TailCorruptionError struct {
    Offset      int    // offset within the segment file
    SegmentPath string // NEW: file path of the corrupted segment, for diagnostics + Recover's truncation target
    Err         error
}

func (e *TailCorruptionError) Error() string {
    if e.SegmentPath != "" {
        return fmt.Sprintf("wal: tail corruption in %s at offset %d: %v", e.SegmentPath, e.Offset, e.Err)
    }
    return fmt.Sprintf("wal: tail corruption at offset %d: %v", e.Offset, e.Err)
}

必要 docstring：字段含义 + 用途（diagnostics + Recover 的截断目标）。

A.2 ReplaySegmentFile 加 isLastSegment 参数

文件：wal/recovery.go

// 改前：
func ReplaySegmentFile(filePath string, startSequence uint64, replayer BatchReplayer) (nextSequence uint64, err error)

// 改后：
func ReplaySegmentFile(filePath string, startSequence uint64, isLastSegment bool, replayer BatchReplayer) (nextSequence uint64, err error)

函数体改动：

nextSequence = startSequence
records, parseErr := ParseRecordsFromFile(filePath)
if parseErr != nil && !IsTailCorruption(parseErr) {
    return nextSequence, fmt.Errorf("wal: parse segment records: %w", parseErr)
}

// Attach SegmentPath to parseErr for downstream diagnostics + truncation target.
// Do this BEFORE the non-last hard-error conversion so both paths benefit.
if parseErr != nil {
    var tce *TailCorruptionError
    if errors.As(parseErr, &tce) {
        tce.SegmentPath = filePath
    }
}

// C4 fix: tail corruption in non-last segment is hard corruption per
// design §3.2 line 704. Use %v (NOT %w) so IsTailCorruption returns false
// for this wrapped error — otherwise errors.As would still find the
// underlying *TailCorruptionError and Recover would treat it as truncatable.
if parseErr != nil && !isLastSegment {
    return nextSequence, fmt.Errorf("wal: corruption in non-last segment %s (hard corruption): %v",
        filePath, parseErr)
}

collector := NewFragmentCollector()
for _, record := range records {
    // ... (unchanged)
}

if parseErr != nil {
    return nextSequence, parseErr
}
if collector.State() == FragmentCollecting {
    if isLastSegment {
        return nextSequence, &TailCorruptionError{
            Offset:      0,
            SegmentPath: filePath,
            Err:         errors.New("incomplete fragmented batch at segment tail"),
        }
    }
    // Non-last segment with incomplete fragments = middle corruption.
    // Plain error (no TailCorruptionError wrapping) — IsTailCorruption is false.
    return nextSequence, fmt.Errorf("wal: incomplete fragmented batch in non-last segment %s (hard corruption)", filePath)
}

return nextSequence, nil

A.3 RecoverFromSegments 传递 isLastSegment

文件：wal/recovery.go

nextSequence = segments[0].StartSequence
for i, segment := range segments {
    if segment.StartSequence != nextSequence {
        return nextSequence, fmt.Errorf("wal: segment start sequence %d does not match expected sequence %d", segment.StartSequence, nextSequence)
    }

    isLastSegment := i == len(segments)-1
    nextSequence, err = ReplaySegmentFile(segment.FilePath, nextSequence, isLastSegment, replayer)
    if err != nil {
        if IsTailCorruption(err) {
            return nextSequence, err
        }
        return nextSequence, fmt.Errorf("wal: replay segment: %w", err)
    }
}

A.4 Recover 使用 tce.SegmentPath（防御性）

文件：wal/recover.go，C5+H8 修复后的 truncation 块：

if len(segments) > 0 {
    // After C4 fix, TailCorruptionError is only returned for the last segment.
    // Use tce.SegmentPath as authoritative truncation target (defensive:
    // fall back to segments[last] if missing).
    corruptedPath := segments[len(segments)-1].FilePath
    var tce *TailCorruptionError
    if errors.As(err, &tce) && tce.SegmentPath != "" {
        corruptedPath = tce.SegmentPath
    }

    lastCompleteBatchEnd, findErr := findLastCompleteBatchEnd(corruptedPath)
    if findErr != nil {
        return nil, fmt.Errorf("wal: recover: find truncation offset: %w", findErr)
    }

    var emptyTrailing []string
    if err := truncateAndPersist(corruptedPath, lastCompleteBatchEnd, dir, emptyTrailing); err != nil {
        return nil, fmt.Errorf("wal: recover: persist tail truncation: %w", err)
    }
}

关键不变量：C4 fix 后 TailCorruptionError 只来自最后一段，所以 corruptedPath == segments[last].FilePath。tce.SegmentPath 是防御性显式表达，未来如果不变量破坏能立即发现。

关于 errors.As 用法：err 在 Recover 函数签名里是 outer 变量（nextSequence, err := RecoverFromSegments(...)），shadowing 后再 errors.As(err, &tce) 读的是当前 outer err。需要确认 Go 的语义在这里正确（应该是的，但加注释或测试覆盖）。

Phase B：测试（60-90 分钟）

B.1 ReplaySegmentFile 单元测试（isLastSegment 参数）

新增到 wal/recovery_test.go：

// Regression guards for C4: isLastSegment controls whether CollectingFragments
// at end is tail corruption (truncatable) or hard corruption.

func TestReplaySegmentFile_LastSegmentCollectingFragmentsIsTailCorruption(t *testing.T) {
    // Build segment with: [Batch A][First][Middle* no Last].
    // Call ReplaySegmentFile with isLastSegment=true.
    // Expect: TailCorruptionError (can be truncated).
    // Expect: tce.SegmentPath == filePath.
}

func TestReplaySegmentFile_NonLastSegmentCollectingFragmentsIsHardError(t *testing.T) {
    // Same segment content as above.
    // Call ReplaySegmentFile with isLastSegment=false.
    // Expect: hard error (NOT TailCorruptionError).
    // Expect: error mentions "non-last segment" or "hard corruption".
}

func TestReplaySegmentFile_LastSegmentParseErrorIsTailCorruption(t *testing.T) {
    // Build segment with: [Batch A][CRC-corrupted bytes].
    // Call with isLastSegment=true.
    // Expect: TailCorruptionError.
}

func TestReplaySegmentFile_NonLastSegmentParseErrorIsHardError(t *testing.T) {
    // Same segment content.
    // Call with isLastSegment=false.
    // Expect: hard error.
}

B.2 RecoverFromSegments / Recover 集成测试

新增到 wal/recover_test.go：

// Regression guard for C4: middle segment corruption must hard-fail Recover,
// NOT truncate the (valid) last segment. This is the key bug Oracle flagged:
// "wal/recover.go:61 总是对 segments[len(segments)-1] 调用 truncateSegment,
// 但 RecoverFromSegments 的 TailCorruptionError 可能来自非尾段".
func TestRecoverMiddleSegmentCorruptionHardFails(t *testing.T) {
    dir := t.TempDir()
    // Construct 3 segments with valid startSequences.
    // segment-0: [Batch seq 0-1]               (2 entries → next=2)
    // segment-1: [Batch seq 2-3] + [First][Middle no Last]  ← middle corruption
    //            (2 complete entries → next=4 if recovery reached end)
    // segment-2: [Batch seq 4-5]               (2 entries; valid; never reached)
    writeTestSegment(t, dir, 0, 0, [][]*WalEntry{
        {makePutEntry("k0", "v0"), makePutEntry("k1", "v1")},
    })

    seg1Path := writeTestSegment(t, dir, 1, 2, [][]*WalEntry{
        {makePutEntry("k2", "v2"), makePutEntry("k3", "v3")},
    })
    // Append First+Middle fragments to segment-1 (no Last).
    appendFileBytes(t, seg1Path, EncodePhysicalRecord(RecFirst, []byte("first-frag")))
    appendFileBytes(t, seg1Path, EncodePhysicalRecord(RecMiddle, []byte("middle-frag")))

    writeTestSegment(t, dir, 2, 4, [][]*WalEntry{
        {makePutEntry("k4", "v4"), makePutEntry("k5", "v5")},
    })

    seg2Path := filepath.Join(dir, "segment-2.wal")
    fiBefore, _ := os.Stat(seg2Path)

    _, err := Recover(dir, &mockReplayer{})
    if err == nil {
        t.Fatal("expected Recover to fail on middle segment corruption")
    }
    if IsTailCorruption(err) {
        t.Errorf("expected hard error (not tail corruption) for middle segment; got %v", err)
    }

    // CRITICAL: segment-2 must NOT be truncated (it's completely valid).
    fiAfter, _ := os.Stat(seg2Path)
    if fiAfter.Size() != fiBefore.Size() {
        t.Errorf("segment-2 was modified: before=%d after=%d (C4-2 regression)",
            fiBefore.Size(), fiAfter.Size())
    }
}

// Regression guard for C4: single-segment tail corruption still truncates
// correctly (existing behavior preserved).
func TestRecoverLastSegmentCorruptionTruncatesCorrectly(t *testing.T) {
    dir := t.TempDir()
    filePath := writeTestSegment(t, dir, 0, 0, [][]*WalEntry{
        {makePutEntry("k0", "v0")},
        {makePutEntry("k1", "v1")},
    })
    encA, _ := EncodeWalBatch(0, []*WalEntry{makePutEntry("k0", "v0")})
    endOfBatch1 := int64(WalFileHeaderSize) +
        int64(PhysicalRecordHeaderSize+len(encA)) +
        int64(PhysicalRecordHeaderSize+len(encA))

    // Append partial fragments to last (only) segment.
    appendFileBytes(t, filePath, EncodePhysicalRecord(RecFirst, []byte("first")))
    appendFileBytes(t, filePath, EncodePhysicalRecord(RecMiddle, []byte("middle")))

    result, err := Recover(dir, &mockReplayer{})
    if err != nil {
        t.Fatalf("Recover: %v", err)
    }
    if !result.Truncated {
        t.Fatal("Truncated = false, want true for last-segment corruption")
    }

    fi, _ := os.Stat(filePath)
    if fi.Size() != endOfBatch1 {
        t.Errorf("file size = %d, want %d", fi.Size(), endOfBatch1)
    }
}

// Regression guard for C4: multi-segment with last-segment corruption still
// works correctly (the legitimate tail-truncation case).
func TestRecoverMultiSegmentLastSegmentCorruptionTruncatesLast(t *testing.T) {
    dir := t.TempDir()
    // segment-0: valid batches
    // segment-1: valid batches + partial tail (corruption in LAST segment)
    writeTestSegment(t, dir, 0, 0, [][]*WalEntry{
        {makePutEntry("k0", "v0")},
    })
    seg1Path := writeTestSegment(t, dir, 1, 1, [][]*WalEntry{
        {makePutEntry("k1", "v1")},
        {makePutEntry("k2", "v2")},
    })
    // Append partial fragments to segment-1 (the LAST segment).
    appendFileBytes(t, seg1Path, EncodePhysicalRecord(RecFirst, []byte("first")))
    appendFileBytes(t, seg1Path, EncodePhysicalRecord(RecMiddle, []byte("middle")))

    // Compute expected truncation point: end of Batch B in segment-1.
    encA, _ := EncodeWalBatch(1, []*WalEntry{makePutEntry("k1", "v1")})
    encB, _ := EncodeWalBatch(2, []*WalEntry{makePutEntry("k2", "v2")})
    endOfBatch2 := int64(WalFileHeaderSize) +
        int64(PhysicalRecordHeaderSize+len(encA)) +
        int64(PhysicalRecordHeaderSize+len(encB))

    result, err := Recover(dir, &mockReplayer{})
    if err != nil {
        t.Fatalf("Recover: %v", err)
    }
    if !result.Truncated {
        t.Fatal("Truncated = false, want true")
    }

    fi, _ := os.Stat(seg1Path)
    if fi.Size() != endOfBatch2 {
        t.Errorf("segment-1 size = %d, want %d (end of Batch 2)", fi.Size(), endOfBatch2)
    }
}

B.3 现有测试更新

wal/recover_test.go 中的 TestRecoverIdempotentAfterTruncation 和 TestRecoverPartialFragmentTailIdempotent 都是单 segment 场景，行为不变。但它们间接调用了 Recover → RecoverFromSegments → ReplaySegmentFile，需要确认新签名 isLastSegment=true 在 RecoverFromSegments 内部正确传递。

wal/recovery_test.go 现有的直接调用 ReplaySegmentFile 的测试需要补 isLastSegment 参数。3 个调用点（Oracle bg_22edec11 提醒）：

grep -n "ReplaySegmentFile" wal/recovery_test.go wal/recover_test.go

• wal/recovery_test.go:124（TestReplaySegmentFile 之类）→ 加 true
• wal/recovery_test.go:150（另一个 ReplaySegmentFile 测试）→ 加 true
• wal/recover_test.go:121（TestRecoverWithTailCorruption 内部，验证截断后重放）→ 加 true

每个调用点加 true（默认按 last segment 处理，保留现有行为）。

B.4 补 middle-segment parser corruption 集成测试（Oracle 新增）

Oracle 修订（bg_22edec11）NICE-TO-HAVE：原计划只覆盖 middle-segment CollectingFragments，没覆盖 middle-segment parser corruption（CRC 失败）。需要加一个。

// Regression guard for C4 (parser corruption half): middle segment CRC
// corruption must hard-fail Recover, NOT truncate the (valid) last segment.
// This tests the path where ParseBlock returns TailCorruptionError and
// ReplaySegmentFile converts it to hard error for non-last segment.
func TestRecoverMiddleSegmentCRCCorruptionHardFails(t *testing.T) {
    dir := t.TempDir()
    // segment-0: [Batch seq 0-1]               (2 entries → next=2)
    // segment-1: [Batch seq 2-3] + [CRC-corrupted bytes]  ← middle parser corruption
    // segment-2: [Batch seq 4-5]               (valid; never reached)
    writeTestSegment(t, dir, 0, 0, [][]*WalEntry{
        {makePutEntry("k0", "v0"), makePutEntry("k1", "v1")},
    })

    seg1Path := writeTestSegment(t, dir, 1, 2, [][]*WalEntry{
        {makePutEntry("k2", "v2"), makePutEntry("k3", "v3")},
    })
    // Append CRC-corrupted bytes (will fail DecodePhysicalRecord's CRC check).
    // 10 bytes of 0xFF — looks like a record header but CRC won't match.
    appendFileBytes(t, seg1Path, []byte{0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF})

    writeTestSegment(t, dir, 2, 4, [][]*WalEntry{
        {makePutEntry("k4", "v4"), makePutEntry("k5", "v5")},
    })

    seg2Path := filepath.Join(dir, "segment-2.wal")
    fiBefore, _ := os.Stat(seg2Path)

    _, err := Recover(dir, &mockReplayer{})
    if err == nil {
        t.Fatal("expected Recover to fail on middle segment CRC corruption")
    }
    if IsTailCorruption(err) {
        t.Errorf("expected hard error (not tail corruption) for middle segment CRC; got %v", err)
    }

    // CRITICAL: segment-2 must NOT be truncated.
    fiAfter, _ := os.Stat(seg2Path)
    if fiAfter.Size() != fiBefore.Size() {
        t.Errorf("segment-2 was modified: before=%d after=%d (C4-2 regression)",
            fiBefore.Size(), fiAfter.Size())
    }
}

Phase C：验证（15 分钟）

# 1. 编译
go build ./...

# 2. 重点测试
go test ./wal -run 'TestReplaySegmentFile|TestRecoverMiddleSegment|TestRecoverLastSegment|TestRecoverMultiSegment|TestRecoverIdempotent|TestRecoverPartialFragmentTail' -count=1 -v

# 3. wal 包全量
go test ./wal/... -count=1

# 4. 全仓
go test ./... -count=1

# 5. race
go test -race ./... -count=1

# 6. vet
go vet ./...

Phase D：Commit message draft

fix: treat non-last segment corruption as hard error (C4)

Per design §3.2 line 704, tail corruption in a non-last WAL segment is
middle corruption, which must hard-fail recovery instead of being
silently truncated. The previous code in wal/recovery.go always returned
TailCorruptionError for CollectingFragments state or parse errors,
regardless of segment position. Recover then always truncated
segments[last], which could corrupt a valid last segment when the actual
corruption was in a middle segment.

Oracle bg_ef425776 flagged an additional failure mode: "wal/recover.go:61
总是对 segments[len(segments)-1] 调用 truncateSegment，但
RecoverFromSegments 的 TailCorruptionError 可能来自非尾段".

Changes:
- wal/record_parser.go: add SegmentPath field to TailCorruptionError for
  diagnostics and defensive truncation target identification.
- wal/recovery.go:
  - ReplaySegmentFile now takes isLastSegment bool parameter.
  - When parse error or CollectingFragments occurs in non-last segment,
    return hard error instead of TailCorruptionError.
  - When in last segment, return TailCorruptionError with SegmentPath set.
  - RecoverFromSegments passes isLastSegment based on iteration index.
- wal/recover.go:
  - Use tce.SegmentPath as authoritative truncation target (defensive
    fallback to segments[last] if missing). After C4 fix, TailCorruptionError
    is only returned for last segment, so this is always segments[last]
    in practice.

Tests:
- wal/recovery_test.go: 4 unit tests for ReplaySegmentFile covering
  last/non-last × CollectingFragments/parse-error matrix. Existing tests
  that directly call ReplaySegmentFile updated to pass isLastSegment=true.
- wal/recover_test.go: 3 integration tests covering middle-segment
  corruption (must hard-fail), last-segment corruption in single-segment
  WAL (must truncate), last-segment corruption in multi-segment WAL
  (must truncate only last segment).

Verified: each new test fails on pre-fix code (non-last corruption
silently truncated valid last segment) and passes after the fix. Full
suite green including go test -race ./... .

Audit context: docs/audit-3.2.md C4 (Oracle-verified bg_ef425776).

---

验收清单

• Phase A.1：TailCorruptionError.SegmentPath 字段存在
• Phase A.2：ReplaySegmentFile 签名包含 isLastSegment bool
• Phase A.2：非尾段 CollectingFragments 返回硬错误（不是 TailCorruptionError）
• Phase A.2：非尾段 parse error（TailCorruption from parser）转硬错误
• Phase A.3：RecoverFromSegments 传递正确 isLastSegment
• Phase A.4：Recover 用 tce.SegmentPath 做截断目标
• Phase B.1：4 个 ReplaySegmentFile 单元测试存在
• Phase B.2：3 个集成测试存在（middle/last-single/last-multi）
• Phase B.3：现有 ReplaySegmentFile 直接调用更新签名
• go test ./wal/... -count=1 全绿
• go test ./... -count=1 全绿
• go test -race ./... -count=1 全绿
• go vet ./... 无新增警告
• 单次 commit，message 引用 audit C4

---

不在本次范围内（后续 issue）

编号	为什么不放进来
C8	segment_manager.go:64-66 把字节偏移当 startSequence。本 patch 测试用 writeTestSegment 绕过，但生产环境多 segment recovery 仍受 C8 影响
C1	CRC 多项式 IEEE → crc32c，独立
C7	Put/Close 竞态，独立
H1-H7	其他 High，独立

---

修订记录

• v1（原始）：C4 修复方案初稿，送 Momus 审
• v1.0（Momus 审核 bg_423473e7）：[OKAY]，无 blocking。思考过程敏锐发现 TestRecoverMiddleSegmentCorruptionHardFails 中 segment-2 的 startSequence=6 与 segment-1 的实际 complete batch 数不匹配（应该是 4）。修正为 4
• v1.1（Oracle 修订 bg_22edec11）：
  • BLOCKING：非尾段 parser corruption 用 %v 而不是 %w —— IsTailCorruption 用 errors.As 遍历链，%w 包装后仍能匹配 *TailCorruptionError，会被当 tail corruption 处理
  • BLOCKING：漏了 wal/recover_test.go:121 的 ReplaySegmentFile 直接调用，需要更新签名。共 3 个直接调用点
  • NEW：加 TestRecoverMiddleSegmentCRCCorruptionHardFails 集成测试，覆盖 parser corruption 中段损坏（CRC fail）的端到端路径
  • MINOR：把 SegmentPath attachment 移到非尾段硬错误转换之前，让两条路径都能设置字段（虽然非尾段硬错误不需要，但保持顺序一致）