logViewer/AGENTS.md

logViewer

High-performance log file viewer. Rust workspace with 4 crates, edition 2024, MSRV 1.92 (pinned in rust-toolchain.toml).

Workspace Layout

crates/core  — log-viewer-core   (library) — I/O, parsing, types, config, file watching
crates/tui   — log-viewer-tui    (binary)  — terminal UI (ratatui + crossterm)
crates/gui   — log-viewer-gui    (binary)  — GUI (egui + eframe) — STUB, not yet functional
crates/bench — log-viewer-bench  (binary)  — mmap vs pread benchmark harness

core owns all I/O, parsing, data types, and file watching. TUI/GUI consume core's public API. GUI is a placeholder with no core integration yet.

default-members = ["crates/core"] — bare cargo run / cargo test target core, not the TUI. Always pass -p log-viewer-tui (or --workspace) explicitly.

Commands

# Run the TUI (primary interface)
cargo run -p log-viewer-tui -- <log-file>

# Build/check/test/lint the whole workspace
cargo check --workspace
cargo test --workspace
cargo fmt --check --all
cargo clippy --workspace -- -D warnings

# Single crate
cargo test -p log-viewer-core
cargo test -p log-viewer-tui

# Benchmarks (test file is NOT auto-generated — see "Benchmarks" section below)
cargo run -p log-viewer-bench
cargo run -p log-viewer-bench -- --quick --suites startup,render --output results.md

CI Gate (must pass before merge)

Runs on ubuntu-latest + windows-latest (.github/workflows/ci.yml):

1. cargo fmt --check --all
2. cargo check --workspace
3. cargo test --workspace
4. cargo clippy --workspace -- -D warnings

Toolchain installed via dtolnay/rust-toolchain@stable reading rust-toolchain.toml (channel 1.92). No rustfmt.toml or clippy.toml — uses toolchain defaults.

Core Architecture

All ten modules are declared pub in lib.rs and re-exported as log_viewer_core::<module>.

Key Modules

• types — LogLevel, LogEntry, SearchQuery, SearchFilter, Bookmark, FileSession, DuplicateKey, SearchResult
• error — CoreError (12 variants via thiserror: Io, Parse, Search, Index, Config, TomlSerialize, Encoding, Watch, Mmap, Cache, FileNotFound, Other) + Result<T> alias. From impls for io::Error / serde_json::Error / toml::de::Error / toml::ser::Error / notify::Error mean ? works freely.
• io::file_reader — mmap-backed FileReader with TOCTOU mitigation (post-mmap stat check), append/reload detection
• io::progressive_reader — ProgressiveFileReader state machine (ReaderState::Sampling { .. } | Ready { .. }), background indexer via crossbeam-channel, VisualHeightIndex for wrapped-line scroll
• io::line_index — Sparse index (every BLOCK_SIZE = 256 lines, memchr SIMD). Serializable to disk.
• io::index_cache — Persistent cache with xxh3 content hash, atomic writes via temp files
• io::line_sampler / io::cache_util — shared helpers used by the index path
• io::wrap — Unicode-aware line wrapping (CJK/emoji/tab), JSON pretty-printing; enforces MAX_WRAP_INPUT_LEN
• io::read_cache — LRU read cache (not yet integrated into the live reader; future pread backend)
• parser::json — NDJSON parser with BOM handling, duplicate key detection
• parser::level — JSON-first level detection (detect_level); falls back to bounded keyword scan with word-boundary check on non-JSON lines
• watcher::file_watcher — File event watcher (notify + crossbeam) with append/truncate/rotation detection
• config — ColorConfig TOML load/save with per-field serde defaults

Stubs (single-line TODOs — planned, not yet built)

• filter::Filter
• bookmark::BookmarkManager
• session::SessionManager
• search::engine::SearchEngine

TUI Architecture

• main.rs — clap CLI (files: Vec<String>), TerminalGuard RAII for raw mode + alternate screen. CRITICAL: Drop does not run on std::process::exit or panic = "abort" — never call process::exit inside the guarded scope; return Err and use ? instead. Event loop: poll indexer → poll watcher → draw → poll keys, 100ms timeout.
• app.rs — App struct (~4000 lines including ~135 inline tests, test mod starts at #[cfg(test)] mod tests). AppLoadingState: Empty | Loading { reader, estimated_lines, progress_percent } | Ready { reader } | Error(String). AppMode: Normal | Settings. Viewport cache for scroll. All key handling.
• ui.rs — ratatui rendering: title bar, content area, status bar, settings popup
• color.rs — LogLevel → ratatui Color via ColorConfig

TUI Keybindings

j/k scroll, Ctrl+d/u half-page, Ctrl+f/b full-page, G/gg jump end/top, Tab toggle JSON formatting, S settings panel, q/Esc quit. Settings: j/k select level, ←/→ cycle colors, Enter save, Esc cancel.

Testing

All tests are inline (#[cfg(test)] mod tests blocks). No tests/ directories, no integration tests, no async tests.

• ~450 tests total: core 251, tui 136, bench 63, gui 0
• Temp-file helpers vary per module — check the local tests mod before assuming an API:
  • make_temp_file(content) -> PathBuf — tui/app.rs, tui/ui.rs
  • make_test_file(lines) -> NamedTempFile — core/io/index_cache.rs
  • make_file(data) -> NamedTempFile — core/io/read_cache.rs
  • struct TempFile { ... } — core/io/line_sampler.rs
• insta is a declared dev-dependency but currently unused (no insta:: calls anywhere in the workspace)
• TUI render tests use ratatui TestBackend (see tui/ui.rs)
• Bench crate has its own unit tests for mmap/pread backends, metrics, and data_gen helpers

Benchmarks

Custom harness (not criterion). Wall-clock + /proc/self/ RSS/page fault metrics. 7 suites: startup, render, jump, memory, growth, rotation, concurrent. Tests mmap (plain/sequential/random/populate/phase_aware) vs pread (plain/random/sequential) backends. Output: markdown tables in benchmark-report.md.

The harness does NOT auto-generate the test file. main.rs requires /tmp/test-logviewer/extreme.log to already exist; if missing it prints the dd command and exits. Generate it manually first:

mkdir -p /tmp/test-logviewer
dd if=/dev/urandom of=/tmp/test-logviewer/extreme.log bs=1M count=5000   # ~5GB

(crates/bench/src/data_gen.rs has generate_test_file / ensure_test_file helpers and is used by the growth and rotation suites for per-run growable files, but it is not wired into main.rs for the primary test file.)

Note: log-viewer-bench depends directly on memmap2 / nix / libc / memchr — it does not depend on log-viewer-core, so it can be built standalone with cargo build -p log-viewer-bench.