dailzandSisyphus
0910d4e85f
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
109 lines
4.5 KiB
Markdown
109 lines
4.5 KiB
Markdown
# logViewer
|
|
|
|
**[中文文档](README-zh.md)**
|
|
|
|
A high-performance terminal log file viewer built in Rust, designed to handle multi-gigabyte files with minimal memory overhead.
|
|
|
|
Uses memory-mapped I/O with a sparse line index and progressive background loading to open files instantly — even 5GB+ logs scroll smoothly from the first keystroke.
|
|
|
|
## Features
|
|
|
|
- **Instant file open** — mmap-backed reader with background progressive indexing; start scrolling before indexing finishes
|
|
- **Handles huge files** — sparse line index (1 entry per 256 lines) keeps memory usage at ~8MB even for 5GB files
|
|
- **Live file tracking** — watches for appends, truncations, and log rotation via `notify`
|
|
- **JSON log support** — NDJSON parsing with BOM handling, duplicate key detection, and toggleable pretty-printing
|
|
- **Unicode-aware wrapping** — correct line wrapping for CJK, emoji, and tabs
|
|
- **Persistent index cache** — line indexes saved to disk with xxh3 content hashing; re-opens are near-instant
|
|
- **Vim-like keybindings** — familiar navigation for terminal users
|
|
- **Customizable colors** — per-log-level colors via TOML config, adjustable from within the TUI
|
|
|
|
## Quick Start
|
|
|
|
```bash
|
|
# Build and run
|
|
cargo run -p log-viewer-tui -- path/to/logfile.log
|
|
|
|
# Or build first
|
|
cargo build --release -p log-viewer-tui
|
|
./target/release/log-viewer-tui path/to/logfile.log
|
|
```
|
|
|
|
Requires Rust 1.92+ (see `rust-toolchain.toml`).
|
|
|
|
## Keybindings
|
|
|
|
| Key | Action |
|
|
|-----|--------|
|
|
| `j` / `↓` | Scroll down one line |
|
|
| `k` / `↑` | Scroll up one line |
|
|
| `Ctrl+d` | Scroll down half page |
|
|
| `Ctrl+u` | Scroll up half page |
|
|
| `Ctrl+f` / `PgDn` | Scroll down full page |
|
|
| `Ctrl+b` / `PgUp` | Scroll up full page |
|
|
| `G` / `End` | Jump to end of file |
|
|
| `gg` / `Home` | Jump to start of file |
|
|
| `Tab` | Toggle JSON pretty-printing |
|
|
| `S` | Open color settings |
|
|
| `q` / `Esc` | Quit |
|
|
|
|
Settings panel: use `j`/`k` to select a log level, `←`/`→` to cycle colors, `Enter` to save, `Esc` to cancel.
|
|
|
|
## Workspace Structure
|
|
|
|
```
|
|
crates/core — Shared library: I/O, parsing, types, config, file watching
|
|
crates/tui — Terminal UI (ratatui + crossterm)
|
|
crates/gui — GUI (egui + eframe) — placeholder, not yet functional
|
|
crates/bench — mmap vs pread benchmark harness
|
|
```
|
|
|
|
## Development
|
|
|
|
```bash
|
|
# Check, test, lint
|
|
cargo check --workspace
|
|
cargo test --workspace
|
|
cargo fmt --check --all
|
|
cargo clippy --workspace -- -D warnings
|
|
|
|
# Run benchmarks (requires a pre-generated ~5GB test file)
|
|
mkdir -p /tmp/test-logviewer
|
|
dd if=/dev/urandom of=/tmp/test-logviewer/extreme.log bs=1M count=5000
|
|
cargo run -p log-viewer-bench
|
|
cargo run -p log-viewer-bench -- --quick --suites startup,render --output results.md
|
|
|
|
# Test a single crate
|
|
cargo test -p log-viewer-core
|
|
cargo test -p log-viewer-tui
|
|
```
|
|
|
|
### CI
|
|
|
|
The CI gate runs on ubuntu-latest and windows-latest:
|
|
1. `cargo fmt --check --all`
|
|
2. `cargo check --workspace`
|
|
3. `cargo test --workspace`
|
|
4. `cargo clippy --workspace -- -D warnings`
|
|
|
|
No custom rustfmt or clippy config — uses toolchain defaults.
|
|
|
|
## Architecture
|
|
|
|
The core library (`log-viewer-core`) owns all I/O, parsing, and data types. Key design decisions:
|
|
|
|
- **Sparse line index** — samples every 256 lines using memchr SIMD acceleration. A 1M-line file produces a ~32KB index instead of ~8MB.
|
|
- **Progressive loading** — `ProgressiveFileReader` starts with a quick head+tail sample for instant line estimates, then builds the full index in a background thread via crossbeam-channel.
|
|
- **mmap with TOCTOU mitigation** — post-mmap stat check to detect file changes. `read_cache` module exists as a future pread-based alternative to eliminate SIGBUS risk entirely.
|
|
- **Persistent cache** — line indexes are serialized to disk with xxh3 content hashing for invalidation. Atomic writes via temp files.
|
|
- **Visual height index** — prefix-sum over wrapped-line heights, enabling O(log n) scroll-to-line mapping for long wrapped lines.
|
|
|
|
Planned but not yet built: filtering, bookmarks, sessions, search engine (stub modules exist).
|
|
|
|
## Benchmarks
|
|
|
|
Custom harness comparing mmap vs pread backends across 7 suites (startup, render, jump, memory, growth, rotation, concurrent). Uses wall-clock timing with `/proc/self/` RSS and page fault metrics. Results are written as markdown tables to `benchmark-report.md`.
|
|
|
|
The harness expects `/tmp/test-logviewer/extreme.log` to already exist. It does not generate the primary 5GB test file automatically; create it first with the `dd` command shown above.
|
|
|
|
The benchmark binary includes ~75 unit tests for the reader backends and metrics collection.
|