Reuse the existing count_existing_lines() (reader.lines().count())
instead of a manual read_until loop, eliminating duplicate line-counting
logic in data_gen.rs.
Closes#40
- variants: use direct tuple comparison instead of format! string, add sort() after dedup
- Memory section: sort rows by (test_name, backend, variant) before output
- Extra Metrics section: sort rows by (test_name, variant_label) before output
- add [lib] target to Cargo.toml to enable unit tests
- add regression test: same data in different input order produces identical report
Issue #38: warn_reset_hwm() silently swallowed non-permission I/O errors
from /proc/self/clear_refs (e.g. missing /proc, read-only procfs, kernel
incompatibility). This left users unaware that VmHWM reset failed and
memory peak data could be contaminated across suites.
Changes:
- runner.rs: all errors now produce a warning with specific failure reason;
PermissionDenied retains 'try running as root' hint; AtomicBool warn-once
prevents duplicate output across 7 suite runs
- main.rs: preflight check now uses warn_reset_hwm() instead of the vague
can_reset_vm_hwm(), sharing the same warn-once mechanism
- metrics.rs: remove dead can_reset_vm_hwm() (no callers remaining)
- tests: add hwm_warned_flag_prevents_reentry and warn_reset_hwm_does_not_panic
Introduce Suite enum (runner.rs) replacing stringly-typed suite matching.
BenchConfig.suites is now Option<Vec<Suite>>, making invalid states
unrepresentable. Unknown suite names produce a clear error listing all
valid values.
Fixes: #37
The reader's line_index and file_size were frozen at open time.
After current_line exceeded the initial 150K lines, get_line_impl
returned None for all subsequent reads. With the background thread
appending ~10K lines/sec, ~40% of measured frame latencies were
actually the cost of a None return, not real I/O.
- Add PreadReaderCore::refresh_index(&mut self): seek to start,
rebuild LineIndex, update file_size, invalidate read cache
- Add PreadReaderPlain::refresh_index forwarding method
- Add ReadCache::invalidate to force cache miss after reindex
- Rewrite bench_scroll_during_append: time-based refresh (250ms),
only record latencies for successful reads, assert max_line > initial
- Add regression tests for refresh_index with appended lines
Add BufWriter::with_capacity(64KB) to generate_test_file,
generate_growable_file, and append_lines in data_gen.rs.
Previously each writeln! triggered an individual write syscall,
making 5GB/74M-line benchmark data generation extremely slow.
BufWriter batches writes into 64KB chunks, reducing syscalls
by ~1000x.
Explicit flush()? + drop before subsequent reads ensures data
visibility and propagates flush errors (BufWriter::drop swallows
them).
Closes#35
Replace `static mut OLD_SIGBUS_HANDLER` with AtomicU8 + AtomicPtr to
remove data race UB when concurrent benchmarks call open() from multiple
threads.
Key changes:
- Use `Once::call_once` to guarantee single handler installation
- Publish old handler to atomics BEFORE installing new handler (closes
the handler-active-but-state-unpublished race window)
- Read atomics with Acquire in signal handler (async-signal-safe)
- Align si_addr to page boundary before mmap(MAP_FIXED)
- Add concurrent test: 8 threads open all 5 variants simultaneously
Loading state silently dropped FileEvent::Appended/Truncated via _ => {}.
After Loading→Ready transition the FileReader was based on a stale snapshot.
- Add reload_after_loading flag to defer reload until Ready state
- Extract reload_ready_reader() from handle_file_truncated
- Explicit 3-branch match: Ready handles, Loading sets flag, rest ignores
- Clear flag on IndexerMessage::Error to prevent stale dirty bit
- 4 regression tests covering append/truncate/collapse/error paths
The previous code called std::process::exit(1) on file load failure,
bypassing all terminal restoration (disable_raw_mode, LeaveAlternateScreen,
show_cursor). This left the user's shell in a broken state with no echo
and no visible cursor.
Introduce TerminalGuard with Drop-based cleanup that fires on every exit
path: normal return, ? error propagation, and panic unwind. The guard
also handles partial initialization rollback (e.g. raw mode enabled but
alternate screen fails). Remove all process::exit calls from the guarded
scope.
Replace tx.send() with tx.try_send() in file watcher notify callback
to avoid blocking the notify thread when the bounded channel (cap 100)
is full. Events are silently dropped instead of blocking, preventing
shutdown delays and event loss during consumer stalls.
Closes#7
Background threads (spawn_indexer, spawn_visual_height_rebuild) previously
held mmap during entire file scan, risking SIGBUS if file was truncated
externally. Now uses BufReader streaming scan with mmap created only
after scan completes, plus stat validation.
Changes:
- spawn_indexer: replace mmap scan with BufReader fill_buf/consume loop,
create mmap post-scan with fd stat validation
- spawn_visual_height_rebuild: replace mmap/FileReader with sequential
BufReader scan, discard results on line count mismatch
- FileReader::open/reload/update_for_append: add stat-after-mmap check
- LineIndex: make fields pub(crate) for direct construction from scan loop
- Add 3 regression tests for truncation scenarios
spawn_indexer builds LineIndex from mmap snapshot but IndexCache::save()
re-opened the file to compute the hash. If the file changed between those
two steps, the cached index would be stored under the wrong hash.
- Add IndexCache::save_with_hash() that computes hash from in-memory data
- Add compute_data_hash() public function (same algorithm as compute_file_hash)
- Update spawn_indexer, FileReader::save_cache, and test callers
Replace direct indexing with .get()? to return None instead of panicking
when sampled_offsets is shorter than expected (e.g. corrupt bincode cache).
Closes#3
Replace LineIndex::from_reader(BufReader) with LineIndex::from_bytes(&mmap)
in both open() and reload(), ensuring the line index is always built from
the same mmap snapshot rather than a separate read through the file descriptor.
This closes the race window where an external file modification between
mmap() and from_reader() could cause line_index offsets to disagree with
the mmap data, leading to get_line() returning wrong content or panicking.
Closes#2
File truncation/rotation during mmap lifetime caused SIGBUS crash
because update_for_append() ignored new_size < old_size, leaving
a stale mapping that would fault on access.
Introduce AppendStatus enum (Unchanged/Appended/Reloaded) so the
caller can distinguish shrink events from normal appends. On shrink,
reload() rebuilds the mmap and line index. The TUI layer clamps
cursor and invalidates viewport cache on Reloaded, matching the
existing handle_file_truncated() behavior.
Fixes#1
During Loading state (before VHI is built), j/k used to jump by logical
line, visually skipping multiple wrapped rows. Now uses v_sub_offset to
track position within a wrapped line, enabling smooth 1-visual-row scroll.
- Add v_sub_offset field to App for sub-line visual position tracking
- scroll_down/up_line else branch: advance v_sub_offset, wrap to next line
- ensure_viewport_cache Loading path: pass v_sub_offset as offset_in_line
- ensure_cursor_visible: skip during Loading (scroll functions manage it)
- Reset v_sub_offset on Loading→Ready, scroll_to_top, scroll_to_bottom
- Add 3 tests for Loading-state sub-offset scrolling behavior
When a JSON line wraps to many visual rows (e.g. 73 rows), pressing j
would skip the entire logical line, making wrapped content unreadable.
Now j/k scroll by 1 visual row when a VisualHeightIndex is available,
with cursor tracking the viewport center. Falls back to logical-line
scroll in Loading/no-index modes or when all content fits the viewport.
Adds 4 tests: visual scroll down, visual scroll up, small-file fallback,
j/k roundtrip.
Add SCAN_AHEAD_LIMIT (10000 lines) to get_line() in Sampling state. Without this, jumping to end-of-file (G) during progressive loading would scan the entire file byte-by-byte on the main thread, blocking the UI and consuming excessive memory. Lines beyond the scanned region + limit now return None, which the TUI renders as empty.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
total_lines() was returning sampled_line_count() (only ~300 lines from the initial 64KB scan) during the Loading state, capping the scroll range. Use estimated_lines instead so the user can scroll to any position while indexing runs in the background. get_line() already supports incremental forward scanning on demand.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Complete FileWatcher implementation using notify 8.x crate with get_inode() for cross-platform file identity. Support append detection for incremental index updates and truncate detection for full reloads.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Replace synchronous file loading with AppLoadingState state machine (Empty/Loading/Ready/Error) for instant interactivity. Add ViewportCache for on-demand viewport computation, replacing global wrap/level caches. Integrate background indexer polling and file watcher events into the TUI event loop. Add loading UI with progress percentage, estimated line numbers with ~ prefix, and error state display. Eliminate all O(N) linear scans using VisualHeightIndex binary search.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Add ProgressiveFileReader state machine (Sampling/Ready/Error) with scanned_newlines cache for O(1) line lookups. Implement spawn_indexer() for background mmap-based indexing with cancellation and progress reporting. Add VisualHeightIndex with prefix-sum + binary search for O(log N) visual height queries. Register all new io submodules and extend FileReader with reload(), save_cache(), and accessor methods.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Move wrap_line_chars and format_json_line from app.rs to core/io/wrap.rs with MAX_WRAP_INPUT_LEN guard. Add serde derives, pub getters, and extend_from_bytes() to LineIndex for incremental index building.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Implement head/tail 64KB sampling to estimate total line count without scanning the entire file. Also provide read_first_lines() for reading the first N lines for immediate display during progressive loading.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Add xxhash-rust and bincode workspace dependencies for fast hashing and serialization. Implement cache_util for cache directory/path resolution with versioning, and IndexCache for saving/loading line indices to disk with file-hash validation.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Replace Vec<u8> with memmap2::Mmap for file content. Build line index via BufReader streaming (from_reader) instead of scanning mmap'd data, keeping RSS at ~3MB for 5GB files instead of ~5GB.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Rewrite LineIndex to use sparse sampling (every 256 lines) instead of per-line offsets. Add from_reader() for low-RSS streaming index construction via BufReader fill_buf()/consume(), reducing 5GB file RSS from 5122MB to 3.4MB.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Replace Line::styled with Span-based rendering: gutter (DarkGray fg) + content (level-based fg color). Add centered settings popup (S key) with j/k navigation, arrow key color cycling, number key shortcuts, Enter to save, Esc to cancel.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Add AppMode enum (Normal/Settings), level_cache field populated in recompute_wrap_cache via detect_level(), color_config loaded from TOML in main.rs. Refactor handle_key for mode dispatch with S key to enter settings.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Add level detection that tries JSON parsing first (trusts level field), then falls back to keyword scanning with word-boundary checks. Supports SEVERE, FATAL, ERROR, WARN, INFO, DEBUG, TRACE and their abbreviations.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Replace empty AppConfig stub with ColorConfig struct that stores level-to-color mappings as strings. Supports loading/saving from XDG config directory via TOML, with graceful degradation on missing/invalid files.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Press Tab to switch JSON log lines between raw single-line and indented
multi-line format (2-space indent, like jq output). Only JSON Object lines
are affected; plain text lines stay unchanged.
- Add json_format bool field to App, toggled by Tab
- Add format_json_line() standalone function with serde_json::to_string_pretty
- Integrate into recompute_wrap_cache with \n-split sub-line wrapping
- Center viewport on cursor after cache rebuild to prevent jump on toggle
- Add 12 unit tests covering format, toggle, and edge cases
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
