CI / ci (windows-latest) (push) Has been cancelled
Pure formatting pass - no semantic changes. Mostly long-line reflow,
import statement reordering (alphabetical), and trailing-newline fixes
that had been accumulating in the working tree.
Affected files:
* crates/bench/src/{data_gen,main,mmap_reader,pread_reader,report,runner}.rs
* crates/bench/src/suites/{concurrent,growth,jump,memory,render,rotation,startup}.rs
* crates/core/src/io/{file_reader,index_cache,read_cache}.rs
* crates/core/src/{types.rs, watcher/file_watcher.rs}
* line_index.rs: replace manual `x % BLOCK_SIZE == 0` with
`.is_multiple_of(BLOCK_SIZE)` (Rust 1.81+).
* line_index.rs: gate three pub(crate) accessor methods
(`sampled_offsets`, `total_lines`, `has_trailing_newline`) behind
`#[cfg(test)]`. They are only consumed by tests in file_reader.rs;
marking them test-only removes them from production builds entirely,
eliminating the dead_code warning without suppressing it.
* json.rs: drop redundant `Some(... .ok()?)` wrapper in
parse_json_object_with_duplicates. `.ok()` already returns Option,
so wrapping it in Some and unwrapping with ? was a no-op.
No behavior change. Unblocks `cargo clippy --workspace -- -D warnings`.
Four changes that compound to a 2-5x speedup on VHI rebuild for large
files, plus eliminating one full wasted VHI pass on initial open.
* spawn_indexer no longer computes VHI inline (progressive_reader.rs).
The inline VHI used a hardcoded width=80, was invalidated immediately
on Loading->Ready transition (gutter digit-boundary), and triggered
a second full rebuild. Now returns visual_height_index: None and lets
the UI trigger one rebuild with the correct post-layout width.
* spawn_visual_height_rebuild rewritten as a single linear pass over
mmap bytes using memchr::memchr (SIMD) to find newlines, replacing
the byte-by-byte BufRead::read_until scan. Cancel check throttled
to every 1MB instead of per-line.
* wrap_line_count(&str, width) -> usize added as a zero-allocation
counterpart to wrap_line_chars. Property tests cover ASCII, CJK,
emoji, combining marks, tabs, zero-width chars, control chars, and
real log lines across many widths - all match wrap_line_chars().len()
exactly. compute_text_visual_height now uses wrap_line_count, removing
10M Vec allocations on a 10M-line file.
* format_json_line returns Cow<'_, str> instead of String. Non-JSON
lines borrow the input (zero allocation); only pretty-printed JSON
objects allocate. Removes another 10M allocations when json_format=true.
Removes now-unused compute_visual_heights (only caller was the deleted
inline VHI path).
Replace deterministic .index.tmp path with per-save unique temp files
via tempfile::Builder. Eliminates race condition where background
indexer thread and TUI main thread could collide on the same temp path,
causing data truncation or corrupt cache writes.
Changes:
- Add write_cache_atomically() helper using tempfile::Builder
- Refactor save_with_hash() and save() to use the helper
- Extract encode_cache() to deduplicate serialization logic
- Move tempfile from [dev-dependencies] to [dependencies]
- Add 2 concurrent tests validating no corruption under parallel writes
Fixes#17
Background worker threads used blocking tx.send() on bounded channels.
If the consumer stopped draining, threads hung forever with no way to
reach the cancel check. Drop-issued cancellation was ineffective.
Changes:
- Progress messages: tx.try_send() (discard if full, never blocks loop)
- Terminal messages (Complete/Error): new send_cancelable<T>() helper
using crossbeam select! — sleeps efficiently until send succeeds or
cancel arrives
- Drop cancellation: tx.try_send() — Drop must never block
- spawn_visual_height_rebuild: same fix for its bounded(1) channel
- 5 new tests covering full-channel + cancel scenarios
- Add early return for len==0 (Ok(&[])) matching std::io semantics
- Add slot.len > 0 guard to cache hit predicate to prevent empty-slot
false matches
- Replace unchecked arithmetic with checked_add/saturating_add for
request_end, block_end, and post-read coverage check
- Fix misleading comment about get(file,0,0) behavior on miss path
- Strengthen clear() to fully reset block_offset and last_access
- Register read_cache module in io/mod.rs
- Add 4 regression tests: zero-len on fresh/populated cache,
zero-len at u64::MAX, overflow error on nonzero read at u64::MAX
When a file does not end with a newline, appending content extends the
last logical line's text and thus its visual height. The incremental
extend path in handle_file_appended only computed heights for newly
created logical lines, missing the old last line whose content changed.
Add VisualHeightIndex::replace_last_line_height() — an O(1) method that
rewrites the final prefix sum entry and total. Called before
extend_from_heights so the correct line is targeted.
Changes:
- progressive_reader.rs: add replace_last_line_height, pub with_params,
7 VHI unit tests
- app.rs: save old_reader_line_count before update, recompute last old
line height in extend path, 2 integration regression tests
Background threads (spawn_indexer, spawn_visual_height_rebuild) previously
held mmap during entire file scan, risking SIGBUS if file was truncated
externally. Now uses BufReader streaming scan with mmap created only
after scan completes, plus stat validation.
Changes:
- spawn_indexer: replace mmap scan with BufReader fill_buf/consume loop,
create mmap post-scan with fd stat validation
- spawn_visual_height_rebuild: replace mmap/FileReader with sequential
BufReader scan, discard results on line count mismatch
- FileReader::open/reload/update_for_append: add stat-after-mmap check
- LineIndex: make fields pub(crate) for direct construction from scan loop
- Add 3 regression tests for truncation scenarios
spawn_indexer builds LineIndex from mmap snapshot but IndexCache::save()
re-opened the file to compute the hash. If the file changed between those
two steps, the cached index would be stored under the wrong hash.
- Add IndexCache::save_with_hash() that computes hash from in-memory data
- Add compute_data_hash() public function (same algorithm as compute_file_hash)
- Update spawn_indexer, FileReader::save_cache, and test callers
Replace direct indexing with .get()? to return None instead of panicking
when sampled_offsets is shorter than expected (e.g. corrupt bincode cache).
Closes#3
Replace LineIndex::from_reader(BufReader) with LineIndex::from_bytes(&mmap)
in both open() and reload(), ensuring the line index is always built from
the same mmap snapshot rather than a separate read through the file descriptor.
This closes the race window where an external file modification between
mmap() and from_reader() could cause line_index offsets to disagree with
the mmap data, leading to get_line() returning wrong content or panicking.
Closes#2
File truncation/rotation during mmap lifetime caused SIGBUS crash
because update_for_append() ignored new_size < old_size, leaving
a stale mapping that would fault on access.
Introduce AppendStatus enum (Unchanged/Appended/Reloaded) so the
caller can distinguish shrink events from normal appends. On shrink,
reload() rebuilds the mmap and line index. The TUI layer clamps
cursor and invalidates viewport cache on Reloaded, matching the
existing handle_file_truncated() behavior.
Fixes#1
Add SCAN_AHEAD_LIMIT (10000 lines) to get_line() in Sampling state. Without this, jumping to end-of-file (G) during progressive loading would scan the entire file byte-by-byte on the main thread, blocking the UI and consuming excessive memory. Lines beyond the scanned region + limit now return None, which the TUI renders as empty.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Add ProgressiveFileReader state machine (Sampling/Ready/Error) with scanned_newlines cache for O(1) line lookups. Implement spawn_indexer() for background mmap-based indexing with cancellation and progress reporting. Add VisualHeightIndex with prefix-sum + binary search for O(log N) visual height queries. Register all new io submodules and extend FileReader with reload(), save_cache(), and accessor methods.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Move wrap_line_chars and format_json_line from app.rs to core/io/wrap.rs with MAX_WRAP_INPUT_LEN guard. Add serde derives, pub getters, and extend_from_bytes() to LineIndex for incremental index building.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Implement head/tail 64KB sampling to estimate total line count without scanning the entire file. Also provide read_first_lines() for reading the first N lines for immediate display during progressive loading.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Add xxhash-rust and bincode workspace dependencies for fast hashing and serialization. Implement cache_util for cache directory/path resolution with versioning, and IndexCache for saving/loading line indices to disk with file-hash validation.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Replace Vec<u8> with memmap2::Mmap for file content. Build line index via BufReader streaming (from_reader) instead of scanning mmap'd data, keeping RSS at ~3MB for 5GB files instead of ~5GB.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Rewrite LineIndex to use sparse sampling (every 256 lines) instead of per-line offsets. Add from_reader() for low-RSS streaming index construction via BufReader fill_buf()/consume(), reducing 5GB file RSS from 5122MB to 3.4MB.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
dailz