Root cause (proven via live lag-triage capture + deterministic PTY repro): fish latches feature flags from its startup env before config.fish runs, so the existing no-query-term settings never applied to zellij-spawned shells. With query-term on, fish queries the terminal after every command; one reply zellij fails to relay permanently degrades that fish process's reader. Subshells were immune (inherited the exported var), which is why the lag always 'disappeared' when tested in a new shell. Also adds the lag-triage/unlag/term-probe diagnostic toolkit used to find this, and upstream_repro.py for filing the fish-shell bug.
15 KiB
Changelog
2026-08-29 (root cause found and fixed)
- Root-caused and fixed the recurring post-TUI typing lag (fish + Zellij + Ghostty) using a
lag-triagecapture from a live lagging shell plus a deterministic PTY reproduction (presets/programs/lag-triage/upstream_repro.py):- Root cause chain: (1) fish latches feature flags from its startup environment, before
config.fishruns — so the existingset -gx fish_features no-query-terminshellInitnever applied to the shell that set it, only to its children. (2) Zellij spawns pane shells viadefault_shellwith nofish_featuresin the environment, so every pane's fish latchedquery-termon (the fish 4.8.1 default; the triage log from the lagging shell confirmedquery-term onwhile$fish_featureswas correctly set tono-query-term). (3) With query-term on, fish sends OSC 11 + CPR (\e[6n) + DA1 (\e[0c) after every command and waits for replies relayed by Zellij. (4) Reproduced on fish 4.8.1: if the terminal fails to reply during just one such cycle — answering everything before and after — that fish process's interactive reader is permanently degraded (keystroke echo >3s, never recovers; ~35ms before). In production Zellij drops/mangles a relay during TUI teardown or heavy output (cf. zellij-org/zellij#5158), e.g. afternh home switch, nvim, jjui, yazi. - Why every previous observation finally makes sense: subshells and
exec fishwere never "resetting" anything — they inherited the exportedfish_features=no-query-termfrom config.fish, latched query-term off at startup, and were therefore immune. The parent zellij-spawned shell never had the variable at startup and stayed vulnerable. Raw keystroke capture in the lagging pane showed instant plain bytes (input path fine) and no stuck terminal modes — the damage was inside the fish process, exactly as the repro shows. - Fix:
zellij.nixnow spawns panes through afish-no-query-termwrapper (export fish_features=no-query-term; exec fish), so the feature is latched off in every pane shell. Verified: interactive fish through the built wrapper with the real config reportsquery-term off; the PTY repro withno-query-termin the environment shows ~35ms echo through all failure phases. - Correction to the earlier 2026-08-29 entry:
query-termdoes not default to off in fish 4.8.1 — it defaults on; it only appeared off in non-interactive checks because the user config'sset -gxtakes effect forfish -c(no reader latch) but not for interactive shells. - Upstream: fish-shell should bound the reader's wait for query replies instead of degrading permanently (repro script kept at
presets/programs/lag-triage/upstream_repro.pyfor filing); Zellij's reply relaying is the trigger (zellij-org/zellij#5158). lag-triagenow checksstatus featuresand calls outquery-term onas the known root cause, and warns that itsread-prompt typing tests may not exhibit main-commandline lag (which produced a false "fixed by stage A" in the first capture).
- Root cause chain: (1) fish latches feature flags from its startup environment, before
2026-08-29 (later)
- Added a diagnostic toolkit (
lag-triage/unlagfish functions +term-probebinary,presets/programs/lag-triage/) for the still-recurring post-TUI typing lag in fish + Zellij + Ghostty, instead of another blind fix. Findings that motivated it:- All three prior fixes were either no-ops or insufficient:
fish_features = no-query-termis a no-op becausequery-termalready defaults to off in fish 4.8.1 (verified withstatus features); disabling Ghostty's fish integration inside Zellij and settingsupport_kitty_keyboard_protocol = falsedid not stop recurrence. - PTY captures of fish 4.8.1 (
TERM=xterm-256color, with and without$ZELLIJ) show fish never writes Kitty keyboard sequences to the wire — it uses modifyOtherKeys (\e[>4;1m), application keypad (\e=), bracketed paste (?2004), and color-theme reporting (?2031), enabling them at every prompt and disabling them before every external command. Crucially, a fresh subshell's startup bytes are identical to the parent's post-command re-enable bytes, so "a subshell fixes the lag" cannot be explained by a simple terminal-state reset — leaving two competing hypotheses that only live capture can separate: (1) fish-internal reader state poisoned by stray/partial escape bytes (e.g. leaked from a closing floating pane), cleared only by a new fish process; (2) Zellij/Ghostty-level stuck state (Zellij 0.45'sStdinAnsiParseris already a proven source of input delays — see the Alt-Shift-P fix below). - Also note: the floating-pane TUIs (jjui via Alt-Shift-J, yazi via Alt-Shift-Y, scrollback editor) run in their own panes and never pass through the shell's fish process at all, while
nvimruns inside the shell pane — the triage log records which path preceded the lag. - Next occurrence: run
lag-triagein the lagging shell BEFORE starting a new shell. It snapshots the environment, queries pane terminal state (kitty flags, modifyOtherKeys, DEC modes, DA1 round-trip latency), captures raw keystroke bytes+timing bypassing fish, then applies staged resets (kitty pop/clear, modifyOtherKeys off, keypad/cursor, mouse/focus/sync, altscreen, stty, DECSTR) — the stage that cures it names the stuck layer. Logs to~/.local/state/lag-triage/for an upstream issue.unlagis the one-shot convenience version (ifunlagnever helps butexec fishdoes, the bug is fish-internal).
- All three prior fixes were either no-ops or insufficient:
2026-08-29
- Fixed 1.5-second latency when pressing
Alt-Shift-Pto triggerzellij-sessionin Zellij 0.45.0 + Ghostty:- Root Cause: Zellij 0.45.0 introduced
StdinAnsiParser(zellij-client/src/stdin_ansi_parser.rs) usingtermwiz::InputParserto parse ANSI control strings (OSCs, CSIs, DCSs) arriving on stdin. When pressingAlt-Shift-P(Option-Shift-P) withsupport_kitty_keyboard_protocol = false, Ghostty sent\x1bP(ESC+ uppercaseP). In ECMA-48 / VT100 standards,ESC Pis the 7-bit ASCII representation ofDCS(Device Control String).StdinAnsiParserbuffered\x1bPwaiting for a DCS string payload and string terminator (ST/\x1b\), hitting a ~1.5-second escape timeout before flushing\x1bPas residue to the keyboard handler. - Fix: Added
alt+shift+p=text:\x1b[112;4uandsuper+shift+p=text:\x1b[112;4uinghostty.nixto send the explicit CSI-u sequence forAlt+Shift+p('p'with modifier 4 =ALT | SHIFT).StdinAnsiParserimmediately recognizes\x1b[112;4uas non-DCS input and passes it straight to the keyboard handler with 0ms latency.
- Root Cause: Zellij 0.45.0 introduced
2026-08-26
-
Fixed macOS shortcuts (
Cmd+T,Ctrl+Tab,Cmd+Shift+],Cmd+Shift+[,Cmd+K,Cmd+Shift+E) in Zellij + Ghostty after disabling the Kitty keyboard protocol:- Mapped Ghostty keybindings (
super+t,super+shift+],super+shift+[,ctrl+tab,ctrl+shift+tab,super+k,super+shift+e) to send standardAlt(ESC-prefix) text sequences (\x1bt,\x1b},\x1b{,\x1bK,\x1bE). - Added matching
Altkeybindings (Alt t,Alt ],Alt },Alt [,Alt {,Alt Shift k,Alt Shift e) inzellij.nixfor tab creation, tab navigation, scroll mode, and scrollback editing. Symbols like]and}are parsed by Zellij's termwiz input engine as distinct character codes ('}'vs']'), so binding bothAlt }andAlt Shift ]ensures\x1b}triggers tab navigation correctly. - Keeps Kitty keyboard protocol disabled in Zellij (
support_kitty_keyboard_protocol = false) so no CSI-u flags leak into Fish shell, guaranteeing zero post-TUI typing lag while restoring all shortcuts.
- Mapped Ghostty keybindings (
-
Fixed persistent Fish typing lag after long TUI sessions (Neovim, jjui, Yazi) inside Zellij + Ghostty, which the
no-query-term/ Ghostty-integration fixes from 2026-08-25 did not resolve:- Verified on Fish 4.8.1 that the
query-termfeature already defaults tooff, so exportingfish_features = no-query-termis a no-op on this Fish version — it isn't the cause of (or fix for) this class of lag. - Set
support_kitty_keyboard_protocol = falseinzellij.nix. Zellij and Ghostty have several open upstream bugs (zellij-org/zellij#3887, #3723, #4178) where the Kitty keyboard protocol's enhancement-flag stack is left in an elevated state after a full-screen TUI exits without properly popping it. Every subsequent keystroke then arrives as a CSI-u sequence that Fish must wait out an escape-disambiguation timeout to parse, which reads as typing lag that worsens the longer the TUI session ran, and persists until the pane's protocol state resets (e.g. a fresh shell/pane). Disabling the protocol support in Zellij avoids the whole bug class; trades off precise modifier reporting (e.g. distinguishing Ctrl+Shift+key) for TUIs running inside Zellij panes, which this setup doesn't otherwise depend on (Shift+Enter is handled via a literal Ghosttytext:keybind, not the Kitty protocol).
- Verified on Fish 4.8.1 that the
2026-08-25
- Fixed Nix evaluation warnings for
stdenvdeprecation andgemini-cli:- Replaced deprecated
stdenv.isLinuxandstdenv.isDarwinchecks across module presets and package definitions withstdenv.hostPlatform.isLinuxandstdenv.hostPlatform.isDarwin. - Replaced deprecated
pkgs.gemini-cliwithpkgs.antigravity-cli(and updated binary invocation toagy) inexperimental.nixprofile anddaily-summary.nixlaunchd service.
- Replaced deprecated
2026-08-25
- Fixed multi-second hang and permanent typing latency in Fish after exiting TUIs inside Zellij and Ghostty:
- Exported
fish_features = "no-query-term"inhome.sessionVariablesand addedset -gx fish_features no-query-termto Fish's top-levelshellInit. Previous attempt (set -a fish_features no-query-termininteractiveShellInit) set a local variable inside an anonymous initialization function block that went out of scope immediately after startup. Furthermore, Fish readsfish_featuresat binary launch before interactive init functions run. Withoutno-query-termexported prior to Fish startup, Fish attempted terminal feature queries (Primary Device AttributesDA1/\e[?cand termcap) whenever a TUI (e.g. Neovim, Lazygit, Yazi) exited and returned control to Fish. Zellij drops or delays DA1 response sequences, causing Fish to block on a multi-second stdin timeout, followed by severe input reader desynchronization and typing latency on every subsequent keystroke. - Disabled
programs.ghostty.enableFishIntegrationand conditionally sourced Ghostty's shell integration script inshellInitonly when NOT running inside a multiplexer (not set -q ZELLIJandnot set -q TMUX). Sourcing Ghostty's shell integration inside Zellij sent duplicate and conflicting OSC 133 prompt markers and DECSCUSR cursor escape sequences to Zellij's PTY parser.
- Exported
2026-08-16
- Fixed Zellij new tab directory tracking by adding
__fish_update_cwd_oscoverride inpresets/programs/zellij.nix. Fish's default OSC 7 sequence includes$hostname, which on macOS or dynamic network environments evaluates toNoah-MacBook-Pro.localor a domain suffix. Zellij compares the OSC 7 hostname against its system hostname (Noah-MacBook-Pro), finds a mismatch, and silently ignores the CWD update, leaving new tabs stuck in a previous directory or session default. Overriding__fish_update_cwd_oscto sendfile://<PWD>(empty hostname) ensures Zellij always updates its cached CWD on everycdand prompt render. - Fixed Firefox "profile cannot be loaded" error on macOS by removing
home.file."Library/Application Support/Firefox/installs.ini". Hardcoding an installation hash ininstalls.inibroke whenever Firefox was updated or rebuilt in the Nix store because the nix store path changed, causing Firefox to compute a new installation hash, fail to match or write to the read-onlyinstalls.inisymlink, and error out. Firefox on macOS usesprofiles.ini(managed by Home Manager) andMOZ_LEGACY_PROFILES=1(exported by nixpkgs' launcher wrapper).
2026-08-03
-
Added
presets/security/corporate-ca.nix(nix-darwin) and enabled it on thelookingglasshost to trust a corporate TLS-intercepting proxy's root CA. Behind the corp network, Nix fetches failed withSSL peer certificate ... self-signed certificate in certificate chain (19)because Nix's stock Mozilla CA bundle doesn't contain the interception root. The module appends the cert tosecurity.pki.certificateFiles, which rebuilds/etc/ssl/certs/ca-certificates.crt(read by both the Nix daemon and, viaNIX_SSL_CERT_FILE, client-side flake fetches).The cert is kept out of this public repo and referenced by absolute path. It is passed as a string (not a Nix path literal) so pure flake evaluation doesn't read it at eval time, and it lives at a root-owned, world-readable path because the unprivileged
nixbldbuild user cannot traverse$HOME(mode0750) to read it at build time.One-time setup on a machine behind the proxy:
# 1. Extract the self-signed corporate root from any TLS connection it MITMs # (the last cert in the chain, subject == issuer). Any HTTPS host works: echo | openssl s_client -connect example.com:443 -servername example.com \ -showcerts 2>/dev/null \ | awk '/BEGIN CERT/{c++} c==2' > /tmp/CorpCA.pem openssl x509 -in /tmp/CorpCA.pem -noout -subject -issuer # sanity check # 2. Install to the root-owned path the config points at: sudo install -d -m 0755 -o root -g wheel /etc/ssl/corp-ca sudo install -m 0644 -o root -g wheel /tmp/CorpCA.pem \ /etc/ssl/corp-ca/CorpCA.pem # 3. Bootstrap the first rebuild (which must fetch inputs over the proxy) # with a combined bundle, then it's permanent: cat /etc/ssl/certs/ca-certificates.crt /etc/ssl/corp-ca/CorpCA.pem \ > /tmp/combined-ca.crt NIX_SSL_CERT_FILE=/tmp/combined-ca.crt nh darwin switch . --configuration lookingglass -
Fixed the
zellij-sessionfish function inpresets/programs/zellij.nixtruncating the session name (derived from the target directory basename) to 20 characters. Zellij names each session's Unix-domain IPC socket$TMPDIR/zellij-<uid>/<version>/<name>, and on macOS the socket path is capped at 103 bytes. The/var/folders/...$TMPDIRprefix consumes ~79 of those, leaving only ~24 chars for the name, so switching into directories with long basenames overflowed the socket path. Becauseswitch-sessionhad already detached from the current session by the time the new one failed to bind, the failure took down the entire terminal instead of erroring gracefully.
2026-07-25
- Added jjui config generation to
jujutsu.nixin Home Manager to include custom GitHub ruleset bypass commands (ctrl+bandctrl+shift+b). - Added
overlays/cheetah3.nixto disablepythonMetadataCheckPhaseforcheetah3. This fixes an issue where the NixOS rebuild fails forsabnzbddue toimportlib.metadata.PackageNotFoundError: No package metadata was found for cheetah3during the Python package evaluation innixos-unstable.
2026-07-20
- Added
overlays/paho-mqtt.nixto disable paho-mqtt's flaky, socket-based test suite. ItscheckPhasehangs in the Nix sandbox and times out with aKeyboardInterruptafter ~150s, which was breaking theflamerebuild (paho-mqtt is pulled in transitively, e.g. via mealie). The override is applied throughpythonPackagesExtensionsso it covers every Python package set.