mirror of
https://github.com/nmasur/dotfiles
synced 2026-08-30 10:17:33 +00:00
fix post-TUI typing lag: latch fish query-term off in zellij pane shells
Root cause (proven via live lag-triage capture + deterministic PTY repro): fish latches feature flags from its startup env before config.fish runs, so the existing no-query-term settings never applied to zellij-spawned shells. With query-term on, fish queries the terminal after every command; one reply zellij fails to relay permanently degrades that fish process's reader. Subshells were immune (inherited the exported var), which is why the lag always 'disappeared' when tested in a new shell. Also adds the lag-triage/unlag/term-probe diagnostic toolkit used to find this, and upstream_repro.py for filing the fish-shell bug.
This commit is contained in:
Noah Masur
@@ -1,5 +1,15 @@
|
||||
# Changelog
|
||||
|
||||
## 2026-08-29 (root cause found and fixed)
|
||||
|
||||
- **Root-caused and fixed the recurring post-TUI typing lag** (fish + Zellij + Ghostty) using a `lag-triage` capture from a live lagging shell plus a deterministic PTY reproduction (`presets/programs/lag-triage/upstream_repro.py`):
|
||||
- **Root cause chain**: (1) fish latches feature flags from its **startup environment**, before `config.fish` runs — so the existing `set -gx fish_features no-query-term` in `shellInit` never applied to the shell that set it, only to its children. (2) Zellij spawns pane shells via `default_shell` with no `fish_features` in the environment, so every pane's fish latched `query-term` **on** (the fish 4.8.1 default; the triage log from the lagging shell confirmed `query-term on` while `$fish_features` was correctly set to `no-query-term`). (3) With query-term on, fish sends OSC 11 + CPR (`\e[6n`) + DA1 (`\e[0c`) after **every** command and waits for replies relayed by Zellij. (4) Reproduced on fish 4.8.1: if the terminal fails to reply during just **one** such cycle — answering everything before and after — that fish process's interactive reader is **permanently degraded** (keystroke echo >3s, never recovers; ~35ms before). In production Zellij drops/mangles a relay during TUI teardown or heavy output (cf. zellij-org/zellij#5158), e.g. after `nh home switch`, nvim, jjui, yazi.
|
||||
- **Why every previous observation finally makes sense**: subshells and `exec fish` were never "resetting" anything — they *inherited* the exported `fish_features=no-query-term` from config.fish, latched query-term off at startup, and were therefore **immune**. The parent zellij-spawned shell never had the variable at startup and stayed vulnerable. Raw keystroke capture in the lagging pane showed instant plain bytes (input path fine) and no stuck terminal modes — the damage was inside the fish process, exactly as the repro shows.
|
||||
- **Fix**: `zellij.nix` now spawns panes through a `fish-no-query-term` wrapper (`export fish_features=no-query-term; exec fish`), so the feature is latched off in every pane shell. Verified: interactive fish through the built wrapper with the real config reports `query-term off`; the PTY repro with `no-query-term` in the environment shows ~35ms echo through all failure phases.
|
||||
- **Correction** to the earlier 2026-08-29 entry: `query-term` does **not** default to off in fish 4.8.1 — it defaults on; it only *appeared* off in non-interactive checks because the user config's `set -gx` takes effect for `fish -c` (no reader latch) but not for interactive shells.
|
||||
- Upstream: fish-shell should bound the reader's wait for query replies instead of degrading permanently (repro script kept at `presets/programs/lag-triage/upstream_repro.py` for filing); Zellij's reply relaying is the trigger (zellij-org/zellij#5158).
|
||||
- `lag-triage` now checks `status features` and calls out `query-term on` as the known root cause, and warns that its `read`-prompt typing tests may not exhibit main-commandline lag (which produced a false "fixed by stage A" in the first capture).
|
||||
|
||||
## 2026-08-29 (later)
|
||||
|
||||
- Added a diagnostic toolkit (`lag-triage` / `unlag` fish functions + `term-probe` binary, `presets/programs/lag-triage/`) for the still-recurring post-TUI typing lag in fish + Zellij + Ghostty, instead of another blind fix. Findings that motivated it:
|
||||
|
||||
@@ -45,6 +45,23 @@ begin
|
||||
end >>$logfile 2>&1
|
||||
_lt "Captured shell + environment snapshot."
|
||||
|
||||
# Proven root cause of the 2026-08 lag (see docs/CHANGELOG.md 2026-08-29):
|
||||
# fish latches feature flags from its startup env before config.fish runs, so
|
||||
# a shell with query-term ON sends terminal queries after every command; one
|
||||
# reply zellij fails to relay permanently degrades this process's reader.
|
||||
if status features | string match -qr '^query-term\s+on'
|
||||
_lt ""
|
||||
_lt "!! query-term is ON in this shell: fish did NOT get fish_features="
|
||||
_lt "!! no-query-term in its STARTUP environment (config.fish is too late)."
|
||||
_lt "!! This is the proven root cause of the post-TUI lag — a query reply"
|
||||
_lt "!! lost by zellij permanently degrades this fish process's reader."
|
||||
_lt "!! Fix: spawn fish with the variable exported (zellij default_shell"
|
||||
_lt "!! wrapper fish-no-query-term). Subshells are immune because they"
|
||||
_lt "!! inherit the exported variable — that's why a new shell 'fixes' it."
|
||||
else
|
||||
_lt "query-term is off in this shell (good — the known root cause is ruled out)."
|
||||
end
|
||||
|
||||
# ---- 3. Terminal state below the shell --------------------------------------
|
||||
_lt ""
|
||||
_lt "Querying terminal state (takes a few seconds)..."
|
||||
@@ -76,6 +93,10 @@ set -l fixed none
|
||||
_lt ""
|
||||
_lt "Now applying resets one at a time. After each, type into the test prompt"
|
||||
_lt "to judge whether the lag is gone."
|
||||
_lt "CAVEAT: fish's read prompt may NOT exhibit lag even when the main"
|
||||
_lt "commandline does. If typing at these test prompts never feels laggy at"
|
||||
_lt "all, answer 'u' (unsure) instead of 'y' — a 'y' here is only meaningful"
|
||||
_lt "if you could feel the lag at the test prompts before the reset."
|
||||
|
||||
if test $fixed = none
|
||||
_lt ""
|
||||
|
||||
@@ -0,0 +1,151 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Deterministic repro of permanent fish reader degradation (fish 4.8.1).
|
||||
|
||||
This is evidence for an upstream fish-shell report, and the proof behind the
|
||||
fish-no-query-term wrapper in presets/programs/zellij.nix. Not installed by
|
||||
the nix module; run directly: python3 upstream_repro.py [queryterm|noqueryterm]
|
||||
|
||||
Finding: with the query-term feature enabled (latched from the startup env,
|
||||
which is fish's default), fish sends OSC 11 + CPR (\e[6n) + DA1 (\e[0c)
|
||||
after every external command and waits for the replies. If the terminal
|
||||
fails to reply during ONE such cycle -- even though it answered every query
|
||||
before and answers every query after -- that fish process's interactive
|
||||
reader is PERMANENTLY degraded: keystrokes are no longer echoed (>3s each,
|
||||
never recovers). In production this happens when zellij drops/mis-relays a
|
||||
reply during TUI teardown or heavy output (cf. zellij-org/zellij#5158), and
|
||||
it presents as permanent typing lag cured only by replacing the process.
|
||||
With fish_features=no-query-term in the startup environment, the same
|
||||
sequence has zero effect (~35ms echo throughout).
|
||||
|
||||
Phases:
|
||||
A. terminal answers all queries -> echo ~35ms (both variants)
|
||||
B. replies dropped for one command -> queryterm: echo dead, permanently
|
||||
C. replies restored, another command -> queryterm: still dead
|
||||
"""
|
||||
import os, pty, re, select, subprocess, sys, time, fcntl, termios
|
||||
|
||||
VARIANT = sys.argv[1] if len(sys.argv) > 1 else "queryterm"
|
||||
|
||||
env = dict(os.environ)
|
||||
env["TERM"] = "xterm-ghostty"
|
||||
env["ZELLIJ"] = "0"
|
||||
env["ZELLIJ_SESSION_NAME"] = "repro"
|
||||
env["FISH_DEBUG"] = "term-support"
|
||||
env["FISH_DEBUG_OUTPUT"] = f"/tmp/fish-lagrepro-{VARIANT}.log"
|
||||
env.pop("fish_features", None)
|
||||
if VARIANT == "noqueryterm":
|
||||
env["fish_features"] = "no-query-term"
|
||||
|
||||
master, slave = pty.openpty()
|
||||
# give it a size
|
||||
fcntl.ioctl(master, termios.TIOCSWINSZ, b"\x00\x28\x00\x78\x00\x00\x00\x00")
|
||||
proc = subprocess.Popen(
|
||||
["fish", "-i", "--no-config"],
|
||||
stdin=slave, stdout=slave, stderr=slave, env=env,
|
||||
preexec_fn=lambda: (os.setsid(), fcntl.ioctl(0, termios.TIOCSCTTY, 0)),
|
||||
close_fds=True,
|
||||
)
|
||||
os.close(slave)
|
||||
|
||||
RESPOND = True
|
||||
transcript = []
|
||||
|
||||
def respond(data):
|
||||
"""Answer terminal queries the way a well-behaved terminal would."""
|
||||
out = b""
|
||||
for m in re.finditer(rb"\x1b\[6n", data):
|
||||
out += b"\x1b[40;1R" # CPR
|
||||
for m in re.finditer(rb"\x1b\[0?c", data):
|
||||
out += b"\x1b[?62;22c" # DA1
|
||||
for m in re.finditer(rb"\x1b\[\?u", data):
|
||||
out += b"\x1b[?0u" # kitty flags
|
||||
for m in re.finditer(rb"\x1b\]11;\?", data):
|
||||
out += b"\x1b]11;rgb:2828/2828/2828\x1b\\" # OSC 11
|
||||
for m in re.finditer(rb"\x1b\[>0?q", data):
|
||||
out += b"\x1bP>|ghostty 1.3.1\x1b\\" # XTVERSION
|
||||
for m in re.finditer(rb"\x1bP\+q[0-9a-fA-F;]+\x1b\\", data):
|
||||
out += b"\x1bP0+r\x1b\\" # XTGETTCAP: not found
|
||||
for m in re.finditer(rb"\x1b\[\?(\d+)\$p", data):
|
||||
out += b"\x1b[?%s;2$y" % m.group(1) # DECRQM: reset
|
||||
return out
|
||||
|
||||
def pump(timeout):
|
||||
"""Read fish output for `timeout` seconds, answering queries if RESPOND."""
|
||||
buf = b""
|
||||
end = time.monotonic() + timeout
|
||||
while time.monotonic() < end:
|
||||
r, _, _ = select.select([master], [], [], 0.03)
|
||||
if r:
|
||||
try:
|
||||
data = os.read(master, 65536)
|
||||
except OSError:
|
||||
return buf
|
||||
buf += data
|
||||
transcript.append(data)
|
||||
if RESPOND:
|
||||
reply = respond(data)
|
||||
if reply:
|
||||
os.write(master, reply)
|
||||
return buf
|
||||
|
||||
def send(s):
|
||||
os.write(master, s if isinstance(s, bytes) else s.encode())
|
||||
|
||||
def measure_echo(chars, settle=0.1):
|
||||
"""Send chars one at a time; measure time until each is echoed."""
|
||||
results = []
|
||||
for ch in chars:
|
||||
pump(settle)
|
||||
t0 = time.monotonic()
|
||||
send(ch)
|
||||
deadline = time.monotonic() + 3.0
|
||||
latency = None
|
||||
buf = b""
|
||||
while time.monotonic() < deadline:
|
||||
buf += pump(0.02)
|
||||
if ch.encode() in buf:
|
||||
latency = (time.monotonic() - t0) * 1000
|
||||
break
|
||||
results.append((ch, latency))
|
||||
return results
|
||||
|
||||
print(f"=== variant: {VARIANT} ===")
|
||||
pump(1.2) # startup, queries answered
|
||||
|
||||
send("echo warmup\r")
|
||||
pump(0.8)
|
||||
|
||||
print("phase A: terminal responsive, echo latency per key:")
|
||||
for ch, ms in measure_echo("abcde"):
|
||||
print(f" {ch}: {ms:.0f} ms" if ms else f" {ch}: NO ECHO in 3s")
|
||||
send("\x15") # ctrl-u clear line
|
||||
pump(0.3)
|
||||
|
||||
# Run external command, then STOP answering queries (simulate lost relay)
|
||||
send("sh -c true\r")
|
||||
time.sleep(0.05)
|
||||
RESPOND = False
|
||||
pump(1.0)
|
||||
|
||||
print("phase B: after external command with query replies DROPPED:")
|
||||
for ch, ms in measure_echo("fghij"):
|
||||
print(f" {ch}: {ms:.0f} ms" if ms else f" {ch}: NO ECHO in 3s")
|
||||
send("\x15")
|
||||
pump(0.3)
|
||||
|
||||
# Does it persist across further commands, with responses restored?
|
||||
RESPOND = True
|
||||
send("sh -c true\r")
|
||||
pump(1.0)
|
||||
print("phase C: responses restored, after another external command:")
|
||||
for ch, ms in measure_echo("klmno"):
|
||||
print(f" {ch}: {ms:.0f} ms" if ms else f" {ch}: NO ECHO in 3s")
|
||||
|
||||
send("\x15")
|
||||
pump(0.2)
|
||||
send("exit\r")
|
||||
pump(0.5)
|
||||
try:
|
||||
proc.wait(timeout=3)
|
||||
except subprocess.TimeoutExpired:
|
||||
proc.kill()
|
||||
@@ -9,6 +9,22 @@ let
|
||||
inherit (config.nmasur.settings) username;
|
||||
cfg = config.nmasur.presets.programs.zellij;
|
||||
|
||||
# fish latches feature flags from its startup ENVIRONMENT before config.fish
|
||||
# runs, so the `set -gx fish_features no-query-term` in config.fish only
|
||||
# protects CHILD fish processes — which is why subshells/exec fish were always
|
||||
# immune to the post-TUI typing lag while zellij-spawned pane shells were not.
|
||||
# With query-term latched on, fish sends OSC 11 + CPR + DA1 queries after
|
||||
# every command and waits for replies; if zellij fails to relay even one
|
||||
# reply (a race during TUI teardown or heavy output), that fish process's
|
||||
# reader is PERMANENTLY degraded — reproduced deterministically in a PTY
|
||||
# harness on fish 4.8.1 (see docs/CHANGELOG.md 2026-08-29 and
|
||||
# presets/programs/lag-triage/upstream_repro.py). Spawning fish with the
|
||||
# variable already exported makes every pane shell immune.
|
||||
fish-no-query-term = pkgs.writeShellScriptBin "fish-no-query-term" ''
|
||||
export fish_features=no-query-term
|
||||
exec ${lib.getExe pkgs.fish} "$@"
|
||||
'';
|
||||
|
||||
zellij-switch-to-last = pkgs.writeShellScriptBin "zellij-switch-to-last" ''
|
||||
TARGET_SESSION=$(cat ~/.local/state/zellij-last-session)
|
||||
if [ -z "$TARGET_SESSION" ]; then
|
||||
@@ -130,7 +146,10 @@ in
|
||||
# Spawn fish directly instead of trusting $SHELL, which inherits the
|
||||
# macOS login shell. On darwin that login shell is no longer managed by
|
||||
# nix-darwin, so $SHELL can point at a stale /run/current-system path.
|
||||
default_shell = lib.getExe pkgs.fish;
|
||||
# Wrapped to export fish_features=no-query-term BEFORE fish starts —
|
||||
# see the fish-no-query-term comment above for why this must happen in
|
||||
# the environment rather than in config.fish.
|
||||
default_shell = lib.getExe fish-no-query-term;
|
||||
# default_layout = "compact-top";
|
||||
# Remove border
|
||||
pane_frames = false;
|
||||
|
||||
Reference in New Issue
Block a user