TermRemoteCtl/docs/superpowers/plans/2026-04-06-terminal-reconnect-recovery.md

537 lines
20 KiB
Markdown

# Terminal Reconnect Recovery Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Add backend-owned reconnect restore state and a raw terminal I/O journal so ordinary shell sessions reconnect with visible input and output intact.
**Architecture:** Keep the existing helper-backed ConPTY runtime and websocket transport, but separate session runtime, raw journal, and restore state. Move reconnect truth to the backend by emitting an explicit `restore` payload during websocket attach, while the Flutter client applies that payload as authoritative terminal state instead of inferring from output replay alone.
**Tech Stack:** ASP.NET Core, WebSocket, helper-backed ConPTY, Flutter, Riverpod, xterm, JSON lines storage, xUnit, Flutter widget tests.
---
### Task 1: Add Raw Terminal Journal Models And Storage
**Files:**
- Create: `apps/windows_agent/src/TermRemoteCtl.Agent/History/SessionIoEvent.cs`
- Create: `apps/windows_agent/src/TermRemoteCtl.Agent/History/SessionIoJournalStore.cs`
- Modify: `apps/windows_agent/src/TermRemoteCtl.Agent/Configuration/AgentOptions.cs`
- Modify: `apps/windows_agent/src/TermRemoteCtl.Agent/Program.cs`
- Test: `apps/windows_agent/tests/TermRemoteCtl.Agent.Tests/Sessions/SessionRegistryTests.cs`
- [ ] **Step 1: Write the failing journal storage tests**
```csharp
[Fact]
public async Task AppendIoEventAsync_Persists_Input_And_Output_In_Order()
{
using var harness = SessionRegistryHarness.Create();
var store = new SessionIoJournalStore(harness.DataRoot);
await store.AppendAsync(new SessionIoEvent("session-1", 1, "input", "dir", DateTimeOffset.UtcNow), CancellationToken.None);
await store.AppendAsync(new SessionIoEvent("session-1", 2, "output", "dir\r\n", DateTimeOffset.UtcNow), CancellationToken.None);
var lines = await File.ReadAllLinesAsync(Path.Combine(harness.DataRoot, "sessions", "session-1.io.jsonl"));
Assert.Equal(2, lines.Length);
Assert.Contains("\"kind\":\"input\"", lines[0]);
Assert.Contains("\"kind\":\"output\"", lines[1]);
}
```
- [ ] **Step 2: Run test to verify it fails**
Run: `dotnet test apps/windows_agent/tests/TermRemoteCtl.Agent.Tests/TermRemoteCtl.Agent.Tests.csproj --filter AppendIoEventAsync_Persists_Input_And_Output_In_Order`
Expected: FAIL with missing `SessionIoJournalStore` or `SessionIoEvent`
- [ ] **Step 3: Add the journal event record**
```csharp
namespace TermRemoteCtl.Agent.History;
public sealed record SessionIoEvent(
string SessionId,
long Sequence,
string Kind,
string Payload,
DateTimeOffset TimestampUtc);
```
- [ ] **Step 4: Add the journal file store**
```csharp
public sealed class SessionIoJournalStore
{
private readonly string _sessionRoot;
public SessionIoJournalStore(string rootPath)
{
_sessionRoot = Path.Combine(rootPath, "sessions");
Directory.CreateDirectory(_sessionRoot);
}
public async Task AppendAsync(SessionIoEvent ioEvent, CancellationToken cancellationToken)
{
var filePath = Path.Combine(_sessionRoot, $"{ioEvent.SessionId}.io.jsonl");
var line = JsonSerializer.Serialize(ioEvent) + Environment.NewLine;
await File.AppendAllTextAsync(filePath, line, new UTF8Encoding(false), cancellationToken).ConfigureAwait(false);
}
}
```
- [ ] **Step 5: Add configuration and service registration**
```csharp
public sealed class AgentOptions
{
public string DataRoot { get; set; } = string.Empty;
public int RingBufferLineLimit { get; set; } = 4000;
public int SessionJournalRetentionDays { get; set; } = 7;
}
```
```csharp
builder.Services.AddSingleton<SessionIoJournalStore>(sp =>
{
var options = sp.GetRequiredService<IOptions<AgentOptions>>().Value;
return new SessionIoJournalStore(options.DataRoot);
});
```
- [ ] **Step 6: Run the backend unit test to verify it passes**
Run: `dotnet test apps/windows_agent/tests/TermRemoteCtl.Agent.Tests/TermRemoteCtl.Agent.Tests.csproj --filter SessionRegistryTests`
Expected: PASS
- [ ] **Step 7: Commit**
```bash
git add apps/windows_agent/src/TermRemoteCtl.Agent/History/SessionIoEvent.cs apps/windows_agent/src/TermRemoteCtl.Agent/History/SessionIoJournalStore.cs apps/windows_agent/src/TermRemoteCtl.Agent/Configuration/AgentOptions.cs apps/windows_agent/src/TermRemoteCtl.Agent/Program.cs apps/windows_agent/tests/TermRemoteCtl.Agent.Tests/Sessions/SessionRegistryTests.cs
git commit -m "feat: add terminal session io journal"
```
### Task 2: Extend Session Registry With Restore Snapshot State
**Files:**
- Create: `apps/windows_agent/src/TermRemoteCtl.Agent/Sessions/SessionRestoreSnapshot.cs`
- Modify: `apps/windows_agent/src/TermRemoteCtl.Agent/Sessions/SessionRegistry.cs`
- Modify: `apps/windows_agent/src/TermRemoteCtl.Agent/History/PendingInputEchoTracker.cs`
- Test: `apps/windows_agent/tests/TermRemoteCtl.Agent.Tests/Sessions/SessionRegistryTests.cs`
- [ ] **Step 1: Write failing restore snapshot tests**
```csharp
[Fact]
public void GetRestoreSnapshot_Includes_Pending_Visible_Input()
{
using var harness = SessionRegistryHarness.Create();
var session = harness.Registry.Create("Shell", DateTimeOffset.UtcNow);
harness.Registry.RecordInputEcho(session.SessionId, "git status");
var snapshot = harness.Registry.GetRestoreSnapshot(session.SessionId);
Assert.Equal(string.Empty, snapshot.ScreenText);
Assert.Equal("git status", snapshot.PendingInput);
Assert.True(snapshot.Sequence > 0);
}
```
```csharp
[Fact]
public async Task GetRestoreSnapshot_Does_Not_Duplicate_Acknowledged_Input()
{
using var harness = SessionRegistryHarness.Create();
var session = harness.Registry.Create("Shell", DateTimeOffset.UtcNow);
harness.Registry.RecordInputEcho(session.SessionId, "dir\r");
await harness.Registry.AppendOutputAsync(session.SessionId, "PS> dir\r\nnext> ", CancellationToken.None);
var snapshot = harness.Registry.GetRestoreSnapshot(session.SessionId);
Assert.Equal("PS> dir\r\nnext> ", snapshot.ScreenText);
Assert.Equal(string.Empty, snapshot.PendingInput);
}
```
- [ ] **Step 2: Run test to verify it fails**
Run: `dotnet test apps/windows_agent/tests/TermRemoteCtl.Agent.Tests/TermRemoteCtl.Agent.Tests.csproj --filter "GetRestoreSnapshot_"`
Expected: FAIL with missing `GetRestoreSnapshot`
- [ ] **Step 3: Add the restore snapshot model**
```csharp
namespace TermRemoteCtl.Agent.Sessions;
public sealed record SessionRestoreSnapshot(
string SessionId,
long Sequence,
string ScreenText,
string PendingInput,
int? CursorRow,
int? CursorColumn);
```
- [ ] **Step 4: Extend `SessionRegistry` to track restore sequence**
```csharp
private readonly ConcurrentDictionary<string, long> _sequenceBySession = new();
public long NextSequence(string sessionId)
{
return _sequenceBySession.AddOrUpdate(sessionId, 1, (_, current) => current + 1);
}
```
- [ ] **Step 5: Add restore snapshot retrieval**
```csharp
public SessionRestoreSnapshot GetRestoreSnapshot(string sessionId)
{
var replay = _replayBySession.GetOrAdd(sessionId, _ => new TerminalReplayBuffer(ReplayCharacterLimit));
var pending = _pendingInputEchoBySession.GetOrAdd(sessionId, _ => new PendingInputEchoTracker());
var sequence = _sequenceBySession.GetOrAdd(sessionId, 1);
return new SessionRestoreSnapshot(
sessionId,
sequence,
replay.GetSnapshot(),
pending.GetVisibleSuffix(),
null,
null);
}
```
- [ ] **Step 6: Update input/output mutation points**
```csharp
public void RecordInputEcho(string sessionId, string input)
{
var tracker = _pendingInputEchoBySession.GetOrAdd(sessionId, _ => new PendingInputEchoTracker());
tracker.Record(input);
NextSequence(sessionId);
}
```
```csharp
public async Task AppendOutputAsync(string sessionId, string chunk, CancellationToken cancellationToken)
{
// existing history and replay mutations
_pendingInputEchoBySession.GetOrAdd(sessionId, _ => new PendingInputEchoTracker()).ObserveOutput(chunk);
NextSequence(sessionId);
await _historyStore.AppendAsync(sessionId, chunk, cancellationToken).ConfigureAwait(false);
}
```
- [ ] **Step 7: Run the backend unit tests to verify they pass**
Run: `dotnet test apps/windows_agent/tests/TermRemoteCtl.Agent.Tests/TermRemoteCtl.Agent.Tests.csproj --filter SessionRegistryTests`
Expected: PASS
- [ ] **Step 8: Commit**
```bash
git add apps/windows_agent/src/TermRemoteCtl.Agent/Sessions/SessionRestoreSnapshot.cs apps/windows_agent/src/TermRemoteCtl.Agent/Sessions/SessionRegistry.cs apps/windows_agent/src/TermRemoteCtl.Agent/History/PendingInputEchoTracker.cs apps/windows_agent/tests/TermRemoteCtl.Agent.Tests/Sessions/SessionRegistryTests.cs
git commit -m "feat: add session restore snapshot state"
```
### Task 3: Emit Restore Payload And Journal Events On Websocket Attach
**Files:**
- Modify: `apps/windows_agent/src/TermRemoteCtl.Agent/Realtime/TerminalWebSocketHandler.cs`
- Modify: `apps/windows_agent/src/TermRemoteCtl.Agent/Terminal/PowerShellSessionHost.cs`
- Test: `apps/windows_agent/tests/TermRemoteCtl.Agent.IntegrationTests/Realtime/TerminalWebSocketHandlerTests.cs`
- [ ] **Step 1: Write failing websocket restore tests**
```csharp
[Fact]
public async Task Reattach_Returns_Restore_Payload_With_Pending_Input()
{
await using var fixture = new TerminalApiFixture();
var registry = fixture.Services.GetRequiredService<SessionRegistry>();
var session = registry.Create("Shell", DateTimeOffset.UtcNow);
registry.RecordInputEcho(session.SessionId, "dir");
using WebSocket socket = await fixture.Server.CreateWebSocketClient().ConnectAsync(
new Uri($"ws://localhost/ws/terminal?sessionId={session.SessionId}"),
CancellationToken.None);
_ = await ReceiveTextAsync(socket, CancellationToken.None);
var restoreFrame = await ReceiveTextAsync(socket, CancellationToken.None);
Assert.Contains("\"type\":\"restore\"", restoreFrame);
Assert.Contains("\"pendingInput\":\"dir\"", restoreFrame);
}
```
- [ ] **Step 2: Run test to verify it fails**
Run: `dotnet test apps/windows_agent/tests/TermRemoteCtl.Agent.IntegrationTests/TermRemoteCtl.Agent.IntegrationTests.csproj --filter Reattach_Returns_Restore_Payload_With_Pending_Input`
Expected: FAIL because only replay text is sent
- [ ] **Step 3: Add the restore response contract**
```csharp
private sealed record TerminalRestoreResponse(
string SessionId,
long Sequence,
string ScreenText,
string PendingInput,
int? CursorRow,
int? CursorColumn,
string Type = "restore");
```
- [ ] **Step 4: Send restore payload after attach acknowledgement**
```csharp
var restore = registry.GetRestoreSnapshot(sessionId);
await SendJsonAsync(socket, new TerminalAttachResponse(sessionId), sendGate, context.RequestAborted).ConfigureAwait(false);
await SendJsonAsync(
socket,
new TerminalRestoreResponse(
restore.SessionId,
restore.Sequence,
restore.ScreenText,
restore.PendingInput,
restore.CursorRow,
restore.CursorColumn),
sendGate,
context.RequestAborted).ConfigureAwait(false);
```
- [ ] **Step 5: Journal websocket lifecycle and PTY traffic**
```csharp
await journalStore.AppendAsync(new SessionIoEvent(sessionId, registry.NextSequence(sessionId), "attach", string.Empty, DateTimeOffset.UtcNow), context.RequestAborted);
```
```csharp
registry.RecordInputEcho(sessionId, message.Input);
await journalStore.AppendAsync(new SessionIoEvent(sessionId, registry.NextSequence(sessionId), "input", message.Input, DateTimeOffset.UtcNow), cancellationToken);
await host.WriteInputAsync(sessionId, message.Input, cancellationToken).ConfigureAwait(false);
```
```csharp
_ = _sessionRegistry.AppendOutputAsync(args.SessionId, args.Chunk, CancellationToken.None);
_ = _journalStore.AppendAsync(new SessionIoEvent(args.SessionId, _sessionRegistry.NextSequence(args.SessionId), "output", args.Chunk, DateTimeOffset.UtcNow), CancellationToken.None);
```
- [ ] **Step 6: Run integration tests to verify they pass**
Run: `dotnet test apps/windows_agent/tests/TermRemoteCtl.Agent.IntegrationTests/TermRemoteCtl.Agent.IntegrationTests.csproj --filter TerminalWebSocketHandlerTests`
Expected: PASS
- [ ] **Step 7: Commit**
```bash
git add apps/windows_agent/src/TermRemoteCtl.Agent/Realtime/TerminalWebSocketHandler.cs apps/windows_agent/src/TermRemoteCtl.Agent/Terminal/PowerShellSessionHost.cs apps/windows_agent/tests/TermRemoteCtl.Agent.IntegrationTests/Realtime/TerminalWebSocketHandlerTests.cs
git commit -m "feat: send terminal restore payload on attach"
```
### Task 4: Teach Flutter To Restore From Backend Snapshot
**Files:**
- Create: `apps/mobile_app/lib/features/terminal/terminal_restore_payload.dart`
- Modify: `apps/mobile_app/lib/features/terminal/terminal_socket_session.dart`
- Modify: `apps/mobile_app/lib/features/terminal/terminal_session_coordinator.dart`
- Modify: `apps/mobile_app/lib/features/terminal/terminal_page.dart`
- Test: `apps/mobile_app/test/features/terminal/terminal_socket_session_test.dart`
- Test: `apps/mobile_app/test/features/terminal/terminal_session_coordinator_test.dart`
- Test: `apps/mobile_app/test/widget_test.dart`
- [ ] **Step 1: Write failing Flutter restore tests**
```dart
testWidgets('terminal reconnect applies restore payload before live frames', (tester) async {
final transportFactory = _QueuedTerminalSocketTransportFactory(
connectionStartupFrames: [
const [
_StartupFrame('{"type":"attached","sessionId":"session-1"}'),
_StartupFrame('{"type":"restore","sessionId":"session-1","sequence":4,"screenText":"PS> gi","pendingInput":"t status"}'),
],
],
);
await _pumpTerminalPage(
tester,
session: _session('session-1', 'codex-main'),
socketFactory: TerminalSocketSessionFactory(transportFactory: transportFactory.create),
);
final terminal = tester.widget<TerminalView>(find.byType(TerminalView)).terminal;
expect(terminal.buffer.getText(), contains('PS> git status'));
});
```
- [ ] **Step 2: Run test to verify it fails**
Run: `C:\tools\flutter\bin\flutter.bat test test/features/terminal/terminal_socket_session_test.dart test/widget_test.dart`
Expected: FAIL because restore frames are not parsed separately
- [ ] **Step 3: Add restore payload model**
```dart
class TerminalRestorePayload {
const TerminalRestorePayload({
required this.sessionId,
required this.sequence,
required this.screenText,
required this.pendingInput,
});
factory TerminalRestorePayload.fromJson(Map<String, dynamic> json) {
return TerminalRestorePayload(
sessionId: json['sessionId'] as String,
sequence: json['sequence'] as int,
screenText: (json['screenText'] as String?) ?? '',
pendingInput: (json['pendingInput'] as String?) ?? '',
);
}
}
```
- [ ] **Step 4: Parse restore frames separately from live output**
```dart
Future<void> connect({
required void Function(String frame) onFrame,
required void Function(TerminalRestorePayload restore) onRestore,
void Function()? onDisconnected,
})
```
```dart
if (decoded is Map && decoded['type'] == 'restore') {
onRestore(TerminalRestorePayload.fromJson(Map<String, dynamic>.from(decoded)));
return;
}
```
- [ ] **Step 5: Apply restore payload as authoritative state in `TerminalPage`**
```dart
void _handleRestorePayload(TerminalRestorePayload restore) {
_resetTerminalForReplay();
final combined = restore.screenText + restore.pendingInput;
if (combined.isNotEmpty) {
terminal.write(combined);
}
}
```
- [ ] **Step 6: Run Flutter tests to verify they pass**
Run: `C:\tools\flutter\bin\flutter.bat test test/features/terminal/terminal_page_input_test.dart test/features/terminal/terminal_socket_session_test.dart test/features/terminal/terminal_session_coordinator_test.dart test/widget_test.dart`
Expected: PASS
- [ ] **Step 7: Commit**
```bash
git add apps/mobile_app/lib/features/terminal/terminal_restore_payload.dart apps/mobile_app/lib/features/terminal/terminal_socket_session.dart apps/mobile_app/lib/features/terminal/terminal_session_coordinator.dart apps/mobile_app/lib/features/terminal/terminal_page.dart apps/mobile_app/test/features/terminal/terminal_socket_session_test.dart apps/mobile_app/test/features/terminal/terminal_session_coordinator_test.dart apps/mobile_app/test/widget_test.dart
git commit -m "feat: restore terminal state from backend snapshot"
```
### Task 5: Make Restore Snapshot The Primary Reconnect Path
**Files:**
- Modify: `apps/mobile_app/lib/features/terminal/terminal_page.dart`
- Modify: `apps/mobile_app/test/widget_test.dart`
- Modify: `docs/testing/manual-smoke-checklist.md`
- [ ] **Step 1: Write failing test that proves reconnect no longer depends on output-only replay**
```dart
testWidgets('terminal reconnect restores pending input without history seed fallback', (tester) async {
final transportFactory = _QueuedTerminalSocketTransportFactory(
connectionStartupFrames: [
const [
_StartupFrame('{"type":"attached","sessionId":"session-1"}'),
_StartupFrame('{"type":"restore","sessionId":"session-1","sequence":7,"screenText":"PS> gi","pendingInput":"t status"}'),
],
],
);
await _pumpTerminalPage(
tester,
session: _session('session-1', 'codex-main'),
apiClient: _FakeAgentApiClient(lines: const <String>[]),
socketFactory: TerminalSocketSessionFactory(transportFactory: transportFactory.create),
);
final terminal = tester.widget<TerminalView>(find.byType(TerminalView)).terminal;
expect(terminal.buffer.getText(), contains('PS> git status'));
});
```
- [ ] **Step 2: Run test to verify it fails**
Run: `C:\tools\flutter\bin\flutter.bat test test/widget_test.dart`
Expected: FAIL until restore payload becomes the primary reconnect source
- [ ] **Step 3: Simplify reconnect restore logic**
```dart
if (connectionState == TerminalConnectionState.reconnecting) {
_resetTerminalForReplay();
_historySeeded = false;
_receivedSocketFrame = false;
}
```
```dart
if (_receivedRestorePayload) {
_cancelHistorySeedTimer();
return;
}
```
- [ ] **Step 4: Update the manual smoke checklist**
```markdown
10. Type a partial command, background the app, reopen it, and confirm the typed command is still visible.
11. Execute a command, reconnect during output, and confirm the command is not duplicated after restore.
```
- [ ] **Step 5: Run final verification**
Run: `C:\tools\flutter\bin\flutter.bat test test/widget_test.dart test/features/terminal/terminal_page_input_test.dart test/features/terminal/terminal_socket_session_test.dart test/features/terminal/terminal_session_coordinator_test.dart`
Expected: PASS
Run: `dotnet test apps/windows_agent/tests/TermRemoteCtl.Agent.Tests/TermRemoteCtl.Agent.Tests.csproj`
Expected: PASS
Run: `dotnet test apps/windows_agent/tests/TermRemoteCtl.Agent.IntegrationTests/TermRemoteCtl.Agent.IntegrationTests.csproj --filter "TerminalWebSocketHandlerTests|SessionHistoryApiTests"`
Expected: PASS
- [ ] **Step 6: Commit**
```bash
git add apps/mobile_app/lib/features/terminal/terminal_page.dart apps/mobile_app/test/widget_test.dart docs/testing/manual-smoke-checklist.md
git commit -m "refactor: make terminal restore snapshot authoritative"
```
## Self-Review
- Spec coverage:
- raw journal is covered by Tasks 1 and 3
- restore snapshot state is covered by Task 2
- websocket attach restore protocol is covered by Task 3
- Flutter restore consumption is covered by Tasks 4 and 5
- Placeholder scan:
- each task contains concrete files, code, commands, and expected results
- Type consistency:
- `SessionIoEvent`, `SessionRestoreSnapshot`, `TerminalRestorePayload`, `GetRestoreSnapshot`, and `RecordInputEcho` are used consistently across the plan
## Execution Handoff
Plan complete and saved to `docs/superpowers/plans/2026-04-06-terminal-reconnect-recovery.md`. Two execution options:
**1. Subagent-Driven (recommended)** - I dispatch a fresh subagent per task, review between tasks, fast iteration
**2. Inline Execution** - Execute tasks in this session using executing-plans, batch execution with checkpoints
**Which approach?**