sladro e071f73490 feat: add backend-owned terminal reconnect restore

2026-04-06 11:19:01 +08:00

20 KiB

Raw Blame History

Terminal Reconnect Recovery Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Add backend-owned reconnect restore state and a raw terminal I/O journal so ordinary shell sessions reconnect with visible input and output intact.

Architecture: Keep the existing helper-backed ConPTY runtime and websocket transport, but separate session runtime, raw journal, and restore state. Move reconnect truth to the backend by emitting an explicit restore payload during websocket attach, while the Flutter client applies that payload as authoritative terminal state instead of inferring from output replay alone.

Tech Stack: ASP.NET Core, WebSocket, helper-backed ConPTY, Flutter, Riverpod, xterm, JSON lines storage, xUnit, Flutter widget tests.

Task 1: Add Raw Terminal Journal Models And Storage

Files:

Create: apps/windows_agent/src/TermRemoteCtl.Agent/History/SessionIoEvent.cs
Create: apps/windows_agent/src/TermRemoteCtl.Agent/History/SessionIoJournalStore.cs
Modify: apps/windows_agent/src/TermRemoteCtl.Agent/Configuration/AgentOptions.cs
Modify: apps/windows_agent/src/TermRemoteCtl.Agent/Program.cs
Test: apps/windows_agent/tests/TermRemoteCtl.Agent.Tests/Sessions/SessionRegistryTests.cs
Step 1: Write the failing journal storage tests

[Fact]
public async Task AppendIoEventAsync_Persists_Input_And_Output_In_Order()
{
    using var harness = SessionRegistryHarness.Create();
    var store = new SessionIoJournalStore(harness.DataRoot);

    await store.AppendAsync(new SessionIoEvent("session-1", 1, "input", "dir", DateTimeOffset.UtcNow), CancellationToken.None);
    await store.AppendAsync(new SessionIoEvent("session-1", 2, "output", "dir\r\n", DateTimeOffset.UtcNow), CancellationToken.None);

    var lines = await File.ReadAllLinesAsync(Path.Combine(harness.DataRoot, "sessions", "session-1.io.jsonl"));
    Assert.Equal(2, lines.Length);
    Assert.Contains("\"kind\":\"input\"", lines[0]);
    Assert.Contains("\"kind\":\"output\"", lines[1]);
}

Step 2: Run test to verify it fails

Run: dotnet test apps/windows_agent/tests/TermRemoteCtl.Agent.Tests/TermRemoteCtl.Agent.Tests.csproj --filter AppendIoEventAsync_Persists_Input_And_Output_In_Order Expected: FAIL with missing SessionIoJournalStore or SessionIoEvent

Step 3: Add the journal event record

namespace TermRemoteCtl.Agent.History;

public sealed record SessionIoEvent(
    string SessionId,
    long Sequence,
    string Kind,
    string Payload,
    DateTimeOffset TimestampUtc);

Step 4: Add the journal file store

public sealed class SessionIoJournalStore
{
    private readonly string _sessionRoot;

    public SessionIoJournalStore(string rootPath)
    {
        _sessionRoot = Path.Combine(rootPath, "sessions");
        Directory.CreateDirectory(_sessionRoot);
    }

    public async Task AppendAsync(SessionIoEvent ioEvent, CancellationToken cancellationToken)
    {
        var filePath = Path.Combine(_sessionRoot, $"{ioEvent.SessionId}.io.jsonl");
        var line = JsonSerializer.Serialize(ioEvent) + Environment.NewLine;
        await File.AppendAllTextAsync(filePath, line, new UTF8Encoding(false), cancellationToken).ConfigureAwait(false);
    }
}

Step 5: Add configuration and service registration

public sealed class AgentOptions
{
    public string DataRoot { get; set; } = string.Empty;
    public int RingBufferLineLimit { get; set; } = 4000;
    public int SessionJournalRetentionDays { get; set; } = 7;
}

builder.Services.AddSingleton<SessionIoJournalStore>(sp =>
{
    var options = sp.GetRequiredService<IOptions<AgentOptions>>().Value;
    return new SessionIoJournalStore(options.DataRoot);
});

Step 6: Run the backend unit test to verify it passes

Run: dotnet test apps/windows_agent/tests/TermRemoteCtl.Agent.Tests/TermRemoteCtl.Agent.Tests.csproj --filter SessionRegistryTests Expected: PASS

Step 7: Commit

git add apps/windows_agent/src/TermRemoteCtl.Agent/History/SessionIoEvent.cs apps/windows_agent/src/TermRemoteCtl.Agent/History/SessionIoJournalStore.cs apps/windows_agent/src/TermRemoteCtl.Agent/Configuration/AgentOptions.cs apps/windows_agent/src/TermRemoteCtl.Agent/Program.cs apps/windows_agent/tests/TermRemoteCtl.Agent.Tests/Sessions/SessionRegistryTests.cs
git commit -m "feat: add terminal session io journal"

Task 2: Extend Session Registry With Restore Snapshot State

Files:

Create: apps/windows_agent/src/TermRemoteCtl.Agent/Sessions/SessionRestoreSnapshot.cs
Modify: apps/windows_agent/src/TermRemoteCtl.Agent/Sessions/SessionRegistry.cs
Modify: apps/windows_agent/src/TermRemoteCtl.Agent/History/PendingInputEchoTracker.cs
Test: apps/windows_agent/tests/TermRemoteCtl.Agent.Tests/Sessions/SessionRegistryTests.cs
Step 1: Write failing restore snapshot tests

[Fact]
public void GetRestoreSnapshot_Includes_Pending_Visible_Input()
{
    using var harness = SessionRegistryHarness.Create();
    var session = harness.Registry.Create("Shell", DateTimeOffset.UtcNow);

    harness.Registry.RecordInputEcho(session.SessionId, "git status");

    var snapshot = harness.Registry.GetRestoreSnapshot(session.SessionId);

    Assert.Equal(string.Empty, snapshot.ScreenText);
    Assert.Equal("git status", snapshot.PendingInput);
    Assert.True(snapshot.Sequence > 0);
}

[Fact]
public async Task GetRestoreSnapshot_Does_Not_Duplicate_Acknowledged_Input()
{
    using var harness = SessionRegistryHarness.Create();
    var session = harness.Registry.Create("Shell", DateTimeOffset.UtcNow);

    harness.Registry.RecordInputEcho(session.SessionId, "dir\r");
    await harness.Registry.AppendOutputAsync(session.SessionId, "PS> dir\r\nnext> ", CancellationToken.None);

    var snapshot = harness.Registry.GetRestoreSnapshot(session.SessionId);

    Assert.Equal("PS> dir\r\nnext> ", snapshot.ScreenText);
    Assert.Equal(string.Empty, snapshot.PendingInput);
}

Step 2: Run test to verify it fails

Run: dotnet test apps/windows_agent/tests/TermRemoteCtl.Agent.Tests/TermRemoteCtl.Agent.Tests.csproj --filter "GetRestoreSnapshot_" Expected: FAIL with missing GetRestoreSnapshot

Step 3: Add the restore snapshot model

namespace TermRemoteCtl.Agent.Sessions;

public sealed record SessionRestoreSnapshot(
    string SessionId,
    long Sequence,
    string ScreenText,
    string PendingInput,
    int? CursorRow,
    int? CursorColumn);

Step 4: Extend SessionRegistry to track restore sequence

private readonly ConcurrentDictionary<string, long> _sequenceBySession = new();

public long NextSequence(string sessionId)
{
    return _sequenceBySession.AddOrUpdate(sessionId, 1, (_, current) => current + 1);
}

Step 5: Add restore snapshot retrieval

public SessionRestoreSnapshot GetRestoreSnapshot(string sessionId)
{
    var replay = _replayBySession.GetOrAdd(sessionId, _ => new TerminalReplayBuffer(ReplayCharacterLimit));
    var pending = _pendingInputEchoBySession.GetOrAdd(sessionId, _ => new PendingInputEchoTracker());
    var sequence = _sequenceBySession.GetOrAdd(sessionId, 1);

    return new SessionRestoreSnapshot(
        sessionId,
        sequence,
        replay.GetSnapshot(),
        pending.GetVisibleSuffix(),
        null,
        null);
}

Step 6: Update input/output mutation points

public void RecordInputEcho(string sessionId, string input)
{
    var tracker = _pendingInputEchoBySession.GetOrAdd(sessionId, _ => new PendingInputEchoTracker());
    tracker.Record(input);
    NextSequence(sessionId);
}

public async Task AppendOutputAsync(string sessionId, string chunk, CancellationToken cancellationToken)
{
    // existing history and replay mutations
    _pendingInputEchoBySession.GetOrAdd(sessionId, _ => new PendingInputEchoTracker()).ObserveOutput(chunk);
    NextSequence(sessionId);
    await _historyStore.AppendAsync(sessionId, chunk, cancellationToken).ConfigureAwait(false);
}

Step 7: Run the backend unit tests to verify they pass

Run: dotnet test apps/windows_agent/tests/TermRemoteCtl.Agent.Tests/TermRemoteCtl.Agent.Tests.csproj --filter SessionRegistryTests Expected: PASS

Step 8: Commit

git add apps/windows_agent/src/TermRemoteCtl.Agent/Sessions/SessionRestoreSnapshot.cs apps/windows_agent/src/TermRemoteCtl.Agent/Sessions/SessionRegistry.cs apps/windows_agent/src/TermRemoteCtl.Agent/History/PendingInputEchoTracker.cs apps/windows_agent/tests/TermRemoteCtl.Agent.Tests/Sessions/SessionRegistryTests.cs
git commit -m "feat: add session restore snapshot state"

Task 3: Emit Restore Payload And Journal Events On Websocket Attach

Files:

Modify: apps/windows_agent/src/TermRemoteCtl.Agent/Realtime/TerminalWebSocketHandler.cs
Modify: apps/windows_agent/src/TermRemoteCtl.Agent/Terminal/PowerShellSessionHost.cs
Test: apps/windows_agent/tests/TermRemoteCtl.Agent.IntegrationTests/Realtime/TerminalWebSocketHandlerTests.cs
Step 1: Write failing websocket restore tests

[Fact]
public async Task Reattach_Returns_Restore_Payload_With_Pending_Input()
{
    await using var fixture = new TerminalApiFixture();
    var registry = fixture.Services.GetRequiredService<SessionRegistry>();
    var session = registry.Create("Shell", DateTimeOffset.UtcNow);

    registry.RecordInputEcho(session.SessionId, "dir");

    using WebSocket socket = await fixture.Server.CreateWebSocketClient().ConnectAsync(
        new Uri($"ws://localhost/ws/terminal?sessionId={session.SessionId}"),
        CancellationToken.None);

    _ = await ReceiveTextAsync(socket, CancellationToken.None);
    var restoreFrame = await ReceiveTextAsync(socket, CancellationToken.None);

    Assert.Contains("\"type\":\"restore\"", restoreFrame);
    Assert.Contains("\"pendingInput\":\"dir\"", restoreFrame);
}

Step 2: Run test to verify it fails

Run: dotnet test apps/windows_agent/tests/TermRemoteCtl.Agent.IntegrationTests/TermRemoteCtl.Agent.IntegrationTests.csproj --filter Reattach_Returns_Restore_Payload_With_Pending_Input Expected: FAIL because only replay text is sent

Step 3: Add the restore response contract

private sealed record TerminalRestoreResponse(
    string SessionId,
    long Sequence,
    string ScreenText,
    string PendingInput,
    int? CursorRow,
    int? CursorColumn,
    string Type = "restore");

Step 4: Send restore payload after attach acknowledgement

var restore = registry.GetRestoreSnapshot(sessionId);
await SendJsonAsync(socket, new TerminalAttachResponse(sessionId), sendGate, context.RequestAborted).ConfigureAwait(false);
await SendJsonAsync(
    socket,
    new TerminalRestoreResponse(
        restore.SessionId,
        restore.Sequence,
        restore.ScreenText,
        restore.PendingInput,
        restore.CursorRow,
        restore.CursorColumn),
    sendGate,
    context.RequestAborted).ConfigureAwait(false);

Step 5: Journal websocket lifecycle and PTY traffic

await journalStore.AppendAsync(new SessionIoEvent(sessionId, registry.NextSequence(sessionId), "attach", string.Empty, DateTimeOffset.UtcNow), context.RequestAborted);

registry.RecordInputEcho(sessionId, message.Input);
await journalStore.AppendAsync(new SessionIoEvent(sessionId, registry.NextSequence(sessionId), "input", message.Input, DateTimeOffset.UtcNow), cancellationToken);
await host.WriteInputAsync(sessionId, message.Input, cancellationToken).ConfigureAwait(false);

_ = _sessionRegistry.AppendOutputAsync(args.SessionId, args.Chunk, CancellationToken.None);
_ = _journalStore.AppendAsync(new SessionIoEvent(args.SessionId, _sessionRegistry.NextSequence(args.SessionId), "output", args.Chunk, DateTimeOffset.UtcNow), CancellationToken.None);

Step 6: Run integration tests to verify they pass

Run: dotnet test apps/windows_agent/tests/TermRemoteCtl.Agent.IntegrationTests/TermRemoteCtl.Agent.IntegrationTests.csproj --filter TerminalWebSocketHandlerTests Expected: PASS

Step 7: Commit

git add apps/windows_agent/src/TermRemoteCtl.Agent/Realtime/TerminalWebSocketHandler.cs apps/windows_agent/src/TermRemoteCtl.Agent/Terminal/PowerShellSessionHost.cs apps/windows_agent/tests/TermRemoteCtl.Agent.IntegrationTests/Realtime/TerminalWebSocketHandlerTests.cs
git commit -m "feat: send terminal restore payload on attach"

Task 4: Teach Flutter To Restore From Backend Snapshot

Files:

Create: apps/mobile_app/lib/features/terminal/terminal_restore_payload.dart
Modify: apps/mobile_app/lib/features/terminal/terminal_socket_session.dart
Modify: apps/mobile_app/lib/features/terminal/terminal_session_coordinator.dart
Modify: apps/mobile_app/lib/features/terminal/terminal_page.dart
Test: apps/mobile_app/test/features/terminal/terminal_socket_session_test.dart
Test: apps/mobile_app/test/features/terminal/terminal_session_coordinator_test.dart
Test: apps/mobile_app/test/widget_test.dart
Step 1: Write failing Flutter restore tests

testWidgets('terminal reconnect applies restore payload before live frames', (tester) async {
  final transportFactory = _QueuedTerminalSocketTransportFactory(
    connectionStartupFrames: [
      const [
        _StartupFrame('{"type":"attached","sessionId":"session-1"}'),
        _StartupFrame('{"type":"restore","sessionId":"session-1","sequence":4,"screenText":"PS> gi","pendingInput":"t status"}'),
      ],
    ],
  );

  await _pumpTerminalPage(
    tester,
    session: _session('session-1', 'codex-main'),
    socketFactory: TerminalSocketSessionFactory(transportFactory: transportFactory.create),
  );

  final terminal = tester.widget<TerminalView>(find.byType(TerminalView)).terminal;
  expect(terminal.buffer.getText(), contains('PS> git status'));
});

Step 2: Run test to verify it fails

Run: C:\tools\flutter\bin\flutter.bat test test/features/terminal/terminal_socket_session_test.dart test/widget_test.dart Expected: FAIL because restore frames are not parsed separately

Step 3: Add restore payload model

class TerminalRestorePayload {
  const TerminalRestorePayload({
    required this.sessionId,
    required this.sequence,
    required this.screenText,
    required this.pendingInput,
  });

  factory TerminalRestorePayload.fromJson(Map<String, dynamic> json) {
    return TerminalRestorePayload(
      sessionId: json['sessionId'] as String,
      sequence: json['sequence'] as int,
      screenText: (json['screenText'] as String?) ?? '',
      pendingInput: (json['pendingInput'] as String?) ?? '',
    );
  }
}

Step 4: Parse restore frames separately from live output

Future<void> connect({
  required void Function(String frame) onFrame,
  required void Function(TerminalRestorePayload restore) onRestore,
  void Function()? onDisconnected,
})

if (decoded is Map && decoded['type'] == 'restore') {
  onRestore(TerminalRestorePayload.fromJson(Map<String, dynamic>.from(decoded)));
  return;
}

Step 5: Apply restore payload as authoritative state in TerminalPage

void _handleRestorePayload(TerminalRestorePayload restore) {
  _resetTerminalForReplay();
  final combined = restore.screenText + restore.pendingInput;
  if (combined.isNotEmpty) {
    terminal.write(combined);
  }
}

Step 6: Run Flutter tests to verify they pass

Run: C:\tools\flutter\bin\flutter.bat test test/features/terminal/terminal_page_input_test.dart test/features/terminal/terminal_socket_session_test.dart test/features/terminal/terminal_session_coordinator_test.dart test/widget_test.dart Expected: PASS

Step 7: Commit

git add apps/mobile_app/lib/features/terminal/terminal_restore_payload.dart apps/mobile_app/lib/features/terminal/terminal_socket_session.dart apps/mobile_app/lib/features/terminal/terminal_session_coordinator.dart apps/mobile_app/lib/features/terminal/terminal_page.dart apps/mobile_app/test/features/terminal/terminal_socket_session_test.dart apps/mobile_app/test/features/terminal/terminal_session_coordinator_test.dart apps/mobile_app/test/widget_test.dart
git commit -m "feat: restore terminal state from backend snapshot"

Task 5: Make Restore Snapshot The Primary Reconnect Path

Files:

Modify: apps/mobile_app/lib/features/terminal/terminal_page.dart
Modify: apps/mobile_app/test/widget_test.dart
Modify: docs/testing/manual-smoke-checklist.md
Step 1: Write failing test that proves reconnect no longer depends on output-only replay

testWidgets('terminal reconnect restores pending input without history seed fallback', (tester) async {
  final transportFactory = _QueuedTerminalSocketTransportFactory(
    connectionStartupFrames: [
      const [
        _StartupFrame('{"type":"attached","sessionId":"session-1"}'),
        _StartupFrame('{"type":"restore","sessionId":"session-1","sequence":7,"screenText":"PS> gi","pendingInput":"t status"}'),
      ],
    ],
  );

  await _pumpTerminalPage(
    tester,
    session: _session('session-1', 'codex-main'),
    apiClient: _FakeAgentApiClient(lines: const <String>[]),
    socketFactory: TerminalSocketSessionFactory(transportFactory: transportFactory.create),
  );

  final terminal = tester.widget<TerminalView>(find.byType(TerminalView)).terminal;
  expect(terminal.buffer.getText(), contains('PS> git status'));
});

Step 2: Run test to verify it fails

Run: C:\tools\flutter\bin\flutter.bat test test/widget_test.dart Expected: FAIL until restore payload becomes the primary reconnect source

Step 3: Simplify reconnect restore logic

if (connectionState == TerminalConnectionState.reconnecting) {
  _resetTerminalForReplay();
  _historySeeded = false;
  _receivedSocketFrame = false;
}

if (_receivedRestorePayload) {
  _cancelHistorySeedTimer();
  return;
}

Step 4: Update the manual smoke checklist

10. Type a partial command, background the app, reopen it, and confirm the typed command is still visible.
11. Execute a command, reconnect during output, and confirm the command is not duplicated after restore.

Step 5: Run final verification

Run: C:\tools\flutter\bin\flutter.bat test test/widget_test.dart test/features/terminal/terminal_page_input_test.dart test/features/terminal/terminal_socket_session_test.dart test/features/terminal/terminal_session_coordinator_test.dart Expected: PASS

Run: dotnet test apps/windows_agent/tests/TermRemoteCtl.Agent.Tests/TermRemoteCtl.Agent.Tests.csproj Expected: PASS

Run: dotnet test apps/windows_agent/tests/TermRemoteCtl.Agent.IntegrationTests/TermRemoteCtl.Agent.IntegrationTests.csproj --filter "TerminalWebSocketHandlerTests|SessionHistoryApiTests" Expected: PASS

Step 6: Commit

git add apps/mobile_app/lib/features/terminal/terminal_page.dart apps/mobile_app/test/widget_test.dart docs/testing/manual-smoke-checklist.md
git commit -m "refactor: make terminal restore snapshot authoritative"

Self-Review

Spec coverage:
- raw journal is covered by Tasks 1 and 3
- restore snapshot state is covered by Task 2
- websocket attach restore protocol is covered by Task 3
- Flutter restore consumption is covered by Tasks 4 and 5
Placeholder scan:
- each task contains concrete files, code, commands, and expected results
Type consistency:
- SessionIoEvent, SessionRestoreSnapshot, TerminalRestorePayload, GetRestoreSnapshot, and RecordInputEcho are used consistently across the plan

Execution Handoff

Plan complete and saved to docs/superpowers/plans/2026-04-06-terminal-reconnect-recovery.md. Two execution options:

1. Subagent-Driven (recommended) - I dispatch a fresh subagent per task, review between tasks, fast iteration

2. Inline Execution - Execute tasks in this session using executing-plans, batch execution with checkpoints

Which approach?

20 KiB Raw Blame History

Terminal Reconnect Recovery Implementation Plan

Task 1: Add Raw Terminal Journal Models And Storage

Task 2: Extend Session Registry With Restore Snapshot State

Task 3: Emit Restore Payload And Journal Events On Websocket Attach

Task 4: Teach Flutter To Restore From Backend Snapshot

Task 5: Make Restore Snapshot The Primary Reconnect Path

Self-Review

Execution Handoff

20 KiB

Raw Blame History