# Multi-Device Conversation Sync — Implementation Plan

## Context

meowlib already has scaffolding for multi-device sync:

| Existing artefact | Where |
|---|---|
| `Identity.Device *KeyPair` | `client/identity.go:35` |
| `Identity.OwnedDevices PeerList` | `client/identity.go:40` |
| `Peer.Type string` | `client/peer.go:52` |
| `ToServerMessage.device_messages` (field 10) | `pb/messages.proto:75` |
| `FromServerMessage.device_messages` (field 9) | `pb/messages.proto:99` |
| `BackgroundJob.Device *KeyPair` | `client/identity.go:334` |

The server (`server/router.go`) does **not** yet implement `device_messages` routing; it goes through `messages`/`Chat` today.

---

## Chosen Sync Scheme: Event-Driven Delta Sync over Existing Message Infrastructure

### Rationale

| Approach | Pros | Cons | Verdict |
|---|---|---|---|
| Full DB sync | Complete history | Huge payloads, merge conflicts, wasteful | ❌ |
| Inbox/outbox file sharing | Simple to reason about | File-level granularity, no dedup, breaks privacy model | ❌ |
| **Event-driven delta sync** | Minimal data, no merge needed, reuses existing crypto + server stack | Requires dedup table | ✅ |

Each message event (received, sent, status change) is forwarded immediately to sibling devices through the **same server infrastructure** as regular peer messages. Each device maintains its own complete local DB. Convergence is eventual; dedup via `ConversationStatus.Uuid`.

### Key Design Decisions

1. **Zero server changes required.** Device sync messages are addressed to the sibling device's lookup key and travel through the existing `msg:{lookup_key}` Redis sorted-set on the server, returned in `from_server.Chat` — identical to peer messages.

2. **Device peers reuse the `Peer` struct** with `Type = "device"`, stored in `Identity.OwnedDevices`. They have their own three keypairs (`MyIdentity`, `MyEncryptionKp`, `MyLookupKp`) and `MyPullServers`.

3. **A new proto message `DeviceSyncPayload`** is added to `messages.proto`. It is serialised and placed in `UserMessage.Appdata`; the parent `UserMessage.Type` is set to `"device_sync"`. This lets the client recognise sync messages without any server-side awareness.

4. **`GetRequestJobs()`** is extended to include device lookup keys alongside peer lookup keys for the appropriate servers, so the background poll thread picks up device sync messages without any extra call.

5. **Dedup** is handled by a small SQLite table `device_sync_seen` (one table per identity folder, not per peer) keyed on `DeviceSyncPayload.DedupId`.

---

## New Protobuf Message

Add to `pb/messages.proto` before re-generating:

```protobuf
// Payload carried inside UserMessage.appdata for device-to-device sync.
// The enclosing UserMessage.type MUST be "device_sync".
message DeviceSyncPayload {
    string sync_type  = 1; // "msg" | "status" | "peer_event"
    string peer_uid   = 2; // local UID of the peer conversation on the sending device
    DbMessage db_message = 3; // the DbMessage to replicate
    string dedup_id   = 4; // globally unique ID (= DbMessage.status.uuid or generated)
}
```

Run `cd pb && ./protogen.sh` after adding this.

---

## Implementation Phases

### Phase 1 — Device Pairing

**Files to touch:** `client/identity.go`, `client/helpers/` (new file `deviceHelper.go`)

**Goal:** Allow two app instances owned by the same user to establish a shared keypair relationship, mirroring the peer invitation flow but flagging the peer as `Type = "device"`.

#### 1.1 `Identity.InitDevicePairing(myDeviceName string, serverUids []string) (*Peer, error)`
- Identical to `InvitePeer` but sets `peer.Type = "device"`.
- Stores the resulting peer in `Identity.OwnedDevices` (not `Peers`).
- Returns the peer so the caller can produce a `ContactCard` QR/file.

#### 1.2 `Identity.AnswerDevicePairing(myDeviceName string, receivedContact *meowlib.ContactCard) (*Peer, error)`
- Mirrors `AnswerInvitation`, stores in `OwnedDevices`.

#### 1.3 `Identity.FinalizeDevicePairing(receivedContact *meowlib.ContactCard) error`
- Mirrors `FinalizeInvitation`, operates on `OwnedDevices`.

#### 1.4 Helper functions (new file `client/helpers/deviceHelper.go`)
```go
// DevicePairingCreateMessage – wraps an invitation step-1 for a device peer.
func DevicePairingCreateMessage(peer *client.Peer, serverUid string) ([]byte, string, error)

// DevicePairingAnswerMessage – wraps invitation step-3 answer for a device peer.
func DevicePairingAnswerMessage(peer *client.Peer, serverUid string) ([]byte, string, error)
```
These reuse `invitationCreateHelper.go`/`invitationAnswerHelper.go` logic.

#### 1.5 Extend `PeerStorage` operations for OwnedDevices
`OwnedDevices` is currently a `PeerList` (in-memory slice). For scalability it should use the same Badger-backed `PeerStorage` mechanism as `Peers`. Consider adding a second `PeerStorage` field `DeviceStorage` to `Identity` with its own `DbFile`.

---

### Phase 2 — Sync Payload Helpers

**Files to touch:** `client/helpers/deviceHelper.go` (continued), `client/dbmessage.go`

#### 2.1 Build a sync message for one sibling device

```go
// BuildDeviceSyncMessage wraps a DbMessage into a UserMessage addressed to a
// sibling device peer. The caller then calls peer.ProcessOutboundUserMessage.
func BuildDeviceSyncMessage(
    devicePeer *client.Peer,
    syncType   string,           // "msg" | "status" | "peer_event"
    peerUid    string,
    dbm        *meowlib.DbMessage,
    dedupId    string,
) (*meowlib.UserMessage, error)
```

Implementation:
1. Serialise `DeviceSyncPayload{SyncType, PeerUid, DbMessage, DedupId}` with `proto.Marshal`.
2. Create a `UserMessage` with `Type = "device_sync"`, `Destination = devicePeer.ContactLookupKey`, `Appdata = serialisedPayload`.
3. Set `Status.Uuid = dedupId`.

#### 2.2 Dispatch sync to all sibling devices

```go
// DispatchSyncToDevices sends a DeviceSyncPayload to every device peer whose
// pull server list overlaps with the available servers.
// It enqueues a SendJob per device, reusing the existing bgSendHelper queue.
func DispatchSyncToDevices(
    storagePath string,
    syncType    string,
    peerUid     string,
    dbm         *meowlib.DbMessage,
    dedupId     string,
) error
```

Iterates `identity.OwnedDevices`, builds and queues one `SendJob` per device (just like `CreateUserMessageAndSendJob` but using device peer keys and putting the message in `outbox/` with a recognisable prefix, e.g. `dev_{devPeerUid}_{dedupId}`).

The message is packed into `ToServerMessage.Messages` (same field as regular chat). No server changes needed.

---

### Phase 3 — Integrate Dispatch into Send/Receive Paths

**Files to touch:** `client/helpers/messageHelper.go`, `client/helpers/bgPollHelper.go`

#### 3.1 After outbound message stored (`CreateAndStoreUserMessage`)

At the end of `CreateAndStoreUserMessage` (after `peer.StoreMessage`), add:

```go
// Async: do not block the caller
go DispatchSyncToDevices(storagePath, "msg", peerUid, dbm, usermessage.Status.Uuid)
```

The `dbm` is obtained from `UserMessageToDbMessage(true, usermessage, nil)` (files are excluded from sync — they stay on the originating device or are re-requested).

#### 3.2 After inbound message stored (`ConsumeInboxFile`)

After `peer.StoreMessage(usermsg, filenames)` succeeds:

```go
dbm := client.UserMessageToDbMessage(false, usermsg, nil)
go DispatchSyncToDevices(storagePath, "msg", peer.Uid, dbm, usermsg.Status.Uuid)
```

#### 3.3 After ACK status update (`ReadAckMessageResponse` — currently a stub)

When status timestamps (received/processed) are updated in the DB, dispatch a `"status"` sync with the updated `DbMessage`.

---

### Phase 4 — Receive & Consume Device Sync Messages

**Files to touch:** `client/helpers/bgPollHelper.go`, new `client/helpers/deviceSyncHelper.go`

#### 4.1 Extend `GetRequestJobs()` to include device lookup keys

In `identity.go:GetRequestJobs()`, after the loop over `Peers`, add a similar loop over `OwnedDevices`:

```go
for _, devPeer := range id.OwnedDevices {
    for _, server := range devPeer.MyPullServers {
        if job, ok := srvs[server]; ok {
            job.LookupKeys = append(job.LookupKeys, devPeer.MyLookupKp)
        }
    }
}
```

Device messages will now arrive inside `from_server.Chat` alongside regular peer messages. The next step distinguishes them.

#### 4.2 Distinguish device vs peer messages in `ConsumeInboxFile`

After `identity.Peers.GetFromMyLookupKey(packedUserMessage.Destination)` returns `nil`, try:

```go
devPeer := identity.OwnedDevices.GetFromMyLookupKey(packedUserMessage.Destination)
if devPeer != nil {
    err := ConsumeDeviceSyncMessage(devPeer, packedUserMessage)
    // continue to next message
    continue
}
// original error path
```

#### 4.3 `ConsumeDeviceSyncMessage` (new file `client/helpers/deviceSyncHelper.go`)

```go
func ConsumeDeviceSyncMessage(
    devPeer *client.Peer,
    packed  *meowlib.PackedUserMessage,
) error
```

Steps:
1. Decrypt with `devPeer.ProcessInboundUserMessage(packed.Payload, packed.Signature)`.
2. Check `usermsg.Type == "device_sync"`.
3. Deserialise `DeviceSyncPayload` from `usermsg.Appdata`.
4. Dedup check: call `IsDeviceSyncSeen(payload.DedupId)`. If yes, skip.
5. Mark seen: `MarkDeviceSyncSeen(payload.DedupId)`.
6. Dispatch by `payload.SyncType`:
   - `"msg"`: find the local peer by `payload.PeerUid`, call `client.StoreDeviceSyncedMessage(peer, payload.DbMessage)`.
   - `"status"`: update the status fields in the existing DB row matched by `payload.DbMessage.Status.Uuid`.
   - `"peer_event"`: (future) update peer metadata.

#### 4.4 `StoreDeviceSyncedMessage` in `client/messagestorage.go`

A thin wrapper around `storeMessage` that:
- Marks the message as synced (a new bool field `Synced` in `DbMessage`, or use a naming convention in `DbMessage.Appdata`).
- Does **not** trigger a second round of sync dispatch (no re-broadcast).
- Handles absent file paths gracefully (files are not synced, only metadata).

---

### Phase 5 — Dedup Store

**Files to touch:** new `client/devicesyncdedup.go`

A single SQLite DB per identity folder: `{StoragePath}/{IdentityUuid}/devicesync.db`.

Schema:
```sql
CREATE TABLE IF NOT EXISTS seen (
    id        TEXT NOT NULL PRIMARY KEY,
    seen_at   INTEGER NOT NULL
);
```

Functions:
```go
func IsDeviceSyncSeen(storagePath, identityUuid, dedupId string) (bool, error)
func MarkDeviceSyncSeen(storagePath, identityUuid, dedupId string) error
func PruneDeviceSyncSeen(storagePath, identityUuid string, olderThan time.Duration) error
```

`PruneDeviceSyncSeen` is called periodically (e.g. weekly) from the background thread to remove entries older than 30 days.

---

## File Change Summary

| File | Change |
|---|---|
| `pb/messages.proto` | Add `DeviceSyncPayload` message |
| `pb/protogen.sh` → re-run | Regenerate `.pb.go` |
| `client/identity.go` | Add `InitDevicePairing`, `AnswerDevicePairing`, `FinalizeDevicePairing`; extend `GetRequestJobs()` |
| `client/peer.go` | No changes needed (Type field already exists) |
| `client/messagestorage.go` | Add `StoreDeviceSyncedMessage` |
| `client/devicesyncdedup.go` | **New** — dedup SQLite helpers |
| `client/helpers/deviceHelper.go` | **New** — `BuildDeviceSyncMessage`, `DispatchSyncToDevices`, pairing message helpers |
| `client/helpers/deviceSyncHelper.go` | **New** — `ConsumeDeviceSyncMessage` |
| `client/helpers/messageHelper.go` | Add `DispatchSyncToDevices` call after outbound store |
| `client/helpers/bgPollHelper.go` | Add device message detection in `ConsumeInboxFile` |

Server package: **no changes required**.

---

## Sync Scope

| Data | Synced | Notes |
|---|---|---|
| Message text / data | ✅ | In `DbMessage.Data` |
| Outbound flag | ✅ | In `DbMessage.Outbound` |
| Message UUID | ✅ | Via `ConversationStatus.Uuid` |
| Sent/received timestamps | ✅ | In `ConversationStatus` |
| File content | ❌ | Not synced; only `FilePaths` metadata synced |
| Peer metadata (keys, servers) | ❌ | Phase 2+ scope |
| Identity blob | ❌ | Out of scope; handled by manual export |

---

## Privacy Properties

- Device sync messages are end-to-end encrypted (same X25519 + PGP as peer messages).
- The server sees only the device lookup key as destination; it has no knowledge this is a sync vs a peer message.
- Including device lookup keys in batch pull requests does not leak which other device belongs to you (same privacy model as multiple peer lookup keys per request).
- `OwnedDevices` peers should be considered "hidden" (not shown in contact lists) and can optionally be stored in the hidden peer store.

---

## Testing Strategy

1. **Unit tests** for `DeviceSyncPayload` serialisation round-trip.
2. **Unit tests** for dedup store (seen/mark/prune lifecycle).
3. **Integration test** extending `TestEndToEnd`:
   - Create identity, two device peers (DeviceA, DeviceB).
   - Send a message on DeviceA.
   - Verify DeviceB's DB contains the synced message after `ConsumeDeviceSyncMessage`.
   - Resend the same dedup_id — verify no duplicate row created.
4. **Integration test** for inbound sync:
   - DeviceA receives a peer message.
   - Verify DeviceB gets the sync and stores it correctly.