Files
meowlib/doc/multi_device_sync_plan.md
ycc f6531e344e
Some checks failed
continuous-integration/drone/push Build is failing
MarkMessageProcessed added
2026-03-05 21:33:03 +01:00

27 KiB
Raw Blame History

Multi-Device Conversation Sync — Implementation Plan

Context

meowlib already has scaffolding for multi-device sync:

Existing artefact Where
Identity.Device *KeyPair client/identity.go:35
Identity.OwnedDevices PeerList client/identity.go:40
Peer.Type string client/peer.go:52
ToServerMessage.device_messages (field 10) pb/messages.proto:75
FromServerMessage.device_messages (field 9) pb/messages.proto:99
BackgroundJob.Device *KeyPair client/identity.go:334

The server (server/router.go) does not yet implement device_messages routing; it goes through messages/Chat today.


Chosen Sync Scheme: Event-Driven Delta Sync over Existing Message Infrastructure

Rationale

Approach Pros Cons Verdict
Full DB sync Complete history Huge payloads, merge conflicts, wasteful
Inbox/outbox file sharing Simple to reason about File-level granularity, no dedup, breaks privacy model
Event-driven delta sync Minimal data, no merge needed, reuses existing crypto + server stack Requires dedup table

Each message event (received, sent, status change) is forwarded immediately to sibling devices through the same server infrastructure as regular peer messages. Each device maintains its own complete local DB. Convergence is eventual; dedup via ConversationStatus.Uuid.

Key Design Decisions

  1. Zero server changes required. Device sync messages are addressed to the sibling device's lookup key and travel through the existing msg:{lookup_key} Redis sorted-set on the server, returned in from_server.Chat — identical to peer messages.

  2. Device peers reuse the Peer struct with Type = "device", stored in Identity.OwnedDevices. They have their own three keypairs (MyIdentity, MyEncryptionKp, MyLookupKp) and MyPullServers.

  3. A new proto message DeviceSyncPayload is added to messages.proto. It is serialised and placed in UserMessage.Appdata; the parent UserMessage.Type is set to "device_sync". This lets the client recognise sync messages without any server-side awareness.

  4. GetRequestJobs() is extended to include device lookup keys alongside peer lookup keys for the appropriate servers, so the background poll thread picks up device sync messages without any extra call.

  5. Dedup is handled by a small SQLite table device_sync_seen (one table per identity folder, not per peer) keyed on DeviceSyncPayload.DedupId.


New Protobuf Messages

Add to pb/messages.proto before re-generating:

// Payload carried inside UserMessage.appdata for device-to-device sync.
// The enclosing UserMessage.type MUST be "device_sync".
message DeviceSyncPayload {
    string sync_type      = 1; // "msg" | "status" | "peer_update" | "identity_update" | "server_add" | "forward"
    string peer_uid       = 2; // local UID of the peer conversation on the sending device
    DbMessage db_message  = 3; // the DbMessage to replicate (sync_type "msg" / "status")
    string dedup_id       = 4; // globally unique ID (= DbMessage.status.uuid or generated)
    bytes peer_data       = 5; // JSON-encoded Peer snapshot (sync_type "peer_update")
    bytes identity_data   = 6; // JSON-encoded identity profile snapshot (sync_type "identity_update")
    bytes forward_payload = 7; // serialized UserMessage for primary to send on behalf of sibling (sync_type "forward")
    string forward_peer_uid = 8; // primary-side peer UID to forward to (sync_type "forward")
}

Run cd pb && ./protogen.sh after adding this.


Implementation Phases

Phase 1 — Device Pairing

Files to touch: client/identity.go, client/helpers/ (new file deviceHelper.go)

Goal: Allow two app instances owned by the same user to establish a shared keypair relationship, mirroring the peer invitation flow but flagging the peer as Type = "device".

1.1 Identity.InitDevicePairing(myDeviceName string, serverUids []string) (*Peer, error)

  • Identical to InvitePeer but sets peer.Type = "device".
  • Stores the resulting peer in Identity.OwnedDevices (not Peers).
  • Returns the peer so the caller can produce a ContactCard QR/file.
  • Sym + DR inherited automatically: because the implementation mirrors InvitePeer, the device peer will have MySymKey, DrKpPublic, DrKpPrivate, DrRootKey, and DrInitiator = true populated automatically. The resulting ContactCard will carry dr_root_key and dr_public_key so the answering device can initialise its own DR session via AnswerDevicePairing.

1.2 Identity.AnswerDevicePairing(myDeviceName string, receivedContact *meowlib.ContactCard) (*Peer, error)

  • Mirrors AnswerInvitation, stores in OwnedDevices.

1.3 Identity.FinalizeDevicePairing(receivedContact *meowlib.ContactCard) error

  • Mirrors FinalizeInvitation, operates on OwnedDevices.

1.4 Helper functions (new file client/helpers/deviceHelper.go)

// DevicePairingCreateMessage  wraps an invitation step-1 for a device peer.
func DevicePairingCreateMessage(peer *client.Peer, serverUid string) ([]byte, string, error)

// DevicePairingAnswerMessage  wraps invitation step-3 answer for a device peer.
func DevicePairingAnswerMessage(peer *client.Peer, serverUid string) ([]byte, string, error)

These reuse invitationCreateHelper.go/invitationAnswerHelper.go logic.

1.5 Extend PeerStorage operations for OwnedDevices

OwnedDevices is currently a PeerList (in-memory slice). This must be migrated to the same Badger-backed PeerStorage mechanism as Peers — it is no longer optional. Device peers carry a Double Ratchet session state (DrStateJson) that advances with every message sent or received. Without persistent storage the DR state is lost on restart, breaking the decryption of all subsequent messages. Add a DeviceStorage PeerStorage field to Identity with its own DbFile, and ensure StorePeer is called on the device peer after every outbound dispatch (in DispatchSyncToDevices) and after every inbound consume (in ConsumeDeviceSyncMessage), mirroring the pattern used in messageHelper.go and bgPollHelper.go for regular peers.


Phase 2 — Sync Payload Helpers

Files to touch: client/helpers/deviceHelper.go (continued), client/dbmessage.go

2.1 Build a sync message for one sibling device

// BuildDeviceSyncMessage wraps a DbMessage into a UserMessage addressed to a
// sibling device peer. The caller then calls peer.ProcessOutboundUserMessage.
func BuildDeviceSyncMessage(
    devicePeer *client.Peer,
    syncType   string,           // "msg" | "status" | "peer_event"
    peerUid    string,
    dbm        *meowlib.DbMessage,
    dedupId    string,
) (*meowlib.UserMessage, error)

Implementation:

  1. Serialise DeviceSyncPayload{SyncType, PeerUid, DbMessage, DedupId} with proto.Marshal.
  2. Create a UserMessage with Type = "device_sync", Destination = devicePeer.ContactLookupKey, Appdata = serialisedPayload.
  3. Set Status.Uuid = dedupId.

2.2 Dispatch sync to all sibling devices

// DispatchSyncToDevices sends a DeviceSyncPayload to every device peer whose
// pull server list overlaps with the available servers.
// It enqueues a SendJob per device, reusing the existing bgSendHelper queue.
func DispatchSyncToDevices(
    storagePath string,
    syncType    string,
    peerUid     string,
    dbm         *meowlib.DbMessage,
    dedupId     string,
) error

Iterates identity.OwnedDevices, builds and queues one SendJob per device (just like CreateUserMessageAndSendJob but using device peer keys and putting the message in outbox/ with a recognisable prefix, e.g. dev_{devPeerUid}_{dedupId}).

After calling peer.ProcessOutboundUserMessage for each device peer, persist the updated DR state: identity.DeviceStorage.StorePeer(devPeer) if devPeer.DrRootKey != "".

The message is packed into ToServerMessage.Messages (same field as regular chat). No server changes needed.


Phase 3 — Integrate Dispatch into Send/Receive Paths

Files to touch: client/helpers/messageHelper.go, client/helpers/bgPollHelper.go

3.1 After outbound message stored (CreateAndStoreUserMessage)

At the end of CreateAndStoreUserMessage (after peer.StoreMessage), add:

// Async: do not block the caller
go DispatchSyncToDevices(storagePath, "msg", peerUid, dbm, usermessage.Status.Uuid)

The dbm is obtained from UserMessageToDbMessage(true, usermessage, nil) (files are excluded from sync — they stay on the originating device or are re-requested).

3.2 After inbound message stored (ConsumeInboxFile)

After peer.StoreMessage(usermsg, filenames) succeeds:

dbm := client.UserMessageToDbMessage(false, usermsg, nil)
go DispatchSyncToDevices(storagePath, "msg", peer.Uid, dbm, usermsg.Status.Uuid)

3.3 After ACK status update (ReadAckMessageResponse — currently a stub)

When status timestamps (received/processed) are updated in the DB, dispatch a "status" sync with the updated DbMessage.


Phase 4 — Receive & Consume Device Sync Messages

Files to touch: client/helpers/bgPollHelper.go, new client/helpers/deviceSyncHelper.go

4.1 Extend GetRequestJobs() to include device lookup keys

In identity.go:GetRequestJobs(), after the loop over Peers, add a similar loop over OwnedDevices:

for _, devPeer := range id.OwnedDevices {
    for _, server := range devPeer.MyPullServers {
        if job, ok := srvs[server]; ok {
            job.LookupKeys = append(job.LookupKeys, devPeer.MyLookupKp)
        }
    }
}

Device messages will now arrive inside from_server.Chat alongside regular peer messages. The next step distinguishes them.

4.2 Distinguish device vs peer messages in ConsumeInboxFile

After identity.Peers.GetFromMyLookupKey(packedUserMessage.Destination) returns nil, try:

devPeer := identity.OwnedDevices.GetFromMyLookupKey(packedUserMessage.Destination)
if devPeer != nil {
    err := ConsumeDeviceSyncMessage(devPeer, packedUserMessage)
    // continue to next message
    continue
}
// original error path

4.3 ConsumeDeviceSyncMessage (new file client/helpers/deviceSyncHelper.go)

func ConsumeDeviceSyncMessage(
    devPeer *client.Peer,
    packed  *meowlib.PackedUserMessage,
) error

Steps:

  1. Decrypt with devPeer.ProcessInboundUserMessage(packed) (takes the full *PackedUserMessagenot payload, signature separately; that API was updated when the sym-encryption + double-ratchet layer was added).
  2. Check usermsg.Type == "device_sync".
  3. Deserialise DeviceSyncPayload from usermsg.Appdata.
  4. Dedup check: call IsDeviceSyncSeen(payload.DedupId). If yes, skip.
  5. Mark seen: MarkDeviceSyncSeen(payload.DedupId).
  6. Persist DR state — after decryption, if devPeer.DrRootKey != "", call identity.OwnedDevices.StorePeer(devPeer) (or the equivalent Badger-backed store) to persist the updated DrStateJson. This mirrors what ConsumeInboxFile does for regular peers.
  7. Dispatch by payload.SyncType:
    • "msg": find the local peer by payload.PeerUid, call client.StoreDeviceSyncedMessage(peer, payload.DbMessage).
    • "status": update the status fields in the existing DB row matched by payload.DbMessage.Status.Uuid.
    • "peer_update": apply payload.PeerData to the local peer record (see Phase 6).
    • "identity_update": apply payload.IdentityData to the local identity profile (see Phase 6).

4.4 StoreDeviceSyncedMessage in client/messagestorage.go

A thin wrapper around storeMessage that:

  • Marks the message as synced (a new bool field Synced in DbMessage, or use a naming convention in DbMessage.Appdata).
  • Does not trigger a second round of sync dispatch (no re-broadcast).
  • Handles absent file paths gracefully (files are not synced, only metadata).

Phase 6 — Peer Metadata and Identity Profile Sync

Files to touch: client/helpers/deviceHelper.go, client/helpers/deviceSyncHelper.go, client/identity.go

The goal is to propagate non-message data across sibling devices: peer names/avatars/settings and the identity profile. This is one-directional fan-out (whichever device makes the change dispatches to all siblings) — no merge protocol is needed because conflicts are resolved by last-write-wins (the dedupId carries a timestamp or UUID sufficient for dedup; ordering is not guaranteed but is acceptable for profile data).

6.1 Peer metadata sync (sync_type = "peer_update")

Dispatch a "peer_update" payload whenever a peer record is meaningfully mutated (name, avatar, notification settings, visibility, blocked state, etc.).

Payload: DeviceSyncPayload.PeerData is a JSON-encoded full Peer struct, including all private key material and DR state. This is safe because:

  • The device sync channel is E2E-encrypted with the same X25519 + sym + DR stack as peer messages.
  • The target server is user-owned; the operator is the user themselves.
  • The recipient is the same person on a different device.

Fields included in PeerData:

  • All keypairs in full: MyIdentity, MyEncryptionKp, MyLookupKp (private + public)
  • MySymKey — shared symmetric key for that peer's channel
  • DrKpPrivate, DrKpPublic, DrRootKey, DrInitiator, ContactDrPublicKey
  • DrStateJson — current live DR session state (see DR note below)
  • All contact keys: ContactPublicKey, ContactEncryption, ContactLookupKey, ContactPullServers
  • All metadata: Name, Avatar, Avatars, MyName, Visible, Blocked, MessageNotification, SendDeliveryAck, SendProcessingAck, CallsAllowed, server lists, etc.

Fields excluded from PeerData:

  • dbPassword — transient in-memory field, never serialised; the receiving device uses its own memory password.

The receiving device upserts the peer into its local Peers store. After applying the sync, the sibling device is a full participant in the conversation: it can send and receive messages using the replicated keypairs, has the same DR session state, and monitors the same lookup key queues.

DR state sync (Phase 6 only): Syncing DrStateJson as part of "peer_update" gives sibling devices a working DR session at the point of pairing and keeps them in sync during normal single-active-device use. Phase 7 supersedes this with independent per-device DR sessions, eliminating all shared-state concerns. If Phase 7 is implemented, the DrStateJson field in PeerData can be omitted from the sync payload (each device initialises its own fresh session via the device introduction flow).

New peers: When Device A completes an invitation with a new contact, it dispatches "peer_update" to all siblings with the full peer record. Device B immediately becomes a full participant — same keypairs, same lookup key, same DR session start state — and can transparently send and receive messages with that contact without any secondary invitation.

6.2 Identity profile sync (sync_type = "identity_update")

Dispatch an "identity_update" whenever Identity.Nickname, Identity.DefaultAvatar, Identity.Avatars, or Identity.Status changes.

Payload: DeviceSyncPayload.IdentityData is a JSON-encoded subset of Identity:

type IdentityProfileSnapshot struct {
    Nickname      string   `json:"nickname"`
    DefaultAvatar string   `json:"default_avatar"`
    Avatars       []Avatar `json:"avatars"`
    Status        string   `json:"status"`
}

The receiving device deserialises this and updates only the listed fields on its local Identity, then calls identity.Save().

Explicitly NOT synced in IdentityData:

  • RootKp — the user's root signing keypair is the trust anchor; it should be established once per identity creation and never transmitted, even over a secure channel. Compromise of the root key invalidates the entire identity.
  • Device — device-specific keypair for server auth; each device has its own.
  • OwnedDevices — the device mesh itself; managed separately by the pairing flow.
  • HiddenPeers — sensitive by design; out of scope.
  • DefaultDbPassword, DbPasswordStore — local security preferences.
  • MessageServers / Peers — covered by their own sync types ("server_add", "peer_update").

6.3 Server list sync (future — sync_type = "server_add")

When a new MessageServer is added to one device's MessageServers, dispatch "server_add" so all siblings discover it. Implementation deferred; placeholder sync_type reserved.

6.4 Dispatch hooks

  • After Identity.InvitePeer / FinalizeInvitation / any peer metadata update: call DispatchSyncToDevices(..., "peer_update", peer.Uid, nil, uuid.New().String()).
  • After Identity.Save() when profile fields changed: call DispatchSyncToDevices(..., "identity_update", "", nil, uuid.New().String()).

Phase 7 — Per-Device DR Sessions (Bullet-proof Forward Secrecy)

Goal: Eliminate the concurrent-send DR race without shared ratchet state and without leaking device count to contacts.

7.0 Privacy constraint

The naive per-device DR approach (introduce all devices to all contacts) has a fundamental privacy problem: every contact learns how many devices you own and receives session material for each. This leaks metadata — device count, device rotation events, possibly device fingerprints. This is unacceptable for a privacy-first library.

Two architecturally sound options are described below. Option B (primary device relay) is recommended because it preserves complete contact-side opacity and requires no protocol extension on the contact side.


Each device is introduced to all contacts via a "device_introduce" message. The contact maintains one independent DR session per device and sends a separate encrypted copy per device on every message.

Property Value
DR race Eliminated
Contact privacy Contacts learn device count and session keys
Contact protocol change Required (handle DeviceInfo list, multi-destination send)
Backward compatibility Old clients can't participate
Server changes None

This is Signal's model. It is appropriate when contacts are expected to be aware of device multiplicity (e.g. a closed ecosystem). It is not appropriate for meowlib's open, privacy-first design.


The device that owns the peer relationship (the one whose keypairs are in the Peer record — call it the primary) is the only device that ever communicates directly with a contact. Its DR session with the contact is singular, unshared, and advances normally.

Sibling devices that want to send a message do so by dispatching a "forward" device sync payload to the primary. The primary re-encrypts with the contact's keys and forwards. From the contact's perspective: one sender, one DR session, zero device awareness.

Property Value
DR race Eliminated (only primary drives the DR session)
Contact privacy Contact is completely unaware of sibling devices
Contact protocol change None required
Backward compatibility Full
Server changes None
Trade-off If primary is offline, sibling outbound messages queue until it returns
7.1 Primary device designation

The device that completes the invitation flow for a peer (calls InvitePeer or FinalizeInvitation) is the primary for that peer. The Peer record synced to sibling devices carries a PrimaryDeviceUid string field (the UID of the device peer that "owns" this peer relationship):

// Add to Peer struct:
PrimaryDeviceUid string `json:"primary_device_uid,omitempty"`
// empty = this device IS the primary for this peer

When a sibling device receives a "peer_update" sync, it sets PrimaryDeviceUid to the sender's device UID. When the primary device sends a peer update, it leaves PrimaryDeviceUid empty (it is the primary).

7.2 New sync type: "forward"

Add to DeviceSyncPayload.sync_type:

"forward" — sibling device requests primary to send a message to a peer on its behalf

New fields needed in DeviceSyncPayload:

bytes forward_payload = 7; // serialized UserMessage (plaintext, will be encrypted by primary)
string forward_peer_uid = 8; // local peer UID on the primary device to forward to
7.3 Send path on a sibling device

When a sibling device (one where peer.PrimaryDeviceUid != "") sends a message to peer P:

  1. Build the UserMessage normally.
  2. Do not call peer.ProcessOutboundUserMessage — the sibling does not have a valid DR state for the contact.
  3. Serialize the UserMessage (plaintext proto bytes).
  4. Build a DeviceSyncPayload{SyncType: "forward", ForwardPayload: serialized, ForwardPeerUid: peer.Uid}.
  5. Dispatch it to the primary device via the normal device sync send path.
  6. Store the message locally with a "pending_forward" status so the UI reflects it immediately.
7.4 Receive and forward path on the primary device

When ConsumeDeviceSyncMessage on the primary sees sync_type == "forward":

  1. Deserialize ForwardPayload into a UserMessage.
  2. Locate the local peer by ForwardPeerUid.
  3. Call peer.ProcessOutboundUserMessage(userMessage) — primary uses its DR session normally.
  4. Enqueue a SendJob to deliver to the contact's server (same path as any outbound message).
  5. Dispatch a "msg" sync back to all siblings with the now-stored DbMessage so they update the message status from "pending_forward" to sent.
7.5 Offline queuing

If the primary device is offline when the sibling dispatches a "forward" sync, the sync message sits in the device sync queue on the server (same Redis sorted-set as all device messages). When the primary comes back online and polls, it picks up the forwarded message and delivers it. No message is lost; latency equals the primary's offline window.

7.6 Result
  • Zero contact protocol changes. Contacts cannot distinguish a primary-only device from a multi-device user.
  • No device count leakage. Device topology is fully opaque to the outside world.
  • No DR race. The primary drives a single ratchet per contact.
  • No server changes.
  • ProcessOutboundUserMessage signature stays (*PackedUserMessage, error) — no ripple through callers.
  • Trade-off is well-bounded: forward latency ≤ primary polling interval, which is already the existing long-poll timeout.

Phase 5 — Dedup Store

Files to touch: new client/devicesyncdedup.go

A single SQLite DB per identity folder: {StoragePath}/{IdentityUuid}/devicesync.db.

Schema:

CREATE TABLE IF NOT EXISTS seen (
    id        TEXT NOT NULL PRIMARY KEY,
    seen_at   INTEGER NOT NULL
);

Functions:

func IsDeviceSyncSeen(storagePath, identityUuid, dedupId string) (bool, error)
func MarkDeviceSyncSeen(storagePath, identityUuid, dedupId string) error
func PruneDeviceSyncSeen(storagePath, identityUuid string, olderThan time.Duration) error

PruneDeviceSyncSeen is called periodically (e.g. weekly) from the background thread to remove entries older than 30 days.


File Change Summary

File Change
pb/messages.proto Add DeviceSyncPayload message (with peer_data and identity_data fields)
pb/protogen.sh → re-run Regenerate .pb.go
client/identity.go Add InitDevicePairing, AnswerDevicePairing, FinalizeDevicePairing; add DeviceStorage PeerStorage field; extend GetRequestJobs(); add profile-change dispatch hooks
client/peer.go No changes needed (Type field already exists)
client/messagestorage.go Add StoreDeviceSyncedMessage
client/devicesyncdedup.go New — dedup SQLite helpers
client/helpers/deviceHelper.go NewBuildDeviceSyncMessage, DispatchSyncToDevices (msg + peer_update + identity_update), pairing message helpers
client/helpers/deviceSyncHelper.go NewConsumeDeviceSyncMessage (handles all sync types)
client/helpers/messageHelper.go Add DispatchSyncToDevices call after outbound store; detect primary vs sibling role on send
client/helpers/bgPollHelper.go Add device message detection in ConsumeInboxFile
client/peer.go Add PrimaryDeviceUid string field; sibling send path dispatches "forward" instead of direct send
client/helpers/deviceSyncHelper.go Handle "forward" sync type: deserialize, re-encrypt, enqueue SendJob, dispatch "msg" sync back

Server package: no changes required.


Sync Scope

Data Synced Notes
Message text / data In DbMessage.Data
Outbound flag In DbMessage.Outbound
Message UUID Via ConversationStatus.Uuid
Sent/received timestamps In ConversationStatus
File content Not synced; only FilePaths metadata synced
Peer full keypairs (private + public) Phase 6 — included in "peer_update" PeerData; channel is E2E-encrypted on user-owned server
Peer symmetric key Phase 6 — included in "peer_update" PeerData
Peer DR session state (DrStateJson) Phase 6 — synced on peer_update; Phase 7 (Option B) eliminates the need: primary drives one DR session, siblings never touch it
Peer metadata (name, avatar, settings) Phase 6 — "peer_update" sync type
New peer (unknown to sibling) Full peer record synced; sibling becomes immediate full participant
Identity profile (nickname, avatar, status) Phase 6 — "identity_update" sync type
Identity root keypair (RootKp) Trust anchor; never transmitted even over secure channel
Known/message server list ⚠️ Future — "server_add" placeholder reserved
Hidden peers Hidden by design; out of scope
Device keypair Per-device; each device authenticates to servers with its own key

Privacy Properties

  • Device sync messages are end-to-end encrypted (same X25519 + sym + DR stack as peer messages).
  • The server sees only the device lookup key as destination; it cannot distinguish sync messages from peer messages.
  • Including device lookup keys in batch pull requests does not leak which device belongs to you (same privacy model as multiple peer lookup keys per request).
  • OwnedDevices peers should be treated as "hidden" (not shown in contact lists) and stored in the device storage, separate from regular peers.
  • Contacts are never made aware of device count or device identity (Phase 7 Option B). The primary device relay model means the outside world observes exactly one sender per user identity, regardless of how many devices are active.
  • The device mesh topology (which devices exist, how many) is known only to the user's own devices, and is carried exclusively over the E2E-encrypted device sync channel on the user-owned server.

Testing Strategy

  1. Unit tests for DeviceSyncPayload serialisation round-trip.
  2. Unit tests for dedup store (seen/mark/prune lifecycle).
  3. Integration test extending TestEndToEnd:
    • Create identity, two device peers (DeviceA, DeviceB).
    • Send a message on DeviceA.
    • Verify DeviceB's DB contains the synced message after ConsumeDeviceSyncMessage.
    • Resend the same dedup_id — verify no duplicate row created.
  4. Integration test for inbound sync:
    • DeviceA receives a peer message.
    • Verify DeviceB gets the sync and stores it correctly.