This commit is contained in:
yves committed 2026-11-20 22:45:37 +01:00
1 parent 006bedc728
commit 64ee239d1d
10 files changed
+1664 -23

No files matched your search

+91 -16
View File
@@ -40,7 +40,7 @@ rust/
│ ├── meow-crypto (lib) primitives : clés, sealed box, KDF, AEAD, Double Ratchet, chiffrement au repos
│ ├── meow-core (lib) protocole pur, sans E/S : cartes, invitation 4 étapes, codec de messages,
│ │ requêtes serveur scellées, authentification des pulls
│ ├── meow-store (lib) [à créer] persistance client : SQLite chiffrée (identité, contacts, serveurs,
│ ├── meow-store (lib) persistance client : sac de blocs chiffrés, espaces cachés (identité, contacts, serveurs,
│ │ sessions, messages, file d'envoi) [phase 4]
│ ├── meow-client (lib) façade asynchrone haut niveau : transport HTTP, envoi/réception,
│ │ files d'envoi, long-poll, événements
@@ -250,22 +250,97 @@ let mut events = client.subscribe(); // Strea
---
## 6. Stockage client
## 6. Stockage client (`meow-store`)
* Une base SQLite par identité (`rusqlite`, `bundled`), en mode WAL.
* Clé maîtresse = Argon2id(mot de passe, sel). Toutes les colonnes sensibles (clés, état DR,
contenu des messages) sont chiffrées en XChaCha20-Poly1305 avec la clé maîtresse et une AAD
`table‖id`. Les index ne contiennent que des identifiants aléatoires.
* Phase 4 : contacts cachés. Des emplacements de taille fixe sont chiffrés avec des clés dérivées
de mots de passe secondaires, et les emplacements vides sont remplis d'aléa indiscernable, ce
qui conserve la déniabilité de la v1.
* Phase 1–3 (provisoire, pour la CLI) : un fichier d'identité unique `*.mwid2` chiffré
(Argon2id + XChaCha20-Poly1305), sauvegardé de façon atomique (écriture puis renommage) et
protégé par un verrou exclusif `*.mwid2.lock` (`File::try_lock`). Deux processus ne peuvent pas
ouvrir la même identité, car le dernier à écrire ferait reculer l'état du ratchet. Ce fichier est
remplacé par `meow-store` en phase 4.
* Invariant : tout changement d'état (ratchet, contact) est **persisté avant** que les octets
correspondants ne quittent l'appareil.
### 6.1 Modèle de menace
L'appareil peut être saisi, une fois ou plusieurs fois, et l'utilisateur peut être contraint de
donner son mot de passe principal.
1. **Sans mot de passe**, l'attaquant ne doit apprendre ni le nombre de contacts, ni quelles
conversations sont actives (« qui sont ses proches »), ni leur taille ou leur date.
2. **Avec le mot de passe principal**, les *espaces cachés* restent invisibles : rien ne permet
de prouver qu'il en existe.
Hors périmètre : la mémoire vive d'un appareil allumé et déverrouillé, et les fuites du système
(notifications, caches d'images, clavier). Pour un utilisateur exposé, la cible est un Linux maîtrisé.
L'application limitera ces fuites, par exemple sans texte de message dans les notifications.
### 6.2 Un sac de blocs uniformes
Un fichier SQLite par identité, qui ne contient que deux tables :
* `meta` : le sel (16 octets, commun à tous les mots de passe) et les paramètres Argon2id.
Ces données sont publiques et non sensibles.
* `blocks(id, data)` : des lignes de **4096 octets** sous des identifiants **aléatoires de
32 octets**. Un bloc utilisé vaut `XChaCha20-Poly1305(clé de l'espace, morceau complété par du
bourrage, aad = id)`. Un bloc de remplissage est de l'aléa. Sans clé, on ne peut pas les distinguer.
Aucune colonne en clair ne relie deux lignes. Un *objet* (profil, contact, segment de
conversation, fichier) est découpé en blocs, et son `ObjectRef { blocks[], len }` est rangé dans
l'objet parent.
```
mot de passe ─Argon2id(sel)─► k ─HMAC─► id de la racine (1 bloc, chiffré par HKDF(k))
└─ Root { type, clé d'espace, ObjectRef de l'index }
Index { meta, contacts[{ id, état, dir }], blocs libres[] }
Dir { segments[] (du plus ancien au plus récent), fichiers[(id de message, ObjectRef)] }
```
### 6.3 Espaces
* Le mot de passe principal ouvre l'espace principal : pseudo, serveurs, contacts visibles, acks
en attente.
* Chaque autre mot de passe ouvre un **espace caché**, qui contient un ou plusieurs contacts avec
leurs conversations et leurs propres acks en attente. Les données d'un espace caché ne sont
jamais rangées dans l'espace principal.
* `unlock(mot de passe)` coûte une dérivation Argon2id et une lecture de la racine, qu'un espace
existe ou non. Le coût est dominé par Argon2id, et il n'y a pas de boucle d'essai sur des
emplacements. Un même champ de saisie sert au mot de passe principal comme aux mots de passe cachés.
* Changer un mot de passe ne fait que déplacer la racine. L'ancienne racine est écrasée par de
l'aléa.
### 6.4 Déni plausible et non-corrélation
* À la création, entre 128 et 1024 blocs d'aléa sont ajoutés **sans être enregistrés nulle
part**, et chaque sauvegarde en ajoute parfois quelques-uns. Tout magasin contient donc des
blocs que le mot de passe principal n'ouvre pas, et les blocs d'un espace caché se fondent parmi eux.
* Un objet réécrit change de blocs : ils sont pris au hasard parmi les blocs libres de l'espace.
Les anciens blocs sont écrasés par de l'aléa (ce qui efface aussi les anciens états de ratchet),
puis libérés. Chaque sauvegarde réécrit aussi quelques blocs libres au hasard. Seules les
racines gardent un identifiant fixe.
* On ne supprime jamais de bloc : les blocs libérés sont recyclés.
* Le WAL est vidé (`wal_checkpoint(TRUNCATE)`) après chaque sauvegarde, et `secure_delete` est
actif.
### 6.5 Segments
Une conversation est une suite de segments d'environ 16 Kio. Ajouter un message ne réécrit que
le dernier segment, et l'historique se lit segment par segment, du plus récent au plus ancien.
C'est l'équivalent des bases de messages de taille fixe de la v1, sans qu'aucun fichier ne
révèle la longueur des conversations.
### 6.6 Limites connues
* **Plusieurs saisies de l'appareil.** Avec deux copies du fichier, l'attaquant voit quels blocs
ont changé entre elles, sans savoir à quel objet ils appartiennent. Il voit aussi que la racine
principale change à chaque sauvegarde. Si un espace caché a été utilisé entre les deux copies,
ses blocs ont changé alors que le mot de passe principal ne les explique pas. C'est la limite
connue des volumes cachés de VeraCrypt : l'éviter demanderait une ORAM, trop coûteuse ici.
Il vaut donc mieux ouvrir les espaces cachés rarement.
* **Le volume total** stocké reste visible, à l'aléa de remplissage près. La date de dernière
modification du fichier aussi.
* **Le temps d'ouverture** : lire l'index d'un espace qui existe prend environ une milliseconde
de plus, ce qui est négligeable devant Argon2id.
### 6.7 Durabilité et exclusivité
* WAL avec `synchronous=FULL` et une transaction par sauvegarde. Quand `save` revient, l'état
est durable : l'invariant **persister avant d'envoyer ou d'acquitter** repose sur cette garantie.
* Un verrou exclusif `<fichier>.lock` (`File::try_lock`) empêche qu'un deuxième processus fasse
reculer l'état d'un ratchet.
* En attendant le lot 4.5, la CLI utilise encore le fichier provisoire `*.mwid2` de
`meow-client` (un profil postcard chiffré d'un seul bloc).
---
Generated
+101 -1
View File
@@ -514,6 +514,18 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "fallible-iterator"
version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2acce4a10f12dc2fb14a218589d4f1f62ef011b2d0cc4b3cb1bba8e94da14649"
[[package]]
name = "fallible-streaming-iterator"
version = "0.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7360491ce676a36bf9bb3c56c1aa791658183a54d2744120f27285738d90465a"
[[package]]
name = "fastrand"
version = "2.5.0"
@@ -550,6 +562,12 @@ version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d9c4f5dac5e15c24eb999c26181a6ca40b39fe946cbe4c263c7209467bc83af2"
[[package]]
name = "foldhash"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "77ce24cb58228fbb8aa041425bb1050850ac19177686ea6e0f41a70416f56fdb"
[[package]]
name = "form_urlencoded"
version = "1.2.2"
@@ -654,7 +672,7 @@ version = "0.15.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9229cfe53dfd69f0609a49f65461bd93001ea1ef889cd5529dd176593f5338a1"
dependencies = [
"foldhash",
"foldhash 0.1.5",
]
[[package]]
@@ -662,6 +680,18 @@ name = "hashbrown"
version = "0.17.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a"
dependencies = [
"foldhash 0.2.0",
]
[[package]]
name = "hashlink"
version = "0.12.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a596f1b20ed2cc5ecac41a164aaebc7258057060f06c0cf7a2ba3991ee7990fb"
dependencies = [
"hashbrown 0.17.1",
]
[[package]]
name = "heck"
@@ -965,6 +995,17 @@ version = "0.2.191"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "65cd87c9a6d60536580b316ad985726104730a5151c99845f2490428ef91240a"
[[package]]
name = "libsqlite3-sys"
version = "0.40.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b06e69070b7e12262b0cd90832f0a7396f142a2b330181255562decf9f0c6215"
dependencies = [
"cc",
"pkg-config",
"vcpkg",
]
[[package]]
name = "linux-raw-sys"
version = "0.12.1"
@@ -1168,6 +1209,19 @@ dependencies = [
"tracing-subscriber",
]
[[package]]
name = "meow-store"
version = "0.2.0"
dependencies = [
"meow-core",
"meow-crypto",
"postcard",
"rusqlite",
"serde",
"thiserror",
"tracing",
]
[[package]]
name = "miette"
version = "7.6.0"
@@ -1312,6 +1366,12 @@ dependencies = [
"spki",
]
[[package]]
name = "pkg-config"
version = "0.3.34"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f6b464fbc74e149a392436b17d523f769e057cb6877f6a5c4618bc6f11800548"
[[package]]
name = "poly1305"
version = "0.8.0"
@@ -1661,6 +1721,30 @@ dependencies = [
"windows-sys 0.52.0",
]
[[package]]
name = "rsqlite-vfs"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "353b74f9810d5b085b6c42c9d86a5f0fdb795553235cdd48753b32e3809d18a7"
dependencies = [
"thiserror",
]
[[package]]
name = "rusqlite"
version = "0.43.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f172c4812a19b26d1c23cdb0070e69db335503dfa378b910eb857386e5b8beba"
dependencies = [
"bitflags",
"fallible-iterator",
"fallible-streaming-iterator",
"hashlink",
"libsqlite3-sys",
"smallvec",
"sqlite-wasm-rs",
]
[[package]]
name = "rustc-hash"
version = "2.1.3"
@@ -1894,6 +1978,16 @@ dependencies = [
"der",
]
[[package]]
name = "sqlite-wasm-rs"
version = "0.6.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a6cd504cecbfe7eef649988840f9a6cb768d11085efec138f8e99f160632d867"
dependencies = [
"cc",
"rsqlite-vfs",
]
[[package]]
name = "stable_deref_trait"
version = "1.2.1"
@@ -2288,6 +2382,12 @@ version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ba73ea9cf16a25df0c8caa16c51acb937d5712a8429db78a3ee29d5dcacd3a65"
[[package]]
name = "vcpkg"
version = "0.2.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "accd4ea62f7bb7a82fe23066fb0957d48ef677f6eeb8215f372f52e48bb32426"
[[package]]
name = "version_check"
version = "0.9.5"
+1
View File
@@ -4,6 +4,7 @@ members = [
"crates/meow-proto",
"crates/meow-crypto",
"crates/meow-core",
"crates/meow-store",
"crates/meow-client",
"crates/meow-server",
"crates/meow-cli",
+8 -6
View File
@@ -15,7 +15,7 @@ Pour reprendre le travail : lire ce fichier, prendre le premier lot ⬜ dont les
### Ordre conseillé pour la suite
1. ~~**3.8** livraison au moins une fois~~ ✅
2. **4.1 → 4.5** `meow-store` (prérequis du mobile).
2. **4.5** brancher `meow-store` dans le client et la CLI (4.1 → 4.3 faits).
3. **5.5, 5.6, 5.9** file d'envoi, accusés, flux d'événements (prérequis de l'UI).
4. **7.1 → 7.3** FFI et intégration dans bastet.
5. Le reste (phases 8–10) selon les priorités.
@@ -24,6 +24,7 @@ Pour reprendre le travail : lire ce fichier, prendre le premier lot ⬜ dont les
| Date | Lots | Notes |
|---|---|---|
| 2026-11-20 | 4.1–4.3 | `meow-store` repensé après relecture du stockage v1 : objectifs anti-saisie (rien ne relie les blocs sans mot de passe, espaces cachés à un ou plusieurs contacts). Sac de blocs uniformes, espaces, segments. Un premier brouillon avec une colonne `contact` en clair a été abandonné. 6 tests de store. Pas encore branché dans le client. |
| 2026-11-20 | 3.8 | Pull non destructif + acks signés dans `MailboxAuth`, `PendingAck` enregistrés avec l'état, redélivrances dédupliquées, pagination (`more`), mise en retraite des mailbox disparues. Redis : liste → stream. Profil client au format 2 (les `.mwid2` de format 1 sont refusés). 31 tests, e2e mémoire et Redis verts, et plus aucun message résiduel dans Redis à la fin de l'e2e. |
| 2026-11-20 | 0.x, 1.1–1.4, 2.1–2.4, 3.1–3.4, 4.0, 5.1–5.4, 6.1–6.5 | Squelette complet et fonctionnel. 29 tests Rust + `tests/e2e.sh` (≈ 60 appels CLI, 2 serveurs, mémoire et Redis, bascule quand un serveur tombe). Décision : protocole v2 sans compatibilité v1. |
@@ -77,11 +78,12 @@ Pour reprendre le travail : lire ce fichier, prendre le premier lot ⬜ dont les
| Lot | Contenu | Fin | État |
|---|---|---|---|
| 4.0 | *Provisoire* : fichier d'identité chiffré unique (`*.mwid2`) dans `meow-client`, avec écriture atomique et verrou exclusif | La CLI fonctionne | ✅ |
| 4.1 | Schéma SQLite (identité, serveurs, contacts, sessions DR, messages, fichiers, file d'envoi) + migrations | Tests | ⬜ |
| 4.2 | Chiffrement applicatif des colonnes (AAD `table‖id`), changement de mot de passe | Tests | ⬜ |
| 4.3 | Contacts cachés : emplacements de taille fixe, déverrouillage par mot de passe secondaire | Tests d'indiscernabilité de taille | ⬜ |
| 4.4 | Pagination de l'historique, recherche, aperçus de fichiers | Tests | ⬜ |
| 4.5 | Bascule de `meow-client` du fichier `.mwid2` vers `meow-store` | e2e vert | ⬜ |
| 4.1 | **Sac de blocs** : un fichier SQLite `meta` + `blocks(id aléatoire, 4096 octets)`, objets découpés en blocs, réécriture vers d'autres blocs avec effacement des anciens, recyclage des blocs libres, aléa de remplissage non enregistré, migrations | Tests : uniformité des blocs, rien en clair, blocs non permutables | ✅ |
| 4.2 | **Espaces** : racine à `HMAC(Argon2id(mot de passe, sel))`, espace principal et espaces cachés (un ou plusieurs contacts), `unlock` à coût constant, changement de mot de passe | Tests : un espace caché est invisible avec le mot de passe principal | ✅ |
| 4.3 | **Segments** de conversation (≈ 16 Kio), historique page par page, fichiers reçus enregistrés avec le message (donc avant l'ack) | Tests | ✅ |
| 4.4 | Recherche dans l'historique, aperçus de fichiers, compactage du fichier (reconstruction avec l'aléa) | Tests | ⬜ |
| 4.5 | Bascule de `meow-client` vers `meow-store`, avec les espaces dans l'API (`unlock`, `create_space`, déplacement d'un contact vers un espace caché) et dans la CLI. Les espaces cachés déverrouillés sont relevés comme les autres | e2e vert | ⬜ **prochain** |
| 4.6 | Test de type « analyse d'appareil » avec plusieurs copies du fichier : mesurer ce qu'un attaquant déduit de la différence entre deux copies, et ajuster la réécriture aléatoire des blocs libres et l'ajout d'aléa (voir ARCHITECTURE §6.6) | Rapport | ⬜ |
## Phase 5 — `meow-client` (façade asynchrone)
+20
View File
@@ -0,0 +1,20 @@
[package]
description = "Meow v2 client persistence (SQLite, application-level encryption)"
name = "meow-store"
version.workspace = true
edition.workspace = true
license.workspace = true
[dependencies]
meow-core = { path = "../meow-core" }
meow-crypto = { path = "../meow-crypto" }
postcard = { version = "1.1.3", default-features = false, features = ["alloc"] }
rusqlite = { version = "0.43.0", features = ["bundled"] }
serde = { version = "1.0.229", features = ["derive"] }
thiserror = "2.0.21"
tracing = "0.1.44"
[lints]
workspace = true
[dev-dependencies]
+211
View File
@@ -0,0 +1,211 @@
//! Fixed-size encrypted blocks and the objects built on them.
//!
//! Every row of the `blocks` table is `(id: 32 random bytes, data: BLOCK_SIZE bytes)`.
//! A used block is `XChaCha20-Poly1305(space key, chunk padded to CAPACITY, aad = id)`;
//! a filler block is random bytes. Without the right key both are indistinguishable.
//!
//! An *object* (any serialised value) is split into chunks stored in blocks whose ids
//! are listed, with the exact length, in an [`ObjectRef`] kept by the parent object.
//! Rewriting an object always allocates other blocks: the old ones are wiped (random
//! bytes) and returned to the space's free list, so addresses do not reveal which
//! object changed.
use meow_crypto::aead::{self, Key};
use rusqlite::{params, Connection, OptionalExtension, Transaction};
use serde::{Deserialize, Serialize};
use crate::{Error, Result};
/// Size of every stored block.
pub const BLOCK_SIZE: usize = 4096;
/// Plaintext bytes per block.
pub const CAPACITY: usize = BLOCK_SIZE - aead::OVERHEAD;
pub type BlockId = [u8; 32];
/// Location and exact length of an object. Trailing blocks beyond `len` are padding.
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
pub struct ObjectRef {
pub blocks: Vec<BlockId>,
pub len: u32,
}
const AAD_LABEL: &[u8] = b"meow/v2/store/block";
/// Uniform random index in `0..n` (n > 0). The tiny modulo bias is irrelevant here.
pub(crate) fn rand_below(n: usize) -> usize {
(u64::from_le_bytes(meow_crypto::random_bytes::<8>()) % n as u64) as usize
}
pub(crate) fn random_block() -> Vec<u8> {
meow_crypto::random_bytes::<BLOCK_SIZE>().to_vec()
}
pub(crate) fn seal_block(key: &Key, id: &BlockId, chunk: &[u8]) -> Vec<u8> {
let mut plain = vec![0u8; CAPACITY];
plain[..chunk.len()].copy_from_slice(chunk);
aead::encrypt(key, &plain, &[AAD_LABEL, id.as_slice()].concat())
}
pub(crate) fn open_block(key: &Key, id: &BlockId, data: &[u8]) -> Result<Vec<u8>> {
if data.len() != BLOCK_SIZE {
return Err(Error::Corrupted("block of unexpected size".into()));
}
aead::decrypt(key, data, &[AAD_LABEL, id.as_slice()].concat())
.map_err(|_| Error::Corrupted("block fails authentication".into()))
}
pub(crate) fn read_block(conn: &Connection, id: &BlockId) -> Result<Option<Vec<u8>>> {
Ok(conn
.query_row("SELECT data FROM blocks WHERE id = ?1", [id.as_slice()], |r| r.get(0))
.optional()?)
}
pub(crate) fn write_block(tx: &Transaction, id: &BlockId, data: &[u8]) -> Result<()> {
tx.execute(
"INSERT INTO blocks (id, data) VALUES (?1, ?2)
ON CONFLICT(id) DO UPDATE SET data = excluded.data",
params![id.as_slice(), data],
)?;
Ok(())
}
/// Read and decrypt an object.
pub(crate) fn get(conn: &Connection, key: &Key, r: &ObjectRef) -> Result<Vec<u8>> {
let mut out = Vec::with_capacity(r.blocks.len() * CAPACITY);
for id in &r.blocks {
let data = read_block(conn, id)?.ok_or_else(|| Error::Corrupted("missing block".into()))?;
out.extend_from_slice(&open_block(key, id, &data)?);
if out.len() >= r.len as usize {
break;
}
}
if out.len() < r.len as usize {
return Err(Error::Corrupted("object shorter than its length".into()));
}
out.truncate(r.len as usize);
Ok(out)
}
pub(crate) fn get_value<T: serde::de::DeserializeOwned>(conn: &Connection, key: &Key, r: &ObjectRef) -> Result<T> {
let bytes = get(conn, key, r)?;
postcard::from_bytes(&bytes).map_err(|e| Error::Corrupted(format!("object: {e}")))
}
/// Writes of one space inside one transaction.
pub(crate) struct Writer<'a> {
tx: &'a Transaction<'a>,
key: &'a Key,
/// Free blocks of the space usable now.
pub free: Vec<BlockId>,
/// Blocks released during this save: wiped and freed by [`Writer::finish`].
released: Vec<BlockId>,
}
impl<'a> Writer<'a> {
pub fn new(tx: &'a Transaction<'a>, key: &'a Key, free: Vec<BlockId>) -> Self {
Self {
tx,
key,
free,
released: Vec::new(),
}
}
/// A free block of the space (random pick), or a brand new id.
fn alloc(&mut self) -> BlockId {
if self.free.is_empty() {
meow_crypto::random_bytes()
} else {
let i = rand_below(self.free.len());
self.free.swap_remove(i)
}
}
fn write_into(&mut self, bytes: &[u8], blocks: &[BlockId]) -> Result<ObjectRef> {
let mut chunks = bytes.chunks(CAPACITY);
for id in blocks {
let chunk = chunks.next().unwrap_or(&[]);
write_block(self.tx, id, &seal_block(self.key, id, chunk))?;
}
Ok(ObjectRef {
blocks: blocks.to_vec(),
len: u32::try_from(bytes.len()).map_err(|_| Error::Invalid("object too large".into()))?,
})
}
/// Store `bytes` in fresh blocks; `old` (if any) is released.
pub fn put(&mut self, bytes: &[u8], old: Option<&ObjectRef>) -> Result<ObjectRef> {
let n = bytes.len().div_ceil(CAPACITY).max(1);
let blocks: Vec<BlockId> = (0..n).map(|_| self.alloc()).collect();
let r = self.write_into(bytes, &blocks)?;
if let Some(old) = old {
self.release(old);
}
Ok(r)
}
pub fn put_value<T: Serialize>(&mut self, value: &T, old: Option<&ObjectRef>) -> Result<ObjectRef> {
let bytes = postcard::to_allocvec(value).map_err(|e| Error::Invalid(e.to_string()))?;
self.put(&bytes, old)
}
pub fn release(&mut self, r: &ObjectRef) {
self.released.extend_from_slice(&r.blocks);
}
/// Wipe released blocks and add them to the free list, rewrite a few random free
/// blocks (churn), and sometimes add filler blocks that nobody records.
pub fn finish_releases(&mut self) -> Result<()> {
for id in std::mem::take(&mut self.released) {
write_block(self.tx, &id, &random_block())?;
self.free.push(id);
}
if !self.free.is_empty() {
for _ in 0..rand_below(4) {
let id = self.free[rand_below(self.free.len())];
write_block(self.tx, &id, &random_block())?;
}
}
if rand_below(4) == 0 {
add_filler(self.tx, 1 + rand_below(4))?;
}
Ok(())
}
/// Store an object whose content depends on the free list itself (the index):
/// its blocks are taken from the free list *before* serialising it.
pub fn put_self_referencing<F>(&mut self, old: Option<&ObjectRef>, serialise: F) -> Result<ObjectRef>
where
F: Fn(&[BlockId]) -> Result<Vec<u8>>,
{
if let Some(old) = old {
// Freed now (wiped below) so they can be listed as free in the object.
for id in &old.blocks {
write_block(self.tx, id, &random_block())?;
}
self.free.extend_from_slice(&old.blocks);
}
let mut blocks = vec![self.alloc()];
loop {
let bytes = serialise(&self.free)?;
if bytes.len() <= blocks.len() * CAPACITY {
return self.write_into(&bytes, &blocks);
}
// Taking a block shrinks the free list, so this converges.
let extra = bytes.len().div_ceil(CAPACITY) - blocks.len();
for _ in 0..extra {
blocks.push(self.alloc());
}
}
}
}
/// Insert `n` random blocks that no space records.
pub(crate) fn add_filler(tx: &Transaction, n: usize) -> Result<()> {
for _ in 0..n {
let id: BlockId = meow_crypto::random_bytes();
write_block(tx, &id, &random_block())?;
}
Ok(())
}
+718
View File
@@ -0,0 +1,718 @@
//! Client persistence for Meow v2, designed to resist device seizure.
//!
//! One SQLite file per identity, used only as a **bag of uniform blocks** (`blocks.rs`):
//! fixed-size rows under random ids. Without a password nothing relates blocks to one
//! another: no contact list, no per-conversation count, size or date.
//!
//! * **Spaces.** A password opens a *space*: Argon2id(password, store salt) gives a key
//! whose HMAC is the id of the space's *root* block. The root holds the space key and
//! the location of the space *index*; the index lists the space's objects (profile,
//! contacts, conversation segments, files) and its free blocks. The main password
//! opens [`SpaceId::MAIN`]; other passwords open hidden spaces holding one or several
//! contacts. Unlocking costs one Argon2id and one lookup, found or not.
//! * **Deniability.** At creation a random number of random filler blocks is added and
//! *not recorded anywhere*; saves sometimes add more. Hidden spaces' blocks are
//! indistinguishable from them, so "blocks I cannot open" exist in every store.
//! * **Unlinkability.** A rewritten object moves to other blocks (picked at random among
//! the space's free ones); the old blocks are wiped. Saves also rewrite random free
//! blocks. Only roots keep a fixed id.
//! * **Segments.** A conversation is a list of pages of about [`PAGE_BYTES`]: appending
//! rewrites only the last page, and history is read page by page.
//! * **Durability.** WAL with `synchronous=FULL`, checkpointed after each save (no
//! history of recent writes left in the WAL). Once [`Store::save`] returns, the data
//! survives a crash: the client's "persist, then send/ack" rule relies on it.
//! * **Exclusivity.** An OS lock on `<path>.lock` is held for the store's lifetime.
//!
//! Known limit: an attacker holding several copies of the file taken at different
//! times sees which blocks changed between them (not which object they belong to).
//! See ARCHITECTURE.md §6.
#![forbid(unsafe_code)]
#![cfg_attr(not(test), warn(clippy::unwrap_used, clippy::expect_used))]
pub mod blocks;
pub mod model;
pub mod schema;
use std::collections::HashMap;
use std::path::{Path, PathBuf};
use blocks::{BlockId, ObjectRef, Writer, CAPACITY};
use meow_core::contact::{Contact, ContactId};
use meow_crypto::aead::Key;
use meow_crypto::password::KdfParams;
use rusqlite::{Connection, OptionalExtension};
use serde::{Deserialize, Serialize};
pub use model::{Batch, Content, HistoryEntry, HistoryPage, PendingAck, RetiredMailbox, SpaceId, State};
#[derive(Debug, thiserror::Error)]
pub enum Error {
#[error("wrong password")]
BadPassword,
#[error("corrupted store: {0}")]
Corrupted(String),
#[error("{0} is in use by another process")]
Locked(String),
#[error("{0} already exists")]
Exists(String),
#[error("store written by a newer version (schema {0})")]
NewerSchema(u32),
#[error("this password cannot be used")]
PasswordUnavailable,
#[error("unknown space")]
UnknownSpace,
#[error("{0}")]
Invalid(String),
#[error("i/o: {0}")]
Io(#[from] std::io::Error),
#[error("sqlite: {0}")]
Sqlite(#[from] rusqlite::Error),
#[error("crypto: {0}")]
Crypto(#[from] meow_crypto::CryptoError),
}
pub type Result<T> = std::result::Result<T, Error>;
/// Target size of a conversation segment.
pub const PAGE_BYTES: usize = 4 * CAPACITY;
/// Random filler blocks added at creation: between MIN and MIN + SPAN.
const FILLER_MIN: usize = 128;
const FILLER_SPAN: usize = 896;
// ───────────────────────────── on-disk objects ─────────────────────────────
#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)]
enum SpaceKind {
Main,
Hidden,
}
/// Single block at `HMAC(password key, label)`.
#[derive(Serialize, Deserialize)]
struct Root {
kind: SpaceKind,
space_key: [u8; 32],
index: ObjectRef,
}
#[derive(Default, Serialize, Deserialize)]
struct Index {
meta: Option<ObjectRef>,
contacts: Vec<ContactEntry>,
free: Vec<BlockId>,
}
#[derive(Clone, Serialize, Deserialize)]
struct ContactEntry {
id: ContactId,
state: ObjectRef,
dir: ObjectRef,
}
/// Per-conversation directory: segments (oldest first) and received files.
#[derive(Default, Serialize, Deserialize)]
struct Dir {
pages: Vec<ObjectRef>,
files: Vec<(Vec<u8>, ObjectRef)>,
}
/// Space-wide data other than contacts.
#[derive(Default, Serialize, Deserialize)]
struct Meta {
nickname: String,
servers: Vec<meow_core::server::ServerInfo>,
pending_acks: Vec<PendingAck>,
retired_mailboxes: Vec<RetiredMailbox>,
}
// ───────────────────────────── keys ─────────────────────────────
struct PasswordKeys {
root_id: BlockId,
root_key: Key,
}
fn password_keys(password: &str, salt: &[u8; 16], kdf: KdfParams) -> Result<PasswordKeys> {
let k = meow_crypto::password::derive_key(password.as_bytes(), salt, kdf)?;
Ok(PasswordKeys {
root_id: meow_crypto::kdf::hmac(&k.0, b"meow/v2/store/root-id"),
root_key: Key(meow_crypto::kdf::derive(&k.0, salt, "meow/v2/store/root-key")?),
})
}
fn space_block_key(space_key: &[u8; 32]) -> Result<Key> {
Ok(Key(meow_crypto::kdf::derive(space_key, &[], "meow/v2/store/blocks")?))
}
fn encode_kdf(p: KdfParams) -> Vec<u8> {
[
p.m_cost_kib.to_be_bytes(),
p.t_cost.to_be_bytes(),
p.p_cost.to_be_bytes(),
]
.concat()
}
fn decode_kdf(b: &[u8]) -> Result<KdfParams> {
let word = |i: usize| -> Result<u32> {
b.get(i..i + 4)
.and_then(|s| s.try_into().ok())
.map(u32::from_be_bytes)
.ok_or_else(|| Error::Corrupted("bad kdf parameters".into()))
};
let p = KdfParams {
m_cost_kib: word(0)?,
t_cost: word(4)?,
p_cost: word(8)?,
};
// Refuse absurd parameters from a tampered file (DoS).
if p.m_cost_kib > 4 * 1024 * 1024 || p.t_cost > 64 || p.p_cost > 16 {
return Err(Error::Corrupted("bad kdf parameters".into()));
}
Ok(p)
}
// ───────────────────────────── files ─────────────────────────────
fn lock_file(path: &Path) -> Result<std::fs::File> {
let f = std::fs::OpenOptions::new()
.create(true)
.truncate(false)
.write(true)
.open(sidecar(path, ".lock"))?;
match f.try_lock() {
Ok(()) => Ok(f),
Err(std::fs::TryLockError::WouldBlock) => Err(Error::Locked(path.display().to_string())),
Err(std::fs::TryLockError::Error(e)) => Err(Error::Io(e)),
}
}
fn sidecar(path: &Path, suffix: &str) -> PathBuf {
let mut p = path.as_os_str().to_owned();
p.push(suffix);
PathBuf::from(p)
}
/// Create the file with owner-only permissions before SQLite opens it.
fn create_private(path: &Path) -> Result<()> {
let mut o = std::fs::OpenOptions::new();
o.write(true).create_new(true);
#[cfg(unix)]
{
use std::os::unix::fs::OpenOptionsExt;
o.mode(0o600);
}
o.open(path)?;
Ok(())
}
fn connect(path: &Path) -> Result<Connection> {
let conn = Connection::open(path)?;
// Fails here (SQLITE_NOTADB) when the file is not a SQLite database.
conn.execute_batch(
"PRAGMA journal_mode = WAL;
PRAGMA synchronous = FULL;
PRAGMA secure_delete = ON;",
)
.map_err(|e| Error::Corrupted(format!("not a meow store ({e})")))?;
Ok(conn)
}
fn checkpoint(conn: &Connection) -> Result<()> {
conn.query_row("PRAGMA wal_checkpoint(TRUNCATE)", [], |_| Ok(()))?;
Ok(())
}
fn encode<T: Serialize>(v: &T) -> Result<Vec<u8>> {
postcard::to_allocvec(v).map_err(|e| Error::Invalid(e.to_string()))
}
// ───────────────────────────── store ─────────────────────────────
struct Space {
root_id: BlockId,
root_key: Key,
kind: SpaceKind,
space_key: [u8; 32],
key: Key,
index: Index,
index_ref: ObjectRef,
/// Plaintext hash of each object as last written, to skip unchanged ones.
written: HashMap<String, [u8; 32]>,
}
/// An open identity store with its unlocked spaces.
pub struct Store {
conn: Connection,
path: PathBuf,
salt: [u8; 16],
kdf: KdfParams,
spaces: Vec<Space>,
_lock: std::fs::File,
}
impl std::fmt::Debug for Store {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("Store")
.field("path", &self.path)
.field("spaces", &self.spaces.len())
.finish_non_exhaustive()
}
}
impl Store {
/// Create a new store whose main space is opened by `password`. Fails if `path`
/// exists, unless `overwrite`.
pub fn create(path: &Path, nickname: &str, password: &str, kdf: KdfParams, overwrite: bool) -> Result<Self> {
let lock = lock_file(path)?;
if path.exists() {
if !overwrite {
return Err(Error::Exists(path.display().to_string()));
}
for p in [path.to_path_buf(), sidecar(path, "-wal"), sidecar(path, "-shm")] {
if p.exists() {
std::fs::remove_file(p)?;
}
}
}
create_private(path)?;
let mut conn = connect(path)?;
schema::migrate(&mut conn)?;
let salt: [u8; 16] = meow_crypto::random_bytes();
{
let tx = conn.transaction()?;
tx.execute("INSERT INTO meta (name, value) VALUES ('salt', ?1)", [salt.as_slice()])?;
tx.execute("INSERT INTO meta (name, value) VALUES ('kdf', ?1)", [encode_kdf(kdf)])?;
blocks::add_filler(&tx, FILLER_MIN + blocks::rand_below(FILLER_SPAN))?;
tx.commit()?;
}
let mut store = Self {
conn,
path: path.to_path_buf(),
salt,
kdf,
spaces: Vec::new(),
_lock: lock,
};
store.new_space(password, SpaceKind::Main)?;
store.save(
SpaceId::MAIN,
&State {
nickname: nickname.to_string(),
..State::default()
},
Batch::default(),
)?;
Ok(store)
}
/// Open a store with its main password, migrating its schema if needed.
pub fn open(path: &Path, password: &str) -> Result<Self> {
if !path.exists() {
return Err(Error::Io(std::io::Error::new(
std::io::ErrorKind::NotFound,
format!("{} does not exist", path.display()),
)));
}
let lock = lock_file(path)?;
let mut conn = connect(path)?;
let meta = |name: &str| -> Result<Vec<u8>> {
conn.query_row("SELECT value FROM meta WHERE name = ?1", [name], |r| r.get(0))
.optional()
.map_err(|e| Error::Corrupted(format!("not a meow store ({e})")))?
.ok_or_else(|| Error::Corrupted(format!("missing {name}")))
};
let salt: [u8; 16] = meta("salt")?
.try_into()
.map_err(|_| Error::Corrupted("bad salt".into()))?;
let kdf = decode_kdf(&meta("kdf")?)?;
// Never modify a store we cannot open: unlock first, then migrate.
let keys = password_keys(password, &salt, kdf)?;
let space = load_space(&conn, keys)?.ok_or(Error::BadPassword)?;
if space.kind != SpaceKind::Main {
return Err(Error::BadPassword);
}
schema::migrate(&mut conn)?;
Ok(Self {
conn,
path: path.to_path_buf(),
salt,
kdf,
spaces: vec![space],
_lock: lock,
})
}
pub fn path(&self) -> &Path {
&self.path
}
/// Unlocked spaces, main first.
pub fn spaces(&self) -> Vec<SpaceId> {
(0..self.spaces.len()).map(SpaceId).collect()
}
/// Try `password` as a hidden-space password. Same cost whether or not a space
/// exists. Returns the space (already unlocked ones included), or `None`.
pub fn unlock(&mut self, password: &str) -> Result<Option<SpaceId>> {
let keys = password_keys(password, &self.salt, self.kdf)?;
if let Some(i) = self.spaces.iter().position(|s| s.root_id == keys.root_id) {
return Ok(Some(SpaceId(i)));
}
match load_space(&self.conn, keys)? {
Some(space) => {
self.spaces.push(space);
Ok(Some(SpaceId(self.spaces.len() - 1)))
}
None => Ok(None),
}
}
/// Create a hidden space opened by `password` (which must not open anything yet).
pub fn create_space(&mut self, password: &str) -> Result<SpaceId> {
self.new_space(password, SpaceKind::Hidden)
}
fn new_space(&mut self, password: &str, kind: SpaceKind) -> Result<SpaceId> {
let keys = password_keys(password, &self.salt, self.kdf)?;
if blocks::read_block(&self.conn, &keys.root_id)?.is_some() {
return Err(Error::PasswordUnavailable);
}
let space_key: [u8; 32] = meow_crypto::random_bytes();
self.spaces.push(Space {
root_id: keys.root_id,
root_key: keys.root_key,
kind,
key: space_block_key(&space_key)?,
space_key,
index: Index::default(),
index_ref: ObjectRef::default(),
written: HashMap::new(),
});
let id = SpaceId(self.spaces.len() - 1);
if let Err(e) = self.save(id, &State::default(), Batch::default()) {
self.spaces.pop();
return Err(e);
}
Ok(id)
}
fn space(&self, id: SpaceId) -> Result<&Space> {
self.spaces.get(id.0).ok_or(Error::UnknownSpace)
}
/// Change the password of a space. Its data is untouched: only the root moves.
pub fn change_password(&mut self, space: SpaceId, new_password: &str) -> Result<()> {
let keys = password_keys(new_password, &self.salt, self.kdf)?;
if blocks::read_block(&self.conn, &keys.root_id)?.is_some() {
return Err(Error::PasswordUnavailable);
}
let sp = self.spaces.get_mut(space.0).ok_or(Error::UnknownSpace)?;
let tx = self.conn.transaction()?;
let root = encode(&Root {
kind: sp.kind,
space_key: sp.space_key,
index: sp.index_ref.clone(),
})?;
blocks::write_block(
&tx,
&keys.root_id,
&blocks::seal_block(&keys.root_key, &keys.root_id, &root),
)?;
// The old root becomes indistinguishable filler.
blocks::write_block(&tx, &sp.root_id, &blocks::random_block())?;
tx.commit()?;
checkpoint(&self.conn)?;
sp.root_id = keys.root_id;
sp.root_key = keys.root_key;
Ok(())
}
/// Load a space's state (everything but history and files).
pub fn load(&mut self, space: SpaceId) -> Result<State> {
let sp = self.spaces.get_mut(space.0).ok_or(Error::UnknownSpace)?;
let mut state = State::default();
if let Some(r) = &sp.index.meta {
let bytes = blocks::get(&self.conn, &sp.key, r)?;
sp.written.insert("meta".into(), meow_crypto::sha256(&[&bytes]));
let m: Meta = postcard::from_bytes(&bytes).map_err(|e| Error::Corrupted(format!("meta: {e}")))?;
state.nickname = m.nickname;
state.servers = m.servers;
state.pending_acks = m.pending_acks;
state.retired_mailboxes = m.retired_mailboxes;
}
for e in &sp.index.contacts {
let bytes = blocks::get(&self.conn, &sp.key, &e.state)?;
sp.written
.insert(format!("contact/{}", e.id.0), meow_crypto::sha256(&[&bytes]));
let c: Contact = postcard::from_bytes(&bytes).map_err(|e| Error::Corrupted(format!("contact: {e}")))?;
if c.id != e.id {
return Err(Error::Corrupted("contact id mismatch".into()));
}
state.contacts.push(c);
}
Ok(state)
}
/// Persist a space's `state` and append `batch`, atomically and durably. Unchanged
/// objects are not rewritten; contacts absent from `state` are deleted with their
/// history and files.
pub fn save(&mut self, space: SpaceId, state: &State, batch: Batch) -> Result<()> {
let sp = self.spaces.get_mut(space.0).ok_or(Error::UnknownSpace)?;
let tx = self.conn.transaction()?;
let mut written = Vec::new();
let new_index_ref;
let new_free;
let new_contacts;
let new_meta;
{
let mut w = Writer::new(&tx, &sp.key, sp.index.free.clone());
let mut meta_ref = sp.index.meta.clone();
let mut contacts_idx = sp.index.contacts.clone();
// Space metadata.
let meta = encode(&Meta {
nickname: state.nickname.clone(),
servers: state.servers.clone(),
pending_acks: state.pending_acks.clone(),
retired_mailboxes: state.retired_mailboxes.clone(),
})?;
let h = meow_crypto::sha256(&[&meta]);
if sp.written.get("meta") != Some(&h) || meta_ref.is_none() {
meta_ref = Some(w.put(&meta, meta_ref.as_ref())?);
written.push(("meta".to_string(), h));
}
// Removed contacts: release everything they own.
let mut kept = Vec::with_capacity(contacts_idx.len());
for e in contacts_idx {
if state.contacts.iter().any(|c| c.id == e.id) {
kept.push(e);
continue;
}
let dir: Dir = blocks::get_value(&tx, &sp.key, &e.dir)?;
dir.pages.iter().for_each(|p| w.release(p));
dir.files.iter().for_each(|(_, f)| w.release(f));
w.release(&e.dir);
w.release(&e.state);
}
contacts_idx = kept;
// Contacts, in the state's order.
let mut entries = Vec::with_capacity(state.contacts.len());
for c in &state.contacts {
let bytes = encode(c)?;
let h = meow_crypto::sha256(&[&bytes]);
let key = format!("contact/{}", c.id.0);
let entry = match contacts_idx.iter().find(|e| e.id == c.id).cloned() {
Some(mut e) => {
if sp.written.get(&key) != Some(&h) {
e.state = w.put(&bytes, Some(&e.state))?;
written.push((key, h));
}
e
}
None => {
written.push((key, h));
ContactEntry {
id: c.id.clone(),
state: w.put(&bytes, None)?,
dir: w.put_value(&Dir::default(), None)?,
}
}
};
entries.push(entry);
}
contacts_idx = entries;
// History and files, per conversation.
type Pending = (ContactId, Vec<HistoryEntry>, Vec<(Vec<u8>, Vec<u8>)>);
let mut by_contact: Vec<Pending> = Vec::new();
for h in batch.history {
match by_contact.iter_mut().find(|(c, _, _)| *c == h.contact) {
Some(slot) => slot.1.push(h),
None => by_contact.push((h.contact.clone(), vec![h], vec![])),
}
}
for (c, mid, data) in batch.files {
match by_contact.iter_mut().find(|(x, _, _)| *x == c) {
Some(slot) => slot.2.push((mid, data)),
None => by_contact.push((c, vec![], vec![(mid, data)])),
}
}
for (cid, history, files) in by_contact {
let entry = contacts_idx
.iter_mut()
.find(|e| e.id == cid)
.ok_or_else(|| Error::Invalid(format!("history for a contact not in this space: {cid}")))?;
let mut dir: Dir = blocks::get_value(&tx, &sp.key, &entry.dir)?;
append_history(&tx, &sp.key, &mut w, &mut dir, history)?;
for (mid, data) in files {
if !dir.files.iter().any(|(m, _)| *m == mid) {
let r = w.put(&data, None)?;
dir.files.push((mid, r));
}
}
entry.dir = w.put_value(&dir, Some(&entry.dir))?;
}
w.finish_releases()?;
// The index lists the free blocks, so its own blocks are taken first.
let old_index = Some(&sp.index_ref).filter(|r| !r.blocks.is_empty());
let r = w.put_self_referencing(old_index, |free| {
encode(&Index {
meta: meta_ref.clone(),
contacts: contacts_idx.clone(),
free: free.to_vec(),
})
})?;
let root = encode(&Root {
kind: sp.kind,
space_key: sp.space_key,
index: r.clone(),
})?;
if root.len() > CAPACITY {
return Err(Error::Invalid("space index too large".into()));
}
blocks::write_block(&tx, &sp.root_id, &blocks::seal_block(&sp.root_key, &sp.root_id, &root))?;
new_index_ref = r;
new_free = w.free;
new_contacts = contacts_idx;
new_meta = meta_ref;
}
tx.commit()?;
checkpoint(&self.conn)?;
// Only now (committed) does the in-memory view change.
sp.index_ref = new_index_ref;
sp.index = Index {
meta: new_meta,
contacts: new_contacts,
free: new_free,
};
sp.written
.retain(|k, _| k == "meta" || state.contacts.iter().any(|c| *k == format!("contact/{}", c.id.0)));
sp.written.extend(written);
Ok(())
}
/// One segment of a conversation (`None`: the latest one), oldest message first.
pub fn history(&self, space: SpaceId, contact: &ContactId, page: Option<usize>) -> Result<HistoryPage> {
let sp = self.space(space)?;
let dir = self.dir(sp, contact)?;
let Some(last) = dir.pages.len().checked_sub(1) else {
return Ok(HistoryPage {
entries: vec![],
page: 0,
prev: None,
});
};
let page = page.unwrap_or(last).min(last);
let entries: Vec<HistoryEntry> = blocks::get_value(&self.conn, &sp.key, &dir.pages[page])?;
Ok(HistoryPage {
entries,
page,
prev: page.checked_sub(1),
})
}
/// Whole conversation, oldest first.
pub fn full_history(&self, space: SpaceId, contact: &ContactId) -> Result<Vec<HistoryEntry>> {
let sp = self.space(space)?;
let dir = self.dir(sp, contact)?;
let mut out = Vec::new();
for p in &dir.pages {
let mut page: Vec<HistoryEntry> = blocks::get_value(&self.conn, &sp.key, p)?;
out.append(&mut page);
}
Ok(out)
}
/// Content of a received file, if stored.
pub fn file(&self, space: SpaceId, contact: &ContactId, message_id: &[u8]) -> Result<Option<Vec<u8>>> {
let sp = self.space(space)?;
let dir = self.dir(sp, contact)?;
dir.files
.iter()
.find(|(m, _)| m == message_id)
.map(|(_, r)| blocks::get(&self.conn, &sp.key, r))
.transpose()
}
fn dir(&self, sp: &Space, contact: &ContactId) -> Result<Dir> {
let e = sp
.index
.contacts
.iter()
.find(|e| e.id == *contact)
.ok_or_else(|| Error::Invalid(format!("unknown contact {contact}")))?;
blocks::get_value(&self.conn, &sp.key, &e.dir)
}
}
/// Load a space from its root, if the password opens one.
fn load_space(conn: &Connection, keys: PasswordKeys) -> Result<Option<Space>> {
let Some(data) = blocks::read_block(conn, &keys.root_id)? else {
return Ok(None);
};
// Filler at that id (practically impossible) or a wrong key: not a root.
let Ok(plain) = blocks::open_block(&keys.root_key, &keys.root_id, &data) else {
return Ok(None);
};
let root: Root = postcard::from_bytes(&plain).map_err(|e| Error::Corrupted(format!("root: {e}")))?;
let key = space_block_key(&root.space_key)?;
let index: Index = blocks::get_value(conn, &key, &root.index)?;
Ok(Some(Space {
root_id: keys.root_id,
root_key: keys.root_key,
kind: root.kind,
space_key: root.space_key,
key,
index,
index_ref: root.index,
written: HashMap::new(),
}))
}
/// Append entries to the last segment (rewritten), opening new segments past
/// [`PAGE_BYTES`]. A message id already present in the last segment is ignored.
fn append_history(
conn: &Connection,
key: &Key,
w: &mut Writer,
dir: &mut Dir,
history: Vec<HistoryEntry>,
) -> Result<()> {
if history.is_empty() {
return Ok(());
}
let mut current: Vec<HistoryEntry> = match dir.pages.last() {
Some(r) => blocks::get_value(conn, key, r)?,
None => Vec::new(),
};
let mut current_ref = dir.pages.pop();
let mut size: usize = 0;
for e in &current {
size += encode(e)?.len();
}
for h in history {
if current.iter().any(|e| e.id == h.id) {
continue;
}
let n = encode(&h)?.len();
if !current.is_empty() && size + n > PAGE_BYTES {
let r = w.put_value(&current, current_ref.as_ref())?;
dir.pages.push(r);
current = Vec::new();
current_ref = None;
size = 0;
}
size += n;
current.push(h);
}
let r = w.put_value(&current, current_ref.as_ref())?;
dir.pages.push(r);
Ok(())
}
+92
View File
@@ -0,0 +1,92 @@
//! Persisted data model.
use meow_core::contact::{Contact, ContactId};
use meow_core::server::ServerInfo;
use meow_crypto::{SignPublic, SigningKeyPair};
use serde::{Deserialize, Serialize};
/// One entry of a conversation history.
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)]
pub struct HistoryEntry {
/// Message id (random, chosen by the sender).
pub id: Vec<u8>,
pub contact: ContactId,
pub outbound: bool,
pub sent_at_ms: u64,
pub received_at: u64,
pub content: Content,
}
/// Displayable content of a message.
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)]
pub enum Content {
Text(String),
/// For received files, the data is kept in the store ([`crate::Store::file`]).
File {
name: String,
size: u64,
},
}
/// One segment of a conversation, oldest message first.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct HistoryPage {
pub entries: Vec<HistoryEntry>,
/// Index of this page.
pub page: usize,
/// Previous (older) page, if any.
pub prev: Option<usize>,
}
/// An unlocked space of the store: [`SpaceId::MAIN`] (opened by the main password)
/// or a hidden space (opened by its own password) holding one or several contacts.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, PartialOrd, Ord)]
pub struct SpaceId(pub usize);
impl SpaceId {
pub const MAIN: SpaceId = SpaceId(0);
}
/// A message received from a server and processed (its effects are stored), whose
/// deletion the server has not confirmed yet. Sent with the next pull to that server;
/// also used to ignore a redelivery of the same message.
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)]
pub struct PendingAck {
pub server: SignPublic,
pub mailbox: Vec<u8>,
pub id: Vec<u8>,
}
/// A mailbox no longer in use (temporary invitation mailbox, removed contact) that
/// still has pending acks: its key is kept just long enough to send them.
#[derive(Clone, Serialize, Deserialize)]
pub struct RetiredMailbox {
pub key: SigningKeyPair,
pub servers: Vec<ServerInfo>,
}
/// What the client keeps in memory for one space. Saved as a whole by
/// [`crate::Store::save`]; unchanged objects are not rewritten. `nickname` and
/// `servers` are only meaningful in the main space.
#[derive(Clone, Default, Serialize, Deserialize)]
pub struct State {
pub nickname: String,
pub servers: Vec<ServerInfo>,
pub contacts: Vec<Contact>,
pub pending_acks: Vec<PendingAck>,
pub retired_mailboxes: Vec<RetiredMailbox>,
}
/// Append-only data written in the same transaction as a [`State`].
#[derive(Default)]
pub struct Batch {
pub history: Vec<HistoryEntry>,
/// Received file contents: (contact, message id, data).
pub files: Vec<(ContactId, Vec<u8>, Vec<u8>)>,
}
impl Batch {
pub fn is_empty(&self) -> bool {
self.history.is_empty() && self.files.is_empty()
}
}
+49
View File
@@ -0,0 +1,49 @@
//! Schema and migrations. `PRAGMA user_version` holds the number of applied migrations.
//!
//! Rules: append migrations, never edit a released one. The schema must stay a bag of
//! uniform blocks: never add a plaintext column that relates rows to each other.
use rusqlite::Connection;
use crate::{Error, Result};
const MIGRATIONS: &[&str] = &[
// 1 — block store (roadmap lot 4.1)
r#"
-- Public parameters: 'salt' (16 bytes, shared by every password), 'kdf' (Argon2id
-- m_cost KiB ‖ t_cost ‖ p_cost, u32 BE each).
CREATE TABLE meta (
name TEXT PRIMARY KEY,
value BLOB NOT NULL
) WITHOUT ROWID;
-- Everything else: fixed-size blocks under random ids (see blocks.rs).
CREATE TABLE blocks (
id BLOB PRIMARY KEY,
data BLOB NOT NULL
) WITHOUT ROWID;
"#,
];
/// Current schema version.
pub const VERSION: u32 = MIGRATIONS.len() as u32;
pub(crate) fn version(conn: &Connection) -> Result<u32> {
Ok(conn.query_row("PRAGMA user_version", [], |r| r.get(0))?)
}
/// Bring the schema up to date. Refuses a store written by a newer version.
pub(crate) fn migrate(conn: &mut Connection) -> Result<()> {
let current = version(conn)?;
if current > VERSION {
return Err(Error::NewerSchema(current));
}
for (i, sql) in MIGRATIONS.iter().enumerate().skip(current as usize) {
let tx = conn.transaction()?;
tx.execute_batch(sql)?;
tx.pragma_update(None, "user_version", (i + 1) as u32)?;
tx.commit()?;
tracing::debug!("store migrated to schema {}", i + 1);
}
Ok(())
}
+373
View File
@@ -0,0 +1,373 @@
use std::collections::HashMap;
use std::path::{Path, PathBuf};
use meow_core::contact::Contact;
use meow_core::server::{ServerInfo, ServerKeys};
use meow_crypto::password::KdfParams;
use meow_store::blocks::BLOCK_SIZE;
use meow_store::{Batch, Content, Error, HistoryEntry, PendingAck, SpaceId, State, Store};
const KDF: KdfParams = KdfParams::INSECURE_TEST;
const MAIN: SpaceId = SpaceId::MAIN;
fn tmp() -> (PathBuf, PathBuf) {
let dir = std::env::temp_dir().join(format!(
"meow-store-{}",
meow_crypto::random_bytes::<8>()
.iter()
.map(|b| format!("{b:02x}"))
.collect::<String>()
));
std::fs::create_dir_all(&dir).unwrap();
(dir.join("id.mwid2"), dir)
}
fn server() -> ServerInfo {
ServerInfo::from_card(&ServerKeys::generate().card("http://s", "s", "", 1)).unwrap()
}
fn contact(name: &str, s: &ServerInfo) -> Contact {
meow_core::invitation::create_offer(name, "me", vec![s.clone()], "", 1000, 3600)
.unwrap()
.contact
}
fn text(c: &Contact, t: &str) -> HistoryEntry {
HistoryEntry {
id: meow_crypto::random_bytes::<16>().to_vec(),
contact: c.id.clone(),
outbound: false,
sent_at_ms: 1,
received_at: 2,
content: Content::Text(t.into()),
}
}
fn texts(entries: &[HistoryEntry]) -> Vec<String> {
entries
.iter()
.map(|e| match &e.content {
Content::Text(t) => t.clone(),
Content::File { name, .. } => format!("file:{name}"),
})
.collect()
}
/// Every block of the file, as an attacker without password sees it.
fn blocks(path: &Path) -> HashMap<Vec<u8>, Vec<u8>> {
let c = rusqlite::Connection::open(path).unwrap();
let mut stmt = c.prepare("SELECT id, data FROM blocks").unwrap();
let rows = stmt
.query_map([], |r| Ok((r.get(0)?, r.get(1)?)))
.unwrap()
.collect::<Result<_, _>>()
.unwrap();
rows
}
#[test]
fn lifecycle_password_and_lock() {
let (path, dir) = tmp();
let store = Store::create(&path, "alice", "pw", KDF, false).unwrap();
assert!(matches!(Store::open(&path, "pw"), Err(Error::Locked(_))));
assert!(matches!(
Store::create(&path, "x", "pw", KDF, true),
Err(Error::Locked(_))
));
drop(store);
assert!(matches!(
Store::create(&path, "x", "pw", KDF, false),
Err(Error::Exists(_))
));
assert!(matches!(Store::open(&path, "nope"), Err(Error::BadPassword)));
let mut store = Store::open(&path, "pw").unwrap();
assert_eq!(store.load(MAIN).unwrap().nickname, "alice");
store.change_password(MAIN, "new").unwrap();
drop(store);
assert!(matches!(Store::open(&path, "pw"), Err(Error::BadPassword)));
let mut store = Store::open(&path, "new").unwrap();
assert_eq!(store.load(MAIN).unwrap().nickname, "alice");
drop(store);
let mut store = Store::create(&path, "bob", "pw2", KDF, true).unwrap();
assert_eq!(store.load(MAIN).unwrap().nickname, "bob");
drop(store);
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
let mode = std::fs::metadata(&path).unwrap().permissions().mode();
assert_eq!(mode & 0o077, 0, "store must not be readable by others");
}
let _ = std::fs::remove_dir_all(dir);
}
#[test]
fn state_segments_and_files() {
let (path, dir) = tmp();
let s = server();
let bob = contact("bob", &s);
let carol = contact("carol", &s);
let mut store = Store::create(&path, "alice", "pw", KDF, false).unwrap();
let mut state = store.load(MAIN).unwrap();
state.servers.push(s.clone());
state.contacts = vec![bob.clone(), carol.clone()];
state.pending_acks.push(PendingAck {
server: s.sign_key,
mailbox: vec![7; 32],
id: b"1-0".to_vec(),
});
// ~1 KiB messages: several segments.
let filler = "x".repeat(1000);
let mut batch = Batch::default();
for i in 0..30 {
batch.history.push(text(&bob, &format!("m{i:02}{filler}")));
}
let dup = batch.history[29].clone();
batch.history.push(dup); // same id: ignored
let file = HistoryEntry {
content: Content::File {
name: "a.bin".into(),
size: 3,
},
..text(&carol, "")
};
batch.files.push((carol.id.clone(), file.id.clone(), vec![1, 2, 3]));
batch.history.push(file.clone());
store.save(MAIN, &state, batch).unwrap();
// A later append goes to the same conversation.
let later = Batch {
history: vec![text(&bob, "last")],
files: vec![],
};
store.save(MAIN, &state, later).unwrap();
drop(store);
let mut store = Store::open(&path, "pw").unwrap();
let loaded = store.load(MAIN).unwrap();
assert_eq!(loaded.servers, state.servers);
assert_eq!(loaded.pending_acks, state.pending_acks);
assert_eq!(
loaded.contacts.iter().map(|c| c.name.as_str()).collect::<Vec<_>>(),
vec!["bob", "carol"]
);
let all = texts(&store.full_history(MAIN, &bob.id).unwrap());
assert_eq!(all.len(), 31);
assert!(all[0].starts_with("m00") && all[29].starts_with("m29") && all[30] == "last");
// Walk the segments from the latest one back.
let mut page = store.history(MAIN, &bob.id, None).unwrap();
assert!(page.prev.is_some(), "30 KiB of messages must span several segments");
assert_eq!(texts(&page.entries).last().unwrap(), "last");
let mut walked = page.entries.len();
while let Some(prev) = page.prev {
page = store.history(MAIN, &bob.id, Some(prev)).unwrap();
walked += page.entries.len();
}
assert_eq!(walked, 31);
assert!(texts(&page.entries)[0].starts_with("m00"));
assert_eq!(texts(&store.full_history(MAIN, &carol.id).unwrap()), vec!["file:a.bin"]);
assert_eq!(store.file(MAIN, &carol.id, &file.id).unwrap(), Some(vec![1, 2, 3]));
assert_eq!(store.file(MAIN, &bob.id, &file.id).unwrap(), None);
// Removing a contact removes its history and files; the blocks are recycled.
let before = blocks(&path).len();
let mut state = loaded;
state.contacts.retain(|c| c.name != "carol");
store.save(MAIN, &state, Batch::default()).unwrap();
assert!(store.full_history(MAIN, &carol.id).is_err());
assert_eq!(store.full_history(MAIN, &bob.id).unwrap().len(), 31);
assert!(blocks(&path).len() <= before + 4, "freed blocks are reused, not leaked");
let _ = std::fs::remove_dir_all(dir);
}
#[test]
fn hidden_spaces() {
let (path, dir) = tmp();
let s = server();
let visible = contact("visible", &s);
let secret = contact("secret", &s);
let secret2 = contact("secret2", &s);
let mut store = Store::create(&path, "alice", "main-pw", KDF, false).unwrap();
let mut main = store.load(MAIN).unwrap();
main.contacts.push(visible.clone());
store.save(MAIN, &main, Batch::default()).unwrap();
// One hidden space holding two contacts.
let h = store.create_space("hidden-pw").unwrap();
assert_ne!(h, MAIN);
let hidden = State {
contacts: vec![secret.clone(), secret2.clone()],
..State::default()
};
let batch = Batch {
history: vec![text(&secret, "for your eyes only")],
files: vec![],
};
store.save(h, &hidden, batch).unwrap();
// A contact of another space cannot receive history here.
let wrong = Batch {
history: vec![text(&secret, "x")],
files: vec![],
};
assert!(store.save(MAIN, &main, wrong).is_err());
assert!(matches!(
store.create_space("hidden-pw"),
Err(Error::PasswordUnavailable)
));
assert!(matches!(store.create_space("main-pw"), Err(Error::PasswordUnavailable)));
drop(store);
// The main password shows nothing of it.
assert!(matches!(Store::open(&path, "hidden-pw"), Err(Error::BadPassword)));
let mut store = Store::open(&path, "main-pw").unwrap();
assert_eq!(store.spaces(), vec![MAIN]);
let names = |st: &State| st.contacts.iter().map(|c| c.name.clone()).collect::<Vec<_>>();
assert_eq!(names(&store.load(MAIN).unwrap()), vec!["visible"]);
assert!(store.full_history(MAIN, &secret.id).is_err());
// Unlocking: wrong password, main password, hidden password.
assert_eq!(store.unlock("not-a-password").unwrap(), None);
assert_eq!(store.unlock("main-pw").unwrap(), Some(MAIN));
let h = store.unlock("hidden-pw").unwrap().unwrap();
assert_eq!(store.unlock("hidden-pw").unwrap(), Some(h));
assert_eq!(names(&store.load(h).unwrap()), vec!["secret", "secret2"]);
assert_eq!(
texts(&store.full_history(h, &secret.id).unwrap()),
vec!["for your eyes only"]
);
// Changing the hidden password.
store.change_password(h, "other-pw").unwrap();
drop(store);
let mut store = Store::open(&path, "main-pw").unwrap();
assert_eq!(store.unlock("hidden-pw").unwrap(), None);
let h = store.unlock("other-pw").unwrap().unwrap();
assert_eq!(names(&store.load(h).unwrap()), vec!["secret", "secret2"]);
let _ = std::fs::remove_dir_all(dir);
}
#[test]
fn an_attacker_without_password_sees_only_uniform_blocks() {
let (path, dir) = tmp();
let s = server();
let bob = contact("bob", &s);
let mut store = Store::create(&path, "alice-the-nickname", "pw", KDF, false).unwrap();
let state = State {
nickname: "alice-the-nickname".into(),
servers: vec![s],
contacts: vec![bob.clone()],
..State::default()
};
let batch = Batch {
history: vec![text(&bob, "very secret text one"), text(&bob, "very secret text two")],
files: vec![],
};
store.save(MAIN, &state, batch).unwrap();
let h = store.create_space("hidden").unwrap();
store.save(h, &State::default(), Batch::default()).unwrap();
drop(store);
// Only two tables: public parameters and blocks.
let conn = rusqlite::Connection::open(&path).unwrap();
let mut stmt = conn
.prepare("SELECT name FROM sqlite_master WHERE type = 'table' ORDER BY name")
.unwrap();
let tables: Vec<String> = stmt
.query_map([], |r| r.get(0))
.unwrap()
.collect::<Result<_, _>>()
.unwrap();
drop(stmt);
drop(conn);
assert_eq!(tables, vec!["blocks", "meta"]);
let all = blocks(&path);
assert!(all.len() > 128, "random filler is always present");
assert!(all.iter().all(|(id, data)| id.len() == 32 && data.len() == BLOCK_SIZE));
// No WAL left behind, and nothing readable in the file.
let wal = std::fs::metadata(format!("{}-wal", path.display())).map_or(0, |m| m.len());
assert_eq!(wal, 0);
let raw = std::fs::read(&path).unwrap();
for needle in ["alice-the-nickname", "very secret", "bob", "http://s"] {
assert!(
!raw.windows(needle.len()).any(|w| w == needle.as_bytes()),
"{needle:?} found in clear"
);
}
// Swapping two blocks breaks authentication (contents are bound to their id).
{
let c = rusqlite::Connection::open(&path).unwrap();
let mut ids: Vec<&Vec<u8>> = all.keys().collect();
ids.sort();
for (a, b) in ids.iter().zip(ids.iter().skip(1)) {
c.execute(
"UPDATE blocks SET data = ?1 WHERE id = ?2",
rusqlite::params![all[*b], a.as_slice()],
)
.unwrap();
}
}
assert!(Store::open(&path, "pw").is_err());
let _ = std::fs::remove_dir_all(dir);
}
#[test]
fn rewrites_move_objects_and_wipe_old_blocks() {
let (path, dir) = tmp();
let s = server();
let bob = contact("bob", &s);
let mut store = Store::create(&path, "alice", "pw", KDF, false).unwrap();
let mut state = store.load(MAIN).unwrap();
state.contacts.push(bob.clone());
store.save(MAIN, &state, Batch::default()).unwrap();
// An unchanged save touches very few blocks (root, index, churn, filler).
let a = blocks(&path);
store.save(MAIN, &state, Batch::default()).unwrap();
let b = blocks(&path);
let changed = b.iter().filter(|(id, d)| a.get(*id) != Some(*d)).count();
assert!(changed <= 2 + 2 + 3 + 4, "{changed} blocks changed for a no-op save");
// Appending a message changes blocks at places unrelated to the previous write.
store
.save(
MAIN,
&state,
Batch {
history: vec![text(&bob, "hello")],
files: vec![],
},
)
.unwrap();
let c = blocks(&path);
assert!(c.len() >= b.len(), "blocks are never deleted");
let _ = std::fs::remove_dir_all(dir);
}
#[test]
fn schema_versions_and_foreign_files() {
let (path, dir) = tmp();
drop(Store::create(&path, "alice", "pw", KDF, false).unwrap());
{
let c = rusqlite::Connection::open(&path).unwrap();
let v: u32 = c.query_row("PRAGMA user_version", [], |r| r.get(0)).unwrap();
assert_eq!(v, meow_store::schema::VERSION);
c.pragma_update(None, "user_version", 99).unwrap();
}
assert!(matches!(Store::open(&path, "pw"), Err(Error::NewerSchema(99))));
let junk = dir.join("junk.mwid2");
std::fs::write(&junk, vec![0x42; 4096]).unwrap();
assert!(matches!(Store::open(&junk, "pw"), Err(Error::Corrupted(_))));
assert!(matches!(
Store::open(&dir.join("missing.mwid2"), "pw"),
Err(Error::Io(_))
));
let _ = std::fs::remove_dir_all(dir);
}