Several vaults per account (phase 3, 1.0.0-alpha.5)

README.md

@@ -13,7 +13,7 @@ package replaces it in one step (`Conflicts`/`Replaces: passman`), and your
 vaults stay in `~/.local/share/passman`, which SecretSpace keeps using —
 nothing is moved or renamed. The vault files are the same.
 
-**`main` is 1.0.0-alpha.4, a SecretSpace 3 test version — not for real passwords yet.**
+**`main` is 1.0.0-alpha.5, a SecretSpace 3 test version — not for real passwords yet.**
 For real use keep the [0.11.0 release](https://github.com/usermaneGermane/SecretSpace-Client-Linux/releases/tag/v0.11.0),
 which the rest of this README describes. It installs from a `.deb` on Ubuntu
 22.04 or newer, or runs from source. It has so far been tested in CI only,
@@ -31,20 +31,23 @@ on a virtual display, not yet on a real desktop.
 **Same vaults as on Windows.** The vault file, the sync protocol and the
 features match SecretSpace for Windows (see the version table under *What is
 different from Windows*): 0.11.0 = Windows 2.13.0, and this test version =
-Windows 3.0.0-alpha.4.
+Windows 3.0.0-alpha.5.
 A vault moves between a Windows PC and a Linux PC unchanged, and both sync
 with either server.
 
 ## SecretSpace 3 — the test version
 
-1.0.0-alpha.4 is the same as [SecretSpace for Windows
-3.0.0-alpha.4](https://github.com/usermaneGermane/SecretSpace-Client-Windows#secretspace-3--a-test-version-not-for-real-passwords-yet):
+1.0.0-alpha.5 is the same as [SecretSpace for Windows
+3.0.0-alpha.5](https://github.com/usermaneGermane/SecretSpace-Client-Windows#secretspace-3--a-test-version-not-for-real-passwords-yet):
 accounts with a Secret Key and a 12-word recovery phrase, Argon2id, and
 importing 2.x vaults with their history. It opens SecretSpace 3 accounts only,
 so do not install it over 0.11.0 — your vaults would not show up (they are
 not changed). Since alpha.3 it syncs through a SecretSpace Server 2.0
 (*Settings → Sync → Set up…*) and signs in on another device with *Sign in
-from a server*. On Linux:
+from a server*. Since alpha.5 an account can have **several vaults**
+(*Vaults* above the list): each with its own key, picked in the menu above
+the list, and all covered by **Sync** — which needs SecretSpace Server
+2.0.0-alpha.2 for more than the Personal vault. On Linux:
 
 - **The Secret Key is kept under your master password** (Argon2id, then
   AES-GCM), not by the system as Windows does with DPAPI. So a copy of the
@@ -54,7 +57,7 @@ from a server*. On Linux:
 - The account folders are `700` and their files `600`, like the vault
   folder; a saved Emergency Kit is `600` too.
 - *Settings → Backup → Open folder* opens the folder with `xdg-open`.
-- The package spells the version `1.0.0~alpha.4`, which apt sorts after
+- The package spells the version `1.0.0~alpha.5`, which apt sorts after
   0.9.0 and before 1.0.0.
 
 | File | What it is |
@@ -187,6 +190,7 @@ Where that README names a Windows version, read the Linux one beside it:
 | 3.0.0-alpha.2 | 1.0.0-alpha.2 | renamed to SecretSpace (package `secretspace`, replaces `passman`) |
 | 3.0.0-alpha.3 | 1.0.0-alpha.3 | accounts sync through SecretSpace Server 2.0; sign in from a server |
 | 3.0.0-alpha.4 | 1.0.0-alpha.4 | a smoother look: smoothed corners, redrawn sign-in backdrop |
+| 3.0.0-alpha.5 | 1.0.0-alpha.5 | several vaults per account; Sync covers them all (Server 2.0.0-alpha.2) |
 
 (0.2.0 added the `.deb` package, which has no Windows counterpart.)
 

sourceFiles/account_crypto.py

@@ -406,6 +406,49 @@ def decrypt_vault(vault_key: bytes, blob: str, expect_vault_id: bytes = None) ->
     return json.loads(body)
 
 
+# ===========================================================================
+# A vault's name, as the server keeps it
+# ===========================================================================
+#: Longest vault name, in characters and in UTF-8 bytes (the server's limit).
+VAULT_NAME_CHARS = 60
+VAULT_NAME_BYTES = 200
+
+
+def new_vault_key() -> bytes:
+    return os.urandom(KEY_LEN)
+
+
+def _name_key(vault_key: bytes, vault_id: bytes) -> bytes:
+    # Its own key, so a name and the vault's entries never share one.
+    return _hkdf(vault_key, vault_id, b"PassMan3 vault name")
+
+
+def seal_vault_name(vault_key: bytes, vault_id: bytes, name: str) -> str:
+    """
+    ``name`` under the vault's key, for the server: it lists an account's
+    vaults without learning what they are called. Bound to the vault id, so
+    one vault's name cannot be shown as another's.
+    """
+    raw = name.encode("utf-8")
+    if not raw or len(raw) > VAULT_NAME_BYTES:
+        raise CryptoError("A vault name must be 1 to 200 bytes long.")
+    nonce = os.urandom(NONCE_LEN)
+    return _b64(nonce + AESGCM(_name_key(vault_key, vault_id)).encrypt(
+        nonce, raw, vault_id))
+
+
+def open_vault_name(vault_key: bytes, vault_id: bytes, sealed: str) -> str:
+    raw = _unb64(sealed, "vault name")
+    if len(raw) < NONCE_LEN + 1 + 16:
+        raise CryptoError("The vault's name is damaged.")
+    try:
+        return AESGCM(_name_key(vault_key, vault_id)).decrypt(
+            raw[:NONCE_LEN], raw[NONCE_LEN:], vault_id).decode("utf-8")
+    except Exception:
+        raise CryptoError("The vault's name could not be read: wrong key, or "
+                          "it is damaged.") from None
+
+
 # ===========================================================================
 # The 12-word recovery phrase (BIP39, English)
 # ===========================================================================

sourceFiles/account_sync.py

@@ -11,9 +11,11 @@
 Three things live here:
 
 - :class:`AccountServer`, the HTTP calls, on sync_client's transport;
-- :class:`AccountSync`, one sync of the personal vault in two halves, with
-  the same three-way merge, conflict choices and base rules as 2.x
+- :class:`AccountSync`, one sync of one vault in two halves, with the same
+  three-way merge, conflict choices and base rules as 2.x
   (sync.merge_vaults, sync_client.base_matches);
+- :func:`reconcile_vaults` (phase 3), which makes the account's list of
+  vaults the same here and on the server before they are synced;
 - the account lifecycle around them: :func:`connect` (an account made here
   goes onto a server), :func:`sign_in` (a new device, or one catching up
   with a password changed elsewhere) and :func:`change_password`.
@@ -23,6 +25,7 @@
 """
 
 import base64
+import os
 import urllib.parse
 
 import account_crypto
@@ -37,6 +40,10 @@
 #: /health; a 1.x server answers without "api" and has only the 2.x one.
 API_VERSION = 2
 
+#: /health "features" entry of a server that keeps several vaults per
+#: account (SecretSpace Server 2.0.0-alpha.2).
+FEATURE_VAULTS = "vaults"
+
 
 class LoginFailed(SyncError):
     """The server did not take this username + password + Secret Key."""
@@ -52,6 +59,7 @@ class AccountServer(SyncClient):
     def __init__(self, url: str, ca_file: str = "",
                  timeout: int = sync_client.DEFAULT_TIMEOUT):
         super().__init__(url, "", ca_file, timeout)
+        self._features = None
 
     @staticmethod
     def _http_error(e) -> SyncError:
@@ -74,6 +82,24 @@ def check_api(self) -> None:
                             "it needs SecretSpace Server 2.0 or newer. (2.x "
                             "vaults sync with it as before.)")
 
+    def has_vaults(self) -> bool:
+        """Can this server keep more than the personal vault? Asked once
+        per connection."""
+        if self._features is None:
+            self._features = list(self.health().get("features") or [])
+        return FEATURE_VAULTS in self._features
+
+    def create_vault(self, vault_id: str, sealed_name: str, wrapped_key: str) -> dict:
+        return self._request("POST", "/v2/vaults", body={
+            "id": vault_id, "name": sealed_name, "wrapped_key": wrapped_key})
+
+    def rename_vault(self, vault_id: str, sealed_name: str) -> dict:
+        return self._request("PUT", f"/v2/vaults/{vault_id}/name",
+                             body={"name": sealed_name})
+
+    def delete_vault(self, vault_id: str) -> None:
+        self._request("DELETE", f"/v2/vaults/{vault_id}")
+
     def prelogin(self, username: str) -> dict:
         return self._request("GET", "/v2/prelogin/" + urllib.parse.quote(
             username, safe=""))
@@ -202,9 +228,12 @@ class AccountSync:
     dance here -- the vault key does not change with the master password.
     """
 
-    def __init__(self, account: accounts.Account, server: AccountServer = None):
+    def __init__(self, account: accounts.Account, server: AccountServer = None,
+                 vault_id: str = None):
         self.account = account
         self.server = server
+        # The vault this sync is for: the personal one unless told otherwise.
+        self.file = account.vault_file(vault_id)
         self.revision = 0
         self.result = None
         self.first_push = False
@@ -213,7 +242,7 @@ def __init__(self, account: accounts.Account, server: AccountServer = None):
 
     def prepare(self, local_vault: dict):
         self.server = logged_in(self.account, self.server)
-        stored = self.server.get_vault(self.account.vault_id.hex())
+        stored = self.server.get_vault(self.file.hex)
         self.revision = stored["revision"]
         self._fingerprint = sync_client.SyncSession.fingerprint(local_vault)
         blob = stored["blob"]
@@ -222,15 +251,15 @@ def prepare(self, local_vault: dict):
             remote = {"accounts": []}
         else:
             try:
-                remote = self.account.decrypt(blob)
+                remote = self.file.decrypt(blob)
             except (CryptoError, ValueError) as e:
                 raise SyncError(f"The vault on the server cannot be read: {e}") from e
         # The 2.x rules for the base (sync_client.SyncSession.prepare): none
         # when the server is empty, and none if it was agreed elsewhere.
         self.base_ignored = False
         base = None
         if blob is not None:
-            base = self.account.load_base()
+            base = self.file.load_base()
             if base is not None and not sync_client.base_matches(
                     base, self.server.url, self.revision):
                 base, self.base_ignored = None, True
@@ -251,12 +280,11 @@ def commit(self, local_vault: dict, choices=None) -> dict:
             merged_vault(local_vault, resolve(self.result, choices)))
         # Saved here first, then pushed: if the network fails in between,
         # this device holds the merge and the next sync pushes it.
-        self.account.save_vault(merged)
+        self.file.save(merged)
         if self.result.needs_push or self.first_push:
             self.revision = self.server.put_vault(
-                self.account.vault_id.hex(), self.account.encrypt(merged),
-                self.revision)
-        self.account.save_base(merged, sync_client.base_origin_for(
+                self.file.hex, self.file.encrypt(merged), self.revision)
+        self.file.save_base(merged, sync_client.base_origin_for(
             self.server.url, self.revision))
         return merged
 
@@ -265,6 +293,137 @@ def close(self) -> None:
             self.server.logout()
 
 
+# ---------------------------------------------------------------------------
+# The account's list of vaults, here and on the server
+# ---------------------------------------------------------------------------
+class VaultChanges:
+    """What :func:`reconcile_vaults` did, for the user to be told."""
+
+    def __init__(self):
+        self.added = []          # names: came from the server to this device
+        self.uploaded = []       # names: made here, now on the server too
+        self.gone = []           # names: deleted on the server, kept here
+        self.renamed = []        # (old, new): renamed on another device
+        self.server_too_old = False
+
+    def __bool__(self):
+        return bool(self.added or self.uploaded or self.gone or self.renamed
+                    or self.server_too_old)
+
+
+def reconcile_vaults(account: accounts.Account, server: AccountServer) -> VaultChanges:
+    """
+    Make the list of vaults the same on this device and on the server.
+    ``server`` is logged in. Vault *contents* are not touched here; the
+    caller syncs each vault afterwards.
+
+    - A vault the server has and this device does not is added here (its
+      entries arrive with its first sync).
+    - A vault made here goes onto the server.
+    - A vault that was on the server and is no longer there was deleted from
+      another device. It is **kept** on this one, marked, and left out of the
+      sync: deleting someone's entries is their decision, not a sync's.
+    - A name changed here is told to the server; one changed elsewhere is
+      taken over.
+
+    A server older than 2.0.0-alpha.2 keeps only the personal vault; the
+    others then simply stay on this device.
+    """
+    changes = VaultChanges()
+    extra_here = [v for v in account.vault_ids() if v != account.personal_id]
+    listed = {v["id"]: v for v in server.me().get("vaults", [])
+              if v.get("kind") != "personal" and v.get("id") != account.personal_id}
+    if not server.has_vaults():
+        changes.server_too_old = bool(extra_here)
+        return changes
+
+    for vault_id in extra_here:
+        record = account.vault_record(vault_id)
+        file = account.vault_file(vault_id)
+        there = listed.get(vault_id)
+        if there is None:
+            if record.get("on_server"):
+                if not record.get("gone_from_server"):
+                    account.mark_vault(vault_id, gone_from_server=True)
+                    changes.gone.append(file.name)
+                continue
+            try:
+                server.create_vault(vault_id, file.sealed_name(),
+                                    record["wrapped_key"])
+            except Conflict409:
+                raise SyncError(f"The server already has another vault with the "
+                                f"id of “{file.name}”. Nothing was changed.") from None
+            account.mark_vault(vault_id, on_server=True, gone_from_server=False)
+            changes.uploaded.append(file.name)
+            continue
+        if not record.get("on_server") or record.get("gone_from_server"):
+            account.mark_vault(vault_id, on_server=True, gone_from_server=False)
+        if record.get("rename_pending"):
+            server.rename_vault(vault_id, file.sealed_name())
+            account.mark_vault(vault_id, rename_pending=False)
+            continue
+        try:
+            name = accounts.clean_vault_name(file.open_name(there.get("name", "")))
+        except (CryptoError, ValueError):
+            continue                    # an unreadable name changes nothing
+        if name != file.name:
+            try:
+                account.rename_vault(vault_id, name, pending=False)
+                changes.renamed.append((file.name, name))
+            except ValueError:
+                pass                    # taken here: keep this device's name
+
+    for vault_id, there in listed.items():
+        if vault_id in extra_here:
+            continue
+        try:
+            file = account.adopt_vault(vault_id, there.get("wrapped_key", ""),
+                                       there.get("name", ""))
+        except (CryptoError, ValueError) as e:
+            raise SyncError(f"The server lists a vault this account cannot "
+                            f"open ({e}). Nothing was changed.") from e
+        changes.added.append(file.name)
+    return changes
+
+
+def syncable_vaults(account: accounts.Account, server: AccountServer) -> list:
+    """Ids of the vaults a sync covers: the personal one, and every other
+    that is on the server."""
+    if not server.has_vaults():
+        return [account.personal_id]
+    return [v for v in account.vault_ids()
+            if v == account.personal_id or (
+                account.vault_record(v).get("on_server")
+                and not account.vault_record(v).get("gone_from_server"))]
+
+
+def delete_vault(account: accounts.Account, vault_id: str) -> None:
+    """
+    Delete a vault for good. On a server it is deleted there first: if the
+    server cannot be reached nothing is deleted anywhere, so a vault never
+    comes back from the server after being deleted here.
+    """
+    record = account.vault_record(vault_id)
+    if account.server and record.get("on_server") \
+            and not record.get("gone_from_server"):
+        server = logged_in(account)
+        try:
+            server.delete_vault(vault_id)
+        finally:
+            server.logout()
+    account.delete_vault(vault_id)
+
+
+def put_vault_back(account: accounts.Account, vault_id: str) -> None:
+    """A vault deleted on the server and kept here goes onto the server
+    again with the next sync (the user's choice, in the Vaults window)."""
+    account.mark_vault(vault_id, on_server=False, gone_from_server=False)
+    try:
+        os.remove(account.vault_file(vault_id).base_path)   # agreed with a vault that is gone
+    except OSError:
+        pass
+
+
 # ---------------------------------------------------------------------------
 # Signing in from a server
 # ---------------------------------------------------------------------------
@@ -326,6 +485,19 @@ def sign_in(url: str, ca_file: str, username: str, password: str,
         if stored["blob"]:
             account.save_base(data, sync_client.base_origin_for(
                 server.url, stored["revision"]))
+    # The account's other vaults come with it, so a new device opens with
+    # everything; a later sync merges whatever this device already had.
+    reconcile_vaults(account, server)
+    for vault_id in syncable_vaults(account, server):
+        file = account.vault_file(vault_id)
+        if file.personal or file.has_file:
+            continue
+        stored = server.get_vault(vault_id)
+        fetched = file.decrypt(stored["blob"]) if stored["blob"] else {"accounts": []}
+        file.save(fetched)
+        if stored["blob"]:
+            file.save_base(fetched, sync_client.base_origin_for(
+                server.url, stored["revision"]))
     server.logout()
     return account, data