Harden S3 DC handling

CHANGELOG.md

@@ -1,5 +1,11 @@
 # Changelog
 
+# Changelog
+
+## 0.5.5 (2025-11-18)
+
+* Validate S3 data-center identifiers so replication only uses the canonical `b2-eu-cen`/`wasabi-eu-central-2-v3`/`scw-eu-fr-v3` keys and update the docs to reflect the upstream requirements
+
 ## 0.5.4 (2025-11-18)
 
 * Respect user-defined S3 data-center identifiers so replication targets use the intended buckets

CloudronManifest.json

@@ -7,7 +7,7 @@
   "contactEmail": "contact@ente.io",
   "website": "https://ente.io",
   "tagline": "Open source, end-to-end encrypted photo backup",
-  "version": "0.5.4",
+  "version": "0.5.5",
   "upstreamVersion": "git-main",
   "healthCheckPath": "/health",
   "httpPort": 3080,

POSTINSTALL.md

@@ -55,7 +55,7 @@ Supported variables (these map directly to the fields described in the upstream
 - `S3_PREFIX` (optional path prefix)
 - `S3_ARE_LOCAL_BUCKETS` (set to `false` when your provider uses HTTPS “real” domains instead of MinIO-style LAN endpoints)
 - `S3_FORCE_PATH_STYLE` (set to `true` for MinIO, Cloudflare R2, Backblaze, or any host that requires `https://host/bucket/object` URLs)
-- `S3_PRIMARY_DC`, `S3_SECONDARY_DC`, `S3_COLD_DC`, `S3_DERIVED_DC` (advanced: override the canonical data-center identifiers if your Museum database already uses different keys; defaults follow the upstream docs: `b2-eu-cen`, `wasabi-eu-central-2-v3`, `scw-eu-fr-v3`)
+- `S3_PRIMARY_DC`, `S3_SECONDARY_DC`, `S3_COLD_DC`, `S3_DERIVED_DC` (advanced: pick from the canonical data-center identifiers listed in the upstream docs. The names are hard-coded in Museum; leave them at `b2-eu-cen`, `wasabi-eu-central-2-v3`, `scw-eu-fr-v3` unless you know you need one of the legacy aliases such as `scw-eu-fr`.)
 - Optional replication: define **both** `S3_SECONDARY_*` and `S3_COLD_*` (endpoints, keys, secrets, optional prefixes, DC names) to mirror uploads to a second hot bucket and a third cold bucket. Replication is only enabled when all three buckets are configured; otherwise the app stays in single-bucket mode. See [Ente’s object storage guide](https://ente.io/help/self-hosting/administration/object-storage) for sample layouts and discussion of reliability.
 
 You should never edit the generated `/app/data/museum/configurations/local.yaml` directly. If you need to append extra settings (for example, defining `internal.super-admins`), create `/app/data/config/museum.override.yaml` and add only the keys you want to override. Copying the entire sample `s3:` block from the docs into that file will erase the credentials that the package renders from `s3.env` and break replication.

README.md

@@ -58,7 +58,7 @@ After installing on Cloudron remember to:
 1. Open the File Manager for the app, edit `/app/data/config/s3.env`, and set the S3-compatible credentials that belong in `museum.yaml`. The upstream documentation expects the canonical keys `b2-eu-cen` (primary), `wasabi-eu-central-2-v3` (secondary) and `scw-eu-fr-v3` (cold); this package renders those blocks automatically from the environment variables below so you don’t have to touch the generated config. At minimum set `S3_ENDPOINT`, `S3_REGION`, `S3_BUCKET`, `S3_ACCESS_KEY`, `S3_SECRET_KEY`, plus the optional `S3_PREFIX`. To enable replication you must also define **both** `S3_SECONDARY_*` and `S3_COLD_*` (endpoint, region, bucket, key, secret, optional prefix/DC overrides); after a restart the package will flip `replication.enabled` on your behalf when all three buckets are present. Advanced knobs from the documentation map to the following variables:
    - `S3_ARE_LOCAL_BUCKETS=false` toggles SSL/subdomain-style URLs (`are_local_buckets` in `museum.yaml`); leave it at `true` for MinIO-style setups.
    - `S3_FORCE_PATH_STYLE=true` translates to `use_path_style_urls=true` (required for R2/MinIO and most LAN storage).
-   - `S3_PRIMARY_DC`, `S3_SECONDARY_DC`, `S3_COLD_DC`, and `S3_DERIVED_DC` let you override the default data-center identifiers if your Museum database already uses different keys.
+   - The data-center identifiers (`b2-eu-cen`, `wasabi-eu-central-2-v3`, `scw-eu-fr-v3`, etc.) are **hard-coded upstream**. Keep the defaults unless you know you are targeting one of the legacy names (as listed in the Ente docs). The start script will ignore unknown values to prevent replication from breaking with empty bucket names.
    - Leave the generated `museum/configurations/local.yaml` alone—if you need to append extra settings, do so via `/app/data/config/museum.override.yaml` and only add the keys you actually want to change. Copy‑pasting the full sample `s3:` block from the docs will overwrite the generated credentials with blanks.
    - If you are using Cloudflare R2 or another hosted S3 provider, configure your bucket’s CORS policy to allow the Ente frontends (e.g. `https://ente.due.ren`, `https://accounts.due.ren`, `https://cast.due.ren`, etc.) so that cast/slideshow playback can fetch signed URLs directly from storage. Backblaze B2 also requires clearing its “native” CORS rules; see the script in `POSTINSTALL.md`.
 2. When prompted during installation, pick hostnames for the Accounts/Auth/Cast/Albums/Family web apps (they are exposed via Cloudron `httpPorts`). Ensure matching DNS records exist; Cloudron-managed DNS creates them automatically, otherwise point CNAME/A records such as `accounts.<app-domain>` at the primary hostname.
@@ -104,7 +104,7 @@ The main photos UI continues to live on the hostname you selected during install
 
 The upstream documentation at [ente.io/help/self-hosting/administration/object-storage](https://ente.io/help/self-hosting/administration/object-storage) is written for bare-metal installs where you edit `museum.yaml` by hand. The Cloudron package wraps those steps so you only maintain `/app/data/config/s3.env`, but the same concepts apply:
 
-- **Canonical bucket names.** Museum’s schema ships with `b2-eu-cen`, `wasabi-eu-central-2-v3`, and `scw-eu-fr-v3`. You can point those names at any S3-compatible provider—the labels are just keys. If you do override them, update `S3_PRIMARY_DC`, `S3_SECONDARY_DC`, `S3_COLD_DC`, and restart so the package renders matching blocks.
+- **Canonical bucket names.** Museum’s schema ships with `b2-eu-cen`, `wasabi-eu-central-2-v3`, and `scw-eu-fr-v3`. You can point those identifiers at any S3-compatible provider, but you cannot rename them—replication logic only understands the upstream keys (or their documented legacy aliases). Leave the defaults in `s3.env` and only change the credentials/endpoints under each key.
 - **Three buckets for replication.** Replication only works when two “hot” buckets and one “cold” bucket are configured. Populate `S3_*`, `S3_SECONDARY_*`, and `S3_COLD_*`; once all three have endpoints/keys/secrets the package automatically writes the `replication.enabled: true` stanza.
 - **Transport settings.** Set `S3_ARE_LOCAL_BUCKETS=true`/`false` and `S3_FORCE_PATH_STYLE=true` to mirror the documentation’s `are_local_buckets`/`use_path_style_urls` toggles when talking to MinIO, Cloudflare R2, or other providers that require path-style URLs over HTTPS.
 - **CORS.** If browsers cannot upload/download because of CORS, apply the recommended JSON from the docs (or the Backblaze helper script in `POSTINSTALL.md`). Ensure `Content-MD5` is listed in `AllowedHeaders` for providers with allow-lists.

start.sh

@@ -272,7 +272,6 @@ S3_SECONDARY_BUCKET="${S3_SECONDARY_BUCKET:-${ENTE_S3_SECONDARY_BUCKET:-}}"
 S3_SECONDARY_ACCESS_KEY="${S3_SECONDARY_ACCESS_KEY:-${ENTE_S3_SECONDARY_ACCESS_KEY:-}}"
 S3_SECONDARY_SECRET_KEY="${S3_SECONDARY_SECRET_KEY:-${ENTE_S3_SECONDARY_SECRET_KEY:-}}"
 S3_SECONDARY_PREFIX="${S3_SECONDARY_PREFIX:-${ENTE_S3_SECONDARY_PREFIX:-}}"
-S3_SECONDARY_DC_RAW="${S3_SECONDARY_DC:-${ENTE_S3_SECONDARY_DC:-}}"
 S3_SECONDARY_ENABLED=false
 S3_SECONDARY_ENDPOINT_HOST=""
 
@@ -282,7 +281,6 @@ S3_COLD_BUCKET="${S3_COLD_BUCKET:-${ENTE_S3_COLD_BUCKET:-}}"
 S3_COLD_ACCESS_KEY="${S3_COLD_ACCESS_KEY:-${ENTE_S3_COLD_ACCESS_KEY:-}}"
 S3_COLD_SECRET_KEY="${S3_COLD_SECRET_KEY:-${ENTE_S3_COLD_SECRET_KEY:-}}"
 S3_COLD_PREFIX="${S3_COLD_PREFIX:-${ENTE_S3_COLD_PREFIX:-}}"
-S3_COLD_DC_RAW="${S3_COLD_DC:-${ENTE_S3_COLD_DC:-}}"
 S3_COLD_ENABLED=false
 S3_COLD_ENDPOINT_HOST=""
 
@@ -339,14 +337,33 @@ S3_ARE_LOCAL_BUCKETS="$(printf '%s' "${S3_ARE_LOCAL_BUCKETS:-${ENTE_S3_ARE_LOCAL
 
 DEFAULT_SECONDARY_DC="wasabi-eu-central-2-v3"
 DEFAULT_COLD_DC="scw-eu-fr-v3"
+S3_VALID_DC_NAMES=("b2-eu-cen" "scw-eu-fr" "scw-eu-fr-locked" "scw-eu-fr-v3" "wasabi-eu-central-2" "wasabi-eu-central-2-v3" "wasabi-eu-central-2-derived" "b5" "b6")
 
-S3_PRIMARY_DC="${S3_PRIMARY_DC:-${ENTE_S3_PRIMARY_DC:-b2-eu-cen}}"
-S3_SECONDARY_DC="$DEFAULT_SECONDARY_DC"
-S3_COLD_DC="$DEFAULT_COLD_DC"
-S3_DERIVED_DC="${S3_DERIVED_DC:-${ENTE_S3_DERIVED_DC:-$S3_PRIMARY_DC}}"
+validate_s3_dc() {
+  local candidate="$1"
+  local fallback="$2"
+  local label="$3"
+  if [ -z "$candidate" ]; then
+    printf '%s\n' "$fallback"
+    return
+  fi
+  for allowed in "${S3_VALID_DC_NAMES[@]}"; do
+    if [ "$candidate" = "$allowed" ]; then
+      printf '%s\n' "$candidate"
+      return
+    fi
+  done
+  log WARN "Ignoring unknown $label S3 data center '$candidate'; falling back to $fallback"
+  printf '%s\n' "$fallback"
+}
+
+S3_PRIMARY_DC="$(validate_s3_dc "${S3_PRIMARY_DC:-${ENTE_S3_PRIMARY_DC:-}}" "b2-eu-cen" "primary")"
+S3_SECONDARY_DC="$(validate_s3_dc "${S3_SECONDARY_DC:-${ENTE_S3_SECONDARY_DC:-}}" "$DEFAULT_SECONDARY_DC" "secondary")"
+S3_COLD_DC="$(validate_s3_dc "${S3_COLD_DC:-${ENTE_S3_COLD_DC:-}}" "$DEFAULT_COLD_DC" "cold")"
+S3_DERIVED_DC="$(validate_s3_dc "${S3_DERIVED_DC:-${ENTE_S3_DERIVED_DC:-}}" "$S3_PRIMARY_DC" "derived")"
 
 S3_SECONDARY_ENV_PRESENT=false
-for value in "$S3_SECONDARY_ENDPOINT" "$S3_SECONDARY_REGION" "$S3_SECONDARY_BUCKET" "$S3_SECONDARY_ACCESS_KEY" "$S3_SECONDARY_SECRET_KEY" "$S3_SECONDARY_PREFIX" "$S3_SECONDARY_DC_RAW"; do
+for value in "$S3_SECONDARY_ENDPOINT" "$S3_SECONDARY_REGION" "$S3_SECONDARY_BUCKET" "$S3_SECONDARY_ACCESS_KEY" "$S3_SECONDARY_SECRET_KEY" "$S3_SECONDARY_PREFIX"; do
   if [ -n "$value" ]; then
     S3_SECONDARY_ENV_PRESENT=true
     break
@@ -367,17 +384,14 @@ if [ "$S3_NOT_CONFIGURED" = "false" ] && [ "$S3_SECONDARY_ENV_PRESENT" = true ];
     S3_SECONDARY_DC=""
   else
     S3_SECONDARY_ENABLED=true
-    if [ -n "$S3_SECONDARY_DC_RAW" ]; then
-      S3_SECONDARY_DC="$S3_SECONDARY_DC_RAW"
-    fi
   fi
 else
-S3_SECONDARY_ENABLED=false
-S3_SECONDARY_DC=""
+  S3_SECONDARY_ENABLED=false
+  S3_SECONDARY_DC=""
 fi
 
 S3_COLD_ENV_PRESENT=false
-for value in "$S3_COLD_ENDPOINT" "$S3_COLD_REGION" "$S3_COLD_BUCKET" "$S3_COLD_ACCESS_KEY" "$S3_COLD_SECRET_KEY" "$S3_COLD_PREFIX" "$S3_COLD_DC_RAW"; do
+for value in "$S3_COLD_ENDPOINT" "$S3_COLD_REGION" "$S3_COLD_BUCKET" "$S3_COLD_ACCESS_KEY" "$S3_COLD_SECRET_KEY" "$S3_COLD_PREFIX"; do
   if [ -n "$value" ]; then
     S3_COLD_ENV_PRESENT=true
     break
@@ -398,9 +412,6 @@ if [ "$S3_NOT_CONFIGURED" = "false" ] && [ "$S3_COLD_ENV_PRESENT" = true ]; then
     S3_COLD_DC=""
   else
     S3_COLD_ENABLED=true
-    if [ -n "$S3_COLD_DC_RAW" ]; then
-      S3_COLD_DC="$S3_COLD_DC_RAW"
-    fi
   fi
 else
   S3_COLD_ENABLED=false