Add comprehensive API documentation to Cloudron setup instructions

- Added detailed API endpoint information in SETUP-INSTRUCTIONS.md
- Documented API usage with Ente CLI
- Enhanced routing configuration for auth/cast/accounts apps
- Updated to version 0.1.64

BUILD-INSTRUCTIONS.md

@@ -0,0 +1,110 @@
+# Ente Cloudron App Build and Installation Instructions
+
+This document provides detailed instructions for building and installing the Ente Cloudron app, an open-source, end-to-end encrypted photo storage and authentication solution.
+
+## Prerequisites
+
+- **Cloudron CLI**: Ensure the Cloudron CLI is installed and configured on your system. Refer to [Cloudron CLI Documentation](https://docs.cloudron.io/packaging/cli/) for setup instructions.
+- **Docker**: Required for local testing or custom builds if needed.
+- **Git**: To clone or manage the repository.
+- **Repository Access**: Ensure you have access to the Ente Cloudron repository at `andreasdueren/ente-cloudron`.
+- **Build Service Token**: A token for the Cloudron build service is required (provided in the command below).
+
+## Build Commands
+
+1. **Clone the Repository** (if not already done):
+   ```bash
+   git clone https://github.com/andreasdueren/ente-cloudron.git
+   cd ente-cloudron
+   ```
+
+2. **Build the App Using Cloudron Build Service**:
+   Use the provided build service and token to build the app. Replace `<version>` with the desired version tag (e.g., `0.1.0` or as per `CloudronManifest.json`).
+   ```bash
+   cloudron build --set-build-service builder.docker.due.ren --build-service-token e3265de06b1d0e7bb38400539012a8433a74c2c96a17955e --set-repository andreasdueren/ente-cloudron --tag 1.0.1
+   ```
+   **Note**: The build process should complete within a reasonable time. Monitor the output for any errors.
+
+## Installation Commands
+
+1. **Install the App on Cloudron**:
+   After a successful build, install the app on your Cloudron instance at the desired location (e.g., `ente.due.ren`).
+   ```bash
+   cloudron install --location ente.due.ren --image andreasdueren/ente-cloudron:1.0.1
+   ```
+   **Important**: Do not wait more than 30 seconds for feedback after running the install command. If there's an error, the process may hang, and you should terminate it to troubleshoot.
+   **Note**: Always uninstall and reinstall during development rather than updating an existing app to ensure a clean setup.
+
+## Testing Procedures
+
+1. **Verify Installation**:
+   - Access the app at `https://ente.due.ren` (or your configured domain).
+   - Ensure the Ente web interfaces (Photos, Accounts, Auth, Cast) load correctly.
+
+2. **Check S3 Configuration**:
+   - Confirm that S3 environment variables are set in Cloudron app settings under the 'Environment Variables' section.
+   - Variables to check: `APP_S3_ENABLED`, `APP_S3_ENDPOINT`, `APP_S3_ACCESS_KEY_ID`, `APP_S3_SECRET_ACCESS_KEY`, `APP_S3_BUCKET`.
+
+3. **Monitor Logs for Errors**:
+   - Use the Cloudron CLI to view logs:
+     ```bash
+     cloudron logs --app ente.due.ren -f
+     ```
+   - Alternatively, shell into the app for detailed log inspection:
+     ```bash
+     cloudron exec --app ente.due.ren
+     tail -f /app/data/logs/*
+     ```
+   - Look for S3 connection errors or other issues.
+
+## Deployment Steps
+
+1. **Post-Installation Configuration**:
+   - If S3 is not working, update the environment variables in Cloudron app settings and restart the app:
+     ```bash
+     cloudron restart --app ente.due.ren
+     ```
+
+2. **User Authentication**:
+   - Ente uses its own authentication system. Ensure user registration and login work as expected.
+   - If OIDC integration is desired in the future, it can be configured using Cloudron's OIDC variables (`CLOUDRON_OIDC_IDENTIFIER`, `CLOUDRON_OIDC_CLIENT_ID`, `CLOUDRON_OIDC_CLIENT_SECRET`).
+
+## Troubleshooting Common Issues
+
+- **S3 Configuration Errors**:
+  - **Symptom**: App falls back to local storage or logs show S3 connection failures.
+  - **Solution**: Verify S3 environment variables in Cloudron settings. Test connectivity manually using AWS CLI (`aws s3 ls s3://<bucket> --endpoint-url <endpoint>`).
+
+- **Build Failures**:
+  - **Symptom**: Build command errors out or hangs.
+  - **Solution**: Check network connectivity to the build service, ensure the token is correct, and review build logs for specific errors.
+
+- **Installation Hangs**:
+  - **Symptom**: Install command does not complete within 30 seconds.
+  - **Solution**: Terminate the command and check Cloudron logs for errors (`cloudron logs --app ente.due.ren`). Reinstall if necessary.
+
+- **App Not Starting**:
+  - **Symptom**: App shows as 'Stopped' or inaccessible after install.
+  - **Solution**: Check logs for startup errors (`cloudron logs --app ente.due.ren`). Ensure database connectivity and correct configuration.
+
+## Configuration Examples
+
+- **S3 Environment Variables** in Cloudron settings:
+  ```
+  APP_S3_ENABLED=true
+  APP_S3_ENDPOINT=s3.amazonaws.com
+  APP_S3_ACCESS_KEY_ID=your_access_key
+  APP_S3_SECRET_ACCESS_KEY=your_secret_key
+  APP_S3_BUCKET=your_bucket_name
+  ```
+
+## Additional Resources
+
+- **Cloudron Documentation**:
+  - [CLI](https://docs.cloudron.io/packaging/cli/)
+  - [Packaging Tutorial](https://docs.cloudron.io/packaging/tutorial/)
+  - [Manifest Reference](https://docs.cloudron.io/packaging/manifest/)
+  - [Addons Guide](https://docs.cloudron.io/packaging/addons/)
+  - [Cheat Sheet](https://docs.cloudron.io/packaging/cheat-sheet/)
+
+For further assistance, contact the Ente team at `contact@ente.io` or refer to the GitHub repository at [https://github.com/ente-io/ente](https://github.com/ente-io/ente).

Caddyfile.simple

@@ -0,0 +1,21 @@
+{
+    admin off
+    auto_https off
+}
+
+:3080 {
+    log {
+        output stdout
+        level DEBUG
+    }
+
+    # Simple health check that always works
+    handle /health {
+        respond "{\"status\": \"OK\"}" 200
+    }
+
+    # Catch-all for debugging
+    handle {
+        respond "Caddy is running on port 3080" 200
+    }
+}

CloudronManifest.json

@@ -7,13 +7,14 @@
   "contactEmail": "contact@ente.io",
   "tagline": "Open Source End-to-End Encrypted Photos & Authentication",
   "upstreamVersion": "1.0.0",
-  "version": "1.0.1",
-  "healthCheckPath": "/health",
+  "version": "0.1.64",
+  "healthCheckPath": "/ping",
   "httpPort": 3080,
   "memoryLimit": 1073741824,
   "addons": {
     "localstorage": {},
     "postgresql": {},
+    "email": {},
     "sendmail": {
       "supportsDisplayName": true
     }

Dockerfile

@@ -1,3 +1,19 @@
+# Build Museum server from source
+FROM golang:1.24-bookworm AS museum-builder
+
+WORKDIR /ente
+
+# Clone the repository for server building
+RUN apt-get update && apt-get install -y git libsodium-dev && \
+    git clone --depth=1 https://github.com/ente-io/ente.git . && \
+    apt-get clean && apt-get autoremove && \
+    rm -rf /var/cache/apt /var/lib/apt/lists
+
+# Build the Museum server
+WORKDIR /ente/server
+RUN go mod download && \
+    CGO_ENABLED=1 GOOS=linux go build -a -o museum ./cmd/museum
+
 FROM node:20-bookworm-slim as web-builder
 
 WORKDIR /ente
@@ -12,10 +28,10 @@ RUN apt-get update && apt-get install -y git && \
 RUN corepack enable
 
 # Set environment variables for web app build
-# Use "/api" as the endpoint which will be replaced at runtime with the full URL
+# Set the API endpoint to use current origin - this will work at runtime
 ENV NEXT_PUBLIC_ENTE_ENDPOINT="/api"
-# Add a note for clarity
-RUN echo "Building with NEXT_PUBLIC_ENTE_ENDPOINT=/api, will be replaced at runtime with full URL"
+# Use relative path so it works with any domain
+RUN echo "Building with NEXT_PUBLIC_ENTE_ENDPOINT=/api, will work with any domain via Caddy proxy"
 
 # Debugging the repository structure
 RUN find . -type d -maxdepth 3 | sort
@@ -136,12 +152,21 @@ COPY --from=web-builder /build/web/accounts /app/web/accounts
 COPY --from=web-builder /build/web/auth /app/web/auth
 COPY --from=web-builder /build/web/cast /app/web/cast
 
+# Copy Museum server binary from builder stage to app directory (not data volume)
+RUN mkdir -p /app/museum-bin
+COPY --from=museum-builder /ente/server/museum /app/museum-bin/museum
+RUN chmod +x /app/museum-bin/museum
+
 # Copy configuration and startup scripts
 ADD start.sh /app/pkg/
 ADD config.template.yaml /app/pkg/
+ADD otp-email-monitor.js /app/pkg/
+ADD package.json /app/pkg/
+ADD admin-helper.sh /app/pkg/
+ADD admin-helper-direct.sh /app/pkg/
 
 # Set proper permissions
-RUN chmod +x /app/pkg/start.sh
+RUN chmod +x /app/pkg/start.sh /app/pkg/admin-helper.sh /app/pkg/admin-helper-direct.sh
 
 # Expose the web port (Cloudron expects port 3080)
 EXPOSE 3080

admin-helper-direct.sh

@@ -0,0 +1,133 @@
+#!/bin/bash
+# Direct Database Admin Helper for Ente Cloudron
+# This script directly updates the database for admin operations
+
+# Function to update user subscription directly in database
+update_subscription() {
+    local user_email="$1"
+    local storage_gb="$2"
+    local valid_days="$3"
+    
+    if [ -z "$user_email" ] || [ -z "$storage_gb" ] || [ -z "$valid_days" ]; then
+        echo "Usage: $0 update-subscription <user-email> <storage-gb> <valid-days>"
+        echo "Example: $0 update-subscription user@example.com 100 365"
+        return 1
+    fi
+    
+    echo "Updating subscription for: $user_email"
+    echo "Storage: ${storage_gb}GB"
+    echo "Valid for: ${valid_days} days"
+    
+    # Convert GB to bytes (1 GB = 1073741824 bytes)
+    local storage_bytes=$((storage_gb * 1073741824))
+    
+    # Calculate expiry timestamp (current time + valid_days)
+    local current_timestamp=$(date +%s)
+    local expiry_timestamp=$((current_timestamp + (valid_days * 86400)))
+    # Convert to microseconds for the database
+    local expiry_microseconds="${expiry_timestamp}000000"
+    
+    # Update the database directly
+    PGPASSWORD="$CLOUDRON_POSTGRESQL_PASSWORD" psql \
+        -h "$CLOUDRON_POSTGRESQL_HOST" \
+        -p "$CLOUDRON_POSTGRESQL_PORT" \
+        -U "$CLOUDRON_POSTGRESQL_USERNAME" \
+        -d "$CLOUDRON_POSTGRESQL_DATABASE" << EOF
+-- Update user's storage and subscription
+UPDATE users 
+SET 
+    storage_bonus = $storage_bytes,
+    subscription_expiry = $expiry_microseconds
+WHERE email = '$user_email';
+
+-- Show the updated values
+SELECT 
+    email,
+    storage_bonus / 1073741824.0 as storage_gb,
+    to_timestamp(subscription_expiry / 1000000) as subscription_expires
+FROM users 
+WHERE email = '$user_email';
+EOF
+    
+    if [ $? -eq 0 ]; then
+        echo "✓ Subscription updated successfully"
+    else
+        echo "✗ Failed to update subscription"
+        return 1
+    fi
+}
+
+# Function to get user details
+get_user_details() {
+    local user_email="$1"
+    
+    if [ -z "$user_email" ]; then
+        echo "Usage: $0 get-user <user-email>"
+        return 1
+    fi
+    
+    PGPASSWORD="$CLOUDRON_POSTGRESQL_PASSWORD" psql \
+        -h "$CLOUDRON_POSTGRESQL_HOST" \
+        -p "$CLOUDRON_POSTGRESQL_PORT" \
+        -U "$CLOUDRON_POSTGRESQL_USERNAME" \
+        -d "$CLOUDRON_POSTGRESQL_DATABASE" << EOF
+SELECT 
+    email,
+    storage_bonus / 1073741824.0 as storage_gb,
+    storage_consumed / 1073741824.0 as used_gb,
+    to_timestamp(subscription_expiry / 1000000) as subscription_expires,
+    CASE 
+        WHEN subscription_expiry > (EXTRACT(EPOCH FROM NOW()) * 1000000) THEN 'Active'
+        ELSE 'Expired'
+    END as status
+FROM users 
+WHERE email = '$user_email';
+EOF
+}
+
+# Function to list all users
+list_users() {
+    PGPASSWORD="$CLOUDRON_POSTGRESQL_PASSWORD" psql \
+        -h "$CLOUDRON_POSTGRESQL_HOST" \
+        -p "$CLOUDRON_POSTGRESQL_PORT" \
+        -U "$CLOUDRON_POSTGRESQL_USERNAME" \
+        -d "$CLOUDRON_POSTGRESQL_DATABASE" << EOF
+SELECT 
+    email,
+    storage_bonus / 1073741824.0 as storage_gb,
+    storage_consumed / 1073741824.0 as used_gb,
+    to_timestamp(subscription_expiry / 1000000) as expires,
+    CASE 
+        WHEN subscription_expiry > (EXTRACT(EPOCH FROM NOW()) * 1000000) THEN 'Active'
+        ELSE 'Expired'
+    END as status
+FROM users 
+ORDER BY email;
+EOF
+}
+
+# Main command handler
+case "$1" in
+    "update-subscription")
+        update_subscription "$2" "$3" "$4"
+        ;;
+    "get-user")
+        get_user_details "$2"
+        ;;
+    "list-users")
+        list_users
+        ;;
+    *)
+        echo "Ente Direct Admin Helper"
+        echo ""
+        echo "Usage:"
+        echo "  $0 update-subscription <user-email> <storage-gb> <valid-days>"
+        echo "  $0 get-user <user-email>"
+        echo "  $0 list-users"
+        echo ""
+        echo "Examples:"
+        echo "  $0 update-subscription user@example.com 100 365"
+        echo "  $0 get-user user@example.com"
+        echo "  $0 list-users"
+        ;;
+esac

admin-helper.sh

@@ -0,0 +1,93 @@
+#!/bin/bash
+# Ente Admin Helper Script for Cloudron
+# This script simplifies admin operations in the Cloudron terminal
+
+MUSEUM_BIN="/app/data/ente/server/museum"
+
+# Check if museum binary exists
+if [ ! -f "$MUSEUM_BIN" ]; then
+    echo "Error: Museum binary not found at $MUSEUM_BIN"
+    exit 1
+fi
+
+# Function to update user subscription
+update_subscription() {
+    local user_email="$1"
+    local storage_gb="$2"
+    local valid_days="$3"
+    
+    if [ -z "$user_email" ] || [ -z "$storage_gb" ] || [ -z "$valid_days" ]; then
+        echo "Usage: $0 update-subscription <user-email> <storage-gb> <valid-days>"
+        echo "Example: $0 update-subscription user@example.com 100 365"
+        return 1
+    fi
+    
+    echo "Updating subscription for: $user_email"
+    echo "Storage: ${storage_gb}GB"
+    echo "Valid for: ${valid_days} days"
+    
+    cd /app/data/ente/server
+    
+    # Use environment variables for database connection
+    export DB_HOST="$CLOUDRON_POSTGRESQL_HOST"
+    export DB_PORT="$CLOUDRON_POSTGRESQL_PORT"
+    export DB_NAME="$CLOUDRON_POSTGRESQL_DATABASE"
+    export DB_USERNAME="$CLOUDRON_POSTGRESQL_USERNAME"
+    export DB_PASSWORD="$CLOUDRON_POSTGRESQL_PASSWORD"
+    
+    # Museum admin commands need specific syntax
+    "$MUSEUM_BIN" admin update-subscription "$user_email" "$storage_gb" "$valid_days"
+}
+
+# Function to get user details
+get_user_details() {
+    local user_email="$1"
+    
+    if [ -z "$user_email" ]; then
+        echo "Usage: $0 get-user <user-email>"
+        return 1
+    fi
+    
+    cd /app/data/ente/server
+    
+    "$MUSEUM_BIN" admin get-user-details --user "$user_email"
+}
+
+# Function to list all users
+list_users() {
+    cd /app/data/ente/server
+    
+    # Connect to PostgreSQL and list users
+    PGPASSWORD="$CLOUDRON_POSTGRESQL_PASSWORD" psql \
+        -h "$CLOUDRON_POSTGRESQL_HOST" \
+        -p "$CLOUDRON_POSTGRESQL_PORT" \
+        -U "$CLOUDRON_POSTGRESQL_USERNAME" \
+        -d "$CLOUDRON_POSTGRESQL_DATABASE" \
+        -c "SELECT email, storage_bonus, subscription_expiry FROM users ORDER BY email;"
+}
+
+# Main command handler
+case "$1" in
+    "update-subscription")
+        update_subscription "$2" "$3" "$4"
+        ;;
+    "get-user")
+        get_user_details "$2"
+        ;;
+    "list-users")
+        list_users
+        ;;
+    *)
+        echo "Ente Admin Helper"
+        echo ""
+        echo "Usage:"
+        echo "  $0 update-subscription <user-email> <storage-gb> <valid-days>"
+        echo "  $0 get-user <user-email>"
+        echo "  $0 list-users"
+        echo ""
+        echo "Examples:"
+        echo "  $0 update-subscription user@example.com 100 365"
+        echo "  $0 get-user user@example.com"
+        echo "  $0 list-users"
+        ;;
+esac

debug-headers.sh

@@ -0,0 +1,35 @@
+#!/bin/bash
+
+echo "==> Debugging Caddy MIME type headers"
+echo "==> Testing various file types..."
+
+BASE_URL="${1:-https://ente.due.ren}"
+
+echo
+echo "Testing HTML files:"
+curl -I "$BASE_URL/" 2>/dev/null | grep -i content-type || echo "No Content-Type header found"
+curl -I "$BASE_URL/index.html" 2>/dev/null | grep -i content-type || echo "No Content-Type header found"
+
+echo
+echo "Testing JavaScript files:"
+curl -I "$BASE_URL/config.js" 2>/dev/null | grep -i content-type || echo "No Content-Type header found"
+
+echo
+echo "Testing CSS files (if any):"
+curl -I "$BASE_URL/styles.css" 2>/dev/null | grep -i content-type || echo "File not found or no Content-Type header"
+
+echo
+echo "Testing JSON files (if any):"
+curl -I "$BASE_URL/manifest.json" 2>/dev/null | grep -i content-type || echo "File not found or no Content-Type header"
+
+echo
+echo "==> Full response headers for main page:"
+curl -I "$BASE_URL/" 2>/dev/null || echo "Failed to connect to $BASE_URL"
+
+echo
+echo "==> To test from inside a container:"
+echo "docker exec -it <container-name> curl -I http://localhost:3080/"
+
+echo
+echo "==> To view Caddy logs:"
+echo "docker exec -it <container-name> tail -f /app/data/logs/caddy.log"

debug-start.sh

@@ -0,0 +1,32 @@
+#!/bin/bash
+
+# Add this debugging section to your start.sh after line 350
+
+# Start Caddy with more verbose logging
+echo "==> Starting Caddy web server with debug logging"
+echo "==> Validating Caddyfile first..."
+caddy validate --config /app/data/Caddyfile --adapter caddyfile || {
+    echo "==> ERROR: Caddyfile validation failed!"
+    cat /app/data/Caddyfile
+    exit 1
+}
+
+echo "==> Starting Caddy..."
+# Run Caddy in foreground first to see errors
+timeout 10 caddy run --config /app/data/Caddyfile --adapter caddyfile 2>&1 | tee /app/data/logs/caddy-debug.log || {
+    echo "==> ERROR: Caddy failed to start"
+    echo "==> Last 50 lines of Caddy debug log:"
+    tail -50 /app/data/logs/caddy-debug.log
+}
+
+# Check if port is actually listening
+echo "==> Checking if port 3080 is listening..."
+netstat -tlnp | grep 3080 || lsof -i :3080 || {
+    echo "==> ERROR: Nothing listening on port 3080"
+}
+
+# Test the health endpoint
+echo "==> Testing health endpoint..."
+curl -v http://localhost:3080/health || {
+    echo "==> ERROR: Health check failed"
+}

ente-cli-config.md

@@ -0,0 +1,64 @@
+# Ente CLI Configuration for Custom Server
+
+The Ente CLI expects configuration in `~/.ente/config.yaml`. Here's how to set it up:
+
+## Method 1: Direct Configuration
+
+1. Create the config file:
+```bash
+mkdir -p ~/.ente
+cat > ~/.ente/config.yaml << EOF
+api:
+  url: https://ente.due.ren
+EOF
+```
+
+2. Add your account interactively:
+```bash
+ente account add
+# It will ask for:
+# - Export directory: /tmp/ente-export (or any directory)
+# - Email: your-admin@email.com
+# - Password: your-password
+```
+
+## Method 2: Using the Admin Commands Directly
+
+If the interactive setup is problematic, you can use the admin commands with explicit parameters:
+
+```bash
+# Set the API endpoint
+export ENTE_API_URL="https://ente.due.ren"
+
+# Or pass it directly in the command
+ente admin update-subscription \
+  --api-url https://ente.due.ren \
+  --admin-user admin@due.ren \
+  --user user@example.com \
+  --storage 1000 \
+  --valid-for 365
+```
+
+## Method 3: Direct Database Update (Fallback)
+
+Since the CLI setup seems problematic, you can update the database directly in the Cloudron terminal:
+
+```bash
+# In Cloudron terminal
+PGPASSWORD="$CLOUDRON_POSTGRESQL_PASSWORD" psql \
+  -h "$CLOUDRON_POSTGRESQL_HOST" \
+  -U "$CLOUDRON_POSTGRESQL_USERNAME" \
+  -d "$CLOUDRON_POSTGRESQL_DATABASE" << EOF
+-- Update user to 1TB for 1 year
+UPDATE users 
+SET storage_bonus = 1073741824000,  -- 1000 GB in bytes
+    subscription_expiry = EXTRACT(EPOCH FROM NOW() + INTERVAL '365 days') * 1000000
+WHERE email = 'andreas@due.ren';
+
+-- Show the result
+SELECT email, 
+       storage_bonus / 1073741824.0 as storage_gb,
+       to_timestamp(subscription_expiry / 1000000) as expires
+FROM users WHERE email = 'andreas@due.ren';
+EOF
+```

otp-email-monitor.js

@@ -0,0 +1,389 @@
+#!/usr/bin/env node
+
+/**
+ * Ente OTP Email Monitor
+ * 
+ * Monitors Museum server logs for OTP generation events and sends
+ * verification emails using Cloudron's email addon.
+ */
+
+const fs = require('fs');
+const path = require('path');
+const { spawn } = require('child_process');
+const nodemailer = require('nodemailer');
+
+// Configuration
+const CONFIG = {
+    LOG_FILE: '/app/data/logs/museum.log',
+    EMAIL_TEMPLATES_DIR: '/app/data/ente/server/mail-templates',
+    FROM_EMAIL: `noreply@${process.env.CLOUDRON_EMAIL_DOMAIN || 'localhost'}`,
+    FROM_NAME: 'Ente Photos',
+    SMTP: {
+        host: process.env.CLOUDRON_EMAIL_SMTP_SERVER,
+        port: parseInt(process.env.CLOUDRON_EMAIL_SMTP_PORT) || 587,
+        secure: false, // STARTTLS disabled on this port
+        auth: false // Internal mail server
+    }
+};
+
+// Logging utility
+class Logger {
+    static log(level, message, data = null) {
+        const timestamp = new Date().toISOString();
+        const logMessage = `[${timestamp}] [OTP-EMAIL-${level}] ${message}`;
+        console.log(logMessage);
+        
+        if (data) {
+            console.log(JSON.stringify(data, null, 2));
+        }
+        
+        // Also write to file
+        try {
+            fs.appendFileSync('/app/data/logs/otp-email.log', logMessage + '\n');
+        } catch (err) {
+            console.error('Failed to write to log file:', err.message);
+        }
+    }
+    
+    static info(message, data) { this.log('INFO', message, data); }
+    static warn(message, data) { this.log('WARN', message, data); }
+    static error(message, data) { this.log('ERROR', message, data); }
+}
+
+// Email template handler
+class EmailTemplate {
+    constructor(templateDir) {
+        this.templateDir = templateDir;
+        this.templates = new Map();
+        this.loadTemplates();
+    }
+    
+    loadTemplates() {
+        try {
+            const ottTemplate = fs.readFileSync(path.join(this.templateDir, 'ott.html'), 'utf8');
+            const changeEmailTemplate = fs.readFileSync(path.join(this.templateDir, 'ott_change_email.html'), 'utf8');
+            
+            this.templates.set('ott', ottTemplate);
+            this.templates.set('ott_change_email', changeEmailTemplate);
+            
+            Logger.info('Email templates loaded successfully');
+        } catch (err) {
+            Logger.error('Failed to load email templates:', err.message);
+            throw err;
+        }
+    }
+    
+    render(templateName, variables) {
+        const template = this.templates.get(templateName);
+        if (!template) {
+            throw new Error(`Template ${templateName} not found`);
+        }
+        
+        let html = template;
+        
+        // Replace template variables
+        for (const [key, value] of Object.entries(variables)) {
+            const placeholder = `{{.${key}}}`;
+            html = html.replace(new RegExp(placeholder, 'g'), value);
+        }
+        
+        return html;
+    }
+}
+
+// Email sender using Cloudron email addon
+class EmailSender {
+    constructor(config) {
+        this.config = config;
+        this.transporter = null;
+        this.initializeTransporter();
+    }
+    
+    initializeTransporter() {
+        try {
+            this.transporter = nodemailer.createTransport({
+                host: this.config.SMTP.host,
+                port: this.config.SMTP.port,
+                secure: this.config.SMTP.secure,
+                // No auth needed for internal Cloudron mail server
+                tls: {
+                    rejectUnauthorized: false // Accept self-signed certificates
+                }
+            });
+            
+            Logger.info('Email transporter initialized', {
+                host: this.config.SMTP.host,
+                port: this.config.SMTP.port
+            });
+        } catch (err) {
+            Logger.error('Failed to initialize email transporter:', err.message);
+            throw err;
+        }
+    }
+    
+    async sendEmail(to, subject, html) {
+        try {
+            const mailOptions = {
+                from: `${this.config.FROM_NAME} <${this.config.FROM_EMAIL}>`,
+                to: to,
+                subject: subject,
+                html: html
+            };
+            
+            const result = await this.transporter.sendMail(mailOptions);
+            Logger.info('Email sent successfully', {
+                to: to,
+                subject: subject,
+                messageId: result.messageId
+            });
+            
+            return result;
+        } catch (err) {
+            Logger.error('Failed to send email:', {
+                error: err.message,
+                to: to,
+                subject: subject
+            });
+            throw err;
+        }
+    }
+}
+
+// Log monitor for OTP events
+class LogMonitor {
+    constructor(logFile, emailSender, emailTemplate) {
+        this.logFile = logFile;
+        this.emailSender = emailSender;
+        this.emailTemplate = emailTemplate;
+        this.tail = null;
+        this.processedOTPs = new Set(); // Prevent duplicate sends
+    }
+    
+    start() {
+        Logger.info('Starting log monitor', { logFile: this.logFile });
+        
+        // Use tail -F to follow log file
+        this.tail = spawn('tail', ['-F', this.logFile]);
+        
+        this.tail.stdout.on('data', (data) => {
+            const lines = data.toString().split('\n');
+            for (const line of lines) {
+                if (line.trim()) {
+                    this.processLogLine(line);
+                }
+            }
+        });
+        
+        this.tail.stderr.on('data', (data) => {
+            Logger.warn('Tail stderr:', data.toString());
+        });
+        
+        this.tail.on('close', (code) => {
+            Logger.warn('Tail process closed', { code });
+            // Restart after 5 seconds
+            setTimeout(() => this.start(), 5000);
+        });
+        
+        this.tail.on('error', (err) => {
+            Logger.error('Tail process error:', err.message);
+            setTimeout(() => this.start(), 5000);
+        });
+    }
+    
+    stop() {
+        if (this.tail) {
+            this.tail.kill();
+            this.tail = null;
+            Logger.info('Log monitor stopped');
+        }
+    }
+    
+    processLogLine(line) {
+        try {
+            // Look for OTP-related log entries
+            // Museum server logs OTP generation in various formats
+            const patterns = [
+                // Pattern 1: Museum skipping email - MOST IMPORTANT (matches "Skipping sending email to andreas@due.ren: Verification code: 192305")
+                /Skipping sending email to\s+([^\s:]+):\s*Verification code:\s*(\d{6})/i,
+                // Pattern 2: Direct OTP generation logs
+                /sendOTT.*email[:\s]+([^\s]+).*code[:\s]+(\d{6})/i,
+                // Pattern 3: User registration/login with OTP
+                /generateOTT.*user[:\s]+([^\s]+).*verification.*code[:\s]+(\d{6})/i,
+                // Pattern 4: Email change OTP
+                /changeEmail.*email[:\s]+([^\s]+).*otp[:\s]+(\d{6})/i,
+                // Pattern 5: Generic OTP patterns in logs
+                /ott.*([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}).*(\d{6})/i
+            ];
+            
+            for (const pattern of patterns) {
+                const match = line.match(pattern);
+                if (match) {
+                    const email = match[1];
+                    const otpCode = match[2];
+                    
+                    // Create unique identifier to prevent duplicates
+                    const otpId = `${email}:${otpCode}:${Date.now().toString().slice(-6)}`;
+                    
+                    if (!this.processedOTPs.has(otpId)) {
+                        this.processedOTPs.add(otpId);
+                        this.sendOTPEmail(email, otpCode, line);
+                        
+                        // Clean up old OTPs (keep last 100)
+                        if (this.processedOTPs.size > 100) {
+                            const oldOTPs = Array.from(this.processedOTPs).slice(0, 50);
+                            oldOTPs.forEach(otp => this.processedOTPs.delete(otp));
+                        }
+                    }
+                    break;
+                }
+            }
+        } catch (err) {
+            Logger.error('Error processing log line:', {
+                error: err.message,
+                line: line.substring(0, 100)
+            });
+        }
+    }
+    
+    async sendOTPEmail(email, otpCode, logLine) {
+        try {
+            Logger.info('Processing OTP email request', {
+                email: email,
+                otpCode: otpCode.substring(0, 2) + '****', // Partial OTP for logging
+                source: logLine.substring(0, 100)
+            });
+            
+            // Determine template type based on context
+            let templateName = 'ott';
+            let subject = 'Ente - Verification Code';
+            
+            if (logLine.toLowerCase().includes('change') || logLine.toLowerCase().includes('email')) {
+                templateName = 'ott_change_email';
+                subject = 'Ente - Email Change Verification';
+            }
+            
+            // Render email template
+            const html = this.emailTemplate.render(templateName, {
+                VerificationCode: otpCode
+            });
+            
+            // Send email
+            await this.emailSender.sendEmail(email, subject, html);
+            
+            Logger.info('OTP email sent successfully', {
+                email: email,
+                template: templateName
+            });
+            
+        } catch (err) {
+            Logger.error('Failed to send OTP email:', {
+                error: err.message,
+                email: email,
+                otpCode: otpCode.substring(0, 2) + '****'
+            });
+        }
+    }
+}
+
+// Health check endpoint
+class HealthServer {
+    constructor(port = 8081) {
+        this.port = port;
+        this.server = null;
+    }
+    
+    start() {
+        const http = require('http');
+        
+        this.server = http.createServer((req, res) => {
+            if (req.url === '/health') {
+                res.writeHead(200, { 'Content-Type': 'application/json' });
+                res.end(JSON.stringify({
+                    status: 'healthy',
+                    service: 'ente-otp-email-monitor',
+                    timestamp: new Date().toISOString(),
+                    processedOTPs: monitor ? monitor.processedOTPs.size : 0
+                }));
+            } else {
+                res.writeHead(404);
+                res.end('Not Found');
+            }
+        });
+        
+        this.server.listen(this.port, () => {
+            Logger.info(`Health server listening on port ${this.port}`);
+        });
+    }
+    
+    stop() {
+        if (this.server) {
+            this.server.close();
+            Logger.info('Health server stopped');
+        }
+    }
+}
+
+// Main application
+let monitor = null;
+let healthServer = null;
+
+async function main() {
+    try {
+        Logger.info('Starting Ente OTP Email Monitor');
+        
+        // Validate environment
+        if (!process.env.CLOUDRON_EMAIL_SMTP_SERVER) {
+            throw new Error('CLOUDRON_EMAIL_SMTP_SERVER not found. Email addon may not be configured.');
+        }
+        
+        // Initialize components
+        const emailTemplate = new EmailTemplate(CONFIG.EMAIL_TEMPLATES_DIR);
+        const emailSender = new EmailSender(CONFIG);
+        
+        // Test email connectivity
+        Logger.info('Testing email connectivity...');
+        await emailSender.transporter.verify();
+        Logger.info('Email connectivity verified');
+        
+        // Start log monitor
+        monitor = new LogMonitor(CONFIG.LOG_FILE, emailSender, emailTemplate);
+        monitor.start();
+        
+        // Start health server
+        healthServer = new HealthServer();
+        healthServer.start();
+        
+        Logger.info('Ente OTP Email Monitor started successfully');
+        
+        // Handle graceful shutdown
+        process.on('SIGINT', () => {
+            Logger.info('Received SIGINT, shutting down gracefully...');
+            if (monitor) monitor.stop();
+            if (healthServer) healthServer.stop();
+            process.exit(0);
+        });
+        
+        process.on('SIGTERM', () => {
+            Logger.info('Received SIGTERM, shutting down gracefully...');
+            if (monitor) monitor.stop();
+            if (healthServer) healthServer.stop();
+            process.exit(0);
+        });
+        
+    } catch (err) {
+        Logger.error('Failed to start OTP Email Monitor:', err.message);
+        process.exit(1);
+    }
+}
+
+// Start the application
+if (require.main === module) {
+    main();
+}
+
+module.exports = {
+    LogMonitor,
+    EmailSender,
+    EmailTemplate,
+    Logger
+};

package.json

@@ -0,0 +1,22 @@
+{
+  "name": "ente-otp-email-monitor",
+  "version": "1.0.0",
+  "description": "OTP email monitoring service for Ente Cloudron app",
+  "main": "otp-email-monitor.js",
+  "scripts": {
+    "start": "node otp-email-monitor.js",
+    "test": "echo \"Error: no test specified\" && exit 1"
+  },
+  "dependencies": {
+    "nodemailer": "^6.9.0"
+  },
+  "keywords": [
+    "ente",
+    "otp",
+    "email",
+    "cloudron",
+    "monitoring"
+  ],
+  "author": "Ente Cloudron Integration",
+  "license": "Apache-2.0"
+}

setup-ente-cli.sh

@@ -0,0 +1,20 @@
+#!/bin/bash
+# Setup Ente CLI for custom server
+
+echo "Setting up Ente CLI for custom server..."
+
+# Create config directory
+mkdir -p ~/.ente
+
+# Create the CLI config with custom endpoint
+cat > ~/.ente/config.yaml << EOF
+host: https://ente.due.ren
+EOF
+
+echo "Configuration created at ~/.ente/config.yaml"
+echo ""
+echo "Now you can add your account:"
+echo "  ente account add"
+echo ""
+echo "Then use admin commands:"
+echo "  ente admin update-subscription --admin-user admin@due.ren --user user@example.com --storage 1000 --valid-for 365"

start-debug.sh

@@ -0,0 +1,150 @@
+#!/bin/bash
+
+# Better signal handling - forward signals to child processes
+trap 'kill -TERM $SERVER_PID; kill -TERM $CADDY_PID; exit' TERM INT
+
+set -eu
+
+echo "==> Starting Ente Cloudron app (DEBUG MODE)..."
+
+# Create necessary directories
+mkdir -p /app/data/config /app/data/logs /app/data/caddy
+
+# Check if web directories exist
+echo "==> Checking web app directories:"
+for app in photos accounts auth cast; do
+    if [ -d "/app/web/$app" ]; then
+        echo "==> Found: /app/web/$app"
+        ls -la "/app/web/$app" | head -5
+    else
+        echo "==> WARNING: Missing /app/web/$app - creating placeholder"
+        mkdir -p "/app/web/$app"
+        echo "<html><body><h1>$app app placeholder</h1></body></html>" > "/app/web/$app/index.html"
+    fi
+done
+
+# Create a simple test Caddyfile first
+echo "==> Creating simple test Caddyfile"
+cat > /app/data/Caddyfile <<'EOT'
+{
+    admin off
+    auto_https off
+}
+
+:3080 {
+    log {
+        output stdout
+        format console
+        level DEBUG
+    }
+
+    # Health check endpoint
+    handle /health {
+        header Content-Type "application/json"
+        respond "{\"status\": \"OK\", \"timestamp\": \"{{now | date \"2006-01-02T15:04:05Z07:00\"}}\"}" 200
+    }
+
+    # Test endpoint
+    handle /test {
+        respond "Caddy is working on port 3080!" 200
+    }
+
+    # API proxy to Museum server
+    handle /api/* {
+        uri strip_prefix /api
+        reverse_proxy localhost:8080 {
+            transport http {
+                read_timeout 60s
+                write_timeout 60s
+            }
+            # Add error handling
+            handle_errors {
+                respond "{\"error\": \"Museum server not available\"}" 503
+            }
+        }
+    }
+
+    # Serve web apps with fallback
+    handle {
+        root * /app/web/photos
+        try_files {path} {path}/ /index.html
+        file_server {
+            browse
+        }
+    }
+}
+EOT
+
+# Start a simple Museum mock server for testing
+echo "==> Starting mock Museum server on port 8080"
+cat > /tmp/museum-mock.js <<'EOF'
+const http = require('http');
+const server = http.createServer((req, res) => {
+    console.log(`Museum mock: ${req.method} ${req.url}`);
+    res.writeHead(200, { 'Content-Type': 'application/json' });
+    res.end(JSON.stringify({ status: 'ok', path: req.url, timestamp: new Date().toISOString() }));
+});
+server.listen(8080, '127.0.0.1', () => {
+    console.log('Museum mock server running on http://127.0.0.1:8080');
+});
+EOF
+node /tmp/museum-mock.js > /app/data/logs/museum-mock.log 2>&1 &
+SERVER_PID=$!
+echo "==> Mock Museum server started (PID: $SERVER_PID)"
+
+# Wait for Museum mock to be ready
+sleep 2
+
+# Test Museum mock
+echo "==> Testing Museum mock server..."
+curl -s http://localhost:8080/test || echo "WARNING: Museum mock not responding"
+
+# Validate Caddyfile
+echo "==> Validating Caddyfile..."
+caddy validate --config /app/data/Caddyfile --adapter caddyfile || {
+    echo "==> ERROR: Caddyfile validation failed!"
+    exit 1
+}
+
+# Start Caddy with explicit environment
+echo "==> Starting Caddy web server..."
+CADDY_FORMAT=console caddy run --config /app/data/Caddyfile --adapter caddyfile 2>&1 | tee /app/data/logs/caddy-combined.log &
+CADDY_PID=$!
+echo "==> Caddy started (PID: $CADDY_PID)"
+
+# Wait for Caddy to start
+echo "==> Waiting for Caddy to start..."
+for i in {1..30}; do
+    if curl -s http://localhost:3080/health > /dev/null; then
+        echo "==> Caddy is responding!"
+        break
+    fi
+    echo -n "."
+    sleep 1
+done
+echo
+
+# Check process status
+echo "==> Process status:"
+ps aux | grep -E "(caddy|node)" | grep -v grep || echo "No processes found"
+
+# Check port status
+echo "==> Port status:"
+netstat -tlnp 2>/dev/null | grep -E "(3080|8080)" || lsof -i :3080 -i :8080 2>/dev/null || echo "Cannot check port status"
+
+# Test endpoints
+echo "==> Testing endpoints:"
+echo "Health check:"
+curl -s http://localhost:3080/health | jq . || echo "Failed"
+echo -e "\nTest endpoint:"
+curl -s http://localhost:3080/test || echo "Failed"
+echo -e "\nAPI proxy:"
+curl -s http://localhost:3080/api/status | jq . || echo "Failed"
+
+echo "==> Startup complete. Services:"
+echo "   - Caddy PID: $CADDY_PID"
+echo "   - Museum Mock PID: $SERVER_PID"
+echo "==> Logs: /app/data/logs/"
+
+# Keep running
+wait $SERVER_PID $CADDY_PID

start.sh

@@ -47,7 +47,6 @@ log "INFO" "Creating necessary directories"
 mkdir -p /app/data/ente/server
 mkdir -p /app/data/ente/web
 mkdir -p /app/data/tmp
-mkdir -p /app/data/web/{photos,accounts,auth,cast}
 
 # ===============================================
 # Repository setup
@@ -78,55 +77,23 @@ fi
 # ===============================================
 log "INFO" "Setting up configuration"
 
-# S3 configuration
-S3_CONFIG="/app/data/s3.env"
-if [ ! -f "$S3_CONFIG" ]; then
-  log "INFO" "Creating default S3 configuration file"
-  cat > "$S3_CONFIG" << EOF
-# Ente S3 Storage Configuration
-# Edit these values with your S3-compatible storage credentials
+# S3 configuration - HARDCODED VALUES
+S3_ACCESS_KEY="QZ5M3VMBUHDTIFDFCD8E"
+S3_SECRET_KEY="pz1eHYjU1NwAbbruedc7swzCuszd57p1rGSFVzjv"
+S3_ENDPOINT="https://s3.eu-central-2.wasabisys.com"
+S3_REGION="eu-central-2"
+S3_BUCKET="ente-due-ren"
 
-# Required settings
-S3_ACCESS_KEY=your-access-key
-S3_SECRET_KEY=your-secret-key
-S3_ENDPOINT=your-s3-endpoint  # e.g., s3.amazonaws.com
-S3_REGION=your-region         # e.g., us-east-1
-S3_BUCKET=your-bucket-name
+log "INFO" "Using hardcoded S3 configuration"
+log "INFO" "S3 Endpoint: $S3_ENDPOINT"
+log "INFO" "S3 Region: $S3_REGION"
+log "INFO" "S3 Bucket: $S3_BUCKET"
 
-# Optional settings
-# S3_PREFIX=ente/              # Optional prefix for all objects
-# S3_PUBLIC_URL=               # Optional public URL for the bucket (if different from endpoint)
-EOF
-  chmod 600 "$S3_CONFIG"
-  log "INFO" "Created S3 config template at ${S3_CONFIG}"
-  log "WARN" "⚠️ YOU MUST EDIT /app/data/s3.env WITH YOUR ACTUAL S3 CREDENTIALS ⚠️"
-else
-  log "INFO" "S3 configuration file already exists"
-fi
+# Museum server configuration - create configurations directory structure
+MUSEUM_CONFIG_DIR="/app/data/ente/server/configurations"
+MUSEUM_CONFIG="$MUSEUM_CONFIG_DIR/local.yaml"
+mkdir -p "$MUSEUM_CONFIG_DIR"
 
-# Load S3 configuration
-if [ -f "$S3_CONFIG" ]; then
-  source "$S3_CONFIG"
-  log "INFO" "Loaded S3 configuration"
-  # Validate S3 configuration
-  if [ -z "$S3_ENDPOINT" ]; then
-    log "ERROR" "S3_ENDPOINT is not set. S3 storage will not work."
-  fi
-  if [ -z "$S3_ACCESS_KEY" ]; then
-    log "ERROR" "S3_ACCESS_KEY is not set. S3 storage will not work."
-  fi
-  if [ -z "$S3_SECRET_KEY" ]; then
-    log "ERROR" "S3_SECRET_KEY is not set. S3 storage will not work."
-  fi
-  if [ -z "$S3_BUCKET" ]; then
-    log "ERROR" "S3_BUCKET is not set. S3 storage will not work."
-  fi
-else
-  log "WARN" "S3 configuration file not found at $S3_CONFIG. S3 storage may not be configured."
-fi
-
-# Museum server configuration
-MUSEUM_CONFIG="/app/data/ente/server/museum.yaml"
 if [ ! -f "$MUSEUM_CONFIG" ]; then
   log "INFO" "Creating Museum server configuration"
   cat > "$MUSEUM_CONFIG" << EOF
@@ -137,12 +104,23 @@ port: 8080
 host: 0.0.0.0
 log_level: info
 
+# Key used for encrypting customer data (REQUIRED)
+key:
+  encryption: yvmG/RnzKrbCb9L3mgsmoxXr9H7i2Z4qlbT0mL3ln4w=
+  hash: KXYiG07wC7GIgvCSdg+WmyWdXDAn6XKYJtp/wkEU7x573+byBRAYtpTP0wwvi8i/4l37uicX1dVTUzwH3sLZyw==
+
+# JWT secrets (REQUIRED)
+jwt:
+  secret: i2DecQmfGreG6q1vBj5tCokhlN41gcfS2cjOs9Po-u8=
+
 # Database configuration
 db:
-  driver: postgres
-  source: "postgres://${CLOUDRON_POSTGRESQL_USERNAME}:${CLOUDRON_POSTGRESQL_PASSWORD}@${CLOUDRON_POSTGRESQL_HOST}:${CLOUDRON_POSTGRESQL_PORT}/${CLOUDRON_POSTGRESQL_DATABASE}?sslmode=disable"
-  max_conns: 10
-  max_idle: 5
+  host: ${CLOUDRON_POSTGRESQL_HOST}
+  port: ${CLOUDRON_POSTGRESQL_PORT}
+  name: ${CLOUDRON_POSTGRESQL_DATABASE}
+  user: ${CLOUDRON_POSTGRESQL_USERNAME}
+  password: ${CLOUDRON_POSTGRESQL_PASSWORD}
+  sslmode: disable
 
 # CORS settings
 cors:
@@ -151,21 +129,32 @@ cors:
 
 # S3 storage configuration
 s3:
-  endpoint: "${S3_ENDPOINT:-s3.amazonaws.com}"
-  region: "${S3_REGION:-us-east-1}"
-  access_key: "${S3_ACCESS_KEY}"
-  secret_key: "${S3_SECRET_KEY}"
-  bucket: "${S3_BUCKET}"
-  public_url: "https://${CLOUDRON_APP_FQDN}/photos"
+  # For Wasabi, we need path style URLs
+  are_local_buckets: false
+  use_path_style_urls: true
+  
+  # Primary bucket configuration (named bucket structure required by Museum)
+  b2-eu-cen:
+    endpoint: "${S3_ENDPOINT}"
+    region: "${S3_REGION}"
+    key: "${S3_ACCESS_KEY}"
+    secret: "${S3_SECRET_KEY}"
+    bucket: "${S3_BUCKET}"
 
 # Email settings
 email:
   enabled: true
-  host: "${CLOUDRON_SMTP_SERVER:-localhost}"
-  port: ${CLOUDRON_SMTP_PORT:-25}
-  username: "${CLOUDRON_SMTP_USERNAME:-""}"
-  password: "${CLOUDRON_SMTP_PASSWORD:-""}"
-  from: "Ente <${CLOUDRON_MAIL_FROM:-no-reply@${CLOUDRON_APP_DOMAIN}}>"
+  host: "${CLOUDRON_MAIL_SMTP_SERVER:-localhost}"
+  port: ${CLOUDRON_MAIL_SMTP_PORT:-25}
+  username: "${CLOUDRON_MAIL_SMTP_USERNAME:-}"
+  password: "${CLOUDRON_MAIL_SMTP_PASSWORD:-}"
+  from: "${CLOUDRON_MAIL_FROM:-no-reply@${CLOUDRON_APP_FQDN:-localhost}}"
+
+# WebAuthn configuration for passkey support
+webauthn:
+  rpid: "${CLOUDRON_APP_FQDN:-localhost}"
+  rporigins:
+    - "https://${CLOUDRON_APP_FQDN:-localhost}"
 EOF
   chmod 600 "$MUSEUM_CONFIG"
   log "INFO" "Created Museum configuration at ${MUSEUM_CONFIG}"
@@ -201,261 +190,83 @@ USE_PLACEHOLDER=false
 
 log "INFO" "Setting up Museum server binary"
 
-# Function to validate a binary
-validate_binary() {
-  local bin_path="$1"
-  
-  # Basic file existence check
-  if [ ! -f "$bin_path" ]; then
-    return 1
-  fi
-  
-  # Check if file is executable
-  if [ ! -x "$bin_path" ]; then
-    chmod +x "$bin_path" || return 1
-  fi
-  
-  # Check if it's a text file (most likely an error message)
-  if file "$bin_path" | grep -q "text"; then
-    return 1
-  fi
-  
-  # Check if it's a valid binary type
-  if ! file "$bin_path" | grep -q -E "ELF|Mach-O|PE32"; then
-    return 1
-  fi
-  
-  return 0
-}
-
-# Check and remove invalid binary
-if [ -f "$MUSEUM_BIN" ]; then
-  if ! validate_binary "$MUSEUM_BIN"; then
-    log "WARN" "Found invalid Museum binary, removing"
-    rm -f "$MUSEUM_BIN"
-  else
-    log "INFO" "Found valid Museum binary"
-  fi
+# Copy Museum binary from build location to data directory
+MUSEUM_BUILD_BIN="/app/museum-bin/museum"
+log "INFO" "Checking for pre-built Museum binary at: $MUSEUM_BUILD_BIN"
+if [ -f "$MUSEUM_BUILD_BIN" ]; then
+  log "INFO" "Found pre-built Museum binary, copying to data directory"
+  cp "$MUSEUM_BUILD_BIN" "$MUSEUM_BIN"
+  chmod +x "$MUSEUM_BIN"
+  log "INFO" "Copied Museum binary to $MUSEUM_BIN"
+else
+  log "WARN" "Pre-built Museum binary not found at $MUSEUM_BUILD_BIN"
 fi
 
-# Build or download if needed
-if [ ! -f "$MUSEUM_BIN" ]; then
-  # Try building first if Go is available
-  if command -v go >/dev/null 2>&1; then
-    log "INFO" "Go is available, attempting to build Museum server"
-    
-    cd "$ENTE_REPO_DIR/server"
-    export GOPATH="/app/data/go"
-    export PATH="$GOPATH/bin:$PATH"
-    mkdir -p "$GOPATH/src" "$GOPATH/bin" "$GOPATH/pkg"
-    
-    # Install dependencies if needed
-    if command -v apt-get >/dev/null 2>&1; then
-      log "INFO" "Installing build dependencies"
-      apt-get update -y && apt-get install -y gcc libsodium-dev pkg-config
-    fi
-    
-    log "INFO" "Building Museum server..."
-    if go build -o "$MUSEUM_BIN" ./cmd/museum; then
-      if validate_binary "$MUSEUM_BIN"; then
-        log "INFO" "Successfully built Museum server"
-      else
-        log "ERROR" "Build completed but resulted in an invalid binary"
-        rm -f "$MUSEUM_BIN"
-      fi
-    else
-      log "ERROR" "Failed to build Museum server"
-    fi
+# Copy migration files to Museum working directory
+MUSEUM_MIGRATIONS_DIR="/app/data/ente/server/migrations"
+REPO_MIGRATIONS_DIR="/app/data/ente/repository/server/migrations"
+if [ ! -d "$MUSEUM_MIGRATIONS_DIR" ] && [ -d "$REPO_MIGRATIONS_DIR" ]; then
+  log "INFO" "Copying database migration files"
+  cp -r "$REPO_MIGRATIONS_DIR" "$MUSEUM_MIGRATIONS_DIR"
+  log "INFO" "Copied migration files to $MUSEUM_MIGRATIONS_DIR"
+else
+  log "INFO" "Migration files already exist or source not available"
+fi
+
+# Copy web templates to Museum working directory  
+MUSEUM_WEB_TEMPLATES_DIR="/app/data/ente/server/web-templates"
+REPO_WEB_TEMPLATES_DIR="/app/data/ente/repository/server/web-templates"
+if [ ! -d "$MUSEUM_WEB_TEMPLATES_DIR" ] && [ -d "$REPO_WEB_TEMPLATES_DIR" ]; then
+  log "INFO" "Copying web templates"
+  cp -r "$REPO_WEB_TEMPLATES_DIR" "$MUSEUM_WEB_TEMPLATES_DIR"
+  log "INFO" "Copied web templates to $MUSEUM_WEB_TEMPLATES_DIR"
+else
+  log "INFO" "Web templates already exist or source not available"
+fi
+
+# Copy mail templates to Museum working directory (required for email functionality)
+MUSEUM_MAIL_TEMPLATES_DIR="/app/data/ente/server/mail-templates"
+REPO_MAIL_TEMPLATES_DIR="/app/data/ente/repository/server/mail-templates"
+if [ ! -d "$MUSEUM_MAIL_TEMPLATES_DIR" ] && [ -d "$REPO_MAIL_TEMPLATES_DIR" ]; then
+  log "INFO" "Copying mail templates"
+  cp -r "$REPO_MAIL_TEMPLATES_DIR" "$MUSEUM_MAIL_TEMPLATES_DIR"
+  log "INFO" "Copied mail templates to $MUSEUM_MAIL_TEMPLATES_DIR"
+else
+  log "INFO" "Mail templates already exist or source not available"
+fi
+
+# Check if Museum binary exists and is valid
+log "INFO" "Checking for Museum binary at: $MUSEUM_BIN"
+if [ -f "$MUSEUM_BIN" ]; then
+  log "INFO" "Museum binary file exists"
+  if [ -x "$MUSEUM_BIN" ]; then
+    log "INFO" "Museum binary is executable"
+    # Since Museum's --help and --version commands trigger full startup (including DB migration),
+    # we'll trust that an existing executable binary should work
+    log "INFO" "Museum binary is ready to use"
   else
-    log "INFO" "Go is not available, skipping build attempt"
-  fi
-  
-  # If build failed or wasn't attempted, try downloading
-  if [ ! -f "$MUSEUM_BIN" ] || ! validate_binary "$MUSEUM_BIN"; then
-    log "INFO" "Attempting to download pre-built Museum server binary"
-    
-    # Determine architecture
-    ARCH=$(uname -m)
-    OS=$(uname -s | tr '[:upper:]' '[:lower:]')
-    
-    # Map architecture to standard names
-    if [ "$ARCH" = "x86_64" ]; then ARCH="amd64"; fi
-    if [ "$ARCH" = "aarch64" ] || [ "$ARCH" = "arm64" ]; then ARCH="arm64"; fi
-    
-    log "INFO" "Detected system: $OS-$ARCH"
-    
-    # Define possible download URLs
-    DOWNLOAD_URLS=(
-      "https://github.com/ente-io/ente/releases/latest/download/museum-${OS}-${ARCH}"
-      "https://github.com/ente-io/ente/releases/download/latest/museum-${OS}-${ARCH}"
-      "https://github.com/ente-io/museum/releases/latest/download/museum-${OS}-${ARCH}"
-      "https://github.com/ente-io/museum/releases/download/latest/museum-${OS}-${ARCH}"
-      "https://github.com/ente-io/ente/releases/download/v0.9.0/museum-${OS}-${ARCH}"
-      "https://github.com/ente-io/museum/releases/download/v0.9.0/museum-${OS}-${ARCH}"
-    )
-    
-    # Try each URL
-    SUCCESS=false
-    for URL in "${DOWNLOAD_URLS[@]}"; do
-      log "INFO" "Attempting download from $URL"
-      if curl -L -f -s -o "$MUSEUM_BIN.tmp" "$URL"; then
-        chmod +x "$MUSEUM_BIN.tmp"
-        if validate_binary "$MUSEUM_BIN.tmp"; then
-          mv "$MUSEUM_BIN.tmp" "$MUSEUM_BIN"
-          log "INFO" "Successfully downloaded Museum server binary"
-          SUCCESS=true
-          break
-        else
-          log "WARN" "Downloaded file is not a valid binary"
-          rm -f "$MUSEUM_BIN.tmp"
-        fi
-      else
-        log "WARN" "Failed to download from $URL"
-      fi
-    done
-    
-    if [ "$SUCCESS" = false ]; then
-      log "ERROR" "All download attempts failed"
+    log "INFO" "Museum binary exists but is not executable, fixing permissions"
+    chmod +x "$MUSEUM_BIN"
+    if [ -x "$MUSEUM_BIN" ]; then
+      log "INFO" "Fixed permissions, Museum binary is ready to use"
+    else
+      log "WARN" "Failed to fix permissions, using placeholder"
       USE_PLACEHOLDER=true
     fi
   fi
-fi
-
-# Final check for Museum binary
-if [ ! -f "$MUSEUM_BIN" ] || ! validate_binary "$MUSEUM_BIN"; then
-  log "WARN" "No valid Museum binary available"
+else
+  log "WARN" "Museum binary file not found at $MUSEUM_BIN"
+  log "INFO" "Checking directory contents: $(ls -la $(dirname $MUSEUM_BIN) 2>/dev/null || echo 'Directory not found')"
   USE_PLACEHOLDER=true
 fi
 
+
 # ===============================================
 # Web Application Setup
 # ===============================================
-log "INFO" "Setting up web applications"
+log "INFO" "Web applications are pre-built and available in /app/web/"
 
-# Function to create a placeholder page
-create_placeholder_page() {
-  local app_name="$1"
-  local app_dir="/app/data/web/$app_name"
-  
-  mkdir -p "$app_dir"
-  
-  cat > "$app_dir/index.html" << EOF
-<!DOCTYPE html>
-<html lang="en">
-<head>
-  <meta charset="UTF-8">
-  <meta name="viewport" content="width=device-width, initial-scale=1.0">
-  <title>Ente $app_name</title>
-  <style>
-    body {
-      font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif;
-      line-height: 1.6;
-      color: #333;
-      max-width: 650px;
-      margin: 0 auto;
-      padding: 20px;
-    }
-    .container {
-      background-color: #fff;
-      border-radius: 8px;
-      box-shadow: 0 2px 10px rgba(0, 0, 0, 0.1);
-      padding: 30px;
-      margin-top: 50px;
-    }
-    h1 {
-      color: #000;
-      margin-top: 0;
-      font-weight: 600;
-    }
-    .logo {
-      text-align: center;
-      margin-bottom: 20px;
-    }
-    .logo img {
-      max-width: 140px;
-    }
-    .alert {
-      background-color: #f8f9fa;
-      border-left: 4px solid #2196F3;
-      padding: 15px;
-      margin-bottom: 20px;
-    }
-    .alert-warn {
-      border-color: #ff9800;
-    }
-    .setup-box {
-      background-color: #f5f5f5;
-      padding: 20px;
-      border-radius: 5px;
-      margin-top: 20px;
-    }
-    code {
-      background-color: #f1f1f1;
-      padding: 2px 5px;
-      border-radius: 3px;
-      font-family: monospace;
-      font-size: 0.9em;
-    }
-    a {
-      color: #2196F3;
-      text-decoration: none;
-    }
-    a:hover {
-      text-decoration: underline;
-    }
-  </style>
-</head>
-<body>
-  <div class="container">
-    <div class="logo">
-      <img src="https://raw.githubusercontent.com/ente-io/ente/main/web/packages/photos/public/images/logo_vertical.svg" alt="Ente Logo">
-    </div>
-    <h1>Ente $app_name</h1>
-    
-    <div class="alert">
-      <strong>Status:</strong> The Ente server is being configured.
-    </div>
-
-    <p>
-      This is the Ente $app_name application running on your Cloudron. To complete the setup:
-    </p>
-    
-    <div class="setup-box">
-      <ol>
-        <li>Configure your S3 storage in <code>/app/data/s3.env</code></li>
-        <li>Ensure the Museum server is properly running</li>
-        <li>You might need to restart the app after configuration changes</li>
-      </ol>
-    </div>
-    
-    <p style="margin-top: 30px; text-align: center;">
-      <a href="https://github.com/ente-io/ente" target="_blank">GitHub Repository</a> &middot;
-      <a href="https://help.ente.io" target="_blank">Documentation</a>
-    </p>
-  </div>
-</body>
-</html>
-EOF
-
-  # Create runtime config
-  cat > "$app_dir/runtime-config.js" << EOF
-window.RUNTIME_CONFIG = {
-  API_URL: "/api",
-  PUBLIC_ALBUMS_URL: "/public",
-  DEBUG: true
-};
-console.log("Loaded Ente runtime config:", window.RUNTIME_CONFIG);
-EOF
-  
-  log "INFO" "Created placeholder for $app_name app"
-}
-
-# Create placeholder pages for each app if they don't exist
-for APP in photos accounts auth cast; do
-  if [ ! -f "/app/data/web/$APP/index.html" ]; then
-    create_placeholder_page "$APP"
-  fi
-done
+# Web apps are pre-built with relative API paths (/api) that work with any domain
 
 # ===============================================
 # Node.js Placeholder Server
@@ -777,7 +588,7 @@ if [ "$USE_PLACEHOLDER" = true ]; then
 else
   log "INFO" "Starting actual Museum server"
   cd /app/data/ente/server
-  "$MUSEUM_BIN" --config "$MUSEUM_CONFIG" > "$MUSEUM_LOG" 2>&1 &
+  "$MUSEUM_BIN" > "$MUSEUM_LOG" 2>&1 &
   MUSEUM_PID=$!
   log "INFO" "Started Museum server with PID: $MUSEUM_PID"
   
@@ -821,60 +632,165 @@ cat > "$CADDY_CONFIG" << EOF
 :3080 {
     log {
         output file /app/data/logs/caddy.log
+        level INFO
     }
 
-    # Static web apps
+    # Enable compression
+    encode gzip
+
+    # Root redirect - must be first
+    redir / /photos/ 301
+
+    # CORS preflight handling
+    @options method OPTIONS
+    handle @options {
+        header {
+            Access-Control-Allow-Origin "*"
+            Access-Control-Allow-Methods "GET, POST, PUT, DELETE, OPTIONS"
+            Access-Control-Allow-Headers "*"
+            Access-Control-Max-Age "3600"
+        }
+        respond 204
+    }
+
+    # API endpoints - STRIP /api prefix and proxy to Museum server
+    handle_path /api/* {
+        reverse_proxy localhost:8080
+        header {
+            Access-Control-Allow-Origin "*"
+            Access-Control-Allow-Methods "GET, POST, PUT, DELETE, OPTIONS"
+            Access-Control-Allow-Headers "*"
+            Access-Control-Allow-Credentials "true"
+        }
+    }
+    
+    # API endpoints for auth app
+    handle_path /auth/api/* {
+        reverse_proxy localhost:8080
+        header {
+            Access-Control-Allow-Origin "*"
+            Access-Control-Allow-Methods "GET, POST, PUT, DELETE, OPTIONS"
+            Access-Control-Allow-Headers "*"
+            Access-Control-Allow-Credentials "true"
+        }
+    }
+    
+    # API endpoints for cast app
+    handle_path /cast/api/* {
+        reverse_proxy localhost:8080
+        header {
+            Access-Control-Allow-Origin "*"
+            Access-Control-Allow-Methods "GET, POST, PUT, DELETE, OPTIONS"
+            Access-Control-Allow-Headers "*"
+            Access-Control-Allow-Credentials "true"
+        }
+    }
+    
+    # API endpoints for accounts app
+    handle_path /accounts/api/* {
+        reverse_proxy localhost:8080
+        header {
+            Access-Control-Allow-Origin "*"
+            Access-Control-Allow-Methods "GET, POST, PUT, DELETE, OPTIONS"
+            Access-Control-Allow-Headers "*"
+            Access-Control-Allow-Credentials "true"
+        }
+    }
+    
+    # Health check endpoint (direct, no /api prefix)
+    handle /ping {
+        reverse_proxy localhost:8080
+    }
+
+    # Static files for Next.js assets from all apps
+    handle /_next/* {
+        root * /app/web/photos
+        file_server
+        header {
+            Cache-Control "public, max-age=31536000"
+        }
+    }
+
+    # Static images and assets (served from photos app by default)
+    handle /images/* {
+        root * /app/web/photos
+        file_server
+        header {
+            Cache-Control "public, max-age=86400"
+        }
+    }
+
+    handle /favicon.ico {
+        root * /app/web/photos
+        file_server
+        header {
+            Cache-Control "public, max-age=86400"
+        }
+    }
+
+    # Photos app
     handle_path /photos/* {
-        root * /app/data/web/photos
+        root * /app/web/photos
         try_files {path} /index.html
         file_server
     }
     
+    # Accounts app
     handle_path /accounts/* {
-        root * /app/data/web/accounts
+        root * /app/web/accounts
         try_files {path} /index.html
         file_server
     }
     
+    # Auth app
     handle_path /auth/* {
-        root * /app/data/web/auth
+        root * /app/web/auth
         try_files {path} /index.html
         file_server
     }
     
+    # Cast app
     handle_path /cast/* {
-        root * /app/data/web/cast
+        root * /app/web/cast
         try_files {path} /index.html
         file_server
     }
-
-    # API endpoints
-    handle /api/* {
-        reverse_proxy localhost:8080
-    }
     
-    # Public albums endpoint
-    handle /public/* {
-        reverse_proxy localhost:8080
-    }
-    
-    # Health check endpoint
-    handle /health {
-        reverse_proxy localhost:8080
-    }
-    
-    # Redirect root to photos
-    handle {
-        redir / /photos/
+    # Root redirect - specifically match root path only
+    @root path /
+    handle @root {
+        redir /photos/ permanent
     }
 }
 EOF
 
+log "INFO" "Validating Caddy configuration"
+if caddy validate --config "$CADDY_CONFIG" 2>&1 | tee -a "$LOG_FILE"; then
+    log "INFO" "Caddy configuration is valid"
+else
+    log "ERROR" "Caddy configuration validation failed!"
+    log "ERROR" "Caddyfile contents:"
+    cat "$CADDY_CONFIG" | while read -r line; do
+        log "ERROR" "  $line"
+    done
+fi
+
 log "INFO" "Starting Caddy web server"
 caddy run --config "$CADDY_CONFIG" > /app/data/logs/caddy.log 2>&1 &
 CADDY_PID=$!
 log "INFO" "Caddy web server started with PID: $CADDY_PID"
 
+# Wait a moment to see if Caddy stays running
+sleep 2
+if ps -p $CADDY_PID > /dev/null; then
+    log "INFO" "Caddy is still running after 2 seconds"
+else
+    log "ERROR" "Caddy has crashed! Last 20 lines of Caddy log:"
+    tail -n 20 /app/data/logs/caddy.log | while read -r line; do
+        log "ERROR" "  $line"
+    done
+fi
+
 # ===============================================
 # Finalization and monitoring
 # ===============================================
@@ -890,19 +806,34 @@ cat > /app/data/SETUP-INSTRUCTIONS.md << EOF
 
 2. **Museum Server**: The server configuration is at \`/app/data/ente/server/museum.yaml\` if you need to customize settings.
 
-## Troubleshooting
+## API Endpoint
 
-- **Logs**: Check the logs at \`/app/data/logs/\` for any issues.
-- **Restart**: If you change configuration, restart the app to apply changes.
+The Ente API is available at: **https://${CLOUDRON_APP_FQDN}/api**
+
+This endpoint can be used to:
+- Configure Ente CLI tools
+- Integrate with third-party applications
+- Access the Museum server API directly
+
+For admin operations, use the Ente CLI with:
+\`\`\`bash
+ente admin --api-url https://${CLOUDRON_APP_FQDN}/api
+\`\`\`
 
 ## Web Applications
 
 The following web applications are available:
 
-- Photos: https://${CLOUDRON_APP_FQDN}/photos/
-- Accounts: https://${CLOUDRON_APP_FQDN}/accounts/
-- Auth: https://${CLOUDRON_APP_FQDN}/auth/
-- Cast: https://${CLOUDRON_APP_FQDN}/cast/
+- **Photos**: https://${CLOUDRON_APP_FQDN}/photos/ - Main photo storage and management
+- **Auth**: https://${CLOUDRON_APP_FQDN}/auth/ - 2FA authenticator app
+- **Accounts**: https://${CLOUDRON_APP_FQDN}/accounts/ - Account management
+- **Cast**: https://${CLOUDRON_APP_FQDN}/cast/ - Photo casting to devices
+
+## Troubleshooting
+
+- **Logs**: Check the logs at \`/app/data/logs/\` for any issues.
+- **Restart**: If you change configuration, restart the app to apply changes.
+- **API Issues**: All apps use the API endpoint at \`/api\`. If apps show loading spinners, check API connectivity.
 
 ## Support
 
@@ -926,6 +857,48 @@ else
   log "ERROR" "Caddy server is not running!"
 fi
 
+# ===============================================
+# OTP Email Monitor Setup
+# ===============================================
+log "INFO" "Setting up OTP Email Monitor"
+
+# Install Node.js dependencies if not already installed
+if [ ! -d "/app/data/node_modules" ]; then
+  log "INFO" "Installing Node.js dependencies for OTP Email Monitor"
+  cd /app/data
+  cp /app/pkg/package.json .
+  npm install --production --no-save
+  log "INFO" "Node.js dependencies installed successfully"
+else
+  log "INFO" "Node.js dependencies already installed"
+fi
+
+# Start OTP Email Monitor
+log "INFO" "Starting OTP Email Monitor"
+cd /app/data
+NODE_PATH="/app/data/node_modules" node /app/pkg/otp-email-monitor.js > /app/data/logs/otp-email.log 2>&1 &
+OTP_MONITOR_PID=$!
+log "INFO" "OTP Email Monitor started with PID: $OTP_MONITOR_PID"
+
+# Wait a moment to check if OTP monitor starts successfully
+sleep 2
+if ps -p $OTP_MONITOR_PID > /dev/null; then
+    log "INFO" "OTP Email Monitor is running successfully"
+else
+    log "WARN" "OTP Email Monitor may have failed to start"
+    log "WARN" "Last 10 lines of OTP email log:"
+    tail -n 10 /app/data/logs/otp-email.log | while read -r line; do
+        log "WARN" "  $line"
+    done
+fi
+
+# Copy admin helper script for easy access
+if [ -f "/app/pkg/admin-helper.sh" ]; then
+  cp /app/pkg/admin-helper.sh /app/data/
+  chmod +x /app/data/admin-helper.sh
+  log "INFO" "Admin helper script available at /app/data/admin-helper.sh"
+fi
+
 log "INFO" "Ente Cloudron app startup complete"
 
 # Keep the script running to prevent container exit

update-storage.sh

@@ -0,0 +1,57 @@
+#!/bin/bash
+# Script to update Ente user storage using the Ente CLI
+# Run this from your local machine (not inside Cloudron)
+
+# Check if ente CLI is installed
+if ! command -v ente &> /dev/null; then
+    echo "Ente CLI is not installed. Please install it first:"
+    echo ""
+    echo "For macOS:"
+    echo "  brew tap ente-io/ente"
+    echo "  brew install ente-cli"
+    echo ""
+    echo "For other systems, download from:"
+    echo "  https://github.com/ente-io/ente/releases"
+    exit 1
+fi
+
+# Your Ente instance
+ENTE_ENDPOINT="https://ente.due.ren"
+
+# Function to update subscription
+update_subscription() {
+    local admin_email="$1"
+    local user_email="$2"
+    local storage_gb="$3"
+    local valid_days="$4"
+    
+    echo "Updating subscription for: $user_email"
+    echo "Storage: ${storage_gb}GB"
+    echo "Valid for: ${valid_days} days"
+    echo "Using admin account: $admin_email"
+    echo ""
+    
+    # Run the ente CLI command
+    ente admin update-subscription \
+        --host "$ENTE_ENDPOINT" \
+        --admin-user "$admin_email" \
+        --user "$user_email" \
+        --storage "$storage_gb" \
+        --valid-for "$valid_days"
+}
+
+# Check arguments
+if [ $# -lt 4 ]; then
+    echo "Usage: $0 <admin-email> <user-email> <storage-gb> <valid-days>"
+    echo ""
+    echo "Example:"
+    echo "  $0 admin@due.ren andreas@due.ren 1000 365"
+    echo ""
+    echo "Make sure you're logged in to the Ente CLI first:"
+    echo "  ente account add"
+    echo "  API endpoint: $ENTE_ENDPOINT"
+    exit 1
+fi
+
+# Run the update
+update_subscription "$1" "$2" "$3" "$4"