Add comprehensive API documentation to Cloudron setup instructions

- Added detailed API endpoint information in SETUP-INSTRUCTIONS.md
- Documented API usage with Ente CLI
- Enhanced routing configuration for auth/cast/accounts apps
- Updated to version 0.1.64

BUILD-INSTRUCTIONS.md

@@ -0,0 +1,110 @@
+# Ente Cloudron App Build and Installation Instructions
+
+This document provides detailed instructions for building and installing the Ente Cloudron app, an open-source, end-to-end encrypted photo storage and authentication solution.
+
+## Prerequisites
+
+- **Cloudron CLI**: Ensure the Cloudron CLI is installed and configured on your system. Refer to [Cloudron CLI Documentation](https://docs.cloudron.io/packaging/cli/) for setup instructions.
+- **Docker**: Required for local testing or custom builds if needed.
+- **Git**: To clone or manage the repository.
+- **Repository Access**: Ensure you have access to the Ente Cloudron repository at `andreasdueren/ente-cloudron`.
+- **Build Service Token**: A token for the Cloudron build service is required (provided in the command below).
+
+## Build Commands
+
+1. **Clone the Repository** (if not already done):
+   ```bash
+   git clone https://github.com/andreasdueren/ente-cloudron.git
+   cd ente-cloudron
+   ```
+
+2. **Build the App Using Cloudron Build Service**:
+   Use the provided build service and token to build the app. Replace `<version>` with the desired version tag (e.g., `0.1.0` or as per `CloudronManifest.json`).
+   ```bash
+   cloudron build --set-build-service builder.docker.due.ren --build-service-token e3265de06b1d0e7bb38400539012a8433a74c2c96a17955e --set-repository andreasdueren/ente-cloudron --tag 1.0.1
+   ```
+   **Note**: The build process should complete within a reasonable time. Monitor the output for any errors.
+
+## Installation Commands
+
+1. **Install the App on Cloudron**:
+   After a successful build, install the app on your Cloudron instance at the desired location (e.g., `ente.due.ren`).
+   ```bash
+   cloudron install --location ente.due.ren --image andreasdueren/ente-cloudron:1.0.1
+   ```
+   **Important**: Do not wait more than 30 seconds for feedback after running the install command. If there's an error, the process may hang, and you should terminate it to troubleshoot.
+   **Note**: Always uninstall and reinstall during development rather than updating an existing app to ensure a clean setup.
+
+## Testing Procedures
+
+1. **Verify Installation**:
+   - Access the app at `https://ente.due.ren` (or your configured domain).
+   - Ensure the Ente web interfaces (Photos, Accounts, Auth, Cast) load correctly.
+
+2. **Check S3 Configuration**:
+   - Confirm that S3 environment variables are set in Cloudron app settings under the 'Environment Variables' section.
+   - Variables to check: `APP_S3_ENABLED`, `APP_S3_ENDPOINT`, `APP_S3_ACCESS_KEY_ID`, `APP_S3_SECRET_ACCESS_KEY`, `APP_S3_BUCKET`.
+
+3. **Monitor Logs for Errors**:
+   - Use the Cloudron CLI to view logs:
+     ```bash
+     cloudron logs --app ente.due.ren -f
+     ```
+   - Alternatively, shell into the app for detailed log inspection:
+     ```bash
+     cloudron exec --app ente.due.ren
+     tail -f /app/data/logs/*
+     ```
+   - Look for S3 connection errors or other issues.
+
+## Deployment Steps
+
+1. **Post-Installation Configuration**:
+   - If S3 is not working, update the environment variables in Cloudron app settings and restart the app:
+     ```bash
+     cloudron restart --app ente.due.ren
+     ```
+
+2. **User Authentication**:
+   - Ente uses its own authentication system. Ensure user registration and login work as expected.
+   - If OIDC integration is desired in the future, it can be configured using Cloudron's OIDC variables (`CLOUDRON_OIDC_IDENTIFIER`, `CLOUDRON_OIDC_CLIENT_ID`, `CLOUDRON_OIDC_CLIENT_SECRET`).
+
+## Troubleshooting Common Issues
+
+- **S3 Configuration Errors**:
+  - **Symptom**: App falls back to local storage or logs show S3 connection failures.
+  - **Solution**: Verify S3 environment variables in Cloudron settings. Test connectivity manually using AWS CLI (`aws s3 ls s3://<bucket> --endpoint-url <endpoint>`).
+
+- **Build Failures**:
+  - **Symptom**: Build command errors out or hangs.
+  - **Solution**: Check network connectivity to the build service, ensure the token is correct, and review build logs for specific errors.
+
+- **Installation Hangs**:
+  - **Symptom**: Install command does not complete within 30 seconds.
+  - **Solution**: Terminate the command and check Cloudron logs for errors (`cloudron logs --app ente.due.ren`). Reinstall if necessary.
+
+- **App Not Starting**:
+  - **Symptom**: App shows as 'Stopped' or inaccessible after install.
+  - **Solution**: Check logs for startup errors (`cloudron logs --app ente.due.ren`). Ensure database connectivity and correct configuration.
+
+## Configuration Examples
+
+- **S3 Environment Variables** in Cloudron settings:
+  ```
+  APP_S3_ENABLED=true
+  APP_S3_ENDPOINT=s3.amazonaws.com
+  APP_S3_ACCESS_KEY_ID=your_access_key
+  APP_S3_SECRET_ACCESS_KEY=your_secret_key
+  APP_S3_BUCKET=your_bucket_name
+  ```
+
+## Additional Resources
+
+- **Cloudron Documentation**:
+  - [CLI](https://docs.cloudron.io/packaging/cli/)
+  - [Packaging Tutorial](https://docs.cloudron.io/packaging/tutorial/)
+  - [Manifest Reference](https://docs.cloudron.io/packaging/manifest/)
+  - [Addons Guide](https://docs.cloudron.io/packaging/addons/)
+  - [Cheat Sheet](https://docs.cloudron.io/packaging/cheat-sheet/)
+
+For further assistance, contact the Ente team at `contact@ente.io` or refer to the GitHub repository at [https://github.com/ente-io/ente](https://github.com/ente-io/ente).

Caddyfile.simple

@@ -0,0 +1,21 @@
+{
+    admin off
+    auto_https off
+}
+
+:3080 {
+    log {
+        output stdout
+        level DEBUG
+    }
+
+    # Simple health check that always works
+    handle /health {
+        respond "{\"status\": \"OK\"}" 200
+    }
+
+    # Catch-all for debugging
+    handle {
+        respond "Caddy is running on port 3080" 200
+    }
+}

CloudronManifest.json

@@ -7,13 +7,14 @@
   "contactEmail": "contact@ente.io",
   "tagline": "Open Source End-to-End Encrypted Photos & Authentication",
   "upstreamVersion": "1.0.0",
-  "version": "1.0.0",
-  "healthCheckPath": "/healthcheck",
+  "version": "0.1.64",
+  "healthCheckPath": "/ping",
   "httpPort": 3080,
   "memoryLimit": 1073741824,
   "addons": {
     "localstorage": {},
     "postgresql": {},
+    "email": {},
     "sendmail": {
       "supportsDisplayName": true
     }

Dockerfile

@@ -1,3 +1,19 @@
+# Build Museum server from source
+FROM golang:1.24-bookworm AS museum-builder
+
+WORKDIR /ente
+
+# Clone the repository for server building
+RUN apt-get update && apt-get install -y git libsodium-dev && \
+    git clone --depth=1 https://github.com/ente-io/ente.git . && \
+    apt-get clean && apt-get autoremove && \
+    rm -rf /var/cache/apt /var/lib/apt/lists
+
+# Build the Museum server
+WORKDIR /ente/server
+RUN go mod download && \
+    CGO_ENABLED=1 GOOS=linux go build -a -o museum ./cmd/museum
+
 FROM node:20-bookworm-slim as web-builder
 
 WORKDIR /ente
@@ -12,10 +28,10 @@ RUN apt-get update && apt-get install -y git && \
 RUN corepack enable
 
 # Set environment variables for web app build
-# Use "/api" as the endpoint which will be replaced at runtime with the full URL
+# Set the API endpoint to use current origin - this will work at runtime
 ENV NEXT_PUBLIC_ENTE_ENDPOINT="/api"
-# Add a note for clarity
-RUN echo "Building with NEXT_PUBLIC_ENTE_ENDPOINT=/api, will be replaced at runtime with full URL"
+# Use relative path so it works with any domain
+RUN echo "Building with NEXT_PUBLIC_ENTE_ENDPOINT=/api, will work with any domain via Caddy proxy"
 
 # Debugging the repository structure
 RUN find . -type d -maxdepth 3 | sort
@@ -78,10 +94,16 @@ RUN mkdir -p /build/web/photos /build/web/accounts /build/web/auth /build/web/ca
 
 FROM cloudron/base:5.0.0@sha256:04fd70dbd8ad6149c19de39e35718e024417c3e01dc9c6637eaf4a41ec4e596c
 
-# Install necessary packages
+# Install necessary packages and Caddy webserver
 RUN apt-get update && \
     apt-get install -y curl git nodejs npm libsodium23 libsodium-dev pkg-config postgresql-client && \
     npm install -g yarn serve && \
+    # Install Caddy for web server
+    apt-get install -y debian-keyring debian-archive-keyring apt-transport-https && \
+    curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg && \
+    curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | tee /etc/apt/sources.list.d/caddy-stable.list && \
+    apt-get update && \
+    apt-get install -y caddy && \
     apt-get clean && apt-get autoremove && \
     rm -rf /var/cache/apt /var/lib/apt/lists
 
@@ -94,7 +116,7 @@ RUN curl -L https://go.dev/dl/go1.24.1.linux-amd64.tar.gz -o go.tar.gz && \
     ln -sf /usr/local/go/bin/gofmt /usr/local/bin/gofmt
 
 # Set up directory structure
-RUN mkdir -p /app/code /app/data/config /app/web
+RUN mkdir -p /app/code /app/data/config /app/data/caddy /app/web
 
 WORKDIR /app/code
 
@@ -130,14 +152,25 @@ COPY --from=web-builder /build/web/accounts /app/web/accounts
 COPY --from=web-builder /build/web/auth /app/web/auth
 COPY --from=web-builder /build/web/cast /app/web/cast
 
+# Copy Museum server binary from builder stage to app directory (not data volume)
+RUN mkdir -p /app/museum-bin
+COPY --from=museum-builder /ente/server/museum /app/museum-bin/museum
+RUN chmod +x /app/museum-bin/museum
+
 # Copy configuration and startup scripts
 ADD start.sh /app/pkg/
 ADD config.template.yaml /app/pkg/
+ADD otp-email-monitor.js /app/pkg/
+ADD package.json /app/pkg/
+ADD admin-helper.sh /app/pkg/
+ADD admin-helper-direct.sh /app/pkg/
 
 # Set proper permissions
-RUN chmod +x /app/pkg/start.sh
+RUN chmod +x /app/pkg/start.sh /app/pkg/admin-helper.sh /app/pkg/admin-helper-direct.sh
 
-# Expose the API port
+# Expose the web port (Cloudron expects port 3080)
+EXPOSE 3080
+# Also expose API port
 EXPOSE 8080
 
 # Start the application

admin-helper-direct.sh

@@ -0,0 +1,133 @@
+#!/bin/bash
+# Direct Database Admin Helper for Ente Cloudron
+# This script directly updates the database for admin operations
+
+# Function to update user subscription directly in database
+update_subscription() {
+    local user_email="$1"
+    local storage_gb="$2"
+    local valid_days="$3"
+    
+    if [ -z "$user_email" ] || [ -z "$storage_gb" ] || [ -z "$valid_days" ]; then
+        echo "Usage: $0 update-subscription <user-email> <storage-gb> <valid-days>"
+        echo "Example: $0 update-subscription user@example.com 100 365"
+        return 1
+    fi
+    
+    echo "Updating subscription for: $user_email"
+    echo "Storage: ${storage_gb}GB"
+    echo "Valid for: ${valid_days} days"
+    
+    # Convert GB to bytes (1 GB = 1073741824 bytes)
+    local storage_bytes=$((storage_gb * 1073741824))
+    
+    # Calculate expiry timestamp (current time + valid_days)
+    local current_timestamp=$(date +%s)
+    local expiry_timestamp=$((current_timestamp + (valid_days * 86400)))
+    # Convert to microseconds for the database
+    local expiry_microseconds="${expiry_timestamp}000000"
+    
+    # Update the database directly
+    PGPASSWORD="$CLOUDRON_POSTGRESQL_PASSWORD" psql \
+        -h "$CLOUDRON_POSTGRESQL_HOST" \
+        -p "$CLOUDRON_POSTGRESQL_PORT" \
+        -U "$CLOUDRON_POSTGRESQL_USERNAME" \
+        -d "$CLOUDRON_POSTGRESQL_DATABASE" << EOF
+-- Update user's storage and subscription
+UPDATE users 
+SET 
+    storage_bonus = $storage_bytes,
+    subscription_expiry = $expiry_microseconds
+WHERE email = '$user_email';
+
+-- Show the updated values
+SELECT 
+    email,
+    storage_bonus / 1073741824.0 as storage_gb,
+    to_timestamp(subscription_expiry / 1000000) as subscription_expires
+FROM users 
+WHERE email = '$user_email';
+EOF
+    
+    if [ $? -eq 0 ]; then
+        echo "✓ Subscription updated successfully"
+    else
+        echo "✗ Failed to update subscription"
+        return 1
+    fi
+}
+
+# Function to get user details
+get_user_details() {
+    local user_email="$1"
+    
+    if [ -z "$user_email" ]; then
+        echo "Usage: $0 get-user <user-email>"
+        return 1
+    fi
+    
+    PGPASSWORD="$CLOUDRON_POSTGRESQL_PASSWORD" psql \
+        -h "$CLOUDRON_POSTGRESQL_HOST" \
+        -p "$CLOUDRON_POSTGRESQL_PORT" \
+        -U "$CLOUDRON_POSTGRESQL_USERNAME" \
+        -d "$CLOUDRON_POSTGRESQL_DATABASE" << EOF
+SELECT 
+    email,
+    storage_bonus / 1073741824.0 as storage_gb,
+    storage_consumed / 1073741824.0 as used_gb,
+    to_timestamp(subscription_expiry / 1000000) as subscription_expires,
+    CASE 
+        WHEN subscription_expiry > (EXTRACT(EPOCH FROM NOW()) * 1000000) THEN 'Active'
+        ELSE 'Expired'
+    END as status
+FROM users 
+WHERE email = '$user_email';
+EOF
+}
+
+# Function to list all users
+list_users() {
+    PGPASSWORD="$CLOUDRON_POSTGRESQL_PASSWORD" psql \
+        -h "$CLOUDRON_POSTGRESQL_HOST" \
+        -p "$CLOUDRON_POSTGRESQL_PORT" \
+        -U "$CLOUDRON_POSTGRESQL_USERNAME" \
+        -d "$CLOUDRON_POSTGRESQL_DATABASE" << EOF
+SELECT 
+    email,
+    storage_bonus / 1073741824.0 as storage_gb,
+    storage_consumed / 1073741824.0 as used_gb,
+    to_timestamp(subscription_expiry / 1000000) as expires,
+    CASE 
+        WHEN subscription_expiry > (EXTRACT(EPOCH FROM NOW()) * 1000000) THEN 'Active'
+        ELSE 'Expired'
+    END as status
+FROM users 
+ORDER BY email;
+EOF
+}
+
+# Main command handler
+case "$1" in
+    "update-subscription")
+        update_subscription "$2" "$3" "$4"
+        ;;
+    "get-user")
+        get_user_details "$2"
+        ;;
+    "list-users")
+        list_users
+        ;;
+    *)
+        echo "Ente Direct Admin Helper"
+        echo ""
+        echo "Usage:"
+        echo "  $0 update-subscription <user-email> <storage-gb> <valid-days>"
+        echo "  $0 get-user <user-email>"
+        echo "  $0 list-users"
+        echo ""
+        echo "Examples:"
+        echo "  $0 update-subscription user@example.com 100 365"
+        echo "  $0 get-user user@example.com"
+        echo "  $0 list-users"
+        ;;
+esac

admin-helper.sh

@@ -0,0 +1,93 @@
+#!/bin/bash
+# Ente Admin Helper Script for Cloudron
+# This script simplifies admin operations in the Cloudron terminal
+
+MUSEUM_BIN="/app/data/ente/server/museum"
+
+# Check if museum binary exists
+if [ ! -f "$MUSEUM_BIN" ]; then
+    echo "Error: Museum binary not found at $MUSEUM_BIN"
+    exit 1
+fi
+
+# Function to update user subscription
+update_subscription() {
+    local user_email="$1"
+    local storage_gb="$2"
+    local valid_days="$3"
+    
+    if [ -z "$user_email" ] || [ -z "$storage_gb" ] || [ -z "$valid_days" ]; then
+        echo "Usage: $0 update-subscription <user-email> <storage-gb> <valid-days>"
+        echo "Example: $0 update-subscription user@example.com 100 365"
+        return 1
+    fi
+    
+    echo "Updating subscription for: $user_email"
+    echo "Storage: ${storage_gb}GB"
+    echo "Valid for: ${valid_days} days"
+    
+    cd /app/data/ente/server
+    
+    # Use environment variables for database connection
+    export DB_HOST="$CLOUDRON_POSTGRESQL_HOST"
+    export DB_PORT="$CLOUDRON_POSTGRESQL_PORT"
+    export DB_NAME="$CLOUDRON_POSTGRESQL_DATABASE"
+    export DB_USERNAME="$CLOUDRON_POSTGRESQL_USERNAME"
+    export DB_PASSWORD="$CLOUDRON_POSTGRESQL_PASSWORD"
+    
+    # Museum admin commands need specific syntax
+    "$MUSEUM_BIN" admin update-subscription "$user_email" "$storage_gb" "$valid_days"
+}
+
+# Function to get user details
+get_user_details() {
+    local user_email="$1"
+    
+    if [ -z "$user_email" ]; then
+        echo "Usage: $0 get-user <user-email>"
+        return 1
+    fi
+    
+    cd /app/data/ente/server
+    
+    "$MUSEUM_BIN" admin get-user-details --user "$user_email"
+}
+
+# Function to list all users
+list_users() {
+    cd /app/data/ente/server
+    
+    # Connect to PostgreSQL and list users
+    PGPASSWORD="$CLOUDRON_POSTGRESQL_PASSWORD" psql \
+        -h "$CLOUDRON_POSTGRESQL_HOST" \
+        -p "$CLOUDRON_POSTGRESQL_PORT" \
+        -U "$CLOUDRON_POSTGRESQL_USERNAME" \
+        -d "$CLOUDRON_POSTGRESQL_DATABASE" \
+        -c "SELECT email, storage_bonus, subscription_expiry FROM users ORDER BY email;"
+}
+
+# Main command handler
+case "$1" in
+    "update-subscription")
+        update_subscription "$2" "$3" "$4"
+        ;;
+    "get-user")
+        get_user_details "$2"
+        ;;
+    "list-users")
+        list_users
+        ;;
+    *)
+        echo "Ente Admin Helper"
+        echo ""
+        echo "Usage:"
+        echo "  $0 update-subscription <user-email> <storage-gb> <valid-days>"
+        echo "  $0 get-user <user-email>"
+        echo "  $0 list-users"
+        echo ""
+        echo "Examples:"
+        echo "  $0 update-subscription user@example.com 100 365"
+        echo "  $0 get-user user@example.com"
+        echo "  $0 list-users"
+        ;;
+esac

debug-headers.sh

@@ -0,0 +1,35 @@
+#!/bin/bash
+
+echo "==> Debugging Caddy MIME type headers"
+echo "==> Testing various file types..."
+
+BASE_URL="${1:-https://ente.due.ren}"
+
+echo
+echo "Testing HTML files:"
+curl -I "$BASE_URL/" 2>/dev/null | grep -i content-type || echo "No Content-Type header found"
+curl -I "$BASE_URL/index.html" 2>/dev/null | grep -i content-type || echo "No Content-Type header found"
+
+echo
+echo "Testing JavaScript files:"
+curl -I "$BASE_URL/config.js" 2>/dev/null | grep -i content-type || echo "No Content-Type header found"
+
+echo
+echo "Testing CSS files (if any):"
+curl -I "$BASE_URL/styles.css" 2>/dev/null | grep -i content-type || echo "File not found or no Content-Type header"
+
+echo
+echo "Testing JSON files (if any):"
+curl -I "$BASE_URL/manifest.json" 2>/dev/null | grep -i content-type || echo "File not found or no Content-Type header"
+
+echo
+echo "==> Full response headers for main page:"
+curl -I "$BASE_URL/" 2>/dev/null || echo "Failed to connect to $BASE_URL"
+
+echo
+echo "==> To test from inside a container:"
+echo "docker exec -it <container-name> curl -I http://localhost:3080/"
+
+echo
+echo "==> To view Caddy logs:"
+echo "docker exec -it <container-name> tail -f /app/data/logs/caddy.log"

debug-start.sh

@@ -0,0 +1,32 @@
+#!/bin/bash
+
+# Add this debugging section to your start.sh after line 350
+
+# Start Caddy with more verbose logging
+echo "==> Starting Caddy web server with debug logging"
+echo "==> Validating Caddyfile first..."
+caddy validate --config /app/data/Caddyfile --adapter caddyfile || {
+    echo "==> ERROR: Caddyfile validation failed!"
+    cat /app/data/Caddyfile
+    exit 1
+}
+
+echo "==> Starting Caddy..."
+# Run Caddy in foreground first to see errors
+timeout 10 caddy run --config /app/data/Caddyfile --adapter caddyfile 2>&1 | tee /app/data/logs/caddy-debug.log || {
+    echo "==> ERROR: Caddy failed to start"
+    echo "==> Last 50 lines of Caddy debug log:"
+    tail -50 /app/data/logs/caddy-debug.log
+}
+
+# Check if port is actually listening
+echo "==> Checking if port 3080 is listening..."
+netstat -tlnp | grep 3080 || lsof -i :3080 || {
+    echo "==> ERROR: Nothing listening on port 3080"
+}
+
+# Test the health endpoint
+echo "==> Testing health endpoint..."
+curl -v http://localhost:3080/health || {
+    echo "==> ERROR: Health check failed"
+}

ente-cli-config.md

@@ -0,0 +1,64 @@
+# Ente CLI Configuration for Custom Server
+
+The Ente CLI expects configuration in `~/.ente/config.yaml`. Here's how to set it up:
+
+## Method 1: Direct Configuration
+
+1. Create the config file:
+```bash
+mkdir -p ~/.ente
+cat > ~/.ente/config.yaml << EOF
+api:
+  url: https://ente.due.ren
+EOF
+```
+
+2. Add your account interactively:
+```bash
+ente account add
+# It will ask for:
+# - Export directory: /tmp/ente-export (or any directory)
+# - Email: your-admin@email.com
+# - Password: your-password
+```
+
+## Method 2: Using the Admin Commands Directly
+
+If the interactive setup is problematic, you can use the admin commands with explicit parameters:
+
+```bash
+# Set the API endpoint
+export ENTE_API_URL="https://ente.due.ren"
+
+# Or pass it directly in the command
+ente admin update-subscription \
+  --api-url https://ente.due.ren \
+  --admin-user admin@due.ren \
+  --user user@example.com \
+  --storage 1000 \
+  --valid-for 365
+```
+
+## Method 3: Direct Database Update (Fallback)
+
+Since the CLI setup seems problematic, you can update the database directly in the Cloudron terminal:
+
+```bash
+# In Cloudron terminal
+PGPASSWORD="$CLOUDRON_POSTGRESQL_PASSWORD" psql \
+  -h "$CLOUDRON_POSTGRESQL_HOST" \
+  -U "$CLOUDRON_POSTGRESQL_USERNAME" \
+  -d "$CLOUDRON_POSTGRESQL_DATABASE" << EOF
+-- Update user to 1TB for 1 year
+UPDATE users 
+SET storage_bonus = 1073741824000,  -- 1000 GB in bytes
+    subscription_expiry = EXTRACT(EPOCH FROM NOW() + INTERVAL '365 days') * 1000000
+WHERE email = 'andreas@due.ren';
+
+-- Show the result
+SELECT email, 
+       storage_bonus / 1073741824.0 as storage_gb,
+       to_timestamp(subscription_expiry / 1000000) as expires
+FROM users WHERE email = 'andreas@due.ren';
+EOF
+```

otp-email-monitor.js

@@ -0,0 +1,389 @@
+#!/usr/bin/env node
+
+/**
+ * Ente OTP Email Monitor
+ * 
+ * Monitors Museum server logs for OTP generation events and sends
+ * verification emails using Cloudron's email addon.
+ */
+
+const fs = require('fs');
+const path = require('path');
+const { spawn } = require('child_process');
+const nodemailer = require('nodemailer');
+
+// Configuration
+const CONFIG = {
+    LOG_FILE: '/app/data/logs/museum.log',
+    EMAIL_TEMPLATES_DIR: '/app/data/ente/server/mail-templates',
+    FROM_EMAIL: `noreply@${process.env.CLOUDRON_EMAIL_DOMAIN || 'localhost'}`,
+    FROM_NAME: 'Ente Photos',
+    SMTP: {
+        host: process.env.CLOUDRON_EMAIL_SMTP_SERVER,
+        port: parseInt(process.env.CLOUDRON_EMAIL_SMTP_PORT) || 587,
+        secure: false, // STARTTLS disabled on this port
+        auth: false // Internal mail server
+    }
+};
+
+// Logging utility
+class Logger {
+    static log(level, message, data = null) {
+        const timestamp = new Date().toISOString();
+        const logMessage = `[${timestamp}] [OTP-EMAIL-${level}] ${message}`;
+        console.log(logMessage);
+        
+        if (data) {
+            console.log(JSON.stringify(data, null, 2));
+        }
+        
+        // Also write to file
+        try {
+            fs.appendFileSync('/app/data/logs/otp-email.log', logMessage + '\n');
+        } catch (err) {
+            console.error('Failed to write to log file:', err.message);
+        }
+    }
+    
+    static info(message, data) { this.log('INFO', message, data); }
+    static warn(message, data) { this.log('WARN', message, data); }
+    static error(message, data) { this.log('ERROR', message, data); }
+}
+
+// Email template handler
+class EmailTemplate {
+    constructor(templateDir) {
+        this.templateDir = templateDir;
+        this.templates = new Map();
+        this.loadTemplates();
+    }
+    
+    loadTemplates() {
+        try {
+            const ottTemplate = fs.readFileSync(path.join(this.templateDir, 'ott.html'), 'utf8');
+            const changeEmailTemplate = fs.readFileSync(path.join(this.templateDir, 'ott_change_email.html'), 'utf8');
+            
+            this.templates.set('ott', ottTemplate);
+            this.templates.set('ott_change_email', changeEmailTemplate);
+            
+            Logger.info('Email templates loaded successfully');
+        } catch (err) {
+            Logger.error('Failed to load email templates:', err.message);
+            throw err;
+        }
+    }
+    
+    render(templateName, variables) {
+        const template = this.templates.get(templateName);
+        if (!template) {
+            throw new Error(`Template ${templateName} not found`);
+        }
+        
+        let html = template;
+        
+        // Replace template variables
+        for (const [key, value] of Object.entries(variables)) {
+            const placeholder = `{{.${key}}}`;
+            html = html.replace(new RegExp(placeholder, 'g'), value);
+        }
+        
+        return html;
+    }
+}
+
+// Email sender using Cloudron email addon
+class EmailSender {
+    constructor(config) {
+        this.config = config;
+        this.transporter = null;
+        this.initializeTransporter();
+    }
+    
+    initializeTransporter() {
+        try {
+            this.transporter = nodemailer.createTransport({
+                host: this.config.SMTP.host,
+                port: this.config.SMTP.port,
+                secure: this.config.SMTP.secure,
+                // No auth needed for internal Cloudron mail server
+                tls: {
+                    rejectUnauthorized: false // Accept self-signed certificates
+                }
+            });
+            
+            Logger.info('Email transporter initialized', {
+                host: this.config.SMTP.host,
+                port: this.config.SMTP.port
+            });
+        } catch (err) {
+            Logger.error('Failed to initialize email transporter:', err.message);
+            throw err;
+        }
+    }
+    
+    async sendEmail(to, subject, html) {
+        try {
+            const mailOptions = {
+                from: `${this.config.FROM_NAME} <${this.config.FROM_EMAIL}>`,
+                to: to,
+                subject: subject,
+                html: html
+            };
+            
+            const result = await this.transporter.sendMail(mailOptions);
+            Logger.info('Email sent successfully', {
+                to: to,
+                subject: subject,
+                messageId: result.messageId
+            });
+            
+            return result;
+        } catch (err) {
+            Logger.error('Failed to send email:', {
+                error: err.message,
+                to: to,
+                subject: subject
+            });
+            throw err;
+        }
+    }
+}
+
+// Log monitor for OTP events
+class LogMonitor {
+    constructor(logFile, emailSender, emailTemplate) {
+        this.logFile = logFile;
+        this.emailSender = emailSender;
+        this.emailTemplate = emailTemplate;
+        this.tail = null;
+        this.processedOTPs = new Set(); // Prevent duplicate sends
+    }
+    
+    start() {
+        Logger.info('Starting log monitor', { logFile: this.logFile });
+        
+        // Use tail -F to follow log file
+        this.tail = spawn('tail', ['-F', this.logFile]);
+        
+        this.tail.stdout.on('data', (data) => {
+            const lines = data.toString().split('\n');
+            for (const line of lines) {
+                if (line.trim()) {
+                    this.processLogLine(line);
+                }
+            }
+        });
+        
+        this.tail.stderr.on('data', (data) => {
+            Logger.warn('Tail stderr:', data.toString());
+        });
+        
+        this.tail.on('close', (code) => {
+            Logger.warn('Tail process closed', { code });
+            // Restart after 5 seconds
+            setTimeout(() => this.start(), 5000);
+        });
+        
+        this.tail.on('error', (err) => {
+            Logger.error('Tail process error:', err.message);
+            setTimeout(() => this.start(), 5000);
+        });
+    }
+    
+    stop() {
+        if (this.tail) {
+            this.tail.kill();
+            this.tail = null;
+            Logger.info('Log monitor stopped');
+        }
+    }
+    
+    processLogLine(line) {
+        try {
+            // Look for OTP-related log entries
+            // Museum server logs OTP generation in various formats
+            const patterns = [
+                // Pattern 1: Museum skipping email - MOST IMPORTANT (matches "Skipping sending email to andreas@due.ren: Verification code: 192305")
+                /Skipping sending email to\s+([^\s:]+):\s*Verification code:\s*(\d{6})/i,
+                // Pattern 2: Direct OTP generation logs
+                /sendOTT.*email[:\s]+([^\s]+).*code[:\s]+(\d{6})/i,
+                // Pattern 3: User registration/login with OTP
+                /generateOTT.*user[:\s]+([^\s]+).*verification.*code[:\s]+(\d{6})/i,
+                // Pattern 4: Email change OTP
+                /changeEmail.*email[:\s]+([^\s]+).*otp[:\s]+(\d{6})/i,
+                // Pattern 5: Generic OTP patterns in logs
+                /ott.*([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}).*(\d{6})/i
+            ];
+            
+            for (const pattern of patterns) {
+                const match = line.match(pattern);
+                if (match) {
+                    const email = match[1];
+                    const otpCode = match[2];
+                    
+                    // Create unique identifier to prevent duplicates
+                    const otpId = `${email}:${otpCode}:${Date.now().toString().slice(-6)}`;
+                    
+                    if (!this.processedOTPs.has(otpId)) {
+                        this.processedOTPs.add(otpId);
+                        this.sendOTPEmail(email, otpCode, line);
+                        
+                        // Clean up old OTPs (keep last 100)
+                        if (this.processedOTPs.size > 100) {
+                            const oldOTPs = Array.from(this.processedOTPs).slice(0, 50);
+                            oldOTPs.forEach(otp => this.processedOTPs.delete(otp));
+                        }
+                    }
+                    break;
+                }
+            }
+        } catch (err) {
+            Logger.error('Error processing log line:', {
+                error: err.message,
+                line: line.substring(0, 100)
+            });
+        }
+    }
+    
+    async sendOTPEmail(email, otpCode, logLine) {
+        try {
+            Logger.info('Processing OTP email request', {
+                email: email,
+                otpCode: otpCode.substring(0, 2) + '****', // Partial OTP for logging
+                source: logLine.substring(0, 100)
+            });
+            
+            // Determine template type based on context
+            let templateName = 'ott';
+            let subject = 'Ente - Verification Code';
+            
+            if (logLine.toLowerCase().includes('change') || logLine.toLowerCase().includes('email')) {
+                templateName = 'ott_change_email';
+                subject = 'Ente - Email Change Verification';
+            }
+            
+            // Render email template
+            const html = this.emailTemplate.render(templateName, {
+                VerificationCode: otpCode
+            });
+            
+            // Send email
+            await this.emailSender.sendEmail(email, subject, html);
+            
+            Logger.info('OTP email sent successfully', {
+                email: email,
+                template: templateName
+            });
+            
+        } catch (err) {
+            Logger.error('Failed to send OTP email:', {
+                error: err.message,
+                email: email,
+                otpCode: otpCode.substring(0, 2) + '****'
+            });
+        }
+    }
+}
+
+// Health check endpoint
+class HealthServer {
+    constructor(port = 8081) {
+        this.port = port;
+        this.server = null;
+    }
+    
+    start() {
+        const http = require('http');
+        
+        this.server = http.createServer((req, res) => {
+            if (req.url === '/health') {
+                res.writeHead(200, { 'Content-Type': 'application/json' });
+                res.end(JSON.stringify({
+                    status: 'healthy',
+                    service: 'ente-otp-email-monitor',
+                    timestamp: new Date().toISOString(),
+                    processedOTPs: monitor ? monitor.processedOTPs.size : 0
+                }));
+            } else {
+                res.writeHead(404);
+                res.end('Not Found');
+            }
+        });
+        
+        this.server.listen(this.port, () => {
+            Logger.info(`Health server listening on port ${this.port}`);
+        });
+    }
+    
+    stop() {
+        if (this.server) {
+            this.server.close();
+            Logger.info('Health server stopped');
+        }
+    }
+}
+
+// Main application
+let monitor = null;
+let healthServer = null;
+
+async function main() {
+    try {
+        Logger.info('Starting Ente OTP Email Monitor');
+        
+        // Validate environment
+        if (!process.env.CLOUDRON_EMAIL_SMTP_SERVER) {
+            throw new Error('CLOUDRON_EMAIL_SMTP_SERVER not found. Email addon may not be configured.');
+        }
+        
+        // Initialize components
+        const emailTemplate = new EmailTemplate(CONFIG.EMAIL_TEMPLATES_DIR);
+        const emailSender = new EmailSender(CONFIG);
+        
+        // Test email connectivity
+        Logger.info('Testing email connectivity...');
+        await emailSender.transporter.verify();
+        Logger.info('Email connectivity verified');
+        
+        // Start log monitor
+        monitor = new LogMonitor(CONFIG.LOG_FILE, emailSender, emailTemplate);
+        monitor.start();
+        
+        // Start health server
+        healthServer = new HealthServer();
+        healthServer.start();
+        
+        Logger.info('Ente OTP Email Monitor started successfully');
+        
+        // Handle graceful shutdown
+        process.on('SIGINT', () => {
+            Logger.info('Received SIGINT, shutting down gracefully...');
+            if (monitor) monitor.stop();
+            if (healthServer) healthServer.stop();
+            process.exit(0);
+        });
+        
+        process.on('SIGTERM', () => {
+            Logger.info('Received SIGTERM, shutting down gracefully...');
+            if (monitor) monitor.stop();
+            if (healthServer) healthServer.stop();
+            process.exit(0);
+        });
+        
+    } catch (err) {
+        Logger.error('Failed to start OTP Email Monitor:', err.message);
+        process.exit(1);
+    }
+}
+
+// Start the application
+if (require.main === module) {
+    main();
+}
+
+module.exports = {
+    LogMonitor,
+    EmailSender,
+    EmailTemplate,
+    Logger
+};

package.json

@@ -0,0 +1,22 @@
+{
+  "name": "ente-otp-email-monitor",
+  "version": "1.0.0",
+  "description": "OTP email monitoring service for Ente Cloudron app",
+  "main": "otp-email-monitor.js",
+  "scripts": {
+    "start": "node otp-email-monitor.js",
+    "test": "echo \"Error: no test specified\" && exit 1"
+  },
+  "dependencies": {
+    "nodemailer": "^6.9.0"
+  },
+  "keywords": [
+    "ente",
+    "otp",
+    "email",
+    "cloudron",
+    "monitoring"
+  ],
+  "author": "Ente Cloudron Integration",
+  "license": "Apache-2.0"
+}

setup-ente-cli.sh

@@ -0,0 +1,20 @@
+#!/bin/bash
+# Setup Ente CLI for custom server
+
+echo "Setting up Ente CLI for custom server..."
+
+# Create config directory
+mkdir -p ~/.ente
+
+# Create the CLI config with custom endpoint
+cat > ~/.ente/config.yaml << EOF
+host: https://ente.due.ren
+EOF
+
+echo "Configuration created at ~/.ente/config.yaml"
+echo ""
+echo "Now you can add your account:"
+echo "  ente account add"
+echo ""
+echo "Then use admin commands:"
+echo "  ente admin update-subscription --admin-user admin@due.ren --user user@example.com --storage 1000 --valid-for 365"

start-debug.sh

@@ -0,0 +1,150 @@
+#!/bin/bash
+
+# Better signal handling - forward signals to child processes
+trap 'kill -TERM $SERVER_PID; kill -TERM $CADDY_PID; exit' TERM INT
+
+set -eu
+
+echo "==> Starting Ente Cloudron app (DEBUG MODE)..."
+
+# Create necessary directories
+mkdir -p /app/data/config /app/data/logs /app/data/caddy
+
+# Check if web directories exist
+echo "==> Checking web app directories:"
+for app in photos accounts auth cast; do
+    if [ -d "/app/web/$app" ]; then
+        echo "==> Found: /app/web/$app"
+        ls -la "/app/web/$app" | head -5
+    else
+        echo "==> WARNING: Missing /app/web/$app - creating placeholder"
+        mkdir -p "/app/web/$app"
+        echo "<html><body><h1>$app app placeholder</h1></body></html>" > "/app/web/$app/index.html"
+    fi
+done
+
+# Create a simple test Caddyfile first
+echo "==> Creating simple test Caddyfile"
+cat > /app/data/Caddyfile <<'EOT'
+{
+    admin off
+    auto_https off
+}
+
+:3080 {
+    log {
+        output stdout
+        format console
+        level DEBUG
+    }
+
+    # Health check endpoint
+    handle /health {
+        header Content-Type "application/json"
+        respond "{\"status\": \"OK\", \"timestamp\": \"{{now | date \"2006-01-02T15:04:05Z07:00\"}}\"}" 200
+    }
+
+    # Test endpoint
+    handle /test {
+        respond "Caddy is working on port 3080!" 200
+    }
+
+    # API proxy to Museum server
+    handle /api/* {
+        uri strip_prefix /api
+        reverse_proxy localhost:8080 {
+            transport http {
+                read_timeout 60s
+                write_timeout 60s
+            }
+            # Add error handling
+            handle_errors {
+                respond "{\"error\": \"Museum server not available\"}" 503
+            }
+        }
+    }
+
+    # Serve web apps with fallback
+    handle {
+        root * /app/web/photos
+        try_files {path} {path}/ /index.html
+        file_server {
+            browse
+        }
+    }
+}
+EOT
+
+# Start a simple Museum mock server for testing
+echo "==> Starting mock Museum server on port 8080"
+cat > /tmp/museum-mock.js <<'EOF'
+const http = require('http');
+const server = http.createServer((req, res) => {
+    console.log(`Museum mock: ${req.method} ${req.url}`);
+    res.writeHead(200, { 'Content-Type': 'application/json' });
+    res.end(JSON.stringify({ status: 'ok', path: req.url, timestamp: new Date().toISOString() }));
+});
+server.listen(8080, '127.0.0.1', () => {
+    console.log('Museum mock server running on http://127.0.0.1:8080');
+});
+EOF
+node /tmp/museum-mock.js > /app/data/logs/museum-mock.log 2>&1 &
+SERVER_PID=$!
+echo "==> Mock Museum server started (PID: $SERVER_PID)"
+
+# Wait for Museum mock to be ready
+sleep 2
+
+# Test Museum mock
+echo "==> Testing Museum mock server..."
+curl -s http://localhost:8080/test || echo "WARNING: Museum mock not responding"
+
+# Validate Caddyfile
+echo "==> Validating Caddyfile..."
+caddy validate --config /app/data/Caddyfile --adapter caddyfile || {
+    echo "==> ERROR: Caddyfile validation failed!"
+    exit 1
+}
+
+# Start Caddy with explicit environment
+echo "==> Starting Caddy web server..."
+CADDY_FORMAT=console caddy run --config /app/data/Caddyfile --adapter caddyfile 2>&1 | tee /app/data/logs/caddy-combined.log &
+CADDY_PID=$!
+echo "==> Caddy started (PID: $CADDY_PID)"
+
+# Wait for Caddy to start
+echo "==> Waiting for Caddy to start..."
+for i in {1..30}; do
+    if curl -s http://localhost:3080/health > /dev/null; then
+        echo "==> Caddy is responding!"
+        break
+    fi
+    echo -n "."
+    sleep 1
+done
+echo
+
+# Check process status
+echo "==> Process status:"
+ps aux | grep -E "(caddy|node)" | grep -v grep || echo "No processes found"
+
+# Check port status
+echo "==> Port status:"
+netstat -tlnp 2>/dev/null | grep -E "(3080|8080)" || lsof -i :3080 -i :8080 2>/dev/null || echo "Cannot check port status"
+
+# Test endpoints
+echo "==> Testing endpoints:"
+echo "Health check:"
+curl -s http://localhost:3080/health | jq . || echo "Failed"
+echo -e "\nTest endpoint:"
+curl -s http://localhost:3080/test || echo "Failed"
+echo -e "\nAPI proxy:"
+curl -s http://localhost:3080/api/status | jq . || echo "Failed"
+
+echo "==> Startup complete. Services:"
+echo "   - Caddy PID: $CADDY_PID"
+echo "   - Museum Mock PID: $SERVER_PID"
+echo "==> Logs: /app/data/logs/"
+
+# Keep running
+wait $SERVER_PID $CADDY_PID

update-storage.sh

@@ -0,0 +1,57 @@
+#!/bin/bash
+# Script to update Ente user storage using the Ente CLI
+# Run this from your local machine (not inside Cloudron)
+
+# Check if ente CLI is installed
+if ! command -v ente &> /dev/null; then
+    echo "Ente CLI is not installed. Please install it first:"
+    echo ""
+    echo "For macOS:"
+    echo "  brew tap ente-io/ente"
+    echo "  brew install ente-cli"
+    echo ""
+    echo "For other systems, download from:"
+    echo "  https://github.com/ente-io/ente/releases"
+    exit 1
+fi
+
+# Your Ente instance
+ENTE_ENDPOINT="https://ente.due.ren"
+
+# Function to update subscription
+update_subscription() {
+    local admin_email="$1"
+    local user_email="$2"
+    local storage_gb="$3"
+    local valid_days="$4"
+    
+    echo "Updating subscription for: $user_email"
+    echo "Storage: ${storage_gb}GB"
+    echo "Valid for: ${valid_days} days"
+    echo "Using admin account: $admin_email"
+    echo ""
+    
+    # Run the ente CLI command
+    ente admin update-subscription \
+        --host "$ENTE_ENDPOINT" \
+        --admin-user "$admin_email" \
+        --user "$user_email" \
+        --storage "$storage_gb" \
+        --valid-for "$valid_days"
+}
+
+# Check arguments
+if [ $# -lt 4 ]; then
+    echo "Usage: $0 <admin-email> <user-email> <storage-gb> <valid-days>"
+    echo ""
+    echo "Example:"
+    echo "  $0 admin@due.ren andreas@due.ren 1000 365"
+    echo ""
+    echo "Make sure you're logged in to the Ente CLI first:"
+    echo "  ente account add"
+    echo "  API endpoint: $ENTE_ENDPOINT"
+    exit 1
+fi
+
+# Run the update
+update_subscription "$1" "$2" "$3" "$4"