Bump version for 1.33.0

CHANGELOG

@@ -274,3 +274,134 @@
 * [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.23.1)
 * There is a denial of service attack (CVE-2020-26257) against the federation APIs in which future events will not be correctly sent to other servers over federation. This affects all servers that participate in open federation. (Fixed in #8776).
 
+[1.15.0]
+* Update Synapse to 1.24.0
+* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.24.0)
+* Add a maximum version for pysaml2 on Python 3.5
+
+[1.16.0]
+* Update Synapse to 1.25.0
+* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.25.0)
+* Add an admin API that lets server admins get power in rooms in which local users have power. (#8756)
+* Add optional HTTP authentication to replication endpoints. (#8853)
+* Improve the error messages printed as a result of configuration problems for extension modules. (#8874)
+* Add the number of local devices to Room Details Admin API. Contributed by @dklimpel. (#8886)
+* Add X-Robots-Tag header to stop web crawlers from indexing media. Contributed by Aaron Raimist. (#8887)
+* Spam-checkers may now define their methods as async. (#8890)
+* Add support for allowing users to pick their own user ID during a single-sign-on login. (#8897, #8900, #8911, #8938, #8941, #8942, #8951)
+* Add an email.invite_client_location configuration option to send a web client location to the invite endpoint on the identity server which allows customisation of the email template. (#8930)
+* The search term in the list room and list user Admin APIs is now treated as case-insensitive. (#8931)
+* Apply an IP range blacklist to push and key revocation requests. (#8821, #8870, #8954)
+* Add an option to allow re-use of user-interactive authentication sessions for a period of time. (#8970)
+* Allow running the redact endpoint on workers. (#8994)
+
+[1.17.0]
+* Update Synapse to 1.26.0
+* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.26.0)
+* During user-interactive authentication via single-sign-on, give a better error if the user uses the wrong account on the SSO IdP. (#9091)
+* Give the public_baseurl a default value, if it is not explicitly set in the configuration file. (#9159)
+* Improve performance when calculating ignored users in large rooms. (#9024)
+* Implement MSC2176 in an experimental room version. (#8984)
+* Add an admin API for protecting local media from quarantine. (#9086)
+* Remove a user's avatar URL and display name when deactivated with the Admin API. (#8932)
+
+[1.18.0]
+* Update Synapse to 1.27.0
+* Use base image v3
+* Update python to 3.8
+* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.27.0)
+* Add an admin API for getting and deleting forward extremities for a room. (#9062)
+* Add an admin API for retrieving the current room state of a room. (#9168)
+* Add an admin API endpoint for shadow-banning users. (#9209)
+
+[1.19.0]
+* Update Synapse to 1.28.0
+* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.28.0)
+* New admin API to get the context of an event: /_synapse/admin/rooms/{roomId}/context/{eventId}. (#9150)
+* Further improvements to the user experience of registration via single sign-on. (#9300, #9301)
+* Add hook to spam checker modules that allow checking file uploads and remote downloads. (#9311)
+* Add support for receiving OpenID Connect authentication responses via form POSTs rather than GETs. (#9376)
+* Add the shadow-banning status to the admin API for user info. (#9400)
+
+[1.20.0]
+* Update Synapse to 1.29.0
+* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.29.0)
+* Add rate limiters to cross-user key sharing requests. (#8957)
+* Add order_by to the admin API GET /_synapse/admin/v1/users/<user_id>/media. Contributed by @dklimpel. (#8978)
+* Add some configuration settings to make users' profile data more private. (#9203)
+* The no_proxy and NO_PROXY environment variables are now respected in proxied HTTP clients with the lowercase form taking precedence if both are present. Additionally, the lowercase https_proxy environment variable is now respected in proxied HTTP clients on top of existing support for the uppercase HTTPS_PROXY form and takes precedence if both are present. Contributed by Timothy Leung. (#9372)
+* Add a configuration option, user_directory.prefer_local_users, which when enabled will make it more likely for users on the same server as you to appear above other users. (#9383, #9385)
+* Add support for regenerating thumbnails if they have been deleted but the original image is still stored. (#9438)
+
+[1.21.0]
+* Update Synapse to 1.30.0
+* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.30.0)
+* Add prometheus metrics for number of users successfully registering and logging in. (#9510, #9511, #9573)
+* Add synapse_federation_last_sent_pdu_time and synapse_federation_last_received_pdu_time prometheus metrics, which monitor federation delays by reporting the timestamps of messages sent and received to a set of remote servers. (#9540)
+* Add support for generating JSON Web Tokens dynamically for use as OIDC client secrets. (#9549)
+* Optimise handling of incomplete room history for incoming federation. (#9601)
+* Finalise support for allowing clients to pick an SSO Identity Provider (MSC2858). (#9617)
+* Tell spam checker modules about the SSO IdP a user registered through if one was used. (#9626)
+
+[1.21.1]
+* Update Synapse to 1.30.1
+
+[1.22.0]
+* Update Synapse to 1.31.0
+* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.31.0)
+* Add support to OpenID Connect login for requiring attributes on the userinfo response. Contributed by Hubbe King. (#9609)
+* Add initial experimental support for a "space summary" API. (#9643, #9652, #9653)
+* Add support for the busy presence state as described in MSC3026. (#9644)
+* Add support for credentials for proxy authentication in the HTTPS_PROXY environment variable. (#9657)
+
+[1.22.1]
+* Update Synapse to 1.32.0
+* Add a Synapse module for routing presence updates between users. (#9491)
+* Add an admin API to manage ratelimit for a specific user. (#9648)
+* Include request information in structured logging output. (#9654)
+* Add order_by to the admin API GET /_synapse/admin/v2/users. Contributed by @dklimpel. (#9691)
+* Replace the room_invite_state_types configuration setting with room_prejoin_state. (#9700)
+* Add experimental support for MSC3083: restricting room access via group membership. (#9717, #9735)
+* Update experimental support for Spaces: include m.room.create in the room state sent with room-invites. (#9710)
+* Synapse now requires Python 3.6 or later. It also requires Postgres 9.6 or later or SQLite 3.22 or later. (#9766)
+* Prevent synapse_forward_extremities and synapse_excess_extremity_events Prometheus metrics from initially reporting zero-values after startup. (#8926)
+* Fix recently added ratelimits to correctly honour the application service rate_limited flag. (#9711)
+* Fix longstanding bug which caused duplicate key value violates unique constraint "remote_media_cache_thumbnails_media_origin_media_id_thumbna_key" errors. (#9725)
+* Fix bug where sharded federation senders could get stuck repeatedly querying the DB in a loop, using lots of CPU. (#9770)
+* Fix duplicate logging of exceptions thrown during federation transaction processing. (#9780)
+
+[1.22.2]
+* Update Synapse to 1.32.0
+* Add a Synapse module for routing presence updates between users. (#9491)
+* Add an admin API to manage ratelimit for a specific user. (#9648)
+* Include request information in structured logging output. (#9654)
+* Add order_by to the admin API GET /_synapse/admin/v2/users. Contributed by @dklimpel. (#9691)
+* Replace the room_invite_state_types configuration setting with room_prejoin_state. (#9700)
+* Add experimental support for MSC3083: restricting room access via group membership. (#9717, #9735)
+* Update experimental support for Spaces: include m.room.create in the room state sent with room-invites. (#9710)
+* Synapse now requires Python 3.6 or later. It also requires Postgres 9.6 or later or SQLite 3.22 or later. (#9766)
+* Prevent synapse_forward_extremities and synapse_excess_extremity_events Prometheus metrics from initially reporting zero-values after startup. (#8926)
+* Fix recently added ratelimits to correctly honour the application service rate_limited flag. (#9711)
+* Fix longstanding bug which caused duplicate key value violates unique constraint "remote_media_cache_thumbnails_media_origin_media_id_thumbna_key" errors. (#9725)
+* Fix bug where sharded federation senders could get stuck repeatedly querying the DB in a loop, using lots of CPU. (#9770)
+* Fix duplicate logging of exceptions thrown during federation transaction processing. (#9780)
+
+[1.22.3]
+* Update Synapse to 1.32.1
+* Fix a regression in Synapse 1.32.0 which caused Synapse to report large numbers of Prometheus time series, potentially overwhelming Prometheus instances. (#9854)
+
+[1.22.4]
+* Update Synapse to 1.32.2
+* Fix a regression in Synapse 1.32.0 and 1.32.1 which caused LoggingContext errors in plugins. (#9857)
+
+[1.23.0]
+* Update Synapse to 1.33.0
+* Update experimental support for MSC3083: restricting room access via group membership. (#9800, #9814)
+* Add experimental support for handling presence on a worker. (#9819, #9820, #9828, #9850)
+* Return a new template when an user attempts to renew their account multiple times with the same token, stating that their account is set to expire. This replaces the invalid token template that would previously be shown in this case. This change concerns the optional account validity feature. (#9832)
+* Fixes the OIDC SSO flow when using a public_baseurl value including a non-root URL path. (#9726)
+* Fix thumbnail generation for some sites with non-standard content types. Contributed by @rkfg. (#9788)
+* Add some sanity checks to identity server passed to 3PID bind/unbind endpoints. (#9802)
+* Limit the size of HTTP responses read over federation. (#9833)
+* Fix a bug which could cause Synapse to get stuck in a loop of resyncing device lists. (#9867)
+* Fix a long-standing bug where errors from federation did not propagate to the client. (#9868)

CloudronManifest.json

@@ -5,7 +5,7 @@
   "description": "file://DESCRIPTION.md",
   "changelog": "file://CHANGELOG",
   "tagline": "Secure & decentralized communication",
-  "version": "1.14.1",
+  "version": "1.23.0",
   "healthCheckPath": "/",
   "httpPort": 8008,
   "memoryLimit": 536870912,
@@ -21,7 +21,7 @@
   "contactEmail": "support@cloudron.io",
   "icon": "file://logo.png",
   "tags": [
-    "im", "collaboration", "voip", "videochat", "chat", "slack", "zulip"
+    "im", "collaboration", "voip", "videochat", "chat", "slack", "zulip", "federated"
   ],
   "mediaLinks": [
     "https://screenshots.cloudron.io/org.matrix.synapse/1.png",

DESCRIPTION.md

@@ -1,4 +1,4 @@
-This app packages Synapse <upstream>1.23.1</upstream>.
+This app packages Synapse <upstream>1.33.0</upstream>.
 
 **This package only provides the Matrix backend. A variety of clients are available
 [here](https://matrix.org/clients/). Riot is a popular web frontend for Matrix and

Dockerfile

@@ -1,26 +1,23 @@
-FROM cloudron/base:2.0.0@sha256:f9fea80513aa7c92fe2e7bf3978b54c8ac5222f47a9a32a7f8833edf0eb5a4f4
+FROM cloudron/base:3.0.0@sha256:455c70428723e3a823198c57472785437eb6eab082e79b3ff04ea584faf46e92
 
 RUN mkdir -p /app/pkg
 
 WORKDIR /app/code
 
-ARG VERSION=v1.23.1
-
 # https://pythonspeed.com/articles/activate-virtualenv-dockerfile/
 RUN virtualenv -p python3 /app/code/env
 ENV VIRTUAL_ENV=/app/code/env
 ENV PATH="$VIRTUAL_ENV/bin:$PATH"
 
+ARG VERSION=v1.33.0
+
 # Synapse (https://github.com/matrix-org/synapse/blob/master/INSTALL.md)
 # lxml - required for previews
 RUN pip install --upgrade pip && \
     pip install --upgrade setuptools && \
     pip install matrix-synapse==${VERSION} psycopg2-binary python-ldap matrix-synapse-ldap3 lxml publicsuffix2
 
-RUN curl -sL https://github.com/mikefarah/yq/releases/download/3.2.1/yq_linux_amd64 -o /usr/bin/yq && \
-    chmod +x /usr/bin/yq
-
-RUN ln -sf /app/data/index.html /app/code/env/lib/python3.6/site-packages/synapse/static/index.html
+RUN ln -sf /app/data/index.html /app/code/env/lib/python3.8/site-packages/synapse/static/index.html
 
 RUN chown -R cloudron.cloudron /app/code
 

start.sh

@@ -21,62 +21,63 @@ if [[ ! -f /app/data/configs/homeserver.yaml ]]; then
     # fix logging configuration
     cp /app/pkg/homeserver.yaml.template /app/data/configs/homeserver.yaml
     mv /app/data/configs/${server_name}.log.config /app/data/configs/log.config
-    yq w -i /app/data/configs/homeserver.yaml log_config /app/data/configs/log.config
-    yq w -i /app/data/configs/log.config handlers.file.filename /run/synapse/homeserver.log
+    yq eval -i ".log_config=\"/app/data/configs/log.config\"" /app/data/configs/homeserver.yaml
+    yq eval -i ".handlers.file.filename=\"/run/synapse/homeserver.log\"" /app/data/configs/log.config
 
     mv /app/data/configs/${server_name}.signing.key /app/data/configs/signing.key
 
-    yq w -i /app/data/configs/homeserver.yaml server_name "${server_name}"
-    yq w -i /app/data/configs/homeserver.yaml registration_shared_secret "$(pwgen -1s 64)"
+    yq eval -i ".server_name=\"${server_name}\"" /app/data/configs/homeserver.yaml
+    yq eval -i ".registration_shared_secret=\"$(pwgen -1s 64)\"" /app/data/configs/homeserver.yaml
 
-    yq w -i /app/data/configs/homeserver.yaml auto_join_rooms "[]"
-    yq w -i /app/data/configs/homeserver.yaml auto_join_rooms\[0\] "#discuss:${server_name}"
+    yq eval -i ".auto_join_rooms=[]" /app/data/configs/homeserver.yaml
+    yq eval -i ".auto_join_rooms[0]=\"#discuss:${server_name}\"" /app/data/configs/homeserver.yaml
 
     if [[ -z "${CLOUDRON_LDAP_SERVER:-}" ]]; then
-        yq w -i /app/data/configs/homeserver.yaml enable_registration true
-        yq w -i /app/data/configs/homeserver.yaml password_config.pepper "$(pwgen -1s 12)"
+        yq eval -i ".enable_registration=true" /app/data/configs/homeserver.yaml
+        yq eval -i ".password_config.pepper=\"$(pwgen -1s 12)\"" /app/data/configs/homeserver.yaml
     fi
 fi
 
 [[ ! -f /app/data/index.html ]] && cp /app/pkg/index.html /app/data/index.html
 
 echo "==> Configuring synapse"
-yq w -i /app/data/configs/homeserver.yaml public_baseurl "${CLOUDRON_APP_ORIGIN}"
+yq eval -i ".public_baseurl=\"${CLOUDRON_APP_ORIGIN}\"" /app/data/configs/homeserver.yaml
 
 # database
-yq w -i /app/data/configs/homeserver.yaml database.args.user "${CLOUDRON_POSTGRESQL_USERNAME}"
-yq w -i /app/data/configs/homeserver.yaml database.args.password "${CLOUDRON_POSTGRESQL_PASSWORD}"
-yq w -i /app/data/configs/homeserver.yaml database.args.database "${CLOUDRON_POSTGRESQL_DATABASE}"
-yq w -i /app/data/configs/homeserver.yaml database.args.host "${CLOUDRON_POSTGRESQL_HOST}"
+yq eval -i ".database.args.user=\"${CLOUDRON_POSTGRESQL_USERNAME}\"" /app/data/configs/homeserver.yaml
+yq eval -i ".database.args.password=\"${CLOUDRON_POSTGRESQL_PASSWORD}\"" /app/data/configs/homeserver.yaml
+yq eval -i ".database.args.database=\"${CLOUDRON_POSTGRESQL_DATABASE}\"" /app/data/configs/homeserver.yaml
+yq eval -i ".database.args.host=\"${CLOUDRON_POSTGRESQL_HOST}\"" /app/data/configs/homeserver.yaml
 
 # email
-yq w -i /app/data/configs/homeserver.yaml email.smtp_host "${CLOUDRON_MAIL_SMTP_SERVER}"
-yq w -i /app/data/configs/homeserver.yaml email.smtp_port "${CLOUDRON_MAIL_SMTP_PORT}"
-yq w -i /app/data/configs/homeserver.yaml email.smtp_user "${CLOUDRON_MAIL_SMTP_USERNAME}"
-yq w -i /app/data/configs/homeserver.yaml email.smtp_pass "${CLOUDRON_MAIL_SMTP_PASSWORD}"
-yq w -i /app/data/configs/homeserver.yaml email.notif_from "%(app)s <${CLOUDRON_MAIL_FROM}>"
+yq eval -i ".email.smtp_host=\"${CLOUDRON_MAIL_SMTP_SERVER}\"" /app/data/configs/homeserver.yaml
+yq eval -i ".email.smtp_port=${CLOUDRON_MAIL_SMTP_PORT}" /app/data/configs/homeserver.yaml
+yq eval -i ".email.smtp_user=\"${CLOUDRON_MAIL_SMTP_USERNAME}\"" /app/data/configs/homeserver.yaml
+yq eval -i ".email.smtp_pass=\"${CLOUDRON_MAIL_SMTP_PASSWORD}\"" /app/data/configs/homeserver.yaml
+yq eval -i ".email.notif_from=\"%(app)s <${CLOUDRON_MAIL_FROM}>\"" /app/data/configs/homeserver.yaml
 
 # ldap
 if [[ -n "${CLOUDRON_LDAP_SERVER:-}" ]]; then
-    yq w -i /app/data/configs/homeserver.yaml 'password_providers[0].config.uri' "${CLOUDRON_LDAP_URL}"
-    yq w -i /app/data/configs/homeserver.yaml 'password_providers[0].config.start_tls' false
-    yq w -i /app/data/configs/homeserver.yaml 'password_providers[0].config.base' "${CLOUDRON_LDAP_USERS_BASE_DN}"
-    yq w -i /app/data/configs/homeserver.yaml 'password_providers[0].config.bind_dn' "${CLOUDRON_LDAP_BIND_DN}"
-    yq w -i /app/data/configs/homeserver.yaml 'password_providers[0].config.bind_password' "${CLOUDRON_LDAP_BIND_PASSWORD}"
-    yq w -i /app/data/configs/homeserver.yaml 'password_providers[0].config.filter' "(objectClass=user)"
+    yq eval -i ".password_providers[0].config.uri=\"${CLOUDRON_LDAP_URL}\"" /app/data/configs/homeserver.yaml
+    yq eval -i ".password_providers[0].config.start_tls=false" /app/data/configs/homeserver.yaml
+    yq eval -i ".password_providers[0].config.base=\"${CLOUDRON_LDAP_USERS_BASE_DN}\"" /app/data/configs/homeserver.yaml
+    yq eval -i ".password_providers[0].config.bind_dn=\"${CLOUDRON_LDAP_BIND_DN}\"" /app/data/configs/homeserver.yaml
+    yq eval -i ".password_providers[0].config.bind_password=\"${CLOUDRON_LDAP_BIND_PASSWORD}\"" /app/data/configs/homeserver.yaml
+    yq eval -i ".password_providers[0].config.filter=\"(objectClass=user)\"" /app/data/configs/homeserver.yaml
+
 else
-    yq w -i /app/data/configs/homeserver.yaml password_config.localdb_enabled true
+    yq eval -i ".password_config.localdb_enabled=true" /app/data/configs/homeserver.yaml
 fi
 
 # turn (https://github.com/matrix-org/synapse/blob/master/docs/turn-howto.md#synapse-setup)
-yq w -i /app/data/configs/homeserver.yaml turn_uris "[]"
-yq w -i /app/data/configs/homeserver.yaml turn_uris\[0\] "turn:${CLOUDRON_TURN_SERVER}:${CLOUDRON_TURN_TLS_PORT}?transport=udp"
-yq w -i /app/data/configs/homeserver.yaml turn_uris\[1\] "turn:${CLOUDRON_TURN_SERVER}:${CLOUDRON_TURN_TLS_PORT}?transport=tcp"
-yq w -i /app/data/configs/homeserver.yaml turn_shared_secret "${CLOUDRON_TURN_SECRET}"
+yq eval -i ".turn_uris=[]" /app/data/configs/homeserver.yaml
+yq eval -i ".turn_uris[0]=\"turn:${CLOUDRON_TURN_SERVER}:${CLOUDRON_TURN_TLS_PORT}?transport=udp\"" /app/data/configs/homeserver.yaml
+yq eval -i ".turn_uris[1]=\"turn:${CLOUDRON_TURN_SERVER}:${CLOUDRON_TURN_TLS_PORT}?transport=tcp\"" /app/data/configs/homeserver.yaml
+yq eval -i ".turn_shared_secret=\"${CLOUDRON_TURN_SECRET}\"" /app/data/configs/homeserver.yaml
 
 # fix permissions
 echo "==> Fixing permissions"
 chown -R cloudron.cloudron /app/data /run/synapse
 
 echo "==> Starting synapse"
-gosu cloudron:cloudron python3 -m synapse.app.homeserver --config-path /app/data/configs/homeserver.yaml
+gosu cloudron:cloudron python3 -m synapse.app.homeserver --config-path /app/data/configs/homeserver.yaml -n

test/package.json

@@ -9,12 +9,9 @@
   "author": "",
   "license": "ISC",
   "dependencies": {
-    "chromedriver": "^86.0.0",
-    "ejs": "^3.1.5",
+    "chromedriver": "^90.0.0",
     "expect.js": "^0.3.1",
-    "mkdirp": "^1.0.4",
-    "mocha": "^8.1.3",
-    "rimraf": "^3.0.2",
+    "mocha": "^8.3.2",
     "selenium-server-standalone-jar": "^3.141.59",
     "selenium-webdriver": "^3.6.0",
     "superagent": "^6.1.0"