Version 1.84.0

when external registration is enabled, this means that we don't move
all of the external users as oidc accounts

CHANGELOG

@@ -1,652 +0,0 @@
-[0.1.0]
-* Initial version
-
-[0.1.1]
-* Removed the Riot part of the installation due to security
-* Changed from Sqlite to Postgres
-* Added LDAP support
-* Fixed upload limit in nginx_matrix.conf
-* Added bogus index.html so cloudron recognizes the matrix server as online
-* Added coturn as TURN server
-
-[0.1.2]
-* Updated logo (transparent)
-* Enabled email settings
-
-[0.1.3]
-* Changed log location of nginx due to backup crashes
-
-[0.1.4]
-* Changed log location of synapse due to backup crashes
-* Updated some configs
-
-[0.1.5]
-* Update to synapse v0.31.2
-
-[0.1.6]
-* Update to synapse v0.33.8
-
-[0.1.7]
-* Update to synapse v0.33.9
-
-[0.1.8]
-* Update to synapse v0.99.0
-* Fixed missing LDAP stuff
-
-[0.1.9]
-* Update to synapse v0.99.3
-
-[0.2.0]
-* Update to synapse v0.99.4
-
-[0.2.1]
-* Update to synapse v0.99.5.1
-
-[0.2.2]
-* Update to synapse v0.99.5.2
-
-[0.2.3]
-* Update to synapse v1.0.0
-* Fixed url preview
-* Fixed voip
-
-[0.2.4]
-* Fixed federation
-
-[0.2.5]
-* Update to synapse v1.1.0
-
-[0.2.6]
-* Update to synapse v1.2.1
-
-[0.2.7]
-* Update to synapse v1.3.0
-
-[0.2.8]
-* Update to synapse v1.3.1
-
-[0.2.9]
-* Update to synapse v1.4.0 (lots of changes)
-
-[0.3.0]
-* Update to synapse v1.4.1
-
-[0.3.1]
-* Update to synapse v1.5.1
-
-[0.3.3]
-* Update to synapse v1.6.0
-
-[0.3.4]
-* Update to synapse v1.6.1
-
-[0.3.5]
-* Update to synapse v1.8.0
-
-[0.3.6]
-* Update to synapse v1.9.1
-
-[0.3.7]
-* Update to synapse v1.10.0
-
-[0.3.8]
-* Update to synapse v1.11.0
-
-[0.4.0]
-* Update to synapse v1.12.0
-
-[0.4.1]
-* Update to synapse v1.12.2
-
-[0.5.0]
-* New reworked app
-
-[0.6.0]
-* Fix title
-
-[0.7.0]
-* Set turn_uris to an array and not a string
-
-[0.7.1]
-* Users will now automatically join the #discuss channel (only in new installations)
-
-[1.0.0]
-* Use latest base image
-* Update to synapse v1.12.4
-
-[1.1.0]
-* Update Synapse to 1.13.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.13.0)
-* Set Referrer-Policy header to no-referrer on media downloads. (#7009)
-* Admin API POST /_synapse/admin/v1/join/<roomIdOrAlias> to join users to a room like auto_join_rooms for creation of users. (#7051)
-* Add options to prevent users from changing their profile or associated 3PIDs. (#7096)
-* Allow server admins to define and enforce a password policy (MSC2000). (#7118)
-* Improve the support for SSO authentication on the login fallback page. (#7152, #7235)
-* Always whitelist the login fallback in the SSO configuration if public_baseurl is set. (#7153)
-* Admin users are no longer required to be in a room to create an alias for it. (#7191)
-* Require admin privileges to enable room encryption by default. This does not affect existing rooms. (#7230)
-
-[1.2.0]
-* Update Synapse to 1.14.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.14.0)
-
-[1.3.0]
-* Add optional sso support
-
-[1.4.0]
-* Update Synapse to 1.15.1
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.15.0)
-* Advertise support for Client-Server API r0.6.0 and remove related unstable feature flags. (#6585)
-* Add an option to disable autojoining rooms for guest accounts. (#6637)
-* Add admin APIs to allow server admins to manage users' devices. Contributed by @dklimpel. (#7481)
-* Add support for generating thumbnails for WebP images. Previously, users would see an empty box instead of preview image. Contributed by @WGH-. (#7586)
-* Support the standardized m.login.sso user-interactive authentication flow. (#7630)
-
-[1.5.0]
-* Update Synapse to 1.15.2
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.15.2)
-* A malicious homeserver could force Synapse to reset the state in a room to a small subset of the correct state. This affects all Synapse deployments which federate with untrusted servers. (96e9afe6)
-* HTML pages served via Synapse were vulnerable to clickjacking attacks. This predominantly affects homeservers with single-sign-on enabled, but all server administrators are encouraged to upgrade. (ea26e9a9)
-
-[1.6.0]
-* Update Synapse to 1.16.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.16.0)
-* Add an option to enable encryption by default for new rooms. (#7639)
-* Add support for running multiple media repository workers. See docs/workers.md for instructions. (#7706)
-* Media can now be marked as safe from quarantined. (#7718)
-* Expand the configuration options for auto-join rooms. (#7763)
-
-[1.6.1]
-* Update Synapse to 1.16.1
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.16.1)
-* Drop table local_rejections_stream which was incorrectly added in Synapse 1.16.0. (#7816, b1beb3ff5)
-
-[1.7.0]
-* Update Synapse to 1.17.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.17.0)
-* Fix inconsistent handling of upper and lower case in email addresses when used as identifiers for login, etc. Contributed by @dklimpel. (#7021)
-* Fix "Tried to close a non-active scope!" error messages when opentracing is enabled. (#7732)
-* Fix incorrect error message when database CTYPE was set incorrectly. (#7760)
-* Fix to not ignore set_tweak actions in Push Rules that have no value, as permitted by the specification. (#7766)
-* Fix synctl to handle empty config files correctly. Contributed by @kotovalexarian. (#7779)
-* Fixes a long standing bug in worker mode where worker information was saved in the devices table instead of the original IP address and user agent. (#7797)
-* Fix 'stuck invites' which happen when we are unable to reject a room invite received over federation. (#7804, #7809, #7810)
-
-[1.8.0]
-* Update Synapse to 1.18.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.18.0)
-* Include room states on invite events that are sent to application services. Contributed by @Sorunome. (#6455)
-* Add delete room admin endpoint (POST /_synapse/admin/v1/rooms/<room_id>/delete). Contributed by @dklimpel. (#7613, #7953)
-* Add experimental support for running multiple federation sender processes. (#7798)
-* Add the option to validate the iss and aud claims for JWT logins. (#7827)
-* Add support for handling registration requests across multiple client reader workers. (#7830)
-* Add an admin API to list the users in a room. Contributed by Awesome Technologies Innovationslabor GmbH. (#7842)
-* Allow email subjects to be customised through Synapse's configuration. (#7846)
-* Add the ability to re-activate an account from the admin API. (#7847, #7908)
-* Support oEmbed for media previews. (#7920)
-
-[1.9.0]
-* Update Synapse to 1.19.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.19.0)
-* Add option to allow server admins to join rooms which fail complexity checks. Contributed by @lugino-emeritus. (#7902)
-* Add an option to purge room or not with delete room admin endpoint (POST /_synapse/admin/v1/rooms/<room_id>/delete). Contributed by @dklimpel. (#7964)
-* Add rate limiting to users joining rooms. (#8008)
-* Add a /health endpoint to every configured HTTP listener that can be used as a health check endpoint by load balancers. (#8048)
-* Allow login to be blocked based on the values of SAML attributes. (#8052)
-* Allow guest access to the GET /_matrix/client/r0/rooms/{room_id}/members endpoint, according to MSC2689. Contributed by Awesome Technologies Innovationslabor GmbH. (#7314)
-
-[1.9.1]
-* Update Synapse to 1.19.1
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.19.1)
-* Fix a bug introduced in v1.19.0 where appservices with ratelimiting disabled would still be ratelimited when joining rooms. (#8139)
-* Fix a bug introduced in v1.19.0 that would cause e.g. profile updates to fail due to incorrect application of rate limits on join requests. (#8153)
-
-[1.10.0]
-* Update Synapse to 1.19.3
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.19.3)
-* Partially mitigate bug where newly joined servers couldn't get past events in a room when there is a malformed event. (#8350)
-* Make index.html customizable
-
-[1.11.0]
-* Update Synapse to 1.20.1
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.20.0)
-* Add an endpoint to query your shared rooms with another user as an implementation of MSC2666. (#7785)
-* Iteratively encode JSON to avoid blocking the reactor. (#8013, #8116)
-* Add support for shadow-banning users (ignoring any message send requests). (#8034, #8092, #8095, #8142, #8152, #8157, #8158, #8176)
-* Use the default template file when its equivalent is not found in a custom template directory. (#8037, #8107, #8252)
-* Add unread messages count to sync responses, as specified in MSC2654. (#8059, #8254, #8270, #8274)
-* Optimise /federation/v1/user/devices/ API by only returning devices with encryption keys. (#8198)
-
-[1.12.0]
-* Update Synapse to 1.21.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.21.0)
-* Require the user to confirm that their password should be reset after clicking the email confirmation link. (#8004)
-* Add an admin API GET /_synapse/admin/v1/event_reports to read entries of table event_reports. Contributed by @dklimpel. (#8217)
-* Consolidate the SSO error template across all configuration. (#8248, #8405)
-* Add a configuration option to specify a whitelist of domains that a user can be redirected to after validating their email or phone number. (#8275, #8417)
-* Add experimental support for sharding event persister. (#8294, #8387, #8396, #8419)
-* Add the room topic and avatar to the room details admin API. (#8305)
-* Add an admin API for querying rooms where a user is a member. Contributed by @dklimpel. (#8306)
-
-[1.12.1]
-* Updat Synapse to 1.21.1
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.21.1)
-
-[1.12.2]
-* Update Synapse to 1.21.2
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.21.2)
-* Security: HTML pages served via Synapse were vulnerable to cross-site scripting (XSS) attacks. All server administrators are encouraged to upgrade
-* Fix rare bug where sending an event would fail due to a racey assertion. (#8530)
-
-[1.13.0]
-* Update Synapse to 1.22.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.22.0)
-* Add ability for ThirdPartyEventRules modules to query and manipulate whether a room is in the public rooms directory. (#8292, #8467)
-* Add support for olm fallback keys (MSC2732). (#8312, #8501)
-* Add support for running background tasks in a separate worker process. (#8369, #8458, #8489, #8513, #8544, #8599)
-* Add support for device dehydration (MSC2697). (#8380)
-* Add support for MSC2409, which allows sending typing, read receipts, and presence events to appservices. (#8437, #8590)
-* Change default room version to "6", per MSC2788. (#8461)
-* Add the ability to send non-membership events into a room via the ModuleApi. (#8479)
-* Increase default upload size limit from 10M to 50M. Contributed by @Akkowicz. (#8502)
-* Add support for modifying event content in ThirdPartyRules modules. (#8535, #8564)
-
-[1.13.1]
-* Update Synapse to 1.22.1
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.22.1)
-* Fix a bug where an appservice may not be forwarded events for a room it was recently invited to. Broke in v1.22.0. (#8676)
-* Fix Object of type frozendict is not JSON serializable exceptions when using third-party event rules. Broke in v1.22.0. (#8678)
-
-[1.14.0]
-* Update Synapse to 1.23.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.23.0)
-* Add a push rule that highlights when a jitsi conference is created in a room. (#8286)
-* Add an admin api to delete a single file or files that were not used for a defined time from server. Contributed by @dklimpel. (#8519)
-* Split admin API for reported events (GET /_synapse/admin/v1/event_reports) into detail and list endpoints. This is a breaking change to #8217 which was introduced in Synapse v1.21.0. Those who already use this API should check their scripts. Contributed by @dklimpel. (#8539)
-* Support generating structured logs via the standard logging configuration. (#8607, #8685)
-* Add an admin API to allow server admins to list users' pushers. Contributed by @dklimpel. (#8610, #8689)
-* Add an admin API GET /_synapse/admin/v1/users/<user_id>/media to get information about uploaded media. Contributed by @dklimpel. (#8647)
-* Add an admin API for local user media statistics. Contributed by @dklimpel. (#8700)
-* Add displayname to Shared-Secret Registration for admins. (#8722)
-
-[1.14.1]
-* Update Synapse to 1.23.1
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.23.1)
-* There is a denial of service attack (CVE-2020-26257) against the federation APIs in which future events will not be correctly sent to other servers over federation. This affects all servers that participate in open federation. (Fixed in #8776).
-
-[1.15.0]
-* Update Synapse to 1.24.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.24.0)
-* Add a maximum version for pysaml2 on Python 3.5
-
-[1.16.0]
-* Update Synapse to 1.25.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.25.0)
-* Add an admin API that lets server admins get power in rooms in which local users have power. (#8756)
-* Add optional HTTP authentication to replication endpoints. (#8853)
-* Improve the error messages printed as a result of configuration problems for extension modules. (#8874)
-* Add the number of local devices to Room Details Admin API. Contributed by @dklimpel. (#8886)
-* Add X-Robots-Tag header to stop web crawlers from indexing media. Contributed by Aaron Raimist. (#8887)
-* Spam-checkers may now define their methods as async. (#8890)
-* Add support for allowing users to pick their own user ID during a single-sign-on login. (#8897, #8900, #8911, #8938, #8941, #8942, #8951)
-* Add an email.invite_client_location configuration option to send a web client location to the invite endpoint on the identity server which allows customisation of the email template. (#8930)
-* The search term in the list room and list user Admin APIs is now treated as case-insensitive. (#8931)
-* Apply an IP range blacklist to push and key revocation requests. (#8821, #8870, #8954)
-* Add an option to allow re-use of user-interactive authentication sessions for a period of time. (#8970)
-* Allow running the redact endpoint on workers. (#8994)
-
-[1.17.0]
-* Update Synapse to 1.26.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.26.0)
-* During user-interactive authentication via single-sign-on, give a better error if the user uses the wrong account on the SSO IdP. (#9091)
-* Give the public_baseurl a default value, if it is not explicitly set in the configuration file. (#9159)
-* Improve performance when calculating ignored users in large rooms. (#9024)
-* Implement MSC2176 in an experimental room version. (#8984)
-* Add an admin API for protecting local media from quarantine. (#9086)
-* Remove a user's avatar URL and display name when deactivated with the Admin API. (#8932)
-
-[1.18.0]
-* Update Synapse to 1.27.0
-* Use base image v3
-* Update python to 3.8
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.27.0)
-* Add an admin API for getting and deleting forward extremities for a room. (#9062)
-* Add an admin API for retrieving the current room state of a room. (#9168)
-* Add an admin API endpoint for shadow-banning users. (#9209)
-
-[1.19.0]
-* Update Synapse to 1.28.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.28.0)
-* New admin API to get the context of an event: /_synapse/admin/rooms/{roomId}/context/{eventId}. (#9150)
-* Further improvements to the user experience of registration via single sign-on. (#9300, #9301)
-* Add hook to spam checker modules that allow checking file uploads and remote downloads. (#9311)
-* Add support for receiving OpenID Connect authentication responses via form POSTs rather than GETs. (#9376)
-* Add the shadow-banning status to the admin API for user info. (#9400)
-
-[1.20.0]
-* Update Synapse to 1.29.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.29.0)
-* Add rate limiters to cross-user key sharing requests. (#8957)
-* Add order_by to the admin API GET /_synapse/admin/v1/users/<user_id>/media. Contributed by @dklimpel. (#8978)
-* Add some configuration settings to make users' profile data more private. (#9203)
-* The no_proxy and NO_PROXY environment variables are now respected in proxied HTTP clients with the lowercase form taking precedence if both are present. Additionally, the lowercase https_proxy environment variable is now respected in proxied HTTP clients on top of existing support for the uppercase HTTPS_PROXY form and takes precedence if both are present. Contributed by Timothy Leung. (#9372)
-* Add a configuration option, user_directory.prefer_local_users, which when enabled will make it more likely for users on the same server as you to appear above other users. (#9383, #9385)
-* Add support for regenerating thumbnails if they have been deleted but the original image is still stored. (#9438)
-
-[1.21.0]
-* Update Synapse to 1.30.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.30.0)
-* Add prometheus metrics for number of users successfully registering and logging in. (#9510, #9511, #9573)
-* Add synapse_federation_last_sent_pdu_time and synapse_federation_last_received_pdu_time prometheus metrics, which monitor federation delays by reporting the timestamps of messages sent and received to a set of remote servers. (#9540)
-* Add support for generating JSON Web Tokens dynamically for use as OIDC client secrets. (#9549)
-* Optimise handling of incomplete room history for incoming federation. (#9601)
-* Finalise support for allowing clients to pick an SSO Identity Provider (MSC2858). (#9617)
-* Tell spam checker modules about the SSO IdP a user registered through if one was used. (#9626)
-
-[1.21.1]
-* Update Synapse to 1.30.1
-
-[1.22.0]
-* Update Synapse to 1.31.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.31.0)
-* Add support to OpenID Connect login for requiring attributes on the userinfo response. Contributed by Hubbe King. (#9609)
-* Add initial experimental support for a "space summary" API. (#9643, #9652, #9653)
-* Add support for the busy presence state as described in MSC3026. (#9644)
-* Add support for credentials for proxy authentication in the HTTPS_PROXY environment variable. (#9657)
-
-[1.22.1]
-* Update Synapse to 1.32.0
-* Add a Synapse module for routing presence updates between users. (#9491)
-* Add an admin API to manage ratelimit for a specific user. (#9648)
-* Include request information in structured logging output. (#9654)
-* Add order_by to the admin API GET /_synapse/admin/v2/users. Contributed by @dklimpel. (#9691)
-* Replace the room_invite_state_types configuration setting with room_prejoin_state. (#9700)
-* Add experimental support for MSC3083: restricting room access via group membership. (#9717, #9735)
-* Update experimental support for Spaces: include m.room.create in the room state sent with room-invites. (#9710)
-* Synapse now requires Python 3.6 or later. It also requires Postgres 9.6 or later or SQLite 3.22 or later. (#9766)
-* Prevent synapse_forward_extremities and synapse_excess_extremity_events Prometheus metrics from initially reporting zero-values after startup. (#8926)
-* Fix recently added ratelimits to correctly honour the application service rate_limited flag. (#9711)
-* Fix longstanding bug which caused duplicate key value violates unique constraint "remote_media_cache_thumbnails_media_origin_media_id_thumbna_key" errors. (#9725)
-* Fix bug where sharded federation senders could get stuck repeatedly querying the DB in a loop, using lots of CPU. (#9770)
-* Fix duplicate logging of exceptions thrown during federation transaction processing. (#9780)
-
-[1.22.2]
-* Update Synapse to 1.32.0
-* Add a Synapse module for routing presence updates between users. (#9491)
-* Add an admin API to manage ratelimit for a specific user. (#9648)
-* Include request information in structured logging output. (#9654)
-* Add order_by to the admin API GET /_synapse/admin/v2/users. Contributed by @dklimpel. (#9691)
-* Replace the room_invite_state_types configuration setting with room_prejoin_state. (#9700)
-* Add experimental support for MSC3083: restricting room access via group membership. (#9717, #9735)
-* Update experimental support for Spaces: include m.room.create in the room state sent with room-invites. (#9710)
-* Synapse now requires Python 3.6 or later. It also requires Postgres 9.6 or later or SQLite 3.22 or later. (#9766)
-* Prevent synapse_forward_extremities and synapse_excess_extremity_events Prometheus metrics from initially reporting zero-values after startup. (#8926)
-* Fix recently added ratelimits to correctly honour the application service rate_limited flag. (#9711)
-* Fix longstanding bug which caused duplicate key value violates unique constraint "remote_media_cache_thumbnails_media_origin_media_id_thumbna_key" errors. (#9725)
-* Fix bug where sharded federation senders could get stuck repeatedly querying the DB in a loop, using lots of CPU. (#9770)
-* Fix duplicate logging of exceptions thrown during federation transaction processing. (#9780)
-
-[1.22.3]
-* Update Synapse to 1.32.1
-* Fix a regression in Synapse 1.32.0 which caused Synapse to report large numbers of Prometheus time series, potentially overwhelming Prometheus instances. (#9854)
-
-[1.22.4]
-* Update Synapse to 1.32.2
-* Fix a regression in Synapse 1.32.0 and 1.32.1 which caused LoggingContext errors in plugins. (#9857)
-
-[1.23.0]
-* Update Synapse to 1.33.0
-* Update experimental support for MSC3083: restricting room access via group membership. (#9800, #9814)
-* Add experimental support for handling presence on a worker. (#9819, #9820, #9828, #9850)
-* Return a new template when an user attempts to renew their account multiple times with the same token, stating that their account is set to expire. This replaces the invalid token template that would previously be shown in this case. This change concerns the optional account validity feature. (#9832)
-* Fixes the OIDC SSO flow when using a public_baseurl value including a non-root URL path. (#9726)
-* Fix thumbnail generation for some sites with non-standard content types. Contributed by @rkfg. (#9788)
-* Add some sanity checks to identity server passed to 3PID bind/unbind endpoints. (#9802)
-* Limit the size of HTTP responses read over federation. (#9833)
-* Fix a bug which could cause Synapse to get stuck in a loop of resyncing device lists. (#9867)
-* Fix a long-standing bug where errors from federation did not propagate to the client. (#9868)
-
-[1.23.1]
-* Update Synapse to 1.33.1
-* Fix bug where /sync would break if using the latest version of attrs dependency, by pinning to a previous version. (#9937)
-
-[1.23.2]
-* Update Synapse to 1.33.2
-* This release fixes a denial of service attack (CVE-2021-29471) against Synapse's push rules implementation.
-
-[1.24.0]
-* Update Synapse to 1.34.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.34.0)
-* Add support for DELETE /_synapse/admin/v1/rooms/<room_id>. (#9889)
-* Improve performance after joining a large room when presence is enabled. (#9910, #9916)
-* Support stable identifiers for MSC1772 Spaces. m.space.child events will now be taken into account when populating the experimental spaces summary response. Please see the upgrade notes if you have customised room_invite_state_types in your configuration. (#9915, #9966)
-* Improve performance of backfilling in large rooms. (#9935)
-
-[1.25.0]
-* Update Synapse to 1.35.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.35.0)
-* Add experimental support to allow a user who could join a restricted room to view it in the spaces summary. (#9922, #10007, #10038)
-* Reduce memory usage when joining very large rooms over federation. (#9958)
-* Add a configuration option which allows enabling opentracing by user id. (#9978)
-* Enable experimental support for MSC2946 (spaces summary API) and MSC3083 (restricted join rules) by default. (#10011)
-
-[1.25.1]
-* Update Synapse to 1.35.1
-* Fix a bug introduced in v1.35.0 where invite-only rooms would be shown to all users in a space, regardless of if the user had access to it. (#10109)
-
-[1.26.0]
-* Update Synapse to 1.36.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.36.0)
-
-[1.27.0]
-* Update Synapse to 1.38.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.38.0)
-* Implement refresh tokens as specified by MSC2918. (#9450)
-* Add support for evicting cache entries based on last access time. (#10205)
-* Omit empty fields from the /sync response. Contributed by @deepbluev7. (#10214)
-* Improve validation on federation send_{join,leave,knock} endpoints. (#10225, #10243)
-* Mark events received over federation which fail a spam check as "soft-failed". (#10263)
-* Add metrics for new inbound federation staging area. (#10284)
-* Add script to print information about recently registered users. (#10290)
-
-[1.27.1]
-* Update Synapse to 1.38.1
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.38.1)
-
-[1.28.0]
-* Update Synapse to 1.39.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.39.0)
-
-[1.29.0]
-* Update Synapse to 1.40.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.40.0)
-
-[1.30.0]
-* Update Synapse to 1.41.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.41.0)
-
-[1.30.1]
-* Send logs to the console
-* Fix postinstall message about federation
-
-[1.30.2]
-* Update Synapse to 1.41.1
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.41.1)
-* GHSA-3x4c-pq33-4w3q / CVE-2021-39164: Enumerating a private room's list of members and their display names.
-* GHSA-jj53-8fmw-f2w2 / CVE-2021-39163: Disclosing a private room's name, avatar, topic, and number of members.
-
-[1.31.0]
-* Update Synapse to 1.42.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.42.0)
-* Support room version 9 from MSC3375. (#10747)
-* Add support for MSC3231: Token authenticated registration. Users can be required to submit a token during registration to authenticate themselves. Contributed by Callum Brown. (#10142)
-* Add support for MSC3283: Expose enable_set_displayname in capabilities. (#10452)
-* Port the PresenceRouter module interface to the new generic interface. (#10524)
-* Add pagination to the spaces summary based on updates to MSC2946. (#10613, #10725)
-
-[1.32.0]
-* Update Synapse to 1.43.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.43.0)
-
-[1.33.0]
-* Update Synapse to 1.44.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.44.0)
-* Only allow the MSC2716 /batch_send?chunk_id=xxx endpoint to connect to an already existing insertion event. (#10776)
-* Improve oEmbed URL previews by processing the author name, photo, and video information. (#10814, #10819)
-* Speed up responding with large JSON objects to requests. (#10868, #10905)
-* Add a user_may_create_room_with_invites spam checker callback to allow modules to allow or deny a room creation request based on the invites and/or 3PID invites it includes. (#10898)
-
-[1.34.0]
-* Update Synapse to 1.45.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.45.0)
-* Fix a long-standing bug when using multiple event persister workers where events were not correctly sent down /sync due to a race. (#11045)
-* Fix a bug introduced in Synapse 1.45.0rc1 where the user directory would stop updating if it processed an event from a user not in the users table. (#11053)
-* Fix a bug introduced in Synapse 1.44.0 when logging errors during oEmbed processing. (#11061)
-
-[1.34.1]
-* Update Synapse to 1.45.1
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.45.1)
-* Revert change to counting of deactivated users towards the monthly active users limit, introduced in 1.45.0rc1. (#11127)
-
-[1.35.0]
-* Update Synapse to 1.46.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.46.0)
-* Fix a bug introduced in v1.46.0rc1 where URL previews of some XML documents would fail. (#11196)
-
-[1.36.0]
-* Update Synapse to 1.47.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.47.0)
-* Fix a bug introduced in 1.47.0rc1 which caused worker processes to not halt startup in the presence of outstanding database migrations. (#11346)
-* Fix a bug introduced in 1.47.0rc1 which prevented the 'remove deleted devices from device_inbox column' background process from running when updating from a recent Synapse version. (#11303, #11353)
-
-[1.36.1]
-* Update Synapse to 1.47.1
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.47.1)
-* GHSA-3hfw-x7gx-437c / CVE-2021-41281: Path traversal when downloading remote media.
-
-[1.37.0]
-* Update Synapse to 1.48.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.48.0)
-
-[1.38.0]
-* Update Synapse to 1.49.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.49.0)
-
-[1.38.1]
-* Update Synapse to 1.49.2
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.49.2)
-
-[1.38.2]
-* Update Synapse to 1.50.1
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.50.1)
-
-[1.38.3]
-* Update Synapse to 1.50.2
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.50.2)
-* Fix a bug introduced in Synapse 1.40.0 that caused Synapse to fail to process incoming federation traffic after handling a large amount of events in a v1 room. (#11806)
-
-[1.38.4]
-* Update Synapse to 1.51.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.51.0)
-
-[1.38.5]
-* Update Synapse to 1.52.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.52.0)
-* Remove account data (including client config, push rules and ignored users) upon user deactivation. (#11621, #11788, #11789)
-* Add an admin API to reset connection timeouts for remote server. (#11639)
-* Add an admin API to get a list of rooms that federate with a given remote homeserver. (#11658)
-* Add a config flag to inhibit M_USER_IN_USE during registration. (#11743)
-* Add a module callback to set username at registration. (#11790)
-* Allow configuring a maximum file size as well as a list of allowed content types for avatars. (#11846)
-
-[1.38.6]
-* Update Synapse to 1.53.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.53.0)
-* Add experimental support for sending to-device messages to application services, as specified by MSC2409. (#11215, #11966)
-* Add a background database update to purge account data for deactivated users. (#11655)
-* Experimental support for MSC3666: including bundled aggregations in server side search results. (#11837)
-* Enable cache time-based expiry by default. The expiry_time config flag has been superseded by expire_caches and cache_entry_ttl. (#11849)
-* Add a callback to allow modules to allow or forbid a 3PID (email address, phone number) from being associated to a local account. (#11854)
-* Stabilize support and remove unstable endpoints for MSC3231. Clients must switch to the stable identifier and endpoint. See the upgrade notes for more information. (#11867)
-* Allow modules to retrieve the current instance's server name and worker name. (#11868)
-* Use a dedicated configurable rate limiter for 3PID invites. (#11892)
-* Support the stable API endpoint for MSC3283: new settings in /capabilities endpoint. (#11933, #11989)
-* Support the dir parameter on the /relations endpoint, per MSC3715. (#11941)
-* Experimental implementation of MSC3706: extensions to /send_join to support reduced response size. (#11967)
-
-[1.39.0]
-* Update Synapse to 1.54.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.54.0)
-* Fix a bug introduced in Synapse 1.54.0rc1 preventing the new module callbacks introduced in this release from being registered by modules. (#12141)
-* Fix a bug introduced in Synapse 1.54.0rc1 where runtime dependency version checks would mistakenly check development dependencies if they were present and would not accept pre-release versions of dependencies. (#12129, #12177)
-
-[1.40.0]
-* Update Synapse to 1.55.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.55.0)
-* Add third-party rules callbacks check_can_shutdown_room and check_can_deactivate_user. (#12028)
-* Improve performance of logging in for large accounts. (#12132)
-* Support the stable identifiers from MSC3440: threads. (#12151)
-* Add a new Jinja2 template filter to extract the local part of an email address. (#12212)
-
-[1.40.1]
-* Update Synapse to 1.55.2
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.55.2)
-
-[1.41.0]
-* Update Synapse to 1.57.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.57.0)
-
-[1.41.1]
-* Update Synapse to 1.57.1
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.57.1)
-
-[1.42.0]
-* Update Synapse to 1.58.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.58.0)
-* Implement MSC3383 for including the destination in server-to-server authentication headers. Contributed by @Bubu and @jcgruenhage for Famedly. (#11398)
-* Enable processing of device list updates asynchronously. (#12365, #12465)
-* Implement MSC2815 to allow room moderators to view redacted event content. Contributed by @tulir @ Beeper. (#12427)
-
-[1.43.0]
-* Update Synapse to 1.59.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.59.0)
-
-[1.43.1]
-* Update Synapse to 1.59.1
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.59.1)
-
-[1.44.0]
-* Update Synapse to 1.60.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.60.0)
-* Add an option allowing users to use their password to reauthenticate for privileged actions even though password login is disabled. (#12883)
-* Explicitly close ijson coroutines once we are done with them, instead of leaving the garbage collector to close them. (#12875)
-* Improve URL previews by not including the content of media tags in the generated description. (#12887)
-
-[1.45.0]
-* Update Synapse to 1.61.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.61.0)
-* Add new media_retention options to the homeserver config for routinely cleaning up non-recently accessed media. (#12732, #12972, #12977)
-* Experimental support for MSC3772: Push rule for mutually related events. (#12740, #12859)
-* Update to the check_event_for_spam module callback: Deprecate the current callback signature, replace it with a new signature that is both less ambiguous (replacing booleans with explicit allow/block) and more powerful (ability to return explicit error codes). (#12808)
-* Add storage and module API methods to get monthly active users (and their corresponding appservices) within an optionally specified time range. (#12838, #12917)
-* Support the new error code ORG.MATRIX.MSC3823.USER_ACCOUNT_SUSPENDED from MSC3823. (#12845, #12923)
-* Add a configurable background job to delete stale devices. (#12855)
-* Improve URL previews for pages with empty elements. (#12951)
-* Allow updating a user's password using the admin API without logging out their devices. Contributed by @jcgruenhage. (#12952)
-
-[1.45.1]
-* Add s3 storage provider module
-
-[1.45.2]
-* Update Synapse to 1.61.1
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.61.1)
-* Linkify GHSA commit
-
-[1.46.0]
-* Update Synapse to 1.62.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.62.0)
-
-[1.47.0]
-* Update Synapse to 1.63.0
-* [Full changelog](https://github.com/matrix-org/synapse/releases/tag/v1.63.0)
-

CloudronManifest.json

@@ -3,19 +3,19 @@
   "title": "Matrix Synapse",
   "author": "Matrix synapse authors",
   "description": "file://DESCRIPTION.md",
-  "changelog": "file://CHANGELOG",
+  "changelog": "file://CHANGELOG.md",
   "tagline": "Secure & decentralized communication",
-  "version": "1.47.0",
-  "upstreamVersion": "1.63.0",
+  "version": "1.84.0",
+  "upstreamVersion": "1.98.0",
   "healthCheckPath": "/",
   "httpPort": 8008,
   "memoryLimit": 536870912,
   "addons": {
     "localstorage": {},
-    "ldap": {},
+    "oidc": { "loginRedirectUri": "/_synapse/client/oidc/callback" },
     "postgresql": {},
     "sendmail": { "supportsDisplayName": true },
-    "turn": {}
+    "turn": { "optional": true }
   },
   "manifestVersion": 2,
   "website": "https://matrix.org",
@@ -29,9 +29,8 @@
     "https://screenshots.cloudron.io/org.matrix.synapse/2.png",
     "https://screenshots.cloudron.io/org.matrix.synapse/3.png"
   ],
-  "changelog": "file://CHANGELOG",
   "postInstallMessage": "file://POSTINSTALL.md",
-  "minBoxVersion": "7.2.0",
+  "minBoxVersion": "7.5.1",
   "forumUrl": "https://forum.cloudron.io/category/50/matrix-synapse-riot",
   "documentationUrl": "https://docs.cloudron.io/apps/synapse/",
   "optionalSso": true

Dockerfile

@@ -1,4 +1,4 @@
-FROM cloudron/base:3.2.0@sha256:ba1d566164a67c266782545ea9809dc611c4152e27686fd14060332dd88263ea
+FROM cloudron/base:4.2.0@sha256:46da2fffb36353ef714f97ae8e962bd2c212ca091108d768ba473078319a47f4
 
 RUN mkdir -p /app/pkg
 
@@ -9,15 +9,21 @@ RUN virtualenv -p python3 /app/code/env
 ENV VIRTUAL_ENV=/app/code/env
 ENV PATH="$VIRTUAL_ENV/bin:$PATH"
 
-ARG VERSION=v1.63.0
+ARG VERSION=1.98.0
+
+# https://github.com/matrix-org/synapse-s3-storage-provider
+ARG STORAGE_PROVIDER_VERSION=1beb6af95e1f5caedb8e6e7e1cc176cdb2106d37
 
 # Synapse (https://github.com/matrix-org/synapse/blob/master/INSTALL.md)
 # lxml - required for previews
 RUN pip install --upgrade pip && \
     pip install --upgrade setuptools && \
-    pip install matrix-synapse==${VERSION} psycopg2-binary python-ldap matrix-synapse-ldap3 lxml publicsuffix2 git+https://github.com/matrix-org/synapse-s3-storage-provider.git@ffd3fa477321608e57d27644197e721965e0e858
+    pip install matrix-synapse==v${VERSION} psycopg2-binary python-ldap matrix-synapse-ldap3 lxml publicsuffix2 git+https://github.com/matrix-org/synapse-s3-storage-provider.git@${STORAGE_PROVIDER_VERSION} matrix-synapse[oidc]
 
-RUN ln -sf /app/data/index.html /app/code/env/lib/python3.8/site-packages/synapse/static/index.html
+# workaround (https://github.com/matrix-org/synapse/issues/15873) . remove after 1.87.0
+RUN sed -e "s/Image.ANTIALIAS/Image.LANCZOS/" -i /app/code/env/lib/python3.10/site-packages/synapse/media/thumbnailer.py
+
+RUN ln -sf /app/data/index.html /app/code/env/lib/python3.10/site-packages/synapse/static/index.html
 
 RUN chown -R cloudron.cloudron /app/code
 

POSTINSTALL.md

@@ -1,5 +1,5 @@
 Account ids are created with the username and the second level domain under which the
-app is installed e.g. `@$CLOUDRON-USERNAME@$CLOUDRON-APP-DOMAIN`.
+app is installed e.g. `@$CLOUDRON-USERNAME:$CLOUDRON-APP-DOMAIN`.
 
 For federation to work, the delegation URI `https://$CLOUDRON-APP-DOMAIN/.well-known/matrix/server`
 must be configured. See the [docs](https://docs.cloudron.io/apps/synapse/#post-installation) on how to do this.

start.sh

@@ -35,12 +35,12 @@ if [[ ! -f /app/data/configs/homeserver.yaml ]]; then
     yq eval -i ".auto_join_rooms=[]" /app/data/configs/homeserver.yaml
     yq eval -i ".auto_join_rooms[0]=\"#discuss:${server_name}\"" /app/data/configs/homeserver.yaml
 
-    if [[ -z "${CLOUDRON_LDAP_SERVER:-}" ]]; then
+    if [[ -z "${CLOUDRON_OIDC_ISSUER:-}" ]]; then
         yq eval -i ".enable_registration=true" /app/data/configs/homeserver.yaml
-        yq eval -i ".password_config.pepper=\"$(pwgen -1s 12)\"" /app/data/configs/homeserver.yaml
         # just setting enabled to false is not enough. see https://github.com/matrix-org/matrix-synapse-ldap3/issues/123
         yq eval -i "del(.password_providers)" /app/data/configs/homeserver.yaml
     fi
+    yq eval -i ".password_config.pepper=\"$(pwgen -1s 12)\"" /app/data/configs/homeserver.yaml # always set this so that users can enable password login if needed
 fi
 
 echo "==> Ensure we log to console"
@@ -65,15 +65,25 @@ yq eval -i ".email.smtp_user=\"${CLOUDRON_MAIL_SMTP_USERNAME}\"" /app/data/confi
 yq eval -i ".email.smtp_pass=\"${CLOUDRON_MAIL_SMTP_PASSWORD}\"" /app/data/configs/homeserver.yaml
 yq eval -i ".email.notif_from=\"${CLOUDRON_MAIL_FROM_DISPLAY_NAME:-Matrix} <${CLOUDRON_MAIL_FROM}>\"" /app/data/configs/homeserver.yaml
 
-# ldap
-if [[ -n "${CLOUDRON_LDAP_SERVER:-}" ]]; then
-    yq eval -i ".password_providers[0].config.uri=\"${CLOUDRON_LDAP_URL}\"" /app/data/configs/homeserver.yaml
-    yq eval -i ".password_providers[0].config.start_tls=false" /app/data/configs/homeserver.yaml
-    yq eval -i ".password_providers[0].config.base=\"${CLOUDRON_LDAP_USERS_BASE_DN}\"" /app/data/configs/homeserver.yaml
-    yq eval -i ".password_providers[0].config.bind_dn=\"${CLOUDRON_LDAP_BIND_DN}\"" /app/data/configs/homeserver.yaml
-    yq eval -i ".password_providers[0].config.bind_password=\"${CLOUDRON_LDAP_BIND_PASSWORD}\"" /app/data/configs/homeserver.yaml
-    yq eval -i ".password_providers[0].config.filter=\"(objectClass=user)\"" /app/data/configs/homeserver.yaml
+# oidc
+if [[ -n "${CLOUDRON_OIDC_ISSUER:-}" ]]; then
+    yq eval -i "del(.password_providers)" /app/data/configs/homeserver.yaml # remove old ldap config
+    echo " ==> Configuring OIDC auth"
+    yq eval -i ".oidc_providers[0].idp_id=\"cloudron\"" /app/data/configs/homeserver.yaml
+    yq eval -i ".oidc_providers[0].idp_name=\"Cloudron\"" /app/data/configs/homeserver.yaml
+    yq eval -i ".oidc_providers[0].issuer=\"${CLOUDRON_OIDC_ISSUER}\"" /app/data/configs/homeserver.yaml
+    yq eval -i ".oidc_providers[0].client_id=\"${CLOUDRON_OIDC_CLIENT_ID}\"" /app/data/configs/homeserver.yaml
+    yq eval -i ".oidc_providers[0].client_secret=\"${CLOUDRON_OIDC_CLIENT_SECRET}\"" /app/data/configs/homeserver.yaml
 
+    yq eval -i ".oidc_providers[0].scopes=[\"openid\", \"email\", \"profile\"]" /app/data/configs/homeserver.yaml
+    yq eval -i ".oidc_providers[0].authorization_endpoint=\"${CLOUDRON_OIDC_AUTH_ENDPOINT}\"" /app/data/configs/homeserver.yaml
+    yq eval -i ".oidc_providers[0].token_endpoint=\"${CLOUDRON_OIDC_TOKEN_ENDPOINT}\"" /app/data/configs/homeserver.yaml
+    yq eval -i ".oidc_providers[0].userinfo_endpoint=\"${CLOUDRON_OIDC_PROFILE_ENDPOINT}\"" /app/data/configs/homeserver.yaml
+    # https://s3lph.me/ldap-to-oidc-migration-3-matrix.html
+    yq eval -i ".oidc_providers[0].allow_existing_users=true" /app/data/configs/homeserver.yaml
+    yq eval -i ".oidc_providers[0].skip_verification=true" /app/data/configs/homeserver.yaml
+    yq eval -i ".oidc_providers[0].user_mapping_provider.config.localpart_template=\"{{ user.sub }}\"" /app/data/configs/homeserver.yaml
+    yq eval -i ".oidc_providers[0].user_mapping_provider.config.display_name_template=\"{{ user.name }}\"" /app/data/configs/homeserver.yaml
 else
     yq eval -i ".password_config.localdb_enabled=true" /app/data/configs/homeserver.yaml
     # just setting enabled to false is not enough. see https://github.com/matrix-org/matrix-synapse-ldap3/issues/123
@@ -81,10 +91,12 @@ else
 fi
 
 # turn (https://github.com/matrix-org/synapse/blob/master/docs/turn-howto.md#synapse-setup)
+if [[ -n "${CLOUDRON_TURN_SERVER:-}" ]]; then
     yq eval -i ".turn_uris=[]" /app/data/configs/homeserver.yaml
     yq eval -i ".turn_uris[0]=\"turn:${CLOUDRON_TURN_SERVER}:${CLOUDRON_TURN_TLS_PORT}?transport=udp\"" /app/data/configs/homeserver.yaml
     yq eval -i ".turn_uris[1]=\"turn:${CLOUDRON_TURN_SERVER}:${CLOUDRON_TURN_TLS_PORT}?transport=tcp\"" /app/data/configs/homeserver.yaml
     yq eval -i ".turn_shared_secret=\"${CLOUDRON_TURN_SECRET}\"" /app/data/configs/homeserver.yaml
+fi
 
 # fix permissions
 echo "==> Fixing permissions"

test/package.json

@@ -9,11 +9,9 @@
   "author": "",
   "license": "ISC",
   "dependencies": {
-    "chromedriver": "^103.0.0",
+    "chromedriver": "^120.0.0",
     "expect.js": "^0.3.1",
-    "mocha": "^10.0.0",
-    "selenium-server-standalone-jar": "^3.141.59",
-    "selenium-webdriver": "^4.3.1",
-    "superagent": "^8.0.0"
+    "mocha": "^10.2.0",
+    "selenium-webdriver": "^4.16.0"
   }
 }

test/test.js

@@ -1,214 +1,360 @@
 #!/usr/bin/env node
 
 /* jshint esversion: 8 */
-/* global describe */
-/* global before */
-/* global after */
-/* global it */
-/* global xit */
+/* global it:false */
+/* global xit:false */
+/* global describe:false */
+/* global before:false */
+/* global after:false */
 
 'use strict';
 
 require('chromedriver');
 
-var execSync = require('child_process').execSync,
+const execSync = require('child_process').execSync,
     expect = require('expect.js'),
     path = require('path'),
-    superagent = require('superagent'),
     { Builder, By, Key, until } = require('selenium-webdriver'),
     { Options } = require('selenium-webdriver/chrome');
 
+
+if (!process.env.USERNAME || !process.env.PASSWORD) {
+    console.log('USERNAME and PASSWORD env vars need to be set');
+    process.exit(1);
+}
+
 describe('Application life cycle test', function () {
     this.timeout(0);
 
+    const ELEMENT_LOCATION = 'element-test';
     const LOCATION = 'test';
-    const TEST_TIMEOUT = 10000;
+    const TEST_TIMEOUT = parseInt(process.env.TIMEOUT, 10) || 10000;
     const EXEC_ARGS = { cwd: path.resolve(__dirname, '..'), stdio: 'inherit' };
+    const USERNAME = process.env.USERNAME;
+    const PASSWORD = process.env.PASSWORD;
+    const ROOM_ID = Math.floor((Math.random() * 100) + 1);
+    const ROOM_NAME = 'Test room ' + ROOM_ID;
+    const MSG_TEXT = 'Test message ';
 
-    const username = process.env.USERNAME;
-    const password = process.env.PASSWORD;
-
-    var app, browser;
-    var token, roomId;
+    let browser, app;
+    let athenticated_by_oidc = false;
 
     before(function () {
-        if (!process.env.USERNAME) throw new Error('USERNAME env var not set');
-        if (!process.env.PASSWORD) throw new Error('PASSWORD env var not set');
+        const options = new Options().windowSize({ width: 1280, height: 1024 });
+        if (process.env.HEADLESS) options.addArguments('headless');
 
-        browser = new Builder().forBrowser('chrome').setChromeOptions(new Options().windowSize({ width: 1280, height: 1024 })).build();
+        browser = new Builder().forBrowser('chrome').setChromeOptions(options).build();
     });
 
     after(function () {
         browser.quit();
     });
 
+    function sleep(millis) {
+        return new Promise(resolve => setTimeout(resolve, millis));
+    }
+
+    async function waitForElement(elem) {
+        await browser.wait(until.elementLocated(elem), TEST_TIMEOUT);
+        await browser.wait(until.elementIsVisible(browser.findElement(elem)), TEST_TIMEOUT);
+    }
+
     function getAppInfo() {
-        var inspect = JSON.parse(execSync('cloudron inspect'));
+        const inspect = JSON.parse(execSync('cloudron inspect'));
         app = inspect.apps.filter(function (a) { return a.location.indexOf(LOCATION) === 0; })[0];
         expect(app).to.be.an('object');
     }
 
+    function getElementAppInfo() {
+        const inspect = JSON.parse(execSync('cloudron inspect'));
+        app = inspect.apps.filter(function (a) { return a.location.indexOf(ELEMENT_LOCATION) === 0; })[0];
+        expect(app).to.be.an('object');
+    }
+
+    function getMessage() {
+        return MSG_TEXT + Math.floor((Math.random() * 100) + 1);
+    }
+
+    async function updateSynapseConfig() {
+        console.log(`Setting Synapse Matrix server location to "https://${app.fqdn}"`);
+        execSync(`cloudron exec --app ${ELEMENT_LOCATION} -- bash -c "jq '.default_server_config[\\"m.homeserver\\"].base_url = \\"https://${app.fqdn}\\"' /app/data/config.json | sponge /app/data/config.json"`);
+        execSync(`cloudron restart --app ${ELEMENT_LOCATION}`);
+        // wait when all services are up and running
+        await sleep(15000);
+    }
+
     async function checkLandingPage() {
         await browser.get(`https://${app.fqdn}`);
         await browser.wait(until.elementLocated(By.xpath('//h1[contains(text(),"Synapse is running")]')), TEST_TIMEOUT);
     }
 
-    // https://matrix.org/docs/spec/client_server/latest#user-interactive-api-in-the-rest-api
-    function registerUser(done) {
-        superagent.post('https://' + app.fqdn + '/_matrix/client/r0/register?kind=user').send({
-            username: username,
-            password: password,
-            inhibit_login: false
-        }).end(function (error, result) {
-            // we will first get a 401
-            let session = result.body.session;
-            console.log('session is', session);
-            if (result.statusCode !== 401) return done(new Error('Expecting a 401 ' + result.statusCode));
-
-            superagent.post('https://' + app.fqdn + '/_matrix/client/r0/register?kind=user').send({
-                auth: {
-                    type: 'm.login.dummy',
-                    session: session
-                },
-                username: username,
-                password: password,
-                inhibit_login: false
-            }).end(function (error, result) {
-                if (error) return done(error);
-                if (result.statusCode !== 200) return done(new Error('Login failed with status ' + result.statusCode));
-
-                console.log('registered user with id', result.body.user_id);
-
-                done();
-            });
-        });
+    async function registerUser() {
+        await browser.get(`https://${app.fqdn}/#/register`);
+        await waitForElement(By.xpath('//input[@label="Username"]'));
+        await browser.findElement(By.xpath('//input[@label="Username"]')).sendKeys(USERNAME);
+        await browser.sleep(2000);
+        await browser.findElement(By.xpath('//input[@label="Password"]')).sendKeys(PASSWORD);
+        await browser.sleep(2000);
+        await browser.findElement(By.xpath('//input[@label="Confirm password"]')).sendKeys(PASSWORD);
+        await browser.sleep(2000);
+        await browser.findElement(By.xpath('//input[@value="Register"]')).click();
+        await browser.sleep(2000);
+        await waitForElement(By.xpath('//h1[text()="You\'re in"]'));
+        await browser.sleep(2000);
+        await browser.findElement(By.xpath('//div[@role="button" and text()="Skip"]')).click();
+        await browser.sleep(2000);
+        await waitForElement(By.xpath(`//h1[contains(., "Welcome")]`));
     }
 
-    // https://matrix.org/docs/spec/client_server/latest
-    function checkLogin(done) {
-        superagent.post('https://' + app.fqdn + '/_matrix/client/r0/login').send({
-            type: 'm.login.password',
-            user: username,
-            password: password
-        }).end(function (error, result) {
-            if (error) return done(error);
-            if (result.statusCode !== 200) return done(new Error('Login failed with status ' + result.statusCode));
+    async function loginOIDC(username, password) {
+        browser.manage().deleteAllCookies();
+        await browser.get(`https://${app.fqdn}/#/login`);
+        await browser.sleep(6000);
 
-            token = result.body.access_token;
-            if (!token) return done(new Error('No token'));
+        await waitForElement(By.xpath('//div[@role="button" and contains(., "Continue with Cloudron")]'));
+        await browser.findElement(By.xpath('//div[@role="button" and contains(., "Continue with Cloudron")]')).click();
+        await browser.sleep(2000);
 
-            done();
-        });
+        if (!athenticated_by_oidc) {
+            await waitForElement(By.xpath('//input[@name="username"]'));
+            await browser.findElement(By.xpath('//input[@name="username"]')).sendKeys(username);
+            await browser.findElement(By.xpath('//input[@name="password"]')).sendKeys(password);
+            await browser.sleep(2000);
+            await browser.findElement(By.xpath('//button[@type="submit" and contains(text(), "Sign in")]')).click();
+            await browser.sleep(2000);
+
+            athenticated_by_oidc = true;
         }
 
-    function checkAutoJoinRoom(done) {
-        superagent.get('https://' + app.fqdn + '/_matrix/client/r0/joined_rooms?access_token=' + token).end(function (error, result) {
-            if (error) return done(error);
-            if (result.statusCode !== 200) return done(new Error('Room listing failed with status ' + result.statusCode));
+        await waitForElement(By.xpath('//p[@class="confirm-trust" and contains(., "Continuing will grant ")]'));
+        await browser.findElement(By.xpath('//a[contains(., "Continue")]')).click();
+        await browser.sleep(2000);
 
-            if (result.body.joined_rooms.length !== 1) return done(new Error('User must have auto-joined discuss channel:' + result.statusCode));
-            done();
-        });
+        if (await browser.findElements(By.xpath('//div[@aria-label="Skip verification for now"]')).then(found => !!found.length)) {
+            await skipVerification();
         }
 
-    function createRoom(done) {
-        superagent.post('https://' + app.fqdn + '/_matrix/client/r0/createRoom?access_token=' + token).send({
-            room_alias_name: 'general'
-        }).end(function (error, result) {
-            if (error) return done(error);
-            if (result.statusCode !== 200) return done(new Error('Room creation failed with status ' + result.statusCode));
-
-            roomId = result.body.room_id;
-            if (!roomId) return done(new Error('No room id'));
-
-            done();
-        });
+        await browser.sleep(3000);
+        await waitForElement(By.xpath(`//h1[contains(., "Welcome")]`));
     }
 
-    function checkRoom(done) {
-        superagent.get('https://' + app.fqdn + '/_matrix/client/r0/joined_rooms?access_token=' + token).end(function (error, result) {
-            if (error) return done(error);
-            if (result.statusCode !== 200) return done(new Error('Room listing failed with status ' + result.statusCode));
+    async function login() {
+        await browser.get('https://' + app.fqdn + '/#/login');
+        await browser.wait(until.elementLocated(By.xpath('//input[@value="Sign in"]')), TEST_TIMEOUT);
+        await browser.findElement(By.xpath('//input[@name="username"]')).sendKeys(USERNAME);
+        await browser.findElement(By.xpath('//input[@name="password"]')).sendKeys(PASSWORD);
+        await browser.findElement(By.xpath('//input[@value="Sign in"]')).click();
+        await browser.sleep(5000);
 
-            if (!result.body.joined_rooms.includes(roomId)) return done(new Error('No room in list: ' + JSON.stringify(result.body)));
+        if (await browser.findElements(By.xpath('//div[@aria-label="Skip verification for now"]')).then(found => !!found.length)) {
+            await skipVerification();
+        }
 
-            done();
-        });
+        await browser.wait(until.elementLocated(By.xpath('//span[text()="Rooms"]')), TEST_TIMEOUT);
+    }
+
+    async function skipVerification() {
+        await browser.wait(until.elementLocated(By.xpath('//div[@aria-label="Skip verification for now"]')), TEST_TIMEOUT);
+        await browser.sleep(5000);
+        await browser.findElement(By.xpath('//div[@aria-label="Skip verification for now"]')).click();
+        await browser.wait(until.elementLocated(By.xpath('//div[contains(text(), "verify later")]')), TEST_TIMEOUT);
+        await browser.sleep(5000);
+        await browser.findElement(By.xpath('//div[contains(text(), "verify later")]')).click();
+        await browser.sleep(5000);
+    }
+
+    async function logout() {
+        await browser.get('https://' + app.fqdn + '/#/home');
+        await browser.sleep(5000);
+        await waitForElement(By.xpath('//div[@role="button" and @title="User menu"]'));
+
+        await browser.findElement(By.xpath('//div[@role="button" and @title="User menu"]')).click();
+        await browser.sleep(2000);
+
+        await browser.findElement(By.xpath('//li[@role="menuitem" and @aria-label="Sign out"]')).click();
+        await browser.sleep(2000);
+
+        if (await browser.findElements(By.xpath('//button[contains(text(), "I don\'t want my encrypted messages")]')).then(found => !!found.length)) {
+            await browser.findElement(By.xpath('//button[contains(text(), "I don\'t want my encrypted messages")]')).click();
+            await browser.sleep(3000);
+        }
+
+        await waitForElement(By.xpath('//h1[text()="Sign in"]'));
+    }
+
+    async function isLoggedIn() {
+        await browser.get('https://' + app.fqdn + '/#/home');
+        await browser.wait(until.elementLocated(By.xpath('//span[text()="Rooms"]')), TEST_TIMEOUT);
+    }
+
+    async function createRoom() {
+        await browser.get('https://' + app.fqdn + '/#/home');
+        await browser.sleep(4000);
+        await waitForElement(By.xpath('//div[@role="button" and @aria-label="Add room"]'));
+        await browser.findElement(By.xpath('//div[@role="button" and @aria-label="Add room"]')).click();
+        await browser.sleep(2000);
+        await browser.findElement(By.xpath('//li[@role="menuitem" and @aria-label="New room"]')).click();
+        await browser.sleep(2000);
+
+        await browser.findElement(By.xpath('//input[@label="Name"]')).sendKeys(ROOM_NAME);
+        await browser.sleep(2000);
+
+        await browser.findElement(By.xpath('//button[text()="Create room"]')).click();
+        await browser.sleep(2000);
+
+        await waitForElement(By.xpath('//div[@role="button" and @aria-label="Add room"]'));
+
+        await waitForElement(By.xpath('//div[@class="mx_RoomTile_titleContainer"]/div[@title="' + ROOM_NAME + '"]'));
+    }
+
+    async function checkRoom() {
+        await browser.get('https://' + app.fqdn + '/#/home');
+        await browser.sleep(4000);
+        await waitForElement(By.xpath('//div[@role="treeitem" and @aria-label="' + ROOM_NAME + '"]'));
+        await browser.findElement(By.xpath('//div[@role="treeitem" and @aria-label="' + ROOM_NAME + '"]')).click();
+        await browser.sleep(2000);
+        await waitForElement(By.xpath('//h2[text()="' + ROOM_NAME + '"]'));
+    }
+
+    async function sendMessage() {
+        await checkRoom();
+
+        await browser.findElement(By.xpath('//div[contains(@class, "mx_BasicMessageComposer_input")]')).sendKeys(getMessage());
+        await browser.sleep(2000);
+
+        await browser.findElement(By.xpath('//div[@role="button" and @aria-label="Send message"]')).click();
+        await browser.sleep(2000);
     }
 
     xit('build app', function () { execSync('cloudron build', EXEC_ARGS); });
 
     // No SSO
     it('install app (no sso)', function () { execSync('cloudron install --no-sso --location ' + LOCATION, EXEC_ARGS); });
-
     it('can get app information', getAppInfo);
-
     it('check landing page', checkLandingPage);
-    it('can register new user', registerUser);
-    it('can login', checkLogin);
-    it('check autojoin', checkAutoJoinRoom);
-    it('create room', createRoom);
-    it('check room', checkRoom);
 
+    it('can install element-web app', function () { execSync('cloudron install --appstore-id im.riot.cloudronapp --location ' + ELEMENT_LOCATION, EXEC_ARGS); });
+    it('update element-app config', updateSynapseConfig);
+
+    it('can get Element app info', getElementAppInfo);
+    it('can register new user', registerUser);
+    it('create room', createRoom);
+    it('can send message', sendMessage);
+
+    it('can logout', logout);
+
+    it('can login', login);
+    it('check room', checkRoom);
+    it('can logout', logout);
+
+    it('can get app info', getAppInfo);
+
+    it('uninstall element-web app', async function () {
+        await browser.get('about:blank');
+        execSync(`cloudron uninstall --app ${ELEMENT_LOCATION}`, EXEC_ARGS);
+    });
     it('uninstall app', function () { execSync('cloudron uninstall --app ' + app.id, EXEC_ARGS); });
 
     // SSO
     it('install app', function () { execSync('cloudron install --location ' + LOCATION, EXEC_ARGS); });
+    it('can get app info', getAppInfo);
 
-    it('can get app information', getAppInfo);
+    it('can install element-web app', function () { execSync('cloudron install --appstore-id im.riot.cloudronapp --location ' + ELEMENT_LOCATION, EXEC_ARGS); });
+    it('update element-app config', updateSynapseConfig);
 
-    it('check landing page', checkLandingPage);
-    it('can login', checkLogin);
-    it('check autojoin', checkAutoJoinRoom);
+    it('can get Element app info', getElementAppInfo);
+    it('can login via OIDC', loginOIDC.bind(null, USERNAME, PASSWORD));
     it('create room', createRoom);
+    it('can send message', sendMessage);
+    it('can get app info', getAppInfo);
+
+    it('can restart app', function () { execSync(`cloudron restart ${app.id}`); });
+
+    it('backup app', function () { execSync(`cloudron backup create --app ${app.id}`, EXEC_ARGS); });
+
+    it('can get Element app info', getElementAppInfo);
+    it('is logged in', isLoggedIn);
     it('check room', checkRoom);
+    it('can get app info', getAppInfo);
 
-    it('can restart app', function () { execSync('cloudron restart'); });
+    it('restore app', async function () {
+        const backups = JSON.parse(execSync(`cloudron backup list --raw --app ${app.id}`));
 
-    it('check landing page', checkLandingPage);
-    it('check room', checkRoom);
+        await browser.get('about:blank');
+        execSync(`cloudron uninstall --app ${app.id}`, EXEC_ARGS);
+        execSync(`cloudron install --location ${LOCATION}`, EXEC_ARGS);
 
-    it('backup app', function () { execSync('cloudron backup create --app ' + app.id, EXEC_ARGS); });
-
-    it('check landing page', checkLandingPage);
-    it('check room', checkRoom);
-
-    it('restore app', function () {
-        const backups = JSON.parse(execSync('cloudron backup list --raw'));
-        execSync('cloudron uninstall --app ' + app.id, EXEC_ARGS);
-        execSync('cloudron install --location ' + LOCATION, EXEC_ARGS);
         getAppInfo();
+
         execSync(`cloudron restore --backup ${backups[0].id} --app ${app.id}`, EXEC_ARGS);
     });
 
-    it('check landing page', checkLandingPage);
+    it('can get Element app info', getElementAppInfo);
+    it('is logged in', isLoggedIn);
     it('check room', checkRoom);
+    it('can send message', sendMessage);
+    it('can logout', logout);
+    it('can get app info', getAppInfo);
 
-    it('move to different location', function () {
+    it('move to different location', async function () {
         browser.manage().deleteAllCookies();
-        execSync('cloudron configure --location ' + LOCATION + '2', EXEC_ARGS);
+        await browser.get('about:blank');
+
+        execSync(`cloudron configure --location ${LOCATION}2`, EXEC_ARGS);
         getAppInfo();
+        // wait when all services are up and running
+        await sleep(15000);
     });
 
-    it('check landing page', checkLandingPage);
-    it('check room', checkRoom);
+    it('update element-app config', updateSynapseConfig);
 
-    it('uninstall app', function () { execSync('cloudron uninstall --app ' + app.id, EXEC_ARGS); });
+    it('can get Element app info', getElementAppInfo);
+    it('can login via OIDC', loginOIDC.bind(null, USERNAME, PASSWORD));
+    it('check room', checkRoom);
+    it('can send message', sendMessage);
+
+    it('can logout', logout);
+    it('can get app info', getAppInfo);
+
+    it('uninstall app', async function () {
+        await browser.get('about:blank');
+        execSync(`cloudron uninstall --app ${app.id}`, EXEC_ARGS);
+    });
 
     // test update
-    it('can install app', function () { execSync('cloudron install --appstore-id org.matrix.synapse --location ' + LOCATION, EXEC_ARGS); });
+    it('can install app for update', function () { execSync('cloudron install --appstore-id org.matrix.synapse --location ' + LOCATION, EXEC_ARGS); });
+    it('can get app info', getAppInfo);
+    it('update element-app config', updateSynapseConfig);
 
-    it('can get app information', getAppInfo);
+    it('can get Element app info', getElementAppInfo);
 
-    it('check landing page', checkLandingPage);
-    it('can login', checkLogin);
+    it('can login via OIDC', loginOIDC.bind(null, USERNAME, PASSWORD));
+    it('is logged in', isLoggedIn);
     it('create room', createRoom);
-    it('check room', checkRoom);
+    it('can send message', sendMessage);
+    it('can logout', logout);
+    it('can get app info', getAppInfo);
 
-    it('can update', function () { execSync('cloudron update --app ' + LOCATION, EXEC_ARGS); });
-
-    it('check landing page', checkLandingPage);
-    it('check room', checkRoom);
-
-    it('uninstall app', function () { execSync('cloudron uninstall --app ' + app.id, EXEC_ARGS); });
+    it('can update', async function () {
+        await browser.get('about:blank');
+        execSync(`cloudron update --app ${app.id}`, EXEC_ARGS);
+        // wait when all services are up and running
+        await sleep(15000);
+    });
+
+    it('can get Element app info', getElementAppInfo);
+    it('can login via OIDC', loginOIDC.bind(null, USERNAME, PASSWORD));
+    it('is logged in', isLoggedIn);
+    it('check room', checkRoom);
+    it('can send message', sendMessage);
+    it('can get app info', getAppInfo);
+
+    it('uninstall app', async function () {
+        await browser.get('about:blank');
+        execSync(`cloudron uninstall --app ${app.id}`, EXEC_ARGS);
+    });
+
+    it('uninstall element-web app', function () {
+        execSync(`cloudron uninstall --app ${ELEMENT_LOCATION}`, EXEC_ARGS);
+    });
 });