We published Synapse version 1.75 as the new stable release this week. Synapse admins are encouraged to upgrade to it at their convenience. It seems like the blog post for version 1.74 was eaten by Santa's reindeer, so this post will also cover changes from it.

Announcements

There were no special announcements for the 1.75 release.

Synapse 1.74: ICU dependency for improved user search

Synapse's search functionality has long been poor when searching for non-English terms. Synapse 1.74 introduced support for an optional improved user search experience which has a better awareness of Unicode. To do so, we use the ICU library when Synapse is installed alongside the binding library PyICU.

Synapse installations using Matrix.org docker images or debian packages will automatically have the new search mode enabled.

From-source installations will need to include the user-search extra when pip install Synapse, e.g. pip install matrix-synapse[user-search]. NB: because PyICU is not distributed as source-only on PyPI, you will need to ensure the ICU development headers are available on your system. See the upgrade notes for more info.

Please try out the new search mode and let us know how you find it in practice.

The new stuff

Support RFC7636 Proof Key for Code Exchange for OAuth single sign-on

Synapse 1.75 adds support for RFC7636 code challenges in the OAuth 2.0 flow.

This is required by Twitter SSO and enabling it can protect against the "authorization code interception attack".

Experimental support for removing account data (MSC3391)

Experimental support for removing account data has landed in Synapse. It was previously possible to create or update account data but not remove them, this is now possible.

The support is experimental and needs to be enabled with a configuration flag since the MSC hasn't landed yet.

More faster remote room joins fixes and improvements

In case you missed it we are working on dramatically improve performance of remote room joins, you can refer to this blog post for more details.

The last two Synapse releases brings that a lot closer to a proper release; in particular they contain a lot of work to support faster joins in deployments with multiple workers. The project continues in earnest; we hope to have more to show off in the coming weeks.

Everything else

See the full changelog, for a complete list of changes in the release. Also please have a look at the upgrade notes for 1.74 version.

Synapse is a Free and Open Source Software project, and we'd like to extend our thanks to everyone who contributed to these releases, including (in no particular order): Ashish Kumar, Dirk Klimpel, Jeremy Kescher, Jeyachandran Rathnam, Nick Mills-Barrett, Jason Little, Villepeh and Vertux. We are also grateful to anyone helping us make Synapse better by sharing their feedback and reporting issues, or helping with community support questions.

And here is another update to your beloved Matrix homeserver implementation, Synapse 1.73.

Announcements

Legacy Prometheus metric names removed

When releasing Synapse 1.69 a couple of months ago, we also announced the removal of old Prometheus metrics that have been replaced by more aptly named ones. he list of these metrics can be found here.

Synapse 1.73 implements the final phase of this plan and entirely removes support for those metrics. As a result, the enable_legacy_metrics configuration option, which was introduced in Synapse 1.71, has also been removed.

Server administrators who are still relying on these legacy metric names are encouraged to update their dashboards at their earliest convenience. For more information, please refer to the upgrade notes.

The new stuff

Performance

A bunch of performance improvements have been included in this release, specifically around the /messages endpoint.

Improvements to event filtering on the client-server API gave the matrix.org homeserver a first nice bump as visible on this graph:

Various optimizations around fetching bundled aggregations resulted in yet another nice improvement:

Note that the graph from the first image, and the second graph from the second image are apdexes, which is a measure that shows improvement when it goes up (as opposed to e.g. response times, which improve when they go down).

Extensible Events experimental support

Experimental support for Extensible Events has landed in Synapse.

This is exciting since this global rework of events presentation has been in talks for a while, and having an implementation to experiment with greatly helps bringing the feature closer to completion.

Note that this support is still very much experimental as the related MSCs are still under review and could change at any time, and therefore not recommended for use in production.

Everything else

See the full changelog, for a complete list of changes in the release. Also please have a look at the upgrade notes for this version.

Synapse is a Free and Open Source Software project, and we'd like to extend our thanks to everyone who contributed to this release, including (in no particular order) schmop, Ashish Kumar, realtyem, and Brennan Chapman as well as anyone helping us make Synapse better by sharing their feedback and reporting issues.

It's not Christmas yet, but it's time for a new release! Say hello to Synapse 1.72.

It seems like this blog didn't self update through AI for the 2 preceding updates so this post will cover 1.70 to 1.72, sorry if this is a bit long but it's been a while :)

Announcements

Dropping support for PostgreSQL 10

PostgreSQL 10 has reached End Of Life: Synapse will no longer support it beginning with version 1.72 so please upgrade your database if you have not already done so.

Legacy Prometheus metric names disabled and to be removed

In previous versions of Synapse, some Prometheus metrics were emitted under two different names, an older name that was non-compliant with OpenMetrics and Prometheus conventions and a new compliant name.

Synapse 1.71 and later have the old metric names switched off by default.

For now it's still possible to get them by using enable_legacy_metrics: true, however server administrators still using legacy metric names are highly encouraged to migrate, as they will be removed in 1.73.

You can find the full list of renamed metrics here.

Changes to the events received by application services (interest)

Following the implementation of MSC3905, Synapse now only considers local users to be interesting to application services. In other words, the users namespace regex in an app service's registration file is only applied against local users of the homeserver.

Please note, this probably doesn't affect the expected behavior of your application service, since an interesting local user in a room still means all messages in the room (from local or remote users) will still be considered interesting.

You can find a bit more info in the MSC and in the upgrade notes.

The new stuff

Threads, threads, threads!

Several MSCs related to threads got implemented:

• MSC3856 provide an API to fetch threads and related metadata.
• MSC3771 and MSC3773 implementing per thread read receipts and per thread notification counts.
• MSC3874 allows to filter out messages belonging to threads from the main timeline (still considered experimental). Along with MSC3856, this should noticeably improve performance of rooms that use threads heavily.

This should significantly improve user experience related to threads, being through behavior or performance impact.

Linking events together

Relationships are great, even more between events than humans!

MSC3664 allows Matrix clients to be notified in real time of related events, so you can now be made quickly aware of this cat emoji reaction that your cat photo clearly deserved.

Additionally, Synapse 1.72 includes an implementation of MSC3912, allowing users to redact the relations of a message alongside the message itself. This is particularly helpful in cases like edits, where users usually want to see their edits redacted at the same time as the original message. Note that this implementation is currently incomplete and still experimental, though, so watch this space!

Faster joins, continued

We continue our journey to get everything going as transparently as possible when doing fast remote room joins.

If you missed it you can refer to this previous blog post to get a lot more infos, and feel free to grep Synapse changelog and the numerous related issues/PRs for all the gory details.

Everything else

See the full changelogs (1.70, 1.71, 1.72) for a complete list of changes in the releases. Also please have a look at the upgrade notes for this version.

Synapse is a Free and Open Source Software project, and we'd like to extend our thanks to everyone who contributed to this release, including (in no particular order) Nico, sando38, realtyem, aceArt GmbH, Tuomas Ojamies, Ashish Kumar, asymmetric, Beeper, Ryan Miguel, Paul Tötterman, Abdullah Osama, Finn, Ivan Shapovalov, Dirk Klimpel, Jonathan de Jong, MichaIng and Aaron Raimist as well as anyone helping us make Synapse better by sharing their feedback and reporting issues.

Hey all,

We’ve just released Matrix 1.5, a largely maintenance update for the spec. We intentionally haven’t landed any major features in this release as Matrix 1.4, just shy of 2 months ago, had introduced fairly large features for clients and servers to consider. As with all spec releases, we encourage implementations to gradually update over the next few months rather than expect them to have support for everything on release day.

Matrix 1.5 sees just 2 MSCs get merged, though this is to be expected from a maintenance release. We expect that the next release (in Q1 2023) will have a few more exciting features to it :)

We’ve covered both MSCs below, but read on to the full changelog for the full picture.

MSC3267: Reference relations

Already supported implicitly by the spec up until now, reference relations are a way to simply reference another event. Usually these sorts of relations are used for events which need to be related to each other, but a dedicated relationship type doesn’t make a lot of sense.

In-room verification and MSC3381: Polls are examples of how these relations get used.

MSC3905: Clarify appservice interest in user IDs

MSC3905 fixes an issue in the specification where appservices (usually bridges) specifying a users regex without homeserver domain would end up receiving far more event traffic than they would have intended. With the MSC, appservices are now only considered interested in “local” users, regardless of how vague their users namespace is.

Overall this should have no effect on most bridges/appservices, however if an appservice in the wild really does need to listen to all users on all homeservers, it can specify a non-exclusive namespace on all rooms instead.

While writing this MSC into the spec we took some time to clarify the appservice registration requirements more generally: check them out here.

The full changelog

MSCs are how the spec changes in the way it does - adding, fixing, and maintaining features for the whole ecosystem to use. Check out the full changelog below, and the Spec Change Proposals page for more information on how these MSCs got merged (hint: they submitted a proposal, which anyone can do - take a look at the Matrix Live episode where Matthew covers the proposal process).

Client-Server API

• Add m.reference relations, as per MSC3267. (#1206)
• Add missing documentation for m.key.verification.request msgtype for in-room verification. (#1271)

• Fix various typos throughout the specification. (#1260, #1265, #1276)
• Fix naming of device_one_time_keys_count in /sync. (#1266)
• Improve display of event subtypes. (#1283)
• Improve documentation about ephemeral events. (#1284)
• Define a 400 response from /_matrix/client/v3/directory/rooms/{roomAlias}. (#1286)
• Clarify parts of the end-to-end encryption sections. (#1294, #1345)
• Various clarifications throughout the specification. (#1306)
• Replace set_sound push rule action by set_tweak. (#1318)
• Clarify the behavior of PUT /_matrix/client/v3/pushrules/{scope}/{kind}/{ruleId}. (#1319)
• Clarify that .m.rule.master has a higher priority than any push rule. (#1320)
• Require request field refresh_token at endpoint POST /_matrix/client/v3/refresh. (#1323)
• Fix a number of broken links in the specification. (#1330)
• Add example read receipt to GET /_matrix/client/v3/sync response example. (#1341)

Server-Server API

• Fix a number of broken links in the specification. (#1330)

Application Service API

• Clarify that application services can only register an interest in local users, as per MSC3905. (#1305)

Identity Service API

• Fix a number of broken links in the specification. (#1330)

Push Gateway API

No significant changes.

Room Versions

• Reword the event auth rules to clarify that users cannot demote other users with the same power level. (#1269)
• Various clarifications to the text on event authorisation rules. (#1270)
• Fix a number of broken links in the specification. (#1330)

Appendices

No significant changes.

Internal Changes/Tooling

• Update docsy theme to v0.5.0 + matrix.org modifications (https://github.com/matrix-org/docsy/commit/a0032f8db919a6c67ba6cdef2c455f105b6272a2). (#1295)

• Improve error messages emitted by resolve-additional-types template. (#1303)
• Fix link to API viewer. (#1308)
• Stop rendering the subsections of the Client-Server API and Room Versions specs as their own separate pages. (#1317)
• Use a link checker to ensure that we do not have broken links. (#1329, #1338)
• Update instructions to preview Swagger definitions. (#1331)
• Make definition anchors more unique. (#1339)
• Generate the unstable changelogs with towncrier, for consistency. (#1340)
• Update CONTRIBUTING.md to mention that non-content changes to this repo should have an "internal" changelog entry. (#1342)
• Update module summary table with new modules: Event Replacements, Threading and Reference Relations. (#1344)
• Disable RSS generation for the spec. (#1346)

Hey everyone, it's time for a new Synapse release! Synapse 1.69 is out, fresh out of the oven. But before we take a look at it, here's a quick announcement:

We have recently disclosed a moderate severity security vulnerability, which was fixed in Synapse 1.62 (released on July 5th 2022). This issue affects all homeservers running a version of Synapse older than 1.62 with open federation. If this is the case for your deployment, please update to a more recent version of Synapse at your earliest convenience.

See advisory GHSA-jhjh-776m-4765 and CVE-2022-31152 for more information.

Now let's see what's new in Synapse 1.69!

Caching in modules

Synapse modules allow third-party developers to extend Synapse's feature range to include very custom features.

Synapse 1.69 includes tools modules can use to interact with Synapse's caching system, which includes features such as synchronising caches across multiple Synapse workers.

Starting with this version, modules can import the @cache() decorator from synapse.module_api, to decorate functions which results must be cached. Cached functions need to be registered with ModuleApi.register_cached_function at the module's initialisation. Invalidating a cached function's result for a given set of arguments can be done using ModuleApi.invalidate_cache.

Upcoming removal of deprecated Prometheus metric names

Back in Synapse 1.2, we deprecated a number of Prometheus metrics, to replace them with more aptly named equivalents that fit existing naming conventions better.

With Synapse 1.69, we're announcing the plan for the removal for these deprecated metrics. Synapse 1.71 (due to be released on November 8) will turn these metrics off by default (though it will still be possible to enable them back). Synapse 1.73 (due to be released on November 22) will remove them altogether.

Server administrators using metrics dashboards that rely on the deprecated metrics are encouraged to update their dashboards with the new metric names. See the upgrade notes for more information.

Everything else

Synapse 1.69 also deprecates the generate_short_term_login_token module API method in favour of the asynchronous create_login_token one. The deprecated method will be removed in Synapse 1.71. See the upgrade notes for more information.

Additionally, Synapse 1.69 introduces a forwards- and backwards-incompatible change to the replication of read receipts. This means that read receipts replicated by workers running Synapse 1.68 will not be able to be processed by workers running Synapse 1.69 and vice-versa. Once all workers are made to run Synapse 1.69 (or 1.68), replication of read receipts will resume as normal. See the upgrade notes for more information.

See the full changelog for a complete list of changes in this release. Also please have a look at the upgrade notes for this version.

Synapse is a Free and Open Source Software project, and we'd like to extend our thanks to everyone who contributed to this release, including (in no particular order) lukasdenk, Dirk Klimpel, Ashish Kumar, Famedly and Beeper, as well as anyone helping us make Synapse better by sharing their feedback and reporting issues.

Hey all,

It’s finally here: threads, edits, and private read receipts. v1.4 has been a little later than usual in the quarter because we wanted to make sure we nailed down all the core MSCs for threads before publishing the spec itself, but we’ve done that now and we’re excited about it.

Matrix 1.3 was just over 3 months ago, though with the relatively large features there and the colossal implementation effort of v1.4 we’re not expecting everyone to have implementations on day 1. Instead, as with all spec releases, we encourage implementations to gradually update over the next few months. We’re additionally planning to make v1.5 (Q4 2022) more of a maintenance update to help make the backlog a bit easier on everyone, though we might prioritize a couple cool features in there too :)

Matrix 1.4 sees 14 MSCs get merged, but we can’t possibly go into detail on them all here. We’ve instead focused on the 3 major features we’re excited about - check out the changelog at the end for the full picture.

🧵 Threads

Threads, a critical feature in terms of parity with other chat systems, have landed thanks to a whopping 6 MSCs: MSC3440, MSC3816, MSC3856, MSC3715, MSC3771, and MSC3773. It’s been a lot of iteration on the reference implementations to get threads this far, and we’re excited to see how the client implementations evolve to provide more structured and less noisy communication for everyone - keep us updated with TWIM posts, please!

Threads have involved changes to event relationships (which were fixed in v1.3), read receipts, and notification counts, resulting in several different models and ways of solving the problem. We think we’ve reached a point that works for conversational threads, though there’s work on the horizon for threads-only rooms, Twitter-style free-form threads, etc to cover more needs of users.

Currently, the demonstration implementation of threads in Synapse and Element is working its way out of the proof-of-concept and beta stages (which were sufficient to validate the MSCs) and are on their way to exiting beta. Keep an eye on the respective blogs for news about production-ready threads!

📝 Edits

Similar to reactions, err, aggregations, MSC2676 and its predecessors have been around for a long while. Edits have existed in the Element clients seemingly forever, and other clients have had troubles trying to implement the same feature: with it now in the spec, it should be a lot easier to bring edits into clients.

For edits, we’ve taken the route of making the MSC match reality, for better or worse. MSCs which improve or add functionality to the system are very much welcome - writing an MSC is relatively easy and helps the whole community.

👥 Private read receipts

Not everyone wants to broadcast that they’ve read a message, but also with read receipts tied into notifications those same people also probably don’t want stuck notifications. To address the problem, we’ve added a new m.read.private receipt type which behaves exactly like m.read, but is only visible to you.

We still have a rework of the notifications system in our long-term plan, but this should hopefully cover the privacy concerns for the time being :)

The full changelog

MSCs are how the spec changes in the way it does - adding, fixing, and maintaining features for the whole ecosystem to use. The blog post can’t cover them all, but that doesn’t make them any less important! Check out the full changelog below, and the Spec Change Proposals page for more information on how these MSCs got merged (hint: they submitted a proposal, which anyone can do - take a look at the Matrix Live episode where Matthew covers the proposal process).

Client-Server API

• Remove unused policy room sharing mechanism, as per MSC3844. (#1196)

• Add a .m.rule.room.server_acl push rule to match m.room.server_acl events, as per MSC3786. (#1190, #1201)
• Add Cross-Origin-Resource-Policy (CORP) headers to media repository, as per MSC3828. (#1197)
• Copy a room's type when upgrading it, as per MSC3818. (#1198)
• Add room_types filter and room_type response to /publicRooms, as per MSC3827. (#1199)
• Add m.replace relations (event edits), as per MSC2676. (#1211)
• Add m.read.private receipts, as per MSC2285. (#1216)
• Make m.fully_read optional on /read_markers, as per MSC2285. (#1216)
• Allow m.fully_read markers to be set from /receipts, as per MSC2285. (#1216)
• Add threading via m.thread relations, as per MSC3440, MSC3816, MSC3856, and MSC3715. (#1254)
• Add per-thread notifications and read receipts, as per MSC3771 and MSC3773. (#1255)

• Mention that the /rooms/{roomId}/invite endpoint will return a 200 response if the user is already invited to the room. (#1084)
• Fix various typos throughout the specification. (#1135, #1161, #1164, #1170, #1180, #1215, #1238, #1243)
• Describe return codes for account data endpoints, and clarify that per-room data does not inherit from the global data. (#1155)
• Clarify that policy rule globs work like ACL globs. Contributed by Nico. (#1165)
• Clarify the format of some structures in the End-to-end encryption module. (#1166)
• Add HTML anchors for object definitions in the formatted specification. (#1174)
• Tweak the styling of <code> snippets in tables rendered from OpenAPI definitions. (#1179)
• Update "API Standards" section to clarify how JSON is used. (#1185)
• Clarify that the "device_id", "user_id" and "access_token" fields are required in the response body of POST /_matrix/client/v3/login. (#1210)
• Reinforce the relationship of refreshed access tokens to transaction IDs. (#1236)
• Clarify enum values by separating possible values with commas. (#1240)

Server-Server API

• Add per-thread notifications and read receipts, as per MSC3771 and MSC3773. (#1255)

• Add HTML anchors for object definitions in the formatted specification. (#1174)
• Tweak the styling of <code> snippets in tables rendered from OpenAPI definitions. (#1179)
• Update "API Standards" section to clarify how JSON is used. (#1185)

Application Service API

• Replace homeserver authorization approach with an Authorization header instead of access_token when talking to the application service, as per MSC2832. (#1200)

• Add HTML anchors for object definitions in the formatted specification. (#1174)

Identity Service API

• Add HTML anchors for object definitions in the formatted specification. (#1174)
• Update "API Standards" section to clarify how JSON is used. (#1185)

Push Gateway API

• Add HTML anchors for object definitions in the formatted specification. (#1174)

Room Versions

• For room versions 1 through 10, clarify that events with rejected auth_events must be rejected. (#1137)
• For room versions 2–10: correct a mistaken clarification to the state resolution algorithm. (#1158)
• For room versions 7 through 10: Clarify that invite->knock is actually a legal transition. (#1175)

Appendices

No significant changes.

Internal Changes/Tooling

• Add internal changes changelog section. (#1194)

• Render HTML anchors for object definition tables. (#1191)
• Give rendered-data sections a background and some padding. (#1195)
• Fix rendering of shortcodes within the client-server API. (#1205)
• Fix the spacing of mapping types generated from the OpenAPI spec. (#1230)