The events of the last week have been utterly terrifying as we’ve seen a highly sophisticated targeted attack on open source infrastructure play out in public, in the form of the liblzma backdoor. Matrix is not impacted by the attack (none of our code or infrastructure is using liblzma or xz 5.6), but it has been a massive wakeup call in terms of understanding the risks posed by overstretched open source maintainership.
2023 has been a pivotal year for Matrix, with huge changes landing both organisationally and technically to prepare the protocol for future generations. The ecosystem has once again gone from strength to strength, with active users (based on Synapse opt-in phone-home reporting) doubling across the public network, and more projects building on Matrix than we can count (look out for the “This Year in Matrix” community wrap-up blog post) - and more organisations than we can track joining Matrix for all their secure decentralised communication needs.
On the governance side, we are in an incredibly exciting new era with Josh joining the Matrix.org Foundation as its first ever Managing Director (and employee!), with a mandate to cement sustainable funding for Matrix as an independent foundation, governed by the forthcoming elected open Governance Board. Now, Matrix needs funding more than ever - but rather than turning the entirety of this post into a plea for donations, I’m going to let Josh fly the flag in the coming weeks. Meanwhile, if you want Matrix to keep existing (especially if you’re an organisation who builds on Matrix) please join the Foundation and donate.
On the technical side: the theme of the year has been one of focus: extreme, overdue, focus.
Over the years, it’s fair to say that the core team has tried to strike a balance between building the core foundational technology of Matrix (the spec, a stable server implementation, client SDKs, end-to-end encryption, VoIP, etc)... and long-term forward-looking projects designed to futureproof Matrix (e.g. Account Portability, P2P Matrix, Dendrite, Hydrogen) and/or inspire developers to build on Matrix for more than just chat (e.g. Third Room, Applications Beyond Chat). In retrospect, this was wildly optimistic: we underestimated the amount of remaining work needed to polish the foundational tech to mainstream quality - and despite Matrix uptake going through the roof, this hasn’t translated into sufficient funding to have the luxury to support folks to proactively work on next-gen projects (or foundational projects, for that matter).
So, this year, we’ve ended up focused on one thing: getting the foundational Matrix featureset to better-than-mainstream quality, performance and stability. We’ve dubbed the overall initiative Matrix 2.0, and kicked it off at FOSDEM 2023 with our Matrix 2.0: How we’re making Matrix go vooooom main-stage talk.
The Road to Matrix 2.0
Matrix 2.0 isn’t (yet) an actual versioned release of the Matrix specification - instead, it describes the various foundational projects needed to get quality, performance and stability up to and beyond that of today’s mainstream messaging apps. These projects are:
Sliding Sync (MSC3575): the ability to instantly log in, launch and sync Matrix clients no matter how large or busy the account.
Native E2EE Group VoIP (MSC3898 + TBA): scalable video and voice conferencing and calling built natively on Matrix and so benefiting from Matrix’s end-to-end encryption.
Native OIDC (MSC3861): replacing Matrix’s historical authentication mechanisms with industry-standard Open ID Connect (giving us two factor authentication, multi-factor auth, passkeys, and radically simplifying auth implementations for both client and server developers).
Faster Remote Room Joins (MSC3902): letting servers rapidly join rooms on other servers by incrementally participating in the room.
Over the course of the year Matrix 2.0 has gone from the initial demo on stage at FOSDEM to concrete implementations which users can play with today as announced in our Matrix 2.0: The Future of Matrix post in September. Since then, we’ve been busy polishing away. On Sliding Sync, the proxy has pretty much stabilised - although the protocol itself can and should be simplified before we think seriously about native implementations (in practice, having the server track room list ordering gets very fiddly when only clients can really determine the final ordering, due to E2EE). Element X and matrix-rust-sdk has been the main implementation driving forwards Sliding Sync and much of the other Matrix 2.0 work, for those itching to play with it.
On Native Group VoIP: we’ve gone through many iterations over the year - starting off with Full Mesh calling (good for ~7 users per call); then switching to the experimental waterfall Selective Forwarding Unit (SFU) to provide scalable but not-E2EE conferencing; and then switching to a hybrid solution using LiveKit to provide an E2EE-capable scalable SFU, but with the signalling and encryption all handled by Matrix. Element Call is the main implementation driving forwards the underlying Matrix work here, and Element Call Beta 3 showed off the new LiveKit based implementation in July - which was then integrated with Element X complete with end-to-end encryption in November. There’s still some polishing remaining here, with a new layout engine in the wings for Element Call, and enabling full encrypted-per-sender conferencing by default in both Element Web and Element X, but it really feels like the hardest work is behind us now: the core team has been successfully doing all of its collaboration on Element Call for months now, like so:
On Open ID Connect: things are also shaping up well. This will be the first time that we’ve replaced a large chunk of the Matrix spec with something else, and in order to manage your account in Matrix 2.0-native clients like Element X homeserver admins will need to migrate their authentication to the new OIDC World using matrix-authentication-service (MAS). There’s a great blog post from September which explains what this will entail - and since then, we even have the beginnings of syn2mas: a migration script to migrate from Synapse-managed accounts to MAS-managed accounts (warning: still experimental). The Matrix.org homeserver hasn’t been migrated yet (as we need to support social login first), but an increasing number of standalone Matrix servers are going OIDC-native, so arguably the migration has already begun! We’ll keep https://areweoidcyet.com updated as the project progresses.
Finally, the core of Faster Remote Room Joins (FRRJ) shipped in Synapse back in February. There’s still some major speedups that FRRJ could unlock, but the other tracks of Matrix 2.0 have been taking priority.
So: Matrix 2.0 is palpably on the horizon - all that remains is polish on the example clients (Element X & Element Call), full support for migrating to OIDC, and landing the MSCs into the spec. For instance, Element X just added read receipts and (early) E2EE backup support in the last few days - the gap is closing! It’s worth noting that significant amounts of this work has been funded by BWI for BwMessenger and BundesMessenger: huge thanks to BWI for supporting core Matrix development by contracting Element.
Levelling up on Encryption
Encryption stability received a huge amount of attention this year. It turns out that reliable end-to-end encryption is surprisingly tricky in a decentralised environment, and historically we’ve been playing on hard mode by implementing three entirely separate implementations of the Matrix layer of encryption between matrix-js-sdk, matrix-ios-sdk and matrix-android-sdk2, each with their own bugs - more than tripling the costs of development, audits, and maintenance by the Foundation (quite ignoring the independent implementations from the community in mtxclient, libquotient, matrix-dart-sdk, trixnity etc).
So a huge project has been underway to converge on a single auditable codebase for the core team’s E2EE implementation so that any bugs or future features can be resolved in a single place. That codebase is matrix-rust-sdk’s matrix-sdk-crypto crate (and our underlying vodozemac double ratchet implementation) - and we’re proud to say that we are using it for encryption in matrix-rust-sdk itself (as showcased by Element X and Fractal 5), matrix-ios-sdk and matrix-android-sdk2 (as used in the old Element iOS & Android apps), and have now merged it in matrix-js-sdk too (available for new logins on develop.element.io). The process of rustifying the encryption in Element Web and the old iOS & Android apps has been nicknamed “Element R”.
The process of switching matrix-js-sdk to use Rust encryption has been particularly gruelling, requiring compiling matrix-sdk-crypto down to WASM as matrix-rust-sdk-crypto-wasm and then doing heart surgery to replace the old JS crypto implementation… while also needing to extensively loop from WASM back into the browser to use IndexedDB for storage, all while outperforming the old implementation. It’s tantalisingly close now: while develop.element.io has it turned on by default for new logins, there are still a few remaining performance edge cases to be chased down related to online backup before we migrate everyone to it. The remaining blocking issues can be found on GitHub for those interested in tracking progress.
matrix-crypto-sdk is already manifestly more reliable than the old implementations (in terms of the chances of hitting infamous Unable To Decrypt errors) - and now that we are so close to converging on it everywhere, the race is on to ensure that any remaining defects get flushed out for once and for all. One of the new initiatives here is called complement-crypto - a full end-to-end torture testing suite specifically for matrix-crypto-sdk. You can read all about it in the announcement post a few weeks ago, but suffice it to say it’s a super exciting project which stress-tests both matrix-rust-sdk and matrix-js-sdk (with the new rust crypto implementation) against federated Synapse containers in order to test E2EE under the most horrible failure modes imaginable. It’s already picked up some elusive bugs which have plagued us for literally years, and it looks set to be the main framework by which we will hunt down and kill any remaining issues. See the Test hitlist for the full scope we’re targeting.
Now that everyone’s (almost) converged on matrix-sdk-crypto, the next big project for the Crypto Team is going to be improving the E2EE usability (at last!). The big news here is that we’re shifting to Trust On First Use (TOFU) for user trust. Specifically: this means that we will only encrypt messages to devices whose owner has explicitly cross-signed them (essentially trusting the owner by default). You will still be able to explicitly verify that other users are not being impersonated (via QR scan or emoji comparison), but this should improve the default behaviour to be much more secure. Alongside TOFU will come other radical simplifications of the E2EE UX (both around login, self-verification, cross-verification and backup), so watch this space: the game is afoot to finally fix Matrix’s E2EE usability, now we can make all the changes in one place!
Finally, work continues to progress at matrix-dmls on supporting a decentralised dialect of Messaging Layer Security (MLS, RFC9420) on top of Matrix as an alternative to our normal Olm/Megolm encryption, with recent work focused on making it play nice with matrix-sdk-crypto. https://arewemlsyet.com is the place to track updates (although it’s a bit overdue for an update).
In other news
Faced with limited funding and the decision to focus exclusively on stability, reliability and performance, there have inevitably been some major changes impacting the core team.
One of the biggest changes is that Element (the company formed by the core Matrix team back in 2017 to try to fund our work on Matrix) can no longer financially afford to donate its work on Synapse and other server components to the Matrix Foundation under the permissive Apache licence. Instead, Element is continuing development under the copyleft AGPLv3 licence at github.com/element-hq/synapse going forwards. This is to let Element sell AGPL exceptions to commercial Matrix vendors in order to fund their underlying Matrix development: you can read more about it at Element’s announcement - or you can listen to this week’s Matrix Live for a firsthand explanation:
The other major change is that we’ve had no choice but pause development on the majority of the core team’s next-generation Matrix projects. We had high hopes of being able to secure dedicated funding for Third Room (especially after the awesome Tech Preview 2: Creator Update in June), but the interested parties did not come through, and the team has now disbanded. Meanwhile, P2P Matrix and Low Bandwidth Matrix is on hiatus until there’s dedicated funding - and Account Portability work is also temporarily paused in favour of commercial Element work, despite the fantastic progress made recently with Pseudo IDs (MSC4014) and Cryptographic identifiers (MSC4080). Given P2P Matrix and Account Portability were the main projects driving Dendrite development recently, this may also cause a slow-down in Dendrite development, although Dendrite itself will still be maintained.
Needless to say, this is far from an ideal situation: we sent up distress flares loud and clear at the beginning of last year’s holiday update; and we’ve now had to shrink to focus exclusively on the core projects. However, we’re optimistic that the tighter focus in the medium term will help us get back to the point where we can resume the longer-term projects - assuming that organisations (and individuals) dependent on Matrix sign up to support the project.
Conclusion
Despite the downsides of 2023, right now we’re feeling distinctly optimistic: Matrix 2.0 clients like Element X already outperform the best proprietary mainstream options by many metrics - and focusing purely on improving the foundations is only going to improve that. We may not have taken the most direct route to get to where we are today, but it genuinely feels like 2024 will be the year where Matrix overtakes the incumbents.
Talking of which, there’s just one last thing to mention: amidst the economic challenges of 2023, one future-facing core team Matrix project has survived: our work around the EU Digital Markets Act (DMA). The DMA is leading antitrust legislation from the European Commission, which aims to stop big centralised tech companies (so called ‘gatekeepers’) from carelessly suppressing innovation, competition and consumer choice by trapping users inside their walled gardens.
Needless to say, we’ve been tracking the DMA closely throughout its gestation, and we’re now in the final sprint: in March 2024, messaging services which have been identified as gatekeepers will have to open their networks to allow interoperability with requesting messaging services (while preserving E2EE, if they’re encrypted). So far, only WhatsApp and Facebook Messenger have been identified as gatekeeper services (Apple is doing everything they can to wriggle out of it). However, it looks like WhatsApp is taking it seriously, which could prove very interesting indeed.
As Matrix, we’ve taken a two-pronged approach: on one side, showcasing how Matrix as it stands today can already bridge existing chat systems together as a highest-common-denominator protocol (including preserving E2EE, if they happen to already use the Double Ratchet). On the other side, we’ve also been contributing significantly to MIMI, the IETF Working Group dedicated to standardising a lowest-common-denominator protocol specifically for DMA interoperability.
2023 has involved a lot of work on MIMI, participating in the Design Team alongside Phoenix, Wire, Cisco, Google and Wickr - and at IETF 118 in Prague in November we collectively proposed the first draft of the protocol (see also the architecture presentation for context). MIMI has ended up taking some inspiration from Matrix (and Linearized Matrix, a simplified dialect we proposed which does away with full-mesh decentralisation), while focusing very tightly on the specific problems of server-to-server interop between existing communication services - leaning on MLS (where available) for synchronising state across the services, while also leaving the door open for using the Double Ratchet to provide an on-ramp for pragmatic bridging to today’s services (including Matrix!).
MIMI’s tight focus means that it doesn’t currently provide conversation history, arbitrary state events, or decentralised conversations - and is focused purely on server-server communication. However, there’s certainly a world where Matrix could evolve to be compatible with MIMI - such a hypothetical Matrix 3.0 would effectively layer Matrix’s richer existing semantics (decentralised conversations, extensible state events, state resolution, group VoIP etc) on top of MIMI’s subset of functionality. It’ll be interesting to see how this plays out. Meanwhile, we’ll continue to provide both Matrix for pragmatic DMA interoperability for today - and participate in MIMI for IETF-track interoperability.
Anyhow: hopefully it’s clear that 2024 is going to be a super interesting year for Matrix - whether that’s simply by nailing Matrix 2.0, or whatever excitements DMA interoperability brings. And if you’re relying on Matrix: please donate.
Meanwhile - have a fantastic end of the year; and thanks once again for flying Matrix.
TL;DR: If you want to play with a shiny new Matrix 2.0 client, head over to Element X.
Matrix has been going for over 9 years now, providing an open standard for secure, decentralised communication for the open Web - and it’s been quite the journey to get to where we are today. Right now, according to Synapse’s opt-in usage reporting, in total there are 111,873,374 matrix IDs on the public network, spanning 17,289,201 rooms, spread over 64,256 servers. This is just scratching the surface, given we estimate that 66% of servers in the public network don’t report stats, and there are many enormous private networks of servers too. We’ve come a long way from creating Matrix HQ as the first ever room on today’s public network, back on Aug 13th 2014 :)
Meanwhile, the Matrix ecosystem has continued to grow unbelievably - with huge numbers of independent clients, bots and bridges maturing into ecosystems of their own, whole new companies forming around the protocol, and organisations ranging from open source projects to governments, NGOs and Fortune 100 companies adopting Matrix as a way to run their own secure, decentralised, standards-based self-sovereign communication.
The world needs Matrix more than ever. Every day the importance of decentralisation is more painfully obvious, as we concretely see the terrifying risks of centralised Internet services - whether that’s through corporate takeover, state censorship, blanket surveillance, Internet shutdowns, surveillance capitalism, or the spectre of gigantic centralised data breaches. It’s been amazing to see the world pivot in favour of decentralisation over the time we’ve been building Matrix, and our mission has never been more important.
On one hand it feels we’re creeping ever closer to that goal of providing the missing communication layer for the open Web. The European Union’s Digital Markets Act (DMA) is a huge step in that direction - regulation that mandates that if the large centralised messaging providers are to operate in the EU, they must interoperate. We’ve been busy working away to make this a reality, including participating in the IETF for the first time as part of the MIMI working group - demonstrating concretely how (for instance) Android Messages could natively speak Matrix in order to interoperate with other services, while preserving end-to-end encryption.
On the other hand, Matrix has often got stuck in focusing on solving the Hard Problems of decentralisation, decentralised end-to-end encryption, and the logistical complexities of supporting a massive heterogeneous public communication network and its surrounding heterogeneous ecosystem. It’s fair to say that in the early days our focus was on making something that worked at all - and then later, we shifted to focusing on something that worked and scaled correctly… but we hadn’t managed to focus on ensuring that Matrix provides the building blocks necessary to create blazingly fast, hyper-efficient communication apps which has potential to outperform the centralised mainstream messaging services…
…until now!
Matrix 2.0
Back at FOSDEM we announced the idea of Matrix 2.0 - a series of huge step changes in terms of Matrix’s usability and performance, made up of Sliding Sync (instant login/launch/sync), Native OIDC (industry-standard authentication), Native Group VoIP (end-to-end encrypted large-scale voice & video conferencing) and Faster Joins (lazy-loading room state when your server joins a room).
Now, we’re excited to announce that as of today everyone can start playing with these Matrix 2.0 features. There’s still some work to bring them formally into the specification, but we’re putting it out there for folks to experience right now. Developers: watch this space for updates on the spec front.
Practically speaking, this means there are now implementations of the four pillars of Matrix 2.0 available today which you can use to power a daily-driver Matrix 2.0 client. The work here has been driven primarily by Element, using their new Element X client as the test-bed for the new Matrix 2.0 functionality and to prove that the new APIs are informed by real-world usage and can concretely demonstrably create an app which begins to outperform iMessage, WhatsApp and Telegram in terms of usability and performance… all while benefiting from being 100% built on Matrix.
matrix-rust-sdk and Element X
The mission of Matrix 2.0 has been to provide a huge step forwards in real-world performance, usability and stability - and that means using a real client codebase as a guinea pig to ensure the new protocol is fit for purpose. matrix-rust-sdk has been the main vehicle for this, with Element X as the app primarily driving the new features (although other clients built on matrix-rust-sdk such as Fractal 5 can then automatically benefit from the work should they wish).
To see what all the fuss is about, your best bet is probably to head over to the Element X launch blog post and read all about it! But from the Matrix perspective, this is a flag day in terms of the existence of a Matrix client which empirically outperforms the mainstream clients both in terms of usability and performance: it shows that Matrix is indeed viable to power communication for billions of users, should we get the chance.
From a client perspective: this has meant implementing Sliding Sync (MSC3575) in matrix-rust-sdk - and then creating the entirely new matrix-sdk-ui crate in order to expose higher level APIs to help apps efficiently drive their UI, without each app having to keep reinventing the wheel and risking getting it wrong. The new UI crate gives APIs for efficiently managing a lazy-loaded room list, lazy-loaded room timelines (including edits, reactions, aggregations, redactions etc), and even when the app should show a sync spinner or not. As a result, the vast majority of the heavy lifting can be handled in matrix-rust-sdk, ensuring that the app layer can focus on UI rather than Matrix guts - and performance improvements (e.g. roomlist caching and timeline caching) can all be handled in one place to the benefit of all clients using the SDK.
This is a huge breakthrough relative to the old days of Matrix where each client would have no choice but burn significant amounts of time hand-carving its own timeline and encryption glue logic (although of course clients are still very welcome to do so if they wish!) - but for those wanting higher-level building blocks, matrix-rust-sdk now provides an excellent basis for experimenting with Matrix 2.0 clients. It’s worth noting that the library is still evolving fast, though, and many APIs are not long-term stable. Both the Sliding Sync API and the UI crates are still subject to significant change, and while the crypto crate and its underlying vodozemac E2EE implementation is pretty stable, features such as E2EE Backup are still being added to the top-level matrix-rust-sdk (and thence Element X).
In order to hook matrix-rust-sdk up to Element X, the Element team endedupcontributingcancellableasync bindings to uniffi, Mozilla’s language binding generator, so you can now call matrix-rust-sdk directly from Swift, Kotlin and (in theory) other languages, complete with beautifully simple async/await non-blocking semantics. This looks to be a pretty awesome stack for doing modern cross-platform development - so even if you have a project which isn’t natively in Rust, you should be able to lean on matrix-rust-sdk if you so desire! We hope that other projects will follow the Rust + Swift/Kotlin pattern for their extreme performance needs :)
Sliding Sync
The single biggest change in Matrix 2.0 is the proposal of an entirely new sync API called Sliding Sync (MSC3575). The goal of Sliding Sync is to ensure that the application has the option of loading the absolutely bare essential data required to render its visible user interface - ensuring that operations which have historically been horribly slow in Matrix (login and initial sync, launch and incremental sync) are instant, no matter how many rooms the user is in or how large those rooms are.
While matrix-rust-sdk implements both Sync v2 (the current API in Matrix 1.8) as well as Sliding Sync, Element X deliberately only implements Sliding Sync, in order to focus exclusively on getting the fastest UI possible (and generally to exercise the API). Therefore to use Element X, you need to be running a homeserver with Sliding Sync support, which (for now) means running a sliding-sync proxy which bolts Sliding Sync support on to existing homeservers. You can check out Thib’s excellent tutorial for how to get up and running (or Element Server Suite provides packages from the Element team)
Now, implementing Sliding Sync in matrix-rust-sdk has been a bit of a journey. Since we showed off the very first implementation at FOSDEM, two big problems came to light. For a bit of context: the original design of Sliding Sync was heavily inspired by Discord’s architecture - where the server calculates an ordered list of large numbers of items (your room list, in Matrix’s case); the client says which window into the list it’s currently displaying; and the server sends updates to the client as the view changes. The user then scrolls around that list, sliding the window up and down, and the server sends the appropriate updates - hence the name Sliding Sync.
Sliding Sync was originally driven by our work on Low Bandwidth Matrix - as it makes no sense to have a fancy line protocol which can run over a 2400 baud modem… if the first thing the app tries to do is download a 100MB Sync v2 initial-sync response, or for that matter a 10MB incremental-sync response after having been offline for a few days (10MB takes 9 hours to shift over a 2400 baud modem, for those who missed out on the 80s). Instead, you clearly only want to send the absolute essentials to the client, no matter how big their account is, and that’s what Sliding Sync does.
The first minor flaw in the plan, however, is that the server doesn’t necessarily have all the data it needs to order the room list. Room ordering depends on what the most recent visible events are in a room, and if the room’s end-to-end encrypted, the server has no way of knowing which events are going to be visible for a given client or not. It also doesn’t know which rooms have encrypted mentions inside them, and we don’t want to leak mention metadata to the server, or design out keyword mentions. So, MSC3575 proposed some complicated contortions to let the client tweak the order client-side based on its superior knowledge of the ordering (given most clients would need to sync all the encrypted rooms anyway, in order to index them and search for keyword notifications etc). Meanwhile, the order might be ‘good enough’ even without those tweaks.
The second minor flaw in the plan was that having implemented Sliding Sync in Element X, it turns out that the user experience on mobile of incrementally loading in room list entries from the server as the user scrolls around the list is simply not good enough, especially on bad connectivity - and the last thing we want to do is to design out support for bad connectivity in Matrix. Users have been trained on mobile to expect to be able to swipe rapidly through infinite-scrolling lists of tens of thousands of photos in their photo gallery, or tens of thousands of emails in their mail client, without ever seeing a single placeholder, even for a frame. So if the network roundtrip time to your server is even 100ms, and Sliding Sync is operating infinitely quickly, you’re still going to end up showing a placeholders for a few frames (6 frames, at 60fps, to be precise) if the user starts scrolling rapidly through their room list. And empirically that doesn’t look great - the 2007-vintage iOS team have a lot to answer for in terms of setting user expectations!
So, the obvious way to solve both of these problems is simply to pull in more data in the background, to anticipate the user scrolling around. In fact, it turns out we need to do that anyway, and indeed pull in all the room data so that room-search is instantly responsive; waiting 100ms or more to talk to the server whenever the user tries to search their roomlist is no fun at all, and it transpires that many users navigate their roomlist entirely by search rather than scrolling. As a result, the sliding sync implementation in matrix-rust-sdk has ended up maintaining an ‘all rooms’ list, which starts off syncing the roomlist details for the most recent N rooms, and then in the background expands to sync all the rest. At which point we’re not really sliding a window around any more: instead it’s more of a QoSed incremental sync.
So, to cut a long story short: while the current Sliding Sync implementation in matrix-rust-sdk and Element X empirically works very well, it’s ended up being a bit too complicated and we expect some pretty significant simplifications in the near future based on the best practices figured out with clients using it. Watch this space for updates, although it’s likely that the current form of MSC3575 will prevail in some respect in order to support low-bandwidth environments where roomlist ordering and roomsearch latency is less important than preserving bandwidth. Critically, we want to figure this out before we encourage folks to implement native server implementations - so for now, we’ll be keeping using the sliding-sync proxy as a way to rapidly experiment with the API as it evolves.
Native Matrix Group VoIP
Another pillar of Matrix 2.0 is that we finally have native Matrix Group VoIP calling (MSC3401)! Much like Sliding Sync has been developed using Element X as a testbed, Element Call has been the guinea pig for getting fully end-to-end-encrypted, scalable group voice/video calling implemented on top of Matrix, building on top of matrix-js-sdk. And as of today, Element Call finally has it working, complete with end-to-end encryption (and integrated in Element X, for that matter)!
Much like Sliding Sync, this has also been a bit of a journey. The original implementations of Element Call strictly followed MSC3401, using full mesh conferencing to effectively have every participant place a call to every other participant - thus decentralising the conference and avoiding the need for a conferencing ‘focus’ server… but limiting the conference to 7 or 8 participants given all the duplication of the sent video required. In Element Call Beta 2, end-to-end encryption was enabled; easy, given it’s just a set of 1:1 calls.
Then the real adventure began: to implement a Selective Forwarding Unit (SFU) which can be used to scale up to hundreds of users - or beyond. The unexpected first move came from Sean DuBois, project lead of the awesome Pion WebRTC stack for Golang - who wrote a proof-of-concept called sfu-to-sfu to demonstrate the viability of decentralised heterogenous cascading SFUs, as detailed in MSC3898. This would not only let calls on a single focus scale beyond hundreds of users, but also share the conferencing out across all the participating foci, providing the world’s first heterogeneous decentralised video conferencing. Element took the sfu-to-sfu implementation, hooked it up to Element Call on a branch, and renamed it as waterfall.
However, when Sean first contributed sfu-to-sfu, he mentioned to us that if Matrix is serious about SFUs, we should take a look at LiveKit - an open source startup not dissimilar to Element who were busy building best-in-class SFUs on top of Pion. And while waterfall worked well as a proof of concept, it became increasingly obvious that there’s a lot of work to be done around tuning congestion control, error correction, implementing end-to-end encryption etc which the LiveKit team had already spent years doing. So, Element reached out to the LiveKit team, and started experimenting with what it might take to implement a Matrix-capable SFU on top of the LiveKit engine.
The end result was Element Call Beta 3, which is an interesting hybrid between MSC3401 and LiveKit’s existing signalling: the high-level signalling of the call (its existence, membership, duration etc) is advertised by Matrix - but the actual WebRTC signalling is handled by LiveKit, providing support for hundreds of users per call.
Finally, today marks the release of Element Call Beta 4, which adds back end-to-end encryption via the LiveKit SFU (currently by using a shared static secret, but in the near future will support full Matrix-negotiated end-to-end encryption with sender keys) - and also includes a complete visual refresh. The next steps here include bringing back support for full mesh as well as SFU, for environments without an SFU, and updating all the MSCs to recognise the hybrid signalling model that reality has converged on when using LiveKit. Meanwhile, head over to https://call.element.io to give it a go, or read more about it in the Element X Ignition blog post!
Native Open ID Connect
Finally, last but not least, we’re proud to announce that the project to replace Matrix’s venerable existing authentication APIs with industry-standard Open ID Connect in Matrix 2.0 has taken a huge leap forwards today, with matrix-authentication-service now being available to add Native OIDC support to Synapse, as well as Element X now implementing account registration, login and management via Native OIDC (with legacy support only for login/logout).
This is a critical step forwards in improving the security and maintainability for Matrix’s authentication, and you can read all about it in this dedicated post, explaining the rationale for adopting OpenID Connect for all forms of authentication throughout Matrix, and what you need to know about the transition.
Conclusion
There has been an enormous amount of work that has gone into Matrix 2.0 so far - whether that’s implementing sliding sync in matrix-rust-sdk and sliding-sync proxy, matrix-authentication-service and all the native OIDC infrastructure on servers and clients, the entirety of Element Call and its underpinning matrix-js-sdk and SFU work, or indeed Faster Joins in Synapse, which shipped back in Jan.
It’s been a pretty stressful sprint to pull it all together, and huge thanks go to everyone who’s contributed - both from the team at Element, but also contributors to other projects like matrix-rust-sdk who have got caught in the crossfire :) It’s also been amazing seeing the level of support, high quality testing and excellent feedback from the wider community as folks have got excited about the promise of Matrix 2.0.
On the Foundation side, we’d like to thank the Members whose financial support has been critical in providing bandwidth to enable the progress on Matrix 2.0 - and for those who want to help accelerate Matrix, especially those commercially building on top of Matrix, please consider joining the Foundation as a member! Also, in case you missed it, we’re super excited to welcome Josh Simmons as Managing Director for the Foundation - focusing on running the Foundation membership programme and generally ensuring the growth of the Foundation funding for the benefit of the whole Matrix community. Matthew and Amandine continue to lead the overall project (alongside their day jobs at Element), with the support of the other three independent Guardians - but Josh is working full time exclusively on running the non-profit foundation and gathering funds to support Matrix.
Talking of funding, we should mention that we’ve had to pause work in other places due to lack of Matrix funding - especially while focusing on successfully shipping Matrix 2.0. Major next-generation projects including Third Room, P2P Matrix, and Low Bandwidth Matrix have all been paused unless there’s a major shift in circumstances - so, if you have money and you’re interested in a world where the more experimental next-generation Matrix projects progress with folks working on them as their day job, please get in touch with the Foundation.
What’s next?
While this is the first usable release of Matrix 2.0 implementations, there’s loads of work still to be done - obvious work on Matrix 2.0 includes:
Getting Native OIDC enabled on matrix.org, and providing migration tools to Native OIDC for existing homeservers in general
Reworking Sliding Sync based on the lessons learned implementing it in matrix-rust-sdk
Actually getting the Matrix 2.0 MSCs stabilised and matured to the point they can be approved and merged into the spec
Adding encrypted backups to matrix-rust-sdk
Reintroducing full-mesh support for Native Matrix Group VoIP calling
Having a big Matrix 2.0 launch party once the spec lands!
Outside of Matrix 2.0 work, other big items on the horizon include:
Adding Rust matrix-sdk-crypto to matrix-js-sdk, at which point all the official Matrix.org client SDKs will (at last!) be using the same stable performant E2EE implementation
Continuing to contribute Matrix input to the MIMI working group in IETF for Digital Markets Act interoperability
Next generation moderation tooling and capabilities
Account Portability and Multihomed accounts
…and much much more.
So: welcome to our brave new Matrix 2.0 world. We hope you’re excited about it as we are - and thanks to everyone for continuing to use Matrix and build on it. Here’s to the beginning of a whole new era!
Today is a big day! As you know, over the last few months we’ve been searching for a Managing Director to join the Matrix.org Foundation full-time, focused on managing the Foundation’s finances, organising the Foundation’s membership programme, helping raise funding to support Foundation work, working with the Guardians to ensure the Foundation stays on mission, and ensuring the Foundation can operate successfully as a fully independent entity.
Given our commitment to open standards and interoperability, we’re delighted to see MLS be ratified by the IETF as RFC9420.
MLS is a new encryption standard defined by the IETF, the standards body that maintains much of what makes the internet work. In the same way that Transport Layer Security (TLS, another IETF standard) defines the way to provide encryption between users and servers, or between two different servers, MLS provides a standard way for users of a messaging service to communicate securely without servers being able to eavesdrop on their conversations.
We launched the Matrix Public Archive publicly on June 2nd, 2023.
We decided to take it down on Sunday, June 25th out of precaution after a
member of OFTC staff warned us that the archive made the content of two OFTC IRC
channels bridged to Matrix available on the Internet.
After investigating the issue, we determined that the Matrix Public Archive's
behaviour was expected for these channels, given an IRC chanop had
explicitly configured the Matrix side of the rooms to be world-readable.
Let's talk about how room visibility works in vanilla Matrix, how it works with
bridges, and what are the next steps.
Back in September 2022 we launched the very first public technology preview of Third Room - our entirely open source, open standards-based platform for creating decentralised multiparty spatial apps and virtual worlds on top of Matrix.
The mission of Third Room is to ensure that a truly open and equitable platform exists for powering shared 3D environments - providing an alternative to the closed walled gardens of the bigger vendors, and generally safeguard against a repeat of the fragmented dystopia that has plagued instant messaging and VoIP systems. In short, just as Matrix aims to be the missing secure communication layer of the open Web, Third Room aims to be the spatial collaboration layer.
Today, we’re incredibly excited to announce Third Room Technology Preview 2: The Creator Update. As more and more 3D hardware enters the market, the race is on to provide tools to developers and creators so they can build on an open, vendor-agnostic platform - and in this update we’ve focused on building out the scripting, editing and authoring capabilities of Third Room to provide a solid platform for building and running collaborative 3D apps of any kind. Check out the new release at https://thirdroom.io.
As a reminder: the Third Room team is a tiny band formed by Robert, Nate and Ajay and operates outside of all the rest of our work on Matrix: the other 97% of our effort goes into making the core of Matrix amazing (particularly the underpinnings for Element X and the next generation of Matrix clients). However, Matrix is about more than just chat and VoIP, and Third Room provides an excellent showcase of Matrix’s abilities as a general purpose communication fabric.
The DMA is new antitrust/competition regulation from Europe which came into force in November, whose objective is to make digital markets more competitive by forcing gatekeepers (i.e. large tech companies) to reconsider some of their anti-competitive or self-preferencing practices. Gatekeepers are defined as companies which have a clear position of influence in a given market (based on revenue / market cap / number of users thresholds), and “an entrenched and durable position”. The process for designating which companies count as gatekeepers will start in May 2023.
The DMA touches upon different key topics, from self-preferencing behaviour to app store management practices - but most importantly includes interoperability for “number-independent interpersonal communication services” (NIICS), otherwise known as chat and voice/video calling and conferencing services (social media was left out for now).
This particular workshop was focused on the latter: interoperability between messaging services, with the aim of getting the different stakeholders of the industry in the same place to discuss how the legislation could be implemented. The whole idea is to figure out a practical way in which WhatsApp could interoperate with iMessage, Google Messages and others, creating an interoperable communication network where users are no longer locked into communication silos and pick their preferred service provider without compromising on who they can talk to.
About 900 people participated online, and around 80 people were present in person: the maximum the room could hold. It was particularly fun to see representatives from the whole industry turning up in person, including folks from XMPP, MIMI (the new IETF working group on messaging interoperability), MLS, us from Matrix obviously (alongside Matrix ecosystem representatives from Beeper and NeoChat!) - all together with the Body of European Regulators for Electronic Communications (BEREC), civil society representatives (like the Federation of German Consumer Organisations (VZBV) and European Digital Rights (EDRi)), mobile network operators, local network agencies, and obviously some of those who are likely to be designated as gatekeepers, such as Meta, Apple and Google.
2022 has been a rollercoaster of a year for Matrix.
On one hand, the network has doubled in size (44.1M to 80.3M visible matrix IDs). The wider world is having a grand awakening to the importance of decentralisation thanks to the situation at Twitter. We’ve seen an amazing number of major new players entering the Matrix ecosystem: Reddit appears to be building out new Chat functionality using Matrix; TeamSpeak announced Matrix-based chat in TS5; Discourse is working on adding Matrix support; Thunderbird launched Matrix support; Governments from Luxembourg to Ukraine have launched their own Matrix-powered chat infrastructure; and hundreds of other organisations ranging from startups to massive private & public sector entities are betting on the protocol. The European Parliament has used Matrix as a proof-point for the viability for communication interoperability between gatekeepers in the Digital Markets Act. FOSDEM 2022 ran smoothly via Matrix with over 23,000 attendees, making it the world's largest open source conference (with 70% of attendees using their own servers!). Sweden has published case studies on the benefits of Matrix for messaging interoperability. Meanwhile existing players like Germany’s BWI have expanded their scope to providing Matrix messaging to the whole German State; Automattic is busy building Matrix plugins for Wordpress; Rocket.Chat launched federation via Matrix, Gematik has been busy progressing their TI Messenger initiative for interoperable messaging within Germany’s healthcare industry, and Tchap in France is continuing to expand.
On the other hand, only a handful of these initiatives have resulted in funding reaching the core Matrix team. This is directly putting core Matrix development at risk. We are witnessing a classic tragedy of the commons. We’ve released all the foundational code of Matrix as permissively-licensed open source and got it to the point that anyone can successfully run it at scale themselves. The network is expanding exponentially. But in return, it transpires that the vast majority of these commercial deployments fail to contribute financially to the Matrix Foundation - whether by donating directly or supporting indirectly by working with Element, who fund the vast majority of core Matrix development today.
In short: folks love the amazing decentralised encrypted comms utopia of Matrix. But organisations also love that they can use it without having to pay anyone to develop or maintain it. This is completely unsustainable, and Element is now literally unable to fund the entirety of the Matrix Foundation on behalf of everyone else - and has had to lay off some of the folks working on the core team as a result.
The only viable solution to this is for organisations building on Matrix to contribute to sharing the costs of maintaining Matrix’s core projects. We made a proposal to address this a few weeks ago, which we’ll iterate on further in the new year to find an approach which both empowers the community and encourages organisations to participate. In the interim, if you are an organisation who’s building on Matrix and you want the project to continue to flourish, please mail [email protected] to discuss how you can support the foundations that you are depending on.
As a reminder, the work the Foundation does today for the benefit of the Matrix includes:
Developing a reference push notification server (sygnal)
Developing a reference identity directory server (sydent)
Procuring and publishing independent public audits of Matrix's encryption and wider stack
Publishing the matrix.org website and blog
Publishing the weekly "Matrix Live" video podcast
Publishing the weekly "This Week In Matrix" news letter
Organising regular meetups (e.g. "Open Tech Will Save Us")
Promoting Matrix at open source conferences
Running the matrix.org homeserver
Moderating the matrix.org project rooms
Running free public bridges to networks such as IRC networks and XMPP.
This list is not remotely exhaustive (turns out there are over 240 projects in the matrix.org GitHub org!) but it serves to illustrate the sheer scale of work that the Foundation performs today. Keeping the core team funded to work on Matrix as our day job is critical for Matrix’s long-term success, and so we really hope that organisations depending on Matrix (or passing philanthropists who appreciate Matrix’s value) will drop a line to [email protected] and help keep the show on the road.
Turbocharging Matrix
Aside from the nightmares of funding open source software, 2022 has very much been a year of building - focusing on implementing a step change in Matrix’s performance and usability: ensuring that the protocol can punch its weight (and more!) against centralised proprietary alternatives. After all, Matrix clients need to be at least as good as the centralised alternatives in order to get widespread uptake.
This work has ended up taking many forms: on the server-side, Synapse sprouted Rust support to accelerate its hot paths, starting with push rule evaluation. It’s super exciting to see Synapse performance heading into a new era, building on the foundations of what’s become a very mature and stable homeserver implementation.
Meanwhile work is in the final stages on “Faster Joins”, finally letting servers rapidly join rooms over federation by synchronising only the minimal subset of state needed to join, rather than proactively synchronising the room’s full current state. Faster joins became available for testing in Synapse in October, and since then the team has been working through making it support workers and addressing the various edge cases and bugs that have shown up during testing. Current join performance is a roughly 25x speedup on large rooms, although we’re confident that we can improve this even more, and we’re aiming to land it in time for FOSDEM at the beginning of Feb.
On the client-side, the work to transform Matrix client performance has centred around “Sliding Sync” - our entirely new API for synchronising the minimal data to a client needed for it to render its UI, thus making login, launch and sync instant. Sliding sync (originally called “sync v3”) has been a long time in the making; the API has gone through countless iterations as we worked away throughout 2022 implementing it in real-life clients, and adding all the extensions (MSC3884, MSC3885) needed to get to parity with sync v2. The wait has been well worth it, though: support in Element Web is in the final stages of development - and moreover the next-generation Element X mobile clients will only speak Sliding Sync.
Element X itself is shaping up to be a showcase of just how snappy and performant Matrix can be: built on matrix-rust-sdk, it uses native Swift UI on iOS/macOS and Jetpack Compose on Android to couple together the best possible platform-native user experience with the ultimate underlying native-code SDK implementation, backed by sliding sync. The goal is to be at least as snappy as Telegram, iMessage or WhatsApp (we’ve taken to counting the frames in screen recordings to compare things like time-to-launch and time-to-load scrollback). Element X is currently in late alpha on iOS, and the hope is to enter public beta in time for FOSDEM. You can see a sneak peek here of the iPad-style layout (running under macOS) though!
Finally, in terms of usability, there have been leaps and bounds forwards across Matrix - particularly with Element’s mobile UI being entirely refreshed by the design team in September as a stepping stone to the forthcoming final Element X design. Any remaining UX quirks should be flushed out with Element X, but the visuals are already a clear step forwards towards an excellent alternative to the centralised encumbents.
Encryption
We had great plans for E2EE in Matrix this year; starting off in a huge rush to get vodozemac finished and audited as our shiny new native-Rust implementation of Olm/Megolm. The plan was then to integrate vodozemac into matrix-rust-sdk’s crypto crate, and then replace the various old fragmented E2EE implementations across matrix-js-sdk, matrix-ios-sdk, matrix-android-sdk2 and matrix-rust-sdk itself with One True audited implementation - with audits booked with Least Authority to get further assurance around matrix-rust-sdk-crypto, matrix-rust-sdk itself and finally the full stack (Element X + Synapse).
Unfortunately, things went sideways when security researchers from Royal Holloway University London & elsewhere got in touch to explain that they had found some nasty implementation vulnerabilities in the venerable matrix-js-sdk implementation. So, we had no choice but to pause “Element R” - the project to converge matrix-{js,ios,android}-sdk on matrix-rust-sdk-crypto, and instead start analysing and addressing the issues across all current shipping Matrix clients in order to resolve them as rapidly as possible. Ironically, it turned out in the end that only matrix-{js,ios,android}-sdk were affected - all other independent implementations, including matrix-rust-sdk, were okay. As such, the Element R work would have protected us from these vulnerabilities had it been ready, and failing that it would have let us solve them in a single place. Instead, Element R ended up getting pushed back for months while we worked through the various issues in triplicate across the legacy SDKs, while also checking all the other client implementations we could find, and dealing with additional issues which the RHUL researchers discovered as they dug deeper. Eventually we finished the analysis and agreed a coordinated disclosure at the end of September.
(EDIT: to be clear, we are very grateful to the security researchers for discovering and disclosing the vulns responsibly to us. The frustration here stems from the irony that if we'd finished the matrix-rust-sdk-crypto rewrite a few months earlier, we'd have mitigated the severe vulns - but instead, the rewrite got pushed back even further. It's obviously our fault though, not the researchers'.)
Since then, work has been split three ways: firstly, Element R work has resumed - and in fact Element R on iOS is pretty much usable as of today, other than needing some work to support E2EE push notifications (which will also be required for Element X). Element R on Android is very close too, and meanwhile Element R on Web decrypted its first event on Dec 19th! We’re hoping to get Element R in production on all platforms by Feb.
Secondly, we’ve been addressing other points raised by the RHUL researchers to ensure that malicious servers cannot add malicious devices or users to conversations, rather than warning as we do today. This is not a trivial problem to solve, but we’re making progress via MSC3917 (Cryptographically Constrained Room Membership) and MSC3834 (Opportunistic user key pinning (TOFU)). However, this work is effectively blocked on Element R landing first, as there’s no way we’re going to fix this in triplicate on the legacy SDKs.
Thirdly, we’ve been pushing ahead on implementing Decentralised MLS as a next-generation encryption protocol for Matrix to potentially replace Olm and Megolm. This work was badly disrupted by RHUL mitigations, but we’re making good progress again - you can follow all the details over at https://arewemlsyet.com. Matrix over DMLS is currently in alpha, but the aim is to start beta testing Decentralised MLS in 2023.
Finally, we’ve been working hard on completely reworking the overall UX of how E2EE should work in Matrix clients - specifically, requiring users to cross-sign their devices in order to use E2EE, and so end up in a much higher trust world (alongside Trust On First Use). Can’t wait to finally simplify the E2EE UX!
All new features
It’s not all been performance and stability work this year - there have been some large areas of feature work happening too.
One of the most visible projects has been Threads, which launched in beta in April, and subsequently has undergone huge amounts of polish to improve performance, notification semantics, unread behaviour and thread-aware read receipts. The end result is feeling great now, and threads exited beta in Element Mobile on Dec 20th. Web narrowly missed the window due to a final ‘stuck notification’ bug which is still being tracked down, but will follow shortly afterwards and then threads will be finally out of beta!
Another big project in 2022 has been to create a general purpose Rich Text Editor to provide WYSIWYG (What You See Is What You Get) message composition for Matrix clients. This has ended up being a very ambitious project to define all the core editing semantics in a shared rust library, with platform-specific bindings to link it into the editing UI available on Web, iOS & Android. The end result lives at https://github.com/matrix-org/matrix-rich-text-editor - and you can play with it by enabling it in labs on Element Web/iOS/Android or experiment with the live demo. The core behaviour is feeling excellent, although predictably some of the fine detail is very fiddly to get right. It’s almost there, though, and thanks to its built-in rust test harness generator(!) we are confident we’ll catch and control all the edge cases, and this should form an incredibly strong platform for all future rich text editing requirements in Matrix (and beyond!). This work was very kindly sponsored by one of Element’s public sector customers in order to get Element to parity with Teams - thank you!
Location Sharing was another feature which landed in 2022 - powered by MSC3488 and MSC3489, and implemented in matrix-{js,ios,android}-sdk in Element Web/iOS/Android, letting users share static and live locations and view them on an OpenStreetMap compatible map tileserver of their server’s choice. The Live Location Sharing is controversial in that it stores location data in the room history (and as such is hidden behind a labs flag on Element), but should eventually be replaced by MSC3672 to share locations are custom ephemeral events instead (once custom EDUs land) in the spec. Around the same time, Polls also went live thanks to MSC3381 - it’s worth noting that both Location Sharing and Polls are excellent examples of “extensible events” in the wild: ensuring that clients which understand the custom event type will render them appropriately, but letting other clients fall back to showing them as simple timeline events.
Open ID Connect
The transition to using Open ID Connect for Matrix authentication has been progressing steadily throughout 2022 - with Third Room being the first OIDC-native Matrix client, closely followed by Element X. matrix-authentication-service now exists as a basic OIDC identity provider suitable for being linked into Synapse, and meanwhile Third Room demonstrates how you can integrate Keycloak as a third party IDP (complete with reCAPTCHA and guest access!). The team also went on a very exciting detour to figure out how to perform login-and-E2EE-setup in a single operation by scanning a QR code (MSC3906), and how it might integrate into OIDC in future.
Element X looks set to be the showcase for native OIDC in a typical Matrix client going forwards, so watch this space to see how it feels!
2022 was the year that Matrix finally got native multiparty VoIP. After launching Element Call Beta 1 in March followed by Beta 2 in June, we’ve been busy embedding Element Call as a “matryoshka” widget into Element Web, using it to replace Jitsi in powering video rooms and video calls. You can read all about this in detail in the summer blog post.
Meanwhile, lots of progress is underway on Waterfall - the name we picked for the Pion-based decentralised Selective Forwarding Unit (i.e. conferencing focus) contributed by Sean DuBois earlier in the year, including adding simulcast support to support large scale conferences.
There’s only one catch, which is that Element Call is still in (very very late) beta, thanks to a handful of bugs which have been hard to track down, which has in turn kept all the other dependencies (embedded Element Call; video rooms etc) in beta too. However, we think we’re pretty much there now - which is perfect timing given how Waterfall is coming together, meaning that both stable and scalable native Matrix conferences are on the horizon!
Even better, the plan is for Element X to rely entirely on embedding Element Call for VoIP - so we should be able to jump forwards pretty rapidly to having excellent native multiparty VoIP and video rooms on mobile as well as on Web. So, once Element Call exits beta, everything should follow. Just for a change, we’re aiming to get this done by the end of January - but there are a lot of unknown unknowns still flying around, so watch this space…
IETF & MIMI
Another massive new initiative this year has been the process of proposing Matrix to the IETF as a candidate for use in interoperable instant messaging standardisation. The MIMI (More Instant Messaging Interoperability) working group emerged earlier in the year within IETF as an initiative to define how MLS could be used to interoperate between different instant messaging silos - as shortly to be required by the Digital Markets Act.
One of the things that MIMI aims to do is to define a common application layer protocol to exchange messages. At first CPIM was proposed (an ancient message format that looks a lot like email) - and then an entirely new JSON message format proposal emerged which looks somewhat Matrix (but isn’t). At this point it became obvious that we should throw our hat into the ring and encourage MIMI to use Matrix rather than reinvent it, and so we set about proposing Matrix as at least the message format and message transport layer of the stack. It’s quite surreal to see Matrix starting to fly around as IETF Drafts!
The next step here is to re-express the relevant bits of the current Matrix spec as self-contained IETF Drafts (rather than backreferencing the current spec from the drafts). The idea is that the normal Matrix spec will continue to evolve much as it always has, but we’ll effectively donate a Long Term Supported dialect of it to IETF which can then evolve according to IETF process and be immortalise as RFCs for use in MIMI. We’ll then backport those changes into spec.matrix.org in order to avoid fragmentation, while retaining the same ability we have to rapidly iterate and extend Matrix with MSCs. This work is well under way (taking opportunity to use Extensible Events from the outset!), and we should see explosions of further IETF Drafts emanating from Travis as 2023 progresses.
Trust & Safety
2022 saw a real uptick in spam and abuse across Matrix, and there have been some valiant attempts to improve our moderation tooling over the course of the year. Unfortunately it hasn’t come together as rapidly as we might have hoped, however, and we’ve seen several large communities give up on Matrix and move back to Discord thanks in part to needing better anti-abuse mechanisms.
In 2023 we’re resetting our trust & safety work, with Mjolnir dev returning to its original development team, and we’ll be working as tactically as possible to ensure that all communities on Matrix can easily block abuse using whatever mechanisms they need.
P2P & Dendrite
Meanwhile, Dendrite (our second generation homeserver implementation) development has continued at pace throughout the year. According to sytest we are now at 93% client-server API compliance with 577 out of 620 tests passing, and the server-server API compliance is at 97% with 111 out of 114 tests passing! None of the missing tests are showstoppers, so it’s fair to say that Dendrite is very nearly ready for primetime.
The interesting plot twist is that Dendrite development has ended up increasingly focusing on embedded matrix server use cases - particularly to power Peer-to-Peer Matrix, where clients require a server to be embedded within them. So while Synapse has ended up increasingly focusing on large-scale deployments, Dendrite has ended up pursuing smaller instances (which is ironic, given originally it was meant to be the other way round!).
P2P Matrix work has been progressing well too - you can follow the blow-by-blow updates over at https://arewep2pyet.com. After a lot of back and forth evaluating hard-state routing versus soft-state routing in Pinecone, we’ve ended up converging on soft-state routing (which is chattier, but easier to reason about in terms of mitigating attacks). However, the chattiness means that it doesn’t scale as well as one might hope - so we’re now working on a “tiered” approach where separate Pinecone networks can be tiered together into one inter-network, giving us scalability at the expense of being slightly less decentralised. It’s fair to say that the journey here has been pretty frustrating in its twists and turns, and sadly Neil Alexander chose to move on a few months ago. However, Devon has stepped up to fill his shoes as primary Pinecone and P2P wrangler, and is making amazing progress on the remaining work - firstly implementing Store and Forward relaying in Dendrite so that today’s Pinecone networks can exchange messages even if the recipient node is offline. Next up will be bridging P2P Matrix with today’s Matrix network - and then working on tiering to provide the scalability we need. The expectation is that today’s serverside Dendrite instances will effectively turn into static pinecone peers, store and forwarding messages on behalf of P2P nodes, and providing tiering between respective pinecone subnets.
Hydrogen & Chatterbox
Development on Hydrogen as a super-lightweight progressive-web-app Matrix client has also been progressing throughout the year (with a few detours to help out with end-to-end testing via trafficlight both for the benefit of Hydrogen and other clients).
The biggest change has been Hydrogen sprouting a separate SDK layer, letting the engine be embedded into other webapps in order to add noninvasive Matrix messaging with as minimal a footprint as possible. This was showcased in Element’s chatterbox offering in July - providing an open source chatbox which can be trivially embedded into existing sites, and also powers the Chatrix wordpress plugin that Automattic is working on.
Hydrogen also added independent support for MSC3401 multiparty voice/video calling (albeit on a branch), letting us showcase heterogeneous Element Call <-> Hydrogen group calling and prove MSC3401 as fit for purpose as a true open interoperable call signalling - and in turn Hydrogen SDK, complete with the multiparty voice/video calling, powers the Matrix engine within Third Room - our metaverse-on-Matrix platform.
We’re looking forwards to Hydrogen continuing to reach full feature parity with Element over the next year, and popping up in increasingly unexpected places as everyone’s favourite embedded Matrix client!
Third Room
Finally, it’s hard to believe that Third Room, our Matrix-based open platform for decentralised realtime spatial collaboration, barely existed at the beginning of the year. Third Room serves to demonstrate that Matrix is way more than just chat and VoIP, but can power the spatial communication layer of the open web. This has ended up driving forwards a tonne of new capabilities for Matrix - showcasing native OIDC auth; scalable multiparty VoIP in Hydrogen SDK, efficient binary-diffed file storage, and more recently has been defining how to store extensible behaviour for Matrix rooms as WASM objects stored in the Matrix room itself.
Third Room itself is a Hydrogen-based Matrix client, which lets you view Matrix rooms as interactive multiparty 3D environments (using MSC3815) - with the world defined as glTF blobs stored in the Matrix room, and the ability to script and customise any aspect of that world using WASM blobs stored in Matrix rooms, which execute on the participating clients, exposing a new scenegraph API called WebSceneGraph in order to manipulate the glTF that makes up the world. We also expect to see a variant of Matrix’s normal widget API to be exposed to these WASM blobs, introducing the concept of sandboxed clientside widgets, bots or other integrations - letting users customise and extend Matrix without ever having to run serverside bots again.
The intention is to provide a platform which can be used to build any kind of interactive realtime spatial multiparty app in an open standardised, decentralised, end-to-end encrypted way - whether that’s for gaming, social, or professional activity such as building “digital twins” for manufacturing, agriculture, smart cities, search & rescue, etc. You can read more about the vision at thirdroom.io/preview, or via press coverage at TheNewStack or Golem. We were also incredibly flattered to be invited to present Third Room at SIGGRAPH Asia a few weeks ago. The official recording has yet to emerge, but you can find a cheeky bootleg here.
We launched Tech Preview 1 of Third Room at the end of September, and since then all of the work has been around building out WebSceneGraph and the WASM scripting environment - letting users build their own functionality in JS via QuickJS or C (and in future Rust or Zig too). We’ve also been working on making the networking (via Matrix WebRTC-negotiated data channels) more robust, switching to an ‘authoritative’ simulation model rather than having each client run its own physics simulation, in order to kick the hard problem of decentralised physics simulations down the road a bit further. We’re also adding in a much-needed ‘discover’ page to help users find new rooms and explore everything that’s possible in the platform. And finally, we’re adding WebXR support so that folks can use ThirdRoom with VR and AR hardware if they so desire. All this should culminate in Tech Preview 2, due in the coming weeks.
So there you have it: it’s been a mixed year for Matrix, but at least the project itself is moving forwards faster than ever, for now. If you look back at the predictions from last year’s holiday blog post you’ll see that most of them even came true. This year, we’ll keep the predictions simple: our plans for 2023 are to ensure that the Foundation is well funded, ship all of the step-change improvements in performance and usability which are currently in beta as rapidly as possible - and demonstrate for once and for all that Matrix can indeed punch its weight against the proprietary centralised alternatives.
If you can afford it, please consider donating to the Matrix.org Foundation to support our work. The most efficient way to support us is to donate via donorbox. Our Patreon is not going anywhere, so if you wish to keep supporting it there we're happy to count you in our supporters.
TL;DR: you can now officially join the Matrix Foundation as an organisational
or individual member in order to sustainably support core Matrix development,
help steer the direction of the protocol and how best to fund it! Organisations
can join by filling this form and we will get back in touch, or individuals can
now donate directly here (as a more efficient alternative to Patreon, which
remains online for Patrons used to it).
Hi all,
Only two days late for Giving Tuesday (and 4 years late on the Foundation
scale), we are super excited to announce that we are finally expanding the
Matrix Foundation into a fully-fledged non-profit fund-raising organisation to
help support the core Matrix development and the wider open source Matrix
ecosystem!
Has your organisation been using Matrix to communicate via the open source
server and clients and you want to ensure that improvements and features still
keep coming? Become a member, and feed into the roadmap.
Has your company been building on top of Matrix, making the most of its openness
and flexibility, but you’ve never figured out how to contribute back in order
to ensure the resilience of the tech which you rely on for the success of your
business? Become a member, and ensure the core development is funded and that
Matrix is here to thrive.
Are you about to be designated a gatekeeper by the EU, and have to rapidly figure out how to implement DMA-compliant interoperability (complete with end-to-end encryption compatibility) into your services? Become a member, and discover how Matrix can solve all your problems, while providing your input too.
Are you looking to implement DMA interoperability but don't fall into the gatekeeper designation? Become a member, and help ensure Matrix fits your needs too!
Are you a non-profit organisation with a mission to provide secure and sovereign communications to those who need it the most, providing an alternative to the current players? Become a member, and help us take our mission forward.
Are you an individual who strongly supports the mission of Matrix and wants to see it thrive and become the open backbone of the world’s communications? Become a member, and support the future of Matrix.
In short, this finally gives the wider world a way to contribute concretely to the significant costs of funding folks to work fulltime on core Matrix development - which now covers over 243(!) projects in github.com/matrix-org. Core Matrix work ranges from:
managing the Matrix spec itself
maintaining the reference client SDKs and encryption and getting them independently audited
maintaining example server implementations in the form of Synapse and Dendrite
writing the test suite
publishing the matrix.org website
promoting awareness of Matrix
running the matrix.org homeserver
and so much more...
In exchange for supporting the Foundation, and beyond providing certainty that our work can continue, organisations and individuals who contribute financially will be able to directly provide input to the trajectory of the core Matrix developments by becoming official members of the Matrix.org Foundation.
Introducing Foundation Memberships and the Governing Board
Practically speaking, this means that we are going to create a "Governing Board" to the Matrix.org Foundation: a new team, made up of members voted by the overall membership, a subset of the Guardians and a subset of the Spec Core Team. The Governing Board will have the responsibility of determining how Foundation funds are distributed and used, how the Spec Core Team roadmap is prioritised, how to best grow Matrix awareness, etc.
In other words, we are literally expanding the day-to-day steering of the direction of the Matrix Foundation to the wider community. In order to run the Governing Board and the overall work of the Foundation, we are also hiring an Executive Director, so please get in touch via [email protected] if you’re a non-profit foundation-running expert! In the interim, while this search proceeds, Matthew and Amandine will fill the role with the support of the other Guardians of Matrix. The Guardians themselves retain their existing function as the Foundation's non-executive board - responsible for safeguarding the overall mission of Matrix, appointing the membership of the Spec Core Team, approving membership applications, and defining the overall changes we’re outlining here.
Membership comes at various levels, each with different rewards:
Ability to vote in the appointment of up to 2 ‘community representatives’ to the Foundation's governing board.
Name on the Matrix.org website
Silver member: between £2,000 and £80,000 per year, depending on organisation size
Ability to vote on the appointment of up to 2 ‘Silver representative’ to the Foundation's governing board
Supporter logo on the front page of the new Matrix.org website
Gold member: £200,000 / year, adds:
Ability to vote on the appointment of up to 3 ‘Gold representatives’ to the Foundation's governing board.
Press release announcing the sponsorship
1 original post on the Matrix.org blog per year
Participation in the internal Spec Core Team room
Larger logo on the front page of Matrix.org
Platinum member: £500,000 / year, adds:
Ability to vote on the appointment of up to 5 ‘platinum representatives’ to the Foundation's governing board.
1 sponsored Matrix Live episode per year
Largest logo on the front page of Matrix.org
As the activities of the Foundation increase, we expect to add more benefits to this list, for example discounts on sponsoring a future Matrix Conference, or similar.
Governing Board elections will occur yearly, with the first election planned towards the end of this year once we’ve gathered together the first wave of members and candidates have proposed themselves for election.
For anyone building on Matrix, memberships are a no-brainer as they will ensure the perpetuity and future-proof-ness of the standard. But for anyone supporting the mission of Matrix, this membership can be key to define its future.
Meanwhile, with Matrix looking increasingly integral to implementing interoperable communication for compliance with the EU’s Digital Markets Act (whether that’s as pure Matrix, or as part of the IETF MIMI initiative), this is an incredible opportunity for the organisations impacted by DMA to get a front-row seat within the Foundation to ensure that Matrix thrives and solves the challenges posed by the act. To get involved, please apply via the membership application form.
So, why is this all happening?
Since 2017, core Matrix development has been funded primarily by Element - the company founded by the team who created Matrix. Over the years, Element has put tens of millions of dollars into Matrix - which in turn has come from both selling Matrix hosting (EMS), on-premise Matrix solutions, and VC investment in Element. To put it in perspective, even though there are over 5000 contributors to github.com/matrix-org - over 90% of the actual committed lines of code come from Element employees. Similarly, while we are enormously thankful for the past and existing generous donations from the wider Matrix community, today they only come to $6,000 a month, relative to the $400,000 a month that Element has been funding.
Over the last year, we’ve seen a palpable shift within the Matrix ecosystem. Matrix is growing faster than ever. Synapse has improved immeasurably, using less RAM than ever and even sprouting Rust to optimise its hot paths. Element has improved immeasurably too, with an entirely new design on mobile, and tons of new features including threads, voice messages, location share, video rooms and more. Monthly active users reported via Synapse’s phone-home stats have almost doubled and are growing at their fastest ever rate. The number of servers has increased equivalently. We hear about major new commercial Matrix deployments almost every day. However, while usage is going through the roof - we haven’t seen a matching increase in players looking to support the project.
In fact, we’ve seen the opposite: commercial vendors forking the protocol while trying to break up the core team. Matrix tenders lost to “preferred” vendors who know absolutely nothing about Matrix. Vendors selling Matrix hosting or services without contributing anything back to the project at all. Organisations with huge amounts of money (governments, $$B massive enterprises) have enthusiastically launched proprietary Matrix solutions by building on the liberally-licensed Apache reference Matrix implementations… while contributing back nothing. Now, we are enormously grateful for the commercial Matrix deployments who do actually work with Element to fund core development or contribute code themselves - but this is very clearly the minority. Obviously it’s great to see folks building on Matrix - but it’s rather galling if it ends up with insufficient funding trickling down to the core Matrix team to be able to build the foundational technology that everyone else is relying on.
Element in particular has staffed up in order to both support the core Matrix ecosystem (the spec, Synapse, Dendrite, Client SDKs, Encryption implementations etc) as well as Element-specific work (the Element apps, EMS, the Element Enterprise Installer, etc). As a result, Element employs roughly twice as many developers as you might expect, and while Matrix is here to stay, this turns out not to be sustainable for Element unless the wider ecosystem helps support the foundational work
We believe the world needs secure decentralised communication more than ever, and ensuring the Foundation can distribute funding to those contributing to the core platform (be that Element or individuals or other organisations) is key to that. So: please consider this a call to arms - if you believe the world needs Matrix, and particularly if you depend on it for your business, please join the Foundation and participate!
Fill in the form to apply for membership, or if you are not acting on behalf of an organisation, you can go straight to our donation page.
Matthew, Amandine & the whole Matrix.org Foundation
