Matrix v1.11 release

20.06.2024 16:52 — Releases Travis Ralston

Hey all,

We’ve just released the milestone Matrix 1.11 update for the protocol. It’s been almost exactly 3 months since the last release, Matrix 1.10, keeping us on track for our once-a-quarter release schedule.

There are 9 MSCs released in Matrix 1.11 today, but there’s one specific MSC we’d like to draw your attention to: MSC3916 - Authenticated Media. Until today, Matrix had a design flaw which allows a user to access media unauthenticated if they knew the URL. This has been used to share files in social media posts, link images outside of chats, and generally imply that a homeserver is a CDN for the internet. Some of these use cases are legitimate, though many are not. This is fixed with MSC3916.

This post covers MSC3916 and its implementation guidelines in more detail, but the full changelog for Matrix 1.11 is available at the end, as always.

MSC3916: Authenticated Media

The Trust & Safety team has been hard at work fixing the design flaw in media mentioned above, taking MSC3916 under their wing and bringing it to release here. The goal has been to roll out the feature as fast as possible, and that’s culminated in the following plan after consultations with us on the Spec Core Team.

Servers are encouraged to freeze unauthenticated media access (i.e. only allow historical media to be accessed without auth) before Matrix 1.12 is released,which we anticipate will happen in about August 2024. Any media which is cached or uploaded before that freeze will remain accessible on the unauthenticated endpoints indefinitely to avoid breaking existing links in social media, external chats, etc. Media which is cached or uploaded after the freeze takes place will only be accessible through the new authenticated endpoints - callers will receive a 404 error on the unauthenticated side.

The MSC has a bit more detail about the side effects of this approach, and more on the considerations leading to this proposed plan. Implementations might choose to support a config flag to enable/disable the freeze as needed, or simply release a version which freezes unauthenticated media on startup. Either are effective approaches for improving user safety and privacy.

Each homeserver should additionally consider its local ecosystem before freezing unauthenticated media by ensuring bridges and the popular clients among users are ready for authenticated media in particular. Ideally, users should see no visible impact to this change being rolled out. To help encourage this, the matrix.org homeserver will be rolling out authenticated media relatively quickly itself - watch for news on the blog next week for specific timelines.

Client and server developers are encouraged to support authenticated media as soon as they can to help server admins make the switch. Servers have a bit more work to do than clients, unfortunately, but both should have a reasonably easy time adding in the relevant support. The T&S team have made themselves available in #matrix-client-developers:matrix.org and #matrix-homeserver-developers:matrix.org to help developers implement this new feature - feel free to visit the respective room for your project and ask questions.

Below is some implementation guidance for developers, though is not a replacement for the spec. Where this post and the spec differ, please prefer the spec :)

Client implementation guidance

Thankfully, supporting authenticated media should be relatively trivial for most clients. When the homeserver declares support for Matrix 1.11 through /versions, the client should use the new /_matrix/client/v1/media/* endpoints, noting that they now require authentication. The user’s access token can be supplied using the Authorization header. (Note that using the query string ?access_token method is not recommended, and deprecated in Matrix 1.11 for all endpoints.)

If the server doesn’t support Matrix 1.11, the existing /_matrix/media/* endpoints should continue to be used.

This may end up looking something like the following in a client:

Clients which rely on a web browser may have to use a Service Worker to append the required header, particularly for img tags and similar.

Clients should additionally note that, as well as /_matrix/media/v3/download/{serverName}/{mediaId}, the following other endpoints have moved to the /_matrix/client/v1/media/* namespace:

The media /upload, /upload/{serverName}/{mediaId}, and /create endpoints have not yet moved; they are expected to make the switch in a future spec version.

Server implementation guidance

As mentioned above, servers are a bit more tricky. The Client-Server API endpoints are relatively straightforward: they are the exact same as the previous endpoints, just with authentication added and at a new path. The Federation API endpoints are a bit more challenging. Most server projects have already started implementation, but for those which haven’t (or need a bit of guidance), there’s some information here.

Server authors are also encouraged to review the client guidance above to gain a better understanding of what their connected clients will be looking for and doing.

Up next in Matrix 1.12 and beyond

Over the next 2-3 months, we’ll be focusing on the following MSCs:

  • Early review of Matrix 2.0 features as they become ready.
    • Sliding sync - MSC3575, or its Simplified variation, and applicable extensions.
    • Group VoIP - Exact MSCs to be determined, as they may change following implementation.
    • OIDC Authentication - primarily MSC3861.
    • Encryption stability - primarily MSC4153.
  • Pre-implementation review of Trust & Safety MSCs as required.
    • Exact MSCs to be determined. Likely to include some “reporting v2” work.
  • Extensible Events and Custom Emoji/Stickers unblocking as required and able.

Additionally, we’ll be continuing to define the expectations of a Spec Core Team member, particularly as it relates to the Governing Board of the Foundation. This exercise has been extremely valuable to us so far, and will help identify areas the Foundation and SCT may need support from each other.

The full changelog

Read on for full details of what’s in Matrix 1.11:

Client-Server API

Deprecations

  • Authentication using a query string is now deprecated, as per MSC4126. The Authorization header should be used instead. (#1808)
  • Use of the /_matrix/media/* endpoints is now deprecated. New, authenticated, endpoints are available instead. (#1858)

New Endpoints

Backwards Compatible Changes

  • Add support for muting in VoIP calls, as per MSC3291. (#1755)
  • Add optional animated query string option to GET /thumbnail, as per MSC2705. (#1757)
  • Specify terms of services at registration, as per MSC1692. (#1812)
  • Add support for mathematical messages, as per MSC2191. (#1816)
  • Do not require UIA when first uploading cross-signing keys, as per MSC3967. (#1828)
  • Add the new unsigned.membership property to events, as per MSC4115. (#1847)
  • Media downloads and thumbnails are now authenticated, as per MSC3916. (#1858)
  • Some media endpoints are now consistently under /_matrix/client/{version}/media/* instead of /_matrix/media/*, as per MSC3916. (#1858)

Spec Clarifications

  • Add /logout and clarify the endpoints which do not take a JSON request body. (#1644)
  • Clarify that the type of the POST /login request must be one of the types returned by the GET /login response. (#1776)
  • Link to existing grammar where possible in types. (#1813)
  • Rename "recovery key" to "backup decryption key". (#1819)
  • Clarify that the device's Ed25519 signing key should be used in QR code verification (as opposed to the device's Curve25519 identity key). (#1829)
  • Fix various typos throughout the specification. (#1832, #1841, #1852, #1853)
  • Specify the encoding to be used when generating QR codes for device verification. (#1839)
  • Clarify that an access token is optional on the POST /account/password and POST /account/deactivate endpoints. (#1843)
  • Use RFC 2119 keywords more consistently. (#1846, #1861)
  • Move size limits for user, room and event IDs into the appendix and clarify that the length is to be measured in bytes. (#1850)
  • Clarify that relations recursion should be capped at a certain depth. (#1854)
  • Add missing secrets, third-party invites and room tagging modules to feature profiles table. (#1860)
  • Clarify when server name is used and link to the definition. (#1862)
  • Clarify where keys reside when checking an m.room.encrypted event. (#1863)
  • Clarify that /media/v3/upload/{serverName}/{mediaId} requires authentication. (#1872)

Server-Server API

Deprecations

  • Use of the Client-Server API /_matrix/media/* endpoints is now deprecated. New, authenticated, endpoints are available instead. (#1858)

New Endpoints

Backwards Compatible Changes

  • Media downloads and thumbnails are now authenticated, as per MSC3916. (#1858, #1869)

Spec Clarifications

  • Link to existing grammar where possible in types. (#1813)
  • Clarify that whitespace around commas is allowed in the X-Matrix Authorization header value params list. (#1818)
  • Clarify that the event field of the /v2/send_join response is only required when the event is signed by the resident server. (#1834, #1840)
  • Replace references to RFC 7235 and RFC 7230 that are obsoleted by RFC 9110. (#1844)
  • Fix various typos throughout the specification. (#1877)

Application Service API

Spec Clarifications

  • Clarify that appservices should be notified of events relating to the sender_localpart user. (#1810)

Identity Service API

Deprecations

  • Authentication using a query string is now deprecated, as per MSC4126. The Authorization header should be used instead. (#1808)

Push Gateway API

No significant changes.

Room Versions

Spec Clarifications

  • Clarify that redaction events are still subject to all applicable auth rules. (#1824)
  • Fix various typos throughout the specification. (#1827, #1848)
  • Fix the rendering of the event format for room versions 1 and 2. (#1883)
  • Generate the Table of Contents with Hugo rather than JavaScript. (#1884)

Appendices

Deprecations

  • Deprecate linking to events in rooms identified by alias, as per MSC4132. (#1823)

Spec Clarifications

  • Define 'Opaque Identifier Grammar'. (#1791)
  • Define common cryptographic key representation. (#1819)
  • Move size limits for user, room and event IDs into the appendix and clarify that the length is to be measured in bytes. (#1850)

Internal Changes/Tooling

Spec Clarifications

  • Update the spec release process and related documentation. (#1759)
  • Fix npm release script for @matrix-org/spec. (#1765)
  • Formatting fixes in CONTRIBUTING.rst. (#1769)
  • Improve rendering on mobile devices. (#1770, #1771)
  • Fix the OpenAPI definition of the security schemes. (#1772)
  • Simplify uses of resolve-refs partial. (#1773)
  • Fix Hugo warnings. (#1775, #1788)
  • Fix github-labels.rst. (#1781)
  • Update dependencies. (#1786, #1803, #1804)
  • Solve allOf recursively in OpenAPI and JSON Schemas. (#1787)
  • Fix property type resolution in render-object-table partial. (#1789)
  • Factor out common definition of Tag type. (#1793)
  • Update the version of Hugo used to render the spec to v0.124.1. (#1794)
  • Add support for pattern formats for patternProperties. (#1796)
  • Clean up unnecessary allOfs in OpenAPI definitions. (#1797)
  • Show information about "Additional Properties" in object tables. (#1798)
  • Fix anchors for schemas under oneOf. (#1799)
  • Use reference to OneTimeKeys schema in OpenAPI definitions. (#1800)
  • Do not use the title of objects containing only additionalProperties or patternProperties. (#1801)
  • Add anchors in definition shortcode. (#1802)
  • Set python version for the Towncrier CI job. (#1805)
  • Replace set-output with environment files in CI. (#1806)
  • Render response headers. (#1809)
  • Use patternProperties in more places with supported formats. (#1813)
  • Add support for rendering string formats. (#1814)
  • Refactor the OpenAPI definitions of the content repository endpoints. (#1822)
  • Clean up pull request template. (#1831)
  • Do not add empty arrays to examples. (#1849)
  • Generate the Table of Contents with Hugo rather than JavaScript. (#1851, #1885)
  • Fix syntax errors in the spec release issue template. (#1856)
  • Use environment variables for Netlify build job. (#1865)
  • Render added/changed in info on request and response content types. (#1876)
  • Fix validation errors in generated HTML. (#1880)
  • Ensure most generated HTML IDs are unique. (#1881)
  • Allow to specify a prefix for generated HTML IDs of API endpoints. (#1882)

This Week in Matrix 2024-06-14

14.06.2024 19:00 — This Week in Matrix Thib

Dept of Status of Matrix 🌡️

Thib (m.org) reports

The Matrix.org Foundation and the Matrix Community Summit team are pleased to announce The Matrix Conference!

It is meant to be the place where the ecosystem at large meets. From individual hackers to organisations working with Matrix, from the Governing Board to public sector representatives we expect the whole community to gather for an exciting event that shapes the future of Matrix!

A particular thank you to HarHarLinks, Yan and Nadine as well as Plain Schwarz who all contributed to the organisation.

Find all the details about the conference and the venue at https://2024.matrix.org and submit your proposals at https://cfp.matrix.org.

Continue reading…

Announcing The Matrix Conference

12.06.2024 13:15 — Conference Thib
Last update: 18.06.2024 15:00

The Matrix.org Foundation is happy to launch the first Matrix Conference, this September 19th to 22nd in Berlin, Germany, at Mitosis Labs! Click on the picture bellow to learn more!

An picture with an abstract background made of lines, and the matrix conference logo that looks like the regular matrix logo.

Building on the success of the Matrix Community Summit, the Foundation is joining forces with the Summit team to open the conference to wider audiences. This is the event that policy makers, public and private sector leaders, open source enthusiasts, and technologists attend to share knowledge and learn what’s next in the decentralised and secure communications sector.

This is THE gathering place for hackers, project managers, digital sovereignty leaders and innovators. With several tracks covering everything from sovereignty & collaboration in the public sector to digital rights and advocacy, all profiles will find content for their interest. Outside of the tracks there will be plenty of time and space to get to know others from the ecosystem and exchange ideas.

Continue reading…

This Week in Matrix 2024-06-07

07.06.2024 19:00 — This Week in Matrix MTRNord

Matrix Live S09E30 — The Account Migrator

The Foundation is hard at work to let you move your Matrix account around. Tadzik walks us through a pragmatic solution to several problems we have.

Foundation

Policy and Regulations blog series

Denise [away] says

we're starting a policy and regulation blog series over on the Foundation's blog. Over the next few months I'll be covering various pieces of legislation that are already in place, as well as incoming regulation, and what it all means for Matrix.

https://matrix.org/blog/2024/06/regulatory-update/

Dept of elections 🗳️

Josh Simmons (he/they) says

The votes have been counted! Introducing the first elected Governing Board of the Matrix.org Foundation 🎉

Thanks to everyone who ran and everyone who voted, and congratulations to those who have been elected!

This is a huge milestone for Matrix, and now we can tackle the challenges we face with greater community involvement: https://matrix.org/blog/2024/06/election-results/

Continue reading…

Policy and regulation update 2024: Matrix and the GDPR

06.06.2024 07:00 — Foundation Denise Almeida

If you have been following the matrix.org blog for some time, you will know that we’ve never been ones to shy away from complex topics like public policy and its impacts on Matrix. With this blog post series, our aim is to introduce a more regular cadence to our regulatory updates and to be more transparent about where we are focusing our efforts in this area.

Each blog post in the series will focus on a given theme or piece of law, as well as its relevant jurisdiction. We will start this series by taking a deep dive into EU regulation, starting with the General Data Protection Regulation (GDPR). Future blog posts in the series will cover the digital services package (DMA and DSA), the incoming CRA and the highly controversial CSAM regulation. These will be followed by a series dedicated to the UK, particularly UK applications of European law such as the GDPR and ePrivacy directive, as well as the Online Safety Act and the IPA amendment bill. Finally, we will conclude the series by looking across the pond and diving into the Cloud Act, as well as KOSA and other existing proposals.

Continue reading…

Introducing our first elected Governing Board

03.06.2024 19:00 — Foundation Josh Simmons

It is an honor and a pleasure to unveil the election results and introduce the first elected Governing Board for the Matrix.org Foundation!

Congratulations to the winning candidates, we thank you for your willingness to serve the community. We’re also grateful to everyone who threw their hat in the ring, and hope that the candidates who did not get elected consider running again in the future – noting that we have an election of Gold, Silver, Individual, and Associate Members scheduled for next year.

Thanks also to all of the people who cast ballots in the election, and to everyone who asked questions along the way! We learned a lot in this first election process that we look forward to incorporating into the next one.

The level of engagement with the process was a very encouraging sign for the health of the Matrix ecosystem, and we’re proud to have had 100% voter turnout in all but one constituency.

Read on to see who is on the Governing Board, a brief discussion of next steps, and reflections on some of the work that remains to improve representation.

Continue reading…

This Week in Matrix 2024-05-31

31.05.2024 19:00 — This Week in Matrix Thib

Matrix Live

Dept of Spec 📜

Andrew Morgan (anoa) {he/him} reports

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/proposals.

MSC Status

New MSCs:

MSCs in Final Comment Period:

Accepted MSCs:

  • No MSCs were accepted this week.

Closed MSCs:

  • No MSCs were closed/rejected this week.

Spec Updates

This last week's focus was primarily spent on ensuring MSCs are moving towards Final Comment Period (FCP; the step before an MSC is accepted) at a healthy pace. Below are the MSCs which were on Tuesday's pings, and their current status:

  • MSC3916: Authenticated media
    • Needs just 2 more checkmarks SCT members to enter FCP (started the week needing 6).
  • MSC4138: More methods in CORS
    • Needs 1 more checkmark from the SCT to enter FCP (started the week needing 1).
    • Currently blocked on clarified text regarding its scope.
  • MSC2781: Removing reply/edit fallbacks
    • Needs just 2 more checkmark from the SCT to enter FCP (started the week needing 2).
    • Currently blocked on some clarified behaviour and process ordering.
  • MSC2867: Marking rooms as unread
    • Needs checkmarks from 3 more SCT members to enter FCP (started the week needing 3).
    • Currently blocked on ensuring that this addition to spec doesn't make notifications more complicated.

Client and server developers are encouraged to review these MSCs in particular because they're on a path to being included in the next spec version. This doesn't have to mean implementing them, but if there's something which doesn't sound quite right in the proposal text then leave a comment on the diff to raise a concern. The SCT will take these comments into consideration as the MSC enters FCP - the 5 day countdown to ensure any last comments are considered before the proposal is accepted into the spec as stable.

Next week's focus will largely be the same as usual: we'll focus on unblocking Matrix 2.0 MSCs/features as a priority, and MSCs which require more checkboxes to enter FCP will be raised for prompt review. We're always on the lookout for MSCs ready for FCP, but can sometimes miss something - let us know in #sct-office:matrix.org if it looks like something is ready to go but hasn't caught our eye yet.

Continue reading…

This Week in Matrix 2024-05-24

25.05.2024 00:00 — This Week in Matrix Thib

Matrix Live

Dept of elections 🗳️

Josh Simmons (away, back May 9th) says

Voting continues in the Governing Board elections, and we're close to having 100% turnout in every constituency except Individual Members! Check your email inbox for an email from [email protected] and please cast matrix-orgyour ballot today.

Separately, we're very pleased to welcome NeoChat and Cinny as our latest Ecosystem Members!

Do you have a community that's centered around Matrix? Or an organization that is aligned with our mission? Perhaps you're part of a company that profits from the use of Matrix. Whichever describes you, become a member today. Your involvement helps us ensure we get representative feedback and the financial support we need to continue stewarding the specification.

Continue reading…

This Week in Matrix 2024-05-17

17.05.2024 19:00 — This Week in Matrix MTRNord

Matrix Live

Dept of elections 🗳️

Josh Simmons (away, back May 9th) announces

Voting has started for the Governing Board elections and runs till May 31 – but don't delay, vote today! 🗳 Huge thanks to all of the nominees who have thrown their hat in the ring.

All eligible voters should have received an email from the election system. All of the results will be published on the blog on June 3. Read our announcement post or visit our election center for more info.

Dept of Spec 📜

Andrew Morgan (anoa) says

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://spec.matrix.org/proposals.

MSC Status

New MSCs:

MSCs in Final Comment Period:

Accepted MSCs:

Closed MSCs:

Spec Updates

As an early heads up, Trust & Safety at the Foundation is working on an important update to Matrix, MSC3916 - Authenticated Media. This change will mean that all clients (and servers) will need to present a valid access token in their Authentication header to access media - which is critical to ensure that URLs are only visible to the correct users, and prevents abuse of Matrix for hosting binaries. More details will be published as we work to get everything released - we wanted to get the information out there as early as possible in the meantime. Let us know if you have any questions.

Matrix.org plans to freeze unauthenticated media endpoints within a couple of months after the spec release, which is expected in the next few weeks. "Freezing" means that media uploaded or cached before the freeze will remain accessible via unauthenticated endpoints indefinitely, but any media cached or uploaded after the freeze will require authentication. The unauthenticated endpoints will be deprecated but will still serve old media on matrix.org.

To ensure a smooth transition, we encourage you to start testing against the unstable endpoints and unreleased server builds. The changes for Synapse are being developed here, and for MMR here. Both are expected to release their changes soon. Once MSC3916 passes FCP, stable endpoints will become available. While releasing unstable support to users isn't required, having patches ready will help speed up the rollout.

We know this is a quicker rollout than usual, but with your help, we can improve user safety and security across the ecosystem. Most clients should find this update straightforward, but if issues are encountered, please reach out in #matrix-client-developers:matrix.org or on the MSC discussion. The team is monitoring the room to help clients adopt the change.

Web browser clients might face the most challenges, given the need to specify an Authentication HTTP header on media requests, so reviewing this pull request and its dependencies could provide useful implementation insights.

Thank you for your support. If you have any questions, let us know. We look forward to a smooth transition with minimal user-visible impact 🙂

Continue reading…

Voting has begun in the Governing Board elections

17.05.2024 00:00 — Foundation Josh Simmons

Voting has started for the Governing Board elections and runs till May 31! This is our first election and we are very excited. All of the results will be published here on the blog on June 3.

You can learn about all of the candidates on our 2024 election page. All eligible voters should have received an email from OpaVote, the election system we have chosen for this year’s elections.

If you believe you are eligible to participate but have not heard from us, first check the inbox and spam folders of the email address you have on file with us (such as through Donorbox or Patreon). Please email us if you have any questions.

Continue reading…