This Week in Matrix 2020-06-26

Matrix Live 🎙

It's demos this week!

• Riot X video calls (valere)
• Room list (TravisR)
• Notifications - (Michael (t3chguy) & richvdh)
• E2E Onboarding (Dave & Nad)
• Modular SAML support (ChristianP)

Dept of Status of Matrix 🌡

Merch Returns!

Yes it's back! For the first time in months we are shipping branded IRL-items featuring Matrix branding. Head to The Shop to stock-up.

Dept of Spec 📜

anoa announced:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

MSC Status

Merged MSCs:

• MSC2631: Add default_payload to PusherData

MSCs in Final Comment Period:

• MSC2604: Accept device information for the login fallback endpoint (merge)

New MSCs:

• [WIP] MSC2654: Unread counts

Spec Core Team

In terms of Spec Core Team MSC focus for this week, unfortunately our three MSCs from last week ( MSC2366 (verification flows), MSC2403 (knocking), and MSC2630 (SAS security)) are still pre-FCP. Most of the team has been quite busy with implementation for the past few weeks. Instead of advertising those 3 MSCs again, we're going to switch the focus to "implementation" for a bit until we're freed up again.

So the focus for this week is: MSC implementation work. However, this should not discourage any MSC authors from responding to MSC feedback in the meantime 🙂

Dept of GSoC 🎓

Ruma GSoC

devinr528 reported:

This week in the ruma/matrix Google Summer of Code project, ruma-events was made ready for use! After adding stripped and sync event generation to the event_enum! macro there were only a few small tweaks needed to try it out in some dependent rust crates. I spent a few days converting matrix-rust-sdk to use the ruma-monorepo. Since ruma is used on both client and server-side, I also opened a PR to update Conduit, a homeserver implementation written in Rust. To test that everything worked together, I updated rumatui, my command-line client written in rust.

Then I could test that Conduit sent, and matrix-rust-sdk received the new ruma events successfully. While updating, I felt the pain of not having accessor methods for the Any*Event enums to get at the event fields held within. I have opened a pull request to add the generation of these methods to the event_enums! macro. Hopefully, the Conduit and matrix-sdk PR's will be merged and the ruma monorepo can be tested in the wild!

matrix-ircd GSOC

karlik told us:

Moved matrix protocol, irc protocol, and bridge module to futures 0.3. Converted some utility functions to new futures, and updated the http implementation to use the standardized Hyper library instead of using a custom http implementation.

go-neb

nikofil said:

• Created PR for enabling e2ee across all services! This required a few changes to how the bot client is initialised, as well as changes to all services to use the new functionality instead of directly sending messages to a room. https://github.com/matrix-org/go-neb/pull/324

• Added code from another of tulir's projects to Mautrix to allow storing the crypto material (olm / megolm sessions, accounts etc.) in a SQL database, adding a second way besides using Gob storage. https://github.com/tulir/mautrix-go/pull/10

• Future plans are to work on the library itself to add any features that might be missing.

HTML embeddable chat rooms

arnav-t told us:

This week I've worked on the following features for the project "HTML embeddable chat rooms" under GSoC '20:

Added support for signing-in

Via sign-in popup

The user can enter their credentials into the modal dialogue box.

Via postMessage interface

iframe.contentWindow.postMessage({
    cmd: 'login',

    args: {
        user: 'username',

        passwd: 'password'
    }

}, origin);

Arguments:

args (object):

• username (string) - Username

• passwd (string) - Password

Response:

{

    status: 'success', 
    message: 'Attempting sign in...'

}

Added support for default avatars

Using the same design as Riot. However, I am just using flexbox and text instead of drawing on an HTML5 canvas. The HSL color is generated by hashing the user ID.

Quoting and replying

Added on-hover buttons and reply popup above message composer similar to Riot web. Quoting prepends the message (quoted) to the message composer similar to Riot web and replying would also work similar to Riot web. It will be fully functional once markdown parsing is added.

Guest access

The client now supports guest access. If no access token or user ID is provided in the configuration file, the client attempts to register a guest account on the home server. Room contents are viewed using peekInRoom.

If the guest attempts to send a message, only then joinRoom is called to avoid spam (each page load would lead to a guest joining the room).

Nheko

Chethan told us:

SAS Verification is completed and works !

Supports both Emoji and Decimal Verification Only uses curve25519-hkdf-sha256 as key_agreement protocol

Brand new UserProfile written entirely in qml. https://github.com/Nheko-Reborn/nheko/pull/203

Mtxclient

Updated with needed APIs for SAS verification.

https://github.com/Nheko-Reborn/mtxclient/pull/32

Opsdroid

tyagdit reported:

Port to Matrix-nio

• The matrix connector is now using matrix-nio!

• Bots can send and receive texts, images, files as before

• No change in configuration required, just update opsdroid and run as before

• Encryption is not yet implemented

• You can check out the PR here

Enabling E2EE

• Sending and receiving encrypted texts, edits, replies works just as normal messages

• Images and files are being worked on

• User shouldn't have to change much about their configuration to use encryption

• Some testing is due

• Check out the progress here

Dept of Servers 🏢

Dendrite

Dendrite is a next-generation homeserver written in Go

kegan reported:

This week has been mainly about testing to ensure that all the progress over the past few months is kept up-to-date and correct. We've also added a few features in order to get certain sytests working. To that end:

• Invites can now be declined over federation and they will be reflected in /sync responses.

• Errors encountered when joining a room over federation are now sent back to the client.

• Errors encountered when accepting an invite over federation are now sent back to the client.

• Dendrite will now check server names meet the server name grammar in the specification.

• A bug which caused client-api-proxy to not actually proxy correctly has been fixed, thanks @fantashley !

• /send now abides by the limits in the specification: 50 PDUs / 100 EDUs.

• The docker-compose scripts now include appservice_api, thanks @fantashley !

• Sending invites over federation will now fall back to v1 if v2 fails with a 404.

• Dendrite now implements room.timeline.limit completely (in both in-line and stored filter formats).

• Dendrite now sets the limited flag on /sync responses correctly.

In addition, we now have support for collecting code coverage output from SyTest. This indicates we are testing roughly 70% of the Dendrite codebase. The remaining 30% are hard to reach via integration tests (e.g database failures, communication problems between internal APIs).

Spec compliance:

• Client-Server APIs: 45%, up from 40% last week

• Server-Server APIs: 50%, up from 38% last week

In total, we've made an additional 45 sytests pass this week.

Mascarene

Nico reported:

Mascarene v0.2.0 has been released.

This version implements a minimal set of client API endpoints to work with Riot and allow user registration,

room creation, invitation and messages sending on the same running instance (no federation support).

Docker image are also available. See this documentation

for rough installation instructions.

Join us at #mascarene:beerfactory.org

Conduit

Conduit is a Matrix homeserver written in Rust https://conduit.rs

timo told us:

This week I finished the cross-signing PR:

To try it out yourself, open Riot (preferably https://riot.im/develop) and join the <https://conduit.koesters.xyz:14004> homeserver.

Thanks to everyone who supports me on Liberapay or Bitcoin!

Synapse

Neil told us:

This week we’ve been working on further improvements to event persistence and ironed out a nasty bug where an unusually long state resolution could block the reactor overall and impact send times. We seem to have got to the bottom of this and m.org has improved a lot as of today.

More generally we’ve been trying to characterise matrix.org performance so that we can continue to improve over the coming months.

We are going to focus on:-

• Client send event

• Outbound Federation Latency

• Inbound Federation Latency

• Room joins

As well as tracking the CS API generally.

We are using apdex with a satisfied limit of 250 ms and a tolerating limit of 1000 ms. By the end of the Summer we will aim to hit an apdex score of 0.9 for each area.

For instance here is Federation Send Event Apdex graphed overtime. You can see that we are averaging about ~0.8 currently, so plenty to keeping us busy!

Next up will be to work on Outbound Federation Latency

Aside from that we’ve been working hard on the upcoming Notifications improvements. Mapping the push rules to the demands of the UI has been challenging and we’ve been through several iterations. If you’d like to learn more take a look at Michael and Rich’s explanation in this week’s Matrix Live. Rich’s presentation has Sheltie pictures #justsayin’

Dept of Bridges 🌉

It's a Tulir kind of week.

mautrix-whatsapp

Tulir told us:

The WhatsApp bridge got a bunch of improvements this week, such as:

• Matrix -> WhatsApp gif bridging

• Bridge status notices if your phone battery is low

• A command to get group invite links and another command to join groups using invite links

• Matrix <-> WhatsApp bridging of most membership changes

mautrix-facebook

Tulir reported:

The Messenger bridge mostly got bugfixes, but also Matrix->Facebook bridging of all media and location messages

mautrix-telegram

Tulir offered:

The Telegram bridge now supports logging in by scanning a QR code, although it requires using the master branch of Telethon instead of a release. I also fixed bridging captions in file messages, so they're now bridged as separate messages like with images

Half-Shot bridges roundup

Half-Shot offered:

Hey all, I've got several releases to talk about this week!

First, matrix-appservice-slack has been updated to 1.4.0 with several quality of life changes such as automatically setting the bot profile on startup, supporting logging out of slack accounts and adding a health checkpoint. There are also quite a few bugfixes so make sure you update.

matrix-appservice-irc was also bumped to 0.18.0 with the headline features being Node 14.x support.

We've also shipped 0.19.0-rc1 which has stopped support for Node 10.x. Why? Because we're adding worker support to the bridge! This release starts to make use of the new(ish) worker_threads feature so that we can dish out processing to separate threads (running their own UV event loops, for node enthusiasts). The first thing to be workerized is metrics, so that metrics may still be reported should the bridge become saturated, but we plan to split out more work as things progress.

Finally matrix-appservice-bridge got a few fixes to support our new worker land, as well as being updated to support matrix-js-sdk 6.0.0. You can checkout the changes for 1.13.1 here.

Dept of Clients 📱

Fluffychat

sorunome offered:

Fluffychat Version 0.15.0 is released, and should be available in the Play Store, on F-Droid and in IOS Testflight soon! This makes Fluffychat the first non-Riot matrix client that supports Cross-Signing.

Features:

• New room list app bar design

• Chat app bar transparent

• Implement web file picker

• Minor design and UX improvements

• Implement Cross Signing

• Restore keys from online key backup

• Added translations: Czech, Spanish, Slovakian

Changes:

• Show presences of users sharing a direct chat

• Big refactoring

Fixes:

• Various fixes, including e2ee fixes and olm session recovery

Pattle: version 0.18.0

Wilko said:

A new version has been released and will soon become available on Google Play, TestFlight and F-Droid!

Changes

• You can now play videos!

• The main public address (room alias) of chats are now shown in chat's details

• The time of sending is now always show on the right side for images

• Notifications are grouped nicer, there's now one notification for Pattle, grouped by chat

• Fix sending an image crashing the app

• Fix urls not being accounted for in time placement (#132)

• Fix chat input not scrolling

• Fix multiple issues regarding showing notifications

• F-Droid: All left-over proprietary code is now removed, thanks to Bubu for notifying me!

Get Pattle

• Android

  • F-Droid
  • Google Play

• iOS (TestFlight)

Get involved

• Follow development: #app:pattle.im

• Issues or suggestions: Post them here!

• Donate: If you want to support Pattle, you can donate!

Mirage

miruka* reported:

0.5.2 is now out:

Added

• Sessions/device list: you can now inspect, rename and manually verify

  your devices from the account settings page. The interface is still work in progress, keyboard navigation and signing

  out sessions will be added in a next version.

• Re-add client-side unread/highlight room indicators.

  If your account has push notifications disabled, which precise cross-client counters depend on, the local indicators will be used as fallback.

• Support the MIRAGE_CACHE_DIR environment variable to override where

  files and thumbnails are downloaded

• A bunch of theme additions, check the full log

Changed

• Overhauled account settings to match the design of other tabbed pages.

  The horizontal layout design has been removed due to complicated code and being impossible to extend without breaking it.

• The display name field in account settings is now colored,

  preview your new display name's color as you type

• For rooms without image avatars set, the room settings's avatar color now

  responds to the name field as you type

• Overhauled scrollbars:

  • Now match the Mirage theme and much better visibility

  • No more right margin for the timeline's bar

  • Minimum height to prevent the bar from becoming impossible to grab

• Use brighter text for room names of rooms that have unread messages

• Buttons, tabs, text fields and areas now have animated bottom borders

  to represent keyboard focus instead of being highlighted like when hovered

• Text fields and areas can now have rounded corners, following the theme

• Tabbed pages (Sign In, Add Chat, etc) can now be swiped left and right

• Popups can now be scrolled when their content is bigger than the

  window's height

• Replace most generic checkmark icons for apply buttons in popups

• Pressing escape in forms will consistently trigger corresponding

  cancel buttons

Fixed

• Various fixes, see full changelog

Nheko

Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at

Nico (@deepbluev7:neko.dev) offered:

• Most of our time this week was spent trying out device verification with Chethan. It's fun and I could finally file bugs in Nheko against someone else!

• We fixed some issues that should hopefully make text in the timeline less blurry again.

• Fixed a bug which cause some clients like fluffychat to break in E2EE rooms (we accidentally sent a null relation when not replying...)

• Fixed an issue, where Nheko didn't verify the format of html formatted messages correctly, causing it to render messages in a way which wasn't compliant to the spec.

• Thanks to the work of a Pirate and his friends, current versions of Nheko should be available in backports for Debian Buster once again!

Riot Chat for Nextcloud

Gary Kim announced:

Riot Chat for Nextcloud 0.5.0 The new version updated the Riot.im version to 1.6.6 and added the ability for admins to set their own custom config for Riot rather then using the settings interface in addition to a few bug fixes.

Riot Chat for Nextcloud allows individuals and organizations with a Nextcloud instance to easily set up and configure their own Riot instance with just a few clicks on a web interface without the need to write a config file.

Join the development Matrix room at #riotchatfornextcloud-general:garykim.dev. Check out the source code here.

Riot-iOS

Manu told us:

This week, we completed UX for cross-signing and secure backup. We made associated settings but we still need to polish them.

Sygnal and the new push extension have been updated to match Apple requirement and our privacy concerns. Events content are no more sent anymore. We started to implement the new room notifications settings UI.

RiotX

benoit told us:

This week we were mainly working on cross-signing, room settings, VoIP stabilization and performance improvement.

Riot-Android

benoit reported:

We will publish at the beginning of next week a beta version of the migration to RiotX codebase on the beta channel of the PlayStore, to be able to ensure the migration works fine, before release it to production.

Dept of SDKs and Frameworks 🧰

Matrix Dart SDK: v0.33.0

Wilko told us:

• Support custom events using RawRoomEvent and RawStateEvent

  Check the README for details.

• Add ability to register using Homeserver.register

• Add VideoMessageEvent

• Add Room.canonicalAlias and Room.alternativeAliases

• Add format field to TextMessage (thank you Cyril!)

• Remove equatable dependency

Get it on pub!

Net::Matrix::Webhook

joepie91 discovered this project, presented yesterday at Conference in the Cloud:

Net::Matrix::Webhook implements a webhook, so you can easily post messages to your matrix chat rooms via HTTP requests. It uses IO::Async to start a web server and connect as a client to matrix. It will then forward your messages.

matrix_scrape_emoji_translations script project

sorunome announced:

Soru wrote a quick program that scrapes riot-web, riot-x and riot-ios for translations of the emoji names for emoji verification and combines them all into an easily-readable json file, so that other client developers can use it. Since all three riot versions have a different set of translations, it might also be helpful for them. You can find the source code along with the outputted json files here.

Dept of Bots 🤖

"Scandinavia and the World" comic bot

Tulir offered:

To add to my existing xkcd and CommitStrip maubot plugins, there's now a similar plugin for Scandinavia and the World comics: https://github.com/maubot/satwcomic

Like the other bots, you can self-host it, use my instance (@satw:maunium.net) in your own room or simply join #satwcomic:maunium.net to automatically get the latest comics in Matrix.

matrix-webhooker

kinta announced:

A bot that will allow to room administrators and moderators to generate room custom commands (in a similar way how telegram commands are thought).

When they are invoked it will post the message event object data and a predefined context object along with a token to a custom url. More information in: https://gitlab.com/communia/matrix-webhooker

Matrix webhook

kinta told us:

https://www.drupal.org/project/matrix_webhook

Drupal module to receive links shared from a matrix room. Get links provided by a matrix bot in a room. With this module one can get media from a matrix own bot to any drupal site. Once installed and bot is created through matrix-webhooker bot. A blog entry about it (in spanish) in https://planet.communia.org/content/enlaces-desde-matrix

Dept of Interesting Projects 🛰

Pollvis - new poll visualiser project

We have a couple of conferences coming up who are planning to use Matrix as part of their offering, much as we do with Open Tech Will Save Us. To help add some more features, I created an MVP "poll visualiser", which watches a room and works in tandem with the poll-bot from Brendan Abolivier . This project is still at the beginning, but might be interesting to some! Find the code at https://github.com/benparsons/pollvis.

Dept of Ping 🏓

Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.

Rank	Hostname	Median MS
1	fairydust.space	331
2	services.pyrahex.com	537
3	heitkoetter.net	604
4	nitro.chat	646.5
5	mchus.pro	660
6	privacytools.io	671
7	matrix.vgorcum.com	727.5
8	eiselecloud.de	740
9	aruiz.io	855
10	neko.dev	923.5

That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!