Open Tech Will Save Us 🎙

Second edition of OTWSU took place this week! Chance to watch it if you haven't already, or (pro-tip), watch it again.

  • Mike Hoye from Mozilla talks about "Verbs in the fediverse"
  • Eugen (@[email protected]) discusses "how we keep mastodon.social running at scale" as a Q&A
  • ... and a new VIP tour of Dendrite, the next-gen Matrix Homeserver, from our own neilalexander and Kegan!

Dept of Status of Matrix 🌡

Bundeswehr switching to Matrix

Oleg said:

German army reported in December 2019 their intentions for Matrix.

Now they decided to replace Stashcat with Matrix in this autumn.

Around 50'000 account are planned.

Riot will be used as a Matrix client. The E2EE by default and device Cross-Signing will be extensively used.

Full article (in German): https://www.heise.de/newsticker/meldung/Bundeswehr-setzt-kuenftig-auf-Matrix-als-Messenger-4719474.html

Mozilla are happy with their deployment!

About a year after this tweet, four months after deployment of our open source @matrixdotorg instance, community participation in the Mozilla project has increased from 4x to 10x across all the projects that previously relied on IRC.https://t.co/E5HRsfHkzk

— mhoye (@mhoye) May 12, 2020

For more from Mike check out the OTSWU recording above, he's on first.

Dept of Spec 📜

Spec

anoa announced:

Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.

MSC Status

Merged MSCs:

  • No MSCs were merged this week.

MSCs in Final Comment Period:

New MSCs:

Spec Core Team

In terms of Spec Core Team MSC focus for this week, MSC2526 has entered FCP. We'd like to keep the E2E MSCs in flight in focus, but are also conscious that room v6 is coming up. We're sticking the MSCs for the latter alongside the existing E2E ones in the focus queue, for a total of 5. Those MSCs are: MSC2366 (verification flows), MSC2399 (UISI messaging), MSC2540 (canonical JSON validation), MSC2209 (notifications auth rules), and MSC2240 (room version 6).

Dept of Servers 🏢

Dendrite / gomatrixserverlib

Dendrite is a next-generation homeserver written in Go

Neil Alexander announced:

  • Dendrite now implements /get_missing_events, which improves inbound federation reliability significantly

  • State resolution v2 implementation in gomatrixserverlib now uses significantly less overall memory and performs far fewer allocations

  • A skeleton keyserver component has been added - it returns no keys for now but that is enough to satisfy Riot when creating DMs

  • Canonical alias is now set if needed when the room is created

  • Reduced the number of HTTP 500s returned by the media API

  • Sync API topology logic and tokens have been reworked

  • Server keys are now cached in memory using an LRU cache, regardless of database or fetchers used

  • Federated joins now attempt all listed servers more reliably

  • Federated joins in response to an incoming invite should now work, particularly in cases where the room had previously been joined and left

  • Some Prometheus statistics are now available for the LRU caches

  • Sync API storage layer has been deduplicated quite a bit

  • Display name and avatar URL are now sent in join content again after a regression

  • Fixed a bug in the membership updater when matching added and removed state

  • Fixed a bug where a response to a federated device query would not contain all required fields

  • Fixed a bug where /sync could panic after joining a room

Get a real demo of Dendrite from Open Tech Will Save Us #2

Conduit

Conduit is a Matrix homeserver written in Rust https://conduit.rs

timo said:

This week I worked on presence and end to end encryption, but they are not working yet.

Meanwhile, Valkum got sytest integration working! This means each time a new commit is pushed to a branch, CI automatically picks up on that and lets a lot of tests run on the homeserver - the same tests that synapse uses. Currently we have 8% of all tests positive on Conduit.


Client-Server APIs: 8% (56/667 tests)

*------------------

  Registration             :  53% (17/32 tests)
  Login                    :  26% (5/19 tests)

  V1 CS APIs               :  10% (3/30 tests)
  Misc CS APIs             : 100% (1/1 tests)

  Profile                  :  33% (2/6 tests)
  Devices                  :  17% (2/12 tests)

  Presence                 :   0% (0/10 tests)
  Create Room              :  31% (4/13 tests)

  Sync API                 :   1% (1/85 tests)
  Room Membership          :   0% (0/15 tests)

  Room State APIs          :   0% (0/14 tests)
  Public Room APIs         :   0% (0/7 tests)

  Room Aliases             :   0% (0/15 tests)

At the same time, MTRNord tried to integrate opentracing into Conduit and Paul worked on a docker file to make installation easier.

Thanks to my supporters on Liberapay.

See the bottom of this post for some more interesting numbers!

Synapse

anoa offered:

Synapse v1.13.0rc2 shipped this week with the following notable improvements:

  • SSO support in User-Interactive Authentication sessions

  • Cross-signing device list bug fixes

  • The ability to run in worker mode using redis as a backend (which will soon be considered the default transport for Synapse workers)

  • Some admin api improvements and additions

...as well as all the usual bug fixes and especially performance improvements. See the changelog for the full details and please test if you can! A full release is expected early next week.

Speaking of next week: work will involve getting room version 6 ready to ship, alongside even more performance work - especially moving event processing off the master process (the code for which is already running successfully on Half-Shot's own homeserver)!

Dept of Bridges 🌉

mx-puppet-steam

Icewind told us:

I've been working on a new puppet bridge for steam chat mx-puppet-steam, while I would still consider it to be in alpha state, most of the things needed for day to day chatting are functional.

It bridges both one on one and group messages, shows the proper name, avatar and online status of your frends and handles embedded images in one on one chats.

I have a docker image for it if that helps: https://hub.docker.com/r/icewind1991/mx-puppet-steam

mautrix-whatsapp

Tulir reported:

As promised last week, mautrix-whatsapp now has end-to-bridge encryption (currently on the e2be branch, will be merged to master soon™). It works mostly the same way as the other bridges.

Dept of Clients 📱

It's a big, big week for client news! We must take a deeper look at some of these projects soon!

Ditto v0.4.1 Released!

Annie offered:

What is Ditto?

Ditto is a mobile client for iOS and Android! It's available on TestFlight and Google Play (beta) - download link available on the website below.

What's New?

Added

  • Reactions support! 😄🎉

  • Add a visual hint if a message is sending or could not be sent

  • Send pending messages when the connection to the homeserver is back

Changed

  • Update to React Native 0.62.2

  • Major performance improvements!

Fixed

  • Redacted events have a proper message

  • Switched push gateway to https

Matrix-TV

Half-Shot said:

Link: https://github.com/Half-Shot/matrix-tv

I got really bored in lockdown, as we all have. I stared at my TV, and it stared back at me. While it's definitely a privacy nightmare to start developing apps for these things, I couldn't help myself. I got stuck in, and secured myself a LG Developer Account.

The LG set of smart TVs use WebOS, as like the name suggests, runs apps in it's special variant of Chromium. I was lucky to be running a recent OS and had access to Chromium 53! And NodeJS 0.12!! Sarcasm aside, there are some very troubling versions of WebOS with much older versions of the browser, so I was happy to be developing against something with strong ES6 support. Despite the absolutely awful onboarding experience, the WebOS SDK itself was pretty good. matrix-tv was written to be a general large screen matrix client, so it talks to the TV's own libraries through an abstraction layer :)

Anyway, this client is the culmination of a lazy Sunday afternoon's work. It features very basic support for viewing a room, but I hope to expand it at some point. It supports running in a web browser and the aforementioned OS, but support for more variants is easy to add. I'm looking into how to design for the large screen beyond making fonts bigger, because the unique constraints on memory and UI are fun challenge.

Nheko

Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at

Nico (@deepbluev7:neko.dev) announced:

  • Nheko now supports SSO. If your server uses SSO, you may want to test that, if it works for you, but I did successfully sign into the Mozilla homeserver.

  • We improved hinting on the login and registration pages, so it should be easier to enter correct credentials.

  • Replies now have a max height by default. If you want to see the full reply, you need to click on it to navigate to the reply.

  • Nheko now has a red unlocked padlock, when an unencrypted message is sent in an encrypted room.

  • red_sky has been hacking along on reactions. Displaying them is now in master, for sending he is currently reworking the emoji picker, so that it can be attached to the timeline more easily and it is easier to search, etc.

Nheko has it's own weblate btw, if anyone feels like translating a bit of stuff: https://weblate.nheko.im/engage/nheko/?utm_source=widget

Pattle: version 0.16.0

Pattle is a client written for Flutter using Dart

Wilko said:

A new version has been made available and will become available on Google Play and TestFlight!

Because of corona this might take longer than usual.

Changes

  • Huge performance improvements! All SDK processing now happens in a different isolate (comparable to a thread)!

  • Also huge start up time improvement! When opening the app you should see your chats in less than or equal to a second! (will vary per device)

  • You'll now see two checkmarks when a message is read!

  • Add Dutch translations! (thanks to Louis!)

  • You can logout now!

  • An error along with a stack trace is now shown when the initial sync fails. You can use it to inform me and make an issue!

  • For the launcher icons vectors are now used on Android when it's supported

  • Contributing is now easier, you don't have to sign a CLA anymore!

Get Pattle

Issues or suggestions

If you stumble upon any issues or would like to suggest features, please do so here!

You can login via GitHub and Gitlab.com!

Follow development

Follow development in #app:pattle.im!

RiotX

benoit reported:

We are making progress on Identity server feature implementation (PR in review). Also we are working to integrated call with WebRTC, integration manager (widget, Jitsi, sticker) and we are stabilizing the application, especially on the cross-signing part.

Expect a new corrective release at the end of the day!

gomuks

gomuks is a terminal based Matrix client written in Go. Source on GitHub

Tulir offered:

I was able to solve a few annoying bugs like message history disappearing and switching rooms taking time (it turned out that read receipts were being sent synchronously), so I decided to release gomuks v0.1.0.

fluffychat

krille said:

FluffyChat 0.13.1 is out with:

  • New status feature

  • HTML rendering of messages

  • Markdown support

  • Enhanced chat list design

  • New translations (Polish, Hungarian)

  • Lots of minor fixes and refactoring

2020-05-15-vsWn8-img_1589450046798.png

Nio

Nio is a client written with SwiftUI

kilian told us:

Since the last updates, there's been a few cool new features and changes added to Nio:

  • message text field supports multiple lines of input

  • edit badges instead of "(edited)"

  • markdown message rendering

  • spanish translation

  • blurhash support for incoming images (so find some Nheko-using friends! 😄)

  • sending images

  • and a few minor and internal fixes

Join us in #niochat:matrix.org for discussion and more fun 😊

Riot-Android 0.9.11

benoit reported:

Riot-Android: Version 0.9.11 (with Matrix SDK 0.9.34) has been released on the PlayStore, and should be available for all during the week-end. It contains fixes on the identity server feature (the "missing token" thing), and also support Jitsi call with configured domain.

Riot-iOS

Manu offered:

On the cross-signing side, we have been working on the implementation of SSSS (to recover all private keys from the HS) and on UI polishments.

On the maintenance side, the new push mechanism from Apple has been implemented but it needs more work to work properly.

Riot integration with NextCloud

Riot-web integration with NextCloud exists! Source at https://github.com/gary-kim/riotchat.

Dept of SDKs and Frameworks 🧰

matrix-spring-boot-sdk

Benedict offered:

This SDK is now on maven central. I also introduced support for implementing Application Services. The project can be found here: https://github.com/benkuly/matrix-spring-boot-sdk . It will be used by Matrix-SMS-Bridge, which only needs one telephone number. The latter will be released next week, when everything works fine (https://github.com/benkuly/matrix-sms-bridge).

Matrix Dart SDK: v0.31.0

Wilko said:

  • Add the ability to defer all processing to a different isolate

    When you use isolated: true on Homeserver.login or MyUser.fromStore, everything will be handled for you and the SDK works the same.

    This only works on platforms with the Dart VM (not web).

  • Updaters are now globally registered per MyUser id.

    This means for now only one Updater per MyUser is allowed. Updaters are created when a MyUser instance is retrieved via Homeserver.login or MyUser.fromStore

  • Add ability to specify what rooms to load from store when retrieving user

  • The previousBatch param from Room.copyWith is removed (wasn't used)

  • Add ability to override context in MyUser.copyWith and MyUser.merge

  • Updater and Homeserver are less coupled now

  • When a Context is changed with copyWith, all child objects' contexts are also changed

  • Fix bug in read receipt parsing (thanks to MatMaul!)

  • Sync errors are now pushed to the updates stream

  • Fix access token not being passed to logout API

  • Add type to RequestUpdate, so you can see which type of call

    the update was caused by

  • Moor has been updated

  • And other minor fixes!

Get it on pub, and see the source here!

Dept of Bots 🤖

JupyBot - First proof of concept

rdi said:

What is JupyBot

The purpose of the bot is to provide a link with Jupyter kernels running either on your local computer or somewhere else on a server. You can access your kernels through the bot with the chat interface. The kernels will answer to your commands through the same interface.

Since this is still a proof of concepts, the number of features is limited (command line text, string and pictures are the only output formats supported. No widget, no funny javascript) and they are few safeguards against infinite loops and other entertaining bugs.

TODO

This is right now just a proof of concept. We want to see if such a bot can be useful for people and how. What features would be interesting, what can be integrated in the present clients,... Therefore, in the short term, we want to focus on the possibility of easy testing for users and easy contributions for people who want to add their ideas. On the mid- and long term, a lot of ideas can be realized; it will depend on the feedback. These are just some examples. Short term:

  • proper error catching

  • fallback for non supported Jupyter formats on riot.im (i.e. a lot)

  • testing on termux on a tablet or smartphone

  • solve the problem with nio-matrix and upload in chunks of images.

  • Dockerized version

Middle term:

  • use of opsdroid as a framework

Longer term:

  • local server integrated to locally command the bot service.

  • make full use of jupyter capabilities with a client suited for it (for instance a Jupyterlab plugin)

2020-05-15--q5mx-screenshot1.png

2020-05-15-0k1yx-screenshot2.png

snowstorm-matrix

lub reported:

I made a little bot that scrapes https://news.blizzard.com/en-us , because they've disabled their RSS feeds for some time now. It regularly scrapes all posts of a configured category and sends messages for new articles to all rooms the bot is a member of. It works like a RSS bot, except the source is a website instead of a RSS feed.

In the readme there are also some public instances, which you can just invite instead of hosting your own bot.

https://gitea.lubiland.de/lub/snowstorm-matrix

midori jabbot matrix.org

Torrey Searle told us:

I've added matrix support into a chat bot, you can see the chat bot in action at #midori-dev:hackerspaces.be and it's project page is here https://github.com/midoricorp/jabbot.
it's main highlight is anybody can add new commands to it in chat using its scripting language
any any existing bot command can be used as a function in new scripts you write
The scripting language to add new commands can be found here https://github.com/midoricorp/script
you can do things like
!script print "HELLO world";
or make a new command
!script sub hello { print "HELLO World"; }
that would result in a new command !hello being made
print also has 2 output stream, one for rich format and one for basic formatting
!script print "Hello World!"; print HTML "Hello <b>World</b>";

Dept of Events and Talks 🗣

Matrix will be featured in a Rust London meetup on the 27th May 2020.

New Public Rooms 🏟

Ludwig offered:

Hey, I have created a room for designers working on matrix stuff to join: #matrix-design:disko.media

The goal is to improve cohesion and collaboration for all things design (UI, UX, interaction...) Over time, it should become a place for people to

  • ask design-specific questions

  • share cool stuff and be praised for all the good work you're doing

  • develop and document a common understanding of the semantics of matrix

  • make sure the user experience is amazing, even as we all figure out how to federate. Probably most designers work on single clients right now, but there are important UX questions that can only be answered ecosystem-wide. Maybe we'll end up with a design guide for people to use alongside the spec.

Final Thoughts 💭

Let's test homeservers against sytest, then unfairly pit them against one another! This week we're comparing Conduit and Dendrite - let's see the results!

NameConduitDendrite
Federation APIs1% (1/103 tests)27% (28/103 tests)
State APIs0% (0/13 tests)46% (6/13 tests)
Key API17% (1/6 tests)33% (2/6 tests)
Query API0% (0/5 tests)80% (4/5 tests)
send_join API0% (0/8 tests)38% (3/8 tests)
make_join API0% (0/2 tests)50% (1/2 tests)
Auth0% (0/19 tests)5% (1/19 tests)
room versions0% (0/7 tests)29% (2/7 tests)
Federation API0% (0/14 tests)36% (5/14 tests)
get_missing_events API0% (0/6 tests)33% (2/6 tests)
Backfill API0% (0/4 tests)0% (0/4 tests)
Invite API0% (0/10 tests)10% (1/10 tests)
send_leave API0% (0/1 tests)0% (0/1 tests)
Public Room API0% (0/1 tests)100% (1/1 tests)
Device Key APIs0% (0/7 tests)0% (0/7 tests)
Non-Spec API0% (0/50 tests)0% (0/50 tests)
Client-Server APIs8% (56/667 tests)33% (225/672 tests)
Registration53% (17/32 tests)62% (20/32 tests)
Login26% (5/19 tests)32% (6/19 tests)
V1 CS APIs10% (3/30 tests)10% (3/30 tests)
Misc CS APIs100% (1/1 tests)100% (1/1 tests)
Profile33% (2/6 tests)100% (6/6 tests)
Devices17% (2/12 tests)17% (2/12 tests)
Presence0% (0/10 tests)0% (0/10 tests)
Create Room31% (4/13 tests)92% (12/13 tests)
Sync API1% (1/85 tests)40% (34/85 tests)
Room Membership0% (0/15 tests)47% (7/15 tests)
Room State APIs0% (0/14 tests)36% (5/14 tests)
Public Room APIs0% (0/7 tests)14% (1/7 tests)
Room Aliases0% (0/15 tests)67% (10/15 tests)
Joining Rooms0% (0/8 tests)75% (6/8 tests)
Leaving Rooms0% (0/1 tests)100% (1/1 tests)
Inviting users to Rooms0% (0/14 tests)7% (1/14 tests)
Banning users0% (0/5 tests)20% (1/5 tests)
Sending events0% (0/3 tests)100% (3/3 tests)
Getting events for Rooms0% (0/10 tests)60% (6/10 tests)
Typing API0% (0/5 tests)0% (0/5 tests)
Receipts0% (0/4 tests)0% (0/4 tests)
Read markers0% (0/1 tests)100% (1/1 tests)
Media APIs10% (2/21 tests)24% (5/21 tests)
Capabilities API0% (0/2 tests)50% (1/2 tests)
Logout0% (0/4 tests)75% (3/4 tests)
Push APIs21% (12/58 tests)8% (5/59 tests)
Account APIs20% (2/10 tests)40% (4/10 tests)
Ephemeral Events0% (0/1 tests)0% (0/1 tests)
Power Levels0% (0/2 tests)83% (5/6 tests)
Redaction0% (0/5 tests)0% (0/5 tests)
Third-Party ID APIs16% (3/19 tests)16% (3/19 tests)
Guest APIs0% (0/35 tests)49% (17/35 tests)
Room Auth0% (0/25 tests)52% (13/25 tests)
Forget APIs0% (0/4 tests)25% (1/4 tests)
Context APIs0% (0/4 tests)0% (0/4 tests)
Room Upgrade APIs0% (0/21 tests)0% (0/21 tests)
Room Versions0% (0/40 tests)75% (30/40 tests)
Device Keys3% (1/37 tests)3% (1/37 tests)
Tagging APIs0% (0/8 tests)75% (6/8 tests)
Search APIs0% (0/6 tests)0% (0/6 tests)
OpenID API0% (0/3 tests)0% (0/3 tests)
Send-to-Device APIs0% (0/12 tests)0% (0/12 tests)
Server Admin API0% (0/1 tests)0% (0/1 tests)
Ignore Users0% (0/3 tests)0% (0/3 tests)
User Directory APIs0% (0/11 tests)0% (0/11 tests)
Application Services API4% (1/25 tests)20% (5/25 tests)

That's all I know 🏁

See you next week, and be sure to stop by #twim:matrix.org with your updates!

The Foundation needs you

The Matrix.org Foundation is a non-profit and only relies on donations to operate. Its core mission is to maintain the Matrix Specification, but it does much more than that.

It maintains the matrix.org homeserver and hosts several bridges for free. It fights for our collective rights to digital privacy and dignity.

Support us