This is part of a series of reports on the six projects assigned to Matrix for Google Summer of Code 2020.
Hello! I am Nikos, an MSc student in Computer Security. My original goal for this GSoC was to implement the basics of end-to-end encryption for Go-NEB, which is a popular and easily extendable bot for the Matrix protocol. I chose this project because it ticked off a lot of the boxes that I look for in a project: The Matrix protocol is very interesting and the e2e specification is fascinating, Go-NEB is written in Go which I wanted to learn better, plus I believe personally that crypto should be easy for everyone to use and adopted more.
My initial plan was to first create bindings in gomatrix for libolm, the C library that implements most of the necessary algorithms for Olm/Megolm, the protocols used in Matrix' end-to-end encryption. Then, I was going to write some nice APIs around those bindings so that most of the work done is hidden from the client that uses the library, and finally I would call these APIs from Go-NEB when needed to set up the encrypted sessions between devices in a room and use them to encrypt/decrypt messages as necessary. What we found out soon after GSoC started was that another library called mautrix-go that was based on gomatrix already did most of these things. My task then became to change the uses of gomatrix in Go-NEB for mautrix-go and make some slight changes to the latter if there were some incompatibilities. A month later, that task was done.
With two months to go in GSoC, my mentors came up with a new task: to implement a service for Go-NEB that allows other client developers to test their e2ee implementations easily. That resulted in a service called "cryptotest" which allows other clients in a specified room to execute some Neb commands which would trigger functions like the forwarding of keys between clients, another feature that had to be implemented in mautrix. Something else that I wanted to add and grabbed my interest was SAS device verification, a multi-step process that involves users comparing a string of numbers or emojis out-of-band to configure they are indeed talking to each other and not to a MITM. These were my main tasks during the second month and I was happy to have achieved them and made a decent verification API for clients to use. They were soon merged into mautrix and Go-NEB.
For my third month I proposed to implement a new Matrix feature called cross-signing, which was similar to SAS verification but dealt with verifying other users instead of their devices who would in turn verify their own devices, creating a graph of trust between users and devices. This functionality was strongly coupled with SSSS, another new feature that allows clients to store their encrypted secrets (in this case the multiple necessary signing keys) on the server, as well as in-room verification which allows verification between users. This task was more challenging as it uses multiple algorithms (some of which not in libolm) for deriving the keys, using them to encrypt other keys which in turn sign other keys. It was also more satisfying when I finally managed to generate the keys, store and retrieve them and then use them to sign another user and when I was done I felt like my understanding of cryptography levelled up.
All in all, I'm very satisfied with the overall experience. The spec was very clear in most cases and when it wasn't the community was always helpful and responsive. They were also happy to discuss the spec with me and explain the more intricate details. For that reason I wasn't asking questions directly to my mentors (besides for defining my tasks) but to the overall community. I'm also very grateful to my mentors, @uhoreg and @Kegan for picking me for this project and helping me with the planning aspect and reviewing my PRs. I'd lastly like to thank the maintainer of mautrix, @tulir, for reviewing and merging a lot of my work into his library.
This is the most students we've have taken on in a single year, made possible by Matrix now acting as an umbrella organisation for multiple projects - this year that's Ruma, Nheko and OpsDroid. There are also students working on projects from the core team: go-neb, matrix-js-sdk and matrix-ircd.
We recieved dozens of applications this year, which made narrowing our focus to six students difficult, but we are proud to announce:
Devin Ragotzy, who has been involved with the Ruma project in the past, will work on Ruma procedural macro refactoring and more
Arnav Tiwari will work with Travis and myself to create HTML Embeddable Matrix Chat Rooms
Nikolaos Filippakis will work to support E2E encryption for go-neb. Kegan, one of the original go-neb authors is particularly excited about this one
Brooks Karlik will update matrix-ircd: Move matrix-ircd to async/await. This will be a welcome upgrade for anyone using that project!
Chethan Reddy, last but not least, will work with Nico and the Nheko gang on Adding Features in End-to-End encryption for Nheko-Reborn
Congratulations all, and also congratulations and thank you to the projects the students will be working on!
Find out all about Matrix @ GSoC on Google's dedicated site, and keep reading the Matrix blog where we'll have updates from the students.
PS Alejandro Domínguez has a Matrix related project too: they'll be working under the GNOME organisation to add Multi account support to Fractal.
*libQMatrixClient was renamed to libQuotient
Regarding my initial proposal, I’ve completed tasks enough for message receiving, and not finished tasks related to message sending.
Focused on libQuotient, I've also added changes to Quaternion client to support the proposed API and test the results. I've reused libQtOlm, which is a Qt wrapper to the matrix olm library and contributed to provide better compatibility during its building and deployment. This also required to dive into the olm library itself and provide minor patch for the olm CMake files too.
So, the basic structure of the project changed a bit, libQtOlm was added as a dependency to support libolm:
+--------------------------------------------+ | Quaternion/Spectral/Telepathy|Tank/etc | +------------------+ +--------------------------------------------+ | CS API | <----> | libQuotient | +------------------+ +-------------------------------+------------+ |Synapse homeserver| | Qt | libQtOlm | +------------------+ +--------------------------------------------+ | libolm | +------------+
During the coding period, I've resorted to the specification, the guide and the last year GSoC python implementation actively. And added minor fix for the documentation about names constants at the documentation examples.
Talking about future work, the status is going to be captured at libQuotient project board. Next steps are:
While olm account management architecture and
device_one_time_keys_count sync data handling is here, such tasks as session management after a restart and device verification still requires additional efforts. After that encrypted attachments support and key backups could also be added.
I'm going to take care about that, but it definitely will be harder as just a side project. I still have enthusiasm though :)
To take part in the Google Summer of Code project was my dream. I'm very happy I've managed to took part this year, since it was the last year of my study, while I had doubts about my personal results until I've received final confirmation and the review.
From the beginning of the project, I've realized I'm very lucky since I've got all the chances to provide perfect results. @Kaffeine helped me a lot with my TelepathyIM contribution. He helped me to evolve my CMake, git and Qt/C++ skills and that's how I've started contributing to the libQuotient before GSoC. After that, it turns out that I've accepted and received even two mentors. @uhoreg from the Matrix team, who helped me with End-to-End Encryption logic and olm/megolm understanding. And @kitsune from the Quotient project, who helped me a lot with the code review, with the architecture decisions and the library logic, and even with the time management (he was the one who watched carefully about my results). The last year GSoC python implementation and guide from @Zil0 were also here, and it turns out that Spectral developer @BHat provided QtOlm wrapper right before GSoC stated.
However, planning and updating the plan were my weak points, where I've made key mistakes, such as poor combination with my regular part-time job. And I definitely should have reserved a small vacation at least for the final period of the project to handle tasks better.
In the end, I've managed to debug the mistakes and provided encrypted messages receiving that could already be used at the clients. Also, I evolved my skills and dived into the megolm E2EE subject. I'm willing to continue my contribution to develop libQuotient as full-featured Qt-based Matrix library.
In general, I am not disappointed. I'm wishing luck to all the future students who are reading this. I'm happy to receive support and contribute to an international open project not only for myself, but also for the other developers and users.
GSoC project page: link
GSoC Matrix room: link
Like many others passionate about Dendrite, I've been eager to see the day when it finally joins the decentralised, federated Matrix network, so I worked on the project to see how much I could do to bridge the gap between it and the current reference homeserver implementation, Synapse, in terms of feature completeness.
Working on Dendrite in just one area would be insufficient to propel it, so my work covers various components in the project, including the Client/Server API, Sync Server, Room Server, and Federation Component. I also spent time refining the project's documentation, improving its testing/continuous integration process, as well as reviewing pull requests from Matrix.org members and the community.
Below is the major portion of my work presented as pull requests, categorised by the components they belong to; a list of links to all the pull requests I created/reviewed is also available at the end of this report.
This component is the main handler of HTTP requests from the clients, except for
/sync requests - see next part.
The Sync Server is responsible for handling the long polling notification
/sync requests) and some other related requests from the clients.
The Room Server is, as its name suggests, where room events, state, etc. are handled.
The Federation Component sends requests to and receives requests from other Matrix homeservers.
Here is a list of links to all the pull requests I have created/reviewed (by 26 Aug, 2019):
|matrix-org/dendrite||Created (36) / Reviewed (27)|
|matrix-org/gomatrixserverlib||Created (12) / Reviewed (1)|
I believe that Dendrite is a pretty special project to work on in the GSoC program.
At its current state, Dendrite isn't a complete homeserver yet. This has left a large portion of work on Dendrite wide open: As I worked on Dendrite, more than often I'd find myself in a place to answer questions related to topics like design or project architecture which, if not thoroughly thought through, might have consequences in the future development of the project. For this reason, I think working on Dendrite involves a set of challenges quite different from what attempting to improve an already functioning piece of software would have presented.
For the same reason, I think what Dendrite has taught me during the summer was less about how to become more fluent in Go or more familiar with various tools, but more about developing the mindset that powers software projects behind the scenes. But this mindset, to myself, seems more valuable than the former, indeed.
And at the end of this report, I'd like to thank my mentors anoa and Brendan, as well as everyone else at Matrix who has answered my questions, discussed with me, and helped me understand the code, and at the same time being super responsive. Without your help, my GSoC experience wouldn't have been this enjoyable! :)
The Google Summer of Code 2019 is coming to an end for me, so it means that it’s time for the final report.
I’ve been taking care of the project “Matrix Visualisations” during these past months. This project aimed at initiating the development of a tool which allows the real-time visualisation of the events DAG of a given Matrix room, as seen from the perspective of one or more homeservers (HS’s).
Regarding my initial proposal, I’ve completed every task proposed and even implemented some additional functionalities. The application is not finished yet and there could be a lot of improvements added to it (especially regarding the design of the UI) but the core functionalities have been implemented.
I am going to precise what has been accomplished and then give some ideas of features to improve.
During GSoC, I have used two separate repositories for the frontend and the backend. I will keep both of them because I’m referencing PRs from them (as PRs are easier to link than lengthy lists of commits).
However, this is the repository regrouping these two parts and this one will be moved to matrix-org for the continuation of this project.
During the application period, I wrote a prototype for this project. This prototype implemented some requests to the CS API (like /login, /logout or /sync) but there were more requests to implement in order to be able to fully use this API:
fromin the /sync request.
I also did a lot of clean up in the source code from the prototype during this task. You can have more details in this PR.
First of all, a lot of work had to be made in order to properly update the displayed DAG when adding new events to it. At this point, I previously used the
setData method of the
Network object of the visjs library (which is used for displaying the graph and interacting with it) each time a node was added, but it was resetting the display each time it was called.
The proper solution was to progressively add nodes and edges to the
DataSet object passed to the constructor of the network (see the documentation of DataSet and Network for more details).
The DAG has been set to be displayed vertically, the events with the same depth are at the same level, the deepest events are at the bottom. The events which origin is the HS on which we are making the observation are in green, those which are coming from other servers are in orange. Here is a picture of what it looks like:
In the graph, there is a special node just after the earliest event which allows us to ask the application to load more events. Here is what it looks like:
When you double click on the node of an event, there is a text area on the right which displays its complete JSON body, like this:
I’ve also implemented the possibility to choose which particular fields of the event can be directly displayed in the labels of the nodes of the displayed graph:
Next, I implemented a backend for retrieving events from the PostgreSQL database of a Synapse HS. It is a small HTTP server which receives requests from the frontend application and then makes queries to the Synapse database to get the requested events.
You can find details about the API for using this backend in this readme, in the “HTTP REST API” section.
You can find more details about this initial implementation: here, here, here (my mentor helped me on this one, thanks to him), and here.
I’ve added the support of this backend in the frontend application, as well as a way to choose which backend to use (between this one and the CS API) in this PR.
I implemented the ability to observe the same DAG of a room from multiple HS’s. This is done with “views”.
In the picture above, you can see that there is a drop down menu from which you can select the view. The fields under this line are used to control the selected view: indicate where it will be connecting, for which room (you could as well observe a different room in a different view), etc…
By default, there is only one view but you can add as many as you want by clicking “Add a view”.
All the DAGs from the different views are displayed side-by-side within the same canvas, like this:
You can have a look at the details of the implementation in this PR.
The next major task I had to do was the implementation of a backend for retrieving events via the Federation API. I thought a lot about the possible options for the location of this backend and I decided to extend the web server created for the PostgreSQL Synapse backend, so that we could launch it in a “postgres mode” or “federation mode”. But the backend would offer the same HTTP API in both modes.
The backend uses the Federation API in the following way:
The full details of the implementation are in this PR. My mentor also helped me get the usage of the
Futures right thanks to this PR.
There has been a small modification in the frontend too, because of the addition of the
/stop endpoint in the backend’s HTTP API, these modifications are in this PR.
For each event, there is a state of the room associated with it, which describes what was the state of the room at the moment this event was accepted (the name of the room, its topic, its members and parameters, etc…).
So I’ve added a way to display this: when you have selected and displayed the JSON body of a given event, you can also request the associated room’s state. I have made it possible to use this feature with every backends: the CS API, the PostgreSQL database and the Federation API. You can have the full details of the implementation here (for the backend) and here (for the frontend).
You can see the result of this feature in the picture below (there is a button “Room state at the selected event”, which allows to ask the application to fetch the state, and the text area under this button where the state is displayed):
Lastly, I’ve applied small fixes to the code of the backend, you can see them in this PR.
The objective of this project was to develop the core functionalities of this application, however there are a lot of improvements to bring to it, like:
This experience has been really rewarding for me. I could discover more about the Matrix community and how the Matrix ecosystem works (on a technical point of view). I want to thank my mentor, Erik Johnston, for his guidance during these past months, and the people in this community who gave me advice.
GSoC has also allowed me to further improve my programming skills in general and discover many various things: the WASM technology, how to use Rust in this context thanks to the various existing libraries/frameworks available, the practical usage of SQL requests as well as TLS certificates and how to apply cryptographic signatures.
It was sometimes challenging to use such experimental technologies (due to the lack of clear documentation) but also very exiting!
Mid-September, I will start my class for my second and final year of my master degree (software engineering, specialised in distributed systems and applications) at Sorbonne Université so I will definitely have less free time. So I don’t think I’ll be able to actively continue to contribute but I will do my best to help other people to continue the work I’ve initiated.
It’s that time of year again! Matrix.org is once again participating in the Google Summer of Code program. We have been allocated four student slots by Google this year, and narrowing the 18 proposals we received down to just four was a very difficult task.
In the end, we have decided on the following four students and their proposed projects:
Alexey Andreyev’s proposal involves adding end-to-end encryption to libQMatrixClient for future support in Qt/libQMatrixClient-based clients such as Quaternion and Spectral. They will be mentored by kitsune, lead developer of libQMatrixClient, and our own end-to-end encryption expert, uhoreg.
Kai Hiller’s proposal for more reliable third-party protocol bridges includes adding the ability to notifiy the user when a message fails to reach its final destination despite being accepted by the bridge. Half-Shot.
Eisha Chen-yen-su’s proposal for Matrix Visualisations aims to “develop a tool which will visualise the event Directed Acyclic Graph data structure which describes the conversation history in a room. It will be a real-time visualisation of the DAG of a given Matrix room, as seen from the perspective of one or more HomeServers (HSes).” They state that “this tool will be useful for debugging or administration of Matrix HSes by making people able to easily see how the federation process works”. They have already posted prototypes of their tool in #gsoc:matrix.org, and it’s all written in Rust! Which makes their mentor, erikj, very happy.
And finally, Cnly’s proposal for working towards completion of Dendrite’s Client-Server API. The proposal also touches on general improvements to the codebase and increasing test coverage. Cnly will be mentored by babolivier and anoa.
Congratulations to the selected students. We look forward to participating with you on completing your project over the course of the summer holidays.
If your proposal was not selected, do not give up hope! Being an active member of the Matrix community and having a deep understanding of the ecosystem and its projects is a big part of what we look for when choosing candidates. If you stick around, you have a strong chance of being chosen in a subsequent year.
We will not be sharing individual’s proposal documents, but students are free to share them as they please.