Since I last wrote a newsletter, a lot of neat stuff has happened. I apologize for the delay and the periodic absences, but I'm almost to a point where I will hopefully have a bit more free time to work on Telodendria.
Architecture
- Source/Sink-Agnostic IO: I abstracted all of the I/O code in Telodendria out into the
Io
and Stream
APIs. These APIs are designed to unify the way we read and write streams, regardless of where they are coming from and where they are going. Io
is heavily inspired by the BSDs' funopen()
, or GNU's fopencookie()
. Essentially, it abstracts out the POSIX system calls read()
, write()
, seek()
, and close()
, and allows a program to specify alternate implementations, basically creating entirely custom IO streams. Stream
builds on Io
to emulate the behavior of the standard C library, essentially replacing FILE
pointers.
Okay cool, what does this have to do with implementing a Matrix homeserver, you ask? Well, it made it so much easier to implement a crucial feature called...
- Transport Layer Security: Telodendria now fully supports TLS! Currently, Telodendria supports OpenSSL and LibreSSL, so it should work out of the box on most Linux distributions and all of the BSDs. Thanks to the new
Io
API, supporting other TLS libraries should also be incredibly easy for those trying to port Telodendria to more obscure systems. In fact, I have even added a template in contrib/
that anyone can use as a starter to add support for additional TLS libraries. My goal is to make Telodendria support the TLS libraries that operating systems ship with, so that building Telodendria doesn't require installing anything additional.
- HTTP Request Router: I finally got around to implementing a real HTTP request router, complete with POSIX regular expression support. Previously, we were routing requests using a bunch of nested
if
-statements, which got very messy very quickly. While extremely fast and memory efficient, it simply isn't sustainable, so I'm willing to sacrifice a few bytes of memory and the runtime overhead of building a routing tree so that we can keep the code maintainable.
- Administrator API: Work has begun on the
/_telodendria/admin/*
endpoints. Currently implemented are these:
/_telodendria/admin/privileges
/_telodnedria/admin/config
(more on that below)
/_telodendria/admin/shutdown
/_telodendria/admin/restart
/_telodendria/admin/stats
Consult the telodendria-admin(7)
page for all the documentation on these endpoints. Prototype endpoint documentation is available in the proposals/admin.7
man page, but these aren't implemented yet, and may be subject to change during implementation.
- Configuration: Telodendria now no longer uses a traditional configuration file. All configuration has now moved to the database. This allows the daemon configuration to be modified at runtime, and in fact, that's the recommended way of doing it now. Use the
/_telodendria/admin/config
API endpoint to update Telodendria's configuration, and restart Telodendria using /_telodendria/admin/restart
. I refactored the configuration parser to be more robust, and now endpoints that require the configuration file can simply do a ConfigLock()
to get access to the current configuration. Do note that the configuration API endpoint is not complete yet. Currently, it only supports retrieving the whole configuration and replacing the whole configuration. I hope to add support for incremental updates to the configuration before the next release.
- New Tools: Telodendria has some new command line tools for performing common tasks. I've implemented
http
, a simple HTTP client in the style of curl
, and json
, a simple JSON parser in the style of jq
. Note that these are both very primitive tools, which is by design. They're meant to fulfill the needs of Telodendria, and nothing more.
Matrix Specification
Here's what's been cooking on the Matrix side of things:
- User Interactive Authentication (UIA): UIA for password and registration token authentication is just about complete. I'm wrapping up the fallback web pages soon, and then I'll build out the registration token administrator API, and then all of the Client Authentication of the Matrix specification should be complete!
- User Profiles: Support for fetching and setting display names and avatar URLs has been added.
- Account Management: Users can now change their passwords and get details about their account.
I'm sure there's been a few other new things as well, but at the moment they are not coming to mind.
What's Next?
Next up on the TODO
list are these:
- Configuration: As mentioned above, there's some work that needs to be done on the configuration API.
- User Data, Capabilities Negotation: I'm hoping to wrap up these small sections of the Matrix specification before the next release. The former might actually be just about complete, I just have to take a look at it to verify.
- Rate Limiting: Rate limiting is going to be a major project. It'll require storing more state, as well as discovering more information about a client.
There are also a few things a bit further out in the future, but I'm thinking about them nonetheless:
- HTTP/1.1: Currently Telodendria is HTTP/1.0 only, which works fine, but requires some unfortunate workarounds. Implementing some very common HTTP/1.1 features will allow Telodendria to take advantage of persistent connections, chunked encoding, and more. Persistent connections will be useful particularly now that the overhead of TLS is a factor. HTTP/1.1 will require some major refactoring, so expect it to be a large project.
- Testing: Over the last few weeks or so, I've been learning Perl, so we can potentially get some Sytest action going once Telodendria has some more functionality. I expect these tests to rapidly and drastically increase the quality of Telodendria.
Finally, if you're looking for a way to get involved, look no further than this list:
- Continuous Integration: We briefly talked about CI a few weeks ago in the other Matrix rooms. I'm continuing to think about what this looks like for Telodendria, whether it's GitHub Actions, or something home-grown. I tend to lean toward the latter in the long run, but the former may be a good way to get our feet wet with CI.
- Administrator Tools: At some point, I would like to build a command line client for the administrator API. Whether this looks like an
ncurses
application, or a more simple command line application, I am not yet sure, but if you have an interest in this or want to start building such a thing, feel free to discuss this with me more so we can work out some of the specifics.
- Documentation: There's always documentation that has to get updated before the release. The code isn't worth very much if it isn't maintainable, and documentation goes a long way in ensuring that Telodendria is maintainable. I've been thinking about a way to put the documentation in the header files as specially-formatted C comments, and then generate the man pages from the header files. I think that would make maintaining the documentation much easier, and it would force people that touch the header files to make sure the documentation is up to date—particularly me, since I have a bad habit of not documenting new functions—but I know I don't want to bring in something like Doxygen, so I'd want to roll my own documentation generator, which wouldn't be difficult at all, I just don't know exactly what that looks like quite yet.
These things can happen fairly independently of Telodendria's development, so it doesn't matter when they happen, and I'll get to them eventually, but if you are looking for something to do, maybe these things would be a good place to start.
Telodendria Project
Maybe you all are sick of me continually bringing this up, but open source projects don't build themselves! It takes a lot of time and effort to build up a large and successful open source project, which is exactly what I'm hoping to accomplish with Telodendria. I know that process is not quick or easy, but you can help make it quicker and easier!
- Get Involved: Don't be afraid to get involved! If you have thoughts or ideas, please don't hesitate to share them, and if you are willing and able, do write some code. Your help is always greatly appreciated. Consult the list above if you need some inspiration.
- Donations: Telodendria has expenses. I don't have any trouble meeting those expenses at the moment, but I certainly would appreciate some help covering them, and if you like what I'm doing, maybe throw a little something my way for my efforts as well. Telodendria is currently not something I can make a living at, which unfortunately means it is relegated to a side project, but if you want me to put more effort in to the project, this is a great incentive, because the more I can make on Telodendria, the less time I have to spend doing other things in an effort to sustain myself.
- Reach Out: The public Matrix rooms are a great place for discussion, and I'd encourage you to keep all discussion on Telodendria there, but if you want to reach out to me personally, whether it's to discuss an idea you have or express a concern in private, you are more than welcome to do so.