As the Matrix Specification stands today, it is easy for client developers to
assume that all events will be structured as documented. For example, the
displayname
key of an m.room.member
event is
specified to be "string
or null
"; and the info
key of an m.image
message event must
be a Javascript object.
In reality, these are not safe assumptions. This MSC proposes that the specification be updated to make it clear that developers should treat all event data as untrusted.
Firstly, let's examine the reasons why such malformed events can exist. Some of these reasons may appear to have trivial solutions; these are discussed below.
-
Most obviously, Synapse as it currently stands does very little validation of event bodies sent over the Client-Server API. There is nothing stopping any user sending a malformed
m.room.message
event with a simplecurl
command or Element-Web's/devtools
.Any event sent in this way will be returned to the clients of all the other users in the room.
-
Events may be encrypted. Any client implementing E2EE must be prepared to deal with any encrypted content, since by definition a server cannot validate it.
-
In order to allow the Matrix protocol to be extensible, server implementations must tolerate unknown event types, and allow them to be passed between clients. It is obvious that a pair of custom clients implementing a
com.example.special.event
event type cannot rely on a standard server implementation to do any validation for them.However, this problem extends to event types and keys which have been added to the specification. For example, the
m.room.encrypted
event type was added in Client-Server API r0.4.0. It therefore follows that a server implementing CS-API r0.3.0 would have no way to validate anm.room.encrypted
event, so if a client is connected to such a server, it could receive malformed events. -
To extend from point 3, the problem is not even resolved by upgrading the server. There may now be rooms which contain historical events which would no longer be accepted, but these will still be served by the server.
This problem also applies to non-room data such as account data. For example, Client-Server API r0.6.0 added the
m.identity_server
account data event type. It is possible, if unlikely, that a client could have uploaded anm.identity_server
event before the administrator upgraded the server. -
Event redaction removes certain keys from an event. This is a bit of a trivial case, though it is worth noting that the rules for event redaction vary between room versions, so it's possible to see a variety of "partial" events.
-
All the cases above can occur without federation. Federation adds additional complexities due to the structure of Matrix rooms. In particular, a server implementation cannot simply ignore any malformed events since such events may either be critical to the structure of the room (for example, they may be
m.room.membership
events), or at the very least ignoring them would leave "holes" in the event graph which would prevent correct back-pagination.
The problems above appear to have some easy solutions. Let's describe some of those ideas, before considering the fundamental problem that none of them can solve.
In this idea, we would require all server implementations to strictly validate any data which is sent over the Client-Server API, to ensure that it complied with the specified formats.
This evidently solves problem 1 above, in that it would prevent local users from creating malformed events of any event types that the server supports; however, it would do nothing to address any of the other problems.
We could require that server implementations validate any data that they are about to serve to a client; we might recommend that malformed data be stripped out of the response, or redacted, or similar.
It is worth mentioning that this would be tricky to implement efficiently on the server side, but it at least helps address most of the problems above, such as historical data, or malformed events received over federation.
Similar to the previous idea, but rather than validating data each time it is served to a client, any stored data could be re-validated to check that it complies with new validation requirements.
This could be more efficient in the case that changes to validation rules are rare, but it could still be a huge amount of data processing on a large server.
As outlined above, one of the big problems in this area is how we deal with
events sent over federation; in particular, if subsets of the servers in a room
have different ideas as to which events are "valid", then their concepts of the
room state can begin to drift, and the room can eventually become
"split-brained". This makes it hard to simply say, for example,
"m.room.member
events with a non-string displayname
are invalid and should
not form part of the room state": we have a risk that some servers will accept
the event, and some will not.
One approach to solving this is via room versions. By specifying that a change of rules only applies for a future room version, we can eliminate this potential disagreement.
The process of changing a room from one version to another is intrusive, not least because it requires that all servers in a room support the new room version (or risk being locked out). For that reason, it is extremely undesirable that any new feature require a new room version: whenever possible, it should be possible to use new features in existing rooms. It therefore follows that we cannot rely on room versions to provide validation of event data.
This idea is included for completeness, though it is unclear how it would work in practice.
It has been suggested that we create a new room version which explicitly states that events which fail the current event schema, whatever that is at that moment in time, should be rejected.
Let's imagine that in future, the m.room.member
event schema is extended to
include an optional location
key, which, if given, must be a string. The
implication of this idea is that servers should reject any m.room.member
event whose location
is not a string. We now have a problem: any servers in
the room which are updated to the latest spec will reject such malformed
events, but any other servers yet to be upgraded will allow it. So is that user
in the room or not?
Even if all the servers in the room can be upgraded at once, what about any
m.room.member
events which were sent before the rule change?
The ideas above all mitigate the problems discussed earlier to a greater or lesser extent, and may indeed be worth doing on their own merits. However, none of them can address the problem of outdated server implementations.
For example, consider the case of a new key being added to an event body, say
m.relates_to
. Now, we may have decided as above that all new specced keys
must be validated by the server, so vN+1
of Synapse dutifully implements such
validation and refuses to accept events with a malformed m.relates_to
.
The problem comes for users whose server is still Synapse vN
. It knows
nothing of m.relates_to
, so accepts and passes it through even if
malformed. The only potential solution is for clients seeking to implement
m.relates_to
to refuse to talk to servers which do not declare support for
it.
However, this is an entirely client-side feature: it is illogical to require
that servers must be upgraded before it can be used. Consider that the hosted
element-web at https://app.element.io
is upgraded to support the new feature;
in this scenario, that would lock out any user whose homeserver had not yet
been upgraded. This is not an acceptable user experience.
In short, we are left with the reality that clients must still handle the unvalidated data.
Short of closely coupling server and client versions - which violates the fundamental ethos of the Matrix project - there is nothing that can completely prevent clients from having to handle untrusted data. In addition, encrypted events eliminate any possibility of server-side validation.
With that in mind, the advantages of the ideas above are diminished. If clients must handle untrusted data in some circumstances, why not in all? "You can trust the content of this data structure, provided you have checked that the server knows how to validate it, in which case you need to treat it as untrusted" is not a useful message for a client developer.
It may be possible to assert that specific, known cases can be treated as trusted data, but these should be called out as specific cases. The default message should be that clients must treat all event data as untrusted.