master
Sven Slootweg 3 years ago
commit e0fea190ce

@ -0,0 +1,11 @@
{
"extends": "@joepie91/eslint-config",
"parserOptions": {
"ecmaVersion": 2020,
"sourceType": "script"
},
"parser": "babel-eslint",
"plugins": [
"babel"
]
}

2
.gitignore vendored

@ -0,0 +1,2 @@
node_modules
private

@ -0,0 +1 @@
{ "presets": ["@babel/preset-env"] }

@ -0,0 +1,9 @@
"use strict";
module.exports = {
createClient: function (hostname, options) {
return {
// FIXME
};
}
};

@ -0,0 +1,437 @@
IDEA: Add a strictValidation flag that strictly validates all inputs on the client library, to catch errors early? Default enabled, allow disabling for eg. people working on protocol extensions
IDEA: Bot(?) library with E2EE support
Packages:
- create-session (for each auth method! produce a {homeserver, accessToken, protocolVersions, unstableFeatures} object named a 'session')
- sync-to-events (parse sync response, turn into list of events)
- event-stream (return ppstream with events that keeps polling /sync)
- various method packages for individual operations, that take a 'session' as their argument
- client, which combines all(?) of the above into a single API... or maybe just the method packages and event-stream? and let a `client` be initialized with a 'session' object from elsewhere
TODO: Write a package for generating a filename based on description + mimetype?
- If description is already a filename with a valid extension for the mimetype (or no mimetype), return as-is
- Otherwise, if mimetype is set, return description + primary extension for mimetype
- If neither a valid filename nor a mimetype is set, return `undefined`
How to deal with event validation: it's probably easiest to have a two-pass validatorm, where the first pass does any normalization and critical validation, and the second pass does "fail-able" validation - ie. validators that can safely fail and should just produce a warning.
TODO: Figure out a failsafe mechanism to prevent unencrypted file uploads in service of an encrypted m.file/m.video/etc. event. Maybe default-reject unencrypted files in makeEncryptedMessage wrapper, unless a permitUnencryptedAttachments flag is set (eg. for forwarding attachments from an unencrypted room)?
FIXME: Split makeMessage stuff out of send* modules once we need to encrypt events for E2EE
TODO: Find a way to reduce validation schema duplication for this
FIXME: Gracefully handle 500 M_UNKNOWN: Internal server error (use exponential backoff?)
Reading magic bytes from a blob: https://gist.github.com/topalex/ad13f76150e0b36de3c4a3d5ba8dc63a
------
# PaginatedResource (cursor) interface
items: the items in the current chunk
getNext({ [limit] }): retrieves the next chunk of items, in the 'direction of travel'
getPrevious({ [limit] }): retrieves the previous chunk of items, in the 'direction of travel'
getNext and getPrevious may reject with a NoMoreItems error if the end of pagination in that direction has been reached; either immediately, or after making another speculative request, depending on the underlying technical requirements
not every implementation may support the `limit` override; in those cases, there is either no fixed limit, or the limit can only be set during the initial request
-------
# messages endpoint
Forwards:
Request initial set somehow
Request again, dir f, from <last token of initial set>
Response:
start:
--------
# parjs combinators
## Characters
digit ASCII(?) digit in <base>
hex ASCII(?) digit in base 16 (hex)
uniDecimal unicode digit in base 10 (decimal)
letter ASCII letter
uniLetter unicode letter
lower ASCII lower-case letter
uniLower unicode lower-case letter
upper ASCII upper-case letter
uniUpper unicode upper-case letter
space single ASCII space
spaces1 one or more ASCII spaces (ie. space+)
whitespace zero or more ASCII "whitespace characters", whatever that means
anyChar a single character of any kind
anyCharOf a single character in the specified <set: string>
noCharOf a single character NOT in the specified <set: string>
## Control flow
replaceState apply the specified parser, but within an isolated <userState> scope (which may also be specified as a function(parentState) -> scopedState)
backtrack apply the specified parser and return the result, but do not advance the parser position
later placeholder parser that does nothing, and on which the `.init` method needs to be called later, to (mutably!) fill in the actual logic
between apply the <before: parser>, and then the specified parser, and then <after: parser>, and return the result of the middle one
thenq apply the specified parser and then the <next> parser, and return the result of the former
qthen apply the specified parser and then the <next> parser, and return the result of the latter
then apply the specified parser and then the <next> parser, and return array containing the results of the [former, latter]
thenPick apply the specified parser and then call the <selector: function(result, userState)>, which dynamically returns the next parser to apply
## Boolean operations
not invert the result of the specified parser
or attempt each of the specified <parsers ...> until one succeeds or all fail
## Repeats
exactly apply the specified parser <count> times, and return an array of results
many apply the specified parser until it runs out of matches, and return an array of results
manyBetween apply <start: parser>, then the specified parser, until the <end: parser> is encountered, then call <projection: function(results[], end, userState)>
manySepBy apply the specified parser until it runs out of matches, and expect each match to be separated with <delimiter: parser>; return an array of results
manyTill apply the specified parser until the <end: parser> is encountered, and return the results of the specified parser
## Result and state handling
must apply the specified parser, and expect its result to pass <predicate: function(result)>
mustCapture apply the specified parser, and expect the parser position to have advanced
each apply the specified parser, then call <projection: function(result, userState)>, and return the *original* result (userState may be mutated)
map apply the specified parser, then call <projection: function(result, userState)>, and return the result of the projection function
mapConst apply the specified parser, then throw away the result and return <constantResult> instead
maybe apply the specified parser, and return its result if it succeeds, or <constantResult> otherwise (ie. a default value)
flatten like map((result) => result.flat())
stringify like map((result) => String(result)), but with extra stringification logic
## Error handling
recover apply the specified parser, and if it hard-fails, call <recovery: function(failureState)> and, if a new parserState is returned from it, return that instead of the original
--------
Message history access patterns:
- Window seek (fetch N messages before/after marker/ID)
- Insert N messages before/after marker/ID
- Update (message with given ID)
- Purge (messages that have not been accessed for N time / that have been least recently accessed)
Each 'message' should have an internal list of all applicable events which modify it, eg. edits and reactions
Maybe have the data structure expose an async 'seek' API which will transparently fetch messages if not locally available?
--------
# /sync response
Sync response:
```js
{
// Required. The batch token to supply in the since param of the next /sync request.
next_batch: "token",
// Updates to rooms.
rooms: { // Rooms
// The rooms that the user has joined.
join: {
"!room_id:example.com": { // Joined Room
// Information about the room which clients may need to correctly render it to users.
summary: { // RoomSummary
// The users which can be used to generate a room name if the room does not have one. Required if the room's m.room.name or m.room.canonical_alias state events are unset or empty.
"m.heroes": ["@foo:example.com", "@bar:example.com"],
// The number of users with membership of join, including the client's own user ID. If this field has not changed since the last sync, it may be omitted. Required otherwise.
"m.joined_member_count": 10,
// The number of users with membership of invite. If this field has not changed since the last sync, it may be omitted. Required otherwise.
"m.invited_member_count": 10
},
// Updates to the state, between the time indicated by the since parameter, and the start of the timeline (or all state up to the start of the timeline, if since is not given, or full_state is true).
state: { // State
// List of events.
events: [ stateEvent, stateEvent, stateEvent ]
},
// The timeline of messages and state changes in the room.
timeline: { // Timeline
// List of events.
events: [ roomEvent, roomEvent, roomEvent ],
// True if the number of events returned was limited by the limit on the filter.
limited: true,
// A token that can be supplied to the from parameter of the rooms/{roomId}/messages endpoint.
prev_batch: "string"
},
// The ephemeral events in the room that aren't recorded in the timeline or state of the room. e.g. typing.
ephemeral: { // Ephemeral
// List of events.
events: [ event, event, event ]
},
// The private data that this user has attached to this room.
account_data: { // Account Data
// List of events.
events: [ event, event, event ]
},
// Counts of unread notifications for this room. Servers MUST include the number of unread notifications in a client's /sync stream, and MUST update it as it changes. Notifications are determined by the push rules which apply to an event. When the user updates their read receipt (either by using the API or by sending an event), notifications prior to and including that event MUST be marked as read.
unread_notifications: { // Unread Notification Counts
// The number of unread notifications for this room with the highlight flag set
highlight_count: 10,
// The total number of unread notifications for this room
notification_count: 10
},
}
},
// The rooms that the user has been invited to.
invite: {
"!room_id:example.com": { // Invited Room
// The state of a room that the user has been invited to. These state events may only have the sender, type, state_key and content keys present. These events do not replace any state that the client already has for the room, for example if the client has archived the room. Instead the client should keep two separate copies of the state: the one from the invite_state and one from the archived state. If the client joins the room then the current state will be given as a delta against the archived state not the invite_state.
invite_state: { // InviteState
// The StrippedState events that form the invite state.
events: [ strippedEvent, strippedEvent, strippedEvent ]
}
}
},
// The rooms that the user has left or been banned from.
leave: {
"!room_id:example.com": { // Left Room
// The state updates for the room up to the start of the timeline.
state: { // State
// List of events.
events: [ stateEvent, stateEvent, stateEvent ]
},
// The timeline of messages and state changes in the room up to the point when the user left.
timeline: { // Timeline
// List of events.
events: [ roomEvent, roomEvent, roomEvent ],
// True if the number of events returned was limited by the limit on the filter.
limited: true,
// A token that can be supplied to the from parameter of the rooms/{roomId}/messages endpoint.
prev_batch: "string"
},
// The private data that this user has attached to this room.
account_data: { // Account Data
// List of events.
events: [ event, event, event ]
},
}
}
},
// The updates to the presence status of other users.
presence: { // Presence
// List of events.
events: [ event, event, event ]
},
// The global private data created by this user.
account_data: { // Account Data
// List of events.
events: [ event, event, event ]
},
// Optional. Information on the send-to-device messages for the client device.
to_device: { // ToDevice
// List of send-to-device messages.
events: [ toDeviceEvent, toDeviceEvent, toDeviceEvent ]
},
// Optional. Information on e2e device updates. Note: only present on an incremental sync.
device_lists: { // DeviceLists
// List of users who have updated their device identity keys, or who now share an encrypted room with the client since the previous sync response.
changed: [ "string", "string", "string" ],
// List of users with whom we do not share any encrypted rooms anymore since the previous sync response.
left: [ "string", "string", "string" ]
},
// Optional. For each key algorithm, the number of unclaimed one-time keys currently held on the server for this device.
device_one_time_keys_count: {
[algorithmName]: 10,
[algorithmName]: 10
}
}
```
Event:
```js
{
// Required. The fields in this object will vary depending on the type of event. When interacting with the REST API, this is the HTTP body.
content: varyingObject,
// Required. The type of event. This SHOULD be namespaced similar to Java package naming conventions e.g. 'com.example.subdomain.event.type'
type: "string",
}
```
Room event:
```js
{
... Event, // inherits
// Required. The globally unique event identifier.
event_id: "string",
// Required. The type of event. This SHOULD be namespaced similar to Java package naming conventions e.g. 'com.example.subdomain.event.type'
type: "string",
// Required. Contains the fully-qualified ID of the user who sent this event.
sender: "string",
// Required. The fields in this object will vary depending on the type of event. When interacting with the REST API, this is the HTTP body.
content: varyingObject,
// Required. Timestamp in milliseconds on originating homeserver when this event was sent.
origin_server_ts: 10,
// Required. The ID of the room associated with this event. Will not be present on events that arrive through /sync, despite being required everywhere else.
room_id: "string",
// Contains optional extra information about the event.
unsigned: { // UnsignedData
// The time in milliseconds that has elapsed since the event was sent. This field is generated by the local homeserver, and may be incorrect if the local time on at least one of the two servers is out of sync, which can cause the age to either be negative or greater than it actually is.
age: 10,
// Optional. The event that redacted this event, if any.
redacted_because: event,
// The client-supplied transaction ID, if the client being given the event is the same one which sent it.
transaction_id: "string"
}
}
```
State event:
```js
{
... RoomEvent, // inherits
// Required. A unique key which defines the overwriting semantics for this piece of room state. This value is often a zero-length string. The presence of this key makes this event a State Event. State keys starting with an @ are reserved for referencing user IDs, such as room members. With the exception of a few events, state events set with a given user's ID as the state key MUST only be set by that user.
state_key: "string"
// Optional. The previous content for this event. If there is no previous content, this key will be missing.
prev_content: varyingObject,
}
```
Stripped event:
```js
{
// Required. The type for the event.
type: "string",
// Required. The content for the event.
content: varyingObject,
// Required. The state_key for the event.
state_key: "string",
// Required. The sender for the event.
sender: "string"
}
```
To-device event:
```js
{
// The content of this event. The fields in this object will vary depending on the type of event.
content: varyingObject,
// The Matrix user ID of the user who sent this event.
sender: "string",
// The type of event.
type: "string"
}
```
-----------
Filter creation request:
```js
{
// List of event fields to include. If this list is absent then all fields are included. The entries may include '.' characters to indicate sub-fields. So ['content.body'] will include the 'body' field of the 'content' object. A literal '.' character in a field name may be escaped using a '\'. A server may include more fields than were requested.
event_fields: [ "field_path", "field_path", "field_path" ],
// The format to use for events. 'client' will return the events in a format suitable for clients. 'federation' will return the raw event as received over federation. The default is 'client'.
event_format: ( "client" || "federation" ),
// The presence updates to include.
presence: EventFilter,
// The user account data that isn't associated with rooms to include.
account_data: EventFilter,
// Filters to be applied to room data.
room: {
// A list of room IDs to exclude. If this list is absent then no rooms are excluded. A matching room will be excluded even if it is listed in the 'rooms' filter. This filter is applied before the filters in ephemeral, state, timeline or account_data
not_rooms: [ "room_id", "room_id", "room_id" ],
// A list of room IDs to include. If this list is absent then all rooms are included. This filter is applied before the filters in ephemeral, state, timeline or account_data
rooms: [ "room_id", "room_id", "room_id" ],
// Include rooms that the user has left in the sync, default false
include_leave: false,
// The state events to include for rooms.
state: StateFilter,
// The events that aren't recorded in the room history, e.g. typing and receipts, to include for rooms.
ephemeral: StateFilter,
// The message and state update events to include for rooms.
timeline: StateFilter,
// The per user account data to include for rooms.
account_data: StateFilter
}
}
```
EventFilter:
```js
{
// The maximum number of events to return.
limit: 10,
// A list of event types to include. If this list is absent then all event types are included. A '*' can be used as a wildcard to match any sequence of characters.
types: [ "m.type", "m.type", "m.type" ],
// A list of event types to exclude. If this list is absent then no event types are excluded. A matching type will be excluded even if it is listed in the 'types' filter. A '*' can be used as a wildcard to match any sequence of characters.
not_types: [ "m.type", "m.type", "m.type" ],
// A list of senders IDs to include. If this list is absent then all senders are included.
senders: [ "user_id", "user_id", "user_id" ],
// A list of sender IDs to exclude. If this list is absent then no senders are excluded. A matching sender will be excluded even if it is listed in the 'senders' filter.
not_senders: [ "user_id", "user_id", "user_id" ],
}
```
StateFilter:
```js
{
... EventFilter, // inherits
// A list of room IDs to include. If this list is absent then all rooms are included.
rooms: [ "room_id", "room_id", "room_id" ],
// A list of room IDs to exclude. If this list is absent then no rooms are excluded. A matching room will be excluded even if it is listed in the 'rooms' filter.
not_rooms: [ "room_id", "room_id", "room_id" ],
// If true, enables lazy-loading of membership events. Defaults to false.
lazy_load_members: false,
// If true, sends all membership events for all events, even if they have already been sent to the client. Does not apply unless lazy_load_members is true. Defaults to false.
include_redundant_members: false,
// If true, includes only events with a url key in their content. If false, excludes those events. If omitted, url key is not considered for filtering.
contains_url: ( true || false || null )
}
```
-----------
Junk
----
// Optional. This key will only be present for state events. A unique key which defines the overwriting semantics for this piece of room state.
state_key: "string",
// The MXID of the user who sent this event.
sender: "string",
// The content of this event. The fields in this object will vary depending on the type of event.
content: varyingObject,
// Timestamp in milliseconds on originating homeserver when this event was sent.
origin_server_ts: 10,
// Information about this event which was not sent by the originating homeserver
unsigned: { // Unsigned
// Time in milliseconds since the event was sent.
age: 10,
// Optional. The event that redacted this event, if any.
redacted_because: event
// Optional. The transaction ID set when this message was sent. This key will only be present for message events sent by the device calling this API.
transaction_id: "string",
// Optional. The previous content for this state. This will be present only for state events appearing in the timeline. If this is not a state event, or there is no previous content, this key will be missing.
prev_content: varyingObject,
}
item 14
Filter:
```json
{
"room": {
"state": {
"lazy_load_members": true
}
}
}
```

@ -0,0 +1,77 @@
{
"name": "@modular-matrix/client",
"description": "A Matrix client library",
"version": "0.1.0",
"main": "index.js",
"repository": "http://git.cryto.net/modular-matrix/client.git",
"author": "Sven Slootweg <admin@cryto.net>",
"license": "WTFPL OR CC0-1.0",
"devDependencies": {
"@babel/core": "^7.8.6",
"@babel/node": "^7.8.4",
"@babel/preset-env": "^7.8.6",
"@joepie91/eslint-config": "^1.1.0",
"babel-eslint": "^10.1.0",
"eslint": "^6.2.2",
"eslint-plugin-babel": "^5.3.0"
},
"dependencies": {
"@joepie91/unreachable": "^1.0.0",
"@modular-matrix/is-homeserver-url": "^0.1.0",
"@modular-matrix/is-mxc-url": "^0.1.0",
"@modular-matrix/parse-mxc": "^1.0.1",
"@promistream/buffer": "^0.1.0",
"@promistream/collect": "^0.1.0",
"@promistream/end-of-stream": "^0.1.0",
"@promistream/filter": "^0.1.1",
"@promistream/map": "^0.1.1",
"@promistream/pipe": "^0.1.0",
"@promistream/sequentialize": "^0.1.0",
"@promistream/simple-sink": "^0.1.0",
"@promistream/simple-source": "^0.1.1",
"@validatem/allow-extra-properties": "^0.1.0",
"@validatem/any-property": "^0.1.3",
"@validatem/anything": "^0.1.0",
"@validatem/array-of": "^0.1.2",
"@validatem/core": "^0.3.11",
"@validatem/default-to": "^0.1.0",
"@validatem/either": "^0.1.9",
"@validatem/error": "^1.1.0",
"@validatem/ignore-result": "^0.1.1",
"@validatem/is-boolean": "^0.1.1",
"@validatem/is-function": "^0.1.0",
"@validatem/is-integer": "^0.1.0",
"@validatem/is-non-empty-string": "^0.1.0",
"@validatem/is-number": "^0.1.3",
"@validatem/is-plain-object": "^0.1.1",
"@validatem/is-string": "^0.1.1",
"@validatem/is-url": "^0.1.0",
"@validatem/matches-format": "^0.1.0",
"@validatem/one-of": "^0.1.1",
"@validatem/require-either": "^0.1.0",
"@validatem/required": "^0.1.1",
"@validatem/wrap-error": "^0.1.3",
"as-expression": "^1.0.0",
"assure-array": "^1.0.0",
"axios": "^0.19.0",
"bluebird": "^3.7.2",
"concat-arrays": "^2.0.0",
"create-event-emitter": "^1.0.0",
"default-value": "^1.0.0",
"dotty": "^0.1.0",
"flatten": "^1.0.3",
"image-size": "^0.8.3",
"is-negative-zero": "^2.0.0",
"make-url": "^0.0.1",
"match-value": "^1.1.0",
"p-defer": "^3.0.0",
"p-try": "^2.2.0",
"parjs": "^0.12.7",
"split-filter": "^1.1.3",
"split-limit": "^1.0.4",
"supports-color": "^7.1.0",
"syncpipe": "^1.0.0",
"url-join": "^4.0.1",
"validatem": "^0.2.0"
}
}

@ -0,0 +1,14 @@
"use strict";
const { letter } = require("parjs");
const { then } = require("parjs/combinators");
function combine([ previous, current ]) {
return previous.concat([ current ]);
}
let test = letter().pipe(
then(combine)
);
console.log(test.parse(process.argv[2]));

@ -0,0 +1,2 @@
- Seems that in a `/sync` response, `timeline` can contain both room *and* state events (ie. it can contain events with state-event-specific keys), but this is not actually mentioned anywhere?
- For `/sync`, `Timeline` specifies for `prev_batch`: "can be supplied to the from parameter of the rooms/{roomId}/messages endpoint" -- but it is not specified what direction this is relevant to.

@ -0,0 +1,7 @@
# Error types
## PUT /_matrix/client/r0/rooms/{roomId}/send/{eventType}/{txnId}
- No error types specified at all
## POST /_matrix/media/r0/upload
- No way to distinguish between different 'forbidden' errors; no specific error code defined for 'forbidden filetype', 'exceeded quota', etc.

@ -0,0 +1,24 @@
"use strict";
const Promise = require("bluebird");
const axios = require("axios");
module.exports = function createLoginAgent({ homeserver }) {
return {
getMethods: function () {
return Promise.try(() => {
// axios get /login
}).then((response) => {
// extract available login methods
});
},
login: function ({ token, username, password }) {
// validate either token or (username, password) supplied
return Promise.try(() => {
// axios post /login
}).then((response) => {
// TODO
});
}
};
};

@ -0,0 +1,32 @@
"use strict";
// FIXME: Switch to `match-value` package
function getValue(value, functionsAreLiterals) {
if (typeof value === "function" && !functionsAreLiterals) {
return value();
} else {
return value;
}
}
function doMatchValue(value, arms, functionsAreLiterals) {
if (value == null) {
return value;
} else if (arms[value] !== undefined) {
// NOTE: We intentionally only check for `undefined` here (and below), since we want to allow the mapped-to value to be an explicit `null`.
return getValue(arms[value], functionsAreLiterals);
} else if (arms._ !== undefined) {
return getValue(arms._, functionsAreLiterals);
} else {
throw new Error(`No match arm found for value '${value}'`);
}
}
module.exports = function matchValue(value, arms) {
return doMatchValue(value, arms, true);
};
module.exports.literal = function matchValueLiteral(value, arms) {
return doMatchValue(value, arms, false);
};

@ -0,0 +1,15 @@
"use strict";
module.exports = function awaitImageLoad(imageElement) {
// FIXME: Validation
return new Promise((resolve, reject) => {
imageElement.addEventListener("load", (_event) => {
resolve();
});
imageElement.addEventListener("error", (_event) => {
// The event does not have an actual error stored on it
reject(new Error("Could not load image")); // FIXME: Error type
});
});
};

@ -0,0 +1,15 @@
"use strict";
module.exports = function awaitVideoElementLoad(videoElement) {
// FIXME: Validation
return new Promise((resolve, reject) => {
videoElement.addEventListener("loadeddata", (_event) => {
resolve();
});
videoElement.addEventListener("error", (event) => {
// FIXME: Check that this actually works as expected
reject(event.error);
});
});
};

@ -0,0 +1,19 @@
"use strict";
module.exports = function calculateThumbnailSize({ sourceWidth, sourceHeight, targetWidth, targetHeight }) {
// FIXME: Validation
// TODO: crop method, currently only implements scale/fit (crop would probably require inverting match arms, ceil instead of floor, and providing a drawing offset)
let sourceAspectRatio = sourceWidth / sourceHeight;
let targetAspectRatio = targetWidth / targetHeight;
let resizeFactor = (targetAspectRatio > sourceAspectRatio)
// Target is wider than source, constrain by height
? targetHeight / sourceHeight
// Target is taller than or equally-ratioed to source, constrain by width
: targetWidth / sourceWidth;
return {
width: Math.floor(sourceWidth * resizeFactor),
height: Math.floor(sourceHeight * resizeFactor)
};
};

@ -0,0 +1,12 @@
"use strict";
const calculateThumbnailSize = require("./");
let calculated = calculateThumbnailSize({
sourceWidth: 200,
sourceHeight: 100,
targetWidth: 80,
targetHeight: 60
});
console.log(calculated);

@ -0,0 +1,10 @@
"use strict";
module.exports = function canvasToBlob(canvas, mimeType) {
// FIXME: Validation
return new Promise((resolve, _reject) => {
canvas.toBlob(function(blob) {
resolve(blob);
}, mimeType);
});
};

@ -0,0 +1,14 @@
"use strict";
// TODO: Can probably be made more performant by precomputing a key -> item mapping, at the cost of multiple items with the same key becoming impossible
module.exports = function diffLists(oldArray, newArray, keyFunction = (item) => item) {
// NOTE: This only detects additions and removals, *not* order changes! The order is not relevant for our usecase.
let oldSet = new Set(oldArray.map((item) => keyFunction(item)));
let newSet = new Set(newArray.map((item) => keyFunction(item)));
let removed = oldArray.filter((item) => !newSet.has(keyFunction(item)));
let added = newArray.filter((item) => !oldSet.has(keyFunction(item)));
return { removed, added };
};

@ -0,0 +1,57 @@
"use strict";
const Promise = require("bluebird");
const asExpression = require("as-expression");
const unreachable = require("@joepie91/unreachable")("@modular-matrix/client"); // FIXME when packaging
const getImageDimensions = require("../get-image-dimensions");
const getVideoDimensions = require("../get-video-dimensions");
const calculateThumbnailSize = require("../calculate-thumbnail-size");
const canvasToBlob = require("../canvas-to-blob");
const { validateOptions } = require("@validatem/core");
const required = require("@validatem/required");
const isInteger = require("@validatem/is-integer");
const oneOf = require("@validatem/one-of");
module.exports = function elementToThumbnail(_options) {
// TODO: Add a 'crop' method in the future?
let { element, maximumWidth, maximumHeight, mimetype } = validateOptions(arguments, {
element: [ required ], // FIXME: Element type validation
maximumWidth: [ required, isInteger ], // FIXME: Positive integer
maximumHeight: [ required, isInteger ], // FIXME: Positive integer
mimetype: [ required, oneOf([ "image/png", "image/jpeg" ]) ]
});
let dimensions = asExpression(() => {
if (element instanceof HTMLImageElement) {
return getImageDimensions(element);
} else if (element instanceof HTMLVideoElement) {
return getVideoDimensions(element);
} else {
unreachable("Unrecognized element type");
}
});
let { width, height } = calculateThumbnailSize({
sourceWidth: dimensions.width,
sourceHeight: dimensions.height,
targetWidth: maximumWidth,
targetHeight: maximumHeight
});
let canvas = document.createElement("canvas");
canvas.width = width;
canvas.height = height;
canvas.getContext("2d").drawImage(element, 0, 0, width, height);
return Promise.try(() => {
return canvasToBlob(canvas, mimetype);
}).then((blob) => {
return {
width: width,
height: height,
blob: blob
};
});
};

@ -0,0 +1,9 @@
"use strict";
module.exports = function ensureValidDimension(value) {
if (value > 0) {
return value;
} else {
throw new Error(`Encountered invalid dimension value; this is probably a bug, please report it!`);
}
};

@ -0,0 +1,9 @@
"use strict";
module.exports = function addCommonFields(protocolEvent, newEvent) {
return Object.assign({
id: protocolEvent.event_id, // FIXME: Make sure that all derived timeline events set this correctly, like with power levels
sourceEventID: protocolEvent.event_id,
timestamp: protocolEvent.origin_server_ts
}, newEvent);
};

@ -0,0 +1,7 @@
"use strict";
const itemDeduplicator = require("../item-deduplicator");
module.exports = function createEventDeduplicator() {
return itemDeduplicator((event) => event.event_id);
};

@ -0,0 +1,18 @@
"use strict";
module.exports = function fileToDataURL(dataUrl) {
return new Promise((resolve, reject) => {
const reader = new FileReader();
reader.onload = function(event) {
resolve(event.target.result);
};
reader.onerror = function(event) {
// FIXME: Verify that this works
reject(event.error);
};
reader.readAsDataURL(dataUrl);
});
};

@ -0,0 +1,7 @@
"use strict";
const nanoid = require("nanoid").nanoid;
module.exports = function generateTransactionID() {
return nanoid();
};

@ -0,0 +1,45 @@
"use strict";
const url = require("url");
const makeURL = require("make-url");
const parseMXC = require("@modular-matrix/parse-mxc");
const { validateOptions } = require("@validatem/core");
const required = require("@validatem/required");
const requireEither = require("@validatem/require-either");
const isString = require("@validatem/is-string");
const isMXC = require("@modular-matrix/is-mxc-url");
const isSession = require("@modular-matrix/is-session");
const isHomeserverURL = require("@modular-matrix/is-homeserver-url");
module.exports = function getFileURL(_session, _options) {
let options = validateOptions(arguments, [
required,
requireEither([ "session", "homeserver" ]),
{
session: [ isSession ],
homeserver: [ isHomeserverURL ],
url: [ required, isMXC ],
filename: [ isString ]
}
]);
let homeserverURL = (options.session != null)
? options.session.homeserver
: options.homeserver;
let parsedMXC = parseMXC.parse(options.url);
let urlTemplate = (options.filename != null)
? "/_matrix/media/r0/download/:serverName/:mediaID/:filename"
: "/_matrix/media/r0/download/:serverName/:mediaID";
let path = makeURL(urlTemplate, {
serverName: parsedMXC.homeserver,
mediaID: parsedMXC.id,
filename: options.filename
});
return url.resolve(homeserverURL, path);
};

@ -0,0 +1,18 @@
"use strict";
const Promise = require("bluebird");
const createSession = require("@modular-matrix/create-session");
const getFileURL = require(".");
return Promise.try(() => {
return createSession("https://pixie.town/", { accessToken: require("../../../private/access-token") });
}).then((session) => {
let url = getFileURL({
session: session,
// url: "mxc://pixie.town/NKsNxCkItRtbpRunyNYHCxsW",
url: "mxc://matrix.org/qmhOpvpttfkmucqyycSzcfvk",
filename: "image.jpg"
});
console.log(url);
});

@ -0,0 +1,39 @@
"use strict";
const Promise = require("bluebird");
const fs = require("fs");
function analyzeBuffer(buffer) {
return buffer.length;
}
function analyzeBlob(blob) {
return blob.size;
}
function analyzeStream(stream) {
return Promise.try(() => {
return fs.promises.stat(stream.path);
}).then((stat) => {
return stat.size;
});
}
function analyzeString(string) {
// NOTE: string.length would produce the size in *code units*, but we want the size in *bytes*
return (new TextEncoder().encode(string)).length;
}
module.exports = function getFilesize(file) {
if (typeof file === "string") {
return analyzeString(file);
} else if (typeof Blob !== "undefined" && file instanceof Blob) {
return analyzeBlob(file);
} else if (Buffer.isBuffer(file)) {
return analyzeBuffer(file);
} else if (file._readableState != null && file.path != null) {
return analyzeStream(file);
} else {
throw new Error(`Invalid file passed`); // FIXME: Validate
}
};

@ -0,0 +1,12 @@
"use strict";
const ensureValidDimension = require("../ensure-valid-dimension");
module.exports = function getImageDimensions(imageElement) {
// FIXME: Validation
return {
width: ensureValidDimension(imageElement.naturalWidth),
height: ensureValidDimension(imageElement.naturalHeight)
};
};

@ -0,0 +1,50 @@
"use strict";
const url = require("url");
const makeURL = require("make-url");
const parseMXC = require("@modular-matrix/parse-mxc");
const { validateOptions } = require("@validatem/core");
const required = require("@validatem/required");
const requireEither = require("../require-either");
const isInteger = require("@validatem/is-integer");
const oneOf = require("@validatem/one-of");
const isMXC = require("@modular-matrix/is-mxc-url");
const isSession = require("@modular-matrix/is-session");
const isHomeserverURL = require("@modular-matrix/is-homeserver-url");
module.exports = function getThumbnailURL(_session, _options) {
let options = validateOptions(arguments, [
required,
requireEither([ "session", "homeserver" ]),
{
session: [ isSession ],
homeserver: [ isHomeserverURL ],
url: [ required, isMXC ],
method: [ required, oneOf([ "crop", "scale" ]) ],
minimumWidth: [ required, isInteger ],
minimumHeight: [ required, isInteger ]
}
]);
let homeserverURL = (options.session != null)
? options.session.homeserver
: options.homeserver;
let parsedMXC = parseMXC.parse(options.url);
let path = makeURL("/_matrix/media/r0/thumbnail/:serverName/:mediaID", {
serverName: parsedMXC.homeserver,
mediaID: parsedMXC.id
});
return url.resolve(homeserverURL, url.format({
pathname: path,
query: {
method: options.method,
width: options.minimumWidth,
height: options.minimumHeight
}
}));
};

@ -0,0 +1,66 @@
"use strict";
const Promise = require("bluebird");
const mmAxios = require("@modular-matrix/axios");
const getThumbnailURL = require("../get-thumbnail-url");
const withoutKeys = require("../without-keys");
const universalImageMetadata = require("../universal-image-metadata");
const defaultValue = require("default-value");
const { validateOptions } = require("@validatem/core");
const required = require("@validatem/required");
const requireEither = require("@validatem/require-either");
const isInteger = require("@validatem/is-integer");
const isBoolean = require("@validatem/is-boolean");
const defaultTo = require("@validatem/default-to");
const oneOf = require("@validatem/one-of");
const isMXC = require("@modular-matrix/is-mxc-url");
const isSession = require("@modular-matrix/is-session");
const isHomeserverURL = require("@modular-matrix/is-homeserver-url");
module.exports = function getThumbnail(_options) {
let options = validateOptions(arguments, [
required,
requireEither([ "session", "homeserver" ]),
{
session: [ isSession ],
homeserver: [ isHomeserverURL ],
url: [ required, isMXC ],
method: [ required, oneOf([ "fit", "fill" ]) ],
minimumWidth: [ required, isInteger ],
minimumHeight: [ required, isInteger ],
stream: [ isBoolean, defaultTo(false) ]
}
]);
let axios = mmAxios({
session: options.session,
homeserver: options.homeserver
});
let thumbnailOptions = withoutKeys(options, [ "stream" ]);
let url = getThumbnailURL(thumbnailOptions);
return Promise.try(() => {
return axios.get(url, {
// TODO: Support "blob" in browsers?
responseType: (options.stream === true) ? "stream" : "arraybuffer"
});
}).then((response) => {
return Promise.try(() => {
if (options.stream === false) {
return universalImageMetadata(response.data);
} else {
return { width: undefined, height: undefined, mimetype: undefined };
}
}).then((metadata) => {
return {
buffer: (options.stream === false) ? response.data : undefined,
stream: (options.stream === true) ? response.data : undefined,
mimetype: defaultValue(metadata.mimetype, response.headers["content-type"]),
width: metadata.width,
height: metadata.height
};
});
});
};

@ -0,0 +1,22 @@
"use strict";
const Promise = require("bluebird");
const mmAxios = require("@modular-matrix/axios");
const { validateArguments } = require("@validatem/core");
const required = require("@validatem/required");
const isSession = require("@modular-matrix/is-session");
module.exports = function getThumbnail(_session) {
let [ session ] = validateArguments(arguments, {
session: [ required, isSession ]
});
let axios = mmAxios({ session: session });
return Promise.try(() => {
return axios.get("/media/r0/config");
}).then((response) => {
return { limit: response.data["m.upload.size"] };
});
};

@ -0,0 +1,15 @@
"use strict";
const Promise = require("bluebird");
const createSession = require("@modular-matrix/create-session");
const getUploadSizeLimit = require(".");
return Promise.try(() => {
return createSession("https://pixie.town/", { accessToken: require("../../../private/access-token") });
}).then((session) => {
return Promise.try(() => {
return getUploadSizeLimit(session);
}).then((result) => {
console.log(result);
});
});

@ -0,0 +1,12 @@
"use strict";
const ensureValidDimension = require("../ensure-valid-dimension");
module.exports = function getVideoDimensions(videoElement) {
// FIXME: Validation
return {
width: ensureValidDimension(videoElement.videoWidth),
height: ensureValidDimension(videoElement.videoHeight)
};
};

@ -0,0 +1,11 @@
"use strict";
const required = require("@validatem/required");
const isEvent = require("../is-event");
const isMatrixID = require("../is-matrix-id");
module.exports = {
... isEvent,
sender: [ required, isMatrixID ]
};

@ -0,0 +1,6 @@
"use strict";
const isString = require("@validatem/is-string");
// FIXME: Improve validation
module.exports = isString;

@ -0,0 +1,10 @@
"use strict";
const required = require("@validatem/required");
const isString = require("@validatem/is-string");
const isPlainObject = require("@validatem/is-plain-object");
module.exports = {
type: [ required, isString ],
content: [ required, isPlainObject ]
};

@ -0,0 +1,8 @@
"use strict";
const isURL = require("@validatem/is-url");
// FIXME: Enforce that the path component is empty! Homeservers must always exist at the root of their hostname
// FIXME: Replace all isString instances where this should be used instead
module.exports = isURL(["https"]);
// MARKER: Package and fix this

@ -0,0 +1,6 @@
"use strict";
const isString = require("@validatem/is-string");
// FIXME: Improve validation
module.exports = isString;

@ -0,0 +1,19 @@
"use strict";
const required = require("@validatem/required");
const isString = require("@validatem/is-string");
const arrayOf = require("@validatem/array-of");
const isTimelineEvent = require("../is-timeline-event");
const isStateEvent = require("../is-state-event");
const optionalArray = require("../optional-array");
let isTimelineList = arrayOf([ required, isTimelineEvent ]);
let isStateList = arrayOf([ required, isStateEvent ]);
module.exports = {
start: [ required, isString ],
end: [ isString ],
chunk: optionalArray(isTimelineList),
state: optionalArray(isStateList)
};

@ -0,0 +1,13 @@
"use strict";
const arrayOf = require("@validatem/array-of");
const isBoolean = require("@validatem/is-boolean");
const isInteger = require("@validatem/is-integer");
module.exports = function isPaginatedChunkOf(rules) {
return {
chunk: arrayOf(rules),
limited: isBoolean,
count: isInteger
};
};

@ -0,0 +1,12 @@
"use strict";
const required = require("@validatem/required");
const isEvent = require("../is-event");
const isMatrixID = require("../is-matrix-id");
// NOTE: Unspecced, see https://github.com/matrix-org/matrix-doc/issues/2680 - this would normally just be `isEvent` as per the spec
module.exports = {
... isEvent,
sender: [ required, isMatrixID ]
};

@ -0,0 +1,68 @@
"use strict";
const required = require("@validatem/required");
const isString = require("@validatem/is-string");
const isInteger = require("@validatem/is-integer");
const isPlainObject = require("@validatem/is-plain-object");
const isEvent = require("../is-event");
const isMatrixID = require("../is-matrix-id");
const isEventID = require("../is-event-id");
const isRoomID = require("../is-room-id");
const optionalObject = require("../optional-object");
const isPaginatedChunkOf = require("../is-paginated-chunk-of");
module.exports = {
... isEvent,
event_id: [ required, isString ],
sender: [ required, isMatrixID ],
origin_server_ts: [ required, isInteger ],
// In spec, but missing from Room Event format: https://github.com/matrix-org/matrix-doc/issues/2684
redacts: isEventID, // FIXME: Make required when redaction-type event
// Spec omission: https://github.com/matrix-org/matrix-doc/issues/2685
age_ts: isInteger,
room_id: [ isRoomID ], // FIXME: Not present on /sync, but will need to be required-checked for event validation elsewhere
// Synapse bug: https://github.com/matrix-org/synapse/issues/7925
age: isInteger,
// Synapse bug: https://github.com/matrix-org/synapse/issues/7924
user_id: isMatrixID,
// Synapse bug: https://github.com/matrix-org/synapse/issues/7925#issuecomment-662089208
replaces_state: isEventID,
// Synapse bug: https://github.com/matrix-org/synapse/issues/7925#issuecomment-663247760
redacted_because: isPlainObject,
// Obsolete field originating from a now-defunct Synapse fork running on ponies.im
origin_server_ipts: [ isInteger ],
unsigned: optionalObject({
age: isInteger,
transaction_id: isString,
redacted_because: isPlainObject, // FIXME: Cannot do recursion with isRoomEvent (/isStateEvent) -- fixable via `dynamic` wrapper, so that the rules are only generated *after* the validator has finished being declared?
// Spec omission: https://github.com/matrix-org/matrix-doc/issues/2690
redacted_by: isEventID,
// Spec omission: https://github.com/matrix-org/matrix-doc/issues/1167
replaces_state: isEventID,
// Spec omission and/or Synapse bug: https://github.com/matrix-org/matrix-doc/issues/877
prev_content: isPlainObject,
// Spec omission: https://github.com/matrix-org/matrix-doc/issues/684
prev_sender: isMatrixID,
// MSC 1849, not merged yet: https://github.com/matrix-org/matrix-doc/blob/matthew/msc1849/proposals/1849-aggregations.md
"m.relations": {
"m.annotation": isPaginatedChunkOf({
type: [ required, isString ],
key: [ required, isString ],
// Should be required according to MSC, but currently missing in Synapse: https://github.com/matrix-org/synapse/issues/7941#issuecomment-663238820
origin_server_ts: [ /* required, */ isInteger ],
count: [ required, isInteger ]
}),
"m.reference": isPaginatedChunkOf({
// Should be required according to MSC, but currently missing in Synapse: https://github.com/matrix-org/synapse/issues/7941
type: [ /* required, */ isString ],
event_id: [ required, isEventID ]
}),
"m.replace": {
event_id: [ required, isEventID ],
origin_server_ts: [ required, isInteger ],
sender: [ required, isMatrixID ]
}
}
})
};

@ -0,0 +1,6 @@
"use strict";
const isString = require("@validatem/is-string");
// FIXME: Improve validation
module.exports = isString;

@ -0,0 +1,16 @@
"use strict";
const required = require("@validatem/required");
const isString = require("@validatem/is-string");
const isPlainObject = require("@validatem/is-plain-object");
const isInteger = require("@validatem/is-integer");
const isRoomEvent = require("../is-room-event");
module.exports = {
... isRoomEvent,
state_key: [ required, isString ],
prev_content: isPlainObject,
// Spec violation by Synapse: https://github.com/matrix-org/synapse/issues/6226
membership: [ isString ]
};

@ -0,0 +1,8 @@
"use strict";
const wrapError = require("@validatem/wrap-error");
const matchesFormat = require("@validatem/matches-format");
module.exports = wrapError("Must be a validly-formatted stream token", "modular-matrix.is-stream-token", [
matchesFormat(/^[a-zA-Z0-9.=_-]+$/)
]);

@ -0,0 +1,13 @@
"use strict";
const required = require("@validatem/required");
const isString = require("@validatem/is-string");
const isEvent = require("../is-event");
const isMatrixID = require("../is-matrix-id");
module.exports = {
... isEvent,
state_key: [ required, isString ],
sender: [ required, isMatrixID ]
};

@ -0,0 +1,112 @@
"use strict";
// FIXME: own package
const required = require("@validatem/required");
const anyProperty = require("@validatem/any-property");
const arrayOf = require("@validatem/array-of");
const isString = require("@validatem/is-string");
const isInteger = require("@validatem/is-integer");
const isBoolean = require("@validatem/is-boolean");
const anything = require("@validatem/anything");
const isEvent = require("../is-event");
const isPresenceEvent = require("../is-presence-event");
const isStateEvent = require("../is-state-event");
const isDeviceEvent = require("../is-device-event");
const isStrippedEvent = require("../is-stripped-event");
const isTimelineEvent = require("../is-timeline-event");
const isRoomID = require("../is-room-id");
const isMatrixID = require("../is-matrix-id");
const optionalObject = require("../optional-object");
const optionalArray = require("../optional-array");
let isStateList = arrayOf([ required, isStateEvent ]);
let isEventList = arrayOf([ required, isEvent ]);
let isPresenceEventList = arrayOf([ required, isPresenceEvent ]);
let isStrippedEventList = arrayOf([ required, isStrippedEvent ]);
let isDeviceEventList = arrayOf([ required, isDeviceEvent ]);
let isTimelineList = arrayOf([ required, isTimelineEvent ]);
module.exports = {
next_batch: [ required, isString ],
// FIXME: also optionalObject for `rooms`
rooms: {
join: optionalObject(anyProperty({
key: [ required, isRoomID ],
value: {
summary: optionalObject({
"m.heroes": arrayOf([ required, isString ]),
"m.joined_member_count": isInteger,
"m.invited_member_count": isInteger,
}),
// NOTE: Despite what the spec currently says, state.events *can* contain membership events when the timeline isn't limited, when lazy-loading is enabled
state: optionalObject({
events: optionalArray(isStateList)
}),
timeline: optionalObject({
events: optionalArray(isTimelineList),
limited: isBoolean,
prev_batch: isString
}),
ephemeral: optionalObject({
events: optionalArray(isEventList)
}),
account_data: optionalObject({
events: optionalArray(isEventList)
}),
unread_notifications: {
highlight_count: isInteger,
notification_count: isInteger
},
// FIXME: expose the below
"org.matrix.msc2654.unread_count": [ isInteger ], // NOTE: https://github.com/matrix-org/matrix-doc/pull/2654
}
})),
invite: optionalObject(anyProperty({
key: [ required, isRoomID ],
value: {
// NOTE: This state needs to be maintained separately from known room state (see spec). FIXME: Represent this in the event list output?
invite_state: optionalObject({
events: optionalArray(isStrippedEventList)
})
}
})),
leave: optionalObject(anyProperty({
key: [ required, isRoomID ],
value: {
state: optionalObject({
events: optionalArray(isStateList)
}),
timeline: optionalObject({
events: optionalArray(isTimelineList),
limited: isBoolean,
prev_batch: isString
}),
account_data: optionalObject({
events: optionalArray(isEventList)
}),
}
}))
},
presence: optionalObject({
events: optionalArray(isPresenceEventList)
}),
account_data: optionalObject({
events: optionalArray(isEventList)
}),
to_device: optionalObject({
events: optionalArray(isDeviceEventList)
}),
device_lists: optionalObject({
changed: arrayOf([ required, isMatrixID ]),
left: arrayOf([ required, isMatrixID ])
}),
device_one_time_keys_count: optionalObject(anyProperty({
key: [ required, isString ], // algorithm name
value: [ required, isInteger ] // key count
})),
groups: anything, // NOTE: Non-standard
// TODO: Validate algorithm names below?
"org.matrix.msc2732.device_unused_fallback_key_types": optionalArray(arrayOf([ required, isString ])), // NOTE: https://github.com/matrix-org/matrix-doc/pull/2732
};

@ -0,0 +1,13 @@
"use strict";
const isNonEmptyString = require("@validatem/is-non-empty-string");
const isInteger = require("@validatem/is-integer");
const isMXC = require("@modular-matrix/is-mxc-url");
module.exports = {
url: [ isMXC ],
displayWidth: [ isInteger ],
displayHeight: [ isInteger ],
mimetype: [ isNonEmptyString ],
filesize: [ isInteger ],
};

12