diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index bfef0cf7..7cebe1a1 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -12,14 +12,13 @@ Lysand is built using the following technologies:
- [Bun](https://bun.sh) - A JavaScript runtime similar to Node.js, but faster and with more features
- [PostgreSQL](https://www.postgresql.org/) - A relational database
- [`pg_uuidv7`](https://github.com/fboulnois/pg_uuidv7) - A PostgreSQL extension that provides a UUIDv7 data type
-- [UnoCSS](https://unocss.dev) - A utility-first CSS framework, used for the login page
+- [Nuxt](https://nuxt.com/) - A Vue.js framework, used for the frontend
- [Docker](https://www.docker.com/) - A containerization platform, used for development and deployment
- [Sharp](https://sharp.pixelplumbing.com/) - An image processing library, used for fast image resizing and converting
- [TypeScript](https://www.typescriptlang.org/) - A typed superset of JavaScript
-- [ESLint](https://eslint.org/) - A JavaScript linter
-- [Prettier](https://prettier.io/) - A code formatter
## Getting Started
+
To get started, please follow these steps:
1. Fork the repository, clone it on your local system and make your own branch
@@ -39,50 +38,9 @@ git clone https://github.com/lysand-org/lysand.git
bun install
```
-3. Set up a PostgreSQL database, using the `pg_uuidv7` extension
+3. Set up a PostgreSQL database (you need a special extension, please look at [the database documentation](database.md))
-You may use the following [Dockerfile](Postgres.Dockerfile) to set it up:
-
-```Dockerfile
-# Use the latest Postgres Docker image based on Alpine
-FROM postgres:alpine
-
-# Set working directory
-WORKDIR /usr/src/app
-
-# Install curl
-RUN apk add --no-cache curl
-
-RUN cd "$(mktemp -d)" \
- && curl -LO "https://github.com/fboulnois/pg_uuidv7/releases/download/v1.3.0/{pg_uuidv7.tar.gz,SHA256SUMS}" \
- && tar xf pg_uuidv7.tar.gz \
- && sha256sum -c SHA256SUMS \
- && PG_MAJOR=$(pg_config --version | sed 's/^.* \([0-9]\{1,\}\).*$/\1/') \
- && cp "$PG_MAJOR/pg_uuidv7.so" "$(pg_config --pkglibdir)" \
- && cp sql/pg_uuidv7--1.3.sql pg_uuidv7.control "$(pg_config --sharedir)/extension"
-# Add a script to run the CREATE EXTENSION command
-RUN echo '#!/bin/sh\npsql -U "$POSTGRES_USER" -d "$POSTGRES_DB" -c "CREATE EXTENSION pg_uuidv7;"' > /docker-entrypoint-initdb.d/init.sh
-
-# Make the entrypoint script executable
-RUN chmod +x /docker-entrypoint-initdb.d/init.sh
-```
-
-4. Copy the `config.toml.example` file to `config.toml` and fill in the values (you can leave most things to the default, but you will need to configure things such as the database connection)
-
-> [!WARNING]
-> You should disable Prisma Redis caching while developing, as it can mess up tests
-
-5. Generate the Prisma client:
-
-```bash
-bun prisma generate
-```
-
-6. Run migrations:
-
-```bash
-bun migrate
-```
+4. Copy the `config/config.toml.example` file to `config/config.toml` and edit it to set up the database connection and other settings.
## Testing your changes
@@ -93,14 +51,14 @@ bun dev
If your port number is lower than 1024, you may need to run the command as root.
-### Running the Vite server
+### Running the FE
-To start the Vite server, run:
+To start the frontend server, run:
```sh
-bun vite:dev
+bun fe:dev
```
-This should be run in a separate terminal window. The Vite server is used to serve the frontend assets and to provide hot module reloading.
+This should be run in a separate process as the server.
## Running tests
@@ -109,37 +67,32 @@ To run the tests, run:
bun test
```
-The tests are located in the `tests/` directory and follow a Jest-like syntax. The server does not need to be started before running the tests, as the tests will spawn their own Lysand server instance.
+The tests are located in the `tests/` directory and follow a Jest-like syntax. The server should be shut down before running the tests.
## Code style
-We use ESLint and Prettier to enforce a consistent code style. To check if your code is compliant, run:
+We use [Biome](https://biomejs.dev) to enforce a consistent code style. To check if your code is compliant, run:
+
```sh
-bun lint
+bunx @biomejs/biome check .
```
To automatically fix the issues, run:
```sh
-bun lint --fix
+bunx @biomejs/biome check . --apply
```
-You should have the [ESLint](https://marketplace.visualstudio.com/items?itemName=dbaeumer.vscode-eslint) and [Prettier](https://marketplace.visualstudio.com/items?itemName=esbenp.prettier-vscode) extensions installed in VSCode, if you use it. From the ESLint extension, you can automatically fix the issues with `Ctrl+Shift+P` and `ESLint: Fix all auto-fixable Problems`.
-
-ESLint and Prettier are also integrated in the CI pipeline, so your code will be automatically checked when you push it. If the pipeline fails, you will need to fix the issues before your pull request can be merged.
-
-Code style such as brackets, spaces/tabs, etc are enforced by Prettier's ESLint plugin. You can find the simple configuration in the `.prettierrc` file.
+You can also install the Biome Visual Studio Code extension and have it format your code automatically on save.
### ESLint rules
-ESLint errors should not be ignored, except if they are false positives, in which case you can use a comment to disable the rule for the line or the file. If you need to disable a rule, please add a comment explaining why.
+Linting should not be ignored, except if they are false positives, in which case you can use a comment to disable the rule for the line or the file. If you need to disable a rule, please add a comment explaining why.
TypeScript errors should be ignored with `// @ts-expect-error` comments, as well as with a reason for being ignored.
### Commit messages
-We use [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) for our commit messages. This allows us to automatically generate the changelog and the version number.
-
-> **Note**: I don't actually enforce this rule, but it would be great if you could follow it.
+We use [Conventional Commits](https://www.conventionalcommits.org) for our commit messages. This allows us to automatically generate the changelog and the version number.
### Pull requests
@@ -151,13 +104,15 @@ We use Bun's integrated testing system to write tests. You can find more informa
Tests **should** be written for all API routes and all functions that are not trivial. If you are not sure whether you should write a test for something, you probably should.
-To help with the creation of tests, you may find [GitHub Copilot](https://copilot.github.com/) useful (or some of its free alternatives like [Codeium](https://codeium.com/)). Please do not blindly copy the code that it generates, but use it as a starting point for your own tests. I recognize that writing tests is very tedious, which is why LLMs can come in handy.
+#### Adding per-route tests
+
+To add tests for a route, create a `route_file_name.test.ts` file in the same directory as the route itself. See [this example](/server/api/api/v1/timelines/home.test.ts) for help writing tests.
### Writing documentation
Documentation for the Lysand protocol is available on [lysand.org](https://lysand.org/). If you are thinking of modifying the protocol, please make sure to send a pull request over there to get it approved and merged before you send your pull request here.
-This project should not need much documentation, but if you think that something needs to be documented, please add it to the README or contribution guide.
+This project should not need much documentation, but if you think that something needs to be documented, please add it to the README, docs or contribution guide.
## Reporting bugs
diff --git a/README.md b/README.md
index bee018ee..9e512b32 100644
--- a/README.md
+++ b/README.md
@@ -2,11 +2,11 @@
-       [](code_of_conduct.md)
+      [](code_of_conduct.md)
## What is this?
-This is a project to create a federated social network based on the [Lysand](https://lysand.org) protocol. It is currently in alpha phase, with basic federation and almost complete Mastodon API support.
+This is a project to create a federated social network based on the [Lysand](https://lysand.org) protocol. It is currently in beta phase, with basic federation and almost complete Mastodon API support.
This project aims to be a fully featured social network, with a focus on privacy, security, and performance. It implements the Mastodon API for support with clients that already support Mastodon or Pleroma.
@@ -15,7 +15,7 @@ This project aims to be a fully featured social network, with a focus on privacy
## Features
-- [x] Inbound federation
+- [x] Federation (partial)
- [x] Hyper fast (thousands of HTTP requests per second)
- [x] S3 or local media storage
- [x] Deduplication of uploaded files
@@ -26,12 +26,11 @@ This project aims to be a fully featured social network, with a focus on privacy
- [x] Automatic image conversion to WebP or other formats
- [x] Scripting-compatible CLI with JSON and CSV outputs
- [ ] Moderation tools
-- [ ] Full Mastodon API support
-- [ ] Outbound federation
+- [x] Mastodon API support (partial)
## Screenshots
-Here are some screenshots of Lysand's built-in web client. This client is built with Vue, and serves to allow users to log in or register. It is kept to only the actions that cannot be done with a Mastodon client, to simplify development. In the future, there may be a full-featured client.
+Here are some screenshots of Lysand's built-in web client. This client is built with Nuxt, and serves to allow users to log in or register. It is kept to only the actions that cannot be done with a Mastodon client, to simplify development. In the future, there will be a full-featured client.
### On Desktop
@@ -114,130 +113,7 @@ Lysand is extremely fast and can handle thousands of HTTP requests per second on
## How do I run it?
-### Requirements
-
-- The [Bun Runtime](https://bun.sh), version 1.0.30 or later (usage of the latest version is recommended)
-- A PostgreSQL database
-- (Optional but recommended) A Linux-based operating system
-- (Optional if you want search) A working Meiliseach instance
-
-> [!WARNING]
-> Lysand has not been tested on Windows or MacOS. It is recommended to use a Linux-based operating system to run Lysand.
->
-> We will not be offerring support to Windows or MacOS users. If you are using one of these operating systems, please use a virtual machine or container to run Lysand.
-
-### Installation
-
-1. Clone this repository
-
-```bash
-git clone https://github.com/lysand-org/lysand.git
-```
-
-2. Install the dependencies
-
-```bash
-bun install
-```
-
-3. Set up a PostgreSQL database, using the `pg_uuidv7` extension
-
-Lysand offers a pre-made Docker image for PostgreSQL with the extension already installed. Use `ghcr.io/lysand-org/postgres:main` as your Docker image name to use it.
-
-You may otherwise use this [Dockerfile](Postgres.Dockerfile) to set it up.
-
-1. Copy the `config.toml.example` file to `config.toml` and fill in the values (you can leave most things to the default, but you will need to configure things such as the database connection)
-
-2. Run migrations:
-
-```bash
-bun migrate
-```
-
-6. (If you want search)
-Create a Meilisearch instance (using Docker is recommended). For a [`docker-compose`] file, copy the `meilisearch` service from the [`docker-compose.yml`](docker-compose.yml) file.
-
-Set up Meiliseach's API key by passing the `MEILI_MASTER_KEY` environment variable to the server. Then, enale and configure search in the config file.
-7. Build everything:
-
-```bash
-bun prod-build
-```
-
-You may now start the server with `bun start`. It lives in the `dist/` directory, all the other code can be removed from this point onwards.
-In fact, the `bun start` script merely runs `bun run dist/index.js --prod`!
-### Running
-
-To run the server, simply run the following command:
-
-```bash
-bun start
-```
-
-### Using the CLI
-
-Lysand includes a built-in CLI for managing the server. To use it, simply run the following command:
-
-```bash
-bun cli help
-```
-
-If you are running a production build, you will need to run `bun run dist/cli.js` or `./entrypoint.sh cli` instead.
-
-You can use the `help` command to see a list of available commands. These include creating users, deleting users and more. Each command also has a `--help,-h` flag that you can use to see more information about the command.
-
-#### Scripting with the CLI
-
-Some CLI commands that return data as tables can be used in scripts. To convert them to JSON or CSV, some commands allow you to specify a `--format` flag that can be either `"json"` or `"csv"`. See `bun cli help` or `bun cli -h` for more information.
-
-Flags can be used in any order and anywhere in the script (except for the `bun cli` command itself). The command arguments themselves must be in the correct order, however.
-
-### Rebuilding the Search Index
-
-You may use the `bun cli index rebuild` command to automatically push all posts and users to Meilisearch, if it is configured. This is useful if you have just set up Meilisearch, or if you accidentally deleted something.
-
-### Using Database Commands
-
-The `bun prisma` commands allows you to use Prisma commands without needing to add in environment variables for the database config. Just run Prisma commands as you would normally, replacing `bunx prisma` with `bun prisma`.
-
-## With Docker
-
-> [!NOTE]
-> Docker is currently broken, as Bun with Prisma does not work well with Docker yet for unknown reasons. The following instructions are for when this is fixed.
->
-> These instructions will probably also work with Podman and other container runtimes.
-
-You can also run Lysand using Docker. To do so, you can:
-
-1. Acquire the Postgres Dockerfile from above
-2. Use this repository's [`docker-compose.yml`](docker-compose.yml) file
-3. Create the `lysand-net` docker network:
-```bash
-docker network create lysand-net
-```
-1. Fill in the config file (see [Installation](#installation))
-2. Run the following command:
-```bash
-docker-compose up -d
-```
-
-You may need root privileges to run Docker commands.
-
-### Running CLI commands inside Docker
-
-You can run CLI commands inside Docker using the following command:
-
-```bash
-sudo docker exec -it lysand sh entrypoint.sh cli ...
-```
-
-### Running migrations inside Docker
-
-You can run migrations inside Docker using the following command (if needed):
-
-```bash
-sudo docker exec -it lysand sh entrypoint.sh prisma migrate deploy
-```
+Please see the [installation guide](docs/installation.md) for more information on how to install Lysand.
## Contributing
@@ -250,9 +126,6 @@ Contributions are welcome! Please see the [CONTRIBUTING.md](CONTRIBUTING.md) fil
## Federation
-> [!WARNING]
-> Federation has not been tested outside of automated tests. It is not recommended to use this software in production.
-
The following extensions are currently supported or being worked on:
- `org.lysand:custom_emojis`: Custom emojis
@@ -304,23 +177,24 @@ Working endpoints are:
- `/api/v1/mutes`
- `/api/v2/media`
- `/api/v1/notifications`
-
-Tests needed but completed:
-
- `/api/v1/media/:id`
- `/api/v2/media`
- `/api/v1/favourites`
- `/api/v1/accounts/:id/followers`
- `/api/v1/accounts/:id/following`
- `/api/v2/search`
+- `/api/v1/follow_requests`
+- `/api/v1/follow_requests/:account_id/authorize`
+- `/api/v1/follow_requests/:account_id/reject`
+- `/api/v1/statuses/:id/mute`
+- `/api/v1/statuses/:id/unmute`
+- `/api/v1/statuses/:id` (`PUT`)
+- `/api/v1/statuses/:id/source`
Endpoints left:
- `/api/v1/reports`
- `/api/v1/accounts/:id/lists`
-- `/api/v1/follow_requests`
-- `/api/v1/follow_requests/:account_id/authorize`
-- `/api/v1/follow_requests/:account_id/reject`
- `/api/v1/follow_suggestions`
- `/api/v1/domain_blocks` (`GET`, `POST`, `DELETE`)
- `/api/v2/filters` (`GET`, `POST`)
@@ -343,11 +217,7 @@ Endpoints left:
- `/api/v1/statuses/:id/translate`
- `/api/v1/statuses/:id/bookmark`
- `/api/v1/statuses/:id/unbookmark`
-- `/api/v1/statuses/:id/mute`
-- `/api/v1/statuses/:id/unmute`
-- `/api/v1/statuses/:id` (`PUT`)
- `/api/v1/statuses/:id/history`
-- `/api/v1/statuses/:id/source`
- `/api/v1/polls/:id`
- `/api/v1/polls/:id/votes`
- `/api/v1/scheduled_statuses`
@@ -379,7 +249,7 @@ Endpoints left:
- `/api/v1/announcements/:id/reactions/:name` (`PUT`, `DELETE`)
- Admin API
-WebSocket Streaming API also needed to be added (and push notifications)
+WebSocket Streaming API also needs to be added (and push notifications)
## License
diff --git a/docs/cli.md b/docs/cli.md
new file mode 100644
index 00000000..7675c15e
--- /dev/null
+++ b/docs/cli.md
@@ -0,0 +1,24 @@
+# Lysand CLI
+
+Lysand includes a built-in, scripting-compatible CLI that can be used to manage the server. This CLI can be used to create and delete users, manage the database and more. It can also output data in JSON or CSV format, making it easy to use in scripts.
+
+## Using the CLI
+
+Lysand includes a built-in CLI for managing the server. To use it, simply run the following command:
+
+```bash
+# Development
+bun cli help
+# Source installs
+bun run dist/cli.js help
+# Docker
+docker compose exec -it lysand /bin/sh /app/entrypoint.sh cli help
+```
+
+You can use the `help` command to see a list of available commands. These include creating users, deleting users and more. Each command also has a `--help,-h` flag that you can use to see more information about the command.
+
+## Scripting with the CLI
+
+Some CLI commands that return data as tables can be used in scripts. To convert them to JSON or CSV, some commands allow you to specify a `--format` flag that can be either `"json"` or `"csv"`. See `bun cli help` or `bun cli -h` for more information.
+
+Flags can be used in any order and anywhere in the script (except for the `bun cli` command itself). The command arguments themselves must be in the correct order, however.Z
\ No newline at end of file
diff --git a/docs/database.md b/docs/database.md
new file mode 100644
index 00000000..3de46f59
--- /dev/null
+++ b/docs/database.md
@@ -0,0 +1,10 @@
+# Installing the database
+
+Lysand uses a special PostgreSQL extension called `pg_uuidv7` to generate UUIDs. This extension is required for Lysand to work properly. To install it, you can either use the pre-made Docker image or install it manually.
+
+## Using the Docker image
+
+Lysand offers a pre-made Docker image for PostgreSQL with the extension already installed. Use `ghcr.io/lysand-org/postgres:main` as your Docker image name to use it.
+
+## Manual installation
+
diff --git a/docs/installation.md b/docs/installation.md
new file mode 100644
index 00000000..3b257197
--- /dev/null
+++ b/docs/installation.md
@@ -0,0 +1,84 @@
+# Installation
+
+### Requirements
+
+- The [Bun Runtime](https://bun.sh), version 1.0.30 or later (usage of the latest version is recommended)
+- A PostgreSQL database
+- (Optional but recommended) A Linux-based operating system
+- (Optional if you want search) A working Meiliseach instance
+
+> [!WARNING]
+> Lysand has not been tested on Windows or MacOS. It is recommended to use a Linux-based operating system to run Lysand.
+>
+> We will not be offerring support to Windows or MacOS users. If you are using one of these operating systems, please use a virtual machine or container to run Lysand.
+
+### With Docker
+
+Docker is the recommended way to run Lysand (podman should also work). To run Lysand with Docker, follow these steps:
+
+1. Download the `docker-compose.yml` file from the repository
+
+```bash
+curl -o docker-compose.yml https://raw.githubusercontent.com/lysand-org/lysand/main/docker-compose.yml
+```
+2. Edit the `docker-compose.yml` file to set up the database connection and other settings
+3. Download the `config.toml.example` file from the repository
+
+```bash
+curl -o config.toml.example https://raw.githubusercontent.com/lysand-org/lysand/main/config.toml.example
+```
+4. Edit the `config.toml.example` file to set up the database connection and other settings, then place it inside `config/` (create the `config/` directory if it does not exist)
+5. Run the following command to start the server:
+
+```bash
+docker-compose up
+```
+
+You may need root privileges to run Docker commands.
+
+### From Source
+
+1. Clone this repository
+
+```bash
+git clone https://github.com/lysand-org/lysand.git
+```
+
+2. Install the dependencies
+
+```bash
+bun install
+```
+
+1. Set up a PostgreSQL database (you need a special extension, please look at [the database documentation](database.md))
+
+2. (If you want search)
+Create a Meilisearch instance (using Docker is recommended). For a [`docker-compose`] file, copy the `meilisearch` service from the [`docker-compose.yml`](docker-compose.yml) file.
+
+Set up Meiliseach's API key by passing the `MEILI_MASTER_KEY` environment variable to the server. Then, enable and configure search in the config file.
+
+3. Build everything:
+
+```bash
+bun prod-build
+```
+
+4. Copy the `config.toml.example` file to `config.toml` inside `dist/config/` and fill in the values (you can leave most things to the default, but you will need to configure things such as the database connection)
+
+You may now start the server with `bun start`. It lives in the `dist/` directory, all the other code can be removed from this point onwards.
+
+### Running the Server
+
+Database migrations are run automatically on startup.
+
+You may use the environment variables `NO_COLORS=true` and `NO_FANCY_DATES=true` to disable colors and date formatting in the console logs: the file logs will never have colors or fancy dates.
+
+Please see the [CLI documentation](cli.md) for more information on how to use the CLI.
+
+### Updating the server
+
+Updating the server is as simple as pulling the latest changes from the repository and running `bun prod-build` again. You may need to run `bun install` again if there are new dependencies.
+
+For Docker, you can run `docker-compose pull` to update the Docker images.
+
+Sometimes, new configuration options are added to `config.example.toml`. If you see a new option in the example file, you should add it to your `config.toml` file.
\ No newline at end of file
diff --git a/packages/server-handler/index.ts b/packages/server-handler/index.ts
index 5f37f6a5..92ef062e 100644
--- a/packages/server-handler/index.ts
+++ b/packages/server-handler/index.ts
@@ -4,7 +4,7 @@ import type { MatchedRoute } from "bun";
import { type Config, config } from "config-manager";
import { LogLevel, type LogManager, type MultiLogManager } from "log-manager";
import { RequestParser } from "request-parser";
-import { type ZodType, z } from "zod";
+import type { ZodType, z } from "zod";
import { fromZodError } from "zod-validation-error";
import {
type AuthData,
@@ -59,18 +59,6 @@ export interface APIRouteExports {
default: RouteHandler;
}
-const exampleZodSchema = z.object({
- allowedMethods: z.array(z.string()),
- ratelimits: z.object({
- max: z.number(),
- duration: z.number(),
- }),
- route: z.string(),
- auth: z.object({
- required: z.boolean(),
- }),
-});
-
export const processRoute = async (
matchedRoute: MatchedRoute,
request: Request,