docs: update plugin documentation to reflect new isolation model
Signed-off-by: NotAShelf <raf@notashelf.dev> Change-Id: I0102be6731db2770408e8a0b445064926a6a6964
This commit is contained in:
parent
1928d26cde
commit
c6697e7c6f
1 changed files with 536 additions and 137 deletions
673
docs/plugins.md
673
docs/plugins.md
|
|
@ -4,8 +4,10 @@ Pinakes is very powerful on its own, but with a goal as ambitious as "to be the
|
|||
last media management system you will ever need" I recognize the need for a
|
||||
plugin system; not everything belongs in the core of Pinakes. Thus, Pinakes
|
||||
supports WASM-based plugins for extending media type support, metadata
|
||||
extraction, thumbnail generation, search, event handling, and theming. Plugins
|
||||
run in a sandboxed wasmtime runtime with capability-based security.
|
||||
extraction, thumbnail generation, search, event handling, and theming.
|
||||
|
||||
Plugins run in a sandboxed wasmtime runtime with capability-based security, fuel
|
||||
metering, memory limits, and a circuit breaker for fault isolation.
|
||||
|
||||
## How It Works
|
||||
|
||||
|
|
@ -14,13 +16,38 @@ A plugin is a directory containing:
|
|||
```plaintext
|
||||
my-plugin/
|
||||
plugin.toml Manifest declaring name, version, capabilities
|
||||
my_plugin.wasm Compiled WASM binary (wasm32-wasi target)
|
||||
my_plugin.wasm Compiled WASM binary (wasm32-unknown-unknown target)
|
||||
```
|
||||
|
||||
The server discovers plugins in configured directories, validates their
|
||||
manifests, checks capabilities against the security policy, compiles the WASM
|
||||
module, and registers the plugin. Plugins are then invoked through exported
|
||||
functions when the system needs their functionality.
|
||||
module, and registers the plugin.
|
||||
|
||||
Extension points communicate via JSON-over-WASM. The host writes a JSON request
|
||||
into the plugin's memory, calls the exported function, and reads the JSON
|
||||
response written via `host_set_result`. This keeps the interface relatively
|
||||
simple and even language-agnostic. Technically, you could write a plugin in
|
||||
_any_ language that compiles to `wasm32-unknown-unknown`. While the JSON model
|
||||
introduces a little bit of overhead, it's the more debuggable approach and thus
|
||||
more suitable for the initial plugin system. In the future, this might change.
|
||||
|
||||
Plugins go through a priority-ordered pipeline. Each plugin declares a priority
|
||||
(0–999, default 500). Built-in handlers run at implicit priority 100, so plugins
|
||||
at priority <100 run _before_ built-ins and plugins at >100 run _after_.
|
||||
Different extension points use different merge strategies:
|
||||
|
||||
<!--markdownlint-disable MD013-->
|
||||
|
||||
| Extension Point | Strategy | What happens |
|
||||
| --------------- | ------------------ | -------------------------------------------------------------- |
|
||||
| Media type | First match wins | First plugin whose `can_handle` returns `true` claims the file |
|
||||
| Metadata | Accumulating merge | All matching plugins run; later fields overwrite earlier ones |
|
||||
| Thumbnails | First success wins | First plugin to succeed produces the thumbnail |
|
||||
| Search | Merge results | Results from all backends are combined and ranked |
|
||||
| Themes | Accumulate | All themes from all providers are available |
|
||||
| Events | Fan-out | All interested plugins receive the event |
|
||||
|
||||
<!--markdownlint-enable MD013-->
|
||||
|
||||
## Plugin Kinds
|
||||
|
||||
|
|
@ -44,222 +71,591 @@ name = "heif-support"
|
|||
version = "1.0.0"
|
||||
api_version = "1.0"
|
||||
kind = ["media_type", "metadata_extractor", "thumbnail_generator"]
|
||||
priority = 50
|
||||
```
|
||||
|
||||
## Writing a Plugin
|
||||
|
||||
### 1. Create a Rust library
|
||||
[dlmalloc]: https://crates.io/crates/dlmalloc
|
||||
|
||||
```bash
|
||||
cargo new --lib my-plugin
|
||||
```
|
||||
|
||||
In `Cargo.toml`:
|
||||
You are no doubt wondering how to write a plugin, but fret not. It is quite
|
||||
simple: plugins target `wasm32-unknown-unknown` with `#![no_std]`. This is a
|
||||
deliberate choice, `no_std` keeps binaries small and avoids libc dependencies
|
||||
that would complicate the WASM target. Use [dlmalloc] for a global allocator and
|
||||
provide a panic handler.
|
||||
|
||||
```toml
|
||||
# Cargo.toml
|
||||
[lib]
|
||||
crate-type = ["cdylib"]
|
||||
|
||||
[dependencies]
|
||||
pinakes-plugin-api = { path = "../path/to/pinakes/crates/pinakes-plugin-api" }
|
||||
serde = { version = "1", features = ["derive"] }
|
||||
serde_json = "1"
|
||||
dlmalloc = { version = "0.2", features = ["global"] }
|
||||
|
||||
[profile.release]
|
||||
opt-level = "s"
|
||||
lto = true
|
||||
```
|
||||
|
||||
### 2. Implement the Plugin trait
|
||||
Let's go over a minimal example that registers a custom `.mytype` file format:
|
||||
|
||||
```rust
|
||||
use pinakes_plugin_api::{Plugin, PluginContext, PluginMetadata};
|
||||
#![no_std]
|
||||
|
||||
pub struct MyPlugin;
|
||||
extern crate alloc;
|
||||
|
||||
impl Plugin for MyPlugin {
|
||||
fn metadata(&self) -> PluginMetadata {
|
||||
PluginMetadata {
|
||||
name: "my-plugin".into(),
|
||||
version: "1.0.0".into(),
|
||||
description: "My custom plugin".into(),
|
||||
author: "Your Name".into(),
|
||||
}
|
||||
use core::alloc::Layout;
|
||||
|
||||
#[global_allocator]
|
||||
static ALLOC: dlmalloc::GlobalDlmalloc = dlmalloc::GlobalDlmalloc;
|
||||
|
||||
#[panic_handler]
|
||||
fn panic_handler(_: &core::panic::PanicInfo) -> ! {
|
||||
core::arch::wasm32::unreachable()
|
||||
}
|
||||
|
||||
unsafe extern "C" {
|
||||
fn host_set_result(ptr: i32, len: i32);
|
||||
fn host_log(level: i32, ptr: i32, len: i32);
|
||||
}
|
||||
|
||||
fn set_response(json: &[u8]) {
|
||||
unsafe { host_set_result(json.as_ptr() as i32, json.len() as i32); }
|
||||
}
|
||||
|
||||
#[unsafe(no_mangle)]
|
||||
pub extern "C" fn alloc(size: i32) -> i32 {
|
||||
if size <= 0 { return 0; }
|
||||
unsafe {
|
||||
let layout = Layout::from_size_align(size as usize, 1).unwrap();
|
||||
let ptr = alloc::alloc::alloc(layout);
|
||||
if ptr.is_null() { -1 } else { ptr as i32 }
|
||||
}
|
||||
}
|
||||
|
||||
fn initialize(&mut self, _ctx: &PluginContext) -> Result<(), String> {
|
||||
Ok(())
|
||||
}
|
||||
#[unsafe(no_mangle)]
|
||||
pub extern "C" fn initialize() -> i32 { 0 }
|
||||
|
||||
fn shutdown(&mut self) -> Result<(), String> {
|
||||
Ok(())
|
||||
#[unsafe(no_mangle)]
|
||||
pub extern "C" fn shutdown() -> i32 { 0 }
|
||||
|
||||
#[unsafe(no_mangle)]
|
||||
pub extern "C" fn supported_media_types(_ptr: i32, _len: i32) {
|
||||
set_response(br#"[{"id":"mytype","name":"My Type","category":"document","extensions":["mytype"],"mime_types":["application/x-mytype"]}]"#);
|
||||
}
|
||||
|
||||
#[unsafe(no_mangle)]
|
||||
pub extern "C" fn can_handle(ptr: i32, len: i32) {
|
||||
let req = unsafe { core::slice::from_raw_parts(ptr as *const u8, len as usize) };
|
||||
let json = core::str::from_utf8(req).unwrap_or("");
|
||||
if json.contains(".mytype") {
|
||||
set_response(br#"{"can_handle":true}"#);
|
||||
} else {
|
||||
set_response(br#"{"can_handle":false}"#);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Then implement whichever additional traits your plugin needs
|
||||
(`MediaTypeProvider`, `MetadataExtractor`, `ThumbnailGenerator`, etc.).
|
||||
|
||||
### 3. Build to WASM
|
||||
### Building
|
||||
|
||||
```bash
|
||||
rustup target add wasm32-wasi
|
||||
cargo build --target wasm32-wasi --release
|
||||
|
||||
# Optional: strip debug info to reduce binary size
|
||||
wasm-tools strip target/wasm32-wasi/release/my_plugin.wasm -o target/wasm32-wasi/release/my_plugin.wasm
|
||||
$ cargo build --target wasm32-unknown-unknown --release
|
||||
```
|
||||
|
||||
### 4. Create the manifest
|
||||
The `RUSTFLAGS=""` override may be needed if your environment sets linker flags
|
||||
(e.g., `-fuse-ld=lld`) that are incompatible with the WASM target. You can also
|
||||
specify a compatible linker explicitly. As pinakes uses a `clang` and `lld`
|
||||
based pipeline, it's necessary to set for, e.g., the test fixtures.
|
||||
|
||||
```toml
|
||||
[plugin]
|
||||
name = "my-plugin"
|
||||
version = "1.0.0"
|
||||
api_version = "1.0"
|
||||
description = "A custom plugin"
|
||||
author = "Your Name"
|
||||
kind = ["metadata_extractor"]
|
||||
The compiled binary will be at
|
||||
`target/wasm32-unknown-unknown/release/my_plugin.wasm`.
|
||||
|
||||
[plugin.binary]
|
||||
wasm = "my_plugin.wasm"
|
||||
### A note on `serde` in `no_std`
|
||||
|
||||
[capabilities]
|
||||
network = false
|
||||
Since `serde_json` requires `std`, you cannot use it in a plugin. Either
|
||||
hand-write JSON strings (as shown above) or use a lightweight no_std JSON
|
||||
library.
|
||||
|
||||
[capabilities.filesystem]
|
||||
read = ["/media"]
|
||||
write = []
|
||||
The test fixture plugin in `crates/pinakes-core/tests/fixtures/test-plugin/`
|
||||
demonstrates the hand-written approach. It is ugly, but it works, and the
|
||||
binaries are tiny (~17KB).
|
||||
|
||||
[capabilities.resources]
|
||||
max_memory_bytes = 67108864
|
||||
max_cpu_time_ms = 5000
|
||||
```
|
||||
|
||||
### 5. Install
|
||||
### Installing
|
||||
|
||||
Place the plugin directory in one of the configured plugin directories, or use
|
||||
the API:
|
||||
|
||||
```bash
|
||||
# Install from local path
|
||||
curl -X POST http://localhost:3000/api/v1/plugins/install \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-d '{"source": "/path/to/my-plugin"}'
|
||||
|
||||
# Install from HTTPS URL (downloads and extracts tar.gz)
|
||||
curl -X POST http://localhost:3000/api/v1/plugins/install \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-d '{"source": "https://example.com/my-plugin-1.0.0.tar.gz"}'
|
||||
```
|
||||
|
||||
## Manifest Reference
|
||||
|
||||
Plugins are required to provide a manifest with explicit intents for everything.
|
||||
Here is the expected manifest format as of 0.3.0-dev version of Pinakes:
|
||||
|
||||
```toml
|
||||
[plugin]
|
||||
name = "my-plugin" # Unique identifier (required)
|
||||
version = "1.0.0" # SemVer version (required)
|
||||
api_version = "1.0" # Plugin API version (required)
|
||||
description = "What it does" # Human-readable description
|
||||
author = "Your Name"
|
||||
kind = [ # One or more roles (required)
|
||||
"media_type",
|
||||
"metadata_extractor",
|
||||
"thumbnail_generator",
|
||||
"event_handler",
|
||||
]
|
||||
priority = 500 # 0-999, lower runs first (default: 500)
|
||||
dependencies = ["other-plugin"] # Plugins that must load first (optional)
|
||||
|
||||
[plugin.binary]
|
||||
wasm = "my_plugin.wasm" # Path to WASM binary relative to plugin dir
|
||||
|
||||
[capabilities]
|
||||
network = false # Allow host_http_request calls
|
||||
allowed_domains = ["api.example.com"] # Domain whitelist (enforced at runtime)
|
||||
environment = ["HOME", "LANG"] # Allowed env vars for host_get_env
|
||||
|
||||
[capabilities.filesystem]
|
||||
read = ["/media"] # Allowed read paths for host_read_file
|
||||
write = ["/tmp/plugin-data"] # Allowed write paths for host_write_file
|
||||
|
||||
[capabilities.resources]
|
||||
max_memory_mb = 64 # Maximum linear memory (default: 512MB)
|
||||
max_cpu_time_secs = 5 # Fuel budget per invocation (default: 60s)
|
||||
```
|
||||
|
||||
## Extension Points
|
||||
|
||||
Every plugin must export these three functions:
|
||||
|
||||
| Export | Signature | Purpose |
|
||||
| ------------ | -------------------- | ---------------------------------- |
|
||||
| `alloc` | `(size: i32) -> i32` | Allocate memory for host to write |
|
||||
| `initialize` | `() -> i32` | Called once on load (0 = success) |
|
||||
| `shutdown` | `() -> i32` | Called before unload (0 = success) |
|
||||
|
||||
Beyond those, which functions you export depends on your plugin's kind(s). The
|
||||
capability enforcer prevents a plugin from being called for functions it hasn't
|
||||
declared; a `metadata_extractor` plugin cannot have `search` called on it.
|
||||
|
||||
### `MediaTypeProvider`
|
||||
|
||||
Plugins with `kind` containing `"media_type"` export:
|
||||
|
||||
**`supported_media_types(ptr, len)`**
|
||||
|
||||
- Request: ignored.
|
||||
- Response:
|
||||
|
||||
```json
|
||||
[{
|
||||
"id": "heif",
|
||||
"name": "HEIF Image",
|
||||
"category": "image",
|
||||
"extensions": ["heif", "heic"],
|
||||
"mime_types": ["image/heif", "image/heic"]
|
||||
}]
|
||||
```
|
||||
|
||||
**`can_handle(ptr, len)`**
|
||||
|
||||
- Request: `{"path": "/media/photo.heif"}`.
|
||||
- Response: `{"can_handle": true}`.
|
||||
|
||||
In the pipeline first-match-wins. Plugins are checked in priority order. The
|
||||
first where `can_handle` returns `true` and has a matching definition claims the
|
||||
file. If no plugin claims it, built-in handlers run as fallback.
|
||||
|
||||
### `MetadataExtractor`
|
||||
|
||||
Plugins with `kind` containing `"metadata_extractor"` export:
|
||||
|
||||
**`supported_types(ptr, len)`**
|
||||
|
||||
- Response: `["heif", "heic"]`
|
||||
|
||||
**`extract_metadata(ptr, len)`**
|
||||
|
||||
- Request: `{"path": "/media/photo.heif"}`.
|
||||
- Response:
|
||||
|
||||
```json
|
||||
{
|
||||
"title": "Sunset",
|
||||
"artist": "Photographer",
|
||||
"album": null,
|
||||
"description": "A sunset photo",
|
||||
"extra": { "camera": "Canon R5" }
|
||||
}
|
||||
```
|
||||
|
||||
All fields are optional. The `extra` map accepts arbitrary key-value pairs.
|
||||
|
||||
The pipeline features accumulating merge. All matching plugins run in priority
|
||||
order. Later plugins' non-null fields overwrite earlier ones. The `extra` maps
|
||||
are merged (later keys win).
|
||||
|
||||
### `ThumbnailGenerator`
|
||||
|
||||
Plugins with `kind` containing `"thumbnail_generator"` export:
|
||||
|
||||
**`supported_types(ptr, len)`**
|
||||
|
||||
Same as [MetadataExtractor](#metadataextractor)
|
||||
|
||||
**`generate_thumbnail(ptr, len)`**
|
||||
|
||||
- Request:
|
||||
|
||||
```json
|
||||
{
|
||||
"source_path": "/media/photo.heif",
|
||||
"output_path": "/cache/thumbs/abc.jpg",
|
||||
"max_width": 320,
|
||||
"max_height": 320,
|
||||
"format": "jpeg"
|
||||
}
|
||||
```
|
||||
|
||||
- Response:
|
||||
`{"path": "/cache/thumbs/abc.jpg", "width": 320, "height": 320, "format": "jpeg"}`
|
||||
|
||||
First-success-wins. The first plugin to return a successful result produces the
|
||||
thumbnail.
|
||||
|
||||
### `SearchBackend`
|
||||
|
||||
Plugins with `kind` containing `"search_backend"` export:
|
||||
|
||||
**`supported_types(ptr, len)`**
|
||||
|
||||
What media types this backend indexes.
|
||||
|
||||
**`search(ptr, len)`**
|
||||
|
||||
- Request: `{"query": "beethoven", "limit": 20, "offset": 0}`.
|
||||
- Response:
|
||||
|
||||
```json
|
||||
{
|
||||
"results": [
|
||||
{
|
||||
"id": "abc-123",
|
||||
"score": 0.95,
|
||||
"snippet": "Beethoven - Symphony No. 9"
|
||||
}
|
||||
],
|
||||
"total_count": 42
|
||||
}
|
||||
```
|
||||
|
||||
**`index_item(ptr, len)`**
|
||||
|
||||
Request contains `id`, `title`, `artist`, `album`, `description`, `tags`,
|
||||
`media_type`, `path`.
|
||||
|
||||
- Response: `{}`.
|
||||
|
||||
**`remove_item(ptr, len)`**
|
||||
|
||||
- Request: `{"id": "abc-123"}`.
|
||||
- Response: `{}`.
|
||||
|
||||
Results from all backends are merged, deduplicated by ID (keeping the highest
|
||||
score), and sorted by score descending. `index_item` and `remove_item` are
|
||||
fanned out to all backends.
|
||||
|
||||
### `EventHandler`
|
||||
|
||||
Plugins with `kind` containing `"event_handler"` export:
|
||||
|
||||
**`interested_events(ptr, len)`**
|
||||
|
||||
- Response: `["MediaImported", "MediaUpdated", "MediaDeleted"]`
|
||||
|
||||
**`handle_event(ptr, len)`**
|
||||
|
||||
- Request:
|
||||
|
||||
```json
|
||||
{
|
||||
"event_type": "MediaImported",
|
||||
"payload": { "media_id": "test-123", "path": "/media/song.mp3" }
|
||||
}
|
||||
```
|
||||
|
||||
- Response: `{}`.
|
||||
|
||||
All interested plugins receive the event. Events are dispatched asynchronously
|
||||
via `tokio::spawn` and do not block the caller. Handler failures are logged but
|
||||
never propagated. Suffice to say that the pipeline is fan-out.
|
||||
|
||||
### `ThemeProvider`
|
||||
|
||||
> [!IMPORTANT]
|
||||
> `ThemeProvider` is experimental, and will likely be subject to change.
|
||||
|
||||
Plugins with `kind` containing `"theme_provider"` export:
|
||||
|
||||
**`get_themes(ptr, len)`**
|
||||
|
||||
- Response:
|
||||
|
||||
```json
|
||||
[{
|
||||
"id": "dark-ocean",
|
||||
"name": "Dark Ocean",
|
||||
"description": "A deep blue theme",
|
||||
"dark": true
|
||||
}]
|
||||
```
|
||||
|
||||
**`load_theme(ptr, len)`**
|
||||
|
||||
- Request: `{"theme_id": "dark-ocean"}`.
|
||||
- Response:
|
||||
|
||||
```json
|
||||
{
|
||||
"css": ":root { --bg: #0a1628; }",
|
||||
"colors": { "background": "#0a1628", "text": "#e0e0e0" }
|
||||
}
|
||||
```
|
||||
|
||||
Themes from all providers are accumulated. When loading a specific theme, the
|
||||
pipeline dispatches to the plugin that registered it.
|
||||
|
||||
### System Events
|
||||
|
||||
The server emits these events at various points:
|
||||
|
||||
| Event | Payload fields | Emitted when |
|
||||
| ------------------- | ----------------------- | -------------------------- |
|
||||
| `ScanStarted` | `roots` | Directory scan begins |
|
||||
| `ScanCompleted` | `roots` | Directory scan finishes |
|
||||
| `MediaImported` | `media_id`, `path` | File imported via scan/API |
|
||||
| `MediaUpdated` | `media_id` | Media metadata updated |
|
||||
| `MediaDeleted` | `media_id` | Media permanently deleted |
|
||||
| `MediaTagged` | `media_id`, `tag` | Tag applied to media |
|
||||
| `MediaUntagged` | `media_id`, `tag` | Tag removed from media |
|
||||
| `CollectionCreated` | `collection_id`, `name` | New collection created |
|
||||
| `CollectionDeleted` | `collection_id` | Collection deleted |
|
||||
|
||||
Plugins can also emit events themselves via `host_emit_event`, enabling
|
||||
plugin-to-plugin communication.
|
||||
|
||||
## Host Functions
|
||||
|
||||
Plugins can call these host functions from within WASM:
|
||||
Plugins can call these host functions from within WASM (imported from the
|
||||
`"env"` module):
|
||||
|
||||
### `host_set_result(ptr, len)`
|
||||
|
||||
Write a JSON response back to the host. The plugin writes the response bytes
|
||||
into its own linear memory and passes the pointer and length. This is how you
|
||||
return data from any extension point function.
|
||||
|
||||
### `host_emit_event(type_ptr, type_len, payload_ptr, payload_len) -> i32`
|
||||
|
||||
Emit a system event from within a plugin. Enables plugin-to-plugin
|
||||
communication.
|
||||
|
||||
Returns `0` on success, `-1` on error.
|
||||
|
||||
### `host_log(level, ptr, len)`
|
||||
|
||||
Log a message. Levels: 0=error, 1=warn, 2=info, 3+=debug.
|
||||
Log a message.
|
||||
|
||||
Levels: 0=error, 1=warn, 2=info, 3+=debug. Messages appear in the server's
|
||||
tracing output.
|
||||
|
||||
### `host_read_file(path_ptr, path_len) -> i32`
|
||||
|
||||
Read a file into the exchange buffer. Returns file size on success, `-1` on IO
|
||||
error, `-2` if the path is outside the plugin's allowed read paths.
|
||||
error, `-2` if the path is outside the plugin's allowed read paths (paths are
|
||||
canonicalized before checking, so traversal tricks won't work).
|
||||
|
||||
### `host_write_file(path_ptr, path_len, data_ptr, data_len) -> i32`
|
||||
|
||||
Write data to a file. Returns `0` on success, `-1` on IO error, `-2` if the path
|
||||
is outside the plugin's allowed write paths.
|
||||
Write data to a file.
|
||||
|
||||
Returns `0` on success, `-1` on IO error, `-2` if the path is outside allowed
|
||||
write paths.
|
||||
|
||||
### `host_http_request(url_ptr, url_len) -> i32`
|
||||
|
||||
Make an HTTP GET request. Returns response size on success (body in exchange
|
||||
buffer), `-1` on error, `-2` if network access is disabled.
|
||||
Make an HTTP GET request.
|
||||
|
||||
Returns response size on success (body in exchange buffer), `-1` on error, `-2`
|
||||
if network access is disabled, `-3` if the domain is not in the plugin's
|
||||
`allowed_domains` list.
|
||||
|
||||
### `host_get_config(key_ptr, key_len) -> i32`
|
||||
|
||||
Read a plugin configuration value. Returns JSON value length on success (in
|
||||
exchange buffer), `-1` if key not found.
|
||||
|
||||
### `host_get_env(key_ptr, key_len) -> i32`
|
||||
|
||||
Read an environment variable.
|
||||
|
||||
Returns value length on success (in exchange buffer), `-1` if the variable is
|
||||
not set, `-2` if environment access is disabled or the variable is not in the
|
||||
plugin's `environment` list.
|
||||
|
||||
### `host_get_buffer(dest_ptr, dest_len) -> i32`
|
||||
|
||||
Copy the exchange buffer into WASM memory. Returns bytes copied. Use this after
|
||||
`host_read_file`, `host_http_request`, or `host_get_config` to retrieve data.
|
||||
Copy the exchange buffer into WASM memory.
|
||||
|
||||
Returns bytes copied. Use this after `host_read_file`, `host_http_request`,
|
||||
`host_get_config`, or `host_get_env` to retrieve data the host placed in the
|
||||
buffer.
|
||||
|
||||
## Security Model
|
||||
|
||||
### Capabilities
|
||||
|
||||
Every plugin declares what it needs in `plugin.toml`. At load time, the
|
||||
`CapabilityEnforcer` validates these against the system policy:
|
||||
Every plugin declares what it needs in `plugin.toml`. The `CapabilityEnforcer`
|
||||
validates these at load time _and_ at runtime; a plugin declaring
|
||||
`kind = ["metadata_extractor"]` cannot have `search` called on it, and a plugin
|
||||
without `network = true` will get `-2` from `host_http_request`.
|
||||
|
||||
- **Filesystem**: Read and write paths must be within allowed directories.
|
||||
Requests for paths outside the allowlist are rejected at load time (the plugin
|
||||
won't load) and at runtime (the host function returns `-2`).
|
||||
- **Filesystem**: Read and write paths must be within allowed directories. Paths
|
||||
are canonicalized before checking to prevent traversal attacks. Requests
|
||||
outside the allowlist are rejected at load time (the plugin won't load) and at
|
||||
runtime (the host function returns `-2`).
|
||||
|
||||
- **Network**: Must be explicitly enabled. Even when enabled, each
|
||||
`host_http_request` call checks the flag. Domain whitelisting is supported in
|
||||
the capability model but not yet enforced at the host function level.
|
||||
- **Network**: Must be explicitly enabled. When `allowed_domains` is configured,
|
||||
each `host_http_request` call also validates the URL's domain against the
|
||||
whitelist (returning `-3` on violation).
|
||||
|
||||
- **Resources**: Maximum memory (default 512MB) and CPU time (default 60s) are
|
||||
enforced via wasmtime's fuel metering system. Each function invocation gets a
|
||||
fresh `Store` with a fuel budget proportional to the configured CPU time
|
||||
limit.
|
||||
- **Environment**: Must be explicitly enabled. When `environment` is configured,
|
||||
only those specific variables can be read.
|
||||
|
||||
- **Memory**: Each function invocation creates a fresh wasmtime `Store` with a
|
||||
memory limit derived from the plugin's `max_memory_mb` capability (default:
|
||||
512MB). Attempts to grow linear memory beyond this limit cause a trap.
|
||||
|
||||
- **CPU**: Enforced via wasmtime's fuel metering. Each invocation gets a fuel
|
||||
budget proportional to the configured CPU time limit.
|
||||
|
||||
### Isolation
|
||||
|
||||
- Each `call_function` invocation creates a fresh wasmtime `Store`, so plugins
|
||||
cannot retain state between calls via the WASM runtime (they can use the
|
||||
filesystem or exchange buffer for persistence).
|
||||
- Each `call_function` invocation creates a fresh wasmtime `Store`. Plugins
|
||||
cannot retain WASM runtime state between calls. If you need persistence, use
|
||||
the filesystem host functions.
|
||||
|
||||
- WASM's linear memory model prevents plugins from accessing host memory.
|
||||
|
||||
- The wasmtime stack is limited to 1MB.
|
||||
|
||||
### Unsigned Plugins
|
||||
### Timeouts
|
||||
|
||||
The `allow_unsigned` configuration option (default `true` in development)
|
||||
permits loading plugins without signature verification. For production
|
||||
deployments, set this to `false` and sign your plugins. Note: signature
|
||||
verification is defined in the configuration but not yet implemented in the
|
||||
loader. This is a known gap.
|
||||
Three tiers of per-call timeouts prevent runaway plugins:
|
||||
|
||||
## Plugin API Endpoints
|
||||
<!--markdownlint-disable MD013-->
|
||||
|
||||
All plugin endpoints require Admin role.
|
||||
| Tier | Default | Used for |
|
||||
| ------------------ | ------- | --------------------------------------------------------------------------------------------- |
|
||||
| Capability queries | 2s | `supported_types`, `interested_events`, `can_handle`, `get_themes`, `supported_media_types` |
|
||||
| Processing | 30s | `extract_metadata`, `generate_thumbnail`, `search`, `index_item`, `remove_item`, `load_theme` |
|
||||
| Event handlers | 10s | `handle_event` |
|
||||
|
||||
| Method | Path | Description |
|
||||
| -------- | ----------------------------- | ------------------------ |
|
||||
| `GET` | `/api/v1/plugins` | List all plugins |
|
||||
| `GET` | `/api/v1/plugins/:id` | Get plugin details |
|
||||
| `POST` | `/api/v1/plugins/install` | Install from path or URL |
|
||||
| `DELETE` | `/api/v1/plugins/:id` | Uninstall a plugin |
|
||||
| `POST` | `/api/v1/plugins/:id/enable` | Enable a disabled plugin |
|
||||
| `POST` | `/api/v1/plugins/:id/disable` | Disable a plugin |
|
||||
| `POST` | `/api/v1/plugins/:id/reload` | Hot-reload a plugin |
|
||||
<!--markdownlint-enable MD013-->
|
||||
|
||||
### Circuit Breaker
|
||||
|
||||
If a plugin fails consecutively, the circuit breaker disables it automatically:
|
||||
|
||||
- After **5 consecutive failures** (configurable, in the config), the plugin is
|
||||
disabled.
|
||||
- A success resets the failure counter.
|
||||
- Disabled plugins are skipped in all pipeline stages.
|
||||
- Reload or toggle the plugin via the API to re-enable.
|
||||
|
||||
### Signatures
|
||||
|
||||
Plugins can be signed with Ed25519. The signing flow:
|
||||
|
||||
1. Compute the BLAKE3 hash of the WASM binary.
|
||||
2. Sign the 32-byte hash with your Ed25519 private key.
|
||||
3. Write the raw 64-byte signature to `plugin.sig` alongside `plugin.toml`.
|
||||
|
||||
On the server side, configure trusted public keys (hex-encoded, 64 hex chars
|
||||
each):
|
||||
|
||||
```toml
|
||||
[plugins]
|
||||
allow_unsigned = false
|
||||
trusted_keys = [
|
||||
"a1b2c3d4...64 hex chars...",
|
||||
]
|
||||
```
|
||||
|
||||
When `allow_unsigned` is `false`, every plugin must carry a `plugin.sig` that
|
||||
verifies against at least one trusted key. If the signature is missing, invalid,
|
||||
or matches no trusted key, the plugin is rejected at load time. Set
|
||||
`allow_unsigned = true` during development to skip this check.
|
||||
|
||||
## Plugin Lifecycle
|
||||
|
||||
1. **Discovery**: On startup, the plugin manager walks configured plugin
|
||||
directories looking for `plugin.toml` files.
|
||||
|
||||
2. **Validation**: The manifest is parsed, and capabilities are checked against
|
||||
2. **Dependency resolution**: Discovered manifests are topologically sorted by
|
||||
their `dependencies` fields. If plugin A depends on plugin B, B is loaded
|
||||
first. Cycles and missing dependencies are detected. Affected plugins are
|
||||
skipped with a warning rather than crashing the server.
|
||||
|
||||
3. **Signature verification**: If `allow_unsigned` is `false`, the loader checks
|
||||
for a `plugin.sig` file and verifies it against the configured trusted keys.
|
||||
Unsigned or invalid-signature plugins are rejected.
|
||||
|
||||
4. **Validation**: The manifest is parsed, and capabilities are checked against
|
||||
the security policy. If validation fails, the plugin is skipped with a
|
||||
warning.
|
||||
|
||||
3. **Loading**: The WASM binary is compiled via wasmtime. The `initialize`
|
||||
export is called if present.
|
||||
5. **Loading**: The WASM binary is compiled via wasmtime. The `initialize`
|
||||
export is called.
|
||||
|
||||
4. **Registration**: The plugin is added to the in-memory registry and the
|
||||
database.
|
||||
6. **Capability discovery**: The pipeline queries each plugin for its supported
|
||||
types, media type definitions, theme definitions, and interested events.
|
||||
Results are cached for fast dispatch.
|
||||
|
||||
5. **Invocation**: When the system needs plugin functionality (e.g., extracting
|
||||
metadata from an unknown file type), it calls the appropriate exported
|
||||
function via `WasmPlugin::call_function()`.
|
||||
7. **Invocation**: When the system needs plugin functionality, the pipeline
|
||||
dispatches to plugins in priority order using the appropriate merge strategy.
|
||||
|
||||
6. **Shutdown**: On server shutdown or plugin disable, the `shutdown` export is
|
||||
called if present.
|
||||
8. **Shutdown**: On server shutdown or plugin disable, the `shutdown` export is
|
||||
called.
|
||||
|
||||
7. **Hot-reload**: The `/plugins/:id/reload` endpoint reloads the WASM binary
|
||||
from disk without restarting the server.
|
||||
9. **Hot-reload**: The `/plugins/:id/reload` endpoint reloads the WASM binary
|
||||
from disk and re-discovers capabilities without restarting the server.
|
||||
|
||||
## Plugin API Endpoints
|
||||
|
||||
<!-- FIXME: this should be moved to API documentation -->
|
||||
|
||||
All plugin endpoints require the Admin role.
|
||||
|
||||
| Method | Path | Description |
|
||||
| -------- | ----------------------------- | ------------------------------- |
|
||||
| `GET` | `/api/v1/plugins` | List all plugins |
|
||||
| `GET` | `/api/v1/plugins/:id` | Get plugin details |
|
||||
| `POST` | `/api/v1/plugins/install` | Install from path or URL |
|
||||
| `DELETE` | `/api/v1/plugins/:id` | Uninstall a plugin |
|
||||
| `POST` | `/api/v1/plugins/:id/enable` | Enable a disabled plugin |
|
||||
| `POST` | `/api/v1/plugins/:id/disable` | Disable a plugin |
|
||||
| `POST` | `/api/v1/plugins/:id/reload` | Hot-reload and re-discover caps |
|
||||
|
||||
Reload and toggle operations automatically re-discover the plugin's
|
||||
capabilities, so changes to supported types or event subscriptions take effect
|
||||
immediately.
|
||||
|
||||
## Configuration
|
||||
|
||||
|
|
@ -270,6 +666,15 @@ In `pinakes.toml`:
|
|||
enabled = true
|
||||
plugin_dirs = ["/etc/pinakes/plugins", "~/.local/share/pinakes/plugins"]
|
||||
allow_unsigned = false
|
||||
trusted_keys = ["a1b2c3d4..."] # Ed25519 public keys (hex-encoded)
|
||||
max_concurrent_ops = 4
|
||||
plugin_timeout_secs = 30
|
||||
max_consecutive_failures = 5
|
||||
|
||||
[plugins.timeouts]
|
||||
capability_query_secs = 2
|
||||
processing_secs = 30
|
||||
event_handler_secs = 10
|
||||
|
||||
[plugins.security]
|
||||
allow_network_default = false
|
||||
|
|
@ -277,20 +682,14 @@ allowed_read_paths = ["/media", "/tmp/pinakes"]
|
|||
allowed_write_paths = ["/tmp/pinakes/plugin-data"]
|
||||
```
|
||||
|
||||
## Known Limitations
|
||||
## Debugging
|
||||
|
||||
- **Plugin signing**: The `allow_unsigned` flag is parsed but signature
|
||||
verification is not implemented. All plugins load regardless of this setting.
|
||||
|
||||
- **Dependency resolution**: The manifest supports a `dependencies` field, but
|
||||
dependency ordering and conflict detection are not implemented.
|
||||
|
||||
- **Domain whitelisting**: `allowed_domains` is part of the capability model but
|
||||
`host_http_request` does not check it.
|
||||
|
||||
- **Environment variables**: The `EnvironmentCapability` is defined in the API
|
||||
but no corresponding host function exists.
|
||||
|
||||
- **Memory limits**: CPU time is enforced via fuel metering. Linear memory
|
||||
limits depend on the WASM module's own declarations; the host does not
|
||||
currently impose an explicit cap beyond wasmtime's defaults.
|
||||
- **`host_log`**: Call from within your plugin to emit structured log messages.
|
||||
They appear in the server's tracing output.
|
||||
- **Server logs**: Set `RUST_LOG=pinakes_core::plugin=debug` to see plugin
|
||||
dispatch decisions, timeout warnings, and circuit breaker state changes.
|
||||
- **Reload workflow**: Edit your plugin source, rebuild the WASM binary, then
|
||||
`POST /api/v1/plugins/:id/reload`. No server restart needed.
|
||||
- **Circuit breaker**: If your plugin is silently skipped, check the server logs
|
||||
for "circuit breaker tripped" messages. Fix the issue, then re-enable via
|
||||
`POST /api/v1/plugins/:id/enable`.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue