# Plugin System Pinakes is very powerful on its own, but with a goal as ambitious as "to be the last media management system you will ever need" I recognize the need for a plugin system; not everything belongs in the core of Pinakes. Thus, Pinakes supports WASM-based plugins for extending media type support, metadata extraction, thumbnail generation, search, event handling, and theming. Plugins run in a sandboxed wasmtime runtime with capability-based security, fuel metering, memory limits, and a circuit breaker for fault isolation. ## How It Works A plugin is a directory containing: ```plaintext my-plugin/ plugin.toml Manifest declaring name, version, capabilities my_plugin.wasm Compiled WASM binary (wasm32-unknown-unknown target) ``` The server discovers plugins in configured directories, validates their manifests, checks capabilities against the security policy, compiles the WASM module, and registers the plugin. Extension points communicate via JSON-over-WASM. The host writes a JSON request into the plugin's memory, calls the exported function, and reads the JSON response written via `host_set_result`. This keeps the interface relatively simple and even language-agnostic. Technically, you could write a plugin in _any_ language that compiles to `wasm32-unknown-unknown`. While the JSON model introduces a little bit of overhead, it's the more debuggable approach and thus more suitable for the initial plugin system. In the future, this might change. Plugins go through a priority-ordered pipeline. Each plugin declares a priority (0–999, default 500). Built-in handlers run at implicit priority 100, so plugins at priority <100 run _before_ built-ins and plugins at >100 run _after_. Different extension points use different merge strategies: | Extension Point | Strategy | What happens | | --------------- | ------------------ | -------------------------------------------------------------- | | Media type | First match wins | First plugin whose `can_handle` returns `true` claims the file | | Metadata | Accumulating merge | All matching plugins run; later fields overwrite earlier ones | | Thumbnails | First success wins | First plugin to succeed produces the thumbnail | | Search | Merge results | Results from all backends are combined and ranked | | Themes | Accumulate | All themes from all providers are available | | Events | Fan-out | All interested plugins receive the event | ## Plugin Kinds A plugin can implement one or more of these roles: | Kind | Purpose | | --------------------- | -------------------------------------------------- | | `media_type` | Register new file formats (extensions, MIME types) | | `metadata_extractor` | Extract metadata from files of specific types | | `thumbnail_generator` | Generate thumbnails for specific media types | | `search_backend` | Provide alternative search implementations | | `event_handler` | React to system events (imports, deletes, etc.) | | `theme_provider` | Supply UI themes | | `general` | General-purpose (uses host functions freely) | Declare kinds in `plugin.toml`: ```toml [plugin] name = "heif-support" version = "1.0.0" api_version = "1.0" kind = ["media_type", "metadata_extractor", "thumbnail_generator"] priority = 50 ``` ## Writing a Plugin [dlmalloc]: https://crates.io/crates/dlmalloc You are no doubt wondering how to write a plugin, but fret not. It is quite simple: plugins target `wasm32-unknown-unknown` with `#![no_std]`. This is a deliberate choice, `no_std` keeps binaries small and avoids libc dependencies that would complicate the WASM target. Use [dlmalloc] for a global allocator and provide a panic handler. ```toml # Cargo.toml [lib] crate-type = ["cdylib"] [dependencies] dlmalloc = { version = "0.2", features = ["global"] } [profile.release] opt-level = "s" lto = true ``` Let's go over a minimal example that registers a custom `.mytype` file format: ```rust #![no_std] extern crate alloc; use core::alloc::Layout; #[global_allocator] static ALLOC: dlmalloc::GlobalDlmalloc = dlmalloc::GlobalDlmalloc; #[panic_handler] fn panic_handler(_: &core::panic::PanicInfo) -> ! { core::arch::wasm32::unreachable() } unsafe extern "C" { fn host_set_result(ptr: i32, len: i32); fn host_log(level: i32, ptr: i32, len: i32); } fn set_response(json: &[u8]) { unsafe { host_set_result(json.as_ptr() as i32, json.len() as i32); } } #[unsafe(no_mangle)] pub extern "C" fn alloc(size: i32) -> i32 { if size <= 0 { return 0; } unsafe { let layout = Layout::from_size_align(size as usize, 1).unwrap(); let ptr = alloc::alloc::alloc(layout); if ptr.is_null() { -1 } else { ptr as i32 } } } #[unsafe(no_mangle)] pub extern "C" fn initialize() -> i32 { 0 } #[unsafe(no_mangle)] pub extern "C" fn shutdown() -> i32 { 0 } #[unsafe(no_mangle)] pub extern "C" fn supported_media_types(_ptr: i32, _len: i32) { set_response(br#"[{"id":"mytype","name":"My Type","category":"document","extensions":["mytype"],"mime_types":["application/x-mytype"]}]"#); } #[unsafe(no_mangle)] pub extern "C" fn can_handle(ptr: i32, len: i32) { let req = unsafe { core::slice::from_raw_parts(ptr as *const u8, len as usize) }; let json = core::str::from_utf8(req).unwrap_or(""); if json.contains(".mytype") { set_response(br#"{"can_handle":true}"#); } else { set_response(br#"{"can_handle":false}"#); } } ``` ### Building ```bash $ cargo build --target wasm32-unknown-unknown --release ``` The `RUSTFLAGS=""` override may be needed if your environment sets linker flags (e.g., `-fuse-ld=lld`) that are incompatible with the WASM target. You can also specify a compatible linker explicitly. As pinakes uses a `clang` and `lld` based pipeline, it's necessary to set for, e.g., the test fixtures. The compiled binary will be at `target/wasm32-unknown-unknown/release/my_plugin.wasm`. ### A note on `serde` in `no_std` Since `serde_json` requires `std`, you cannot use it in a plugin. Either hand-write JSON strings (as shown above) or use a lightweight no_std JSON library. The test fixture plugin in `crates/pinakes-core/tests/fixtures/test-plugin/` demonstrates the hand-written approach. It is ugly, but it works, and the binaries are tiny (~17KB). ### Installing Place the plugin directory in one of the configured plugin directories, or use the API: ```bash curl -X POST http://localhost:3000/api/v1/plugins/install \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $TOKEN" \ -d '{"source": "/path/to/my-plugin"}' ``` ## Manifest Reference Plugins are required to provide a manifest with explicit intents for everything. Here is the expected manifest format as of 0.3.0-dev version of Pinakes: ```toml [plugin] name = "my-plugin" # Unique identifier (required) version = "1.0.0" # SemVer version (required) api_version = "1.0" # Plugin API version (required) description = "What it does" # Human-readable description author = "Your Name" kind = [ # One or more roles (required) "media_type", "metadata_extractor", "thumbnail_generator", "event_handler", ] priority = 500 # 0-999, lower runs first (default: 500) dependencies = ["other-plugin"] # Plugins that must load first (optional) [plugin.binary] wasm = "my_plugin.wasm" # Path to WASM binary relative to plugin dir [capabilities] network = false # Allow host_http_request calls allowed_domains = ["api.example.com"] # Domain whitelist (enforced at runtime) environment = ["HOME", "LANG"] # Allowed env vars for host_get_env [capabilities.filesystem] read = ["/media"] # Allowed read paths for host_read_file write = ["/tmp/plugin-data"] # Allowed write paths for host_write_file [capabilities.resources] max_memory_mb = 64 # Maximum linear memory (default: 512MB) max_cpu_time_secs = 5 # Fuel budget per invocation (default: 60s) ``` ## Extension Points Every plugin must export these three functions: | Export | Signature | Purpose | | ------------ | -------------------- | ---------------------------------- | | `alloc` | `(size: i32) -> i32` | Allocate memory for host to write | | `initialize` | `() -> i32` | Called once on load (0 = success) | | `shutdown` | `() -> i32` | Called before unload (0 = success) | Beyond those, which functions you export depends on your plugin's kind(s). The capability enforcer prevents a plugin from being called for functions it hasn't declared; a `metadata_extractor` plugin cannot have `search` called on it. ### `MediaTypeProvider` Plugins with `kind` containing `"media_type"` export: **`supported_media_types(ptr, len)`** - Request: ignored. - Response: ```json [{ "id": "heif", "name": "HEIF Image", "category": "image", "extensions": ["heif", "heic"], "mime_types": ["image/heif", "image/heic"] }] ``` **`can_handle(ptr, len)`** - Request: `{"path": "/media/photo.heif"}`. - Response: `{"can_handle": true}`. In the pipeline first-match-wins. Plugins are checked in priority order. The first where `can_handle` returns `true` and has a matching definition claims the file. If no plugin claims it, built-in handlers run as fallback. ### `MetadataExtractor` Plugins with `kind` containing `"metadata_extractor"` export: **`supported_types(ptr, len)`** - Response: `["heif", "heic"]` **`extract_metadata(ptr, len)`** - Request: `{"path": "/media/photo.heif"}`. - Response: ```json { "title": "Sunset", "artist": "Photographer", "album": null, "description": "A sunset photo", "extra": { "camera": "Canon R5" } } ``` All fields are optional. The `extra` map accepts arbitrary key-value pairs. The pipeline features accumulating merge. All matching plugins run in priority order. Later plugins' non-null fields overwrite earlier ones. The `extra` maps are merged (later keys win). ### `ThumbnailGenerator` Plugins with `kind` containing `"thumbnail_generator"` export: **`supported_types(ptr, len)`** Same as [MetadataExtractor](#metadataextractor) **`generate_thumbnail(ptr, len)`** - Request: ```json { "source_path": "/media/photo.heif", "output_path": "/cache/thumbs/abc.jpg", "max_width": 320, "max_height": 320, "format": "jpeg" } ``` - Response: `{"path": "/cache/thumbs/abc.jpg", "width": 320, "height": 320, "format": "jpeg"}` First-success-wins. The first plugin to return a successful result produces the thumbnail. ### `SearchBackend` Plugins with `kind` containing `"search_backend"` export: **`supported_types(ptr, len)`** What media types this backend indexes. **`search(ptr, len)`** - Request: `{"query": "beethoven", "limit": 20, "offset": 0}`. - Response: ```json { "results": [ { "id": "abc-123", "score": 0.95, "snippet": "Beethoven - Symphony No. 9" } ], "total_count": 42 } ``` **`index_item(ptr, len)`** Request contains `id`, `title`, `artist`, `album`, `description`, `tags`, `media_type`, `path`. - Response: `{}`. **`remove_item(ptr, len)`** - Request: `{"id": "abc-123"}`. - Response: `{}`. Results from all backends are merged, deduplicated by ID (keeping the highest score), and sorted by score descending. `index_item` and `remove_item` are fanned out to all backends. ### `EventHandler` Plugins with `kind` containing `"event_handler"` export: **`interested_events(ptr, len)`** - Response: `["MediaImported", "MediaUpdated", "MediaDeleted"]` **`handle_event(ptr, len)`** - Request: ```json { "event_type": "MediaImported", "payload": { "media_id": "test-123", "path": "/media/song.mp3" } } ``` - Response: `{}`. All interested plugins receive the event. Events are dispatched asynchronously via `tokio::spawn` and do not block the caller. Handler failures are logged but never propagated. Suffice to say that the pipeline is fan-out. ### `ThemeProvider` > [!IMPORTANT] > `ThemeProvider` is experimental, and will likely be subject to change. Plugins with `kind` containing `"theme_provider"` export: **`get_themes(ptr, len)`** - Response: ```json [{ "id": "dark-ocean", "name": "Dark Ocean", "description": "A deep blue theme", "dark": true }] ``` **`load_theme(ptr, len)`** - Request: `{"theme_id": "dark-ocean"}`. - Response: ```json { "css": ":root { --bg: #0a1628; }", "colors": { "background": "#0a1628", "text": "#e0e0e0" } } ``` Themes from all providers are accumulated. When loading a specific theme, the pipeline dispatches to the plugin that registered it. ### System Events The server emits these events at various points: | Event | Payload fields | Emitted when | | ------------------- | ----------------------- | -------------------------- | | `ScanStarted` | `roots` | Directory scan begins | | `ScanCompleted` | `roots` | Directory scan finishes | | `MediaImported` | `media_id`, `path` | File imported via scan/API | | `MediaUpdated` | `media_id` | Media metadata updated | | `MediaDeleted` | `media_id` | Media permanently deleted | | `MediaTagged` | `media_id`, `tag` | Tag applied to media | | `MediaUntagged` | `media_id`, `tag` | Tag removed from media | | `CollectionCreated` | `collection_id`, `name` | New collection created | | `CollectionDeleted` | `collection_id` | Collection deleted | Plugins can also emit events themselves via `host_emit_event`, enabling plugin-to-plugin communication. ## Host Functions Plugins can call these host functions from within WASM (imported from the `"env"` module): ### `host_set_result(ptr, len)` Write a JSON response back to the host. The plugin writes the response bytes into its own linear memory and passes the pointer and length. This is how you return data from any extension point function. ### `host_emit_event(type_ptr, type_len, payload_ptr, payload_len) -> i32` Emit a system event from within a plugin. Enables plugin-to-plugin communication. Returns `0` on success, `-1` on error. ### `host_log(level, ptr, len)` Log a message. Levels: 0=error, 1=warn, 2=info, 3+=debug. Messages appear in the server's tracing output. ### `host_read_file(path_ptr, path_len) -> i32` Read a file into the exchange buffer. Returns file size on success, `-1` on IO error, `-2` if the path is outside the plugin's allowed read paths (paths are canonicalized before checking, so traversal tricks won't work). ### `host_write_file(path_ptr, path_len, data_ptr, data_len) -> i32` Write data to a file. Returns `0` on success, `-1` on IO error, `-2` if the path is outside allowed write paths. ### `host_http_request(url_ptr, url_len) -> i32` Make an HTTP GET request. Returns response size on success (body in exchange buffer), `-1` on error, `-2` if network access is disabled, `-3` if the domain is not in the plugin's `allowed_domains` list. ### `host_get_config(key_ptr, key_len) -> i32` Read a plugin configuration value. Returns JSON value length on success (in exchange buffer), `-1` if key not found. ### `host_get_env(key_ptr, key_len) -> i32` Read an environment variable. Returns value length on success (in exchange buffer), `-1` if the variable is not set, `-2` if environment access is disabled or the variable is not in the plugin's `environment` list. ### `host_get_buffer(dest_ptr, dest_len) -> i32` Copy the exchange buffer into WASM memory. Returns bytes copied. Use this after `host_read_file`, `host_http_request`, `host_get_config`, or `host_get_env` to retrieve data the host placed in the buffer. ## Security Model ### Capabilities Every plugin declares what it needs in `plugin.toml`. The `CapabilityEnforcer` validates these at load time _and_ at runtime; a plugin declaring `kind = ["metadata_extractor"]` cannot have `search` called on it, and a plugin without `network = true` will get `-2` from `host_http_request`. - **Filesystem**: Read and write paths must be within allowed directories. Paths are canonicalized before checking to prevent traversal attacks. Requests outside the allowlist are rejected at load time (the plugin won't load) and at runtime (the host function returns `-2`). - **Network**: Must be explicitly enabled. When `allowed_domains` is configured, each `host_http_request` call also validates the URL's domain against the whitelist (returning `-3` on violation). - **Environment**: Must be explicitly enabled. When `environment` is configured, only those specific variables can be read. - **Memory**: Each function invocation creates a fresh wasmtime `Store` with a memory limit derived from the plugin's `max_memory_mb` capability (default: 512MB). Attempts to grow linear memory beyond this limit cause a trap. - **CPU**: Enforced via wasmtime's fuel metering. Each invocation gets a fuel budget proportional to the configured CPU time limit. ### Isolation - Each `call_function` invocation creates a fresh wasmtime `Store`. Plugins cannot retain WASM runtime state between calls. If you need persistence, use the filesystem host functions. - WASM's linear memory model prevents plugins from accessing host memory. - The wasmtime stack is limited to 1MB. ### Timeouts Three tiers of per-call timeouts prevent runaway plugins: | Tier | Default | Used for | | ------------------ | ------- | --------------------------------------------------------------------------------------------- | | Capability queries | 2s | `supported_types`, `interested_events`, `can_handle`, `get_themes`, `supported_media_types` | | Processing | 30s | `extract_metadata`, `generate_thumbnail`, `search`, `index_item`, `remove_item`, `load_theme` | | Event handlers | 10s | `handle_event` | ### Circuit Breaker If a plugin fails consecutively, the circuit breaker disables it automatically: - After **5 consecutive failures** (configurable, in the config), the plugin is disabled. - A success resets the failure counter. - Disabled plugins are skipped in all pipeline stages. - Reload or toggle the plugin via the API to re-enable. ### Signatures Plugins can be signed with Ed25519. The signing flow: 1. Compute the BLAKE3 hash of the WASM binary. 2. Sign the 32-byte hash with your Ed25519 private key. 3. Write the raw 64-byte signature to `plugin.sig` alongside `plugin.toml`. On the server side, configure trusted public keys (hex-encoded, 64 hex chars each): ```toml [plugins] allow_unsigned = false trusted_keys = [ "a1b2c3d4...64 hex chars...", ] ``` When `allow_unsigned` is `false`, every plugin must carry a `plugin.sig` that verifies against at least one trusted key. If the signature is missing, invalid, or matches no trusted key, the plugin is rejected at load time. Set `allow_unsigned = true` during development to skip this check. ## Plugin Lifecycle 1. **Discovery**: On startup, the plugin manager walks configured plugin directories looking for `plugin.toml` files. 2. **Dependency resolution**: Discovered manifests are topologically sorted by their `dependencies` fields. If plugin A depends on plugin B, B is loaded first. Cycles and missing dependencies are detected. Affected plugins are skipped with a warning rather than crashing the server. 3. **Signature verification**: If `allow_unsigned` is `false`, the loader checks for a `plugin.sig` file and verifies it against the configured trusted keys. Unsigned or invalid-signature plugins are rejected. 4. **Validation**: The manifest is parsed, and capabilities are checked against the security policy. If validation fails, the plugin is skipped with a warning. 5. **Loading**: The WASM binary is compiled via wasmtime. The `initialize` export is called. 6. **Capability discovery**: The pipeline queries each plugin for its supported types, media type definitions, theme definitions, and interested events. Results are cached for fast dispatch. 7. **Invocation**: When the system needs plugin functionality, the pipeline dispatches to plugins in priority order using the appropriate merge strategy. 8. **Shutdown**: On server shutdown or plugin disable, the `shutdown` export is called. 9. **Hot-reload**: The `/plugins/:id/reload` endpoint reloads the WASM binary from disk and re-discovers capabilities without restarting the server. ## Plugin API Endpoints All plugin endpoints require the Admin role. | Method | Path | Description | | -------- | ----------------------------- | ------------------------------- | | `GET` | `/api/v1/plugins` | List all plugins | | `GET` | `/api/v1/plugins/:id` | Get plugin details | | `POST` | `/api/v1/plugins/install` | Install from path or URL | | `DELETE` | `/api/v1/plugins/:id` | Uninstall a plugin | | `POST` | `/api/v1/plugins/:id/enable` | Enable a disabled plugin | | `POST` | `/api/v1/plugins/:id/disable` | Disable a plugin | | `POST` | `/api/v1/plugins/:id/reload` | Hot-reload and re-discover caps | Reload and toggle operations automatically re-discover the plugin's capabilities, so changes to supported types or event subscriptions take effect immediately. ## Configuration In `pinakes.toml`: ```toml [plugins] enabled = true plugin_dirs = ["/etc/pinakes/plugins", "~/.local/share/pinakes/plugins"] allow_unsigned = false trusted_keys = ["a1b2c3d4..."] # Ed25519 public keys (hex-encoded) max_concurrent_ops = 4 plugin_timeout_secs = 30 max_consecutive_failures = 5 [plugins.timeouts] capability_query_secs = 2 processing_secs = 30 event_handler_secs = 10 [plugins.security] allow_network_default = false allowed_read_paths = ["/media", "/tmp/pinakes"] allowed_write_paths = ["/tmp/pinakes/plugin-data"] ``` ## Debugging - **`host_log`**: Call from within your plugin to emit structured log messages. They appear in the server's tracing output. - **Server logs**: Set `RUST_LOG=pinakes_core::plugin=debug` to see plugin dispatch decisions, timeout warnings, and circuit breaker state changes. - **Reload workflow**: Edit your plugin source, rebuild the WASM binary, then `POST /api/v1/plugins/:id/reload`. No server restart needed. - **Circuit breaker**: If your plugin is silently skipped, check the server logs for "circuit breaker tripped" messages. Fix the issue, then re-enable via `POST /api/v1/plugins/:id/enable`.