pinakes/docs/plugins.md

Plugin System

Pinakes is very powerful on its own, but with a goal as ambitious as "to be the last media management system you will ever need" I recognize the need for a plugin system; not everything belongs in the core of Pinakes. Thus, Pinakes supports WASM-based plugins for extending media type support, metadata extraction, thumbnail generation, search, event handling, and theming.

Plugins run in a sandboxed wasmtime runtime with capability-based security, fuel metering, memory limits, and a circuit breaker for fault isolation.

How It Works

A plugin is a directory containing:

my-plugin/
  plugin.toml          Manifest declaring name, version, capabilities
  my_plugin.wasm       Compiled WASM binary (wasm32-unknown-unknown target)

The server discovers plugins in configured directories, validates their manifests, checks capabilities against the security policy, compiles the WASM module, and registers the plugin.

Extension points communicate via JSON-over-WASM. The host writes a JSON request into the plugin's memory, calls the exported function, and reads the JSON response written via host_set_result. This keeps the interface relatively simple and even language-agnostic. Technically, you could write a plugin in any language that compiles to wasm32-unknown-unknown. While the JSON model introduces a little bit of overhead, it's the more debuggable approach and thus more suitable for the initial plugin system. In the future, this might change.

Plugins go through a priority-ordered pipeline. Each plugin declares a priority (0–999, default 500). Built-in handlers run at implicit priority 100, so plugins at priority <100 run before built-ins and plugins at >100 run after. Different extension points use different merge strategies:

Extension Point	Strategy	What happens
Media type	First match wins	First plugin whose can_handle returns true claims the file
Metadata	Accumulating merge	All matching plugins run; later fields overwrite earlier ones
Thumbnails	First success wins	First plugin to succeed produces the thumbnail
Search	Merge results	Results from all backends are combined and ranked
Themes	Accumulate	All themes from all providers are available
Events	Fan-out	All interested plugins receive the event

Plugin Kinds

A plugin can implement one or more of these roles:

Kind	Purpose
media_type	Register new file formats (extensions, MIME types)
metadata_extractor	Extract metadata from files of specific types
thumbnail_generator	Generate thumbnails for specific media types
search_backend	Provide alternative search implementations
event_handler	React to system events (imports, deletes, etc.)
theme_provider	Supply UI themes
general	General-purpose (uses host functions freely)

Declare kinds in plugin.toml:

[plugin]
name = "heif-support"
version = "1.0.0"
api_version = "1.0"
kind = ["media_type", "metadata_extractor", "thumbnail_generator"]
priority = 50

Writing a Plugin

You are no doubt wondering how to write a plugin, but fret not. It is quite simple: plugins target wasm32-unknown-unknown with #![no_std]. This is a deliberate choice, no_std keeps binaries small and avoids libc dependencies that would complicate the WASM target. Use dlmalloc for a global allocator and provide a panic handler.

# Cargo.toml
[lib]
crate-type = ["cdylib"]

[dependencies]
dlmalloc = { version = "0.2", features = ["global"] }

[profile.release]
opt-level = "s"
lto = true

Let's go over a minimal example that registers a custom .mytype file format:

#![no_std]

extern crate alloc;

use core::alloc::Layout;

#[global_allocator]
static ALLOC: dlmalloc::GlobalDlmalloc = dlmalloc::GlobalDlmalloc;

#[panic_handler]
fn panic_handler(_: &core::panic::PanicInfo) -> ! {
    core::arch::wasm32::unreachable()
}

unsafe extern "C" {
    fn host_set_result(ptr: i32, len: i32);
    fn host_log(level: i32, ptr: i32, len: i32);
}

fn set_response(json: &[u8]) {
    unsafe { host_set_result(json.as_ptr() as i32, json.len() as i32); }
}

#[unsafe(no_mangle)]
pub extern "C" fn alloc(size: i32) -> i32 {
    if size <= 0 { return 0; }
    unsafe {
        let layout = Layout::from_size_align(size as usize, 1).unwrap();
        let ptr = alloc::alloc::alloc(layout);
        if ptr.is_null() { -1 } else { ptr as i32 }
    }
}

#[unsafe(no_mangle)]
pub extern "C" fn initialize() -> i32 { 0 }

#[unsafe(no_mangle)]
pub extern "C" fn shutdown() -> i32 { 0 }

#[unsafe(no_mangle)]
pub extern "C" fn supported_media_types(_ptr: i32, _len: i32) {
    set_response(br#"[{"id":"mytype","name":"My Type","category":"document","extensions":["mytype"],"mime_types":["application/x-mytype"]}]"#);
}

#[unsafe(no_mangle)]
pub extern "C" fn can_handle(ptr: i32, len: i32) {
    let req = unsafe { core::slice::from_raw_parts(ptr as *const u8, len as usize) };
    let json = core::str::from_utf8(req).unwrap_or("");
    if json.contains(".mytype") {
        set_response(br#"{"can_handle":true}"#);
    } else {
        set_response(br#"{"can_handle":false}"#);
    }
}

Building

$ cargo build --target wasm32-unknown-unknown --release

The RUSTFLAGS="" override may be needed if your environment sets linker flags (e.g., -fuse-ld=lld) that are incompatible with the WASM target. You can also specify a compatible linker explicitly. As pinakes uses a clang and lld based pipeline, it's necessary to set for, e.g., the test fixtures.

The compiled binary will be at target/wasm32-unknown-unknown/release/my_plugin.wasm.

A note on serde in no_std

Since serde_json requires std, you cannot use it in a plugin. Either hand-write JSON strings (as shown above) or use a lightweight no_std JSON library.

The test fixture plugin in crates/pinakes-core/tests/fixtures/test-plugin/ demonstrates the hand-written approach. It is ugly, but it works, and the binaries are tiny (~17KB).

Installing

Place the plugin directory in one of the configured plugin directories, or use the API:

curl -X POST http://localhost:3000/api/v1/plugins/install \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"source": "/path/to/my-plugin"}'

Manifest Reference

Plugins are required to provide a manifest with explicit intents for everything. Here is the expected manifest format as of 0.3.0-dev version of Pinakes:

[plugin]
name = "my-plugin"               # Unique identifier (required)
version = "1.0.0"                # SemVer version (required)
api_version = "1.0"              # Plugin API version (required)
description = "What it does"     # Human-readable description
author = "Your Name"
kind = [                         # One or more roles (required)
  "media_type",
  "metadata_extractor",
  "thumbnail_generator",
  "event_handler",
]
priority = 500                   # 0-999, lower runs first (default: 500)
dependencies = ["other-plugin"]  # Plugins that must load first (optional)

[plugin.binary]
wasm = "my_plugin.wasm"          # Path to WASM binary relative to plugin dir

[capabilities]
network = false                  # Allow host_http_request calls
allowed_domains = ["api.example.com"]  # Domain whitelist (enforced at runtime)
environment = ["HOME", "LANG"]   # Allowed env vars for host_get_env

[capabilities.filesystem]
read = ["/media"]                # Allowed read paths for host_read_file
write = ["/tmp/plugin-data"]     # Allowed write paths for host_write_file

[capabilities.resources]
max_memory_mb = 64               # Maximum linear memory (default: 512MB)
max_cpu_time_secs = 5            # Fuel budget per invocation (default: 60s)

Extension Points

Every plugin must export these three functions:

Export	Signature	Purpose
alloc	(size: i32) -> i32	Allocate memory for host to write
initialize	() -> i32	Called once on load (0 = success)
shutdown	() -> i32	Called before unload (0 = success)

Beyond those, which functions you export depends on your plugin's kind(s). The capability enforcer prevents a plugin from being called for functions it hasn't declared; a metadata_extractor plugin cannot have search called on it.

MediaTypeProvider

Plugins with kind containing "media_type" export:

supported_media_types(ptr, len)

• Request: ignored.

• Response:

  [{
    "id": "heif",
    "name": "HEIF Image",
    "category": "image",
    "extensions": ["heif", "heic"],
    "mime_types": ["image/heif", "image/heic"]
  }]
  

can_handle(ptr, len)

• Request: {"path": "/media/photo.heif"}.
• Response: {"can_handle": true}.

In the pipeline first-match-wins. Plugins are checked in priority order. The first where can_handle returns true and has a matching definition claims the file. If no plugin claims it, built-in handlers run as fallback.

MetadataExtractor

Plugins with kind containing "metadata_extractor" export:

supported_types(ptr, len)

• Response: ["heif", "heic"]

extract_metadata(ptr, len)

• Request: {"path": "/media/photo.heif"}.

• Response:

  {
    "title": "Sunset",
    "artist": "Photographer",
    "album": null,
    "description": "A sunset photo",
    "extra": { "camera": "Canon R5" }
  }
  

All fields are optional. The extra map accepts arbitrary key-value pairs.

The pipeline features accumulating merge. All matching plugins run in priority order. Later plugins' non-null fields overwrite earlier ones. The extra maps are merged (later keys win).

ThumbnailGenerator

Plugins with kind containing "thumbnail_generator" export:

supported_types(ptr, len)

Same as MetadataExtractor

generate_thumbnail(ptr, len)

• Request:

  {
    "source_path": "/media/photo.heif",
    "output_path": "/cache/thumbs/abc.jpg",
    "max_width": 320,
    "max_height": 320,
    "format": "jpeg"
  }
  

• Response: {"path": "/cache/thumbs/abc.jpg", "width": 320, "height": 320, "format": "jpeg"}

First-success-wins. The first plugin to return a successful result produces the thumbnail.

SearchBackend

Plugins with kind containing "search_backend" export:

supported_types(ptr, len)

What media types this backend indexes.

search(ptr, len)

• Request: {"query": "beethoven", "limit": 20, "offset": 0}.

• Response:

  {
    "results": [
      {
        "id": "abc-123",
        "score": 0.95,
        "snippet": "Beethoven - Symphony No. 9"
      }
    ],
    "total_count": 42
  }
  

index_item(ptr, len)

Request contains id, title, artist, album, description, tags, media_type, path.

• Response: {}.

remove_item(ptr, len)

• Request: {"id": "abc-123"}.
• Response: {}.

Results from all backends are merged, deduplicated by ID (keeping the highest score), and sorted by score descending. index_item and remove_item are fanned out to all backends.

EventHandler

Plugins with kind containing "event_handler" export:

interested_events(ptr, len)

• Response: ["MediaImported", "MediaUpdated", "MediaDeleted"]

handle_event(ptr, len)

• Request:

  {
    "event_type": "MediaImported",
    "payload": { "media_id": "test-123", "path": "/media/song.mp3" }
  }
  

• Response: {}.

All interested plugins receive the event. Events are dispatched asynchronously via tokio::spawn and do not block the caller. Handler failures are logged but never propagated. Suffice to say that the pipeline is fan-out.

ThemeProvider

Important

ThemeProvider is experimental, and will likely be subject to change.

Plugins with kind containing "theme_provider" export:

get_themes(ptr, len)

• Response:

  [{
    "id": "dark-ocean",
    "name": "Dark Ocean",
    "description": "A deep blue theme",
    "dark": true
  }]
  

load_theme(ptr, len)

• Request: {"theme_id": "dark-ocean"}.

• Response:

  {
    "css": ":root { --bg: #0a1628; }",
    "colors": { "background": "#0a1628", "text": "#e0e0e0" }
  }
  

Themes from all providers are accumulated. When loading a specific theme, the pipeline dispatches to the plugin that registered it.

System Events

The server emits these events at various points:

Event	Payload fields	Emitted when
ScanStarted	roots	Directory scan begins
ScanCompleted	roots	Directory scan finishes
MediaImported	media_id, path	File imported via scan/API
MediaUpdated	media_id	Media metadata updated
MediaDeleted	media_id	Media permanently deleted
MediaTagged	media_id, tag	Tag applied to media
MediaUntagged	media_id, tag	Tag removed from media
CollectionCreated	collection_id, name	New collection created
CollectionDeleted	collection_id	Collection deleted

Plugins can also emit events themselves via host_emit_event, enabling plugin-to-plugin communication.

Host Functions

Plugins can call these host functions from within WASM (imported from the "env" module):

host_set_result(ptr, len)

Write a JSON response back to the host. The plugin writes the response bytes into its own linear memory and passes the pointer and length. This is how you return data from any extension point function.

host_emit_event(type_ptr, type_len, payload_ptr, payload_len) -> i32

Emit a system event from within a plugin. Enables plugin-to-plugin communication.

Returns 0 on success, -1 on error.

host_log(level, ptr, len)

Log a message.

Levels: 0=error, 1=warn, 2=info, 3+=debug. Messages appear in the server's tracing output.

host_read_file(path_ptr, path_len) -> i32

Read a file into the exchange buffer. Returns file size on success, -1 on IO error, -2 if the path is outside the plugin's allowed read paths (paths are canonicalized before checking, so traversal tricks won't work).

host_write_file(path_ptr, path_len, data_ptr, data_len) -> i32

Write data to a file.

Returns 0 on success, -1 on IO error, -2 if the path is outside allowed write paths.

host_http_request(url_ptr, url_len) -> i32

Make an HTTP GET request.

Returns response size on success (body in exchange buffer), -1 on error, -2 if network access is disabled, -3 if the domain is not in the plugin's allowed_domains list.

host_get_config(key_ptr, key_len) -> i32

Read a plugin configuration value. Returns JSON value length on success (in exchange buffer), -1 if key not found.

host_get_env(key_ptr, key_len) -> i32

Read an environment variable.

Returns value length on success (in exchange buffer), -1 if the variable is not set, -2 if environment access is disabled or the variable is not in the plugin's environment list.

host_get_buffer(dest_ptr, dest_len) -> i32

Copy the exchange buffer into WASM memory.

Returns bytes copied. Use this after host_read_file, host_http_request, host_get_config, or host_get_env to retrieve data the host placed in the buffer.

Security Model

Capabilities

Every plugin declares what it needs in plugin.toml. The CapabilityEnforcer validates these at load time and at runtime; a plugin declaring kind = ["metadata_extractor"] cannot have search called on it, and a plugin without network = true will get -2 from host_http_request.

• Filesystem: Read and write paths must be within allowed directories. Paths are canonicalized before checking to prevent traversal attacks. Requests outside the allowlist are rejected at load time (the plugin won't load) and at runtime (the host function returns -2).

• Network: Must be explicitly enabled. When allowed_domains is configured, each host_http_request call also validates the URL's domain against the whitelist (returning -3 on violation).

• Environment: Must be explicitly enabled. When environment is configured, only those specific variables can be read.

• Memory: Each function invocation creates a fresh wasmtime Store with a memory limit derived from the plugin's max_memory_mb capability (default: 512MB). Attempts to grow linear memory beyond this limit cause a trap.

• CPU: Enforced via wasmtime's fuel metering. Each invocation gets a fuel budget proportional to the configured CPU time limit.

Isolation

• Each call_function invocation creates a fresh wasmtime Store. Plugins cannot retain WASM runtime state between calls. If you need persistence, use the filesystem host functions.

• WASM's linear memory model prevents plugins from accessing host memory.

• The wasmtime stack is limited to 1MB.

Timeouts

Three tiers of per-call timeouts prevent runaway plugins:

Tier	Default	Used for
Capability queries	2s	supported_types, interested_events, can_handle, get_themes, supported_media_types
Processing	30s	extract_metadata, generate_thumbnail, search, index_item, remove_item, load_theme
Event handlers	10s	handle_event

Circuit Breaker

If a plugin fails consecutively, the circuit breaker disables it automatically:

• After 5 consecutive failures (configurable, in the config), the plugin is disabled.
• A success resets the failure counter.
• Disabled plugins are skipped in all pipeline stages.
• Reload or toggle the plugin via the API to re-enable.

Signatures

Plugins can be signed with Ed25519. The signing flow:

1. Compute the BLAKE3 hash of the WASM binary.
2. Sign the 32-byte hash with your Ed25519 private key.
3. Write the raw 64-byte signature to plugin.sig alongside plugin.toml.

On the server side, configure trusted public keys (hex-encoded, 64 hex chars each):

[plugins]
allow_unsigned = false
trusted_keys = [
  "a1b2c3d4...64 hex chars...",
]

When allow_unsigned is false, every plugin must carry a plugin.sig that verifies against at least one trusted key. If the signature is missing, invalid, or matches no trusted key, the plugin is rejected at load time. Set allow_unsigned = true during development to skip this check.

Plugin Lifecycle

1. Discovery: On startup, the plugin manager walks configured plugin directories looking for plugin.toml files.

2. Dependency resolution: Discovered manifests are topologically sorted by their dependencies fields. If plugin A depends on plugin B, B is loaded first. Cycles and missing dependencies are detected. Affected plugins are skipped with a warning rather than crashing the server.

3. Signature verification: If allow_unsigned is false, the loader checks for a plugin.sig file and verifies it against the configured trusted keys. Unsigned or invalid-signature plugins are rejected.

4. Validation: The manifest is parsed, and capabilities are checked against the security policy. If validation fails, the plugin is skipped with a warning.

5. Loading: The WASM binary is compiled via wasmtime. The initialize export is called.

6. Capability discovery: The pipeline queries each plugin for its supported types, media type definitions, theme definitions, and interested events. Results are cached for fast dispatch.

7. Invocation: When the system needs plugin functionality, the pipeline dispatches to plugins in priority order using the appropriate merge strategy.

8. Shutdown: On server shutdown or plugin disable, the shutdown export is called.

9. Hot-reload: The /plugins/:id/reload endpoint reloads the WASM binary from disk and re-discovers capabilities without restarting the server.

Plugin API Endpoints

All plugin endpoints require the Admin role.

Method	Path	Description
GET	/api/v1/plugins	List all plugins
GET	/api/v1/plugins/:id	Get plugin details
POST	/api/v1/plugins/install	Install from path or URL
DELETE	/api/v1/plugins/:id	Uninstall a plugin
POST	/api/v1/plugins/:id/enable	Enable a disabled plugin
POST	/api/v1/plugins/:id/disable	Disable a plugin
POST	/api/v1/plugins/:id/reload	Hot-reload and re-discover caps

Reload and toggle operations automatically re-discover the plugin's capabilities, so changes to supported types or event subscriptions take effect immediately.

Configuration

In pinakes.toml:

[plugins]
enabled = true
plugin_dirs = ["/etc/pinakes/plugins", "~/.local/share/pinakes/plugins"]
allow_unsigned = false
trusted_keys = ["a1b2c3d4..."] # Ed25519 public keys (hex-encoded)
max_concurrent_ops = 4
plugin_timeout_secs = 30
max_consecutive_failures = 5

[plugins.timeouts]
capability_query_secs = 2
processing_secs = 30
event_handler_secs = 10

[plugins.security]
allow_network_default = false
allowed_read_paths = ["/media", "/tmp/pinakes"]
allowed_write_paths = ["/tmp/pinakes/plugin-data"]

Debugging

• host_log: Call from within your plugin to emit structured log messages. They appear in the server's tracing output.
• Server logs: Set RUST_LOG=pinakes_core::plugin=debug to see plugin dispatch decisions, timeout warnings, and circuit breaker state changes.
• Reload workflow: Edit your plugin source, rebuild the WASM binary, then POST /api/v1/plugins/:id/reload. No server restart needed.
• Circuit breaker: If your plugin is silently skipped, check the server logs for "circuit breaker tripped" messages. Fix the issue, then re-enable via POST /api/v1/plugins/:id/enable.