pinakes/examples/plugins/heif-support/README.md

HEIF/HEIC Support Plugin

This example plugin adds support for HEIF (High Efficiency Image Format) and HEIC (HEIF Container) to Pinakes.

Overview

HEIF is a modern image format that provides better compression than JPEG while maintaining higher quality. This plugin enables Pinakes to:

• Recognize HEIF/HEIC files as a media type
• Extract metadata from HEIF images
• Generate thumbnails from HEIF images

Features

• Media Type Registration: Registers .heif, .heic, .hif extensions as image media types
• EXIF Extraction: Extracts EXIF metadata including camera info, GPS coordinates, timestamps
• Thumbnail Generation: Generates thumbnails in JPEG, PNG, or WebP format
• Resource Limits: Configurable memory and CPU limits for safe processing
• Large Image Support: Handles images up to 8192x8192 pixels

Supported Formats

• HEIF: High Efficiency Image Format (.heif, .hif)
• HEIC: HEIF Container format used by Apple devices (.heic)
• HEIF Sequences: Multi-image HEIF files
• HEIF with Alpha: HEIF images with transparency

Implementation

The plugin implements three traits:

MediaTypeProvider

#[async_trait]
impl MediaTypeProvider for HeifPlugin {
    fn supported_media_types(&self) -> Vec<MediaTypeDefinition> {
        vec![MediaTypeDefinition {
            id: "heif".to_string(),
            name: "HEIF Image".to_string(),
            category: "image".to_string(),
            extensions: vec!["heif".to_string(), "heic".to_string(), "hif".to_string()],
            mime_types: vec!["image/heif".to_string(), "image/heic".to_string()],
            icon: Some("image".to_string()),
        }]
    }

    async fn can_handle(&self, path: &PathBuf, mime_type: Option<&str>) -> PluginResult<bool> {
        // Check file extension and/or MIME type
    }
}

MetadataExtractor

#[async_trait]
impl MetadataExtractor for HeifPlugin {
    async fn extract_metadata(&self, path: &PathBuf) -> PluginResult<ExtractedMetadata> {
        // 1. Parse HEIF file structure
        // 2. Extract EXIF metadata
        // 3. Get image dimensions
        // 4. Return ExtractedMetadata
    }

    fn supported_types(&self) -> Vec<String> {
        vec!["heif".to_string()]
    }
}

ThumbnailGenerator

#[async_trait]
impl ThumbnailGenerator for HeifPlugin {
    async fn generate_thumbnail(
        &self,
        path: &PathBuf,
        output_path: &PathBuf,
        options: ThumbnailOptions,
    ) -> PluginResult<ThumbnailInfo> {
        // 1. Decode HEIF image
        // 2. Resize to thumbnail dimensions
        // 3. Encode to output format
        // 4. Save to output_path
        // 5. Return ThumbnailInfo
    }

    fn supported_types(&self) -> Vec<String> {
        vec!["heif".to_string()]
    }
}

Dependencies

The plugin uses the following Rust crates (compiled to WASM):

• libheif-rs: HEIF decoding and encoding
• image: Image processing and thumbnail generation
• kamadak-exif: EXIF metadata parsing

Building

Prerequisites

# Install WASM target
rustup target add wasm32-wasi

# Install wasm-tools for optimization (provides strip functionality)
cargo install wasm-tools

Build Process

# Build the plugin
cargo build --target wasm32-wasi --release

# Strip debug symbols to reduce size
wasm-tools strip target/wasm32-wasi/release/heif_support.wasm -o target/wasm32-wasi/release/heif_support.wasm

# Copy to plugin directory
cp target/wasm32-wasi/release/heif_support.wasm .

Size Optimization

# Use wasm-opt for further optimization
wasm-opt -Oz heif_support.wasm -o heif_support_optimized.wasm

Installation

Manual Installation

# Copy plugin directory to Pinakes plugins directory
cp -r examples/plugins/heif-support ~/.config/pinakes/plugins/

Via API

curl -X POST http://localhost:3000/api/v1/plugins/install \
  -H "Content-Type: application/json" \
  -d '{"source": "/path/to/heif-support"}'

Via Plugin Manager

pinakes plugin install /path/to/heif-support

Configuration

The plugin can be configured through the config section in plugin.toml:

EXIF Extraction

• extract_exif: Enable EXIF metadata extraction (default: true)

Thumbnail Generation

• generate_thumbnails: Enable thumbnail generation (default: true)
• thumbnail_quality: JPEG quality for thumbnails, 1-100 (default: 85)
• thumbnail_format: Output format - "jpeg", "png", or "webp" (default: "jpeg")

Resource Limits

• max_memory_mb: Maximum memory the plugin can use in megabytes (default: 256, set in [capabilities])
• max_width: Maximum image width to process (default: 8192)
• max_height: Maximum image height to process (default: 8192)

Security

Capabilities

• Filesystem Read: Only files being processed (via Pinakes)
• Filesystem Write: Thumbnail directory only
• Network: Disabled
• Environment: No access

Resource Limits

• Memory: 256 MB maximum
• CPU Time: 30 seconds maximum per operation

Sandboxing

The plugin runs in a WASM sandbox with:

• No access to host filesystem beyond granted paths
• No network access
• No arbitrary code execution
• Memory and CPU time limits enforced by runtime

Performance

Typical Performance

• Metadata Extraction: ~50-100ms for typical HEIF files
• Thumbnail Generation: ~200-500ms depending on source image size
• Memory Usage: 50-150 MB typical, 256 MB maximum

Optimization Tips

1. Keep source images below 8192x8192 for best performance
2. Use JPEG thumbnail format for smaller file sizes
3. Adjust thumbnail quality vs file size tradeoff with thumbnail_quality

Error Handling

The plugin handles:

• Corrupted Files: Returns descriptive error
• Unsupported Variants: Gracefully skips unsupported HEIF features
• Memory Limits: Fails safely if image too large
• Timeout: Returns error if processing exceeds CPU time limit

Testing

# Run unit tests
cargo test

# Test with sample HEIF files
cargo test --test integration -- --nocapture

Troubleshooting

Plugin Fails to Load

• Check that heif_support.wasm exists in plugin directory
• Verify plugin.toml is valid TOML
• Check Pinakes logs for detailed error messages

Thumbnails Not Generated

• Verify generate_thumbnails = true in config
• Check filesystem write permissions for thumbnail directory
• Ensure source image is below size limits

Out of Memory Errors

• Reduce max_width and max_height in config
• Increase max_memory_mb if source images are large
• Check that source files aren't corrupted

Future Enhancements

• Support for HEIF image sequences (burst photos)
• HDR metadata extraction
• Live Photo support
• AVIF format support (similar to HEIF)

License

MIT