pinakes/examples/plugins/heif-support/README.md

# HEIF/HEIC Support Plugin

This example plugin adds support for HEIF (High Efficiency Image Format) and HEIC (HEIF Container) to Pinakes.

## Overview

HEIF is a modern image format that provides better compression than JPEG while maintaining higher quality. This plugin enables Pinakes to:
- Recognize HEIF/HEIC files as a media type
- Extract metadata from HEIF images
- Generate thumbnails from HEIF images

## Features

- **Media Type Registration**: Registers `.heif`, `.heic`, `.hif` extensions as image media types
- **EXIF Extraction**: Extracts EXIF metadata including camera info, GPS coordinates, timestamps
- **Thumbnail Generation**: Generates thumbnails in JPEG, PNG, or WebP format
- **Resource Limits**: Configurable memory and CPU limits for safe processing
- **Large Image Support**: Handles images up to 8192x8192 pixels

## Supported Formats

- **HEIF**: High Efficiency Image Format (`.heif`, `.hif`)
- **HEIC**: HEIF Container format used by Apple devices (`.heic`)
- **HEIF Sequences**: Multi-image HEIF files
- **HEIF with Alpha**: HEIF images with transparency

## Implementation

The plugin implements three traits:

### MediaTypeProvider

```rust
#[async_trait]
impl MediaTypeProvider for HeifPlugin {
    fn supported_media_types(&self) -> Vec<MediaTypeDefinition> {
        vec![MediaTypeDefinition {
            id: "heif".to_string(),
            name: "HEIF Image".to_string(),
            category: "image".to_string(),
            extensions: vec!["heif".to_string(), "heic".to_string(), "hif".to_string()],
            mime_types: vec!["image/heif".to_string(), "image/heic".to_string()],
            icon: Some("image".to_string()),
        }]
    }

    async fn can_handle(&self, path: &PathBuf, mime_type: Option<&str>) -> PluginResult<bool> {
        // Check file extension and/or MIME type
    }
}
```

### MetadataExtractor

```rust
#[async_trait]
impl MetadataExtractor for HeifPlugin {
    async fn extract_metadata(&self, path: &PathBuf) -> PluginResult<ExtractedMetadata> {
        // 1. Parse HEIF file structure
        // 2. Extract EXIF metadata
        // 3. Get image dimensions
        // 4. Return ExtractedMetadata
    }

    fn supported_types(&self) -> Vec<String> {
        vec!["heif".to_string()]
    }
}
```

### ThumbnailGenerator

```rust
#[async_trait]
impl ThumbnailGenerator for HeifPlugin {
    async fn generate_thumbnail(
        &self,
        path: &PathBuf,
        output_path: &PathBuf,
        options: ThumbnailOptions,
    ) -> PluginResult<ThumbnailInfo> {
        // 1. Decode HEIF image
        // 2. Resize to thumbnail dimensions
        // 3. Encode to output format
        // 4. Save to output_path
        // 5. Return ThumbnailInfo
    }

    fn supported_types(&self) -> Vec<String> {
        vec!["heif".to_string()]
    }
}
```

## Dependencies

The plugin uses the following Rust crates (compiled to WASM):
- `libheif-rs`: HEIF decoding and encoding
- `image`: Image processing and thumbnail generation
- `kamadak-exif`: EXIF metadata parsing

## Building

### Prerequisites

```bash
# Install WASM target
rustup target add wasm32-wasi

# Install wasm-tools for optimization (provides strip functionality)
cargo install wasm-tools
```

### Build Process

```bash
# Build the plugin
cargo build --target wasm32-wasi --release

# Strip debug symbols to reduce size
wasm-tools strip target/wasm32-wasi/release/heif_support.wasm -o target/wasm32-wasi/release/heif_support.wasm

# Copy to plugin directory
cp target/wasm32-wasi/release/heif_support.wasm .
```

### Size Optimization

```bash
# Use wasm-opt for further optimization
wasm-opt -Oz heif_support.wasm -o heif_support_optimized.wasm
```

## Installation

### Manual Installation

```bash
# Copy plugin directory to Pinakes plugins directory
cp -r examples/plugins/heif-support ~/.config/pinakes/plugins/
```

### Via API

```bash
curl -X POST http://localhost:3000/api/v1/plugins/install \
  -H "Content-Type: application/json" \
  -d '{"source": "/path/to/heif-support"}'
```

### Via Plugin Manager

```bash
pinakes plugin install /path/to/heif-support
```

## Configuration

The plugin can be configured through the `config` section in `plugin.toml`:

### EXIF Extraction
- `extract_exif`: Enable EXIF metadata extraction (default: true)

### Thumbnail Generation
- `generate_thumbnails`: Enable thumbnail generation (default: true)
- `thumbnail_quality`: JPEG quality for thumbnails, 1-100 (default: 85)
- `thumbnail_format`: Output format - "jpeg", "png", or "webp" (default: "jpeg")

### Resource Limits
- `max_memory_mb`: Maximum memory the plugin can use in megabytes (default: 256, set in `[capabilities]`)
- `max_width`: Maximum image width to process (default: 8192)
- `max_height`: Maximum image height to process (default: 8192)

## Security

### Capabilities

- **Filesystem Read**: Only files being processed (via Pinakes)
- **Filesystem Write**: Thumbnail directory only
- **Network**: Disabled
- **Environment**: No access

### Resource Limits

- **Memory**: 256 MB maximum
- **CPU Time**: 30 seconds maximum per operation

### Sandboxing

The plugin runs in a WASM sandbox with:
- No access to host filesystem beyond granted paths
- No network access
- No arbitrary code execution
- Memory and CPU time limits enforced by runtime

## Performance

### Typical Performance

- **Metadata Extraction**: ~50-100ms for typical HEIF files
- **Thumbnail Generation**: ~200-500ms depending on source image size
- **Memory Usage**: 50-150 MB typical, 256 MB maximum

### Optimization Tips

1. Keep source images below 8192x8192 for best performance
2. Use JPEG thumbnail format for smaller file sizes
3. Adjust thumbnail quality vs file size tradeoff with `thumbnail_quality`

## Error Handling

The plugin handles:
- **Corrupted Files**: Returns descriptive error
- **Unsupported Variants**: Gracefully skips unsupported HEIF features
- **Memory Limits**: Fails safely if image too large
- **Timeout**: Returns error if processing exceeds CPU time limit

## Testing

```bash
# Run unit tests
cargo test

# Test with sample HEIF files
cargo test --test integration -- --nocapture
```

## Troubleshooting

### Plugin Fails to Load

- Check that `heif_support.wasm` exists in plugin directory
- Verify `plugin.toml` is valid TOML
- Check Pinakes logs for detailed error messages

### Thumbnails Not Generated

- Verify `generate_thumbnails = true` in config
- Check filesystem write permissions for thumbnail directory
- Ensure source image is below size limits

### Out of Memory Errors

- Reduce `max_width` and `max_height` in config
- Increase `max_memory_mb` if source images are large
- Check that source files aren't corrupted

## Future Enhancements

- Support for HEIF image sequences (burst photos)
- HDR metadata extraction
- Live Photo support
- AVIF format support (similar to HEIF)

## License

MIT