examples: add example plugins
Signed-off-by: NotAShelf <raf@notashelf.dev> Change-Id: I9eac30c7d4c1c89178f4930b215e523d6a6a6964
This commit is contained in:
parent
bee18284ee
commit
708f8a0b67
5 changed files with 945 additions and 0 deletions
257
examples/plugins/heif-support/README.md
Normal file
257
examples/plugins/heif-support/README.md
Normal file
|
|
@ -0,0 +1,257 @@
|
|||
# HEIF/HEIC Support Plugin
|
||||
|
||||
This example plugin adds support for HEIF (High Efficiency Image Format) and HEIC (HEIF Container) to Pinakes.
|
||||
|
||||
## Overview
|
||||
|
||||
HEIF is a modern image format that provides better compression than JPEG while maintaining higher quality. This plugin enables Pinakes to:
|
||||
- Recognize HEIF/HEIC files as a media type
|
||||
- Extract metadata from HEIF images
|
||||
- Generate thumbnails from HEIF images
|
||||
|
||||
## Features
|
||||
|
||||
- **Media Type Registration**: Registers `.heif`, `.heic`, `.hif` extensions as image media types
|
||||
- **EXIF Extraction**: Extracts EXIF metadata including camera info, GPS coordinates, timestamps
|
||||
- **Thumbnail Generation**: Generates thumbnails in JPEG, PNG, or WebP format
|
||||
- **Resource Limits**: Configurable memory and CPU limits for safe processing
|
||||
- **Large Image Support**: Handles images up to 8192x8192 pixels
|
||||
|
||||
## Supported Formats
|
||||
|
||||
- **HEIF**: High Efficiency Image Format (`.heif`, `.hif`)
|
||||
- **HEIC**: HEIF Container format used by Apple devices (`.heic`)
|
||||
- **HEIF Sequences**: Multi-image HEIF files
|
||||
- **HEIF with Alpha**: HEIF images with transparency
|
||||
|
||||
## Implementation
|
||||
|
||||
The plugin implements three traits:
|
||||
|
||||
### MediaTypeProvider
|
||||
|
||||
```rust
|
||||
#[async_trait]
|
||||
impl MediaTypeProvider for HeifPlugin {
|
||||
fn supported_media_types(&self) -> Vec<MediaTypeDefinition> {
|
||||
vec![MediaTypeDefinition {
|
||||
id: "heif".to_string(),
|
||||
name: "HEIF Image".to_string(),
|
||||
category: "image".to_string(),
|
||||
extensions: vec!["heif".to_string(), "heic".to_string(), "hif".to_string()],
|
||||
mime_types: vec!["image/heif".to_string(), "image/heic".to_string()],
|
||||
icon: Some("image".to_string()),
|
||||
}]
|
||||
}
|
||||
|
||||
async fn can_handle(&self, path: &PathBuf, mime_type: Option<&str>) -> PluginResult<bool> {
|
||||
// Check file extension and/or MIME type
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### MetadataExtractor
|
||||
|
||||
```rust
|
||||
#[async_trait]
|
||||
impl MetadataExtractor for HeifPlugin {
|
||||
async fn extract_metadata(&self, path: &PathBuf) -> PluginResult<ExtractedMetadata> {
|
||||
// 1. Parse HEIF file structure
|
||||
// 2. Extract EXIF metadata
|
||||
// 3. Get image dimensions
|
||||
// 4. Return ExtractedMetadata
|
||||
}
|
||||
|
||||
fn supported_types(&self) -> Vec<String> {
|
||||
vec!["heif".to_string()]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### ThumbnailGenerator
|
||||
|
||||
```rust
|
||||
#[async_trait]
|
||||
impl ThumbnailGenerator for HeifPlugin {
|
||||
async fn generate_thumbnail(
|
||||
&self,
|
||||
path: &PathBuf,
|
||||
output_path: &PathBuf,
|
||||
options: ThumbnailOptions,
|
||||
) -> PluginResult<ThumbnailInfo> {
|
||||
// 1. Decode HEIF image
|
||||
// 2. Resize to thumbnail dimensions
|
||||
// 3. Encode to output format
|
||||
// 4. Save to output_path
|
||||
// 5. Return ThumbnailInfo
|
||||
}
|
||||
|
||||
fn supported_types(&self) -> Vec<String> {
|
||||
vec!["heif".to_string()]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Dependencies
|
||||
|
||||
The plugin uses the following Rust crates (compiled to WASM):
|
||||
- `libheif-rs`: HEIF decoding and encoding
|
||||
- `image`: Image processing and thumbnail generation
|
||||
- `kamadak-exif`: EXIF metadata parsing
|
||||
|
||||
## Building
|
||||
|
||||
### Prerequisites
|
||||
|
||||
```bash
|
||||
# Install WASM target
|
||||
rustup target add wasm32-wasi
|
||||
|
||||
# Install wasm-tools for optimization (provides strip functionality)
|
||||
cargo install wasm-tools
|
||||
```
|
||||
|
||||
### Build Process
|
||||
|
||||
```bash
|
||||
# Build the plugin
|
||||
cargo build --target wasm32-wasi --release
|
||||
|
||||
# Strip debug symbols to reduce size
|
||||
wasm-tools strip target/wasm32-wasi/release/heif_support.wasm -o target/wasm32-wasi/release/heif_support.wasm
|
||||
|
||||
# Copy to plugin directory
|
||||
cp target/wasm32-wasi/release/heif_support.wasm .
|
||||
```
|
||||
|
||||
### Size Optimization
|
||||
|
||||
```bash
|
||||
# Use wasm-opt for further optimization
|
||||
wasm-opt -Oz heif_support.wasm -o heif_support_optimized.wasm
|
||||
```
|
||||
|
||||
## Installation
|
||||
|
||||
### Manual Installation
|
||||
|
||||
```bash
|
||||
# Copy plugin directory to Pinakes plugins directory
|
||||
cp -r examples/plugins/heif-support ~/.config/pinakes/plugins/
|
||||
```
|
||||
|
||||
### Via API
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:3000/api/v1/plugins/install \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"source": "/path/to/heif-support"}'
|
||||
```
|
||||
|
||||
### Via Plugin Manager
|
||||
|
||||
```bash
|
||||
pinakes plugin install /path/to/heif-support
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
The plugin can be configured through the `config` section in `plugin.toml`:
|
||||
|
||||
### EXIF Extraction
|
||||
- `extract_exif`: Enable EXIF metadata extraction (default: true)
|
||||
|
||||
### Thumbnail Generation
|
||||
- `generate_thumbnails`: Enable thumbnail generation (default: true)
|
||||
- `thumbnail_quality`: JPEG quality for thumbnails, 1-100 (default: 85)
|
||||
- `thumbnail_format`: Output format - "jpeg", "png", or "webp" (default: "jpeg")
|
||||
|
||||
### Resource Limits
|
||||
- `max_memory_mb`: Maximum memory the plugin can use in megabytes (default: 256, set in `[capabilities]`)
|
||||
- `max_width`: Maximum image width to process (default: 8192)
|
||||
- `max_height`: Maximum image height to process (default: 8192)
|
||||
|
||||
## Security
|
||||
|
||||
### Capabilities
|
||||
|
||||
- **Filesystem Read**: Only files being processed (via Pinakes)
|
||||
- **Filesystem Write**: Thumbnail directory only
|
||||
- **Network**: Disabled
|
||||
- **Environment**: No access
|
||||
|
||||
### Resource Limits
|
||||
|
||||
- **Memory**: 256 MB maximum
|
||||
- **CPU Time**: 30 seconds maximum per operation
|
||||
|
||||
### Sandboxing
|
||||
|
||||
The plugin runs in a WASM sandbox with:
|
||||
- No access to host filesystem beyond granted paths
|
||||
- No network access
|
||||
- No arbitrary code execution
|
||||
- Memory and CPU time limits enforced by runtime
|
||||
|
||||
## Performance
|
||||
|
||||
### Typical Performance
|
||||
|
||||
- **Metadata Extraction**: ~50-100ms for typical HEIF files
|
||||
- **Thumbnail Generation**: ~200-500ms depending on source image size
|
||||
- **Memory Usage**: 50-150 MB typical, 256 MB maximum
|
||||
|
||||
### Optimization Tips
|
||||
|
||||
1. Keep source images below 8192x8192 for best performance
|
||||
2. Use JPEG thumbnail format for smaller file sizes
|
||||
3. Adjust thumbnail quality vs file size tradeoff with `thumbnail_quality`
|
||||
|
||||
## Error Handling
|
||||
|
||||
The plugin handles:
|
||||
- **Corrupted Files**: Returns descriptive error
|
||||
- **Unsupported Variants**: Gracefully skips unsupported HEIF features
|
||||
- **Memory Limits**: Fails safely if image too large
|
||||
- **Timeout**: Returns error if processing exceeds CPU time limit
|
||||
|
||||
## Testing
|
||||
|
||||
```bash
|
||||
# Run unit tests
|
||||
cargo test
|
||||
|
||||
# Test with sample HEIF files
|
||||
cargo test --test integration -- --nocapture
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Plugin Fails to Load
|
||||
|
||||
- Check that `heif_support.wasm` exists in plugin directory
|
||||
- Verify `plugin.toml` is valid TOML
|
||||
- Check Pinakes logs for detailed error messages
|
||||
|
||||
### Thumbnails Not Generated
|
||||
|
||||
- Verify `generate_thumbnails = true` in config
|
||||
- Check filesystem write permissions for thumbnail directory
|
||||
- Ensure source image is below size limits
|
||||
|
||||
### Out of Memory Errors
|
||||
|
||||
- Reduce `max_width` and `max_height` in config
|
||||
- Increase `max_memory_mb` if source images are large
|
||||
- Check that source files aren't corrupted
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
- Support for HEIF image sequences (burst photos)
|
||||
- HDR metadata extraction
|
||||
- Live Photo support
|
||||
- AVIF format support (similar to HEIF)
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
Loading…
Add table
Add a link
Reference in a new issue