Rust/WASM + llama.cpp WebGPU

Edge inference smoke test

Run Qwen3.5 page extraction and EmbeddingGemma vectors locally in browser workers. Model bytes are downloaded directly by your browser and are not sent to this server.

Loading WASM runtime

Structured extraction

Qwen3.5 0.8B

Q4_K_M · 508 MiB

WebGPU generation uses a 65,536-token context with F16 KV cache via pinned wllama/llama.cpp. The Rust/Candle CPU fallback remains capped at 2,048 input tokens.

Not loaded

Results appear here.

Semantic retrieval

EmbeddingGemma 300M

Q4_0 · 265 MiB

This path implements bidirectional Gemma attention, mean pooling, both official dense projections, Matryoshka truncation, and L2 normalization in Candle. Use is subject to the Gemma terms.

Not loaded

Results appear here.