Our own LLM engine

Hypothesis 2 · milestone 1 · experimental

Qwen2.5 0.5B Instruct (4-bit GGUF, 429 MB) running on an inference engine written for this project: hand-written WebGPU kernels for the GPU, and C compiled to WebAssembly with SIMD and threads for the CPU. No ML libraries. The model file is cached after the first download. This is an experiment, tested on one laptop (GTX 1050, Chrome on Windows): expect rough edges elsewhere.