On-Device VLM Inference Optimization

2.05× faster inference · Oct–Dec 2025

Optimized Qwen3-VL-2B inference on Snapdragon 8 Elite, achieving a 2.05× speedup over the project baseline. Configured llama.cpp with importance-matrix-based Q4_0 quantization, memory locking (mlock), and CPU affinity for on-device inference. Fine-tuned with LoRA for visual question answering, evaluating both task quality and inference performance.