kernelopt
I rewrote ballform's RF-DETR ball detector in MLX and replaced the slow parts with custom Metal kernels. It's 1.8× faster than ONNX Runtime + Core ML on my M3 Pro (50.9 ms to 28.2 ms per 1080p frame) and still passes every accuracy check.
kernelopt /
Faster ball detection
1.8×faster per framesame detections
fp32 passes all 147 reference detection checks on 11 images. fp16 is faster but fails 3 of them, so it's opt-in.
what each change saved
Each row adds to the one above. The deformable attention row is approximate (about 30 ms).
preprocessing · 4.7ms to 0.42ms
The Mac build of OpenCV hands these resizes to Arm's KleidiCV library, which rounds differently from OpenCV's own code. The kernel reproduces both exactly, because even a one-bit change in the input can reorder which objects the detector keeps.
Measured on 2026-09-24 on an otherwise idle M3 Pro (18-core GPU, 36 GB). Medians of 60 warm calls, from a uint8 frame in to boxes and scores out.
Two detectors on two threads went from 22.8 to 40.0 calls per second, with all 600 concurrent calls matching single-threaded output exactly.
i profiled first and only wrote kernels where the profile said to: a fused residual + layernorm, a deformable attention sampler, an fp32 matrix multiply with gelu built in, and one kernel that does all the preprocessing. the backbone's matrix multiplies now run at 4.8–5.5 tflops, close to the most mlx gets out of this gpu in fp32, so the rest of the wins would have to come from fusing more work into them. fp16 would be faster, but the model ranks 1600 proposals and keeps the top 300, and fp16 rounding is enough to reshuffle about a third of them, so it stays opt-in. ballform uses this as its detector.