Dharma AI Boosts GPU Utilization by 33 Percentage Points

Dharma AI has introduced a constraint-aware GPU allocator designed to optimize cluster scheduling. By replacing traditional First-In, First-Out (FIFO) scheduling with an algorithm that sequences jobs based on priority and workload shape, the company achieved significant efficiency gains on identical hardware. Across five benchmark scenarios under heavy contention, the allocator raised GPU utilization from a baseline band of 52% to 85% up to a range of 72% to 88%. In a training-heavy scenario utilizing eight GPUs, utilization jumped by 33 percentage points, from 53.6% to 87.0%, while priority-weighted output surged by 105%.

The system delivers these improvements without introducing significant latency. The allocator runs in just 1 to 2 milliseconds on the five contended scenarios, and takes only 15 milliseconds during a scale test of 30 jobs across 64 GPUs. In that scale test, while both FIFO and the new allocator achieved identical utilization of 44.9% and completed 27 of 30 jobs, Dharma AI's system delivered 15.9% more priority-weighted value. Even in a uniform-priority test where priority signals were removed, the allocator boosted utilization from 76.8% to 87.5% and increased value by 23.1%, proving that smart placement planning alone yields major benefits.

For practitioners, the allocator solves the friction between elastic real-time inference and rigid batch jobs like training or quantization. Instead of reserving a flat block of GPUs all day to meet peak real-time demand, the allocator treats real-time demand as a dynamic curve, filling the troughs with batch jobs. It enforces five structural constraints, such as ensuring batch jobs occupy contiguous GPU blocks sized to a power of two and preventing running jobs from being preempted. The scheduler optimizes a rolling 24-hour horizon but only commits the current timestep, re-running every 30 to 60 minutes to absorb forecasting errors.

To feed this allocator, Dharma AI uses specialized estimators rather than a single generic predictor. For instance, its training forecaster evaluates 22 features across 10 distinct training variants to predict duration and GPU needs. This orchestration framework allows machine learning engineers to reclaim idle standby capacity and ensure high-priority tasks are not stuck behind low-value workloads.