Core42 has published a new whitepaper arguing that enterprises need to rethink how they measure the cost of artificial intelligence as organisations move from pilot projects to production-scale deployments.
Titled Efficiency Wins the Inference Era, the whitepaper examines how businesses can scale AI while maintaining control over cost, performance and data sovereignty. According to Core42, the focus should shift from the price of individual AI tokens to the overall efficiency of inference, the process through which a trained model generates a response each time it is used.
The paper argues that evaluating AI based solely on token pricing provides an incomplete picture of production costs. Instead, organisations should measure “tokens per second per dollar”, assessing how much useful output a system delivers, how much it costs to generate, and how quickly it reaches users.
Core42 notes that inference costs continue to grow as generative AI becomes embedded in daily business operations. Larger prompts, richer context windows, expectations for lower latency and the emergence of agentic AI systems, where a single request can trigger multiple model calls, are all contributing to increased compute demand.
The whitepaper also argues against running every AI workload on the same model and infrastructure. Instead, it advocates routing workloads according to their specific performance requirements. Core42 said its Compass platform follows this approach by directing workloads to the most appropriate combination of models, accelerators and deployment paths across silicon from NVIDIA, AMD, Qualcomm, and Cerebras.
According to the whitepaper, Compass operates with a 99.5 percent availability commitment, serves more than seven million API requests and processes well over 100 billion tokens each week across production workloads. Its fastest Cerebras inference path delivers throughput improvements of up to 20 times.
Beyond performance, Core42 argues that efficiency, governance and sovereignty should be managed together, combining cost visibility and budget controls with access controls, encryption, in-country data residency, zero logging of customer data and more than 170 security policies.
Discussion about this post