AI infrastructure conversations often begin with a familiar question: How many GPUs do we need? While it's an important question, it's not always the first one enterprises should ask.

A recent industry discussion on right-sizing GPU infrastructure highlights a real problem: undersized infrastructure can hurt performance, while oversized GPU estates can leave expensive capacity idle. It also emphasizes the need to distinguish between training, batch workloads and online inference, because each has different latency, throughput and utilization characteristics.

But for many enterprises, the more strategic question is broader: Which workloads need GPUs, which can run efficiently on CPUs and which should move closer to the edge?

That shift in thinking is central to making AI affordable.

From GPU-first to workload-first AI

GPUs are indispensable for many AI workloads, especially large-scale model training, high-concurrency inference, large-context GenAI and workloads that demand high memory bandwidth. A GPU-first strategy, however, can become expensive when every AI use case is treated as if it requires the same class of accelerator.

Enterprise AI is not one workload. It's a portfolio. It includes retrieval-augmented generation, embeddings, summarization, classification, recommendation engines, classical machine learning, computer vision, anomaly detection, document intelligence, AI copilots, agentic workflows and data preprocessing. Some of these need large GPU clusters. Many don't.

The affordable AI principle is simple: use GPUs where they deliver unique value and use Intel Xeon-based compute where it delivers the right balance of performance, cost, availability, security and operational simplicity.

Intel has continued to invest in CPU-based AI performance. Intel reported that Xeon 6 processors with Performance-cores achieved ~1.9x improvement in average AI inference performance compared to 5th Gen Intel Xeon across six MLPerf benchmarks and up to 17x improvement on BERT versus 3rd Gen Xeon submissions over four years. Intel’s 5th Gen Xeon platform also includes AI acceleration in every core, targeting end-to-end AI workloads before customers need to add discrete accelerators.

Why CPUs matter in enterprise AI

The CPU is already the control plane of enterprise IT. It runs the operating system, networking, security, data movement, application logic, orchestration, preprocessing and postprocessing that surround AI models. In many real-world AI applications, those surrounding tasks are not peripheral; they're the workload.

This is especially true for production AI. After all, a chatbot is more than token generation; a RAG solution is more than a language model; a computer vision system is more than inference. These solutions also include data pipelines, vector search, policy checks, API calls, access controls, monitoring, logging, guardrails, workflow integration and lifecycle operations.

That is where Intel Xeon-based infrastructure becomes highly relevant. With Intel Advanced Matrix Extensions, AI acceleration is built directly into the processor cores, enabling CPU-based inference for workloads where the economics and operational model are better than deploying discrete GPUs. AWS demonstrated CPU-based AI inference acceleration of up to 76% using Intel AMX on Amazon EC2 8th-generation instances and that CPU inference can be suitable when cost, operational complexity and infrastructure compatibility are key factors.

For enterprises, this can mean a more practical AI adoption path: start with existing x86 infrastructure patterns, optimize models through quantization and software acceleration and reserve GPUs for the workloads that truly require them.

The overlooked economics of affordable AI

Right-sizing GPUs can reduce waste. But affordable AI requires optimizing the full cost stack:

• Infrastructure cost: Not every workload should consume scarce GPU capacity
• Utilization: GPUs deliver value when they are highly utilized. Many enterprise inference workloads are bursty, intermittent or latency-sensitive rather than continuously saturated
• Operational complexity: GPU clusters introduce additional requirements around scheduling, drivers, networking, cooling, capacity planning and specialized operations
• Data movement: Moving data to centralized GPU infrastructure can add latency, bandwidth cost and privacy exposure
• Security and compliance: Regulated workloads may benefit from keeping data closer to where it is generated or processed
• Energy and sustainability: The most powerful accelerator is not always the most efficient choice for every task

A CPU-inclusive strategy helps enterprises avoid a false binary: either buy expensive GPU capacity or delay AI adoption. Instead, they can build a tiered AI architecture that matches workload requirements to the right compute layer.

The HCLTech and Intel perspective: AI should be placed, not just sized

HCLTech and Intel bring a long-standing collaboration across cloud, AI, edge, digital workplace and infrastructure modernization. HCLTech’s Intel partnership spans more than 30 years, with dedicated engineering, customer experience labs and AI and cloud native labs supporting enterprise transformation.

This matters because enterprise AI cannot be solved by hardware alone. It requires assessment, architecture, modernization, platform engineering, governance and managed operations.

The HCLTech-Intel AI approach is framed around five principles:

1. CPU-first where practical, GPU where necessary

   Start by classifying workloads. Lightweight and medium inference, embeddings, classification, summarization, data preprocessing, rules-heavy agent workflows and many computer vision use cases may run effectively on Intel Xeon or Intel AI-enabled endpoints. Large-model training, high-concurrency LLM serving and large-context workloads may require GPUs.

   The goal is not to replace GPUs. The goal is to protect GPU investments by using them only where they create differentiated value.

2. Optimize the model before expanding the hardware

   Many enterprises overspend because they scale infrastructure before optimizing models. Quantization, pruning, batching, caching, model distillation and runtime optimization can significantly reduce compute requirements.

   Intel’s software ecosystem, including OpenVINO and oneAPI-based optimizations, supports this shift toward efficient inference across Intel hardware. HCLTech emphasizes model conversion, INT8 quantization, pruning and multi-engine scheduling as part of scalable edge inference patterns.

3. Bring AI closer to the business process

   Affordable AI is not only about the data center. Some use cases become more economical when inference happens closer to the endpoint, plant, branch, workstation or edge location.

   HCLTech has positioned Intel-powered AI endpoint solutions to run AI tasks locally on endpoint devices, improving performance, latency, privacy and security by reducing the need to send data to remote servers. 

   This is especially relevant in manufacturing, retail, healthcare, banking and field operations, where low latency, data privacy and operational resilience are critical.

4. Build a hybrid AI architecture

   The future of enterprise AI is not CPU-only or GPU-only. It's a hybrid.

   A well-designed AI architecture may include:

   • Intel Xeon processors for inference, orchestration, preprocessing, classical ML, application logic and secure enterprise workloads
   • GPUs for large-scale training, fine-tuning and high-throughput model serving
   • AI PCs and edge endpoints for local inference and user-context-aware productivity
   • Cloud, private cloud and on-prem platforms based on data sovereignty, latency and compliance needs

   HCLTech’s cognitive infrastructure work reflects this direction, including CPU-based inferencing, GPU consolidation, workload optimization, private AI and hybrid deployment patterns.

5. Measure cost per outcome, not cost per accelerator

   The unit of value in AI is not the number of GPUs deployed, but the business outcome per dollar, per watt and per unit of operational complexity.

   For a claims summarization engine, that may mean cost per document processed. For an industrial vision system, it may mean cost per inspection line. For a coding assistant, it may mean cost per developer workflow. For a service desk copilot, it may mean cost per resolved ticket.

   This reframes AI infrastructure decisions around measurable value rather than hardware preference.

A practical enterprise AI placement model

Enterprises can use the following decision model:

Affordable AI is an architecture discipline

The next phase of AI adoption will be defined less by who owns the largest GPU cluster and more by who can industrialize AI efficiently.

That means matching workloads to the right silicon, optimizing models before scaling infrastructure, moving inference closer to where decisions happen and creating a managed operating model that enterprises can trust.

Right-sizing GPUs is a valuable practice. But it is only one part of the affordable AI equation. The bigger opportunity is right-placing AI compute across CPU, GPU, edge and cloud environments. With Intel Xeon processors, built-in AI acceleration, optimized software frameworks and HCLTech’s enterprise AI Engineering and operations capabilities, organizations can move from AI experimentation to AI at scale—with cost, control and confidence.

Affordable AI is not about choosing CPUs over GPUs. It's about choosing the right compute for the right workload, every time.