As noted in a recent episode of the HCLTech Trends and Insights podcast, the enterprise world is moving beyond proof-of-concept into real-world generative AI (GenAI) adoption. Govind Chandranani, HCLTech’s Practice Head of Engineering and R&D Services, observes that the market now has “huge expectations… on the role of GenAI for both business and service transformation.” Indeed, leaders across industries see promise in generative AI – but they also face new challenges.

Risks vary by sector. In consumer-facing industries, privacy looms large: studies find that 63% of consumers worry GenAI could “compromise an individual’s privacy by exposing personal data to breaches or… unauthorized access.” In retail or finance, that means consumers and regulators will scrutinize how data is gathered and used in AI models.

Chandranani highlights how this risk intersects with brand reputation: “These new chatbots are representing the company,” he explains, adding that “we’ve seen examples where somebody asked for a refund from an airline and the company had to say, ‘We will not provide what the chatbot committed.’” When AI makes promises on behalf of a brand, hallucinations become more than technical issues. They can become liabilities.

In healthcare, the stakes include patient safety and regulatory compliance. Experts warn that biased AI outputs could lead to inaccurate diagnoses or treatment recommendations and misdiagnoses. As health data is highly sensitive, providers must ensure GenAI tools meet strict standards, such as HIPAA and clinical validation.

Similarly, manufacturing or engineering organizations must weigh the cost of deploying powerful AI models against the need for explainability and safety. Chandranani notes that in verticals like manufacturing or healthcare, “the risk is cost and the accuracy of models, not just response accuracy, [but whether the model is doing what you want, based on policy].”

These industry risks are not hypothetical. For example, a recent US court case underscored the danger of unchecked AI hallucinations: attorneys who used ChatGPT ended up citing fictional case law in briefs, triggering sanctions. Cases like this highlight how even a small AI error can breach compliance or public trust, especially in regulated industries. Senior leaders must therefore view GenAI through the lens of their domain’s specific risks: from data privacy and IP in consumer apps to cost, explainability and safety in healthcare and manufacturing.

Rethinking QA: A new testing mindset

Unlike traditional software, GenAI outputs are not fully predictable. Even the same prompt can yield different valid responses each time. QA teams must abandon the old “one correct answer” mindset.

Chandranani reinforces this shift: “You have to create prompts to break the system,” he says. “You need more knowledge base to generate the right prompt which maps to Responsible AI, your domain functionality and your performance [expectations].” He adds that a fully manual approach will fall short: “GenAI testing cannot be totally manual. Otherwise, it will mislead the system.”

Rather than expecting rigid precision, testers should work in a confidence-based framework: tuning, evaluating and monitoring AI behavior much like they would a new employee. In practice, this means building iterative test suites and prompt libraries to probe the model’s responses, using metrics beyond simple accuracy, including factual consistency, bias scores and diversity of answers.

Chandranani emphasizes that “continuous validation is super important.” Without it, biased or outdated data will produce flawed results. Indeed, researchers note that traditional accuracy metrics “do not capture the vulnerability of LLMs to hallucination-inducing factors,” so novel evaluation criteria, such as hallucination rates and robustness to prompt changes, must be added. By consciously shifting from “exact output” to “acceptable range of output,” QA can help ensure GenAI systems behave reliably even as they learn or drift over time.

Pillars of trustworthy, Responsible AI

Risk management starts with design. Chandranani says that organizations should embed “trustworthy and Responsible AI” from the outset. This means upholding core pillars like transparency, accountability and explainability.

As NIST’s AI Risk Management Framework outlines, trustworthy AI systems should be “accountable and transparent, explainable and interpretable, [and] privacy-enhanced and fair.” In practical terms, that translates to fully documenting data sources and training processes, keeping audit logs of AI decisions and providing human-readable explanations for outputs where needed.

Chandranani underscores the real-world urgency: “Suppose you ask a chatbot, ‘Will I get a refund?’ and it says yes based on recent memory. That means your [system] isn’t accountable enough.” In a complex system, it’s important to demonstrate how the model arrives at results or responses.

For senior leaders, this also means establishing governance policies, including bias audits and ethics reviews, so that no system is treated as a black box. For instance, HCLTech’s own AI engineering platform, AI Force, now integrates input security scanners and output scanners to flag potentially unsafe or sensitive content before it reaches end users.

As Chandranani explains: “We created a system that can generate thousands of prompts very quickly and provide objective matrices, so every [validation engineer or] SME is thinking in a similar way. That improves objectivity, accuracy and [future] explainability.” This tool-assisted guardrail system shows how accountability and explainability can be operationalized: model recommendations are automatically vetted and any flagged issues are traceable to specific inputs.

Regulatory imperatives: Compliance vs. competitive advantage

Governments around the world are catching up with AI. The EU’s AI Act, for example, is “the first-ever legal framework on AI” and institutes strict, risk-based rules for deployment. High-risk AI (in areas like healthcare devices, employment screening or biometric ID) will require rigorous risk assessments, transparent data use, human oversight and proven accuracy. Meanwhile in the US, NIST has issued an AI Risk Management Framework (AI RMF) that, while voluntary, recommends that companies map, manage and measure AI risks around those same trust factors.

Compliance deadlines are looming, so leaders must proactively prepare. Chandranani notes that you have to see two points: “one, your internal policies; and second, your legal compliance, depending on your domain or location.” He adds that it’s not enough to build once and forget: “You have to govern on a continuous basis.” 

History shows being late to compliance can be costly: Italy recently fined OpenAI €15 million for privacy violations in ChatGPT. At the same time, companies are learning that early adoption of AI governance can be a strategic differentiator. An HCLTech study finds that while most organizations acknowledge the importance of Responsible AI, only 15% have mature frameworks in place to implement best practices, leaving few truly equipped to scale GenAI responsibly and turn compliance into competitive advantage. 

Cross-functional governance and KPIs

GenAI quality is not just a technical problem, it’s an organizational one. Successful deployments require cross-functional collaboration across business, legal, data and engineering teams.

“You train the model, tune it, test it, continuously monitor it and go back to the update phase. So many roles are involved. That’s why equal collaboration is needed,” says Chandranani.

Clear KPIs and metrics help bind these groups together. Traditional KPIs like uptime or latency must be supplemented with AI-specific indicators like hallucination rate, response coherence and explainability scores. “These become the communication between roles,” continues Chandranani. “If I create 1,000 prompts and map them to quantifiable matrices, that same data can be used by designers, developers and testers.”

By agreeing on these new “AI KPIs” up front, multi-disciplinary teams can monitor the health of the GenAI pipeline end-to-end – from data ingestion to model output to business impact.

Continuous monitoring and structured validation

Finally, deploying GenAI in production is not a one-off project but an ongoing process. Once live, models must be continuously monitored for drift, safety and performance.

HCLTech’s AI Force platform supports this. “We built [this system] to enable validation engineers, LLM Ops, developers and SMEs to get very objective data, [which leads to] improved accuracy, improved objectivity and better explainability,” says Chandranani.

The goal is a structured validation pipeline: before any model update is pushed, it must pass quality gates, such as accuracy tests, fairness audits and safety checks. Think of it as continuous integration/continuous deployment (CI/CD) for AI, with every code or data change going through a standardized review. This tool-supported approach not only scales QA efforts but also provides auditable evidence of due diligence.

Without compromise 

Realizing the promise of GenAI means balancing agility with rigor. “These are not just risks, they are challenges. And as an IT company, we need to address them for a good customer experience,” says Chandranani. 

By identifying industry-specific risks, adopting a new testing mindset, adhering to core ethical pillars, staying ahead of regulations and investing in cross-functional processes and monitoring tools, enterprises can harness GenAI safely and effectively. 

The result will be innovative AI-driven products that delight customers, without compromising trust or compliance.