30% of GenAI projects fail after POC: Kellton Point of View on Gartner’s prediction

In an analysis, Gartner specialist Rita Sallam forecasted that 30% of Generative Artificial Intelligence (GAI) initiatives will be discarded following Proof of Concept (POC) by the conclusion of 2025 owing to subpar information quality, insufficient risk safeguards, mounting expenses, or ambiguous commercial worth.

Gartner firmly believes leaders are eager to witness returns on GAI expenditures, yet entities grapple to demonstrate and achieve value. As the breadth of endeavors expands, the monetary strain of creating and implementing GAI frameworks is increasingly experienced.

A significant hurdle for entities emerges in rationalizing the considerable investment in GenAI projects for efficiency enhancement, which can be challenging to convert directly into monetary advantage, as Gartner said.

Numerous entities utilize Generative Artificial Intelligence to revolutionize their operational paradigms and generate novel business prospects. Nevertheless, these execution strategies come with considerable expenses, spanning from $5 million to $20 million.

Regrettably, no universal solution exists with GAI, and expenses are less foreseeable than alternative technologies. What enterprises expend, the applications they fund, and the execution strategies they adopt - all determine the outlays. Whether they're a market innovator and desire to infuse AI comprehensively or overly emphasize productivity gains or augment existing procedures, each carries different expense levels, hazards, variability, and strategic influence.

Gartner's investigation suggests that generative artificial intelligence necessitates more acceptance of indirect, prospective financial investment criteria than immediate return on expenditure (ROE). Traditionally, many Chief Financial Officers have yet to be at ease with investing presently for indirect worth in the future. This hesitation can bias investment distribution to tactical versus strategic outcomes.

Surviving the GenAI projects shakeout: Kellton’s POV on Gartner’s prediction

Many businesses are eager to harness its power as Generative AI (GenAI) for their next projects because GenAI is revolutionizing industries with its robust capabilities. However, the high costs associated with GenAI projects can be daunting, especially for smaller organizations. Here’s Kellton point of view on how businesses can successfully navigate GenAI project costs without breaking the bank :

• 1. Leveraging open-source and free models

One of the most effective ways to reduce GenAI project costs is by tapping into the wealth of open-source resources and free models available. Let's dive deeper into the options:

Open-source APIs

Many powerful GenAI tools are available through open-source APIs. These can significantly reduce development costs while still providing robust capabilities. Some notable examples include:

• Hugging face transformers: This library provides thousands of pre-trained models to perform tasks on texts, such as classification, information extraction, question answering, summarization, translation, and text generation.
• TensorFlow hub: Offers a repository of trained machine learning models ready for fine-tuning and deployment.
• PyTorch hub: Similar to TensorFlow Hub, it provides pre-trained models that can be easily integrated into projects.

Microsoft Azure with SLA

Microsoft offers access to some of its AI models through Azure, with Service Level Agreements (SLAs) that can provide peace of mind for businesses relying on these services. Key offerings include:

• Azure cognitive services: Provides AI models for vision, speech, language, and decision-making tasks.
• Azure Machine Learning: Allows businesses to build, train, and deploy machine learning models.

The SLAs ensure a certain level of uptime and performance, which can be crucial for businesses relying on these services for critical operations.

Meta's LLaMA 3.2

Facebook's parent company, Meta, recently released LLaMA 3.3 as an open-source large language model. It's trained on different parameters and can be fine-tuned for specific use cases, offering a cost-effective alternative to proprietary models. Key features include:

• Multiple model sizes: LLaMA 3.2 is available in four model sizes ranging from lightweight models like 1B and 3B to the larger vision-enabled models starting from 11B, and 90B — catering to different levels of computational needs.
• Fine-tuning capabilities: The model supports fast fine-tuning due to its architectural advancements, enabling product engineering developers to respond swifty to the base model without significant overhead.
• Commercial use allowed: In comparison to other LLMs, LLaMA 3.2 is open to commercial use for everyone, making it an ideal choice for businesses seeking to integrate advanced AI features into their applications to support customization and flexibility in deployment.

2. Utilize efficient LLM

Large Language Models (LLMs) are at the heart of many GenAI applications. Opting for more efficient models can dramatically reduce implementation costs:

OpenAI GPT-4o-Mini

This smaller version of the popular OpenAI model offers similar capabilities at a fraction of the computational cost. Benefits include:

• Reduced inference time: Faster response times for real-time applications.
• Lower memory requirements: Can run on less powerful hardware, reducing infrastructure costs.
• Comparable performance: It performs nearly as well as its larger counterpart for many tasks.

Claude 3

Known for its efficiency, Anthropic Claude 3 provides a good balance between ethical AI performance and operational cost. Benefits include:

• Optimized for chat: Well-suited for conversational AI and large-scale deployments.
• Lower token costs: A much cost-efficient option in particular for large-scale conversational AI applications wherein processing of high volumes of text is involved.
• Fast inference: Highly optimized for quick response times with real-time support for applications, chatbots and customer service, with lower latency.

Mistral

Mistral is a preferred choice in the GenAI era as it enables businesses to reap premium benefits like rapid inference and efficient performance in more lightweight settings. Benefits include:

• Lower token costs: It has smaller parameter like the 7B model that offers efficient token pricing, making it ideal choice for cost-conscious businesses.
• Fast Inference: This LLM model excel at fast inference times and performs well in resource-constrained environments with zero compromise on rapid text generation.
• Performance: Mistral offers impressive performance in terms of language understanding, generation, and code-related tasks. It strikes a balance between computational efficiency and high accuracy.

3. Embrace cloud infrastructure

Cloud computing offers a flexible and cost-effective way to implement GenAI projects:

AWS and Azure options

Major cloud providers like Amazon Web Services (AWS) and Microsoft Azure offer robust AI and machine learning services. These platforms allow businesses to set up and run AI models without significant upfront investment in hardware. Key services include:

AWS:

• Amazon SageMaker: Provides tools to build, train, and deploy machine learning models quickly.
• AWS Lambda: Allows running code without provisioning servers, which is ideal for serverless AI applications.
• Amazon Comprehend: Offers natural language processing capabilities for text analysis.
• AWS Bedrock: A fully managed platform by Amazon for developing generative AI-driven applications.

Azure:

• Azure Machine Learning: Provides a cloud-based environment for training, deploying, and managing ML models.
• Azure Cognitive Services: Offers pre-built AI models for various tasks.
• Azure Databricks: Provides a unified analytics platform for big data and machine learning.

One of the most significant advantages of cloud-based AI solutions is the flexibility in pricing. Businesses can opt for pay-as-you-go models to scale their usage (and costs) based on project requirements. They can use AI services without significant capital expenditures and quickly scale resources up or down based on demand.

By leveraging cloud infrastructure, businesses can avoid many technical challenges associated with setting up and maintaining AI systems. This reduces costs and allows teams to focus on their core competencies rather than infrastructure management.

What’s more, cloud providers handle automated updates and maintenance, ensuring that system updates and security patches are handled without manual intervention.
Also, due to the high availability and redundancy built into the infrastructure, businesses are always operational, even during disruptions.

And how can we forget the biggest advantage—the global reach of cloud platforms that allows businesses to deploy GenAI support across multiple geographic regions, enhancing performance while meeting local compliance requirements?

Avoid Post-POC abandonment pitfalls of GenAI projects: Quick cost-saving strategies

Here’s a more detailed breakdown of each cost-saving strategy to avoid post-POC abandonment:

1. Align POC goals with business objectives

Why it matters: A POC often fails to transition to production if its goals are disconnected from business needs. To prevent this, ensure that the POC addresses a pressing issue or opportunity for the company.
How to do it: Before starting the POC, collaborate with business leaders and stakeholders to understand their challenges. Define how the POC will solve these problems and align it with strategic goals, such as improving efficiency, reducing costs, or enhancing customer satisfaction.
Outcome: When the POC is tied to business goals, it’s easier to justify further investment and move the project into production, saving time and costs on unnecessary pivots.

2. Involve stakeholders early

• Why it matters: Stakeholders, including decision-makers, business users, and technical teams, need to buy into the POC from the beginning. Post-POC abandonment often occurs when there’s a lack of support or understanding among these groups.
• How to do it: Invite relevant stakeholders to be part of the POC process early. Provide regular updates, invite feedback, and demonstrate how the POC is progressing towards solving business issues. This creates a sense of ownership among stakeholders and reduces resistance when it’s time to scale.
• Outcome: Early stakeholder engagement builds trust and ensures the resources and support needed to take the POC into production.

3. Start with a scalable solution

• Why it matters: A common pitfall is selecting tools or platforms for the POC that don’t scale well to a full production environment. Rebuilding or switching solutions post-POC can be costly and time-consuming.
• How to do it: When choosing technologies for your POC, assess their ability to scale. Consider factors such as integrating existing systems, future growth, and flexibility to accommodate increased demand.
• Outcome: A scalable solution helps transition smoothly from POC to full deployment without costly infrastructure changes or additional investments.

4. Establish clear success metrics

• Why it matters: Without measurable success criteria, a POC might be perceived as inconclusive or unsuccessful, leading to its abandonment. Clear metrics help demonstrate its value and guide decisions on future investments.
• How to do it: Before starting the POC, define quantifiable KPIs (Key Performance Indicators). These could include time saved, costs reduced, or revenue increased. Ensure all stakeholders agree on these metrics.
• Outcome: With clear success metrics, you can easily show the POC’s impact, making it easier to secure buy-in for scaling the solution.

5. Limit scope to essentials

• Why it matters: Trying to achieve too much in the POC phase can lead to delays, bloated costs, and increased complexity, which could fail. Focusing on core functionalities minimizes these risks.
• How to do it: Stick to the essentials during the POC. Aim to prove a single use case or functionality demonstrating the solution’s potential. Avoid adding features that aren’t critical at this stage.
• Outcome: A streamlined POC allows for faster, more cost-effective execution, making it easier to demonstrate value early and reducing the chances of abandonment due to over-complication.

6. Ensure POC knowledge transfer

• Why it matters: Post-POC abandonment often happens when knowledge is siloed or lost during the transition to production. Ensuring all insights and technical details are transferred helps avoid costly delays and mistakes.
• How to do it: Document every step of the POC process, from challenges faced to solutions developed. Ensure that the team involved in the POC is also involved in the transition or that they properly train other teams.
• Outcome: Proper knowledge transfer enables a seamless transition from POC to production, avoiding costly rework, miscommunication, or the need for new hires to fill knowledge gaps.

These strategies help prevent POC abandonment and avoid unnecessary expenses while positioning the project for a smooth transition to production.

Conclusion

Implementing GenAI projects doesn't have to be prohibitively expensive. By leveraging open-source resources, choosing efficient models, utilizing cloud infrastructure, and adopting additional cost-saving strategies, businesses of all sizes can tap into GenAI's power without breaking the bank.

Remember, aligning your GenAI strategy with your specific business needs and budget constraints is key. Start small, prove value, and then scale your efforts as you see returns on your investment. With these comprehensive strategies, even smaller organizations can compete in the AI-driven marketplace, using GenAI to drive innovation and growth.

As AI evolves rapidly, stay informed about new developments and continuously reassess your approach to ensure you're maximizing the value of your GenAI projects while keeping costs under control.