The Linux Foundation Projects
Skip to main content
search
BlogNews

One Year of OPEA: Lowering GenAI adoption barriers for enterprise

By April 8, 2025No Comments

By Dr. Malini Bhandaru, OPEA Technical Steering Committee Chair, and Intel, Senior Principal Engineer & Rachel Roumeliotis, OPEA Ecosystem & Community Manager and Intel, Director, Open Source Strategy

A year ago, it felt like each week brought an exciting new AI model that promised to change how we do, well, everything. Models still feel like magic: creating art, solving math problems, generating code. But, moving from simple chatbots to enterprise-level generative AI (GenAI) solutions that unlock the full promise of AI felt like a giant leap. Intel’s 2024 Open Source Community Survey revealed that the top GenAI adoption barriers were scalability and performance challenges (37%) and a shortage of AI talent (37%). Could we lower those barriers? Help democratize AI? This is typically where open source comes to the rescue!

It all began in March 2024, as an aspirational internal whitepaper at Intel that aimed to capture all the key components necessary for an enterprise to be truly successful with GenAI. The paper covered the importance of being able to choose which software and hardware components are used to create solutions. It highlighted the need for standards around evaluation—not just for functionality, but also performance and accuracy. An effective enterprise-level GenAI solution would also need to be able to run in both development mode and at scale, and provide customizable end-to-end reference implementations for common use cases. Most importantly, the solution had to be open source and broadly available for true democratization to encourage the rich community collaborations that bring together our best minds across the technology industry.

A Year Later…

The white paper has evolved into the Open Platform For Enterprise AI (OPEA), an Apache 2.0-licensed open source project housed under the Linux AI & Data Foundation, with over 50 community partners. A year of firsts has culminated in a code base that supports multiple GenAI use cases across a variety of hardware platforms.

Some of our favorite big moments: 

OPEA Building Blocks

OPEA has  six sub-repositories, each playing a crucial role in achieving the goals of democratizing GenAI and providing scalable, enterprise-ready solutions.

  1. GenAIExamples is filled with end-to-end examples for common use cases such as a Chatbot (95% usage given its broad applicability to health care, legal services, product literature), Document Summarization (to cope with information glut),  AgenticAI, Code Generation, Code Translation, and Language Translation (increasingly global engagements), to name a few. Each of these can be run in either Docker or Kubernetes, scaled on Kubernetes, and run on your choice of cloud or on-prem, on bare metal or virtual machines, using hardware from Intel, AMD, or Nvidia. Just change an environment variable to use a different embedder or LLM to experiment and determine the best for your use case. 
  2. GenAIComps: A typical GenAI solution consists of multiple components, such as an embedder, a vector database, a model server, guardrails, and more that are chained together. For each component type, often there are multiple alternatives with their own pros and cons. OPEA supports three model serving frameworks, namely vLLM, TGI, and Ollama; three popular component chaining mechanisms, namely LangChain, LlamaIndex, and Haystack; and at least 10 vector database implementations, including graph support, with solutions differing in the amount of data they can handle, whether they auto update on underlying data changes, and more.
  3. GenAIInfra provides scripts and templates to launch workloads on popular cloud offerings such as AWS, GCP, IBM, and Intel® Tiber™ AI Cloud, and Helm charts to further ease deployment of the GenAI examples. It also contains instructions on how to integrate with authentication and authorization systems, rate limiting gateways, and more—each important to delivering an enterprise-grade solution.
  4. GenAIEval houses benchmark scripts in addition to leveraging industry best practices to evaluate accuracy. Production solutions have to comprehend cost and meet criteria such as latency, throughput, and accuracy. These scripts help us develop model deployment profiles to deliver low latency or high throughput solutions, taking us from proof-of-concept solutions to production.
  5. GenAIStudio: A low-code/no-code interface that reduces adoption barriers even further. It supports dragging and dropping components and chaining them together then, with a click of a button, building, packaging, and deploying these using Docker or Kubernetes onto your choice of infrastructure. 
  6. Docs provides crisp documentation on how to use the various OPEA components, while making few assumptions about the reader’s background and expertise to deliver a complete set of instructions. This is a living set of documents, of course, and, as such, we have tooling in place to check for link validity, perform nightly builds and publish to our doc .io site, which is constantly evolving.

Together, these repositories form a cohesive framework that empowers developers to create, scale, and deploy GenAI solutions efficiently, moving us closer to the goal of making AI practical and accessible for enterprises.

Partnerships

Strong partnerships are key to OPEA’s current and future success, enabling us to tackle enterprise AI challenges with innovative solutions and meaningful integrations.

Retrieval augmented generation (RAG), for example, reduces hallucinations and increases response relevancy in generative AI (GenAI), making vector databases a critical ingredient in AI pipelines. Recognizing this, multiple vector database vendors, including Pathway and Qdrant, have joined forces with OPEA to make their offerings first-class citizens within the platform. Similarly, Prediction Guard has integrated guardrail services into OPEA that improve response quality by addressing bias, repetition, politeness, and other factors.

OPEA’s multi-vendor approach is further strengthened by AMD’s contributions, which have enabled ROCm-based hardware-tested solutions for six GenAI examples already (AudioQnA, ChatQnA, DocSum, FaqGen, CodeTrans, and CodeGen). Partners like LlamaIndex and deepset (Haystack) have added flexibility for end-users to customize their GenAI pipelines, while AWS has brought OPEA integrations to its marketplace. Nutanix and NetApp are among the latest partners, acknowledging OPEA’s potential to simplify GenAI adoption across cloud hosting and storage domains.  

These collaborations exemplify how OPEA relies on active partnerships to deliver scalable, enterprise-ready solutions. As the project continues to grow, the contributions of engaged partners will remain critical in driving innovation, expanding adoption, and ensuring the platform meets the evolving needs of enterprises worldwide.

Powered by OPEA

A project gains adoption if it addresses some pain point or fills a gap. OPEA’s customizable GenAI examples, its curated list of GenAI components that can be mixed and matched, and its evolving end-to-end benchmark suites are what make it valuable as a one-stop-GenAI-shop.

ISTE + ASCD, to reduce costs while still maintaining security, resiliency, and performance, adopted OPEA and hosted inference services running in Intel® Tiber™ AI Cloud. The United Nations Innovation Unit (OICT) is also using OPEA as the cornerstone of its “AI Sovereign stack.”  

H3C, in partnership with Intel, unveiled its AIGC LinSeer integrated machine: a full-stack enterprise AI solution built with Intel® Xeon®, Intel® Gaudi®  2, and OPEA. Tailored for sectors like healthcare, energy, and education, the system delivers high-accuracy inference with rapid response capabilities and support for complex workflows. 

Dell recently launched a turnkey GenAI solution using the Dell PowerEdge XE9680 platform and Intel Gaudi 3 accelerators, all powered by OPEA. It demonstrates OPEA’s value in enterprise-grade infrastructure environments.

With these collaborations in place, we’re poised to explore even greater possibilities and innovations in the coming year.

What’s Next

Rapid innovation defines AI today, whether it’s more capable models, better model serving frameworks, or more efficient inference. OPEA will be integrating with AIBrix and KubeAI to reduce token latency and increase throughput, constantly integrate and test new models, publish hardware optimized images, provide more AI agents, and improve documentation. As innovation in AI accelerates, OPEA is paving the way for responsible and impactful advancements in enterprise AI.

We invite the community to learn more about OPEA, use it, and help enhance it through your contributions. Learn about and influence the OPEA project through the OPEA Developer Experience, Evaluation, and Security working groups. Be sure to stop by an OPEA community event this year—next up is our one-year-anniversary series of events, OPEA Week (April 14-18)! Join the effort to unlock the potential of GenAI, safely and responsibly, leveraging the best of open source, to create together something even more valuable and easy to use.

Close Menu