FRAMEWORK
A risk-based framework for moving generative AI beyond proof of concepts to production
- Explore a risk-based framework for deploying generative AI in production, balancing governance, trust, and innovation to unlock its full transformational potential.
Artificial Intelligence companies with eye-watering valuations. Super scalars begging NVIDIA for GPU chips. Reopening of Three-Mile Island, which no one could have ever predicted even a year ago. We are truly at the top of the hype curve. Multi-trillion-dollar bets are being made right now (Deep Seek or not), but will organizations realize the transformational value of generative AI?
The minute large enterprises decide they don’t see the promise and pull back on investment, it could all come crashing down. This cycle isn’t new: we saw it with the dot-com boom and bust and even with early predictions about cloud computing. While some fortunes were made, many were lost. Could generative AI follow the same trajectory?
The roadblock to generative AI adoption
This is the pivotal question everyone is trying to answer. Despite the hype, we are seeing some concerning signs right now. Many proof-of-concept (POC) projects have delivered compelling results by feeding contextual data into large language models (LLMs). RAG is in the BAG. AWS, Azure, and others are building it into the infrastructure. Let the production business applications flow with increased productivity as far as the eye can see.
Except… why is no one putting these solutions into production?
There have been innumerable headlines about this: “Companies Are Struggling to Move GenAI Projects into Production,” “GenAI not production-ready,” and “Gartner Predicts 30% of Generative AI projects will be Abandoned after proof of concept by End of 2025.” The problem is clear: production adoption lags. Major cloud platforms now provide implementation investment funds to only those workloads being put into production, no longer for POCs.
This hesitation stems from a crisis of trust. Gen AI is non-deterministic. We all see this when we ask ChatGPT a question and get an authoritative, well-reasoned, thoughtful answer that is completely wrong. Customer trust in your business is everything. If you lose that, then losing your business is not far behind. Leaders know this and are understandably putting the brakes on truly implementing generative AI solutions until they can ensure safe and reliable outcomes.
So, how do we get out of this situation?
Learning from past technology deployments
We have seen this before. Technology is fantastic at turbocharging your business until you run into a ditch. For example, e-commerce platforms were being deployed fast and furious until someone figured out that you can inject a script that runs an SQL query to dump your customer credit card numbers. Yes, accidents happen, and then you put guardrails in to keep you out of that ditch the next time.
We are not spending enough time on gen AI governance, and unfortunately, the way governance is currently being implemented can be overly constrictive in some areas and not constrictive enough in others. One size does not fit all. Publicly available gen AI governance models like NIST are a very heavyweight set of processes and procedures that may be appropriate and can certainly mitigate risk. Unfortunately, if these governance frameworks are not calibrated with the specific use cases they apply to, they may present a barrier to putting anything into production.
We are not paying enough attention to the risk categories and understanding the risk profile of each specific generative AI workload. The solution lies in tailored governance, which aligns controls with each AI application’s specific risks and potential harms.
A framework for generative AI governance
Inspired by biocontainment levels in biology, consider a part of our generative AI risk framework. This provides a structured approach to governance and matches controls to potential risks. By applying the right-sized governance to specific use cases, organizations can accelerate safe deployment.
Essentially, if you understand the potential risk well, you can have flexible governance based on that specific risk profile. This is one way an organization can move faster and put lower-level categories of items into production with less risk to the organization.
This approach is similar to Anthropic’s Responsible Scaling Policy (RSP), which biocontainment labs also inspired. https://www.anthropic.com/news/announcing-our-updated-responsible-scaling-policy
Anthropic’s approach focuses on the model itself and puts guardrails around certain capabilities, limiting knowledge that could create harm like bio and nuclear weapons. It arguably deals with capabilities several years out. Our calibrated approach is really focused on the potential risks of use cases leaders are assessing today. Adopting these strategies can provide a path to production for many Gen AI transformational applications currently relegated to eternal Proof-of-Concept status.
Level 1
Minimal risk, high opportunity
Level 1 category workloads are low risk and a great place to start for organizations new to working with generative AI workloads before you may fully flesh out your governance procedures. Types of risk drivers are categorized into data sensitivity, access, and capabilities. This somewhat mirrors data classification frameworks used in data governance.
Level 1 has publicly available data, so if it loses containment, it is not a big deal because that information is already available on the public internet. For example, Pariveda worked with a health insurance provider using a RAG strategy to feed an LLM public plan available on their website. This information was used to augment a model helping CSRs answer member questions specific to those members’ plans.
Access refers to who is accessing the specific Gen AI system. For Level 1, we assume you only have specially trained internal employees accessing the model. If there are hallucinations, those employees are ready for those eventualities. Employees can also provide feedback on the responses to help improve the model over time.
Lastly, capabilities refer to how independent the model is. Level 1 is standard RAG chat-type request and response interfaces that are well understood with no autonomy.
Level 2
Low risk, more complexity
Level 2 presents a bit more risk. Now, the data may be protected, as well as contain internal organizational information that may not have legal privacy requirements, but is not something you would want to put out on the public internet.
With access, you may have a much wider set of employees accessing the model; still, people are trained. However, you want your employees to trust it, so you would want to ensure you have good testing and guardrails to prevent hallucinations.
Capabilities could be a bit more robust. An example would be a model analyzing maintenance notes for a manufacturing system (provided it isn’t something that could cause real harm if it failed). If it is wrong, it could be impactful but not catastrophic.
Level 3
Moderate risk, customer impact
Level 3 starts to get more interesting. Some data could have legal or reputational risk implications if it were to get out. Customer or sensitive product data might be a good example.
Access entails an end customer starting to have direct access to the model without employee oversight. You may have heard about a successful lawsuit against a large airline that deployed a chatbot on their website. In this instance, the model misled a customer by providing an answer that was inconsistent with the airline’s standard policy. What they didn’t understand, plan, or test for was that they had deployed a Level 3 model (direct customer access) and didn’t know it.
Capabilities could include some Agentic or autonomous behavior, which gives applications more ability to take actions that are not expressly controlled by human oversight. However, these capabilities are still limited where harm is minimal. For example, making the wrong dinner reservation could be very inconvenient, but it is not the end of the world.
Level 4
High risk, maximum oversight
Level 4 is where the rubber really meets the road. Data is very sensitive, such as PHI health records, and sensitive PII data, such as credit cards, social security numbers, or strategic product designs.
End customers could have access to the model. Based on the model’s output, they could make dangerous decisions, such as something related to their health.
Capabilities could include highly agentic behavior, where the model seeks goals and agendas without human oversight. An example could be a virtual nurse agent that interfaces with patients, providing health advice, and periodically checking in with these patients to ensure they are following doctors’ orders.
Building trust and unlocking value
Generative AI governance must align with business priorities, risk tolerance, and operational realities. Based on your organization’s specific risk tolerance and the domain of the specific use cases, you could tailor this framework to be more aligned with your organization.
The industry is beginning to recognize the importance of proportional, risk-aligned governance. This framework and Anthropic’s Responsible Scaling Policy both show how critical it is to implement frameworks to manage the polarity between AI risks and enabling the transformational AI innovation we see on the horizon.
Deploying higher classification models into production requires more effort, translating to higher costs to ensure they do not create harm. So, consider starting with lower-risk use cases classified as Levels 1 or 2. You can move much faster for these less risky models, putting them out into your production environments, interfacing with customers (potentially through intermediaries) to test and learn, creating value for your organization more quickly.
Based on the difficulty we have all seen deploying Gen AI solutions into production, it is clear that a one-size-fits-all approach has not worked. By classifying workloads based on risk levels and tailoring the governance process accordingly, your organization can deploy generative AI solutions safely and effectively. Focusing on incremental progress, we can move past the endless POC phase and begin realizing the true value of Generative AI.
Case Studies
Explore our success stories
Want to talk?
Looking️ for️ a️ team️ to️ help️ you️ solve️ a️ complex️ problem?️
INSIGHTS
Article
