- H2O.ai's Acceptable AI Use Policy sorts data into three tiers (public, confidential and restricted), and anything not yet classified is treated as restricted.
- Confidential information such as customer names, roadmaps, financials and source code goes only into H2O.ai's own models or approved models in a restricted tenant, never into a public chatbot.
- Every unattended AI agent at H2O.ai needs a named owner, its own least-privilege service account, human approval before high-risk actions, hard spending caps and a kill switch.
- An audit of roughly 325 apps connected to H2O.ai's Google Workspace blocked more than 140 and restricted another 150, leaving 30 trusted enterprise apps.
- Before signing with an AI vendor, ask whether your data is retained or used for training, where its servers and sub-processors are, and whether it has a SOC 2 Type II report.
We all know data is key to making AI solutions successful, but do you think about where it goes once it enters the prompt or predictive model?
As Head of IT at H2O.ai, I see that question from both sides. We build AI products, and we rely on AI every day to run the company. This year, we rewrote our internal Acceptable AI Use Policy. The most important lesson is that responsible AI is mostly ordinary governance applied to a new kind of tool. What follows is the approach we put in place, and what I would recommend to any leadership team undertaking the same work.
The six principles behind it
Responsible AI covers a lot of ground. We organize it into six areas:
- Explainable: people affected by a model's output can understand why it produced that result.
- Interpretable: when the stakes are high, use models whose logic you can inspect directly.
- Fair: test for bias before a model makes decisions about people, and keep testing after launch.
- Secure: treat models as attack surfaces. Prompt injection and model poisoning deserve the same attention as any other software vulnerability.
- Human-centered: AI drafts, people decide.
- Compliant: meet the privacy laws where you operate, such as GDPR and CCPA, along with whatever your customer contracts require.
Keep sensitive work on AI you control
The simplest way to protect data is to keep it inside your own environment. That's the principle behind sovereign AI: your models, data and results stay on infrastructure you control, whether that's a private cloud, on-premises, or fully air-gapped.
Our policy puts this into practice. Confidential information, such as customer names, roadmaps, financials and source code, can only go into our own models or approved models hosted in an H2O.ai restricted tenant with no outside network access. It never goes into a public chatbot.
Two of our products deliver the same protection for customers. H2O Driverless AI builds predictive models with explainability built in, so teams can show a regulator how a decision was reached. Enterprise h2oGPTe runs large language models privately, including fully air-gapped, so prompts containing customer or financial data are never used to train someone else's model. The National Institutes of Health runs it this way, fully air-gapped, assisting 8,000 employees across 28 institutes.
Four rules that do most of the work
To keep it simple for our makers, we can boil our policy down to four simple rules:
- Classify data before you pick a tool. We use three tiers:
- Public material can go into any approved tool.
- Confidential material stays in private models, or in enterprise tools where we've verified the vendor doesn't train on it.
- Restricted data, like sensitive personal information or board material, is off limits without written sign-off from Legal and Security.
- If someone isn't sure which tier applies, we ask that they treat it as restricted.
- Keep one list of approved tools. Our Approved AI Inventory is the final word on what each tool is allowed to touch. Personal genAI accounts are not permitted for business use, even for a quick task.
- Treat every output as a draft. A person reviews and approves anything that affects proprietary source code, contracts, money or security.
- Know where the data lives. If a vendor can't show where its servers and sub-processors are, through a SOC 2 Type II report or its data processing terms, we don't approve it.
Every agent needs an owner
Agents raise the stakes because they take actions on their own. We register any agent that runs unattended, acts for a team, connects several systems or deals with outside parties. Each one has to have:
- A named person who is accountable for its setup, its access and what it does.
- Its own service account with only the permissions it needs. Never a borrowed login.
- A human approval step before high-risk actions, such as writing to production databases or the ERP, moving money, emailing customers, or changing security settings.
- Hard caps on how often it runs, how many API calls it makes and how much it can spend, plus a documented kill switch IT can use at any time.
- Testing against edge cases and prompt injection before it reaches production.
Applying the policy: auditing connected apps
A policy is only as effective as your visibility into what people actually use. Earlier this year, we reviewed every third-party app connected to our Google Workspace environment. There were roughly 325 of them. Many were AI tools that employees had connected on their own, often with broad access to email, files and calendars.
We did not begin by blocking everything. A blanket block would have broken integrations the business depends on and pushed people toward workarounds. Instead, we worked in a deliberate order:
- We first trusted the apps that critical systems rely on, such as our identity provider's provisioning integration, so that tighter controls would not disrupt operations.
- We enabled a request-access workflow, giving employees a clear way to ask for a tool rather than a reason to go around the controls.
- We introduced a middle tier that lets an app use Google sign-in without any access to company data.
- Finally, we blocked the apps that had no business justification.
The outcome was more than 140 apps blocked outright and another 150 restricted in the Google data they can access. Only the 30 enterprise grade apps that are core to our business were “trusted” in Google Workspace. Just as important, we now have a current record of what is connected and why, which feeds directly into our Approved AI Inventory.
Questions to ask any outside AI vendor
Third-party AI tools can deliver real value. However, information entered into them may be retained by the provider, used for training, or turn up in litigation, which can jeopardize legal privilege or trade secret protection. Before signing new agreements, executives should ask:
- Will our data be retained or used to train your models, and how can we verify the opt-out?
- Where are your servers, storage and sub-processors? Can we see your SOC 2 Type II report?
- Do any sub-processors or data-labeling teams operate in high-risk jurisdictions?
- How do you defend against prompt injection, and what vulnerabilities have you disclosed?
- Do you support SSO, and can we control the AI features at the admin level?
Where to start
Organizations do not need an extensive policy to begin. Classify data into tiers, publish a short list of approved tools, and assign an accountable owner to every agent. That covers most of the risk. From there, move the most sensitive work onto AI that you operate yourself.
To see this approach running on your own infrastructure, from private cloud to fully air-gapped, start with Enterprise h2oGPTe.
:quality(75))