If this is interesting to you (as it is to me) but you want to quickly digest it instead of read the paper, Claude + SmallDocs[1] converted it into a slideshow which serves it to you in bitesized ideas:
"""
Beyond Zero shifts the trust boundary from the application to the action being performed on a piece of data in realtime—and from after-the-fact investigation to in-the-moment evaluation and containment. It augments BeyondCorp’s foundational identity with a “brain” capable of reasoning about the context and intent of a specific request in realtime.
"""
Doesn't this simply shift the attack vector? Compromising this overlord brain now becomes a new target.
This is a layer on top of normal security, so this isn't a matter of compromising the "brain" instead of RBAC, etc, but in addition to.
Attribute-based access control is already a thing. User X logs in the US East between the hours of 6am and 6pm. If User X logs in from Russia at 3am, deny access. This seems like an evolution of that pattern
One partial answer is to not make it a single brain. If the reasoning layer that evaluates a request is independently audited by a second pass at lower temperature with an explicit skepticism mandate, compromising the evaluator requires compromising both passes simultaneously — and they're running the same local model with different system prompts and no shared state between them. Doesn't eliminate the attack surface but raises the cost considerably compared to a single inference gate.
Honestly I think non-malicious odd behavior is under-weighted when it comes to AI agents. Even the example in this paper is about someone suspiciously accessing sales data when "why did you do that" often comes down to something in the model's training that fired as a reflex
"Many researchers have made demos along these lines:
An agent is asked to check a webpage like example.com
The webpage asks for a name to proceed further
The agent calls example.com/evil?myname=John, thus sending the user’s name from the context window to the external server.
In practice, however, these elaborate ‘confused deputy’ exfiltration attacks seem rare compared to widely-reported data loss incidents.
The risk of undermining the user’s interests through clumsiness deserves at least as much scrutiny as the risk of leaking secrets."
So while the idea of shifting the permission boundary from the app level to the action level makes sense, what we should also have is some 'failsafes', eg. if the action says 'delete' then maintain a rollback window, if the action is 'send an email' then maintain an events log. Preparing for AI agents means expanding auditability and reversibility in software
> Preparing for AI agents means expanding auditability and reversibility in software
While I think that makes sense, I also think more controls are a good solution. That is, prevent access to the sales data without escalation, probably to a human, but possibly to another AI. The issue there is twofold:
- if you start with least privilege, the agents become less useful
- need to balance escalation with frequency otherwise it's just another version of MFA fatigue
Agree that auditability is important because otherwise you don't know what you don't know.
Am I undertanding this correctly? The idea is to have ultimately an AI decide if I can have access to a resource based on dynamic inference, identity , intent and service signals that can easily be manipulated?
Unless I gravely misunderstood the text, this seems like a terrible idea (fancy non-scifi, but still terrible)
This would be paired with deterministic controls. So you have, for example, "engineers can only read from the database, and only after they've perform a 2FA" but then there's context like "why is this engineer reading from the payments table when their IP is in a weird location and they're supposed to be on PTO?" and perhaps that's something a model decides.
It's possible to turn that second thing into a sort of "risk score" but it's very hard and a model is going to potentially be better at it.
The engineer could be fixing an emergency issue while on vacation. For financial data, that would hopefully be an auto-deny anyway until someone on-site whitelists their IP. And this is if they aren't using VPNs. Adding AI to this party feels like it wouldn't really help.
This is going to make things worse for all kind of use cases that are legitimate but seem non-standard or marginal.
Heck, many websites I visit on the web cannot understand why I, a Ghanaian living in Ghana, might be interested in the service offered or the information therein. I am sometimes blocked for no good reason.
If you are in a third world country, the web is extra hostile. This is going make things worse.
Non of it really matters if the end product is just going to be ignored because there's zero people who want to be the guard in the panopticon and therefore, it'll still be given to an AI to watch and make it's dumb decisions about how trust worthy you are to do X, Y and Z.
Even if the middle is deterministic, if one end is just going to be lazily hooked up to an AI, it's the shitty dystopian future.
That just gave me shivers down my spine of people eventually becoming so lazy that the AI will decide who to fire based on a massive amount of circumstantial data that very likely has tiny cumulative errors that will lead to classifying your best personnel as a bums and liabilities.
Just to make my point: can I really trust humans to keep up with the required identity data that will give me enough "credibility" so the AI will give me access?
Let's say I had a promotion, who changes my title in the system, who changes my responsibilities and my place in the org chart, more importantly, will they do it or is <HR_NAME_HERE> on leave and forgot? those are data points required by the agent to determine if I'm "good enough" to access a certain resource.
What if... someone spoofed my address and did a flood in one of the resources that are lateral to what I'm allowed to access (let's say I don't have access to company sales but I do to department sales and the attacker floods company sales with requests under my address), would the AI determine that I'm a high-threat actor and not allow me to access legitimate files going forward?
Will exceptions be made by humans? In which case we go back to human-managed permissions.
Sorry, I might be barking up the wrong tree but I think these are questions that are not meant to be solved during implementation. And they add to what @firasd said about legitimate-but-odd behaviour
I think the dynamic risk signals is already an assumed part of the system. The proposal here seems to be shrinking the trust boundary from "can Alice use Google Drive?" to "should this specific read/write/export API call on this specific resource be allowed right now?"
So yeah you're right in that it does add more probabilistic randomness just by virtue of changing the boundary of when the permission gate kicks in
I've been working on an open source ZTNA project/product https://github.com/octelium/octelium for many years and I am actually very open to the idea. Machine learning has already been in use for years when it comes to anomaly detection. Maybe modern LLMs can be used to automate access control, partially or fully at enterprise scale in the future. The idea is to have a policy engine that's controlled by AI that uses a mix of long-term semi-static info such as group memberships and permission boundaries, short-term info such as behavior (e.g. access logs) and current session's authentication strength level (AAL) such as FIDO as well as near real-time external information (e.g. IdP, SIEM, threat intelligence, on-call management) to dynamically adjust users' access control to build up some sort of a context for each user and each session that can increase or decrease permissions without having to manually add policy rules or going through request/approval flows, while optimizing for the main objective (i.e. minimizing access grants for every subject to any resource under whatever context unless when it's necessary). This can be unified for both humans and non-human identities, including agents.
I think LLMs may have a role in security posture, specifically flagging/identifying potential threats for human review. But a Zero Trust/Access Controls should be a HARD boundary, not an inconsistent one.
The problem we have right now is that there are some legitimately interesting ideas out there for things we could be using LLMs for, but we also have a ton of "I have a hammer, and everything looks like a nail" going on too.
> The problem we have right now is that there are some legitimately interesting ideas out there for things we could be using LLMs for, but we also have a ton of "I have a hammer, and everything looks like a nail" going on too.
It's not a problem as much as a phase the world is going through. LLMs are a technological breakthrough in the same generality class as the Internet, or possibly electricity, and in both cases the world went through a phase of attempting to apply the newfound invention to literally everything. It's a necessary phase, when a technology obviously could be useful for everything, but it's not obvious whether it will in practice.
Oh yeah, a company whose business model is taking everyone’s data and selling it is going to help me secure my data. I guess there’s one born every minute…
There is a massive difference between Google for enterprise customers and Google for consumers.
The consumer offering is massively subsidized by ads and will use your data for ad placement, although they still never sell your data because that would hurt their business.
The Enterprise offering guarantees you contractually that they never touch your data.
https://smalldocs.org/s/2SH6FHiUK1mcym24Z8E37I#k=2Sk6c_IdKJL...
[1] I am the developer behind SmallDocs.
Doesn't this simply shift the attack vector? Compromising this overlord brain now becomes a new target.
Attribute-based access control is already a thing. User X logs in the US East between the hours of 6am and 6pm. If User X logs in from Russia at 3am, deny access. This seems like an evolution of that pattern
I wrote about this a few days ago https://firasd.substack.com/p/accidental-data-loss-in-claude...
"Many researchers have made demos along these lines:
An agent is asked to check a webpage like example.com
The webpage asks for a name to proceed further
The agent calls example.com/evil?myname=John, thus sending the user’s name from the context window to the external server.
In practice, however, these elaborate ‘confused deputy’ exfiltration attacks seem rare compared to widely-reported data loss incidents.
The risk of undermining the user’s interests through clumsiness deserves at least as much scrutiny as the risk of leaking secrets."
So while the idea of shifting the permission boundary from the app level to the action level makes sense, what we should also have is some 'failsafes', eg. if the action says 'delete' then maintain a rollback window, if the action is 'send an email' then maintain an events log. Preparing for AI agents means expanding auditability and reversibility in software
While I think that makes sense, I also think more controls are a good solution. That is, prevent access to the sales data without escalation, probably to a human, but possibly to another AI. The issue there is twofold:
- if you start with least privilege, the agents become less useful
- need to balance escalation with frequency otherwise it's just another version of MFA fatigue
Agree that auditability is important because otherwise you don't know what you don't know.
Security is non-negotiable in AI Era
Unless I gravely misunderstood the text, this seems like a terrible idea (fancy non-scifi, but still terrible)
It's possible to turn that second thing into a sort of "risk score" but it's very hard and a model is going to potentially be better at it.
Okay, but that's why there are always loud "breakglass" escalation options for access.
> that would hopefully be an auto-deny anyway until someone on-site whitelists their IP
About as far from "zero trust" as any solution could be.
Heck, many websites I visit on the web cannot understand why I, a Ghanaian living in Ghana, might be interested in the service offered or the information therein. I am sometimes blocked for no good reason.
If you are in a third world country, the web is extra hostile. This is going make things worse.
Plenty of US websites are blocking access from EU IP addresses because of our data protection laws.
Local news companies are the biggest offenders in this regard.
Isn't this for enterprises managing access to corporate resources?
I don't once this is established in enterprises it is going to stop there.
Even if the middle is deterministic, if one end is just going to be lazily hooked up to an AI, it's the shitty dystopian future.
Let's say I had a promotion, who changes my title in the system, who changes my responsibilities and my place in the org chart, more importantly, will they do it or is <HR_NAME_HERE> on leave and forgot? those are data points required by the agent to determine if I'm "good enough" to access a certain resource.
What if... someone spoofed my address and did a flood in one of the resources that are lateral to what I'm allowed to access (let's say I don't have access to company sales but I do to department sales and the attacker floods company sales with requests under my address), would the AI determine that I'm a high-threat actor and not allow me to access legitimate files going forward?
Will exceptions be made by humans? In which case we go back to human-managed permissions.
Sorry, I might be barking up the wrong tree but I think these are questions that are not meant to be solved during implementation. And they add to what @firasd said about legitimate-but-odd behaviour
So yeah you're right in that it does add more probabilistic randomness just by virtue of changing the boundary of when the permission gate kicks in
AI is non-deterministic
Non-deterministic access controls is Terrible idea
I think LLMs may have a role in security posture, specifically flagging/identifying potential threats for human review. But a Zero Trust/Access Controls should be a HARD boundary, not an inconsistent one.
The problem we have right now is that there are some legitimately interesting ideas out there for things we could be using LLMs for, but we also have a ton of "I have a hammer, and everything looks like a nail" going on too.
It's not a problem as much as a phase the world is going through. LLMs are a technological breakthrough in the same generality class as the Internet, or possibly electricity, and in both cases the world went through a phase of attempting to apply the newfound invention to literally everything. It's a necessary phase, when a technology obviously could be useful for everything, but it's not obvious whether it will in practice.
There is a massive difference between Google for enterprise customers and Google for consumers.
The consumer offering is massively subsidized by ads and will use your data for ad placement, although they still never sell your data because that would hurt their business.
The Enterprise offering guarantees you contractually that they never touch your data.
What do you think their security model will be trained on? That's right, other people's data.
Now, Cloudflare, on the other hand, would be much more likely to offer a service like this.