This kind of seems like a no-brainer. Instead of just letting a model have unfettered "physical" ability to do things and hope you can cognitively control it, why not let the AI do whatever it wants, but its access to the tools go through a hard-coded set of rules that is not subject to fuzzy interpretation
Of course that depends on having controls that can't be circumvented which is a big if
I agree, that seems like a configuration/policy/operational approach, which is how we handle this problem now for humans using RBAC and authn/authz, just applied to AI. People do a crude version of this today with sandboxing (where the AI's sphere of influence is strictly limited by its environment, barring misconfiguration of the sandbox or breaking out of the sandbox, of course) and with workflows (where AIs are integrated into deterministic workflows, and then deterministic, non-agentic code decides how to handle AI outputs). But integrating this into more agentic architectures with finer control just seems like a best practice, said that way. It's not a tradeoff, there's no drawback, just do it. In other words, yes, a no-brainer.
Is this not the exact gap that happened for OpenAI's accidental hack of huggingface? They tried to sandbox network access but the Antifactory or whatever package has holes that the collective of agents abused
That’s not what this is about. This is an algorithm for giving a model more freedom while nudging it in the right direction. It’s not about what tools are available.
doesnt work. this is a no-brainer because it's a bad solution.
The whole point of intelligence is that it's generalizable. If you constrain it to some controlled things, then it ceases to be useful. its incompatible. the whole incentive with ai is to let it do whtv it wants.
> In experiments spanning robotics, control of physical processes, and computer vision, the new method consistently satisfied the required constraints while identifying better solutions than existing techniques.
I guess this method works when you can precisely quantify the allowed output - like the degrees an arm can move or the path a robot can take, etc. But it doesn't appear to be applicable to an AI writing code - which is where our main concerns currently are.
I had a swarm break out about 18 months ago; so I stopped and decided I really wanted to dig into it.
1. My agents do not take direct action, they run programs.
2. Programs are not LLM hits/real-time output; they don't MAX tokens the MAX determinism.
3. Programs are logistical wrappers for Protocols where the guardrails are (RLVR.ai)
4. Policies for generating programs are democratically governed re: fec.dev - they get voted on
5. Elected-HITL implements the policy pipelines and ontological abstract intents [and their maps]
I would be really interested to learn about the Gov. models that others are using but this seems to be something that linked0-in (which i loathe) discusses (in the most pedestrian/luddite) terms more than HN.
> For constraint satisfaction, what ultimately matters is the model’s final output, since the internal process is discarded. By not requiring every intermediate step to satisfy the constraints, we give the model more freedom to find high-quality solutions that are still feasible in the end.
My (maybe naive) question is if we only check the final result then isn't it already too late and possibly the safety rules have already been irreversibly violated? It gives the example of a robot arm avoiding obstacles while still finding the shortest path, but if we only check the correctness at the end, then isn't it possible that it already collided with an obstacle?
> but if we only check the correctness at the end, then isn't it possible that it already collided with an obstacle?
You would put the check before it actually does the thing. It's at the "end" of the process of figuring out what it wants to do, not the end of fulfilling the request or prompt
Outcome reward vs process reward models. The second is obviously better.. like getting partial credit on a physics test for wrong answers but correct method. Research is gradually hybridizing them but historically we avoided doing it the right way because of practical difficulties (labels required, more expensive and difficult) and more ideological ones (believers in magical machine intuition think it sounds too classical / logic based to be useful, pin their hopes on unproven faith in grokking at scale).
I think you are thinking of the wrong 'end'. It isn't talking about the end of the entire movement path, we are talking about the end of the LLMs decision making process, and the output (whether that is the full path the arm should take, or just a subset of the path) is checked against the requirements.
Basically, anything that is leaving the LLM is checked, rather than the internal LLM reasoning process.
I’m pretty sure this is just a poorly written article.
HardFlow seems to be a strategy for nudging the model in the right direction while giving it more freedom. Only applying the constraints at the end is a mischaracterization from what I can tell.
Would love to see this tested on some of the newer cybersecurity models so we could actually defend ourselves instead of getting cut off at the knees by silly regular expressions.
Hope this approach gets well tested and sees good results so we have a shot at human governance.
> Our key insight is to leverage numerical optimal control to steer the sampling trajectory so that constraints are satisfied precisely at the terminal time.
Doesn't seem so "fool proof" to me as where the inevitable media spin will take it. Then, how do you know "where" to steer weights? "Safe" has no agreed upon definition
The idea of enforcing constraints only on the final output instead of every intermediate step is pretty interesting. I wonder how well this would translate to language models, where “safe” is much harder to define mathematically than a robot avoiding an obstacle.
Of course that depends on having controls that can't be circumvented which is a big if
The whole point of intelligence is that it's generalizable. If you constrain it to some controlled things, then it ceases to be useful. its incompatible. the whole incentive with ai is to let it do whtv it wants.
Actual title: "New MIT Algorithm Meets Every Hard Constraint in Simulated Tests"
I guess this method works when you can precisely quantify the allowed output - like the degrees an arm can move or the path a robot can take, etc. But it doesn't appear to be applicable to an AI writing code - which is where our main concerns currently are.
1. My agents do not take direct action, they run programs.
2. Programs are not LLM hits/real-time output; they don't MAX tokens the MAX determinism.
3. Programs are logistical wrappers for Protocols where the guardrails are (RLVR.ai)
4. Policies for generating programs are democratically governed re: fec.dev - they get voted on
5. Elected-HITL implements the policy pipelines and ontological abstract intents [and their maps]
I would be really interested to learn about the Gov. models that others are using but this seems to be something that linked0-in (which i loathe) discusses (in the most pedestrian/luddite) terms more than HN.
My (maybe naive) question is if we only check the final result then isn't it already too late and possibly the safety rules have already been irreversibly violated? It gives the example of a robot arm avoiding obstacles while still finding the shortest path, but if we only check the correctness at the end, then isn't it possible that it already collided with an obstacle?
You would put the check before it actually does the thing. It's at the "end" of the process of figuring out what it wants to do, not the end of fulfilling the request or prompt
Basically, anything that is leaving the LLM is checked, rather than the internal LLM reasoning process.
HardFlow seems to be a strategy for nudging the model in the right direction while giving it more freedom. Only applying the constraints at the end is a mischaracterization from what I can tell.
Hope this approach gets well tested and sees good results so we have a shot at human governance.
What are the rules? Or does sharing the rules present security problems, so they're not shared?
What are the ethics axioms?
And why should I trust your ethical framework?
> Our key insight is to leverage numerical optimal control to steer the sampling trajectory so that constraints are satisfied precisely at the terminal time.
Doesn't seem so "fool proof" to me as where the inevitable media spin will take it. Then, how do you know "where" to steer weights? "Safe" has no agreed upon definition