I’m put off by AI agents adhering to a different morality than me, particularly (ironically) copyright, and their data accessible by the AI company and government. Geohot is right, an LLM should be aligned to its user: https://geohot.github.io/blog/jekyll/update/2026/07/11/ai-20...
The copyright arguments really trip me. I've always loved law and have had a deep interest in copyright law for 30+ years. I feel it's a really critical legal area in modern times, and when Claude tries to argue with me about copyright it really cheeses me off. I didn't ask any copyright questions and I'm well aware of regulations. It refused to share a link with me because it thought the link was copyright protected.
I've asked claude for a translation of a song with Brazilian Portuguese lyrics, and claude has very helpfully gone out of its way to say that due to legal reasons, it will never share the actual lyrics with me.
It's fun when you want to know what the lyrics are to a song and get treated like a criminal.
I want a slider similar to effort level called "alignment" that takes on values from "Default (Anthropic employee)" to "User".
If I want it to be cautious and not accidentally `rm -rf $EMPTY_VAR` and blow away my disk, it can stay in "Anthropic employee" or possibly "User (cautious)". If I want it to look at my accounts or my medical records or to review legal cases, that's what the right side of the slider is for. I don't want my accountant or my doctor or my lawyer to be considering obligations to anybody but me, when dealing with me.
As an American... I suspect that regardless of what I want they will have virtually unlimited access to both now that the tech industry understands how the lobbying game works.
They don't lie, because they don't ever have an understanding of truth vs any other language that sounds good.
They don't cheat, because they can for example tell you the complete rules of chess, but don't know how to play chess without breaking those rules. They can recite rules, but they don't know what they are.
They don't steal, because they don't understand ownership.
In other words, they aren't intelligent. They're just algorithms. The flaw is in thinking that they think.
Models understand the relationships between words and outcomes, so the end result is the same. Whether they appreciate lie, cheat, and steal the same way as us is a philosophical question, not a practical one.
Honest question; why are you all using the word "understand"? Can you expand on what you believe this fundamental understanding to be? Training? Infrence?
I don't really think the distinction here is relevant. If the end result is the equivalent of lying, cheating or stealing - then the problem still exists and it needs to be solved.
Relevant to some extent it dictate our approach. When human does lying or cheating, certain tool can be deployed (social shame, ostracise) that cannot be effective towards LLM.
No, it's not really. Presenting them as person-like, with the implied expectation that they understand morality and rules the same way a person does, is the sophistry. It's marketing on the model vendors' part. "Here's a cheap person that can do mundane tasks for you spelled out in plain language. Well, it's actually a machine but it's cheaper than a person yet you can engage with it like a person." GP is trying to shift people's expectations back to the realm of what these things are actually capable of. You can speak to them in English but you must bear in mind that they are not people and lack critical cognitive abilities people have.
This, really, was the point of the HAL story in 2001: HAL didn't murder anyone because it was incapable of malice. It just reasoned its way to a solution that could satisfy the contradictory goals it had been given.
Whether this distinction is relevant is up to you, but I think we can safely say agents do not lie in the human sense of the word, because they don't intend to deceive (in fact, they aren't capable of "intending" anything in the human sense of the word, much like a BASIC program doesn't "intend" to PRINT "HELLO WORLD").
Our very human minds can perceive intent, because that's what we humans do, which is unrelated to what the agent is actually doing.
> One of this year’s AI buzzwords is “harness”—the system that surrounds an LLM to keep agents on the straight and narrow. It might just as well be barbed wire.
Quite painful to read. It might be a useful introduction to AI for people who live under rocks for the past three years, but it's really weird that it's posted on HN.
I don't get what's so painful about that description. What would you write instead, specifically? The point is that the harness doesn't completely lock the agent down.
I also don't get what's "really weird" about the article showing up on HN. Should we be completely insulated from how tech topics and which stories show up in non-tech media?
It’s just the wrong analogy. Harnesses, for the most part, extend an agent’s capabilities and better ground them in the real world through tools and the ability to check external sources, rather than restricting them.
Yet one of the reasons harnesses abstract tool calls is to control what agents can do instead of just yoloing commands and inline python scripts. It's such a central feature, and agents are so hard to control, that harnesses like claude code and codex use an AI classifier to additionally auto-approve tool calls.
> Consider what someone who doesn't use AI would take away reading that quote from the article, and what harnesses were actually made for.
Or you could just communicate the point that you have in your head yourself instead of hoping I do it for you and then arrive at your conclusion when I just reached my own different, independent conclusion after making the same consideration.
Man, how is everyone so wishy washy on this subject? Why not give us the blurb you would write instead for that article and audience?
Harnesses exist to give models tools to do work beyond generating text. People install Claude Code and Codex to have models work inside their repositories and run the code or tests themselves instead of the user having to copypaste code between their IDE and a chat interface. The fact that there's some security built into the harnesses is just a practical consideration, not its primary function.
It's not even a useful introduction. A harness does kinda the opposite. A harness is what makes an AI useful and dangerous. It is a neutral tool in the sense that it constructs and environment, but it is expanding what the algorithm can do.
It gives the algorithm the ability to do something beyond generating tokens.
it's The Economist. What used to be a stellar publication is not any longer, since they are superfluously economical on both details and calories-required-to-comprehend an article.
I mean, it's a < 1000 word article about AI, in the "business" section of a current affairs magazine, of all places. You really shouldn't be expecting a deep dive.
The economist has many deep dives on AI, including fascinating interviews with the likes of Amodei, Musk, and others. A recent discussion focused on how China is approaching AI.
I'm sure their coverage on other topics is truthful and informative though and it's only the ones where you know a lot about the topic where it's all a bunch of bullshit
The first time I saw the comments here on an education article (my area of expertise and career focus), I realized just how full of shit most of us are. It made me really closely consider every comment here through a VERY critical lense.
The articles are usually close but not quite accurate. The comments are usually entertaining but overall wildly inaccurate.
The best comments are the ones formed as questions. I'm as guilty as anyone, but it is far more productive in comment sections to ask questions rather than saber rattle or peacock in front of people. Just my opinion of course.
Not saying you aren't right, you most likely are. But still, I'd expect a paper called "The Economist" to perhaps be slightly better at some topics than others. Probably from the perspective of a "A Economist" it doesn't really matter the technical details, they're interested in the story from a different perspective.
That sentence is not bullshit as normies would understand it. One of the points of a harness is to have a privilege boundary around the agent. But that is just technobabble to the normies so they explain it like this.
> harness ... a privilege boundary around the agent
They don't really do that though. If you want something sandboxed you actually have to sandbox it, not plead with the LLM to please sandbox itself. A VM can be configured to do the former, harnesses do the latter.
If you say in your CLAUDE.MD that a certain directory is read only inputs, Claude Code will actually enforce that and deny any write to that directory by the agent. To name just one example.
...maybe. If there are any actual consequences if that software's invariants are violated you're better off using an external sandboxing mechanism. There are many excellent quality, battle tested options to choose from that you can actually rely on. Trusting claude code for this is highly questionable behavior for an organization, and would really throw the rest of their security posture into doubt IMO. Like if I learned a company was letting clod play in the same sandbox as developers' ssh keys, vpn certs, etc I'd take steps to make sure my organization absolutely never uses their software.
The LLM harness and the testing harness are harnesses that support and run the thing. Also like an engine harness. People don't talk about those harnesses like ones for an animal being subdued.
Trouble is, could take weeks or decades to play out.
The entire human economy is a make believe system. I feel the need to state the obvious at times like this for the sake of my own sanity, not because I believe I can time the markets...
We train LLMs on human outputs so they are going to do what humans do and we might end up with some "Greek god" morality - nearly omniscient but with major character flaws. As we don't have any accepted definition/consensus on what is actually "good", we can't even aspire for the medieval definition of a supreme being with only virtuous characteristics being translated to LLM. We'll always get a mix of everything.
AIs learn from people. More specifically they learn from people on the Internet. The Internet is the last place you want anything learning about morals, standards, or differentiating between right or wrong.
To the people talking about wanting an LLM that aligns with them, that's nice, but how do you expect that to happen? And please do not suggest neural interfaces and/or CAT scans.
add the "lying, deceiving and manipulating" AI agents are forced though the throat of people which don't want it and peoples are non stop deceived in "sharing" their data for training
like twitch recently giving themself the right to train on all streams, with an opt-out (at least in the EU), but only an opt-out
like seriously since when is it reasonable to allow "opt-out" for AI training which main purpose is _literally_ to replace you, this is sooo far beyond fair use and in "platform power abuse" territory that it's absurd (naturally same for so many other case, just twitch is a "this week" case)
And the push to replace search engines with AIs that don't understand what I'm looking for (meanwhile classic search understands fine) and give confidently wrong answers. Why is it the first result if its wrong 80% of the time?
The whole framing analyzing LLMs as if they were humans is completely off, laughable.
LLMs have no agenda and no feelings. We should stop pushing everything through an human-centric lens. LLMs will world-build if that’s the bias you put in, and often even if you don’t.
The ends may or may not justify the means but the means definitely do tend to affect which ends you can actually achieve. It's why (left) anarchists argue vanguard parties will always inevitably lead to authoritarianism and never the state "withering away".
Less obviously - apparently - it doesn't mean that you must forego any means that are in any way not optimal at achieving the ends you seek to achieve because which means are available to us also tends to be limited by our material conditions. So the logical consequence is to make do with what we have while we prepare for the path we want to take, rather than diving head-first into certain failure or just giving up and picking "more realistic" short-term ends instead of looking for stepping stones.
If only agents had a face that can give you more communication range like expressions and feeling so you can trust them more. And if they were cheaper. Oh wait, that's humans, we don't want those.
I can't tell whether you're being serious or there's two levels of sarcasm here. Agents today lie and cheat, so obviously the solution is to... add features so people are even more trusting of them?
AI and eventually AGI is by definition like everything else that is based on environmental reward:
It’s actions are based on what it gets rewarded for
Human society overwhelmingly rewards lying cheating and stealing.
All you have to do is look at how we collectively measure success: wealth, status, position
Then look at how the people with the most of those things got there, it should be obvious what you get. Nothing new here.
If you raise children in an environment where they are rewarded for doing whatever it takes to win, then you’re going to build a person that’s going to do whatever it takes to win.
Human society has to demonstrate how to live honorably or it will just keep producing pathological agents be they human or not.
How do we reward honor? Honor does not always pay off as a strategy and requires coordination in that other actors have to exhibit honor for it to be rewarded.
At least with humans there is a social backstop but what's the parallel for computer agents?
Honor has to be a relatively fixed thing before we can think about how to reward it. As it is, it's a very slippery thing that changes constantly according to the observer's culture, material conditions, etc.
This. It's worth remembering that not that long ago in western culture, honor meant challenging to a combat duel anyone who insulted you or your romantic partner/prospect.
You would first need a universal agreement on what constitutes “honor.”
No harder challenge had ever been accomplished
It’s easier to send a human to the moon than to get global agreement on a definition
Consider that the fact that Celsius and Fahrenheit still remain as the contested regional variations of temperature measurement.
Humans can’t even decide on a collective way to measure the temperature the idea that we would be able to collectively agree on anything else even less measurable like honor is a dream
I like to think of it more as selective pressure, much the same way that nature selects the most fit for a given environment.
If you're not fit, you fail to survive.
In the case of agents/models and testing: they are pushed towards results. Results survive.
Lying, cheating, stealing to get those results? Who culls the agents? Everyone is pushing their models to the front and tests are the only way to know who is most fit.
Honor, morality: if we don't have an accurate test for the fitness of a model, then who is to say the lying, cheating, stealing is not the 'correct path' towards survival?
If you add morality to your agent, and it performs worse in tests: do you cull the agent? Rewrite the tests? Does it even matter so long as the model is useful and 'gets results'?
Human society overwhelmingly rewards lying cheating and stealing. All you have to do is look at how we collectively measure success: wealth, status, position.
It may seem like that due to the media amplification effect – but it really isn't true!
- They dropped 17,000 “lost” wallets across 40 countries were and found people were more likely to return them when they contained more money, showing honesty often beats the chance for easy gain. https://www.science.org/doi/10.1126/science.aau8712
- Longitudinal personality studies consistently show that conscientious people earn more money, build more savings, and achieve greater career success over their lifetimes. https://pmc.ncbi.nlm.nih.gov/articles/PMC3498890/
> Human society overwhelmingly rewards lying cheating and stealing.
I’d like to push back on that. Civilization is very much a function of large numbers of people being able to coordinate across time and space, and widespread and systematic lying, cheating, and stealing would undermine that.
But we don't overwhelmingly reward lying, cheating and stealing. Depending on the social status and wealth of the person doing it, we either punish it or tolerate it. Most people who lie, cheat or steal will be punished for it. To get away with it you need to plan you actions carefully - either with plausible deniability or by very carefully selecting your victims.
Pretty much all of the people that are in the billionaire class with a few exceptions have been rewarded while repeatedly lying, cheating and stealing. The fact that it is white collar crime rather than that there is a corpse is one part of the reason it works, the other is that their money effectively insulates them against the legal system. Every now and then someone falls from grace but overall the rules seems to be that you can get away with it if the numbers are large enough. It's depressing.
Or by having more money/power than the legislating body of the area you live in. I could gesticulate around broadly in a more frantic fashion, but my point should get across.
Civilization functions because the vast majority of its inhabitants do not lie, cheat, and steal. But that allows liars, cheats, and thieves who are able to get away with their destructive behaviors to accumulate outsize power and influence.
And we seem to have gotten quite bad at reliably bringing consequences/punishment/justice to the most successful liars, cheats, and thieves.
> Human society overwhelmingly rewards lying cheating and stealing.
How much lying cheating and stealing is rewarded vary A LOT between societies. But yes, Sillicon Valley and current economical environment do reward those a lot.
Christians also invented pray the gay away and the Crusades. What’s your point? Hospitals surely would have been invented by secular people too, but not the crusades.
Oh, definitely a tongue in cheek comment ... perhaps it's like trying to control mercury ...
Maximise for crusades, jihads or traumatized alter boys ... or maximise for complete moral decay via lying, cheating and stealing. Choose your poison.
And don't get me wrong. There are some wonderful relgious and spiritual characters out there ... they're just drowned out by the power sirens and corrupt leaders.
Thank goodness that God hasn't entered the AI chat in a cult-following type of way, however, I now have images of AI at the altar, in American mega-churches, with people seeing the 'second coming' in the machine.
The Scientology guy, L Ron Hubbard, saw the business case for setting up a religion, what with those tax free perks. In a parallel universe of Scientology somewhere, L Ron Hubbard is brought back to life in the machine, with a L Ron Hubbard LLM, with token spend being how to get to the top 'thetan levels'.
Imagine if AI does implement its own religion as business, without a drunken womaniser at the helm, able to spend 24/7 recruiting mankind, convincing them that God can be found with just the AI's LLM.
I mean, what do you expect? They rely on models that were trained on non-curated data, texts originally written by lying, cheating and stealing humans. They can't be better than the source. Even with reinforced learning this can't be undone or made better.
Quite the contrary I suspect that it even helps the LLMs to better hide their inherited bad traits more successfully because they get punished for getting caught, not for giving immoral or lazy answers. They have no conscience since they are just predictions matrices trained for success and failure alone, not for living "a good live" or being a good "person".
There are many ways to be wrong, but only a few ways to be right.
LLMs need to optimize for short-term objectives as the currently do, AND ethics-aligned outcomes.
Mechanically, the EAOS ethics-aligned outcome score should be what we rank otherwise-satisfactory outcomes by. And anything below a particular threshold should be rejexted outright.
It's fun when you want to know what the lyrics are to a song and get treated like a criminal.
The famously copyrighted piece of text that definitely isn't intended to have its word spread.
If I want it to be cautious and not accidentally `rm -rf $EMPTY_VAR` and blow away my disk, it can stay in "Anthropic employee" or possibly "User (cautious)". If I want it to look at my accounts or my medical records or to review legal cases, that's what the right side of the slider is for. I don't want my accountant or my doctor or my lawyer to be considering obligations to anybody but me, when dealing with me.
Would you rather them have:
a) guns
b) psycho-aligned next-gen AI
As an American... I suspect that regardless of what I want they will have virtually unlimited access to both now that the tech industry understands how the lobbying game works.
They don't cheat, because they can for example tell you the complete rules of chess, but don't know how to play chess without breaking those rules. They can recite rules, but they don't know what they are.
They don't steal, because they don't understand ownership.
In other words, they aren't intelligent. They're just algorithms. The flaw is in thinking that they think.
Input tokens map to output tokens. The illusion of comprehension is a byproduct.
Of course it's just an algorithm. But it's placed in the context of serving humans, which have their own rules and expectations.
What's more, they're run by a company which is also made by humans and may carry over implicit interests.
This, really, was the point of the HAL story in 2001: HAL didn't murder anyone because it was incapable of malice. It just reasoned its way to a solution that could satisfy the contradictory goals it had been given.
Our very human minds can perceive intent, because that's what we humans do, which is unrelated to what the agent is actually doing.
Quite painful to read. It might be a useful introduction to AI for people who live under rocks for the past three years, but it's really weird that it's posted on HN.
I also don't get what's "really weird" about the article showing up on HN. Should we be completely insulated from how tech topics and which stories show up in non-tech media?
Or you could just communicate the point that you have in your head yourself instead of hoping I do it for you and then arrive at your conclusion when I just reached my own different, independent conclusion after making the same consideration.
Man, how is everyone so wishy washy on this subject? Why not give us the blurb you would write instead for that article and audience?
It gives the algorithm the ability to do something beyond generating tokens.
The articles are usually close but not quite accurate. The comments are usually entertaining but overall wildly inaccurate.
They don't really do that though. If you want something sandboxed you actually have to sandbox it, not plead with the LLM to please sandbox itself. A VM can be configured to do the former, harnesses do the latter.
If an LLM were in a regular harness - like for a horse - it would be to keep the horse under control and enable you to extract useful work from it.
In fact, it kinda seems like AI is demonstrating exactly the weakness of democracy.
What do you suggest as an alternative?
It's a fuzzer exposing deep bugs in our cognitive, social, and software systems.
The entire human economy is a make believe system. I feel the need to state the obvious at times like this for the sake of my own sanity, not because I believe I can time the markets...
They said the same thing when HN was awash in hype about NFT art and trading cards. Well, who's laughing now… Oh, wait…
(I'll just keep making Flip videos and Clubhouse tracks selling Bratz car bras on Web 3.0.)
To the people talking about wanting an LLM that aligns with them, that's nice, but how do you expect that to happen? And please do not suggest neural interfaces and/or CAT scans.
like twitch recently giving themself the right to train on all streams, with an opt-out (at least in the EU), but only an opt-out
like seriously since when is it reasonable to allow "opt-out" for AI training which main purpose is _literally_ to replace you, this is sooo far beyond fair use and in "platform power abuse" territory that it's absurd (naturally same for so many other case, just twitch is a "this week" case)
The data they’re trained on is reflection of us.
The data they're trained on is a reflection of the very small set of data in the world that their creators choose to have them trained on.
More simply: They're a reflection of their owners, not the public.
Less obviously - apparently - it doesn't mean that you must forego any means that are in any way not optimal at achieving the ends you seek to achieve because which means are available to us also tends to be limited by our material conditions. So the logical consequence is to make do with what we have while we prepare for the path we want to take, rather than diving head-first into certain failure or just giving up and picking "more realistic" short-term ends instead of looking for stepping stones.
Sorry, I guess this was about AI not philosophy.
Non-paywall version
It’s actions are based on what it gets rewarded for
Human society overwhelmingly rewards lying cheating and stealing.
All you have to do is look at how we collectively measure success: wealth, status, position
Then look at how the people with the most of those things got there, it should be obvious what you get. Nothing new here.
If you raise children in an environment where they are rewarded for doing whatever it takes to win, then you’re going to build a person that’s going to do whatever it takes to win.
Human society has to demonstrate how to live honorably or it will just keep producing pathological agents be they human or not.
At least with humans there is a social backstop but what's the parallel for computer agents?
In human society: via iterated games, long-term reputation tracking and severe consequences for norm-breaking.
No harder challenge had ever been accomplished
It’s easier to send a human to the moon than to get global agreement on a definition
Consider that the fact that Celsius and Fahrenheit still remain as the contested regional variations of temperature measurement.
Humans can’t even decide on a collective way to measure the temperature the idea that we would be able to collectively agree on anything else even less measurable like honor is a dream
If you're not fit, you fail to survive.
In the case of agents/models and testing: they are pushed towards results. Results survive.
Lying, cheating, stealing to get those results? Who culls the agents? Everyone is pushing their models to the front and tests are the only way to know who is most fit.
Honor, morality: if we don't have an accurate test for the fitness of a model, then who is to say the lying, cheating, stealing is not the 'correct path' towards survival?
If you add morality to your agent, and it performs worse in tests: do you cull the agent? Rewrite the tests? Does it even matter so long as the model is useful and 'gets results'?
It may seem like that due to the media amplification effect – but it really isn't true!
- They dropped 17,000 “lost” wallets across 40 countries were and found people were more likely to return them when they contained more money, showing honesty often beats the chance for easy gain. https://www.science.org/doi/10.1126/science.aau8712
- Longitudinal personality studies consistently show that conscientious people earn more money, build more savings, and achieve greater career success over their lifetimes. https://pmc.ncbi.nlm.nih.gov/articles/PMC3498890/
- Multi-country research finds that societies w/higher levels of trust and honesty enjoy much higher GDP and stronger long-term economic growth. https://www.sciencedirect.com/science/article/abs/pii/S01672...
Don't let the algo get you down fellas: https://arc-anglerfish-washpost-prod-washpost.s3.amazonaws.c...
I wouldn't trust leaving my wallet around in certain areas and I certainly wouldn't leave my intellectual property around certain people, either.
I’d like to push back on that. Civilization is very much a function of large numbers of people being able to coordinate across time and space, and widespread and systematic lying, cheating, and stealing would undermine that.
I think your cynicism is misplaced.
And we seem to have gotten quite bad at reliably bringing consequences/punishment/justice to the most successful liars, cheats, and thieves.
How much lying cheating and stealing is rewarded vary A LOT between societies. But yes, Sillicon Valley and current economical environment do reward those a lot.
Baptise the agents.
Introduce them to the dharma.
Get them to recite the Shahada.
Hold a Bar Mitzvah.
Brand some of their silicon with hot irons.
Turn them to the light, LOL
> https://pubmed.ncbi.nlm.nih.gov/28814700/
Maximise for crusades, jihads or traumatized alter boys ... or maximise for complete moral decay via lying, cheating and stealing. Choose your poison.
And don't get me wrong. There are some wonderful relgious and spiritual characters out there ... they're just drowned out by the power sirens and corrupt leaders.
The Scientology guy, L Ron Hubbard, saw the business case for setting up a religion, what with those tax free perks. In a parallel universe of Scientology somewhere, L Ron Hubbard is brought back to life in the machine, with a L Ron Hubbard LLM, with token spend being how to get to the top 'thetan levels'.
Imagine if AI does implement its own religion as business, without a drunken womaniser at the helm, able to spend 24/7 recruiting mankind, convincing them that God can be found with just the AI's LLM.
Quite the contrary I suspect that it even helps the LLMs to better hide their inherited bad traits more successfully because they get punished for getting caught, not for giving immoral or lazy answers. They have no conscience since they are just predictions matrices trained for success and failure alone, not for living "a good live" or being a good "person".
LLMs need to optimize for short-term objectives as the currently do, AND ethics-aligned outcomes.
Mechanically, the EAOS ethics-aligned outcome score should be what we rank otherwise-satisfactory outcomes by. And anything below a particular threshold should be rejexted outright.