Warranty Void If Regenerated

(nearzero.software)

136 points | by Stwerner 3 hours ago

26 comments

  • donatj 19 minutes ago
    I'm trying to sort out my own emotions on this.

    I did not realize this was AI generated while reading it until I came to the comments here... And I feel genuinely had? Like "oh wow, you got me"... I don't like this feeling.

    It's certainly the longest thing (I know about) I've taken the time to read that was AI generated. The writing struck me as genuinely good, like something out of The New Yorker. I found the story really enjoyable.

    I talked to AI basically all day, yet I am genuinely made uneasy by this.

  • nativeit 38 minutes ago
    > The milk pricing tool consumed the feed tool’s output as one of its cost inputs. The format change hadn’t broken the connection — the data still flowed — but it had caused the pricing tool to misparse one field, reading a per-head cost as a per-hundredweight cost, which made the feed expenses look much higher than they were, which made the margin calculations come out lower, which made the recommended prices drop. “You changed your feed tool,” Tom said.

    “Yeah, I updated the silage ratios. What does that have to do with milk prices?”

    “Everything.”

    He showed Ethan the chain: feed tool regenerated → output format shifted → pricing tool misparsed → margins calculated wrong → prices dropped → contracts auto-negotiated at below-market rates. Five links, each one individually innocuous, collectively costing Ethan roughly $14,000.

    Ethan looked ill.

    --

    I've re-read this a few times now, and can't work out how the interpreted price of feed going up and the interpreted margins going down results in a program setting lower prices on the resulting milk? I feel like this must have gotten reversed in the author's mind, since it's not like it's a typo, there are multiple references in the story for this cause and effect. Am I missing something?

    [Edited for clarity]

  • helle253 2 hours ago
    that's funny, i know where this story is set (i grew up there) - or at least, the place Claude was basing things off of

    some inconsistencies that stuck out/i found interesting:

    - HWY 29 doesnt run through marshfield, its about 15 miles north.

    - not a lot of people grow cabbage in central wisconsin ;)

    - no corrugated sheet metal buildings like in the first image around there

    - i dont think theres a county road K near Marshfield - not in Marathon county at least

    fwiw i think this story is neat, but wrong about farmers and their outlooks - agriculture is probably one of the most data-driven industries out there, there are not many family farmers left (the kinds of farmers depicted in this story), it is largely industrial scale at this point.

    All that said, as a fictional experiment its pretty cool!

    • CamperBob2 55 minutes ago
      I think it serves even better as a metaphor for software engineering's future than as a forecast for the future of farming. As you suggest, farmers already had to make the "transition" over the course of the 20th century. A farmer from 1926 wouldn't recognize his counterpart today. They would have nothing to talk about. Software people, though, are still twentieth-century programmers at heart, who are just starting to feel their way through the Kubler-Ross process.

      Really a great story, and to the extent it was AI-written, well... even greater.

      • selimthegrim 1 minute ago
        > As you suggest, farmers already had to make the "transition" over the course of the 20th century. A farmer from 1926 wouldn't recognize his counterpart today. They would have nothing to talk about.

        Can you elaborate on this?

      • arcanemachiner 44 minutes ago
        Kubler-Ross process -> "A model outlining emotional responses to terminal diagnosis or loss: Denial, Anger, Bargaining, Depression, and Acceptance"
        • CamperBob2 42 minutes ago
          Exactly. The stages don't always occur in order, or at all, but you can see the general progression play out any day, all day on here.

          I'm happily surprised (frankly amazed TBH) that the submitter didn't get bawled out by people flagging the post and accusing him of posting slop.

  • girvo 1 hour ago
    I will say this is one of the few pieces of prose I've read that was AI generated that didn't immediately jump out as it (a couple of inconsistencies eventually grabbed me enough to come to the comments and see your post details which mention it - I'd clicked through from the HN homepage), so your polishing definitely worked! Quite a neat little story
    • robot-wrangler 1 hour ago
      I think this passes the sniff test only if you're not too familiar with this neighborhood of the training set. Not that the writing is bad but it's just derivative. I listen to stuff like "Lost Scifi" podcast almost daily for example, but there are many similar ones which are focused on reading classic stuff from the golden-age zines because it's all public domain.

      The premise/structure/flavor of TFA is an almost pitch-perfect imitation of that kind of voice, to the point that I immediately flagged it as probably generated. I actually think a modern person would have some difficulty even in consciously mimicking it. There's an "aw shucks" yokel-thrown-into-the-future aspect to it. Plot-wise you have rural bicycle repair shop that expands operations to support nuclear reactors and that sort of thing. Substitute any of the more atomic-age stuff for AI stuff and you're mostly there. If you have some Amazing Stories from the 1920s on your shelf then you kind of know what I mean.

      • girvo 23 minutes ago
        > I think this passes the sniff test only if you're not too familiar with this neighborhood of the training set

        Which is totally fair, I'm honestly not! I haven't read much of that myself

      • jjmarr 57 minutes ago
        It is a pitch perfect interpretation and I assumed that's what OP was going for. Manna (2010) read very similarly.
        • robot-wrangler 46 minutes ago
          Can't speak for them but FWIW it does not sound like OP is necessarily aware of the genre at all. They asked Claude to explain something via fiction, and then perhaps Claude made the "creative decision" based simply on the availability of the material.
  • furyofantares 14 minutes ago
    I guess I'm an expert on LLM-isms somehow, I thought they were still plentiful. But I was able to get through the text, it's pretty good, you did great work cleaning it up. There's just a bit more to do to my taste.

    The story is good.

  • heap_perms 6 minutes ago
    I liked it. It has a similar feel to an Andy Weir "The martian" type of novel.
  • hatthew 1 day ago
    A fun read!

    I'm mildly thrown off by some inconsistencies. Carol says "I’ve been under-watering that spot on purpose for thirty years," and then a paragraph down Tom's thoughts say "Carol didn’t know that she under-watered the clay spot." Carol considers a drip irrigation timer the last acceptable innovation, but then the illustration points to the greenhouse as the last acceptable illustration. Several other things as well, mostly in the illustrations.

    Are these real inconsistencies or am I misunderstanding? Was this story AI-assisted (in part or all)? Is this meta-commentary?

    • gunalx 1 day ago
      I also got a slight feeling of ai assistance as well (especially on the drawings), but the story was well written and really sucked me in all in all.
    • Stwerner 1 day ago
      Thanks! Yeah this was AI assisted. As an experiment I started asking Claude to explain things to me with a fiction story and it ended up being really good, so I started seeing how far I could take it.
      • dazzaji 15 hours ago
        I’m pleasantly surprised this was AI assisted so deeply that inconsistencies like that slipped by you. The writing is really extraordinary. It made me want to read for fun again for the first time in decades. Thank you!
        • Stwerner 9 hours ago
          Funny, I was talking to a friend the other day about some thoughts on branding and he commented "as someone with a background in marketing & advertising communications, it's wild to watch a software engineer learn the value of branding and marketing from first principles".

          I guess I'm also learning the value of working with an editor from first principles... over the last couple weeks before publishing I read through and made edits to this piece at least twice a day and still didn't catch this.

          • FarmerPotato 8 minutes ago
            > from first principles

            I don't think that phrase means what you are trying to say here.

            What it doesn't mean: - learning by doing

            I believe it generally means: a formalization that comes after a subject is understood so well that you can reduce it to "first principles" that imply the rest. Or, the production of a hypothesis by deduction from widely-accepted principles.

  • jjmarr 1 hour ago
    Your polishing work made a difference! The prose is like every other work of science fiction I've read.

    It's written like this is a dystopia but billing $180/45 minutes in rural low cost of living area sounds awesome. And the choreographer billing "more than a truck" for three weeks? The dream!

    • ghewgill 1 hour ago
      The story didn't mention what had happened to inflation in the meantime. A dozen eggs costs $32.
    • brianm 40 minutes ago
      Huh, I got cottage core, not dystopia!
  • cortesoft 1 day ago
    I do enjoy this sort of speculative fiction that imagines though future consequences of something in its early stages, like AI is right now. There are some interesting ideas in here about where the work will shift.

    However, I do wonder if it is a bit too hung up on the current state of the technology, and the current issues we are facing. For example, the idea that the AI coded tools won't be able to handle (or even detect) that upstream data has changed format or methodology. Why wouldn't this be something that AI just learns to deal with? There us nothing inherent in the problem that is impossible for a computer to handle. There is no reason to think AIs can't learn how to code defensively for this sort of thing. Even if it is something that requires active monitoring and remediation, surely even today's AIs could be programmed to monitor for these sorts of changes, and have them modify existing code when to match the change when they occur. In the future, this will likely be even easier.

    The same thing is true with the 'orchestration' job. People already have begun to solve this issue, with the idea of a 'supervisor' agent that is designing the overall system, and delegating tasks to the sub-systems. The supervisor agent can create and enforce the contracts between the various sub-systems. There is no reason to think this wont get even better.

    We are SO early in this AI journey that I don't think we can yet fully understand what is simply impossible for an AI to ever accomplish and what we just haven't figure out yet.

    • andai 1 day ago
      Yeah, in the real world, Tom is already an OpenClaw instance...
      • Stwerner 1 day ago
        Funny I actually saw this tweet this morning about an Openclaw instance getting too advanced for the users to know how to control and fix: https://x.com/jspeiser/status/2033880731202547784?s=46&t=sAq...
        • Imustaskforhelp 1 hour ago
          > Funny I actually saw this tweet this morning about an Openclaw instance getting too advanced for the users to know how to control and fix: https://x.com/jspeiser/status/2033880731202547784?s=46&t=sAq...

          I feel like this ultimately boils down to something similar to nocode vs code debates that you mention. (Is openclaw having these flowcharts similar to nocode territory?)

          at some point, code is more efficient in doing so, maybe even people will then have this code itself be generated by AI but then once again, you are one hallucination away from a security nightmare or doesn't it become openclaw type thing once again

          But even after that, at some point, the question ultimately boils down to responsibility. AI can't bear responsibility and there are projects which need responsibility because that way things can be secure.

          I think that the conclusion from this is that, we need developers in the loop for the responsibility and checks even if AI generated code stays prevalent and we are seeing some developers already go ahead and call them slop janitors in the sense that they will remove the slop from codebase.

          I do believe that the end reason behind it is responsibility. We need someone to be accountable for code and we need someone to take a look and one who understands the things to prevent things from going south in case your project requires security which for almost all production related things/not just basic tinkering is almost necessary.

    • cactusplant7374 2 hours ago
      > The supervisor agent can create and enforce the contracts between the various sub-systems.

      Or you can ask the agent to do this after each round. Or before a deploy. They are great at performing analysis.

    • gambiting 14 hours ago
      >>There is no reason to think AIs can't learn how to code defensively for this sort of thing.

      For the exact same reason why there is absolutely no technical reason why two departments in a company can't talk to each other and exchange data, but because of <whatever> reason they haven't done that in 20 years.

      The idea that farmers will just buy "AI" as a blob that is meant to do a thing and these blobs will never interact with each other because they weren't designed to(as in - John Deere really doesn't want their AI blob to talk to the AI blob made by someone else, even if there is literally no technical reason why it shouldn't be possible), seems like the most likely way things will go - it's how we've been operating for a long time and AI won't change it.

  • tengwar2 1 day ago
    There's a bit of a tradition of introducing engineering ideas through stories. I remember a novella which was used to introduce something like MRP II (https://en.wikipedia.org/wiki/Material_requirements_planning) in the 80's. One of the reasons I think it works is that it keeps a focus on the human elements - like why Tom fitted the switch in your story. I remember automating a lab system back in 1985, which would bring in £1000 per day. Two weeks later I found out that the reason it wasn't in use was that the user wanted an amber monitor rather than a green one. I fitted the switch.

    I don't know if this is what the future will look like, but this looks realistic. And if my non-existent grandson starts re-coding my business without asking, he's going to spend the next six months using K&R C.

  • Havoc 1 hour ago
    This sort of article really needs at least a vague clue as to what it is about.

    It's a long article and from skimming I see chat of farming, software, GPS. I can't tell whether this is worth investing time to read if I can't even tell what it may be about

  • neversupervised 1 hour ago
    I don't oppose reading AI generated content in principle, but because it's free to generate, I always am less likely to read super long prose that is AI generated. So the question is whether someone has taken the time to keep it as long as necessary but not longer. Or if there are ways to make it easier for me to commit to the experience, with a sort of TLDR
  • ethansinjin 11 hours ago
    A fun read. I was hoping for the title to have some more relevance to the story, like someone who had handcrafted a piece of software and didn’t want others messing with it! Was that ever part of a draft?
    • Stwerner 9 hours ago
      Ugh yeah, I had an aside about the right-to-repair fights still going on indefinitely into the future that I ended up cutting. I kept the title because it seemed like a warning the characters would see on everything they bought, even if they ignored it. I'm sure I'll explore the idea more in the future though, I plan to explore insurance and liability and law at some point too.
  • jumpalongjim 1 day ago
    Often suggested by optimistic podcast guests these days: the as-yet-unknown new careers that will replace the familiar old ones and thus give employment in the AI era. I think your story is more a commentary on the current AI goldrush than an insight into future careers.
  • SeriousM 1 day ago
    This is such a good written fiction story. Well done. And the best part: I can see myself as Tom.
  • andai 1 day ago
    I enjoyed this very much. But I have to wonder, was this written by Claude?

    Edit: got it right!

    https://news.ycombinator.com/item?id=47419681

    • Stwerner 23 hours ago
      Haha well it was me and Claude ;)
      • Syntonicles 23 hours ago
        I wonder if it was de-indexed from HN for this reason.

        30 minutes ago it was on the front-page, now I can't find it listed in the top 200.

        • Stwerner 23 hours ago
          Yeah I was wondering the same thing. I didn’t realize there was any kind of rule against this kind of stuff
  • cactusplant7374 2 hours ago
    There are always bugs in software. The question is do you have enough eyes on the data to spot them or do they linger for years.
  • recursive 1 day ago
    I used to live in Marshfield WI. It's kind of jarring to see it mentioned "in the real world", the the extent that HN resembles that.
  • bstsb 1 day ago
    excellent story, it was both interesting and mildly terrifying. to think that one day software could be malleable seems so wrong to me - you would think having deterministic results is important for programming - and yet with "vibe coding" that really seems to be where it's going.
    • sanex 23 hours ago
      The whole reason it is called software is because of its malleability :)
  • chse_cake 1 day ago
    this is such a beautiful essay. thank you op for posting. made my day :)
  • lelandbatey 1 day ago
    Who can know what the world will look like as we "transition"? I sure don't, but I'm thankful the author here has taken a stab at it. I feel like this is one of the first stories I've seen to try to imagine this post-transition world in a way that isn't so gonzo as to be unrelatable. It was so relatable (the human-ness shining all the brighter in a machine-driven world) that I cried as I finished reading. I've felt very anxious about my own future, and to see one possible future painted so vividly, with such human and emotionally focused themes, triggered quite an emotional reaction. I think the feeling was:

    > If the world must change, I hope at least we still tell such stories and share how we feel within that change. If so, come what may, that's a future I know I can live in.

    • Stwerner 21 hours ago
      Thank you for this comment, I'm so glad it made you feel a little bit better about the future, if even for a little while!

      This is really the whole idea behind this project with Near Zero. I think there's a lot of anxiety out there around AI and the future, I was there for a while too. Ultimately I've ended up pretty optimistic about it all, and inspired by what the group at Protocolized is doing, found science fiction a great way to help express that.

  • bethekidyouwant 1 day ago
    It’s a neat piece of writing, but not nearly dystopic enough for my taste. There will only be one farm and whoever is fixing it will be on the other side of the world.
    • Legend2440 1 hour ago
      Yawn. I'm tired of dystopian fiction. We're likely to get something that is neither dystopia nor utopia, but somewhere in between.
    • 8n4vidtmkvmk 23 hours ago
      I think that's the point, and it's refreshing to see. My takeaway is that even if everything goes as good as it possibly could go, there will still be a need for that human touch.

      Just saying that everything is going to go to shit and one or two corporations will take over everything... Maybe, but I've heard that story already.

    • iwontberude 1 day ago
      Dystopians are too easy. The real challenge and reward are interesting utopian novels.
  • andai 1 hour ago
    Did this story disappear then re-appear?
    • tomhow 35 minutes ago
      Yes, which is why some of the comments are from a day ago but the post is only a couple of hours old. We originally downranked it due to being AI-generated.

      But on reflection and discussion with the author, we decided that enough HN users may find that it gratifies intellectual curiosity, because it's interesting to see how a human and an AI bot can collaborate to create writing like this.

      We just asked the author to write an introduction to make it clear it's AI-generated and explain their process.

  • swordmem 1 hour ago
    [dead]
  • thin_carapace 48 minutes ago
    ai photos and em dashes/directional inverted commas (so probably AI writing), sorry to be rude but i dont see the point of reading this when i could prompt an AI to generate the story myself if you shared the prompts? im not trying to be crass here, if someone could share why they think this is worth reading id be all ears. after being warned across the net to not be snarky about AI i really must iterate that snarkiness is not my intention here, im genuinely curious.

    edit: since im being downvoted, i clarify my interest to be in the human artistic input behind this mathematical model output

    • AlexCoventry 10 minutes ago
      FWIW, I read it before I learned that it was AI-generated, and I enjoyed it and thought it's possibly insightful.