Loved this. It's funny how basic all of this can be. We have all these skills and and tricks and incantations whatever else, when it probably all just comes down to giving the agent the right info to set them up for success (just like you'd do with anyone you're delegating anything to).
I found it interesting that Thorsten mentions the importance of attaching screenshots.
In my experience, it varies widely by workload. Vision is great for some UI stuff but extremely terrible for my work, which involves visualizing connectivity and geometric stuff for road structure for autonomous vehicles. It can get super frustrating when I paste a screenshot and go "this polyline isn't connected when it should be" or "those polylines are self intersecting in an invalid way" and then the AI literally can't see what I'm talking about. It would have to then write debug code to dump out the coordinates of the polylines to convince itself that the bug even exists. Then it would often claim that it's fixed when it is clearly not.
I previously complained about GPT 5.6 Sol not being able to solve a simple "follow the lines" puzzle [1] without having to write custom Python code for it. GPT-6 Astra, however, is vastly improved, and could do it without writing Python. Still, for my work, it can occasionally struggle.
I think these AI platforms would guarantee bigger wins for themselves if they figured out how to give these models more of an object-conceptualization of their input than they currently have. Solving math puzzles is cool and all, but 99% of customers don't need to solve math puzzles like that; they're asking the model to answer questions about thins and to translate ideas into a real UI. A model that could reliably determine if/how its input/output isn't visually matching up to a design would be a big deal. Sure, we kind of have that now, but it's still in gimmick territory.
Slick presentation. His prompts look similar to my brain dumps. I’m in the habit of running these through Codex’s plan-mode first to ensure the AI is on the same page.
articles like this need to factor in the overall time/effort tradeoff in developing the set of prompts necessary to reach an overall goal... for example, if I can achieve the same results while being sloppy/lazy/vague with my prompts, I wouldn't bother with the extra discipline or time expended to craft "to spec" prompts
In my experience, it varies widely by workload. Vision is great for some UI stuff but extremely terrible for my work, which involves visualizing connectivity and geometric stuff for road structure for autonomous vehicles. It can get super frustrating when I paste a screenshot and go "this polyline isn't connected when it should be" or "those polylines are self intersecting in an invalid way" and then the AI literally can't see what I'm talking about. It would have to then write debug code to dump out the coordinates of the polylines to convince itself that the bug even exists. Then it would often claim that it's fixed when it is clearly not.
I previously complained about GPT 5.6 Sol not being able to solve a simple "follow the lines" puzzle [1] without having to write custom Python code for it. GPT-6 Astra, however, is vastly improved, and could do it without writing Python. Still, for my work, it can occasionally struggle.
[1] https://news.ycombinator.com/item?id=49334444