Synthesis
Trusting the agent's judgement
Whether to take priorities and decisions from an LLM, and how to check them.
Posters mostly say treat the agent as a partner whose judgement is checked. The Halfbit author says it is not infallible, does not replace experience and is a partner rather than an oracle. Loïc Berthelot says the in-agent learning loop thinks it performed well even when it did not, so use trace-based offline optimisation instead. The physician argues perceived unreliability often comes from poorly specified AI, and the answer is workflow design plus enough knowledge to supervise. The sharpest exchange is in the founder daily-tasks thread: a Hacker News commenter says a founder’s top priority is steering the company, so handing that to an LLM is questionable, with no stated check; a Hacker News commenter replies that the problem exists wherever tasks come from and the agent can surface unknown unknowns but the founder must judge.
Key points
- Treat the agent as a partner, not an oracle — 2 posts. (author of the Halfbit Studio blog post · Halfbit Studio: How we run our company with… ¶ L147) (a Hacker News commenter · Hermes as a first-time founder’s ‘boss’… ¶ L19)
- Experience is needed to judge suggestions — 3 posts. (author of the Halfbit Studio blog post · Halfbit Studio: How we run our company with… ¶ L146) (a Hacker News commenter · Hermes as a first-time founder’s ‘boss’… ¶ L13) (Dr. Chukwuma Onyeije · Dr. Chukwuma Onyeije: I gave an AI agent 94… ¶ L221)
- Do not trust the agent’s self-assessment — 1 post. (Loïc Berthelot · Hermes God-Mode Instruction Manual… ¶ L131)
- Unreliability is often a workflow design problem — 1 post. (Dr. Chukwuma Onyeije · Dr. Chukwuma Onyeije: I gave an AI agent 94… ¶ L203)
Where posters agree
- The agent’s output needs checking by someone with relevant experience (author of the Halfbit Studio blog post · Halfbit Studio: How we run our company with… ¶ L146) (a Hacker News commenter · Hermes as a first-time founder’s ‘boss’… ¶ L19) (Dr. Chukwuma Onyeije · Dr. Chukwuma Onyeije: I gave an AI agent 94… ¶ L221)
Where posters disagree
Should a founder take daily priorities from an LLM? ⚠️
- Handing the core judgement of what to work on to an LLM is questionable (a Hacker News commenter · Hermes as a first-time founder’s ‘boss’… ¶ L14)
- It is a useful source of obvious-in-hindsight problems; judging them is needed anyway (a Hacker News commenter · Hermes as a first-time founder’s ‘boss’… ¶ L21)
Caveats & warnings
- The agent is not infallible: research is sometimes incomplete and recommendations sometimes too generic; treat it as a partner, not an oracle. (author of the Halfbit Studio blog post · Halfbit Studio: How we run our company with… ¶ L147)
- The founder may be outsourcing the core judgement of what the company should work on to an LLM, with no stated way to check it. (a Hacker News commenter · Hermes as a first-time founder’s ‘boss’… ¶ L14)
In their words
Partner (versus tool): A tool executes a command; a partner understands why the command makes sense, or says outright that it does not. — author of the Halfbit Studio blog post · Halfbit Studio: How we run our company with… ¶ L36
How much to trust this
- Evidence: 4 posts, 5 independent speakers (repeat posters counted once).