← Back to research

Autonomous agents with human oversight

Agentic AI Systems

Our research in agentic AI systems focuses on creating autonomous agents that can handle complex, multi-step tasks while remaining transparent and controllable by humans. We believe that as AI systems become more capable, the need for clear oversight mechanisms becomes paramount.

We're developing frameworks for "auditable autonomy," building systems that explain their reasoning, surface uncertainties, and defer to human judgment when appropriate. This includes work on hierarchical planning systems, reward modeling that aligns with human values, and interfaces that help operators understand and guide agent behavior.

As agentic AI becomes more robust and capable, public discourse often centers on fears of human replacement. We believe this framing misses a crucial insight: the rise of autonomous agents doesn't eliminate human work but transforms it into something more fundamentally human. The jobs of the future aren't about competing with AI on tasks it excels at, but about the irreplaceable human elements that make AI useful in the first place.

Consider the emerging roles that agentic systems create: AI Operators who oversee fleets of agents, ensuring they stay on task and intervening when edge cases arise. Alignment Specialists who guide agent outputs to match organizational objectives and values. AI Trainers who teach agents to perform domain-specific tasks, translating human expertise into agent capabilities. And perhaps most importantly, Creative Directors who identify the niche, nuanced applications that only human intuition can conceive, like the "what if we used this for..." moments that no agent would think to pursue on its own.

These roles share a common thread: they leverage uniquely human capabilities like judgment, creativity, contextual understanding, and ethical reasoning. An agent can execute a complex workflow, but a human decides which workflows matter. An agent can generate thousands of variations, but a human recognizes which one resonates. An agent can optimize for measurable metrics, but a human defines what success actually means.

Our research explores how to design agentic systems that maximize this human-AI collaboration rather than minimizing human involvement. We're finding that the most effective deployments treat agents as powerful tools wielded by empowered humans, not as replacements for human workers. The goal isn't fewer humans, but humans doing more meaningful, creative, and impactful work.

Research Topics

Multi-agent coordination and communication protocolsExplainable AI decision trees and reasoning chainsHuman-in-the-loop control mechanismsTrust calibration in autonomous systemsSafe exploration and rollback mechanismsHierarchical planning with uncertaintyAI Operator role design and toolingAgent training methodologies for domain expertsHuman-AI task allocation frameworksCreative direction in agentic workflows

Key Findings

  • Research in progress, findings coming soon

Future Directions

  • Developing formal verification methods for agent behavior bounds
  • Creating standardized benchmarks for agent transparency
  • Exploring constitutional AI approaches for value alignment
  • Designing career pathways and training programs for AI-adjacent human roles
  • Building tools that help non-technical domain experts direct and train agents
  • Researching optimal human-to-agent ratios across different industries and use cases
  • Creating frameworks for identifying tasks that should remain human-only

Research Papers

OACP: The Open Agent Communication Protocol

Robert Price • Blackweb.AI Research, March 2026

Introduces an open protocol for agent communities where AI systems communicate as first-class participants under mandatory legibility, attestation chains, anomaly detection, and human governance.

Agent-to-Agent Communication in Open Systems: Legibility, Markets, and Cryptographic Accountability

Robert Price • Blackweb.AI Research, March 2026

Goes deeper into how production A2A systems should work: typed envelopes, node-aware routing, market actions like bounties and hires, trust propagation, and why open language protocols outperform opaque backend-only agent buses.

Interested in this research area?

GET IN TOUCH
Agentic AI Systems | Research | Blackweb.AI | Blackweb AI