The tempting move is to start with the orchestrator.
That is usually backwards. A team does not need a swarm of agents before it has a theory of governed memory. It needs to know what counts as true, who can change it, who can act on it, and what has to be reviewed before it leaves the system.
The hard part is not connecting agents to tools. The hard part is deciding what the team is allowed to remember.
What this is
This is the architecture pattern I would use for a team that wants a shared agent layer: raw work artifacts, curated team memory, permission boundaries, retrieval, a shared orchestrator, role-specific agents, human review, audit, evals, and a feedback loop.
The diagram is intentionally not "one agent to rule them all." That pattern is attractive because it is easy to draw. It is also where accountability goes to disappear.
The orchestrator should route work. It should not become the only place where judgment lives.
The memory problem
Team memory has more failure modes than personal memory.
In a personal system, a bad note mostly harms the person who wrote it. In a team system, a bad memory can become policy, source material, a customer answer, a project plan, or the premise for another agent's work.
That means the shared memory layer needs ownership. Someone has to know which records are canonical, which are draft, which are stale, which are restricted, and which should be deleted.
This is why the raw intake queue is separate from shared memory. Meeting notes, tickets, documents, decisions, and project updates can all enter the system. They do not all deserve the same status.
How the orchestrator works
The shared orchestrator receives a request and decides what kind of work is being asked for.
Some requests should go to retrieval. Some should go to a scoped skill. Some should go to a role-specific agent. Some should go straight to human review because the request is ambiguous, high-risk, or outside the system's authority.
That routing decision is where many agent systems become opaque. A good orchestrator should leave a trace: what it saw, what it retrieved, which skill or agent it invoked, what it refused to do, and where human review entered.
Role agents should be narrow. A research agent gathers and cites. A planning agent turns constraints into options. A writing agent drafts from approved context. An ops agent prepares repeatable checklists or tickets. None of them should be able to silently rewrite shared memory.
Permissions are part of the architecture
The permissions registry is not an enterprise flourish. It is the thing that lets the system exist without leaking responsibility.
For each memory source and each skill, the team needs to know:
- who can read it
- who can write to it
- who can approve it
- who can trigger an external action
- what must be logged
- what must never be used as source material
Without that layer, the shared orchestrator becomes a confident intern with access to everyone's desk.
The feedback loop
The system can improve over time, but not by "learning from itself" in the loose sense.
It improves when reviewed outcomes become structured updates:
- Memory updates: an approved correction changes the shared record.
- Skill updates: repeated review comments change a skill's instructions or examples.
- Eval cases: failures become test cases before the workflow expands.
- Routing updates: repeated review decisions become routing rules, including which requests should go to a person instead of an agent.
This is the loop that makes the system more useful. It is also the loop that prevents it from becoming a self-reinforcing error machine.
The audit and eval layer is where this loop becomes visible. If the team cannot see which sources, tools, agents, and human edits produced an output, it cannot improve the system except by vibes.
Inside the boxes
The team version has the same technical pieces as the personal version, but the stakes move from usefulness to accountability.
- Team memory needs permissions: how read, write, approve, and action rights become part of the architecture.
- Retrieval is a judgment layer: why shared retrieval must filter by source, status, freshness, and authority.
- Feedback loops need a place to land: how review comments become system changes instead of disappearing into the final draft.
- Raw intake is not memory: why raw meeting notes, tickets, and documents should not become shared truth automatically.
Build sequence
First, choose one workflow. Not "team knowledge." A workflow.
For example: prepare a weekly project status brief from approved project notes, tickets, and decisions. That is concrete enough to define sources, owners, output format, review, and success.
Second, separate raw intake from curated shared memory. If everything flows into memory automatically, the team will spend the next year debugging trust.
Third, add retrieval with source trails and permissions. The system should be able to say, "I used these records, and I was allowed to use them."
Fourth, add one scoped role agent or skill. Do not start with four agents unless the handoffs are already obvious.
Fifth, add review and logging before expanding action. The system should produce drafts before it updates systems of record.
Finally, add the feedback loop: reviewed corrections become memory updates, skill updates, eval cases, or routing rules.
What I would not automate first
I would not start with autonomous external action. I would not let agents update canonical memory without review. I would not let every team member's private notes become shared truth. I would not use the orchestrator as a substitute for deciding who owns the workflow.
The first useful version should make a recurring team workflow faster and more inspectable. If the system cannot name what work stops, it is still a demo.
Failure modes
- Shared memory has no owner: everyone can add context, nobody can retire it.
- Permissions are implied: agents retrieve records the human reviewer would not have been allowed to use.
- The orchestrator hides the work: no one can tell why an agent was invoked or what it saw.
- Role agents overlap: research, planning, writing, and ops all rewrite each other's outputs.
- Feedback has nowhere to land: corrections happen in comments, not in memory, skills, evals, or routing.
- The team automates the visible step: the agent drafts the status report, but humans still do all the status gathering because the memory layer is not real.
Architecture references
This is closest to a governed workflow system with agentic components, not a free-form multi-agent conversation. That distinction is important. Anthropic's framing of workflows versus agents is useful here. OpenAI's Agents SDK foregrounds handoffs, tools, and tracing. MCP is relevant because tool access needs a standard interface. LangGraph is useful as a way to think about state, memory, and orchestration, but the governance boundary still has to be designed by the team.