SKILL.md: why skills exist

I was trying to understand why agents need “skills” when they already have prompts and tools. The distinction is simpler than it first sounds. A tool is usually one function the model can call, while a skill is a complete capability around a task.

It can include instructions, the tools required, reference knowledge, workflow logic, and task-specific guardrails. So a skill is not really another tool. It is a way of packaging the knowledge and procedure needed to use tools properly for a specific job.

The reason this becomes useful is scale. If every capability is loaded into the agent from the beginning, the system is simple and predictable, but unused skills still consume context. Dynamic skill discovery instead allows the agent to select only the skills relevant to the current task. Skills can then be discovered, selected, activated, executed, and removed from context afterwards.

A simple use case: code review

A raw tool might let an agent read files, grep code, or inspect a Git diff. A code-review skill can combine those tools with the instructions and knowledge needed to review changes for bugs, style, and security issues. The skill becomes the reusable unit for that task instead of putting all of those instructions permanently into the agent.

The mental model

The way I understand it now is: the model provides the base capability, tools expose actions, and skills package the task-specific knowledge and procedure around those actions.

This is also why skills and fine-tuning are different. Fine-tuning changes the model more deeply, while skills can be swapped, combined, and updated at runtime without retraining the model.

RAG: what I actually needed to understand

I first understood RAG as “store embeddings in a vector database, retrieve chunks, give them to an LLM.” That is technically correct, but it hides most of the system. The useful way for me to understand RAG is as an information retrieval system that decides what evidence the model gets to see.

1. Why retrieval exists

An LLM stores knowledge inside its parameters. That works for general knowledge, but the model can still be wrong, outdated, or completely unaware of private information such as internal docs, codebases, logs, policies, or deployment history. RAG keeps some knowledge outside the model and retrieves it when needed.

This also separates RAG from fine-tuning for me. Fine-tuning changes the model's behaviour or capability. RAG changes the information available to the model at inference time. They can work together, but they solve different problems.

2. A RAG system is really two pipelines

RAG becomes easier to reason about when indexing and querying are separated. Indexing happens when knowledge enters the system. Retrieval happens when a user asks something.

This gives a simple debugging rule: a bad answer does not automatically mean a bad model. The document may have been parsed badly, chunked badly, embedded badly, or simply never retrieved. By the time generation happens, several earlier decisions have already determined what the model is allowed to know.

3. Retrieval does not mean only vector search

I had mentally equated RAG with embeddings. That is too narrow.

Sparse retrieval such as BM25 works from lexical signals and is still extremely useful when exact terms matter: error codes, IDs, proper nouns, rare terminology, API names. Dense retrieval instead uses embeddings to match semantic meaning, so the query and document do not need to use the same words.

Neither solves every query. Hybrid retrieval keeps both signals. One useful method is Reciprocal Rank Fusion, where results are combined using their ranks instead of trying to directly compare incompatible BM25 and vector similarity scores.

For technical systems this matters a lot. "ERR_CONNECTION_RESET" should probably be found because those exact characters exist. “Why are sessions randomly expiring?” may instead need semantic retrieval to find documentation about refresh-token invalidation.

4. Chunking is a retrieval decision

Chunking looked like preprocessing until I realised that a retriever can only return the units I created during indexing.

A chunk that is too small may match precisely but lose the surrounding meaning. A chunk that is too large carries context but becomes less precise and adds irrelevant tokens. Fixed-size chunking is simple, semantic chunking tries to follow topic changes, and structure-aware chunking respects things such as headings, functions, classes, HTML sections, and tables.

Parent-child chunking made the trade-off especially clear to me: retrieve using a small child chunk for precision, then return the larger parent section to the model for context. Retrieval granularity and generation context do not have to be identical.

5. Basic retrieve → generate breaks quickly

Naive RAG assumes the user's query is already a good search query. Often it is not.

A user might write checkout broken after deploy, while the relevant incident report says payment-service latency regression following release 4.18. The information need is similar, but the language is different.

That is why retrieval systems start adding query transformation. HyDE generates a hypothetical answer and searches using it. Step-back prompting broadens a narrow question. Multi-query retrieval generates several formulations and combines their results.

Then the system can rerank the first-stage results with a more expensive cross-encoder. The first retriever might cheaply reduce millions of chunks to 50 candidates; the reranker can then compare those candidates more carefully and keep the best few. Contextual compression can reduce them further by removing irrelevant sentences.

Most of these improvements happen before generation. That is the part I had underestimated.

6. Agentic RAG changes retrieval into a decision

Standard RAG has fixed control flow:

retrieve → generate

That becomes weak when a question needs several searches, different data sources, or a second query based on what the first retrieval revealed. The source calls out multi-hop questions, ambiguity, heterogeneous sources, and iterative refinement as cases where static retrieval starts breaking.

Agentic RAG turns retrieval into a loop.

The important change is not “add LangGraph” or “use an agent framework.” It is that the system can now decide whether to retrieve, what to retrieve, where to retrieve it from, and whether another retrieval round is necessary.

The source also makes multi-source retrieval explicit: internal documents may belong in a vector database, current information on the web, structured records in SQL, and source code in a code index. One universal index is not automatically the correct abstraction.

7. Evaluation has to locate the failure

A single “answer quality” score is not enough because retrieval and generation can fail independently.

I need to evaluate at least three levels: whether the right passages were retrieved, whether the model used those passages correctly, and whether the complete system solved the user's task.

Recall tells me whether relevant evidence was retrieved at all. Precision tells me how much of what I retrieved was actually useful. MRR and NDCG care about where relevant documents appear in the ranking. Generation metrics then measure things such as whether claims are actually supported by the retrieved context.

There is also a non-obvious trade-off with top_k. Increasing k may improve recall while making the final answer worse because the model now receives more marginally relevant context. More retrieval is not automatically better retrieval.

8. Production RAG starts looking like backend infrastructure

The demo ends at “I can ask my PDF questions.” Production starts immediately after that unfortunate moment of confidence.

Real corpora change. Documents are updated and deleted. Embedding models change. Chunking strategies change. Indexes need migrations. Retrieval has latency budgets. Sources may disagree. Citations need to remain traceable.

The common failures are also broader than hallucination: the correct document may never be retrieved, misleading documents may poison the context, useful evidence may get lost inside long context, too many chunks may dilute the signal, or the model may cite documents that do not actually support its claim.

This is why incremental indexing matters. Updating a document should replace its old chunks rather than leaving stale fragments searchable. Deletion, expiration, document versions, and indexing timestamps become normal backend concerns. Changing the embedding model may require a separate index and migration because old and new vectors are not automatically compatible. Changing the chunking strategy can invalidate the indexed representation even if the source document itself did not change.

Latency becomes its own layer as well: metadata pre-filtering, approximate nearest-neighbour search, embedding caches, parallel retrieval, streaming and quantization all exist because retrieval has to work under actual serving constraints.

What changed in my mental model

I started with:

What I understand now is closer to:

So I no longer think of RAG as “an LLM connected to a vector database.”

I think of it as retrieval engineering around an LLM.

The model generates the final answer, but parsing, chunking, indexing, retrieval, ranking, routing, context construction and evaluation determine what evidence reaches it in the first place.

If that evidence is bad, a better generator mostly gives me a more articulate wrong answer.

Agent Memory: what does it actually mean for an agent to remember?

I used to think agent memory was basically conversation history stored somewhere and retrieved later. That is part of it, but it is not really the interesting problem. An LLM call itself is stateless: it receives some context, produces output, and the next call starts again from whatever context we provide. The context window is therefore working state, not persistent memory. Once an agent starts operating across long tasks, multiple sessions, or repeated interactions, we need an actual system deciding what survives, what gets retrieved, what changes, and what should eventually be forgotten.

1. Why memory exists at all

The first-principles reason is not personalization. It is the context-window boundary.

An agent running for hours or days can accumulate conversations, tool outputs, decisions, failures, code changes, observations and intermediate state. Keeping all of it inside the prompt forever is impossible, and even before hitting the hard context limit it becomes expensive and noisy. Without persistent memory, information that leaves context is gone, previous mistakes cannot be learned from, and every new session effectively begins cold.

So I now separate these two things:

Context = what the model can see right now

Memory  = information the system can preserve
          and bring back when useful

A bigger context window delays the problem. It does not remove the need to decide what information deserves to stay around.

2. “Memory” is not one thing

The useful part of the taxonomy in the chapter is that it separates memory by how information is used rather than treating everything as one giant vector store.

There are four types: working, episodic, semantic and procedural memory.

Working memory is the immediate workspace: recent conversation, current task state, active files, temporary reasoning and whatever is already inside context. It is fast because nothing needs to be retrieved, but it disappears when that context disappears. Episodic memory stores specific experiences: previous conversations, successful attempts, failed attempts, decisions and outcomes.

Semantic memory stores facts and concepts independent of the particular episode where they were learned. Procedural memory stores how to do things: tool-use patterns, workflows and action sequences. The chapter gives a software debugging example that makes the distinction quite clean: the current error is working memory, a similar bug fixed last week is episodic memory, knowledge about asyncio.gather is semantic memory, and reproduce → isolate → hypothesize → test → fix is procedural memory.

That made “memory” much less vague for me.

3. Storing everything is not memory

My first naive memory design would probably have been:

conversation happens
       ↓
store conversation
       ↓
retrieve later

But an unlimited transcript is closer to an archive than useful memory.

The actual problem starts at the write step: should this information be remembered at all?

The source treats writing as a filtering problem. Importance might come from surprise, reward, or the model's own estimate of whether an event matters. Before committing a new fact, the system may also have to check whether it contradicts something already stored.

There is also the question of granularity.

User prefers Python.

is easy to search and update.

An entire 40-message conversation contains much more context but is noisy and expensive to retrieve.

A summary sits somewhere between them.

The chapter suggests that production systems often combine these forms: atomic facts for precise recall, summarized episodes for narrative context, and verbatim history in colder storage for auditability.

That feels much closer to a database design problem than “give the chatbot memory.”

4. Memory has four real operations

The cleanest model I found is:

write
read
update
reflect

Writing determines what deserves persistence.

Reading determines which old information matters now.

Updating deals with changed or contradictory information.

Reflection turns several specific experiences into a more general insight.

Retrieval itself does not have to use only semantic similarity. The query can be rewritten or expanded, and recency can matter alongside relevance. A memory from five minutes ago may deserve more weight than a semantically similar observation from six months ago.

Updating is equally important because memory can become stale. Related memories can be consolidated, old entries can be evicted, and low-value information can decay. The source explicitly discusses LRU eviction, importance-based forgetting and retaining repeatedly accessed memories longer.

So forgetting is not necessarily a failure of memory.

Sometimes it is a feature required to keep memory useful.

5. Reflection is where experience becomes knowledge

This was the part I found most interesting.

Suppose a coding agent fails three times because it keeps missing the same edge case.

Episodic memory can store:

Attempt 1 → failed on empty input
Attempt 2 → failed on empty input
Attempt 3 → failed on empty input

But simply storing three failures does not mean the agent learned anything.

Reflection can turn them into:

Always check empty-input behaviour
before finalizing this class of solution.

The source frames this as moving information between memory types: retrieve specific episodes from episodic memory, reason over them in working memory, then store the generalized insight as semantic memory.

That is a more useful definition of “learning from experience” than simply appending previous chats to the prompt.

The memory system does not just preserve the past. It can compress repeated experience into reusable knowledge.

6. Memory architecture starts looking like a cache hierarchy

One large memory store is possible, but not every memory needs the same latency or availability.

MemGPT's useful analogy is virtual memory in operating systems: keep a small amount immediately available, keep more information in retrievable storage, and push old material into archival storage.

The implementation pattern later in the chapter describes this as hot, warm and cold memory. Hot memory is directly in context, warm memory sits in something like a vector store, and cold memory is archival. Entries can move between tiers depending on relevance, importance and access frequency.

I like this model because the goal is not “make every memory always visible.”

The goal is:

keep the right information at the cheapest useful level, then move it closer when required.

That is suspiciously similar to how basically every computer system solves limited fast storage. Apparently agents do not escape computer architecture merely because someone added a transformer.

7. Persistent conversation memory is really state management

For conversational agents, memory often becomes visible as personalization: preferences, expertise, ongoing goals, previous sessions and project history.

The useful engineering view is not “the AI remembers me.” It is that the system maintains a persistent user model and retrieves relevant parts of it into a new session. The chapter describes session continuity as retrieving user information and previous session summaries at startup, then updating that state after the new interaction.

This is also where privacy stops being optional architecture decoration. Persistent personal memory creates requirements around consent, deletion, access control and retention. The source explicitly calls for privacy-by-default rather than treating stored user history as free raw material.

So good conversational memory is not “store more.”

It is remember selectively, retrieve selectively, and let the memory change when reality changes.

8. Multi-agent memory becomes shared state

Memory becomes even more interesting once there are several agents.

Without shared memory, agents have to continuously send everything they know to each other. That creates duplicated communication and makes coordination dependent on conversation history.

A shared memory pool instead gives them persistent state that any permitted agent can query.

The blackboard architecture goes further: agents communicate indirectly through a shared state rather than sending everything directly to one another. But then ordinary distributed-systems problems arrive, because apparently they always do: concurrent writes, conflicting facts, versions and consistency. The chapter discusses options such as last-write-wins, version histories, voting, confidence-weighted merging and designated ownership.

So shared agent memory is not just an AI problem.

At some point it becomes a distributed state-management problem.

9. How do I know the memory system actually works?

Perfectly storing information is not enough.

A memory system can remember every single event and still be terrible if it constantly retrieves irrelevant information into the model's context.

The source separates evaluation into abilities such as information extraction, multi-session reasoning, temporal reasoning, handling knowledge updates, and correctly abstaining when no relevant memory exists.

Useful metrics therefore include memory recall, precision of retrieved memories, retrieval latency, token efficiency, downstream answer accuracy, faithfulness and contradiction rate. Production systems also need operational metrics such as write frequency, stale-memory retrieval and storage growth.

That last category matters because a system may look good over 20 conversations and slowly rot over 2,000. Contradictions accumulate. Storage expands. Old facts survive newer ones. Retrieval gets noisy.

Memory quality is therefore something that has to be monitored over time rather than tested once.

What changed in my mental model

I started with this:

Now I think agent memory looks more like this:

The main thing I understand now is that memory is not storage.

Storage is one component.

A useful agent memory system needs policies for what to write, how to represent it, how to retrieve it, how to handle time and contradictions, when to forget, and how to turn repeated experiences into more general knowledge. The chapter eventually reduces the same system to four core operations: write, retrieve, update and reflect.

So when someone says an agent “has memory,” the question I would ask now is:

What exactly is being remembered, how is it retrieved, and what happens when that memory stops being true?

That is where the actual engineering starts.

Agent Harness: the layer that actually makes an LLM an agent

I had understood models, tools, memory and orchestration separately, but the term agent harness made the system boundary much clearer. An LLM itself is still basically a stateless tokens → tokens function. It does not persist state, execute tools, manage context, retry failed calls or know when an action needs approval. The harness is the runtime around the model that handles those things. It manages context, memory, tool execution, state, routing, safety and observability while leaving the actual reasoning to the model.

The operating-system analogy is useful here. The model provides intelligence, but the harness gives that intelligence controlled access to memory, tools and execution. A strong model inside a bad harness can still behave badly because the infrastructure determines what information it sees and what actions it is actually allowed to perform.

1. Context is a budget, not just a window

The first thing the harness has to manage is context. I used to think of context length mostly as “how much text can the model accept?” The more useful question is: what deserves to occupy those tokens?

The context is shared between several things: the system prompt, retrieved memory or RAG results, tool definitions, conversation history and space reserved for the model's response. History keeps growing while the maximum context stays fixed, and a single large tool result can suddenly consume a huge part of the budget.

So context management becomes allocation. Recent messages may remain verbatim, older history can be summarized, irrelevant turns can be removed, and important instructions can stay pinned. The chapter also describes hierarchical summarization, where recent information stays detailed while increasingly old information becomes increasingly compressed.

What matters to me is that context management is information selection. A 128k context window does not mean I should put 128k tokens into every request. Every irrelevant token costs latency and competes for attention with information that actually matters.

The harness should also count tokens before every call instead of waiting for the API to decide what disappears when the limit is crossed.

2. The prompt is assembled by the harness

I also stopped thinking of a production prompt as one giant string.

A harness can build the actual prompt at runtime from separate components:

That means the system instructions, memory, tool definitions, history and current request can be versioned and changed independently. The source describes this explicitly as dynamic prompt assembly rather than maintaining one monolithic prompt.

This makes prompts feel less like magic text and more like application configuration.

prompt =
    system
  + relevant memory
  + relevant tools
  + relevant history
  + current task

The important word is relevant.

If an agent has 200 tools, putting 200 tool definitions into every request wastes tokens and gives the model more opportunities to choose the wrong one. The same applies to memory and few-shot examples.

3. Tools need an execution layer

Giving a model a tool definition does not mean the model itself executes anything.

The model produces something like:

tool: search_web
args: {...}

Then the harness has to validate that request, execute the real function, process the result and return it to the model.

Even the tool description matters because it is part of how the model decides which tool to use. A useful tool schema explains what the tool does, when it should and should not be used, its parameters, return structure and side effects.

Then there is the result.

A database query could return 50,000 rows. A web tool could return an entire page. A code execution tool could emit megabytes of logs.

Throwing all of that directly back into context is not sensible.

The harness may need to parse the result, enforce its expected schema, truncate or summarize large outputs, normalize errors and retry transient failures before the model ever sees them.

So:

model chooses action
        ↓
harness executes action
        ↓
harness cleans result
        ↓
model reasons over result

That boundary matters.

4. Tool selection itself eventually becomes retrieval

Another thing I hadn't connected before is that large tool libraries create essentially the same problem as large document corpora.

If I have five tools, I can probably give all five definitions to the model.

If I have 2,000 tools, that stops making sense.

The source describes retrieval-augmented tool selection: search the available tool descriptions first, inject only the top relevant tools into context, then let the model make the final selection.

That is basically RAG applied to capabilities instead of documents.

I find that useful because it connects several agent concepts I had treated separately. Context management, memory retrieval, skill discovery and tool discovery are all versions of the same engineering problem:

there is more information available than the model should see at once, so select the useful subset first.

5. Orchestration is deciding what happens next

The harness also controls the execution loop around the model.

The simplest agent pattern is something close to ReAct:

The model reasons, chooses an action, receives the observation and decides what to do next. The harness runs the loop, parses the tool calls and enforces termination conditions such as maximum iterations.

But ReAct is only one orchestration pattern.

For a task with a predictable structure, the agent can first create a plan and then execute it:

The source calls this Plan-and-Execute. It can use fewer model calls on long tasks, although it is less adaptive when unexpected information appears.

For more complex systems there can be supervisors, specialist agents, peer-to-peer communication, hierarchical agents or handoffs. But the abstraction remains the same:

orchestration defines who acts next and under what condition.

Frameworks are just different ways of expressing that control flow.

6. State is what makes long-running agents possible

Memory and state looked similar to me initially, but I think they are cleaner when separated.

Memory answers:

What information from the past might be useful?

State answers:

Where exactly is this execution right now?

For a long-running task the harness might need:

current plan
completed steps
pending steps
tool calls in flight
approvals waiting
known facts
retry counters
current workflow state

The source separates conversation state, task state, agent state and persistent state. Long-running tasks also need checkpoints so execution can resume after failure instead of restarting from zero.

This is why explicit workflow graphs make sense for serious agents. Instead of control flow being hidden inside an increasingly heroic while True, states and transitions become visible and testable.

And once state matters, schema design matters too. The chapter makes the comparison I like: treat agent state like a database schema, with explicit structure, versions and migration paths rather than a random global dictionary that gradually becomes sentient.

7. Autonomy needs boundaries

A model deciding that a tool should run does not automatically mean that tool should run.

Some actions are cheap and reversible:

read_file()
search_docs()
query_logs()

Others are very much not:

delete_database()
send_email()
deploy_production()
make_purchase()

The harness is where that distinction can actually be enforced.

The chapter describes approval gates, confidence-based escalation and asynchronous pause/resume for long-running workflows.

Tool execution also needs real isolation. Model-generated arguments should be validated, code execution should have resource and permission boundaries, and external tool results should be treated as untrusted data because retrieved content can itself contain prompt-injection instructions.

This makes the model/harness boundary clearer to me:

LLM:
"I want to do this."

Harness:
"Are you allowed to?"

That separation is much safer than asking the model to police itself.

8. Failure is part of the control flow

An agent calling external systems will fail.

APIs timeout. Models rate-limit. Tools return malformed data. The model repeats itself. An agent can spend twenty calls confidently rediscovering the same failure.

So retries and loop detection are not edge features. They are part of the runtime.

The source recommends exponential backoff for transient failures, fallback models, graceful degradation when a tool is unavailable, maximum iteration limits, duplicate-action detection and progress checks.

Graceful failure also means preserving useful partial work and state. A long-running agent that completes nine steps, fails on step ten and then forgets steps one through nine is not robust. It is merely expensive.

9. Observability is what makes any of this debuggable

Normal software often fails with an exception.

Agents can fail while every function returned 200 OK.

The tool worked. The database worked. The model responded.

It just chose the wrong thing.

That is a semantic failure, which means debugging needs visibility into the whole trajectory.

The chapter describes the usual observability triad:

Traces show the sequence of decisions. Logs record individual events. Metrics tell me whether the system is degrading across many executions. Useful agent metrics include task success, number of steps, tool failures, cost and p95 latency.

Replay is especially useful. If I can take a failed production trace, rerun it with another prompt or model and compare the trajectories, debugging becomes much less mystical.

Otherwise the bug report is basically:

“The agent became confused around step 14.”

A magnificent contribution to computer science.

10. Production turns the harness into backend infrastructure

Once many agents are running concurrently, the harness has to deal with the same things every backend eventually meets: latency, concurrency, queues, rate limits and cost.

Independent tools can execute in parallel. Responses can stream. Repeated prompt prefixes and deterministic tool outputs can be cached. Different models can be routed to different steps depending on how much reasoning that step actually needs.

Then concurrency introduces rate limiting and backpressure. Interactive tasks may need higher priority than background work, and when capacity is exhausted it is often better to reject new work explicitly than to create an infinite queue and quietly transform latency into geology.

At this point an agent system stops looking like “LLM + a couple functions” and starts looking like a normal production service with a nondeterministic reasoning component inside it.

That is probably the useful perspective.

What changed in my mental model

My initial model of an agent was roughly:

Now I think the real boundary looks more like:

The model is still the reasoning engine, but almost everything that makes that reasoning useful over time lives outside the model.

That is what an agent harness means to me now.

It is not another agent framework or another abstraction placed on top of an LLM. It is the runtime boundary that manages everything the model itself does not manage: context, tools, state, execution, recovery and control.

And this also explains why two products using the exact same model can behave completely differently.

The model matters, obviously.

But the harness decides what the model sees, what it can touch, what happens after it makes a decision, and whether the system survives when that decision goes wrong.

Agent Design Patterns: when to use workflows, agents, and everything in between

I used to think agent architecture mostly meant choosing a framework and wiring an LLM to some tools. The more useful way to look at it is simpler: who decides what happens next?

If the execution order is defined by code, I have a workflow. If the model decides what to do next based on what it observes, I have an agent. That distinction matters because agents add flexibility, but they also add cost, latency and unpredictability. The source makes the practical recommendation pretty clear: start with workflows and move toward autonomous agents only when the task actually needs dynamic routing or open-ended exploration.

That gives me a much better starting point than asking, “Which agent framework should I use?”

The first question should be:

How much decision-making does the model actually need?

1. Prompt chaining: when the steps are already obvious

Prompt chaining is probably the least exciting pattern, which is also why it is often the correct one.

A complex task is broken into fixed stages and the output of one stage becomes input to the next.

For example:

raw logs
   ↓
extract important events
   ↓
classify failure
   ↓
generate incident summary

The model does not need to decide which stage comes next because I already know the process.

The nice part is that each stage can have its own prompt, model and validation. If step two produces garbage, I can catch it before that garbage flows into step three. That makes the system much easier to inspect than one giant model call pretending to perform an entire pipeline internally.

My takeaway is that breaking a task into multiple LLM calls does not automatically make it agentic.

Sometimes it is simply a good pipeline.

2. Routing: choose the right path once

Routing is useful when the incoming requests are different enough that they should not all use the same prompt, tools or model.

A router can itself be an LLM or something much simpler like a classifier.

The important part is that routing happens once and the system then follows the chosen path.

For example, if I have an internal engineering assistant:

"Why did checkout fail?"
        ↓
incident investigation workflow

"Explain this function"
        ↓
code understanding workflow

"Find our deployment policy"
        ↓
document retrieval workflow

There is no reason one mega-agent needs to dynamically rediscover these categories every time.

The source frames routing exactly this way: classify the request and dispatch it to a specialized handler.

3. Parallelization: do independent work at the same time

Some tasks contain multiple pieces of work that do not depend on each other.

A code review is a clean example.

I could do:

security review
      ↓
performance review
      ↓
style review

Or:

The second version is faster because the independent calls run together.

The chapter separates two useful versions of this pattern. Sectioning divides a problem into independent pieces, while voting asks multiple models or generations to solve the same problem and then selects or combines the answers.

This distinction is useful:

sectioning
different work → parallel calls

voting
same work → multiple attempts

Parallelization is not primarily about intelligence. It is mostly about latency, decomposition and sometimes reliability.

4. Orchestrator-workers: let the model decide the decomposition

This is where things become more agent-like.

With normal parallelization, I decide the subtasks beforehand.

With orchestrator-workers, the model looks at the problem and decides how it should be divided.

Suppose the task is:

Refactor this backend.

I cannot necessarily define the exact subtasks before the system has inspected the repository.

The orchestrator may first discover:

auth module needs cleanup
database layer has duplicated logic
API schemas are inconsistent
tests are missing around payments

Then it can create workers around those discovered problems.

That is the key difference: the decomposition itself is generated at runtime.

This pattern makes sense when I know the goal but cannot know the exact task graph beforehand.

5. Evaluator-optimizer: generate, judge, improve

Another pattern I find useful is separating creation from criticism.

One model generates something.

Another evaluates it against explicit criteria.

If it fails, the feedback goes back into another attempt.

For code:

generate implementation
        ↓
run tests / evaluate
        ↓
tests fail
        ↓
feed failure back
        ↓
repair implementation

This works well when quality can actually be checked.

That last point matters.

If the evaluator has no clear criterion, I have just created two models confidently debating each other at my expense.

The source recommends evaluator-optimizer for tasks where there is an explicit quality bar: tests, translation fidelity, style requirements and similar measurable constraints.

Now the autonomous patterns

The previous patterns still keep most control in normal software.

Autonomous agent patterns move more of that control into the model itself.

That flexibility is useful, but this is also where I should become much more suspicious about complexity.

6. ReAct: decide one step at a time

ReAct is the classic loop:

The agent looks at the current state, chooses a tool or action, sees the result and then decides again.

For example:

Need to understand this error
        ↓
search logs
        ↓
see database timeout
        ↓
inspect database metrics
        ↓
notice connection exhaustion
        ↓
inspect latest deploy
        ↓
form hypothesis

The key part is that I did not predefine this exact sequence.

Each observation changes what the agent decides to do next.

But an open loop needs boundaries. The source calls out tool parsing, explicit termination and maximum iteration limits as basic implementation requirements.

Otherwise:

Congratulations, we have invented a token-burning perpetual motion machine.

7. Planning agents: create a task graph before acting

ReAct decides locally.

Planning agents first step back and build an explicit plan.

The interesting part is that there are different levels of planning.

The source compares four strategies:

• plan once and never change it,
• replan only when something fails,
• reconsider the plan after every step,
• keep a high-level plan fixed while generating smaller subplans dynamically.

That creates a real trade-off.

more planning
    ↓
more adaptability
    ↓
more LLM calls
    ↓
more latency + cost

I like the chapter's framing that the plan should be a living document, not a prophecy.

The plan gives the agent structure without pretending that nothing unexpected will happen.

8. Reflection: learning without changing weights

Reflection is slightly different from planning.

The model reviews what it already did and asks:

Was the output correct?

Where did the trajectory go wrong?

Am I even solving the right problem?

Then it can change strategy.

The Reflexion version goes further by persisting the insight.

For example:

Attempt failed:
forgot empty-input edge case

        ↓

Reflection:
check boundary conditions before final answer

        ↓

next attempt retrieves that lesson

No weights changed.

The system learned by storing a useful piece of experience and feeding it back into future execution.

This connects directly to the agent-memory idea from the previous note: memory becomes useful when past experience changes future behaviour.

9. Tool-use itself has patterns

I had mostly thought about tools individually:

search()
read_file()
query_db()

But the way tools are composed is itself an architectural decision.

The chapter describes five patterns.

Single-turn

One tool answers the information need.

Good for simple lookups.

Parallel

Useful when all tools are independent.

Sequential

Each result determines the input to the next tool.

Nested

An entire agent becomes callable like a tool.

Fallback

The source's point here is important: the harness can own fallback logic instead of forcing the model to reason about infrastructure failures.

That keeps model reasoning focused on the task rather than rate-limit archaeology.

The principle behind all of this

After going through these patterns, I think the most important part is not memorizing their names.

It is understanding how much uncertainty exists in the task.

The source gives essentially the same recommendation in its selection guide: start from the simpler patterns and move downward only when those patterns actually fail.

That is probably the biggest correction to my thinking.

I used to see:

workflow < agent < multi-agent

as increasing levels of sophistication.

Now I see:

workflow
agent
multi-agent

as different amounts of runtime uncertainty and autonomy.

More autonomy is not automatically better architecture.

What I understand now

My earlier mental model was:

That hides the most important design choice.

A better model is:

The design principles at the end of the chapter are basically the ones I would keep:

Use the simplest architecture that works. Keep execution inspectable. Give the model precise tools. Expect tools to fail. Prefer structured outputs. Test behaviour on messy inputs, not only happy paths.

What I take away from agent design patterns is that the engineering problem is not “how do I add more agents?”

It is:

which decisions should remain deterministic, and which decisions genuinely benefit from being delegated to the model?

Once I can answer that, choosing the pattern becomes much easier.