A chatbot answers questions. An AI agent does things. That one distinction explains most of the hype, confusion, and genuine progress happening in AI tooling right now.

If you have written software for any amount of time, you already know the difference between a function that returns a value and a program that orchestrates a workflow. An AI agent is closer to the second. It takes a goal, breaks it into steps, picks the right tools, handles errors, and delivers a result — without someone babysitting every decision.

The core idea

An AI agent is a system built around a large language model that can take actions in the real world. Not just generate text. Read files, query databases, call APIs, write code, send messages, make purchases.

Three properties separate an agent from a regular LLM call:

  1. Tool use. The model calls external functions. A model that says “you should run this SQL query” is a chatbot. A model that runs the query and hands you the results is an agent.

  2. Planning. Given a goal, the agent figures out which steps to take and in what order. No hardcoded script. The LLM reasons about the task, picks the next action, observes what happened, and adjusts.

  3. Autonomy. The agent operates with some independence — branching logic, failure recovery, chaining multiple actions without waiting for a human to approve each one.

Take away any one of these and you have something less than an agent. Tools but no planning? That’s a function router. Planning but no tools? A reasoning engine. The agent is all three working together.

How agents work under the hood

Most production agents follow a loop:

Receive goal -> Plan next step -> Pick tool -> Execute -> Observe result -> Repeat or finish

The LLM sits at the center. On each iteration it receives the conversation history, the available tools, and the latest observation. Then it decides: call a tool, ask for clarification, or return a final answer.

Tools are exposed through interfaces. The Model Context Protocol (MCP) has become the standard way to do this. An MCP server advertises what it can do — read a file, search a database, create a calendar event — and the agent calls those capabilities through a structured interface. Build the tool once, and any MCP-compatible agent can use it.

Context management is the less glamorous half of the problem. Conversation history, tool results, and system instructions all compete for space in the model’s context window. The good frameworks handle this by summarizing old results and keeping the most relevant information in view. The bad ones just truncate and hope for the best.

Then there’s orchestration — the outer loop. Frameworks like LangGraph, CrewAI, and the Anthropic Agent SDK give you control over how the agent plans, retries on failure, and hands off between steps. Some support multi-agent setups where agents with different specializations collaborate on a task using protocols like A2A.

What changes for developers

When models go from “API you call” to “agent that acts,” the work changes.

The highest-value thing you can build is a good tool. A well-defined MCP server with clear descriptions and predictable behavior does more for an agent than hours of prompt tuning. The AgentNDX directory tracks over 25,000 MCP servers across categories like database management, web scraping, and cloud infrastructure. Each one gives agents a new capability without any model retraining.

Error handling stops being an afterthought and becomes a design problem. A tool call returns unexpected data. The model misreads a result. The plan needs to change mid-run. You’re not just coding for the happy path anymore — you’re designing for agents that will absolutely find the unhappy ones.

And security gets harder. Every tool an agent can access is an attack surface. An agent with database writes, email sending, and file system access needs guardrails. Tool-level permissions, authentication patterns, and sandboxing aren’t optional in agentic systems. They’re load-bearing.

AI agents in practice

Here’s what working with agents actually looks like right now.

Code agents — Claude Code, GitHub Copilot, Cursor — use file system tools, terminal access, and search to write, test, and debug code across entire repositories. They read existing code, understand project structure, make changes, and verify the result. Developers extend them with skills for domain-specific work.

Data agents connect to databases through MCP servers, run queries, analyze results, and generate reports. Instead of writing a throwaway script, you point an agent at a database MCP server and describe what you need.

Workflow agents are where it gets interesting. A support agent might look up a customer record, check order status, draft a response, and update a ticket — all in one run. The tools come from MCP servers for CRM, ticketing, and communication platforms. Research agents do something similar with web scraping tools and search APIs, replacing the manual loop of searching, reading, and summarizing that eats hours.

What an agent is not

Not everything with an LLM inside is an agent. A few things that get mislabeled:

  • A chatbot with RAG. Retrieval-augmented generation adds context to a model’s responses, but the model is still just answering questions. No tool use, no planning, no autonomy.
  • A single API call with structured output. Asking a model to extract data and return JSON is useful. It’s a function call, not an agent.
  • A workflow with an LLM step. If the logic is hardcoded and the LLM just fills in one piece (generate email copy, classify a ticket), that’s automation with an LLM component. The model isn’t deciding what to do next.

The line is blurry. But the core test holds: does the model decide what actions to take, execute them, and adapt based on what happens? If yes, agent.

FAQ

Q: Do I need a framework to build an AI agent? A: No. A model API, a tool-calling interface, and a loop will get you surprisingly far. Frameworks like LangGraph and the Anthropic Agent SDK add orchestration, state management, and multi-agent support — useful for production, overkill for a prototype. Start simple.

Q: What is the difference between an AI agent and an MCP server? A: An MCP server provides tools. An agent uses them. The server exposes capabilities (search, write, query) through a standard protocol. The agent decides which tools to call, in what order, and what to do with the results. One agent might hit dozens of MCP servers in a single workflow.

Q: Are AI agents safe to use in production? A: With guardrails, yes. You need tool-level permissions, output validation, rate limiting, and human-in-the-loop checkpoints for high-stakes actions. The real risk isn’t that agents are unreliable — it’s that developers skip the guardrails because the demo worked fine without them.