LLM Agents Ecosystem Handbook
A practical operating manual for building, evaluating, securing, and shipping modern LLM agent systems.
Modern agents are not "a prompt + a tool." They are systems — with identity, memory, skills, tools, MCP integrations, guardrails, observability, evals, and a provider strategy. This handbook teaches the whole stack and ships templates, blueprints, runnable adapters, and curated examples you can adopt today.
What's in this repo
A curated, opinionated, production-oriented handbook in seven parts:
- Concepts — Agent OS, identity, memory, skills, MCP, safety, observability — every layer of the modern agent stack
- Provider ecosystem — adapters + docs for 24+ LLM providers (frontier APIs, fast inference, marketplaces, enterprise clouds, specialty, local runtimes), with a router for fallback chains
- Skills ecosystem — design guide, taxonomy, maturity model, security checklist, and a curated skill catalog
- Prompt engineering — agent prompt patterns, instruction hierarchy, context engineering, prompt-injection defense
- Coding-agent workflows — for Claude Code, Cursor, Codex, Aider, Cline, and custom runtimes — repo instructions, prompts, review checklist, safe refactoring
- Design docs — agent / technical design docs, ADR guide, design reviews, rollout plans, the
DESIGN.mdmachine-readable spec - Curated catalog — 100+ existing agent skeletons, framework comparisons, evaluation tools, tutorials — preserved and improved
Who this is for
| You are… | Start at |
|---|---|
| New to agents | docs/beginners_guide.md → agent_os/README.md |
| Building a production agent | blueprints/ → checklists/production_readiness_checklist.md |
| Picking / wiring providers | providers/README.md → providers/provider_matrix.md |
| Comparing frameworks | docs/framework_comparison.md |
| Adding memory / RAG | memory/ → tutorials/rag_tutorials |
| Adding MCP | mcp/ → mcp/mcp_security.md |
| Designing Skills | skills/ → skills/skill_design_guide.md |
| Working with coding agents | coding_agents/ → coding_agents/prompts/ |
| Writing better prompts | prompt_engineering/ |
| Designing & rolling out | design_docs/ |
| Hardening safety/evals | safety/ → evals/ |
| Coding agent reading this repo | llms.txt → llm_wiki/index.md |
Modern Agent Stack
| Layer | Purpose | Where in this repo |
|---|---|---|
| Model / Provider | LLM choice + abstraction + routing | providers/ |
| Orchestration | Agent loops, planning, handoffs | docs/framework_comparison.md, blueprints/ |
| Tool | Function calling and external actions | agent_os/mcp_layer.md |
| MCP | Standardized external context and tools | mcp/ |
| Memory | Durable user/project/semantic memory | memory/ |
| Skills | Reusable, progressive-loading workflows | skills/ |
| Identity | Personality, mission, refusal style | agent_os/agent_identity.md, templates/ |
| Prompt | System prompt design, instruction hierarchy, defenses | prompt_engineering/ |
| Safety | Guardrails, approvals, policy | safety/ |
| Observability | Tracing, spans, cost, latency, evals | observability/, evals/ |
| Deployment | Shipping agents to production | design_docs/rollout_plan.md |
| Coding-agent harness | Claude Code, Cursor, Codex, Aider, Cline | coding_agents/ |
📖 Deep dive: agent_os/README.md
Provider ecosystem
The handbook ships an LLMProvider abstraction with 24+ providers across six families. Most providers go through a single OpenAI-compatible code path; specialty / local providers are first-class.
| Provider type | Examples | Best for |
|---|---|---|
| Frontier APIs | OpenAI, Anthropic, Google Gemini | Reasoning, tool use, production agents |
| Fast inference | Groq, Cerebras, SambaNova | Low-latency workloads |
| Marketplaces | OpenRouter, Together, Fireworks, DeepInfra | Model choice and routing |
| Enterprise clouds | Azure OpenAI, AWS Bedrock, Vertex AI | Compliance, governance |
| Specialty | xAI, Perplexity, Mistral, Cohere, DeepSeek, Hugging Face, Replicate, NVIDIA NIM, MiniMax | Domain-specific |
| Local runtimes | Ollama, LM Studio, vLLM, llama.cpp | Privacy, cost control, offline dev |
If you want a governed OpenAI-compatible control plane in front of those providers, Tuning Engines is a useful runtime option for policy enforcement, approval gates, MCP and agent tracing, and usage or cost visibility without changing the surrounding agent framework.
Quick start:
from utilities import get_provider
from utilities.provider_router import ProviderRouter
# Use any single provider
out = get_provider("groq").chat(
[{"role": "user", "content": "Summarize MCP."}],
model="llama-3.1-8b-instant",
)
# Or route by task class with fallback
router = ProviderRouter()
out = router.chat(messages, task_class="cheap") # Groq → DeepSeek → Together → OpenRouter
📖 providers/README.md • providers/provider_matrix.md • providers/router_patterns.md • providers/local_models.md
Repository map
.
├── README.md • llms.txt • llms-full.txt
├── agent_os/ ← the Agent OS concept, layers, workspace examples
├── providers/ ← 24+ provider docs + adapters + router patterns
├── templates/ ← AGENTS.md / SOUL.md / MEMORY.md / SKILL.md / DESIGN_DOC / ADR / …
├── skills/ ← design guide + taxonomy + maturity model + curated catalog + 4 examples
├── memory/ ← memory taxonomy, distillation, security, examples
├── mcp/ ← MCP basics, architecture, security, server catalog, examples
├── prompt_engineering/ ← agent prompt patterns, instruction hierarchy, defenses
├── coding_agents/ ← Claude Code, Cursor, Codex, workflows, prompts, review
├── design_docs/ ← agent + technical design docs, ADR guide, design.md spec
├── safety/ ← guardrails, approvals, prompt injection, secure checklist
├── observability/ ← tracing, spans, cost/latency, dashboards
├── evals/ ← eval design, regression / tool / memory / MCP / safety / prompt
├── blueprints/ ← production architectures by use case
├── examples/ ← end-to-end runnable agent workspaces
├── checklists/ ← agent design, prod readiness, MCP security, …
├── llm_wiki/ ← LLM-friendly index, glossary, matrices, wiki pattern
├── docs/ ← framework comparison, best practices, beginners' guide
├── tutorials/ ← RAG, memory, fine-tuning, chat-with-X
├── utilities/ ← LLMProvider + router + provider_config
├── agents/ ← 100+ curated agent skeletons (preserved)
├── complete_apps/, web_apps/, notebooks/, datasets/, design/, resources/, scripts/, tests/, ecosystem/
└── .github/ ← issue / PR templates
Skills ecosystem
A curated, in-repo catalog plus a clear taxonomy and maturity model:
- skills/skill_design_guide.md — write triggers the model picks
- skills/skill_vs_tool_vs_mcp.md — when to use which
- skills/skill_taxonomy.md — domains, tags, risk
- skills/skill_maturity_model.md — experimental → production
- skills/skill_packaging.md — ship a portable skill
- skills/skill_validation.md — lint / smoke / eval
- skills/awesome_skills_catalog.md — broader ecosystem map
- skills/catalog/ — index + per-domain skills
- skills/examples/ — four full reference skills
Curated skills shipped: research-summarizer, repo-auditor, mcp-security-reviewer, agent-memory-curator, api-design-reviewer, pr-summarizer, adr-writer, incident-postmortem, sprint-planner, dataset-profiler.
Prompt engineering
A dedicated section, agent-focused:
- prompt_engineering/agent_prompt_patterns.md
- prompt_engineering/system_prompt_design.md
- prompt_engineering/instruction_hierarchy.md
- prompt_engineering/context_engineering.md
- prompt_engineering/tool_use_prompting.md
- prompt_engineering/planning_and_reflection.md
- [prompt_engineering/memory_prompting.md](prompt_engineerin