Select an appropriate LLM

The big idea

Choose models with a task-first, capability-mapped, test-and-iterate mindset; avoid “one model fits all” and align agents’ roles, backstories, and tools with the right LLMs to balance cost, speed, and reliability in real multiagent workflows.

Quick framework

• Start with task-first thinking: simple tasks need low-cost instruction followers, complex tasks need reasoning depth, and creative tasks need nuance and tone control; map requirements before picking models.
• Map capabilities by category: reasoning, general-purpose, fast/efficient, creative, and open-source; use strengths purposefully rather than brand reputation or generic benchmarks.
• Consider constraints: context window, tool/function-calling reliability, latency, cost, privacy/compliance, and enterprise deployment needs; optimize for the actual workflow.

Multi-model crews

• Use different models per role: a high-reasoning manager for planning/coordination, a creative model for content, and an efficient model for extraction and formatting; wire this explicitly in Agent llm configs and manager_llm at the crew level.
• Set a default crew LLM for baseline, then override only where essential (the 20% of agents doing 80% of the complex reasoning) to control cost and switching overheads.

Role, backstory, tools

• Define specific roles to justify model selection (e.g., “SaaS RevOps Analyst” merits a reasoning model); generic roles dilute performance and waste budget.
• Write rich backstories to amplify model context: domain experience, working style, quality standards, and decision frameworks; this reliably upgrades output quality without changing models.
• Match tool-heavy agents to models with strong function-calling and parameter extraction; don’t put a purely creative model in an API-integration role.

Hierarchy and inheritance

• Understand crew vs. manager vs. agent LLM hierarchy to avoid conflicts and unnecessary switching; plan manager_llm when using hierarchical processes, and let agents inherit unless a role truly needs an override.

Practical checklist

• Audit: identify agents doing complex reasoning vs. routine processing; find tool-heavy roles; check if all agents unnecessarily share one premium LLM.
• Baseline: set a cost-effective default crew LLM; then selectively upgrade critical agents (manager, customer-facing, high-stakes analysts).
• Validate: use AMP-style testing/A-B comparisons, run multiple iterations on real tasks, and weigh cost-performance; share results with collaborators for consistent standards.

What to avoid

• One-model-for-everything: misallocates cost and reduces reliability; diversify by role and task.
• Premature optimization: don’t implement elaborate switching rules before measuring real gains on real data; start simple, then refine.
• Context blind spots: short-context models break in memory-heavy, multi-turn, multi-agent settings; match context windows to communication patterns and CrewAI memory usage.

Model-by-goal guidance

• High-performance crews: use o3, Gemini 2.5 Pro, or Claude 4 Sonnet for manager/strategic agents; pair with efficient models for routine steps to contain spend.
• Cost-conscious crews: pick DeepSeek R1, Llama 4 Scout, or Gemini 2.0 Flash for most roles; reserve premium models for the few critical decisions.
• Specialized workflows: choose by core capability—Claude 4 for coding, Gemini 2.5 Pro for research, Llama 405B for function-calling reliability.
• Enterprise/privacy: consider open-source (Llama 4, DeepSeek V3, Qwen3) on private infra; accept possible performance trade-offs for data control.