Managing Knowledge in CrewAI

In CrewAI, Knowledge is how you give agents access to external information (files, web content, structured data) in a reusable, searchable way. You can think of it as a vectorized reference library that sits beside your agents and crews, powering retrieval‑augmented generation (RAG).

1. What “Knowledge” Means in CrewAI

Knowledge lets agents:

Use domain‑specific information instead of relying only on the base LLM.
Support decisions with grounded, real‑world data.
Maintain context across tasks and conversations.
Reduce hallucinations by retrieving relevant chunks from your own sources.

Under the hood, CrewAI embeds your content into a vector store (ChromaDB by default) and queries it when tasks run.

2. Quickstart: Basic Knowledge Examples

2.1 String knowledge (simple, in‑memory)

# python
from crewai import Agent, Task, Crew, Process, LLM
from crewai.knowledge.source.string_knowledge_source import StringKnowledgeSource

content = "Users name is John. He is 30 years old and lives in San Francisco."
string_source = StringKnowledgeSource(content=content)

llm = LLM(model="gpt-4o-mini", temperature=0)

agent = Agent(
    role="About User",
    goal="You know everything about the user.",
    backstory="You are a master at understanding people and their preferences.",
    verbose=True,
    allow_delegation=False,
    llm=llm,
)

task = Task(
    description="Answer the following questions about the user: {question}",
    expected_output="An answer to the question.",
    agent=agent,
)

crew = Crew(
    agents=[agent],
    tasks=[task],
    verbose=True,
    process=Process.sequential,
    knowledge_sources=[string_source],  # crew-level knowledge
)

result = crew.kickoff(
    inputs={"question": "What city does John live in and how old is he?"}
)

The agent answers from the knowledge store, not just its prior training.

2.2 Web content knowledge

python
from crewai import Agent, Task, Crew, Process, LLM
from crewai.knowledge.source.string_knowledge_source import StringKnowledgeSource

content = "Users name is John. He is 30 years old and lives in San Francisco."
string_source = StringKnowledgeSource(content=content)

llm = LLM(model="gpt-4o-mini", temperature=0)

agent = Agent(
    role="About User",
    goal="You know everything about the user.",
    backstory="You are a master at understanding people and their preferences.",
    verbose=True,
    allow_delegation=False,
    llm=llm,
)

task = Task(
    description="Answer the following questions about the user: {question}",
    expected_output="An answer to the question.",
    agent=agent,
)

crew = Crew(
    agents=[agent],
    tasks=[task],
    verbose=True,
    process=Process.sequential,
    knowledge_sources=[string_source],  # crew-level knowledge
)

result = crew.kickoff(
    inputs={"question": "What city does John live in and how old is he?"}
)

CrewDoclingSource ingests and chunks remote documents; queries run over that embedded content.

3. Supported Knowledge Sources

CrewAI ships multiple knowledge adapters so you can plug in different file types.

All files are usually placed in a ./knowledge folder in your project root for convenience.

3.1 Text sources

Raw string content – StringKnowledgeSource
Text files (.txt) – TextFileKnowledgeSource

from crewai.knowledge.source.text_file_knowledge_source import TextFileKnowledgeSource

text_source = TextFileKnowledgeSource(
    file_paths=["document.txt", "another.txt"]
)

3.2 PDF

from crewai.knowledge.source.pdf_knowledge_source import PDFKnowledgeSource

pdf_source = PDFKnowledgeSource(
    file_paths=["document.pdf", "another.pdf"]
)

3.3 Structured data: CSV, Excel, JSON

from crewai.knowledge.source.csv_knowledge_source import CSVKnowledgeSource
from crewai.knowledge.source.excel_knowledge_source import ExcelKnowledgeSource
from crewai.knowledge.source.json_knowledge_source import JSONKnowledgeSource

csv_source = CSVKnowledgeSource(file_paths=["data.csv"])
excel_source = ExcelKnowledgeSource(file_paths=["spreadsheet.xlsx"])
json_source = JSONKnowledgeSource(file_paths=["data.json"])

You can combine multiple sources in a single crew or agent.

4. Agent‑Level vs Crew‑Level Knowledge

Knowledge can live at agent level or crew level, and both can coexist.

4.1 Agent‑only knowledge (independent)

from crewai import Agent, Task, Crew
from crewai.knowledge.source.string_knowledge_source import StringKnowledgeSource

specialist_knowledge = StringKnowledgeSource(
    content="Specialized technical information for this agent only"
)

specialist_agent = Agent(
    role="Technical Specialist",
    goal="Provide technical expertise",
    backstory="Expert in specialized technical domains",
    knowledge_sources=[specialist_knowledge],  # agent-specific
)

task = Task(
    description="Answer technical questions",
    agent=specialist_agent,
    expected_output="Technical answer",
)

crew = Crew(agents=[specialist_agent], tasks=[task])
result = crew.kickoff()

The agent uses its own knowledge even if the crew has none configured.

4.2 Crew‑wide + agent‑specific

from crewai.knowledge.source.string_knowledge_source import StringKnowledgeSource

crew_knowledge = StringKnowledgeSource(
    content="Company policies and general information for all agents"
)

specialist_knowledge = StringKnowledgeSource(
    content="Technical specifications only the specialist needs"
)

specialist = Agent(
    role="Technical Specialist",
    goal="Provide technical expertise",
    backstory="Technical expert",
    knowledge_sources=[specialist_knowledge],
)

generalist = Agent(
    role="General Assistant",
    goal="Provide general assistance",
    backstory="General helper",
)

crew = Crew(
    agents=[specialist, generalist],
    tasks=[...],
    knowledge_sources=[crew_knowledge],  # crew-level
)

Result

Specialist sees: crew_knowledge + specialist_knowledge.
Generalist sees: crew_knowledge only.

4.3 Multiple agents with different knowledge + embedders

You can give each agent its own knowledge + embedding provider, with a crew‑level fallback.

sales_knowledge   = StringKnowledgeSource(content="Sales procedures and pricing")
tech_knowledge    = StringKnowledgeSource(content="Technical documentation")
support_knowledge = StringKnowledgeSource(content="Support procedures")

sales_agent = Agent(
    role="Sales Representative",
    knowledge_sources=[sales_knowledge],
    embedder={"provider": "openai", "config": {"model": "text-embedding-3-small"}},
)

tech_agent = Agent(
    role="Technical Expert",
    knowledge_sources=[tech_knowledge],
    embedder={"provider": "ollama", "config": {"model": "mxbai-embed-large"}},
)

support_agent = Agent(
    role="Support Specialist",
    knowledge_sources=[support_knowledge],
    # uses crew embedder fallback
)

crew = Crew(
    agents=[sales_agent, tech_agent, support_agent],
    tasks=[...],
    embedder={  # fallback embedder
        "provider": "google-generativeai",
        "config": {"model_name": "gemini-embedding-001"},
    },
)

Each agent gets only its own knowledge and can use a different embedding stack.

5. Knowledge Configuration and Storage

5.1 KnowledgeConfig: controlling retrieval

You can tune how much and how strict knowledge retrieval is.

from crewai.knowledge.knowledge_config import KnowledgeConfig
from crewai import Agent

knowledge_config = KnowledgeConfig(
    results_limit=10,
    score_threshold=0.5,
)

agent = Agent(
    ...,
    knowledge_config=knowledge_config,
)

Core parameters include:

results_limit – max number of chunks returned per query.
score_threshold – minimum similarity score.
Other fields (e.g., collection names, storage config) via KnowledgeStorage.

5.2 Where knowledge is stored (ChromaDB)

By default, CrewAI stores knowledge in a ChromaDB directory under a platform‑specific path.

macOS:

~/Library/Application Support/CrewAI/{project_name}/knowledge/

Linux:

~/.local/share/CrewAI/{project_name}/knowledge/

Windows:

C:\Users\{username}\AppData\Local\CrewAI\{project_name}\knowledge\

Each collection has its own folder; agent and crew collections are separate.

You can inspect the path with:

import os

knowledge_path = os.path.join(db_storage_path(), "knowledge")
print("Knowledge storage location:", knowledge_path)

5.3 Controlling storage location

Option 1: Environment variable (recommended)

import os
from crewai import Crew

os.environ["CREWAI_STORAGE_DIR"] = "./my_project_storage"

crew = Crew(
    agents=[...],
    tasks=[...],
    knowledge_sources=[...],
)

Option 2: Custom KnowledgeStorage

from crewai.knowledge.storage.knowledge_storage import KnowledgeStorage
from crewai.knowledge.source.string_knowledge_source import StringKnowledgeSource

custom_storage = KnowledgeStorage(
    embedder={
        "provider": "ollama",
        "config": {"model": "mxbai-embed-large"},
    },
    collection_name="my_custom_knowledge",
)

knowledge_source = StringKnowledgeSource(
    content="Your knowledge content here",
)
knowledge_source.storage = custom_storage

Option 3: Project‑local directory

import os
from pathlib import Path

project_root = Path(__file__).parent
knowledge_dir = project_root / "knowledge_storage"
os.environ["CREWAI_STORAGE_DIR"] = str(knowledge_dir)

6. Embedding Providers for Knowledge

By default, knowledge embeddings use OpenAI text-embedding-3-small, even if your LLM is from another provider.

You can override this at crew or agent level.

6.1 Matching embeddings to your stack

Example – Voyage AI with Claude:

crew = Crew(
    agents=[agent],
    tasks=[...],
    knowledge_sources=[knowledge_source],
    embedder={
        "provider": "voyageai",
        "config": {
            "api_key": "your-voyage-api-key",
            "model": "voyage-3",
        },
    },
)

Local embeddings (no external API):

crew = Crew(
    agents=[agent],
    tasks=[...],
    knowledge_sources=[knowledge_source],
    embedder={
        "provider": "ollama",
        "config": {
            "model": "mxbai-embed-large",
            "url": "http://localhost:11434/api/embeddings",
        },
    },
)

Agent‑level override (e.g., Gemini embeddings):

agent = Agent(
    role="Researcher",
    goal="Research topics",
    backstory="Expert researcher",
    knowledge_sources=[knowledge_source],
    embedder={
        "provider": "google-generativeai",
        "config": {
            "model_name": "gemini-embedding-001",
            "api_key": "your-google-key",
        },
    },
)

Azure OpenAI embeddings:

agent = Agent(
    role="Researcher",
    goal="Research topics",
    backstory="Expert researcher",
    knowledge_sources=[knowledge_source],
    embedder={
        "provider": "azure",
        "config": {
            "api_key": "your-azure-api-key",
            "model": "text-embedding-ada-002",
            "api_base": "https://your-azure-endpoint.openai.azure.com/",
            "api_version": "2024-02-01",
        },
    },
)

7. Advanced Features: Query Rewriting, Events, and Custom Sources

7.1 Query rewriting (better retrieval)

CrewAI automatically rewrites queries to improve retrieval quality.

When a task uses knowledge, getknowledge_search_query runs.
The agent’s LLM turns the full task prompt into a concise search query.

Example transformation:

Original prompt:
"Answer the following questions about the user's favorite movies:
What movie did John watch last week? Format your answer in JSON."

Rewritten search query:
"What movies did John watch last week?"

This improves recall and relevance in vector search.

7.2 Knowledge events (observability)

CrewAI emits events around knowledge retrieval, e.g.:

KnowledgeRetrievalStartedEvent
KnowledgeRetrievalCompletedEvent
KnowledgeQueryStartedEvent / CompletedEvent / FailedEvent

You can subscribe using the event bus to monitor how knowledge is used.

Check initialization:

print("Before kickoff:", getattr(agent, "knowledge", None))
crew.kickoff()
print("After kickoff:", agent.knowledge)

Verify storage location and contents using db_storage_path() and listing the knowledge folder.
Use ChromaDB’s client to inspect collections and sample documents.

8.2 Resetting knowledge

Use the crew API or CLI:

# In code
crew.reset_memories(command_type="agent_knowledge")  # just agent
crew.reset_memories(command_type="knowledge")        # crew + agent

# CLI
# crewai reset-memories --agent-knowledge
# crewai reset-memories --knowledge

This is useful when you update documents and want a fresh embedding pass.

9. Best Practices for an Agentic Architect

Here are some key patterns:

Use agent‑level knowledge for role‑specific “playbooks” (sales, support, tech).
Use crew‑level knowledge for shared policies, company docs, or product info.
Choose embedding providers explicitly, especially in production.
Keep a dedicated knowledge/ folder and treat it like a content repo.
Monitor storage growth and include knowledge dirs in backups and deployments.
Reset knowledge when major content changes, and test retrieval with small queries.

Framed simply:

Memory is what agents learn over time;
Knowledge is the library you preload so they can answer with accurate, grounded information.