Lesson 24: Human-in-the-Loop Through MCP Servers
In the previous lesson, we used the evaluator-optimizer pattern to automatically refine articles against the writing guidelines. While these automated loops are powerful, writing is inherently subjective. Even the most sophisticated AI models hallucinate and miss the nuance of your personal voice or specific instructions in your input. This escalates further in our use case, where it must follow 100+ instructions simultaneously. The reality is that AI is not yet capable of producing perfect results. But that’s fine. Its goal is not to replace you, but to automate your repetitive tasks, allowing you to focus on what makes the final output special and creative.
That’s why, to build a practical writing assistant, we cannot rely solely on automation. We need a way to intervene, review, and steer the AI. The perfect balance involves using AI to generate and automate the heavy lifting, while you, the domain expert, review and refine the output.
In this lesson, we will finalize the Brown capstone project by adding this essential human element. We will cover:
- The AI generation / human validation loop and why it is essential for practical AI applications.
- How to introduce human feedback into the article reviewer node.
- Implementing two new editing workflows:
edit_articleandedit_selected_text. - Serving Brown as an MCP server with tools, prompts, and resources.
- Integrating Brown with MCP clients like Cursor for a coding-like writing experience.
Understanding the AI Generation Human Validation Loop
AI systems are imperfect. LLMs can display “jagged intelligence,” performing brilliantly on complex reasoning tasks while failing at simple instructions that no human would miss. They can hallucinate facts or misinterpret tone. Therefore, we cannot simply delegate thinking, creating, and planning entirely to the AI.
To build reliable applications, we must design a cooperative workflow in which the human handles planning and high-level thinking, while the AI handles execution.
We call this the AI Generation / Human Validation Loop :
- Generation: The AI generates a draft or performs an action.
- Validation: A human reviews the output, validates correctness, manually edits parts of it if necessary, and provides feedback.
- Iteration: The loop repeats until the human is satisfied.
Image 1: The AI Generation - Human Validation Loop
Andrej Karpathy, former Director of AI at Tesla, emphasizes two key strategies to make this loop effective [1]:
Speeding Up Verification
The bottleneck in this loop is often the human’s ability to verify the AI’s work. To expedite this process, we require application-specific interfaces that utilize our visual processing capabilities. Reading raw text output is slow and cognitively demanding. It forces the brain to parse syntax and semantics linearly.
Instead, presenting changes as visual “diffs”, like red and green lines in code editors, allows humans to audit changes instantly. This visual representation acts as a highway to the brain, bypassing the slow process of reading. We should also enable direct actions. Users should be able to accept or reject changes with a single click or keyboard shortcut, rather than typing out new instructions in a chat box. This reduces friction and keeps the verification loop spinning fast.
We saw this issue in the previous lesson when we tried to understand how the evaluator’s reviews were applied to the new article. The process was cumbersome. That’s why, in this lesson, we use MCP clients like Cursor to automatically diff the old and improved articles.
Keeping the AI on a Leash by Avoiding “Overreaction”
We must avoid “overreaction” from the AI. Receiving a 10,000-line code refactor from an agent is not helpful because verifying it takes longer than writing it yourself. The AI might introduce subtle bugs or security issues that are buried deep within the massive output.
To keep the loop tight, we should constrain the AI to work in small, incremental chunks, focusing on a single, concrete task at a time. We also need to use concrete prompts to increase the probability of a successful generation. Vague prompts lead to vague outputs that fail verification and force the user to restart the loop. By using auditable intermediate artifacts, such as a course syllabus before a full course, we constrain the AI and prevent it from getting “lost in the woods.”
Again, we encountered this issue in our use case when editing the entire article. When fully automated, that can be useful, but when we introduce a human-in-the-loop, we want to improve only the scoped parts of the article. Here is where the selected text logic, which we began implementing in the previous lesson, comes in.
The “Keeping the AI on a Leash” concept directly connects to the “autonomy slider” we discussed in the “Workflows vs. Agents” lesson at the beginning of the course. As you increase an agent’s autonomy, you decrease human control and reliability. Your job as an AI engineer is to determine how much autonomy to give your AI app, which directly translates into a trade-off between more or less control when introducing a human-in-the-loop.
Image 2: A diagram illustrating Andrej Karpathy's "autonomy slider" concept, showing the inverse relationship between an AI agent's level of control and application reliability.
To conclude with a fun analogy from Karpathy. He suggests building products that are “less Iron Man robots and more Iron Man suits.” We aim to augment humans with a rapid generation-verification loop rather than replace them with a fully autonomous agent [1]. In a sentence, that’s our moat behind Nova and Brown.
Now, let’s understand how we can apply these principles to Brown.
Adding Human-In-The-Loop In Our Writing Workflow
We have the generation step from Lessons 22 and 23, but we lack a mechanism for the human to intervene after the initial draft is generated. We need a way, after the article is generated, to allow humans to review it, validate it, and provide further instructions on how to improve or modify it.
Writing content is incredibly similar to writing code. Both processes involve drafting, reviewing, refactoring, and polishing. As engineers, we can adopt the “coding-like” experience found in AI IDEs like Cursor. In these tools, you highlight a section of code, provide instructions (e.g., “refactor this to use async”), and the AI returns a diff of the changes. We want this exact experience for writing articles. We want to highlight a paragraph, ask for a tone shift, and see the changes applied instantly.
To achieve this, we implemented two new editing workflows that apply the evaluator-optimizer pattern with human feedback:
- Edit Article Workflow: Reviews and edits the entire article based on human feedback.
- Edit Selected Text Workflow: Reviews and edits only a specific portion of the text (still based on human feedback).
We instructed our evaluator-optimizer loop to prioritize human feedback over the static profile guidelines. This ensures that if you explicitly request a change, the AI respects your decision, even if it contradicts a general rule.
As shown in Image 3, the editing workflow applies the reviewing and editing step only once. We completely trimmed down the loop from the original article generation workflow. Because, as we presented in the previous section, at this point, the loop is naturally created between you and the AI, not between two LLMs.
Image 3: Brown’s new edit article and selected text workflows, which work in parallel with the article generation workflow presented in previous lessons.
To anchor how these new editing workflows relate to the article generation workflow implemented in the previous lesson, refer to image 4. The main idea is that we reuse the same article reviewer and writer nodes and static inputs, such as the profiles and examples. They are merely combined into a different format at the application level.
Image 4: The complete article generation workflow, with the orchestrator-worker and evaluator-optimizer patterns.
To be more hands-on, to enable this flexible approach between generating the first draft and further editing the article, we decoupled Brown into three independent workflows:
generate_articleedit_articleedit_selected_text
Also, to expose these workflows to clients such as Cursor or Claude, we served the entire Brown application as an MCP server that wraps each workflow as an MCP tool.
This architecture enables you to generate a draft, review it, and then apply editing workflows selectively until you are satisfied. By exposing these as independent tools, we are not tied to a single monolithic script.
Image 5: Flowchart illustrating the human in the loop flow leveraging the new edit tools.
We specifically added the edit_selected_text workflow to avoid “overreaction.” Editing the whole article when you only want to change one paragraph is inefficient and risky. The AI might accidentally change sections you liked. By focusing on selected text, we keep the AI on a leash.
💡To see how the design presented in this lesson relates to Brown's overall architecture, open the whiteboard below.
Live Whiteboard: The End-To-End Architecture of Brown: The Writing Workflow. Click here to expand.
Now, let’s get into the implementation. We will start by introducing the human feedback logic into the entities and nodes. Then we will implement the two new editing LangGraph workflows and ultimately serve everything as an MCP server.
Introducing Human Feedback Into the Article Reviewer
Let’s start by looking at how we modified our ArticleReviewer node to accept human feedback.
- Next, we updated the
ArticleReviewerclass to accept an optionalhuman_feedbackparameter during initialization. This enables the reviewer to operate with or without human input, ensuring backward compatibility with our automated workflows.
- We updated the system prompt to include explicit instructions for handling this feedback. We instruct the LLM to ignore it if it’s empty. But if it’s present, we want to decompose it across one or more human feedback reviews. The idea behind this design choice is that for every independent piece of human feedback, we want to create a distinct Review object that is addressed independently. For example, if we get the human feedback “Make the introduction more engaging and catchy. Also, expand on the definition of both workflows and agents from the first section,” we expect two
Reviewobjects from the article reviewer. One that addresses the introduction and another one that addresses the issue from the first section. __ Also, note how we reduced the human feedback to the sameReviewinterface, as with the other types of reviews, but withprofile="human_feedback". By doing so, we can easily leverage all the infrastructure we used for the evaluator-optimizer pattern.
- In the
ainvokemethod, we inject the feedback into the prompt template. If no feedback exists, we inject an empty string.
- Let’s see this in action. We load our sample article and profiles, create a
HumanFeedbackobject with specific instructions, and run the reviewer.
When we examine the output, we see that the reviewer creates specific reviews with profile= “human_feedback” that directly addresses our request, alongside the standard profile-based reviews for structure and mechanics. Exactly what we wanted!
Now, let’s integrate this into the editing workflows, starting with the one for editing the whole article.
Implementing the Article Editing Workflow
This workflow orchestrates the review and editing process for the entire article based on your feedback. You might think this violates the “overreaction” principle, and it does. However, based on our tests, this is still necessary if the initial number of review/editing iterations used to generate the article is insufficient. Thus, this is useful when you don’t want to check the final output too carefully, but you realize it still needs more refinement at the global level before starting to edit more targeted parts of the article.
As we have said throughout the course. You should always question “the book” and apply only what makes sense for your app. In our use case, that means the next workflow, which edits only a specific part of the text.
Image 6: The Brown project structure.
Because of our clean architecture, we can easily reuse all entities and nodes from the domain layer and recompose them into different business logic in the app layer. In other words, we will use LangGraph to orchestrate the same components into different workflows, such as gluing the ArticleWriter and ArticleReviewer into a simple editing workflow. In software, this is known as building orthogonal components that make sense on their own, and you can reuse them as LEGO blocks. This pattern is extremely popular in frameworks like PyTorch, which provide the building blocks you can use to build whatever you want.
Now, back to our implementation.
- We follow the exact same pattern as we did when building the article generation workflow. Thus, we will expand only on what’s new here. We use LangGraph’s Functional API to orchestrate the workflow. The
build_edit_article_workflowfunction acts as a factory, creating an entry point with a checkpointer for state persistence of its short-term memory.
The input definition is a TypedDict with the input directory and human feedback:
- The main workflow function,
_edit_article_workflow, manages the sequence. It accepts theEditArticleInputalong with the standard LangChainconfig. Within it, we first initialize the stream writer and load the context.
- Next, it runs the review and editing steps. We use the
generate_reviewstask to get feedback and then pass that toedit_based_on_reviewsto apply changes (which we will expand soon).
- Finally, it returns the edited article along with specific instructions for the workflow consumer, such as the MCP client (like Cursor). These instructions are critical because they tell the client how to interpret the results. For example, the
always apply the changes to the source...bullet point is the prompt engineering magic that tells the MCP client to do a diff between the old and new article. This works because this workflow is always called relative to thesource file. Thus, the workflow consumer has in context the concept of the old and new article.
For example, this is how the “diff” logic looks when using Brown within Cursor with human_feedback= “make sure the article adheres to the article_guideline 100%”(We will show how this works at the end of the lesson.)
Image 7: A “diff” in Cursor after calling the edit article workflow through MCP from Cursor.
As you can see, we can fully leverage all Cursor’s built-in features.
- The
generate_reviewstask wraps theArticleReviewer. It uses the@taskdecorator for checkpointing and retries. Note that it specifically uses the “review_article” configuration from our app config. This allows us to tune the model parameters specifically for reviewing, separate from generation.
- The
edit_based_on_reviewstask wraps theArticleWriter. Importantly, when we providereviewsto theArticleWriter, it switches to “editing mode.” It uses the reviews to guide changes rather than writing from scratch. This reuse of theArticleWriternode demonstrates the power of our modular design.
- To run this workflow, we initialize it with an in-memory checkpointer and stream the events. The output shows the progression from loading context to generating reviews and finally producing the edited article.
It outputs:
This workflow shows the power of human feedback. Instead of running 10 automated review/editing loops in hopes of a perfect result, we use fewer review/editing loops in our initial article generation (2 in our use case). Then the user enters and decides whether more is required. This makes the initial generation faster and cheaper. We don’t assume how many iterations are needed. We let you choose when the article is done.
Implementing the Selected Text Editing Workflow
While editing the entire article is useful for broad changes, you often want to refine a specific section without risking mental overload from changes across the entire document.
The edit_selected_text workflow enables this precision. Its structure is nearly identical to edit_article, but it operates on a SelectedText entity instead of a full Article.
- We build the workflow using the same factory pattern.
The input EditSelectedTextInput requires the selected_text string and its line numbers (number_line_before_selected_text, number_line_after_selected_text). These line numbers help the AI locate the text within the article’s larger context, ensuring it understands what precedes and follows the selection to avoid editing the text in a vacuum (remember, context is key!):
- The main logic mirrors the article workflow, but creates a
SelectedTextentity. This entity holds the snippet we want to edit while keeping a reference to the full article for context. Like this, we ensure the selected text is always coupled with the article it originated from.
- We then proceed with the review and edit steps, similar to the article workflow, but operating on the
selected_textobject.
- The returned string is similar to the previous one but is tailored to the selected text, instructing the client to apply changes only to that specific section.
Here is an example of the “diff” logic when running the edit selected text workflow within Cursor. Note that this time the green bars appear only on a specific section of the article.
Image 8: A “diff” in Cursor after calling the edit selected text workflow through MCP from Cursor.
- The tasks
generate_reviewsandedit_based_on_reviewsare adapted to handleSelectedText. TheArticleReviewerandArticleWriternodes are polymorphic. They behave differently depending on whether they receive anArticleorSelectedText. When receivingSelectedText, the reviewer focuses its critique only on that snippet, ignoring the rest of the document.
- The
ArticleWriterreturns aSelectedTextobject containing only the edited portion. This ensures that the output is strictly limited to what the user requested to change.
Why is this important? As we said in the beginning, editing only the selected text prevents “overreaction.” If you want to rewrite a single paragraph, you don’t want the AI to rephrase your introduction or change your conclusion. It allows for faster, cheaper, and safer iterations. You can surgically improve parts of your content without destabilizing the whole.
- Running this workflow follows the same pattern. We extract the text we want to edit (e.g., lines 8-42 of our sample), pass it to the workflow along with feedback, and receive the refined snippet (this will be 10x smoother when using Cursor or Claude as the MCP Client):
Based on the input selected text:
Image 1: A flowchart illustrating a deterministic LLM workflow with clear start and end points, including an LLM call and data operations.
AI Agents
AI agents are systems where an LLM dynamically decides the sequence of steps, reasoning, and actions to achieve a goal. The path is not predefined. Instead, the agent uses a reasoning process to plan its actions based on the task and the current state of its environment. This process is often modeled on frameworks like ReAct (Reason, Act, Observe). This allows agents to be adaptive and capable of handling new or unexpected situations through LLM-driven autonomy. They can select tools, execute actions, evaluate the outcomes, and correct their course until the goal is achieved [1].
It outputs:
Finally, the last step to make interacting with Brown smooth is to run it as an MCP server.
Serving Brown as an MCP Server
To integrate these workflows into tools like Cursor or Claude Desktop, we serve them using MCP. We implemented the server in the brown.mcp module, keeping the serving layer completely separate from our domain and app logic. Which means all the other modules from brown are not aware of brown.mcp module, while the brown.mcp module imports all the domain, app, and infrastructure components it needs from the rest of the application. This clean separation means we can easily replace MCP with a REST API or a CLI in the future without touching the core workflows [3].
Now, let’s start implementing the MCP server.
Tools
- We initialize the MCP server using
FastMCP, similar to how we did in the Nova research agent. Similarly, we have a dedicated lesson on MCP. In this lesson, we will expand only on the particularities used in Brown.
We define three tools corresponding to our workflows. Each tool handles the setup of the workflow, including creating a unique thread ID for state management, the checkpointer, and custom handling of the events within the parse_message function.
Another essential aspect is that the tool definition includes type hints and docstrings, which the MCP client uses to determine how to call the tool. The standard practices for defining the tools consumed by tool calling models.
- We also expose the
edit_articletool, which takes the article path and human feedback as input.
- Finally, the
edit_selected_texttool allows for precise edits. It requires the selected text and its line numbers.
- We use a helper function
parse_messageto interpret the workflow’s streaming events and report progress back to the MCP client through thectxFastMCP object. Doing so, the MCP client can further report these events on a UI, ensuring the user sees real-time updates (e.g., “20%: Reviewing article...”, “30%: Editing...") directly in their dashboard. This feedback is essential for long-running tasks, letting the user know the system is working as expected.
- Let’s see this in action. We can simulate an MCP client in memory to verify our tools work as expected. We create an MCP client connected to our
mcpMCP Server instance, load the text we want to edit, and call theedit_selected_texttool.
The client receives real-time progress logs:
Followed by the final output (the same as we had when we directly called the edit_selected_text_workflow a few sections above):
Prompts
We also exposed three MCP prompts,**** decorated by @mcp.prompt ,**** such as generate_article_prompt, which enable users to trigger complex workflows using preconfigured templates. Instead of remembering how to call, trigger, or configure a particular tool, such as what arguments it requires, you just call a prompt, and it will take care of everything.
Here is what the prompts for editing the article look like:
We can verify the prompt retrieval using our client:
It outputs the formatted message that the client (e.g., Cursor’s chatbot) will use to instruct the LLM:
And similarly for edit_selected_text_prompt:
Which returns:
Remember, these prompts are like recipes that help the MCP client understand how to use the tools from the MCP server. Here, we had only one tool call per recipe, but these can get quite complex, enumerating complex business use cases that aggregate dozens of tools. That’s what we had in the Nova research agent: we compiled the entire research plan into a single plan.
As a useful analogy for memory, you can view the prompts as procedural memory because they instruct the LLM how to use its available actions (tools) to perform a complex task, which is ultimately the definition of procedural memory.
Now, let’s understand the last core component of an MCP server: resources.
Resources
In our use case, we leveraged the resources feature to expose static files used to configure Brown to the client, such as writing profiles and app configuration. In this way, the user can easily understand how Brown is configured. Why is this useful? In a real-world scenario, we will likely have multiple app configurations and writing profiles for social media posts, video transcripts, or different characters. Thus, to understand the current state of the MCP server, it’s essential to know what you are using.
Thus, we exposed resources like resource://config/app and "resource://config/character_profile" to allow the client to read the current configuration or character profile without executing any code. We did that through the @mcp.resource decorator. You could easily extend this functionality to expose all static inputs and configurations.
We can list all the resources:
It outputs:
Or get a single resource:
It outputs the JSON configuration:
We wrapped up the MCP Server implementation. The last step is to understand how to use it effectively.
Hooking to the MCP Server
There are two main ways to connect to Brown’s MCP server: using the built-in CLI script or integrating it directly into Cursor.
Option 1: Brown CLI Script
Brown includes a CLI utility at lessons/writing_workflow/scripts/brown_mcp_cli.py. This script uses an in-memory MCP client to call the tools, which is great for testing or running workflows from the terminal.
It supports all the workflows as follows:
- Generate the article:
- Edit the entire article:
- Edit the selected text:
For more details on running these CLI commands while using Brown as a standalone project, see the documentation from lessons/writing_workflow from the GitHub repository.
Option 2: Cursor Integration
The most powerful way to use Brown is within Cursor. You configure the server in your .cursor/mcp.json file, telling Cursor how to launch the Brown Python project.
Once connected, you get a fluid Human-in-the-Loop Writing Experience :
- Generate: You ask Cursor to “Generate an article based on the guidelines in this folder.”
- Review: You read the output in the editor.
- Feedback: You highlight a paragraph that feels off and type, “This is too formal, make it more conversational.”
- AI Edits: Brown’s
edit_selected_texttool runs, and Cursor presents a diff view. - Iterate: You accept the change and move to the next section.
This workflow puts you in the driver’s seat. The AI handles the grunt work of drafting and rewriting, while you guide the direction and quality. Note that for now, we are using local stdio transport. We will cover remote HTTP-based deployments in Part 3.
Image 9: Screenshot of how Brown looks within Cursor when hooked as an MCP server.
We continued to use Cursor as an example because it was the one we used most to test the application. But as MCP Servers are entirely decoupled from the MCP client, you can easily use other tools to hook to Brown, such as Claude, Windsurf, or any other MCP client of your choice.
Running Brown Through the MCP Server (Video)
Check out the video below for a quick demo of how we used Brown, together with Cursor, to edit an existing article using our editing article and selected-text MCP tools.
Conclusion
Understanding how to add humans to the loop properly is essential for building practical AI applications because AI systems are imperfect. They hallucinate and make reasoning mistakes. The balance we’ve built allows the AI to handle the heavy lifting of generation while keeping humans in control of validation and direction.
In this lesson, we learned:
- The AI Generation / Human Validation Loop: How to design workflows that balance automation with human expertise.
- Edit Article Workflow: Implementing a workflow to refine entire documents based on feedback.
- Edit Selected Text Workflow: Building a precision tool for focused edits, reducing the risk of unwanted changes.
- MCP Server Integration: Serving our application logic through a standardized protocol with tools, prompts, and resources.
- Cursor Integration: Creating a coding-like writing experience.
This wraps up our exploration of the Brown writing workflow. We started with the foundations in Lesson 22, applied the evaluator-optimizer pattern in Lesson 23, and finalized it with human-in-the-loop integration here in Lesson 24. In Part 2D, we will take the final step: orchestrating both Nova (our researcher) and Brown (our writer) into a single, unified system. Then, in Part 3, we will shift gears to production engineering, covering AI Evals, monitoring, and deployment.
Practicing Ideas
- Hooking Brown to Claude Desktop: Instead of using Cursor, integrate Brown with Claude Desktop for a different AI assistant experience.
- Use Resource Templates: Parameterize the writing profiles within the MCP resources and easily add support for all available profiles. Learn more here.
- Serve Brown through FastAPI: Replace the MCP server with a FastAPI REST API for web-based integrations.
- Add Guideline Review Tool: Create a workflow to review and edit your article guidelines before passing them to the article generation step.
References
- Karpathy, A. (2024, June 18). Software Is Changing (Again). YouTube. youtube.com/watch
- Schluntz, E., & Zhang, B. (2024, December 19). Building effective agents. Anthropic. anthropic.com/engineering/building-effective-agents
- The FastMCP Server. (n.d.). FastMCP. gofastmcp.com/servers/server
- Functional API. (n.d.). LangChain. docs.langchain.com/oss/python/langgraph/functional-api
- Use the functional API. (n.d.). LangChain. docs.langchain.com/oss/python/langgraph/use-functional-api