Lesson 23: Reviewing and Editing Through the Evaluator-Optimizer Pattern

Explaining the Evaluator-Optimizer Pattern

In previous lessons, we laid the foundation for our Brown writing workflow. We learned how to structure the project in Lesson 20, designed the system architecture in Lesson 21, and implemented the core drafting logic in Lesson 22 using the orchestrator-worker pattern and context engineering.

However, generating a draft is just the beginning. Any professional writer knows that the real value is created during the editing process. A first draft rarely meets all quality standards, tone requirements, and structural guidelines perfectly.

In this lesson, we will implement the Evaluator-Optimizer pattern to add a self-correcting quality assurance layer to our writing workflow. We will develop an automated review system that evaluates the generated article against our writing profiles and returns the reviews to the writer to improve the content iteratively. Finally, we will orchestrate this entire process using LangGraph’s Functional API and add short-term memory to persist our workflow state.

In this lesson, you will learn:

The architecture of the Evaluator-Optimizer pattern and how it differs from Reflection.
How to model review entities to capture structured feedback.
Implementing an Article Reviewer (Evaluator) that checks content against multiple profiles.
Upgrading the Article Writer (Optimizer) to edit content based on feedback.
Centralizing application configuration using Pydantic and YAML.
Orchestrating the review-edit loop with LangGraph’s Functional API and checkpointing.

Explaining the Evaluator-Optimizer Pattern

The Evaluator-Optimizer pattern is a workflow design that mimics real-world quality assurance. It separates content generation from evaluation, creating a feedback loop that iteratively refines the output until it meets specific criteria [1].

The architecture consists of two distinct components:

Evaluator : Analyzes the output against a set of requirements and identifies specific issues.
Optimizer : Takes the evaluator’s feedback and generates an improved version of the output.

This cycle continues until a stopping condition is met, such as reaching a maximum number of iterations or satisfying a quality threshold [2].

Image 1: A flowchart illustrating the Evaluator-Optimizer pattern applied to generating articles. Image 1: A flowchart illustrating the Evaluator-Optimizer pattern applied to generating articles.

This pattern is widely used in engineering tasks where quality is non-negotiable:

Code generation: An LLM writes code, and a separate component (static analysis or another LLM) reviews it for bugs, security flaws, or style violations.
Video script writing: A script is generated and then reviewed for pacing and brand alignment.
Course material creation: Lessons are drafted and then critiqued for pedagogical effectiveness.
Financial reporting: ensuring outputs adhere to strict data formats and compliance rules.

In our context, this mirrors the professional writing process:

The Writer creates an initial draft.
The Reviewer provides objective feedback based on guidelines.
The Editor (often the writer wearing a different hat) refines the draft based on that feedback.
Steps 2 and 3 repeat until the piece is polished.

Now, let’s see how this applies to our Brown architecture.

Scoping the New Writing Workflow Architecture

In Lesson 22, we implemented a linear three-step workflow:

Load Context : Gather guidelines, research, profiles, and examples.
Generate Media : Create diagrams using the orchestrator-worker pattern.
Write Article : Generate the first draft.

Now, we are extending this into a five-step workflow by adding a review-edit loop:

Article Reviewer (Evaluator) : The ArticleReviewer node checks the draft against the article guideline and all writing profiles (tone, style, structure).
Article Editor (Optimizer) : The ArticleWriter node ingests the reviews and rewrites the article to fix the identified issues.

Image 2: The complete writing workflow, adding the review and edit steps to the previous pipeline. Image 2: The complete writing workflow, adding the review and edit steps to the previous pipeline.

Then, at step 6, we check if we reached the expected number of iterations. If not, we repeat steps 4 and 5 on top of the reviewed article until we reach our configured number of iterations. From our tests, we usually used two review-editing loops. Ultimately, we return the final article to the user.

💡To see how the design presented above relates to Brown's overall architecture, open the whiteboard below.

Live Whiteboard: The End-To-End Architecture of Brown: The Writing Workflow. Click here to expand.

Returning to our evaluator-optimizer design, we must address an important question: “Why rely solely on the maximum number of expected iterations rather than on an evaluation score as well?”

Why We Don’t Use a Quality Score

The standard definition of the evaluator-optimizer pattern suggests using a numerical score to decide when to stop the loop. We deliberately avoid this for two reasons.

First, writing quality is subjective and multi-dimensional. Reducing adherence to our complex style and structure profiles to a single scalar number is noisy and unreliable. An LLM might hallucinate a high score while missing critical structural requirements, or assign a low score to minor stylistic choices, leading to a noisy, unstable evaluation loop. In concrete numbers, we have around 100 rules within our profiles. Assigning a binary score of 0 or 1 to each rule and averaging them can significantly reduce the signal.

One option to solve this problem is to reduce the evaluation to a set of key characteristics we want the article to consistently follow, and compute the score only on those. Unfortunately, at this step, it won’t work for us, because we want the Reviewer to review all the profile instructions. Still, this strategy is effective for AI Evaluations, where we want to automate evaluating the entire Brown application and ensure it adheres to key features.

Second, we prioritize Human-in-the-Loop (HITL). Instead of relying on an arbitrary threshold, we conduct a fixed number of automated review cycles (e.g., two passes) to identify and correct obvious errors. Then, as we will see in Lesson 24, we present the result to a human user who can provide the final “pass” or request specific edits. This approach simplifies implementation, improves reliability, reduces token usage and latency, and ensures the final output meets human standards.

The Evaluator-Optimizer vs. Reflection Pattern

Before diving into the implementation, we want to clarify the difference between the Evaluator-Optimizer and Reflection patterns. We believe this is important because we’ve observed that many people often confuse the two.

The confusing part is that both aim to improve LLM outputs through iteration, but they do so in different ways.

The Evaluator-Optimizer Pattern uses separate LLM calls (or even distinct agents) for generation and evaluation. This separation of concerns enables us to utilize distinct prompts, contexts, or even different models for each task. For example, you might use a highly creative model for writing and a stricter, reasoning-heavy model for reviewing [2].

Image 3: The Evaluator-Optimizer vs the Reflection pattern Image 3: The Evaluator-Optimizer vs the Reflection pattern

The Reflection Pattern , in contrast, happens within a single LLM entity that shares the same context between the generation and evaluation. In other words, the same LLM reflects on top of its own answers and tries to improve them before providing the final answer. The model is prompted to “critique its own thinking” before generating the final answer. This time, the reflection loop happens within an agent’s planning mechanism or within the LLM’s reasoning phase. While efficient, it lacks the transparency, modularity, and control afforded by a distinct evaluator node [3], [4].

Typically, the Evaluator-Optimizer pattern is employed within workflows, whereas the Reflection pattern is purely agentic. However, as the line between workflows and agents blurs, you can apply the Evaluator-Optimizer pattern to an agentic design. For example, in a swarm of agents, you would have specialized evaluator and optimizer agents, and the decision to pass information between them is handled by an orchestrator agent, rather than being hardcoded in the code as we do in a workflow. Ultimately, the most significant difference lies in whether the evaluation is conducted by the same entity (Reflection) or by a different, specialized one (Evaluator-Optimizer).

We chose the Evaluator-Optimizer pattern for Brown because:

Transparency : We want to inspect the Reviews as distinct artifacts. This helps us debug why the agent decided to edit the text.
Configuration : We can use a high-temperature model for the creative writing phase and a low-temperature, deterministic model for the strict reviewing phase.
Process : It aligns with the real-world separation between a writer and an editor.

Now that we have completed the theory and design behind the editing and reviewing loop, let’s begin implementing it.

Modeling Our Review Entities

To implement this pattern, we first need to define structured data models for our feedback. In Lesson 22, we defined most of the entities from the Brown project, such as the Article and ArticleGuideline. Now, for the reviewing logic, we have to introduce a few more.

Within Brown, we support two types of reviews to optimize for cost and speed: Whole Article Reviews and Selected Text Reviews , because often, you only need to fix a specific section, not rewrite the entire piece.

Let’s see how we can model that behavior. In the class diagram below, you can observe that we have attached an ArticleReviews class to the Article, which models a list of Reviews for a given article. Additionally, for reviewing only a single piece of text, we created the SelectedText class, which includes the SelectedTextReviews model that attaches a list of Reviews to the selected text.

Now, let’s implement each to see how this works in practice.

ArticlestringcontentArticleReviewsArticlearticlelist[Review]reviewsReviewstringprofilestringlocationstringcommentSelectedTextarticleArticlestringcontentSelectedTextReviewsArticleArticleSelectedTextselected_textlist[Review]reviewshascontainsfromhascontains

Image 4: Entity Relationship Diagram showing relationships between Article, ArticleReviews, Review, SelectedText, and SelectedTextReviews.

The Review Entity

A Review captures a single issue found in the text. It links the problem to a specific profile and location and provides a comment. As the Review entity is passed directly to the LLM during evaluation, it’s essential to attach a Field object to each attribute that clearly describes what each field is and how it relates to the evaluation process.

from pydantic import BaseModel, Field
from brown.entities.mixins import ContextMixin
 
 
class Review(BaseModel, ContextMixin):
    profile: str = Field(
        description="The profile type listing the constraints based on which we will write the comment."
    )
    location: str = Field(
        description="The location from within the article where the comment is made. For example, the title of a section."
    )
    comment: str = Field(
        description="The comment made by the reviewer stating the issue relative to the profile."
    )
 
    def to_context(self) -> str:
        return f"""
<{self.xml_tag}>
    <profile>{self.profile}</profile>
    <location>{self.location}</location>
    <comment>{self.comment}</comment>
</{self.xml_tag}>
"""

The ArticleReviews Entity

This entity aggregates all reviews for a comprehensive article review. It contains the article itself and the list of reviews.

class ArticleReviews(BaseModel, ContextMixin):
    article: Article
    reviews: list[Review]
 
    def to_context(self, include_article: bool = False) -> str:
        reviews_str = "\n".join([review.to_context() for review in self.reviews])
        return f"""
<{self.xml_tag}>
    {f"<article>{self.article}</article>" if include_article else ""}
    <reviews>
    {reviews_str}
    </reviews>
</{self.xml_tag}>
"""

The SelectedText Entity

To review only a part of the article, we need to define what that part is. The SelectedText entity holds the content and its position (line numbers) within the parent article. We use the first_line_number and last_line_number to help the LLM locate the selected text within the article, providing the full context for review and editing. Without this, reviewing or editing the selected text out of context can introduce issues. For example, let’s say that the RAG acronym was already introduced as “Retrieval-Augmented Generation (RAG)” at the top of the article, and it’s used as “RAG” within the selected text. If the evaluator cannot see that, it will mark it as an error.

class SelectedText(BaseModel, ContextMixin):
    article: Article
    content: str
    first_line_number: int
    last_line_number: int
 
    def to_context(self) -> str:
        return f"""
<{self.xml_tag}>
    <content>{self.content}</content>
    <first_line_number>{self.first_line_number}</first_line_number>
    <last_line_number>{self.last_line_number}</last_line_number>
</{self.xml_tag}>
"""

The SelectedTextReviews Entity

Finally, this entity groups reviews specific to a text selection. It also leverages the Review entity, linking it to a particular SelectedText.

class SelectedTextReviews(BaseModel, ContextMixin):
    article: Article
    selected_text: SelectedText
    reviews: list[Review]
 
    def to_context(self, include_article: bool = False) -> str:
        reviews_str = "\n".join([review.to_context() for review in self.reviews])
        return f"""
<{self.xml_tag}>
    {f"<article>{self.article.to_context()}</article>" if include_article else ""}
    <selected_text>{self.selected_text.to_context()}</selected_text>
    <reviews>
    {reviews_str}
    </reviews>
</{self.xml_tag}>
"""

Now that we have all the entities in place, let’s move on to implementing the evaluator-optimizer pattern from scratch, starting with the evaluator.

Implementing the Article Reviewer (The Evaluator)

We use the same Node abstraction from Lesson 22 to implement the ArticleReviewer node, which generates feedback as review entities that follow the structure presented above.

The ArticleReviewer node works as follows.

1. The Reviewer takes the content to review (polymorphic: either Article or SelectedText, as they share the same behaviour) and the requirements (article guidelines and profiles). We also implement properties that make it easy to decide whether to work with an article or selected text.

class ArticleReviewer(Node):
    system_prompt_template = """..."""# We'll see this shortly
    selected_text_system_prompt_template = """..."""
 
    def __init__(
        self,
        to_review: Article | SelectedText,
        article_guideline: ArticleGuideline,
        model: Runnable,
        article_profiles: ArticleProfiles,
    ) -> None:
        self.to_review = to_review
        self.article_guideline = article_guideline
        self.article_profiles = article_profiles
 
        super().__init__(model, toolkit=Toolkit(tools=[]))
        
    @property
    def is_article(self) -> bool:
        return isinstance(self.to_review, Article)
 
    @property
    def is_selected_text(self) -> bool:
        return isinstance(self.to_review, SelectedText)
 
    @property
    def article(self) -> Article:
        if self.is_article:
            return cast(Article, self.to_review)
        else:
            return cast(SelectedText, self.to_review).article

Notice that the toolkit is empty. Reviewing is a pure generation task as the LLM doesn’t need external tools to critique the text. The magic happens when the proper context and prompt are provided.

2. We use an intermediate Pydantic model, ReviewsOutput, to enforce structured output from the LLM.

class ReviewsOutput(BaseModel):
    reviews: list[Review]
 
def _extend_model(self, model: Runnable) -> Runnable:
    model = cast(BaseChatModel, super()._extend_model(model))
    model = model.with_structured_output(ReviewsOutput)
 
    return model

This decoupling keeps the LLM’s output schema simple (list[Review]) while allowing the node to return rich domain entities (ArticleReviews) that include the original article context.

3. Next, we implement the ainvoke method. This method orchestrates the assembly of the prompt and the model call. First, we construct the system prompt by formatting all profiles and guidelines into context. We also handle the logic for adding specialized instructions when reviewing a selected text segment.

async def ainvoke(self) -> ArticleReviews | SelectedTextReviews:
    system_prompt = self.system_prompt_template.format(
        human_feedback=self.human_feedback.to_context() if self.human_feedback else "",
        article=self.article.to_context(),
        article_guideline=self.article_guideline.to_context(),
        character_profile=self.article_profiles.character.to_context(),
        article_profile=self.article_profiles.article.to_context(),
        structure_profile=self.article_profiles.structure.to_context(),
        mechanics_profile=self.article_profiles.mechanics.to_context(),
        terminology_profile=self.article_profiles.terminology.to_context(),
        tonality_profile=self.article_profiles.tonality.to_context(),
    )
    user_input_content = self.build_user_input_content(inputs=[system_prompt])
    inputs = [
        {
            "role": "user",
            "content": user_input_content,
        }
    ]
    If self.is_selected_text:
        inputs. extend(
            [
                {
                    "role": "user",
                    "content": self.selected_text_system_prompt_template.format(selected_text=self.to_review.to_context()),
                }
            ]
        )

Note how we call to_context() on every entity. This encapsulates the logic for formatting each profile into XML, keeping the node logic clean and scoped.

4. Next, we invoke the model to generate the reviews and package the output into the appropriate domain entity.

reviews = await self.model.ainvoke(inputs)
if not isinstance(reviews, ReviewsOutput):
    raise InvalidOutputTypeException(ReviewsOutput, type(reviews))
    
if self.is_selected_text:
    return SelectedTextReviews(
        article=self.article,
        selected_text=cast(SelectedText, self.to_review),
        reviews=reviews.reviews,
    )
else:
    return ArticleReviews(
        article=self.article,
        reviews=reviews.reviews,
    )

The System Prompt

As always, the system prompt is the brain of our node. Let’s analyze its anatomy, as we did for the article writer, following best practices for prompt structure [5]:

Task Context : Defines the persona and the high-level overview of the task.

system_prompt_template = """
You are Brown, an expert article writer, editor and reviewer specialized in reviewing technical, educative and informational articles.

Your task is to review a given article against a set of expected requirements and provide detailed feedback 
about any deviations. You will act as a quality assurance reviewer, identifying specific issues and suggesting 
how the article fails to meet the expected requirements.

These reviews will further be used to edit the article, ensuring it follows all the requirements.

## Requirements

The requirements are a set of rules, guidelines or profiles that the article should follow. Here they are:

- **article guideline:** the user intent describing how the article should look like. Specific to this particular article.
- **article profile:** rules specific to writing articles. Generic for all articles.
- **character profile:** the character you will emporsonate while writing. Generic for all content.
- **structure profile:** Structure rules guiding the final output format. Generic for all content.
- **mechanics profile:** Mechanics rules guiding the writing process. Generic for all content.
- **terminology profile:** Terminology rules guiding word choice and phrasing. Generic for all content.
- **tonality profile:** Tonality rules guiding the writing style. Generic for all content.
...
"""

Background Data : Injects the article, guidelines, and all profiles (Character, Tonality, Structure, etc.).

"""
...
## Article to Review
 
Here is the article that needs to be reviewed:
 
{article}
 
## Article Guideline
 
The <article_guideline> represents the user intent, describing how the actual article should look like.
 
The <article_guideline> will ALWAYS contain:
- all the sections of the article expected to be wrriten, in the correct order
- a level of detail for each section, describing what each section should contain. Depending on how much detail you have in a
particular section of the <article_guideline>, you will use more or less information from the <research> tags to write the section.
 
The <article_guideline> can ALSO contain:
- length constraints for each section, such as the number of characters, words or reading time. If present, you will respect them.
- important (golden) references as URLs or titles present in the <research> tags. If present, always prioritize them over anything else 
from the <research>.
- information about anchoring the article into a series such as a course or a book. Extremely important when the article is part of 
something bigger and we have to anchor the article into the learning journey of the reader. For example, when introducing concepts
in previous articles that we don't want to reintroduce into the current one.
- concrete information about writing the article. If present, you will ALWAYS priotize the instructions from the <article_guideline> 
over any other instructions.
 
Here is the article guideline:
{article_guideline}
 
## Character Profile
 
To make the writing more personable, we emporsonated the following character profile when writing the article:
{character_profile}
 
## Terminology Profile
 
Here is the terminology profile, describing how to choose the right words and phrases:§
to the target audience:
{terminology_profile}
 
## Tonality Profile
 
Here is the tonality profile, describing the tone, voice and style of the writing:
{tonality_profile}
 
## Mechanics Profile
 
Here is the mechanics profile, describing how the sentences and words should be written:
{mechanics_profile}
 
## Structure Profile
 
Here is the structure profile, describing general rules on how to structure text, such as the sections, paragraphs, lists,
code blocks, or media items:
{structure_profile}
 
## Article Profile
 
Here is the article profile, describing particularities on how the end-to-end article should look like:
{article_profile}
...
"""

Detailed Task Description : Defines the rules. Reviewing Rules establishes a strict hierarchy: Special Rules (from the writing profiles) > Article Guideline (the user input) > Writing Profile > Others. This prevents the model from prioritizing minor stylistic points over core requirements.

"""
...
## Reviewing Process

You will review the article against all the requirements above, creating a one-to-many relationship between each requirement and the 
number of required reviews. In other words, for each requirement, you will create 0 to N reviews. If the article follows the 
requirement 100%, you will not create any reviews for it. If it doesn't follow the requirement, you will create as many reviews 
as required to ensure the article follows the requirement.

Remember that these reviews will further be used to edit the article, ensuring it follows all the requirements. Thus, it's
important to make a thorough review, covering all the requirements and not missing any detail.

## Reviewing Rules

- **The first most important rule:** The requirements can contain some special sections labeled as "rules" or 
"correction rules". You should look for <(.*)?rules(.*)?> XML tags like <correction_media_rules>, 
<abbreviations_or_acronyms_never_to_expand_rules>, <correction_reference_rules>. These are special highlights that 
should always be prioritized over other rules during the review process. They should be respected at all costs when 
writing the article. You will always prioritize these rules over other rules from the requirements making them your 
No.1 focus.
- **The second most important rule:** The adherence to the <article_guideline>.
- **The third most important rule:** The adherence to the <article_profile>.
- **The fourth most important rule:** The adherence to the rest of the requirements.

Other more generic rules:
- Be thorough but fair - only flag genuine issues
- Enphasize WHY something is wrong, not just WHAT is wrong
- Focus on significant deviations, not minor nitpicks 
...
"""

Chain of Thought : Wrap up the system prompt with a chain of thoughts that explicitly states the order of the instructions and reminds the LLM at the end of the context about its core task to ensure it focuses on the right thing:

"""
## Chain of Thoughts

1. Read and analyze the article.
2. Read and analyze the <human_feedback>.
3. Read and analyze all the requirements considering the <human_feedback> as a guiding force.
4. Carefully compare the article against the requirements as instructed by the rules above.
5. For each requirement, create 0 to N reviews
6. Return the reviews of the article.
"""

We could further improve this prompt by adding a few-shot examples , such as calling the Reviewer on a sample article, manually correcting the generated reviews, and feeding those back into the prompt as examples. This strategy could significantly increase the quality and consistency of the feedback.

The Selected Text Prompt

As we want to model both reviewing the whole article or a selected piece of text, we decoupled the two logic branches into two system prompts. The selected text prompt doesn’t repeat the entire logic; it just provides extra context on where the selected text is coming from and short-circuits the Chain of Thoughts with a new set of instructions on the reviewing steps for the selected text. Otherwise, all context, including the article, article guidelines, and writing profiles, is used in both scenarios.

selected_text_system_prompt_template = """
You already reviewed and edited the whole article. Now we want to further review only a specific portion
of the article, which we label as the <selected_text>. Despite reviewing the selected text, instead of the
article as a whole, you will follow the exact same instructions from above as if you were reviewing the article as a whole.

## Selected Text to Review

Here is the selected text that needs to be reviewed:

{selected_text}

As pointed out before, the selected text is part of the larger <article> that is already reviewed.
You will use the full <article> as context and anchoring the reviewing process within the bigger picture.

The <first_line_number> and <last_line_number> numbers from the <selected_text> indicate the first and 
last line/row numbers of the selected text from the <article>. Use them to locate the selected text within the <article>.

## Chain of Thoughts

Here is the new chain of thoughts logic you will follow when reviewing the selected text. You can ignore the
previous chain of thoughts:

1. Read and analyze the article.
2. Locate the <selected_text> within the <article> based on the <first_line_number> and <last_line_number>.
3. Read and analyze the <human_feedback>.
4. Read and analyze all the requirements considering the <human_feedback> as a guiding force.
5. Carefully compare the selected text against the requirements as instructed by the rules above.
6. For each requirement, create 0 to N reviews
7. Return the reviews of the selected text.
"""

By placing the Chain of Thoughts at the end of the prompt chain, we effectively redirect the model’s immediate focus while keeping the global context available.

from brown.loaders import MarkdownArticleGuidelineLoader, MarkdownArticleLoader, MarkdownArticleProfilesLoader
from brown.models import SupportedModels, get_model
from brown.nodes import ArticleReviewer
 
# Load the article guideline
guideline_loader = MarkdownArticleGuidelineLoader(uri=Path("article_guideline.md"))
article_guideline = guideline_loader.load(working_uri=SAMPLE_DIR)
 
# Load the article profiles
profiles_input = {
    "article": PROFILES_DIR / "article_profile.md",
    "character": PROFILES_DIR / "character_profiles" / "paul_iusztin.md",
    "mechanics": PROFILES_DIR / "mechanics_profile.md",
    "structure": PROFILES_DIR / "structure_profile.md",
    "terminology": PROFILES_DIR / "terminology_profile.md",
    "tonality": PROFILES_DIR / "tonality_profile.md",
}
profiles_loader = MarkdownArticleProfilesLoader(uri=profiles_input)
article_profiles = profiles_loader.load(working_uri=SAMPLE_DIR)

And the article generated in the previous lesson.

article_loader = MarkdownArticleLoader(uri=Path("article.md"))
article = article_loader.load(working_uri=SAMPLE_DIR)

Then, run the Reviewer using Gemini 2.5 Flash.

model = get_model(SupportedModels.GOOGLE_GEMINI_25_FLASH)
reviewer = ArticleReviewer(
    to_review=article,
    article_guideline=article_guideline,
    article_profiles=article_profiles,
    model=model,
)
article_reviews = await reviewer.ainvoke()

pretty_print.wrapped(f"Generated {len(article_reviews.reviews)} reviews:", title="Article reviews")
for i, review in enumerate(article_reviews.reviews, 1):
    review_dict = {
        "Profile": review.profile,
        "Location": review.location,
        "Comment": review.comment[:200] + "..." if len(review.comment) > 200 else review.comment,
    }
    pretty_print.wrapped(review_dict, title=f"Review {i}")

The output shows specific, actionable feedback as follows:

------------------------------- Article reviews ------------------------------
  Generated 49 reviews:
---------------------------------- Review 1 ----------------------------------
  {
  "Profile": "article_guideline",
  "Location": "Article level - Outline",
  "Comment": "The article's main title is 'Workflows vs. Agents: The Critical Decision Every AI Engineer Faces'. However, the guideline specifies the title for Section 1 as 'Introduction: The Critical Decision Ever..."
}
---------------------------------- Review 2 ----------------------------------
  {
  "Profile": "article_guideline",
  "Location": "Introduction: The Critical Decision Every AI Engineer Faces - First paragraph",
  "Comment": "The article's introduction contains 84 words, which is within the 100-word limit. However, the guideline specifies the content for the 'The Problem' part of the introduction. The article's content lar..."
}
---------------------------------- Review 3 ----------------------------------
  {
  "Profile": "article_guideline",
  "Location": "Understanding the Spectrum: From Workflows to Agents - First paragraph",
  "Comment": "The article states 'To make the right choice, you first need to understand what LLM workflows and AI agents are. We will look at their core properties and how they are used, rather than their technica..."
}
...
---------------------------------- Review 40 ----------------------------------
  {
  "Profile": "structure_profile",
  "Location": "Introduction: The Critical Decision Every AI Engineer Faces - First paragraph",
  "Comment": "The introduction's first paragraph has 84 words. The 'Sentence and paragraph length patterns' rule states 'Keep paragraphs \u2264 80 words'. This paragraph slightly exceeds the limit. It should be condense..."
}
...
---------------------------------- Review 49 ----------------------------------
  {
  "Profile": "article_profile",
  "Location": "References - Article level",
  "Comment": "The 'References' section is present at the end of the article, listing all sources. This aligns with the 'General Article Structure' and 'References Rules'."
}
-------------------------------------------------------------------------------

You can see that the profile points to one of our writing profiles, the location is the name of a section from the article, and within the comment, we have the actual review feedback.

Now, let’s review only a selected piece of text.

Example: Reviewing Selected Text

We can also review just a specific portion of the article. This is useful when a user wants to check if a particular section meets the requirements without reprocessing the entire document.

First, we extract the text we want to review and wrap it in a SelectedText entity.

from brown.entities.articles import SelectedText
 
# Let's extract a specific section to review
article_lines = article.content.split("\n")
first_line_number = 11
last_line_number = 44
selected_content = "\n".join(article_lines[first_line_number:last_line_number])
 
selected_text = SelectedText(
    article=article,
    content=selected_content,
    first_line_number=first_line_number,
    last_line_number=last_line_number,
)
pretty_print.wrapped(selected_text.to_context(), title="Selected text context (first 1500 characters)")

It outputs:

--------------- Selected text context (first 1500 characters) ----------------
  
<selected_text>
    <content>### LLM Workflows
An LLM workflow is a sequence of tasks orchestrated by developer-written code. It can include LLM calls, but also other operations like reading from a database or calling an API. Think of it like a recipe where each step is explicitly defined. The key characteristic is that the path is determined in advance, resulting in a deterministic or rule-based system. This gives you predictable execution, explicit control over the application's flow, and makes the system easier to test and debug. Because you control every step, you know exactly where a failure occurred and how to fix it.

\`\`\`mermaid
graph TD
    A["Start"] --> B["LLM Call"]
    B --> C["Process Data"]
    C --> D["Store Data"]
    D --> E["End"]
\`\`\`
Image 1: A flowchart illustrating a deterministic LLM workflow with clear start and end points, including an LLM call and data operations.

### AI Agents

AI agents are systems where an LLM dynamically decides the sequence of steps, reasoning, and actions to achieve a goal. The path is not predefined. Instead, the agent uses a reasoning process to plan its actions based on the task and the current state of its environment. This process is often modeled on frameworks like ReAct (Reason, Act, Observe). This allows agents to be adaptive and capable of handling new or unexpected situations through LLM-driven autonomy. They can select tools, execute actions, evaluate the outcomes, and correct their course until the goal is achieved [[1]](https://www.youtube.co
...
	</content>
  <first_line_number>11</first_line_number>
  <last_line_number>44</last_line_number>
</selected_text>

Then, we run the ArticleReviewer exactly as before, but passing the SelectedText object. The node’s polymorphic design handles the rest.

model = get_model(SupportedModels.GOOGLE_GEMINI_25_FLASH)
reviewer = ArticleReviewer(
    to_review=selected_text,  # Now passing SelectedText instead of Article
    article_guideline=article_guideline,
    article_profiles=article_profiles,
    model=model,
)
selected_text_reviews = await reviewer.ainvoke()

pretty_print.wrapped(
    f"Generated {len(selected_text_reviews.reviews)} reviews for selected text:", title="Selected text reviews"
)
for i, review in enumerate(selected_text_reviews.reviews, 1):
    review_dict = {
        "Profile": review.profile,
        "Location": review.location,
        "Comment": review.comment[:200] + "..." if len(review.comment) > 200 else review.comment,
    }
    pretty_print.wrapped(review_dict, title=f"Review {i}")

The output will now contain reviews specific only to that slice of text, anchored by the line numbers.

----------------------- Selected text reviews --------------------------------
  Generated 2 reviews for selected text:
---------------------------------- Review 1 ----------------------------------
  {
  "Profile": "structure_profile",
  "Location": "Understanding the Spectrum: From Workflows to Agents - 
  LLM Workflows sub-section",
  "Comment": "The caption for Image 1 is missing the citation requirements 
  (`Source`, `Image by`, `citation_name`, `citation_identifier`, 
  `citation_url`). The guideline specifies that all media items should 
  contain..."
}

---------------------------------- Review 2 ----------------------------------
  {
  "Profile": "structure_profile",
  "Location": "Understanding the Spectrum: From Workflows to Agents - 
  AI Agents sub-section",
  "Comment": "The caption for Image 2 is missing the citation requirements 
  (`Source`, `Image by`, `citation_name`, `citation_identifier`, 
  `citation_url`). The guideline specifies that all media items should 
  contain..."
}

Now, let’s hook these reviews to the article writer, the Optimizer.

Hooking the Reviews to the Article Writer (The Optimizer)

To implement the Optimizer, we don’t need a new node. We upgrade the existing ArticleWriter to serve a dual purpose: Writer (from scratch) and Editor (from reviews). This keeps all writing logic in one place.

Updating the Article Writer

We update __init__ to accept optional reviews.

class ArticleWriter(Node):
	article_reviews_prompt_template = "..."
	selected_text_reviews_prompt_template = "..."
	
	def __init__(
	    self,
	# ... other args
	    reviews: ArticleReviews | SelectedTextReviews | None = None,
	) -> None:
	# ...
	    self.reviews = reviews

If reviews is None, it writes a new draft. If provided, it switches to editing mode.

We update ainvoke to construct a conversation history that simulates the review process.

        async def ainvoke(self) -> Article | SelectedText:
    			... # The same logic as in lesson 22
    			
    			if self.reviews:
                if isinstance(self.reviews, ArticleReviews):
                    reviews_prompt = self.article_reviews_prompt_template.format(
                        reviews=self.reviews.to_context(include_article=False),
                    )
                elif isinstance(self.reviews, SelectedTextReviews):
                    reviews_prompt = self.selected_text_reviews_prompt_template.format(
                        selected_text=self.reviews.selected_text.to_context(),
                        reviews=self.reviews.to_context(include_article=False),
                    )
                else:
                    raise ValueError(f"Invalid reviews type: {type(self.reviews)}")
                inputs.extend(
                    [
                        {
                            "role": "assistant",
                            "content": self.reviews.article.to_context(),
                        },
                        {
                            "role": "user",
                            "content": reviews_prompt,
                        },
                    ]
                )
          written_output = await self.model.ainvoke(inputs)
          
          ... # The same logic as in lesson 22
          
          if isinstance(self.reviews, SelectedTextReviews):
                return SelectedText(
                    article=self.reviews.article,
                    content=written_output,
                    first_line_number=self.reviews.selected_text.first_line_number,
                    last_line_number=self.reviews.selected_text.last_line_number,
                )
            else:
                return Article(
                    content=written_output,
                )

By simulating the conversation history through the if ``[self.reviews](<http://self.reviews/>) branch, we create the following context:

Here are the rules (System - initially added when writing the article the first time)
Here is what you wrote (Assistant - the article)
Here is what was wrong (User - the optimizer system prompt)
Now fix it.

Let’s see what the optimizer system prompts look like.

The Editing Prompts

As before, we have a system prompt tailored to each scenario: the entire article or a selected piece of text.

The system prompt for reviewing the article is grouped as follows:

The detailed task description.

article_reviews_prompt_template = """
We personally reviewed the article and compiled a list of reviews based on which you have to edit the article 
you wrote one step before.

## Reviewing Logic

Here is how we created the feedback reviews:
- We compared the whole article you wrote against the <article_guideline> to ensure it follows the user intent.
- We compared the whole article you wrote against all the profile constraints: 
<character_profile>, <article_profile>, <structure_profile>, <mechanics_profile>, <terminology_profile> and <tonality_profile>.
- As the article was subject to a manual human review, we created a special type of constrains called "human feedback". These 
are the most important as they reflect the direct need of the user.
- For each of these constrains, we created 0 to N reviews. If a profile was respected 100%, then we did not create any reviews for it. 
Otherwise, for each broken rule from a given profile, we created a review.
- Each review contains the profile from which it broke a rule, the location within the article (e.g., the section) and the actual review,
as a message on what is wrong and how it deviates from the profile.

Your task is to fix all the reviews from the provided list. You will prioritize these reviews over anything else, 
while still keeping the factual information anchored in the <research> and <article_guideline>.
....
"""

Special section ranking the importance of each review. We will cover human feedback in the next lesson. For now, it’s important to highlight that we always prioritize human feedback over everything else:

"""
...
## Ranking the Importance of the Reviews

1. Always prioritize the human feedback reviews above everything else. The human feedback 
is the primary driver for your edits.
2. Next prioritize the reviews based on the <article_guideline>.
3. Finally, prioritize the reviews based on the other profiles.
..
"""

The actual reviews:

"""
...
## Reviews 

Here are the reviews you have to fix:
{reviews}
...
"""

The chain of thought from the end overrides the one from the main system prompt, transforming the ArticleWriter node from generating an article from scratch to reviewing it:

"""
## Chain of Thought

1. Analyze the reviews to understand what needs to be changed.
2. Priotize the reviews based on the importance ranking.
3. Based on the reviews, apply in order, the necessary edits to the article, while still 
following all the necessary instructions from the profiles and guidelines above.
4. Ensure the edited text is still anchored in the <research> and <article_guideline>.
5. Ensure the edited text still flows naturally with the surrounding content and overall article.
6. Return the fully edited article.
"""

For reviewing the selected piece of text, the prompt is pretty similar to the one for editing the whole article, except for some minor details about where the selected text comes from.

selected_text_reviews_prompt_template = """
We personally reviewed only a portion of the article, a selected text, and compiled a list of reviews based on which 
you have to edit only the given selected text you wrote within the article from one step before.

## Selected Text to Edit

Here is the selected text that needs to be edited:

{selected_text}

Remember that this selected text is part of the article from one step before. Anchor your
editing within the broader context of the article.

Selected text editing guidelines:
- After edits, keep the selected text consistent with the surrounding article context
- To locate the selected text within the article, use the specific first and last line numbers passed
along with the <selected_text>.
- Only edit the selected text, don't modify the entire article

## Reviewing Logic

... # Similar to the one for editing the whole article

## Ranking the Importance of the Reviews

... # Similar to the one for editing the whole article

## Reviews 

Here are the reviews you have to fix:
{reviews}

...

"""

The final chain of thought has to carefully instruct how to manipulate only the selected text in the context of the full article.

"""
## Chain of Thought

1. Place the selected text in the context of the full article.
2. Analyze the reviews to understand what needs to be changed.
3. Priotize the reviews based on the importance ranking.
4. Based on the reviews, apply in order, the necessary edits to the selected text, while still 
following all the necessary instructions from the profiles and guidelines above.
5. Ensure the edited selected text is still anchored in the <research> and <article_guideline>.
6. Ensure the edited selected text still flows naturally with the surrounding content and overall article.
7. Return the fully edited selected text.
"""

Now that we have all the moving pieces of the evaluator-optimizer pattern in place, such as the Article Reviewer and Editor, let’s glue them together into an end-to-end example.

E nd-to-End Example

We can now run the full loop manually to verify the logic. This example demonstrates the complete lifecycle: loading context, generating media, writing a draft, reviewing it, and editing it.

Load Context : We initialize our loaders and fetch all necessary data.

from brown.loaders import (
    MarkdownArticleExampleLoader,
    MarkdownArticleGuidelineLoader,
    MarkdownArticleProfilesLoader,
    MarkdownResearchLoader,
)
 
pretty_print.wrapped("STEP 1: Loading Context", width=100)
 
# Load guideline
guideline_loader = MarkdownArticleGuidelineLoader(uri=Path("article_guideline.md"))
article_guideline = guideline_loader.load(working_uri=SAMPLE_DIR)
 
# Load research
research_loader = MarkdownResearchLoader(uri=Path("research.md"))
research = research_loader.load(working_uri=SAMPLE_DIR)
 
# Load profiles
profiles_input = {
    "article": PROFILES_DIR / "article_profile.md",
    "character": PROFILES_DIR / "character_profiles" / "paul_iusztin.md",
    "mechanics": PROFILES_DIR / "mechanics_profile.md",
    "structure": PROFILES_DIR / "structure_profile.md",
    "terminology": PROFILES_DIR / "terminology_profile.md",
    "tonality": PROFILES_DIR / "tonality_profile.md",
}
profiles_loader = MarkdownArticleProfilesLoader(uri=profiles_input)
article_profiles = profiles_loader.load()
 
# Load examples
examples_loader = MarkdownArticleExampleLoader(uri=EXAMPLES_DIR)
article_examples = examples_loader.load()

Generate Media : We run the media generation step (simulated here for brevity).

import asyncio
 
from brown.entities.media_items import MediaItems
from brown.models import SupportedModels, get_model
from brown.nodes import MediaGeneratorOrchestrator, MermaidDiagramGenerator, Toolkit
 
# Create worker tool
diagram_model = get_model(SupportedModels.GOOGLE_GEMINI_25_FLASH)
mermaid_generator = MermaidDiagramGenerator(model=diagram_model)
toolkit = Toolkit(tools=[mermaid_generator.as_tool()])
 
# Create orchestrator
orchestrator_model = get_model(SupportedModels.GOOGLE_GEMINI_25_FLASH)
orchestrator = MediaGeneratorOrchestrator(
    article_guideline=article_guideline,
    research=research,
    model=orchestrator_model,
    toolkit=toolkit,
)
 
# Get media generation jobs
media_jobs = await orchestrator.ainvoke()
media_items = await asyncio.gather(*coroutines)
media_items = MediaItems.build(media_items=media_items)

Write Draft : We invoke the writer to create the first version.

# Workflows vs. Agents: The AI Engineer's Critical Choice
 
Every AI engineer eventually confronts a pivotal architectural decision: 
Will you build a predictable, step-by-step workflow where you dictate 
every action, or will you create an autonomous agent that thinks and 
decides for itself? This choice isn't just a technical detail; it shapes 
everything from development time and operational costs to the application's 
reliability and user experience.
 
Opting for the wrong approach can lead to significant setbacks. You might 
end up with a system too rigid to adapt, breaking down when users deviate 
from expected patterns, or an agent that performs brilliantly 80% of the 
time but fails spectacularly when it matters most. This can result in months 
of wasted development, frustrated users, and executives questioning the 
value proposition of AI investments. Across 2024-2025, billion-dollar AI 
startups have seen their success—or failure—hinge primarily on this 
architectural distinction. The most...

It outputs:

from brown.nodes import ArticleWriter
 
writer_model = get_model(SupportedModels.GOOGLE_GEMINI_25_FLASH)
article_writer = ArticleWriter(
    article_guideline=article_guideline,
    research=research,
    article_profiles=article_profiles,
    media_items=media_items,
    article_examples=article_examples,
    model=writer_model,
    reviews=None,  # No reviews for first draft
)
 
article = await article_writer.ainvoke()
print(article.content[:1000])

Review : The Reviewer analyzes the draft and produces feedback.

reviewer_model = get_model(SupportedModels.GOOGLE_GEMINI_25_FLASH)
article_reviewer = ArticleReviewer(
    to_review=article,
    article_guideline=article_guideline,
    article_profiles=article_profiles,
    model=reviewer_model,
)

article_reviews = await article_reviewer.ainvoke()

print(f"✓ Generated {len(article_reviews.reviews)} reviews")
for i, review in enumerate(article_reviews.reviews, 1):
    pretty_print.wrapped(
        {
            "Profile": review.profile,
            "Location": review.location,
            "Comment": review.comment[:150] + "..." if len(review.comment) > 150 else review.comment,
        },
        title=f"Review {i}",
        width=100,
    )

It outputs:

✓ Generated 95 reviews
-------------------------------- Review 1 ----------------------------------
  {
  "Profile": "structure_profile",
  "Location": "Article Level",
  "Comment": "The article is missing an explicit subtitle as specified 
  in the article template: '### Subtitle' after the main title."
}
-------------------------------- Review 2 ----------------------------------
  {
  "Profile": "article_guideline",
  "Location": "Introduction: The Critical Decision Every AI Engineer Faces - 
  First paragraph",
  "Comment": "The introduction's first paragraph uses the wording 
  'Every AI engineer eventually confronts a pivotal architectural decision: 
  Will you build a predict..."
}
-------------------------------- Review 3 ----------------------------------
  {
  "Profile": "terminology_profile",
  "Location": "Introduction: The Critical Decision Every AI Engineer Faces - 
  First paragraph",
  "Comment": "The sentence 'This choice isn't just a technical detail; 
  it shapes everything from development time and operational costs to the 
  application's reliabi..."
}
-------------------------------- Review 4 ----------------------------------
...

Edit : The writer acts as an editor, applying the reviews to the draft.

editor_model = get_model(SupportedModels.GOOGLE_GEMINI_25_FLASH)
article_editor = ArticleWriter(
    article_guideline=article_guideline,
    research=research,
    article_profiles=article_profiles,
    media_items=media_items,
    article_examples=article_examples,
    model=editor_model,
    reviews=article_reviews,  # Pass reviews to trigger editing mode
)

edited_article = await article_editor.ainvoke()

Compare : We can check the difference between the draft and the final version.

print(f"""Original length: {len(article.content):,} characters
Edited length: {len(edited_article.content):,} characters
Difference: {len(edited_article.content) - len(article.content):+,} characters

Number of reviews addressed: {len(article_reviews.reviews)}""")
print(f"Final length: {len(final_article.content)}")

It outputs:

Original length: 29,441 characters
Edited length: 30,860 characters
Difference: +1,419 characters

Number of reviews addressed: 95

With the current infrastructure, it’s impossible to show how all the reviews are addressed. Let’s focus on the second one in the introduction. This is how Review 2 looked like:

-------------------------------- Review 2 ----------------------------------
 {
  "Profile": "article_guideline",
  "Location": "Introduction: The Critical Decision Every AI Engineer Faces - 
  First paragraph",
  "Comment": "The introduction's first paragraph uses the wording 
  'Every AI engineer eventually confronts a pivotal architectural decision: 
  Will you build a predict..."
}

It changes the article by improving the introduction:

 # Workflows vs. Agents: The AI Engineer's Critical Choice
### The AI Engineer's Guide to Architectural Choices

When building AI applications, engineers face a critical architectural 
decision early in their development process. Should they create a 
predictable, step-by-step workflow where they control every action, or 
should they build an autonomous agent that can think and decide for itself? 
This choice impacts everything from development time and operational costs 
to the application's reliability and user experience.

...

Let’s also use a diff tool to show the difference between the first and second versions of the articles. It doesn’t really matter what tool. For example, we used this free one. Here is what was deleted from the first version:

And added to the second version:

As you can see, checking what actually changed after a review can be tedious. You have to go over the reviews, locate the changes using the location field, and use a diff tool to compare files. However, in future lessons, once we integrate Brown with Cursor/Claude through an MCP server, this process of checking changes and diffs will be fully automated.

The only step left in this lesson is to glue the workflow above with LangGraph. Before we do, let’s define our app configuration layer.

Centralizing Our App Configuration

As our system grows, hardcoding model names, hyperparameters, and paths becomes unmanageable. We need a centralized configuration system that defines all our parameters in a single configuration file and propagates it across the entire application.

To implement this, we use YAML files to store the configuration and load them with Pydantic to map the YAML into Python and define a type-safe schema.

Let’s implement this.

Configuration Classes

Context : Defines paths to guidelines, profiles, and research that define our context.

from pathlib import Path
from typing import Annotated, Any, Literal
 
import yaml
from annotated_types import Ge
from pydantic import BaseModel, DirectoryPath, Field, field_validator
 
from brown.config import get_settings
from brown.models.config import ModelConfig, SupportedModels
 
class Context(BaseModel):
    article_guideline_loader: Literal["markdown"]
    article_guideline_uri: Path = Field(description="URI to the article guideline file.")
 
    research_loader: Literal["markdown"]
    research_uri: Path = Field(description="URI to the research file.")
 
    article_loader: Literal["markdown"]
    article_renderer: Literal["markdown"]
    article_uri: Path = Field(description="URI to the article file.")
 
    profiles_loader: Literal["markdown"]
    profiles_uri: Annotated[DirectoryPath, Field(description="URI to the profiles directory.")]
    character_profile: str
 
    examples_loader: Literal["markdown"]
    examples_uri: Annotated[DirectoryPath, Field(description="URI to the examples directory.")]
 
    def build_article_uri(self, iteration: int) -> Path:
        return self.article_uri.with_stem(f"{self.article_uri.stem}_{iteration:03d}")

ToolConfig : Configures individual tools, such as the diagram generator.

class ToolConfig(BaseModel):
    name: str
    model_id: SupportedModels
    config: ModelConfig

NodeConfig : Allows configuring the model and parameters for each node independently. Note how it’s leveraging the SupportedModels and ModelConfig types to ensure we load only models that we support through our brown.models module.

class NodeConfig(BaseModel):
    model_id: SupportedModels
    config: ModelConfig
    tools: dict[str, ToolConfig]

Memory : Controls the checkpointing strategy.

class Memory(BaseModel):
    checkpointer: Literal["in_memory", "sqlite"]

AppConfig : The root configuration object that loads the config from a YAML file into a Pydantic AppConfig object.

class AppConfig(BaseModel):
    context: Context
    memory: Memory
 
    num_reviews: Annotated[int, Ge(1), Field(description="The number of reviews to perform while generating the article the first time.")]
    nodes: dict[str, NodeConfig]
    
    @classmethod
    def from_yaml(cls, file_path: Path) -> "AppConfig":
        """Load configuration from a YAML file."""
 
        if not file_path.exists():
            raise FileNotFoundError(f"Configuration file not found: {file_path}")
 
        with open(file_path, "r", encoding="utf-8") as f:
            data = yaml.safe_load(f)
 
        return cls(**data)

Example YAML

Let’s look at an example YAML file, such as the one we used across the course, located at configs/course.yaml:

context:
  article_guideline_loader: "markdown"
  article_guideline_uri: "article_guideline.md"
  research_loader: "markdown"
  research_uri: "research.md"
  article_loader: "markdown"
  article_renderer: "markdown"
  article_uri: "article.md"
  profiles_loader: "markdown"
  profiles_uri: "inputs/profiles"
  character_profile: "paul_iusztin.md"
  examples_loader: "markdown"
  examples_uri: "inputs/examples/course_lessons"
memory:
  checkpointer: "in_memory"
num_reviews: 2
nodes:
  generate_media_items:
    model_id: "google_genai:gemini-2.5-flash"
    model_config:
      temperature: 0.0
      include_thoughts: false
      thinking_budget: null
    tools:
      mermaid_diagram_generator:
        model_id: "google_genai:gemini-2.5-flash"
        config:
          temperature: 0.0
          include_thoughts: false
          thinking_budget: null
  write_article:
    model_id: "google_genai:gemini-2.5-flash"
    model_config:
      temperature: 0.7
      include_thoughts: false
      thinking_budget: null
  review_article:
    model_id: "google_genai:gemini-2.5-flash"
    model_config:
      temperature: 0.0
      include_thoughts: false
      thinking_budget: null
  edit_article:
    model_id: "google_genai:gemini-2.5-flash"
    model_config:
      temperature: 0.1
      include_thoughts: false
      thinking_budget: null
  review_selected_text:
    model_id: "google_genai:gemini-2.5-flash"
    model_config:
      temperature: 0.0
      include_thoughts: false
      thinking_budget: null
  edit_selected_text:
    model_id: "google_genai:gemini-2.5-flash"
    model_config:
      temperature: 0.1
      include_thoughts: false
      thinking_budget: null

Note how we have complete control over how many review loops we do, how we configure each model, and even what infrastructure we use to load our context from. This allows us to easily experiment with different setups, such as a creative setting for the first draft and a strict, deterministic setting for reviewing and editing, optimizing the performance of each stage.

Loading and Using Configuration

We lazy-load the configuration once through a Least Recently Used (LRU) cache. Note how we control what app config to load through the settings object that loads the environment variables:

from functools import lru_cache
 
@lru_cache(maxsize=1)
def get_app_config() -> AppConfig:
    return AppConfig.from_yaml(get_settings().CONFIG_FILE)

What’s the difference between the settings and the app config?

Usually, you want to load only sensitive information via env vars, since you never want to commit your credentials to git. That’s why everything that goes into your .env file is sensitive and never exposed to the public.

Meanwhile, the app config contains no sensitive information. On the contrary, you want to store and version it with Git. Also, loading the entire application config from env vars is painful because it doesn’t naturally support complex structures like dicts (it does, but it’s ugly).

That’s why you usually want two config types: one for credentials (our settings) and one for app parameters.

Also, in case you have values that rarely change, it’s totally fine to have a [constants.py](<http://constants.py/>) file that contains static objects, such as file names, character names, relative paths, etc., that aggregate in a single place all your constants. You could also add them directly into your app config as default values. Both options are valid. Ultimately, the most important design decision is to separate the settings and app config.

Now, let’s finally implement the LangGraph workflow that glues together all the steps from the end-to-end example above into a compact unit of work.

Glueing Everything Into Our LangGraph Workflow

Instead of using LangGraph Graph SDK, we use LangGraph’s Functional API to orchestrate the workflow. We chose the Functional API over the Graph SDK because it allows us to write standard Python loops and logic (if/else) while still getting state management and checkpointing [6], [7].

In the first version of Brown, we used the Graph SDK, which overcomplicated the code. Writing simple loops or conditional logic required defining complex edges and state reducers, turning what should have been minutes of coding into hours of debugging graph topology. The Functional API solves this by allowing us to use native Python control flow while leveraging LangChain’s rich ecosystem.

Let’s dig into it.

The Workflow Definition

We define the workflow using LangGraph’s Functional API @entrypoint, @task decorators, and a shared RetryPolicy. Let’s start with the @entrypoint that serves as, well…, the entrypoint to our workflow.

from langchain_core.runnables import RunnableConfig
from langgraph.checkpoint.base import BaseCheckpointSaver
from langgraph.config import get_stream_writer
from langgraph.func import entrypoint, task
from langgraph.types import RetryPolicy
 
from brown.base import Loader
from brown.builders import build_article_renderer, build_loaders, build_model
from brown.config_app import get_app_config
from brown.entities.articles import Article, ArticleExamples
from brown.entities.guidelines import ArticleGuideline
from brown.entities.media_items import MediaItem, MediaItems
from brown.entities.profiles import ArticleProfiles
from brown.entities.research import Research
from brown.entities.reviews import ArticleReviews
from brown.nodes.article_reviewer import ArticleReviewer
from brown.nodes.article_writer import ArticleWriter
from brown.nodes.media_generator import MediaGeneratorOrchestrator
from brown.workflows.types import WorkflowProgress
 
app_config = get_app_config()
retry_policy = RetryPolicy(max_attempts=3, retry_on=Exception)
 
def build_generate_article_workflow(checkpointer: BaseCheckpointSaver):
    """Create a generate article workflow with optional checkpointer."""
    return entrypoint(checkpointer=checkpointer)(_generate_article_workflow)
 
class GenerateArticleInput(TypedDict):
    dir_path: Path

Next, we define the****_generate_article_workflow function that orchestrates the entire flow: load context, generate media, draft, iterative review/edit/render with progress updates, and final render to disk. It glues the end-to-end article generation workflow into a single function. We already went through this in this lesson, so we won’t split it into multiple groups.

The only element that’s new to this function is how we use the writer LangGraph object to emit custom WorkflowProgress events that inform the consumer of the workflow’s progress. Since this workflow can take ~3-5 minutes to run, sending progress messages to the consumer is a vital UX component. Nobody likes waiting for a blank screen. This way, the client can easily display progress messages to the end user or make other decisions, such as sending notifications or emails.

async def _generate_article_workflow(inputs: GenerateArticleInput, config: RunnableConfig) -> str:
    dir_path = inputs["dir_path"]
    dir_path.mkdir(parents=True, exist_ok=True)
 
    writer = get_stream_writer()
 
    writer(WorkflowProgress(progress=0, message="Loading context").model_dump(mode="json"))
    context = {}
    loaders = build_loaders(app_config)
    for context_name in ["article_guideline", "research", "profiles", "examples"]:
        loader = cast(Loader, loaders[context_name])
        context[context_name] = loader.load(working_uri=dir_path)
    writer(WorkflowProgress(progress=2, message="Loaded context").model_dump(mode="json"))
 
    writer(WorkflowProgress(progress=3, message="Genererating media items").model_dump(mode="json"))
    media_items = await generate_media_items(context["article_guideline"], context["research"])
    writer(WorkflowProgress(progress=10, message="Generated media items").model_dump(mode="json"))
 
    writer(WorkflowProgress(progress=15, message="Writing article").model_dump(mode="json"))
    article = await write_article(context["article_guideline"], context["research"], context["profiles"], media_items, context["examples"])
    writer(WorkflowProgress(progress=20, message="Written raw article").model_dump(mode="json"))
 
    article_path = dir_path / app_config.context.build_article_uri(0)
    article_renderer = build_article_renderer(app_config)
    article_renderer.render(article, output_uri=article_path)
    writer(WorkflowProgress(progress=25, message=f"Rendered raw article to `{article_path}`").model_dump(mode="json"))
 
    steps_per_iteration = 3 # review, edit, render
    total_steps = max(1, app_config.num_reviews * steps_per_iteration)
    step_size = 75 / total_steps# remaining percentage after 25for i in range(1, app_config.num_reviews + 1):
        base_step_index = (i - 1) * steps_per_iteration
 
        p_review = int(25 + step_size * (base_step_index + 1))
        p_review = min(p_review, 99)
        writer(WorkflowProgress(progress=p_review, message=f"Rewiewing article [Iteration {i} / {app_config.num_reviews}]").model_dump(mode="json"))
        reviews = await generate_reviews(article, context["article_guideline"], context["profiles"])
        writer(WorkflowProgress(progress=p_review, message="Generated reviews").model_dump(mode="json"))
 
        p_edit = int(25 + step_size * (base_step_index + 2))
        p_edit = min(p_edit, 99)
        writer(WorkflowProgress(progress=p_edit, message="Editing article").model_dump(mode="json"))
        article = await edit_based_on_reviews(
            context["article_guideline"],
            context["research"],
            context["profiles"],
            media_items,
            context["examples"],
            reviews,
        )
        writer(WorkflowProgress(progress=p_edit, message="Edited article").model_dump(mode="json"))
 
        p_render = int(25 + step_size * (base_step_index + 3))
        p_render = min(p_render, 99)
        article_path = dir_path / app_config.context.build_article_uri(i)
        article_renderer.render(article, output_uri=article_path)
        writer(WorkflowProgress(progress=p_render, message=f"Rendered article to `{article_path}`").model_dump(mode="json"))
 
    article_path = dir_path / app_config.context.article_uri
    article_renderer.render(article, output_uri=article_path)
    writer(WorkflowProgress(progress=100, message=f"Final article rendered to `{article_path}`").model_dump(mode="json"))
 
    return f"Final article rendered to`{article_path}`."

The WorkflowProgress pydantic object looks as follows (note that whenever we send it to the writer we have to JSON serialize it by calling .model_dump(mode="json") as the communication between the workflow and the consumer can be done through different processes.

from typing import Annotated
 
from annotated_types import Ge, Le
from pydantic import BaseModel
 
 
class WorkflowProgress(BaseModel):
    progress: Annotated[int, Ge(0), Le(100)]
    message: str

Now, let’s define each step at a time. Each step is decorated with the @task LangGraph decorator that takes the retry_policy as a parameter. This ensures that if an API call fails or a rate limit is hit, LangGraph automatically retries just that specific step without crashing the whole workflow.

The****generate_media_items step implements the orchestrator-worker pattern that generates all diagrams/snippets by fanning out tool calls from the media orchestrator, logging planned jobs, running them in parallel, and returning a typed MediaItems collection. The same logic we presented in the previous lesson.

@task(retry_policy=retry_policy)
async def generate_media_items(article_guideline: ArticleGuideline, research: Research) -> MediaItems:
    writer = get_stream_writer()
 
    model, toolkit = build_model(app_config, node="generate_media_items")
    media_generator_orchestrator = MediaGeneratorOrchestrator(
        article_guideline=article_guideline,
        research=research,
        model=model,
        toolkit=toolkit,
    )
    media_items_to_generate_jobs = await media_generator_orchestrator.ainvoke()
 
    writer(f"Found {len(media_items_to_generate_jobs)} media items to generate using the following tool configurations:")
    for i, job in enumerate(media_items_to_generate_jobs):
        writer(f"  • Tool {i + 1}: {job['name']} - {job.get('args', {}).get('description_of_the_diagram', 'No description')}")
 
    coroutines = []
    for media_item_to_generate_job in media_items_to_generate_jobs:
        tool_name = media_item_to_generate_job["name"]
        tool = media_generator_orchestrator.toolkit.get_tool_by_name(tool_name)
        if tool is None:
            writer(f"⚠️ Warning: Unknown tool '{tool_name}', skipping...")
            continue
        coroutine = tool.ainvoke(media_item_to_generate_job["args"])
        coroutines.append(coroutine)
 
    writer(f"Executing {len(coroutines)} media item generation jobs in parallel.")
    media_items: list[MediaItem] = await asyncio.gather(*coroutines)
    writer(f"Generated {len(media_items)} media items.")
 
    return MediaItems.build(media_items)

The write_article step generates the first full draft using the writer node, combining guidelines, research, profiles, media items, and examples into an Article.

@task(retry_policy=retry_policy)
async def write_article(
    article_guideline: ArticleGuideline,
    research: Research,
    article_profiles: ArticleProfiles,
    media_items: MediaItems,
    article_examples: ArticleExamples,
) -> Article:
    model, _ = build_model(app_config, node="write_article")
    article_writer = ArticleWriter(
        article_guideline=article_guideline,
        research=research,
        article_profiles=article_profiles,
        media_items=media_items,
        article_examples=article_examples,
        model=model,
    )
    article = await article_writer.ainvoke()
    return cast(Article, article)

The****generate_reviews task is used within the evaluator optimizer. It runs the Reviewer on the current article to produce ArticleReviews aligned with guidelines and profiles.

@task(retry_policy=retry_policy)
async def generate_reviews(article: Article, article_guideline: ArticleGuideline, article_profiles: ArticleProfiles) -> ArticleReviews:
    model, _ = build_model(app_config, node="review_article")
    article_reviewer = ArticleReviewer(
        to_review=article,
        article_guideline=article_guideline,
        article_profiles=article_profiles,
        model=model,
    )
    reviews = await article_reviewer.ainvoke()
    return cast(ArticleReviews, reviews)

Then we have the edit_based_on_reviews step that uses the writer in “edit” mode, guided by the generated reviews plus full context, to produce the updated Article.

@task(retry_policy=retry_policy)
async def edit_based_on_reviews(
    article_guideline: ArticleGuideline,
    research: Research,
    article_profiles: ArticleProfiles,
    media_items: MediaItems,
    article_examples: ArticleExamples,
    reviews: ArticleReviews,
) -> Article:
    model, _ = build_model(app_config, node="edit_article")
    article_writer = ArticleWriter(
        article_guideline=article_guideline,
        research=research,
        article_profiles=article_profiles,
        media_items=media_items,
        article_examples=article_examples,
        model=model,
        reviews=reviews,
    )
    article = await article_writer.ainvoke()
    return cast(Article, article)

Ultimately, we can run the whole workflow as a standard LangGraph object calling astream() for streaming intermediate steps from the @task outputs (by picking stream_mode=["values"] and the writer (by picking stream_mode=[”custom”]). This way, we can capture events as they become available, creating the streaming effect you see in chatbots, or, in this case, emit progress events or intermediate results.

import uuid
from brown.memory import build_in_memory_checkpointer
from brown.workflows.generate_article import build_generate_article_workflow
 
async with build_in_memory_checkpointer() as checkpointer:
    workflow = build_generate_article_workflow(checkpointer=checkpointer)
 
    thread_id = str(uuid.uuid4())
    config = {"configurable": {"thread_id": thread_id}}
 
    async for event in workflow.astream(
        {"dir_path": SAMPLE_DIR},
        config=config,
        stream_mode=["custom", "values"],
    ):
        event_type, event_data = event
        if event_type == "custom":
            print(event_data)
        elif event_type == "values":
            print(event_data)

This implementation follows clean code patterns by separating the app layer from the infrastructure layer. The LangGraph workflow represents our app layer, gluing together entities and nodes into usable business logic. The checkpointer acts as the infrastructure layer, providing short-term memory. Notice how the workflow doesn’t care which checkpointer we use (in-memory or SQLite), only that a checkpointer is injected at runtime. This keeps our domain logic independent of infrastructure choices. For example, we could use a different memory builder, such as build_sqlite_checkpointer and easily inject a different checkpointer without changing anything else.

Image 5: The Brown project structure. Image 5: The Brown project structure.

With that in mind, let’s also see how we implemented the short-term memory layer.

Adding Short-Term Memory

We support both in-memory (for testing) and SQLite (for local persistence) checkpointers.

In-Memory Checkpointer : Best for development and testing, as the state is lost when the process ends.

from langgraph.checkpoint.memory import InMemorySaver
 
def build_in_memory_checkpointer():
    return InMemorySaver()

SQLite Checkpointer : Best for local persistence, allowing you to resume between runs if the workflow fails. Even if SQLite is still a file on disk, it can easily support dozens of users during initial testing of your app (aka in PoC mode). Thus, it’s extremely powerful when first deploying your application, as it avoids setting up a database such as Postgres, which already complicates the infrastructure. It’s also super easy to use when testing and debugging locally.

from langgraph.checkpoint.sqlite import SqliteSaver
import sqlite3
 
def build_sqlite_checkpointer(db_path: str = "checkpoints.db"):
    conn = sqlite3.connect(db_path, check_same_thread=False)
    return SqliteSaver(conn)

So what’s the purpose of the checkpointer? It provides:

Debugging : It saves the output from each step. Thus, we can easily run the workflow multiple times from a given step, while freezing the others. This is extremely powerful due to the non-deterministic nature of AI applications. That’s why it’s usually recommended to have one step per LLM call to scope down one non-deterministic component per step.
Resilience : Because the output of each step is saved, we can also resume the workflow in case of failures. For example, if the script crashes during the 2nd review loop, we can resume exactly where it left off.
Human-in-the-Loop : We can pause execution, wait for user input, and resume.

Running the New Writing Workflow

Now that we understand the LangGraph workflow, let’s finally run it. We will use the same code presented a few paragraphs above. Note how we generate a unique thread_id using UUIDs to track this specific run in the checkpointer and stream the events.

import uuid
from brown.memory import build_in_memory_checkpointer
from brown.workflows.generate_article import build_generate_article_workflow
 
SAMPLE_DIR = Path("inputs/tests/02_sample_medium")
 
async with build_in_memory_checkpointer() as checkpointer:
    workflow = build_generate_article_workflow(checkpointer=checkpointer)
 
    thread_id = str(uuid.uuid4())
    config = {"configurable": {"thread_id": thread_id}}
 
    async for event in workflow.astream(
        {"dir_path": SAMPLE_DIR},
        config=config,
        stream_mode=["custom", "values"],
    ):
        event_type, event_data = event
        if event_type == "custom":
            print(event_data)
        elif event_type == "values":
            print(event_data)

The events will look something like this:

-------------------------------------- Event -----------------------------------
  {
  "progress": 0,
  "message": "Loading context"
}
-------------------------------------- Event -----------------------------------
  {
  "progress": 2,
  "message": "Loaded context"
}
-------------------------------------- Event -----------------------------------
  {
  "progress": 3,
  "message": "Genererating media items"
}
-------------------------------------- Event -----------------------------------
  Found 9 media items to generate using the following tool configurations:
-------------------------------------- Event -----------------------------------
    • Tool 1: mermaid_diagram_generator_tool - A flowchart illustrating a simple LLM workflow. It should start with a '_start_' node, proceed to a 'tool_calling_llm' node, then to a 'tools' node representing external operations, and finally conclude with an '_end_' node. This diagram should visually represent a predefined, sequential process where an LLM orchestrates tasks and interacts with external tools or data operations.
-------------------------------------- Event -----------------------------------
    • Tool 2: mermaid_diagram_generator_tool - A flowchart illustrating the AI generation and human verification loop. The loop should start with 'AI Generation', lead to 'Human Review/Verification' (with options to 'Accept' or 'Reject'), and then loop back to 'AI Generation' if rejected, or proceed to 'Task Completion' if accepted. The diagram should emphasize the iterative nature and the human-in-the-loop aspect.
-------------------------------------- Event -----------------------------------
    • Tool 3: mermaid_diagram_generator_tool - A flowchart illustrating an LLM workflow with chaining and routing. It should start with an 'Input' node, lead to an 'LLM Call Router' node that dynamically routes to different 'Specialized LLM Call' nodes (e.g., 'LLM Call A', 'LLM Call B', 'LLM Call C') based on conditions. Each specialized LLM call should then chain to subsequent 'Processing Step' nodes, and finally converge to an 'Output' node.
-------------------------------------- Event -----------------------------------
  Executing 9 media item generation jobs in parallel.
-------------------------------------- Event -----------------------------------
  Generated 9 media items.
-------------------------------------- Event -----------------------------------
  {
  "progress": 10,
  "message": "Generated media items"
}
-------------------------------------- Event -----------------------------------
  {
  "progress": 15,
  "message": "Writing article"
}
-------------------------------------- Event -----------------------------------
  {
  "progress": 20,
  "message": "Written raw article"
}
-------------------------------------- Event -----------------------------------
  {
  "progress": 25,
  "message": "Rendered raw article to `inputs/tests/02_sample_medium/article_000.md`"
}
-------------------------------------- Event -----------------------------------
  {
  "progress": 37,
  "message": "Rewiewing article [Iteration 1 / 2]"
}
-------------------------------------- Event -----------------------------------
  {
  "progress": 37,
  "message": "Generated reviews"
}
-------------------------------------- Event -----------------------------------
  {
  "progress": 50,
  "message": "Editing article"
}
-------------------------------------- Event -----------------------------------
  {
  "progress": 50,
  "message": "Edited article"
}
----------------------------------------------------------------------------------------------------------------------- Event -----------------------------------
  {
  "progress": 62,
  "message": "Rendered article to `inputs/tests/02_sample_medium/article_001.md`"
}
-------------------------------------- Event -----------------------------------
  {
  "progress": 75,
  "message": "Rewiewing article [Iteration 2 / 2]"
}
-------------------------------------- Event -----------------------------------
  {
  "progress": 75,
  "message": "Generated reviews"
}
-------------------------------------- Event -----------------------------------
  {
  "progress": 87,
  "message": "Editing article"
}
-------------------------------------- Event -----------------------------------
  {
  "progress": 87,
  "message": "Edited article"
}
-------------------------------------- Event -----------------------------------
  {
  "progress": 99,
  "message": "Rendered article to `inputs/tests/02_sample_medium/article_002.md`"
}
-------------------------------------- Event -----------------------------------
  {
  "progress": 100,
  "message": "Final article rendered to `inputs/tests/02_sample_medium/article.md`"
}
-------------------------------------- Event -----------------------------------
  Final article rendered to`inputs/tests/02_sample_medium/article.md`.

In the output directory, we see the evolution of the article:

article_000.md: The initial draft.
article_001.md: The result after the first review-edit cycle.
article_002.md: The final polished version.
[article.md](<http://article.md/>): The final article. In this use case, it is the same as article_002.md (because we run the review-edit loop only twice)

To see the article’s evolution across iterations, open the sample we ran for you, which includes all the articles, and compare them using this free tool. You can also open the notebook.ipynb, run it, and generate all the article versions yourself.

Conclusion

In this lesson, we transformed our linear writing workflow into a reliable, self-correcting system by implementing the Evaluator-Optimizer pattern. We modeled structured reviews, implemented a specialized Reviewer node, and upgraded our writer to handle editing tasks. By orchestrating this with LangGraph and adding persistence, we built a resilient engine that iteratively refines its own work.

In the next lesson, we will take this a step further by introducing Human-in-the-Loop into our workflows. We will expose our reviewing and editing capabilities as tools, allowing a human user to intervene, provide specific feedback, and guide the agent to the perfect final draft.

Practicing Ideas

Here are some ideas on how you can further extend this code:

Improve the Reviewer : Tweak the Reviewer prompts or profiles for better results. Get a sense of how it affects the final prompts. The biggest change you could make here is to add few-shot examples to the Reviewer.
Modify the configuration : Change models, temperatures, and num_reviews in YAML and observe output changes.
Add a scoring mechanism to the evaluator : Even though we advised against it, it will be a good exercise to modify the evaluator-optimizer implementation and see how it behaves.
Different AI frameworks : Replace LangGraph with PydanticAI, Google ADK, or other frameworks (clean architecture makes swapping easy).

References

Schluntz, E., & Zhang, B. (n.d.). Building effective agents. Anthropic. anthropic.com/engineering/building-effective-agents
Iusztin, P. (2025, February 23). Stop Building AI Agents. Use These 5 Patterns Instead. Decoding AI Magazine. decodingai.com/p/stop-building-ai-agents-use-these
SwirlAI. (n.d.). Reflection and Working Memory. SwirlAI. newsletter.swirlai.com/p/building-ai-agents-from-scratch-part-8ca
LangChain. (n.d.). Reflection Agents. LangChain Blog. blog.langchain.com/reflection-agents
Moran, H., & Ryan, C. (n.d.). Prompting 101 | Code w/ Claude. YouTube. youtube.com/watch
LangChain. (n.d.). Functional API overview. LangChain. docs.langchain.com/oss/python/langgraph/functional-api
LangChain. (n.d.). Use the functional API. LangChain. docs.langchain.com/oss/python/langgraph/use-functional-api

Lesson 23: Reviewing and Editing Through the Evaluator-Optimizer Pattern

Tip

Tip

Tip

On this page