[Hands-on] Build a Multi-agent Brand Monitoring System

TODAY’S DAILY DOSE OF DATA SCIENCE

​[Hands-on] Build a Multi-agent Brand Monitoring System​

Today, we're building a brand monitoring app that scrapes web mentions of a brand at scale and produces insights about a company.

Tech stack:

• ​Bright Data​ to scrape data at scale.
• CrewAI for orchestration.
• ollama to serve DeepSeek-R1 locally.

Here's the workflow:

• Use ​Bright Data​ to scrape brand mentions across X, Instagram, YouTube, websites, etc.
• Invoke platform-specific Crews to analyze the data and generate insights.
• Merge all insights to get the final report.

Let's implement this!

---

Scraping tool

To monitor a brand, we must scrape data across various sources—X, YouTube, Instagram, websites, etc.

Thus, we'll first gather recent search results from Bright Data's SERP API.

Platform-specific scraping function

The above output will contain links to web pages, X posts, YouTube videos, Instagram posts, etc.

To scrape those sources, we use Bright Data's platform-specific scrapers.

Set up DeepSeek R1 locally

We'll serve R1 locally through Ollama.

To do this:

• First, we download it locally.
• Next, we define it with the CrewAI's LLM class.

Here's the code👇

Crew Setup

We will have multiple Crews, one for each platform (X, Instagram, YouTube, etc.)

Each Crew will have two Agents:

• Analysis Agent → It analyses the scraped content.
• Writer Agent → It produces insights from the analysis.

Below, let's implement the X Crew!

Note: The implementation for other Crews is available in the GitHub repo linked later.

X Analyst Agent

This Agent analyzes the posts scraped by Bright Data and extracts key insights. It is also assigned a task to do so.

X Writer Agent

The Agent takes the output of the X analyst agent and generates insights.

Create a Flow

Finally, we use CrewAI Flows to orchestrate the workflow:

• We start the Flow by using the Scraping tool.
• Next, we invoke platform-specific scrapers.
• Finally, we invoke platform-specific Crews.

We wrap the app in a clear streamlit interface for interactivity and run the Flow.

When Agents use tools, they run into issues like IP blocks, bot traffic, captcha solvers, etc. This hinders the Agent's execution.

​Grab the API_KEY here →​

It lets you:

• Scrape data for Agents at scale without getting blocked.
• Simulate user behavior using advanced browser tools.
• Build Agentic apps with real-time and historical web data.

Thanks to Bright Data for working with us on this demo.

Find the code in this GitHub repo: ​Brand monitoring repo​.

Thanks for reading!

No-Fluff Industry ML resources to

Succeed in DS/ML roles

At the end of the day, all businesses care about impact. That’s it!

• Can you reduce costs?
• Drive revenue?
• Can you scale ML models?
• Predict trends before they happen?

We have discussed several other topics (with implementations) in the past that align with such topics.

Here are some of them:

• Learn sophisticated graph architectures and how to train them on graph data in this crash course.
• So many real-world NLP systems rely on pairwise context scoring. Learn scalable approaches here.
• Run large models on small devices using Quantization techniques.
• Learn how to generate prediction intervals or sets with strong statistical guarantees for increasing trust using Conformal Predictions.
• Learn how to identify causal relationships and answer business questions using causal inference in this crash course.
• Learn how to scale and implement ML model training in this practical guide.
• Learn 5 techniques with implementation to reliably test ML models in production.
• Learn how to build and implement privacy-first ML systems using Federated Learning.
• Learn 6 techniques with implementation to compress ML models.

All these resources will help you cultivate key skills that businesses and companies care about the most.

SPONSOR US

Advertise to 600k+ data professionals

Our newsletter puts your products and services directly in front of an audience that matters — thousands of leaders, senior data scientists, machine learning engineers, data analysts, etc., around the world.

Get in touch today →