Case Study: Slack Knowledge Assistant

Every team has the same problem: information is scattered across tools. The answer to "how does our authentication system work?" lives partly in a Notion doc, partly in a GitHub PR, partly in a Slack thread from three months ago, and partly in a Linear ticket. When someone asks in Slack, the response is either silence or a 10-minute scavenger hunt.

This article walks through building an open-source Slack bot that answers questions by searching across all of your connected tools using Airweave. We cover the architecture, the pipeline design, and the patterns that make it work well in practice.

The full implementation is available at github.com/airweave-ai/slack-knowledge-assistant.

What It Does

The assistant is a Slack bot. Mention it in a channel or send it a DM, and it searches across all of your company's connected sources (GitHub, Notion, Linear, Google Drive, Slack itself, and anything else synced to Airweave), generates an answer grounded in what it finds, and replies with source citations linking back to the original documents.

The bot reacts with a thinking emoji, searches, generates an answer, and posts a rich reply with citations. If a teammate replies in the thread while the bot is still working, it adapts its response so it doesn't repeat what a human already said.

It also handles threaded conversations. Ask a follow-up like "what about the API?" in the same thread, and the assistant rewrites your question into a standalone search query using the conversation history as context. This makes it feel like a real conversation rather than a series of isolated lookups.

Architecture

The assistant is a FastAPI application that receives Slack events via webhook. The pipeline has six stages, each feeding into the next:

The event handler's only job is to acknowledge the Slack event within three seconds (Slack's timeout requirement) and hand off the actual work to a background thread. Everything interesting happens in the pipeline.

Query Contextualization

The most important design decision in the assistant is how it handles follow-up questions. In a thread, users ask things like "what about the API?" or "who built that?" These questions are meaningless without the preceding conversation.

Before searching, the assistant sends the full thread history to a fast model (Claude Haiku) with a simple instruction: rewrite the user's follow-up as a standalone search query. If the thread started with "How does our authentication system work?" and the follow-up is "what about the API?", the rewritten query becomes something like "authentication system API implementation."

This is a small step that has a large impact on search quality. Without it, Airweave would receive "what about the API?" as the search query and return generic API results. With it, the search is grounded in the conversation's actual topic.

# Simplified: contextualize follow-up using thread history
query = await contextualize_query(
    user_message="what about the API?",
    thread_history=thread_messages,
    model="claude-3-5-haiku-latest"
)
# Result: "authentication system API implementation details"

# Simplified: contextualize follow-up using thread history
query = await contextualize_query(
    user_message="what about the API?",
    thread_history=thread_messages,
    model="claude-3-5-haiku-latest"
)
# Result: "authentication system API implementation details"

# Simplified: contextualize follow-up using thread history
query = await contextualize_query(
    user_message="what about the API?",
    thread_history=thread_messages,
    model="claude-3-5-haiku-latest"
)
# Result: "authentication system API implementation details"

# Simplified: contextualize follow-up using thread history
query = await contextualize_query(
    user_message="what about the API?",
    thread_history=thread_messages,
    model="claude-3-5-haiku-latest"
)
# Result: "authentication system API implementation details"

Searching with Airweave

Once the query is contextualized, the assistant searches the Airweave collection. This is a single API call that searches across every connected source:

response = await client.collections.search(
    readable_id=collection_id,
    query=contextualized_query,
    limit=10
)

response = await client.collections.search(
    readable_id=collection_id,
    query=contextualized_query,
    limit=10
)

response = await client.collections.search(
    readable_id=collection_id,
    query=contextualized_query,
    limit=10
)

response = await client.collections.search(
    readable_id=collection_id,
    query=contextualized_query,
    limit=10
)

Behind the scenes, Airweave runs hybrid search (combining semantic similarity with keyword matching) and reranks results for precision. The assistant receives ranked results from GitHub, Notion, Linear, Slack, and any other connected source, each with metadata including the source type, original URL, and relevance score.

This is the step that replaces what would otherwise be a sprawling set of custom API integrations. Without Airweave, you would need a GitHub search client, a Notion search client, a Linear search client, and logic to merge and rank their results. With Airweave, it's one call.

Source-Aware Answer Generation

The assistant doesn't just dump search results into a prompt. It generates an answer using Claude, passing the search results as context, and then polishes the output specifically for Slack.

The polishing step is worth highlighting. The assistant adapts its language based on where information comes from:

• Notion results get framed as "This is documented in..."

• GitHub results become "This is implemented in..."

• Linear or Jira results become "This is tracked in..."

• Slack results become "This was discussed in..."

This source awareness makes answers feel natural rather than robotic. It also builds trust, because readers can immediately tell whether they're looking at official documentation, actual code, a ticket, or a casual conversation.

Each source citation includes a link back to the original document, so readers can verify or dig deeper.

Handling Concurrent Human Replies

There's a subtle timing problem with any Slack bot: what happens when a teammate answers the question while the bot is still processing? Without handling this, the bot posts a response that repeats what a human already said, which feels redundant and annoying.

The assistant solves this by checking the thread for new replies just before posting its answer. If a human has responded in the meantime, the assistant revises its answer to acknowledge the human reply and add only the additional context that the human didn't cover.

This is a small detail that significantly improves the experience in active channels where humans and the bot are both responding.

Confidence Grading

Not all search results are equally reliable. A well-maintained Notion doc is more authoritative than a casual Slack message from six months ago. The assistant grades its confidence based on the quality, recency, and source type of the results it found.

When confidence is low (for example, when the only results are tangentially related Slack messages), the assistant signals this in its response rather than presenting uncertain information with false confidence. This is important for building team trust in the assistant over time.

Deployment

The assistant is a standard FastAPI application. It can be deployed anywhere that runs Python: Railway, Render, Fly.io, or a simple Docker container.

FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

Configuration is handled entirely through environment variables:

For local development, run the server with uvicorn and expose it via ngrok so Slack can reach your event handler.

Key Patterns

Several patterns from this implementation apply broadly to any agent built on Airweave:

Rewrite before searching. If your agent handles multi-turn conversations, always contextualize the user's message before sending it to Airweave. The difference in search quality between "what about the API?" and "authentication system API implementation" is the difference between useless and useful results.

Let Airweave handle the source complexity. The assistant's codebase contains zero source-specific search logic. No GitHub client, no Notion client, no Linear client. Adding a new source (say, Confluence) means connecting it to the Airweave collection. The assistant code doesn't change at all.

Adapt output to source type. When presenting results to users, use the source metadata that Airweave provides. Framing information differently based on whether it came from documentation, code, a ticket, or a conversation makes answers more credible and easier to act on.

Handle the real-world timing issues. Bots that ignore what happens while they're processing feel broken. Check for concurrent activity before posting, and adapt accordingly.

Looking Ahead

The Slack Knowledge Assistant demonstrates a pattern that extends beyond Slack. Any messaging or collaboration interface (Discord, Teams, a custom chat UI) can use the same pipeline: receive a question, contextualize it, search Airweave, generate a grounded answer, and present it with source citations.

The assistant's entire value comes from the quality of the context it retrieves. The pipeline logic (query rewriting, answer generation, formatting) is straightforward. What makes it useful is having a single, continuously updated retrieval layer across all of the tools a team actually uses.

The complete implementation is available at github.com/airweave-ai/slack-knowledge-assistant.