Scout Best Practices

Scout is a powerful tool for helping your customers get quick, accurate responses via chat and email — but the experience you deliver depends on how thoughtfully you set it up and maintain it over time. This guide walks you through what to do to get started, how to shape and refine the way your Scout behaves, and how to build a routine for improving results.

Before diving in, make sure you've completed the initial setup — head to Get Started With Scout to find instructions to set it up if you haven't yet.

Start With Good Sources

Take a moment to think about what you want Scout to do well. The clearest signal to it is the quality of your Docs articles — the AI draws from your Help Scout Docs site (or other knowledge base added to the Scout's sources).

Before adding any other inputs, review your existing articles with fresh eyes:

• Are the articles accurate and up to date?
• Do they answer the questions customers actually ask?
• Are there topics customers frequently contact support about that don't have a Docs article yet?
• Are your articles clear and direct?

Filling those gaps in your Docs is the highest-leverage thing you can do. The better your documentation, the better Scout will be out of the gate.

Scout can also pull from external sources — public websites, PDFs, Word documents, spreadsheets, and more. Keep sources focused. Adding targeted pages with reliable, relevant information will produce more accurate answers than adding an entire website.

Check out Write Docs That Work Well With Scout for more help with shoring up your documentation!

Shape Scout's Behavior

Beyond the knowledge sources you add, three settings shape how your AI Agent communicates with customers: Role, Text sources, and Exclusions. Each has a different purpose, and using them together gives you precise control over the experience.

Head to Scout > Manage to find these settings.

Roles

The field Character under Roles is where you define who your Scout is. Use it to establish the tone, style, and general approach it takes when talking to customers. You can give it direction about your brand's voice, how formal or conversational it should be, and what kinds of things it should or shouldn't do in general.

A few things to keep in mind:

• Character shapes how the AI communicates — it doesn't introduce new information. If you want Scout to know something specific, use an Improvement instead.
• Test Scout after making changes here to confirm the AI is responding the way you expect before customers see it.

Text Sources

Text sources are short pieces of information or instruction that you add directly to Scout. They're the right tool when:

• Scout's answer was mostly right but missed a key detail or took a slightly wrong direction.
• Customers ask about something you don't offer — and you want Scout to respond clearly without it sounding like a refusal.
• A question comes up occasionally that's too specific or unusual to warrant a full Docs article.
• The answer involves account-specific help or requires a team member to step in — add an Improvement to direct the customer to contact support.

You can add a new Text source from a previous conversation by clicking Improve, by heading to Scout > Manage and adding it directly, or from a test conversation at Scout > Test.

Text sources are not a replacement for good Docs. If a question is common and the answer is relevant to most customers, update your Docs. If the issue is specific, rare, or support-dependent, a Text source is the right call.

After adding a Text source, test it. Head to Scout > Test and ask it questions related to what you added. If the answer still isn't quite right, go back to Scout > Manage > Sources, find the Text, and refine it.

Exclusions

Some topics are better handled by your team — billing disputes, account investigations, sensitive escalations, or anything that requires a human judgment call. Exclusions tell Scout to step back from those topics and hand the conversation off rather than attempting an answer.

Add an Exclusion when:

• A topic always requires team involvement, regardless of how the question is phrased.
• You want to make sure the AI never attempts an answer on a sensitive subject.

Each Beacon that uses Scout has a setting for how the agent should respond when a excluded topic comes up. Customize those response settings on each Beacon, just head to Tools > Beacons > [Your Beacon] > Edit Beacon > Scout and scroll down to find Customize messages to make sure the handoff message feels right for your customers.

Evaluate and Improve Over Time

Setting up Scout is the beginning, not the end. The teams that get the best results treat it as an ongoing practice.

Start by reviewing conversations regularly. Head to Reports > Scout, and select your Beacon. Sort by resolution (just click the Resolution header on the table) to find the conversations where customers weren't helped. Read the full conversation — sometimes the last message changes what Scout should have done earlier in the exchange.

A useful question to ask while reviewing: "If a customer sent these same questions in an email, would our team be able to respond more clearly or completely?" If yes, there's an improvement opportunity. If no, the customer may simply not have given enough detail to answer — and that's not something you can fix.

Resolutions are broken down by outcome:

• Contact helped — an answer was provided and the visitor didn't indicate they needed more.
• Contact not helped — the visitor said they still needed help but didn't reach out.
• Human escalation — the visitor chose to chat or send a message for further help.

A high rate of "contact not helped" often points to answers that are technically present but not satisfying. That's a signal to look at the quality of those Docs articles or add targeted Text sources. A high rate of human escalation may indicate a topic that belongs in an Exclusion.

A Note on How AI Works

One thing worth understanding as you manage Scout: AI does not work the same way a database lookup or a search engine does.

A search engine finds an exact match. Given the same query twice, it returns the same result. AI is different — it's probabilistic rather than deterministic. It generates a response based on patterns across your knowledge sources, the question asked, and the context of the conversation. That means two very similar questions might get slightly different answers, and a response that worked well once may vary the next time.

This is normal and expected. It also means that testing is essential. After you make changes to your Identity, add an Improvement, or update your knowledge sources, test with several different phrasings of the same question — not just one. A good answer to one version of a question doesn't guarantee consistent results across all variations.

It also means that more precise inputs help. Vague or conflicting information in your knowledge sources or Improvements introduces more variability. The clearer and more consistent your sources, the more reliably Scout will respond.

Think of managing Scout less like configuring a rule and more like coaching someone: you give guidance, observe results, and refine. Over time, the right habits — regular review, targeted Text sources, clear Docs — add up to a much better customer experience.