Copilot Studio provides business professionals an exciting opportunity to create custom chatbots and virtual agents in a low/no-code environment. However, even though you don’t need to write the code yourself, it’s still important to understand the settings and options you select, as they can significantly impact the final product.
In this post, we’ll dive into a distinction that I found especially confusing when starting out with Copilot Studio: the difference between generative AI orchestration and generative AI knowledge. At first, I wondered how one could function without the other, given that both stem from generative AI. To clear up the confusion, let’s explore what each of these terms means, how AI plays a role, and why they don’t always go hand in hand.
To understand what I’m referring to, visit copilotstudio.microsoft.com and check out the “Overview” section for setting up the agent, where you’ll find these two parts:
Let’s briefly review what each of these entails and how they function on their own, before we dive into comparing and contrasting them.
“Use generative AI to determine how best to respond to users and events”
Generative orchestration lets the AI agent dynamically pick and combine resources like topics, actions, or knowledge sources to respond to user input or events. Think of it as the AI conducting a symphony, blending what’s configured (your custom data or actions) into a smart, context-aware response. It uses its language model to understand intent and craft answers on the fly, relying on your setup rather than broad external knowledge.
This makes modern AI chatbots, like those in Copilot Studio, feel human-like compared to old-school, rule-based bots. Legacy chatbots are stiff—tied to scripts and exact keywords, spitting out canned replies or “I don’t get it” if you veered off-path.
A generative AI agent, though, is designed to understand your meaning even if you’re vague (e.g., “How much time off?” vs. “What’s the vacation policy?”) and pulls a natural answer from your data. Under the hood, it swaps rigid flowcharts for real-time reasoning, handling ambiguity and connecting dots across your resources.
“Allow the AI to use its own general knowledge”
This option, when enabled, permits the AI to tap into its pre-trained, broad knowledge base—essentially the foundational data it was trained on before being fine-tuned or configured in Copilot Studio. This is the “general knowledge” that large language models (like those powering Copilot Studio) acquire during their initial training, covering a wide range of topics from public data sources up to a certain point. When this setting is turned on, the AI can answer questions or provide information that goes beyond the specific knowledge sources, topics, or actions you’ve explicitly provided in the agent’s configuration.
For example, if a user asks, “Why is the sky blue?” and your agent doesn’t have a topic or knowledge source covering that, turning on this option allows the AI to draw from its pre-trained understanding to respond, rather than saying it doesn’t know. However, this comes with a trade-off: the responses might not be grounded in your specific data, and there’s a risk of less control over accuracy or relevance to your domain.
How do these options differ?
So, how do these two options stack up? The first one, generative orchestration, is all about working within the sandbox you’ve built.
It uses your configured resources to generate responses, giving you tight control over what the AI says. The second option, general knowledge, opens the door wider, letting the AI pull from its broader training. This makes it more versatile but less predictable.
If you’re building an assistant for a customer service team, you might want it to stick to company policies (Option 1), whereas a general-purpose bot might benefit from answering random questions (Option 2). For data analysts, the choice often depends on whether you need precision within your dataset or flexibility for unexpected queries. The purpose differs too—Option 1 optimizes your resources, while Option 2 fills gaps when your data doesn’t have the answer.
Can generative AI work without general knowledge?
Here’s a question that might be on your mind: how does a generative AI option work at all without tapping into general knowledge? Here’s how.
Generative AI, at its core, is about understanding language and creating responses based on patterns it’s learned. In Copilot Studio, when you enable Option 1 and disable Option 2, the AI still works: it parses user queries, matches them to your resources, and generates answers, all without needing to know about the Roman Empire or the physics of rainbows. Generative AI doesn’t rely on general knowledge—it thrives within your boundaries, making it perfect for controlled, domain-specific tasks.
When to use which
So, when should you use these options?
If you’re after precision—like an assistant for HR or a customer service bot that only reflects company data—enable Option 1 and skip Option 2. It keeps everything grounded in your Excel files or SharePoint docs, which is gold for analytics work.
If you want a jack-of-all-trades assistant that can handle sales queries and random trivia, turn on both. Just keep an eye on accuracy, especially with general knowledge responses, since they might not always fit your context.
In general, disabling Option 1 entirely isn’t something I’d recommend. It limits the AI to rigid, pre-set behaviors, which isn’t as dynamic or useful for modern analytics tasks. Your choice depends on your goals: tight control or broad capability.
Conclusion
In this post, we delved into the best methods for incorporating generative AI into our Copilot Studio agents, exploring its use in both orchestration and knowledge components. We covered the benefits, drawbacks, and specific scenarios for each method.
I hope this summary shed some light on how agents function and provided a high-level understanding of generative AI’s role.
What are your thoughts or questions about adding generative AI to Copilot Studio, or about Copilot Studio in general? Please let me know in the comments.
Leave a Reply