The Rise of Agentic AI: Lessons from Building with Copilot Studio

In today’s hyper-connected corporate world, speaking the same language—literally—can be the key to unlocking smoother collaboration, deeper understanding, and new opportunities. I saw a chance to bridge a common communication gap and decided to build an accessible, engaging tool to help myself in learning Dutch, especially that tricky pronunciation. The result? An AI Dutch Language Tutor, powered by Microsoft Copilot Studio and seamlessly integrated into the daily workflow of Microsoft Teams.

Let’s pull back the curtain on this exciting project, sharing my vision, the nitty-gritty of the architecture, a deep dive into each component, and the lessons I learned along the way.

1. The Spark: Why a Dutch Language Tutor?

Let’s face it, learning a new language while juggling a demanding job is tough. Finding dedicated time, getting instant feedback on your pronunciation, or using generic apps that feel worlds away from your work life are common hurdles. My vision was simple: bring the learning experience directly to where many of us spend our day – Microsoft Teams, but with the intelligence driven by Microsoft Copilot Studio.

I envisioned a “Dutch Language Tutor” that would:

Feel like a natural, AI-driven conversation.
Offer bite-sized practice modules, kicking off with pronunciation.
Deliver objective, real-time feedback using cutting-edge speech technology.
Be secure and super easy to access.

My goal wasn’t just another app; I wanted to create a supportive learning companion

2. The Blueprint: Designing the AI Tutor

To bring this idea to life, I sketched out a multi-layered architecture, getting various Microsoft technologies to sing in harmony.

Imagine a user wanting to practice Dutch.

Their journey starts in Microsoft Teams, where they chat with the Dutch Language Tutor.
The conversational brain built in Copilot Studio. When they opt for pronunciation practice, the Tutor doesn’t just pick any word; it calls a Power Automate flow. This flow acts as a scout, venturing out to an External API to fetch a random Dutch word.
With the word in hand, the Copilot crafts a special Teams Deep Link and sends it back to the User in Teams.
A click on this link magically opens our custom-built Teams Tab Application – an interactive webpage hosted on Azure Static Web Apps. This is where the real practice happens!
The Tab App first displays the word. To power the pronunciation assessment, it needs a secret key for Azure Speech Services. So, it securely calls a dedicated Azure Function, which acts as a vault, safely providing the necessary API key and region.
Now, the stage is set. The user records their voice.
The Azure Speech SDK, running right in their browser within the Tab App or the Teams desktop client analyzes their pronunciation against the target word, calculating scores for accuracy, fluency, and more. These scores pop up on their screen.
Happy with their practice, the user clicks ‘Submit.’
The Tab App then gathers all these results, including the user’s Teams ID, and sends them via an HTTP POST request to a second Power Automate flow.
This ‘results receiver’ flow then formats a friendly message and uses the Teams connector to send the feedback as a direct message to the user in Teams, closing the loop.

Key Components:

User in MS Teams: The starting and ending point of the interaction.
Copilot Studio (Dutch Tutor Bot): The conversational brain, orchestrating the initial flow.
Power Automate (Flow 1 – DutchVocabPractice): Fetches random words.
External Random Word API: Source of practice words.
Teams Tab Application (index.html): The interactive practice environment.
Hosts: Azure Speech SDK for client-side assessment.
Azure Function (GetSpeechCredentials): Securely provides Speech API keys.
Power Automate (Flow 2 – ReceivePronunciationFeedback): Receives results from the tab and posts feedback to Teams.

A Closer Look at Each Component

Let’s pop the hood and see how each piece of this tech puzzle works.

Core Components

Microsoft Copilot Studio: Manages the user conversation, topic flows, and calls to Power Automate.
Microsoft Teams Tab Application (index.html): A single-page web application hosted on Azure Static Web Apps, providing the UI for pronunciation practice.
Azure Functions:
- GetSpeechCredentials: Securely provides Azure Speech API keys to the Teams Tab App.
- TextToSpeechFunction: (For future vocabulary enhancement) Converts text to spoken audio.
Power Automate Flows:
- DutchVocabPractice: Fetches random Dutch words (and potentially definitions) for practice.
- ReceivePronunciationFeedback: Receives pronunciation scores from the Teams Tab App and posts them as a message in Teams.
- ConvertTextToSpeech: (For future vocabulary enhancement) Orchestrates calling the TextToSpeechFunction.
Teams Developer Portal / App Manifest: Defines the Teams application, its tabs, permissions, and other metadata.
Azure AI Speech Service: Provides Speech-to-Text (for pronunciation assessment) and Text-to-Speech capabilities.
Azure Static Web Apps: Hosts the index.html file for the Teams Tab application.

Configuration Steps:

Azure Resource Setup

Azure AI Speech Service:

Create a “Speech” service resource in the Azure portal.
Note down its Key1 (or Key2) and Region (e.g., westeurope). These will be used by your Azure Functions.

Azure Function App:

Create a “Function App” resource in the Azure portal.
- Note the Function App name (e.g., dutch-language-tutor-fn)
- Runtime stack: Node.js (e.g., version 18 LTS or higher).
- Hosting: Consumption plan is suitable for this use case.

Azure Static Web App:

Create a “Static Web App” resource in the Azure portal.
Configure its deployment source (e.g., GitHub, Azure DevOps, or manual upload via SWA CLI).
This will host your index.html and any related assets (CSS, other JS files if any).
Note its default URL (e.g., https://abcxxxxxx-0a0c85f03.6.azurestaticapps.net).

Teams App Setup (Developer Portal)

Go to the Teams Developer Portal (dev.teams.microsoft.com).
Create a new app or import your existing manifest.
Key Manifest (manifest.json) Details:
- id: Your unique App ID.
- name, developer, description: Fill as appropriate.
- staticTabs: Define your pronunciation practice tab.

{
"staticTabs": [
    "entityId": "pronunciation-tab",
    "name": "Practice Dutch",
    "contentUrl": "https://gray-mushroom-0a0c85f03.6.azurestaticapps.net/index.html", // URL to your hosted index.html
    "websiteUrl": "https://gray-mushroom-0a0c85f03.6.azurestaticapps.net",
    "scopes": ["personal"]
  }
]

validDomains: Must include the domain of your Azure Static Web App and your Azure Function App.
devicePermissions:
- Ensure “media” is included for microphone access.
  - “devicePermissions”: [“media”]
Upload icons.
Install and test the app in Teams.

Azure Function 1: GetSpeechCredentials

Purpose: Securely provide Azure Speech API key and region to the index.html tab app.
Trigger: HTTP Trigger.
Function App Application Settings:
- SPEECH_SUBSCRIPTION_KEY: Your Azure Speech service Key1.
- SPEECH_SERVICE_REGION: Your Azure Speech service Region.

Code (index.js for the function):

Platform CORS Settings (for the Function App):
- In the Azure portal, navigate to your Function App > API > CORS.
- Add your static web app to the “Allowed Origins”. Save.

Function URL: Get and note down the function URL (including the ?code=… key). This will be AZURE_FUNCTION_GET_KEY_URL in index.html.

Teams Tab Application (index.html)

Hosted on Azure Static Web Apps.
HTML Structure (on Github repo)
JavaScript Logic (Key parts):
- Constants:
  const AZURE_FUNCTION_GET_KEY_URL = “YOUR_AZURE_FUNCTION_URL_FOR_GetSpeechCredentials”;
  const POWER_AUTOMATE_FEEDBACK_URL = “YOUR_POWER_AUTOMATE_HTTP_TRIGGER_URL_FOR_ReceivePronunciationFeedback”;
- SDK Loading:
- <script src=”https://res.cdn.office.net/teams-js/2.20.0/js/MicrosoftTeams.min.js”></script> <script src=”https://aka.ms/csspeech/jsbrowserpackageraw”></script>
Initialization (DOMContentLoaded):
- Initialize Teams SDK (microsoftTeams.app.initialize()).
- Get Teams context (microsoftTeams.app.getContext()).
- Determine currentReferenceText (word to practice) in order of preference:
  - teamsClientContext.page.subPageId (from Teams deep link context).
  - ?word= URL query parameter.
  - Default word (e.g., “Hallo”).
- Display currentReferenceText in practiceWordEl.
- Call fetchAzureSpeechCredentials() to get key/region from AZURE_FUNCTION_GET_KEY_URL.
- Configure Azure Speech SDK (SpeechSDK.SpeechConfig.fromSubscription, SpeechSDK.AudioConfig.fromDefaultMicrophoneInput).
- Enable the “Start Recording” button.

startPronunciationAssessment() function:
- Creates SpeechSDK.PronunciationAssessmentConfig.
- Creates SpeechSDK.SpeechRecognizer.
- Includes detailed event logging (sessionStarted, speechStartDetected, etc.).
- Calls currentRecognizer.recognizeOnceAsync().
- In the callback, if sdkResult.reason === SpeechSDK.ResultReason.RecognizedSpeech:
  - Gets PronunciationAssessmentResult.fromResult(sdkResult).
  - Extracts accuracyScore, fluencyScore, completenessScore, pronunciationScore.
  - Calculates overallScore.
  - Updates UI elements to display recognized text and scores.
  - Makes the results section and “Submit Feedback” button visible.
- Handles NoMatch and Canceled reasons.
- Closes the recognizer.
handleSubmitFeedbackViaPowerAutomate() function:
- Constructs feedbackData object including scores and teamsClientContext.user.id.
- Makes an HTTP fetch POST request with feedbackData to POWER_AUTOMATE_FEEDBACK_URL.
- Provides UI feedback (e.g., alert or status update).

The Power of Flows

Power Automate Flow 1: DutchVocabPractice

Trigger: “When Power Virtual Agents calls a flow” / “When an action is called from Copilot Studio”.
Action 1: HTTP GET Request:
- Method: GET
- URI: URL of your chosen random word API (e.g., the new one you found that works).
Action 2: Parse JSON:
- Content: Body from the HTTP action.
- Schema: Generate from a sample response from the word API.
Action 3: Respond to your Copilot / Return value(s) to Power Virtual Agents:
- Define outputs:
  - wordToPractice (Text): Map from the parsed JSON (e.g., body(‘Parse_JSON’)?[‘word’]).
  - Definition (Text, optional): Map from parsed JSON.
  - Pronunciation (Text, optional): Map from parsed JSON.

Power Automate Flow 2: ReceivePronunciationFeedback

Trigger: “When a HTTP request is received”.
Action 1: Parse JSON:
- Content: Body from the HTTP trigger.
- Schema: Use the same schema as defined in the trigger.
Action 2: Microsoft Teams – “Post message in a chat or channel” (or a more specific “post to user” if available and preferred).
- Post as: Flow bot
- Post in: Chat with Flow bot
- Recipient: Dynamic content teamsUserId from the “Parse JSON” step.
- Message: Compose a message using dynamic content from “Parse JSON” to display the word and scores (e.g., “Feedback for ‘{word}’: Accuracy {accuracy}%…”).

Copilot Studio Topic Configuration

Trigger: User phrases (e.g., “Hello”, “Hi”).
Initial Message & Question (Multiple Choice):

“Hi {User.DisplayName}, I am Dutch Language Tutor… What would you like to learn & practice today?”
Options: “Practice Vocabulary”, “Improve Pronunciation”, etc.
Save response to Topic.userIntent.

Condition on Topic.userIntent:

If Topic.userIntent is “Improve Pronunciation”:
- Message: “It’s always nice to have clear pronunciation…”
- Question: “Are you ready to begin?” (Multiple Choice: Yes/No). Save to Topic.vocabReady.
- Condition: Topic.vocabReady is “Yes”.
  - If Yes:
    - Action (Call Flow): DutchVocabPractice.
    - Store outputs: Topic.wordToPractice, Topic.Definition, Topic.Pronunciation.
    - Set Variable Value: Topic.TeamsDeepLinkToTab
    - Formula: (The confirmed working formula to construct the Teams deep link with Topic.wordToPractice in subEntityId).
- Send a Message:
  - Text: “The word to practice is {Topic.wordToPractice}. Please click the link to open the pronunciation tool: ${Topic.TeamsDeepLinkToTab}” (or use Markdown for a cleaner link).

(The topic ends here for this interaction. Feedback will come as a separate message from the Power Automate flow).

Under The Hood:

Microsoft Copilot Studio: The Conversational Conductor

Copilot Studio is the undisputed star of our tutor, handling the chat and logic.

It manages the dialogue, figures out what the user wants (e.g., “I want to practice pronunciation”), and kicks off the right processes.

The Interactive Hub: Teams Tab Application (index.html)

This is where the pronunciation practice comes alive! It’s a single HTML page, beefed up with JavaScript, and hosted on Azure Static Web Apps.

Azure Function (GetSpeechCredentials): The Keymaster

Security first! This Azure Function acts as a secure vault for our Azure Speech API key.
Provides the Speech SDK key and region to index.html without exposing them in client-side code.

Power Automate Flow 1 (DutchVocabPractice): The Word Scout

This flow is our trusty word-fetcher.

Power Automate Flow 2 (ReceivePronunciationFeedback): The Messenger

This flow delivers the practice results right back to the user.

The User’s Journey: A Quick Walkthrough

My Development Adventure: Key Learnings & Troubleshooting Tales

Building this wasn’t always a walk in the park. Here are some of the biggest takeaways:

CORS is King (and a Tricky One!): Getting index.html (client) to talk to the Azure Function (server, different domain) meant diving deep into Cross-Origin Resource Sharing. We needed CORS configured in both the Azure Function’s response headers and at the Azure Function App’s platform level.
Teams SDK Quirks:
- That integrity attribute in the Teams JS SDK script tag? It caused loading fails when the CDN updated. Removing it temporarily got us moving again (lesson: manage SRI hashes carefully!).
- Understanding how subEntityId travels via deep link context and how microsoftTeams.app.getContext() reads it was key for the tab to get the practice word.

API Roulette: Relying on free external APIs for random words can be a gamble (rate limits, downtime). For a production app, a more robust API or internal solution is better.
Copilot Studio Formula Fun: String concatenation (& was the magic character) and variable typing in Copilot Studio needed some trial and error. Simple “Send a message” nodes became our best friends for debugging variable values.
Task Modules vs. Deep Links for Feedback – The Great Debate:
Our dream was for Copilot Studio to launch the tab as a task module and get feedback directly via microsoftTeams.tasks.submitTask(). This would keep the feedback neatly in the same chat. But, wrestling the “Send an activity” node in Copilot Studio to accept the precise JSON-like object proved too tricky for V1.
My current solution (deep link -> tab POSTs to Power Automate -> Power Automate DMs user) works well, but the feedback is in a separate chat. It’s a classic case of balancing ideal UX with platform constraints.
The Unsung Hero: console.log(): When index.html was acting up (like “No speech recognized”), detailed console.log statements for Teams context, fetched credentials, Speech SDK events, and variable states were our lifeline.

What’s Next? The Road Ahead

The Dutch Language Tutor is a solid Version 1.0, but I’m already dreaming up enhancements:

Seamless In-Conversation Feedback: I’ll tackle that “Send an activity” node in Copilot Studio again to launch the tab as a task module. This would let microsoftTeams.tasks.submitTask() send results right back to the Copilot for in-thread feedback.
Track Progress & Gamify: Store practice history (maybe in Dataverse?) so users can see their improvement. Add badges or points for a bit of fun!
Smarter Learning: Adjust word difficulty based on how users are doing.
More to Learn: Introduce modules for vocabulary (flashcards in Adaptive Cards, anyone?), reading, and writing.
Pinpoint Pronunciation: The Azure Speech SDK gives phoneme-level accuracy. Showing this in the tab could offer super targeted feedback.
Hear it Right: Let users hear the correct pronunciation, maybe using Azure Text-to-Speech.

Final Thoughts: A Symphony of Services

Building this Dutch Language Tutor has been an amazing demonstration of how powerful Microsoft’s cloud services are when they work together. From Copilot Studio’s AI smarts to Power Automate’s slick automation, Azure Functions’ secure backend logic, Azure Speech SDK’s client-side magic, and Teams’ user-friendly interface – every piece played a crucial part.

Sure, I hit some bumps, especially with inter-service communication and client-side setups in Teams. But the result is a genuinely useful tool. It proves that with a bit of grit and a methodical approach, you can build sophisticated, AI-enhanced apps that solve real problems and make learning more engaging, right where people work.

This project isn’t just a handy utility; it’s an inspiring peek at what’s possible when you orchestrate these incredible platforms effectively.

What are your thoughts? Have you built similar tools or have ideas for other learning applications? Share in the comments below!

1. The Spark: Why a Dutch Language Tutor?

2. The Blueprint: Designing the AI Tutor

A Closer Look at Each Component