Bringing AI to Meetings with the Sample Builder
We’re excited to share a significant update to the Azure Communication Services Sample Builder. This release integrates Azure’s latest AI and video calling capabilities, implementing meeting transcription and AI-generated call summaries to help organizations deliver insightful and effective meeting experiences. In just a few minutes, without the need for any coding, you can use the Sample Builder to start prototyping video calling with Azure AI integration. Click the link below to begin or continue reading for further information. 👉 Try the Sample Builder Note, this pattern of combining Azure Communication Services and Azure AI for meeting transcription and summarization is not limited to Sample Builder. You can take the code and overall design pattern and rebuild this experience using the underlying APIs and SDKs. What Is the Sample Builder? The Sample Builder is a no-code Azure Portal experience that you use to brand, customize, build, and deploy a GitHub based sample for prototyping. The sample integrates and deploys multiple Azure services for secure and engaging meetings: Application hosting of the meeting front-end is provided by Azure App Services High-definition video calling for mobile and desktop browsers is provided by Azure Communication Services Calling Role-based access for attendees and providers are implemented using Azure Communication Services Rooms Accessible, customizable, fluid user experience is built on the open-source Azure Communication Services UI library Designed for developers, IT teams, and solution architects, the Sample Builder gets you started quickly but doesn’t produce a production-ready application. After prototyping, you can take the code from GitHub, customize user experience, integrate your own systems, and fine-tune the AI interactions for production. Smarter Meetings with Transcription and Summarization Today’s update integrates Azure AI Speech and Azure AI Language services directly into your meetings, transforming how companies capture, understand, and act on conversations. You can fine-tune this integration and can take advantage of the latest innovation from Azure AI, ensuring your end-users benefit from advancements in LLM models, natural language understanding, and conversation summarization. Transcription and meeting summarization are valuable across industries. For example: Healthcare: Automatically document patient-provider interactions, reduce administrative burden, and support clinical accuracy. Financial Services: Capture detailed records of client meetings to meet regulatory requirements and improve transparency. Education: Provide students and instructors with accessible records of virtual sessions, supporting learning and retention. Real-Time Transcription With transcription enabled, Azure Communication Services uses Azure AI Speech to Text to convert spoken language into a live, speaker-attributed transcript. This allows participants to stay fully engaged in the conversation without worrying about taking notes. Key benefits include: Accurate, multilingual transcription for a wide variety of languages. You can see the full list of supported languages here Speaker attribution for clarity and accountability Searchable meeting records for easy reference and knowledge sharing Support for multilingual teams, with transcripts that can be translated or reviewed post-meeting Training and quality assurance, enabling review of real conversations to improve service delivery Transcripts can be stored securely and managed according to your organization’s compliance, privacy, and retention policies, making them suitable for regulated industries. AI-Generated Call Summaries After the meeting, the Azure AI Language Summarization API automatically analyzes the transcript and generates a concise, structured summary. This summary distills the conversation into key takeaways, including: Main discussion points Decisions made Action items and next steps This helps teams: Align quickly on outcomes and responsibilities Brief stakeholders who couldn’t attend Maintain consistent documentation for compliance, audits, or internal reporting Reduce meeting fatigue by eliminating the need to rewatch or reread entire transcripts How to Get Started You can try these new features today by following a few simple steps: Go through the Sample Builder using the official tutorial. Select the Rooms option in the booking and calling steps. Enable auto-transcription or allow users to turn on transcription in the booking and calling steps. Enable meeting summary in the post-call steps. Choose how you want to deploy and go through follow up steps. Once fully deployed, start a call to test. If not using auto-transcription, open the meeting controls and select “Start Transcription”. Choose the spoken language (Click here for the list of supported languages). After the meeting ends, participants can view the AI-generated summary and download the transcript. In the sample experience, transcripts and summaries are available temporarily. In production environments, you can store them securely and use them to support training, compliance, or analytics. Microsoft 365 Integration Today’s update focuses on integrating Azure AI with Azure hosted meetings. However Azure Communication Services are interoperable with Microsoft Teams, and you can use the Sample Builder to deploy a branded Azure application that joins Teams meetings. Interoperability can be incredibly helpful for organizations that are already using Microsoft Teams but want to build a custom meeting experience for business-2-consumer (B2C) interactions. Using Microsoft Teams as the meeting host allows you to leverage: Teams Premium AI features for generating meeting notes and recommending follow-ups. Teams Premium virtual appointment features for scheduling of B2C meetings and sending reminders across SMS, email, and other channels. Teams Phone capabilities so end-users can dial into the meeting using traditional telephony. Get Started Today Explore the new AI-powered features in the Sample Builder and start building smarter virtual appointment experiences: 👉 Try the Sample Builder With transcription and meeting summaries, your meetings can do more than connect people—they can capture insights, drive action, and deliver lasting value.
Conversational AI applications with Azure Communication Services and the Azure OpenAI Realtime API
At Microsoft, we’re dedicated to helping businesses create seamless, interactive experiences that blend advanced communication capabilities with the power of artificial intelligence. We're shipping an updated set of samples, documentations, and videos, to help developers deliver breakthrough integration that leverages Azure Communication Services and Azure OpenAI’s Realtime API. This enables developers to build real-time, low-latency, AI-driven conversational applications that are both scalable and secure. Bridging communication and AI Modern customers expect engaging, instantaneous interactions—whether it’s for customer support, live translations, or intelligent voice command applications. With this integration, you can harness the rich media capabilities of Azure Communication Services alongside Azure OpenAI’s powerful Realtime language models. The result? A fluid, bidirectional communication system in which audio is captured, processed, and responded to in near real time. Why Azure Communication Services? Azure Communication Services is an AI-ready, cloud-based communications platform that provides APIs for voice, video, chat, and SMS—empowering developers to integrate these capabilities into any application. This ecosystem provides: Scalability & Global Reach: Built on Microsoft’s robust cloud that powers scaled applications like Microsoft Teams and Dynamics, Azure Communication Services ensures reliable, low-latency communication across the globe within a unified tech stack. Enterprise-Grade Security: Benefit from Azure’s comprehensive security and compliance certifications. Flexible Integration: Easily combine voice and video streaming with AI insights to deliver enhanced, interactive customer experiences. Learn more about at the Azure Communication Services product page. The power of real-time AI By integrating Azure OpenAI’s Realtime API with Azure Communication Services, you can convert live audio streams into intelligent interactions. Imagine a customer support system that not only listens to your customers but also understands context, analyzes intent, and generates precise, helpful responses—all as the conversation unfolds. How it works Capture and stream: Azure Communication Services captures high-quality audio from your application’s communication channels. Whether you’re building a call center, a virtual assistant, or an interactive voice response system, Azure Communication Services ensures that every sound is transmitted securely and with minimal latency. Process with Azure OpenAI: Azure OpenAI’s Realtime API processes the streamed audio input using advanced language models. The API analyzes the conversation in real time, extracting intent and context. Deliver intelligent responses: The API returns processed data as actionable insights or natural language responses. Azure Communication Services then delivers these responses back to the user, closing the loop in an instant. This seamless, bidirectional flow is inspired by our latest innovations—check out our detailed exploration of bidirectional real-time audio streaming at Ignite 2024. A developer’s journey: getting started To help you hit the ground running, we’ve put together an overview of the integration steps, along with sample code and best practices. Step 1: Set up your environment Begin by provisioning an Azure Communication Services resource. We offer SDKs in multiple languages (JavaScript, .NET, and more) to quickly integrate voice, video, and chat into your application. Detailed documentation and sample projects are available at: https://learn.microsoft.com/en-us/azure/ai-services/openai/realtime-audio-quickstart Step 2: Stream audio to Azure OpenAI’s Realtime API Once your Azure Communication Services instance is live, capture real-time audio streams from your application. The following code illustrates how you might forward audio data to Azure OpenAI for processing: # WebSocket to get bi-directional streaming audio @app.websocket("/ws") async def ws(websocket: WebSocket): await websocket.accept() handler = CommunicationHandler(websocket) await handler.start_conversation_async() while True: # Receive data from the client data = await websocket.receive_json() kind = data["kind"] if kind == "AudioData": audio_data = data["audioData"]["data"] # Send the audio data to the CallAutomationHandler Step 3: Customize and Scale With the integration in place, you can further tailor the interaction: Real-time transcription and translation: Use Azure OpenAI’s capabilities to transcribe and translate audio on the fly. Contextual customer support: Integrate with backend systems to provide personalized support based on conversation context. Voice-activated commands: Enable smart assistants that understand and execute user commands in real time. Scale with GPT-4o-Mini-Realtime-Preview: With the newly announced GPT-4o-Mini-Realtime-Preview API you can scale your Conversational AI application with reduced latency at a significantly reduced cost. Unlock new possibilities By combining Azure Communication Services with Azure OpenAI’s real-time capabilities, you’re not just adding another tool to your stack—you’re creating an ecosystem where communication and intelligence work hand in hand. This integration opens new avenues for customer engagement, automation, and interactive media applications. We’re thrilled to see how developers and businesses will leverage this powerful combination to transform their communication strategies and build the next generation of intelligent applications. Get started today Ready to build real-time conversational AI experiences? Explore the following resources to begin your journey: Get started with the sample code: https://github.com/anujb-msft/communication-services-realtime-voice-agent Chat with Azure Communication Services experts and join our live learning series on how to build with us in April. Read about this event and register here: aka.ms/register-acs-series. Azure Communication Services: Product Page and Documentation. OpenAI Realtime API: Refer to OpenAI’s developer resources for guidance on API integration. Real-Time Audio Streaming Insights: Ignite 2024 Blog Post. At Microsoft, we believe that the future of communication lies in the seamless integration of real-time connectivity and intelligent processing. We can’t wait to see what you create!Catch Up on the Azure Communication Services Fundamentals Series
This April, we partnered with Microsoft Reactor to deliver a four-part webcast series designed to help developers get started with Azure Communication Services. Each 20-minute episode focused on a different communication channel—giving developers the tools to build real-time, scalable, and secure communication experiences into their apps. If you missed the live sessions, don’t worry, they’re all available on-demand! Check out the full playlist here or see the following individual videos. Here’s a quick look at what each episode covered: Episode 1: WhatsApp Messaging We kicked off the series by showing how to integrate WhatsApp Business messaging into your Azure Communication Services applications. We walked through everything from sandbox testing to connecting a verified WhatsApp Business Account and sending messages with SDKs. For a deeper dive on what was covered, read about it here from Gloria herself! Episode 2: Exploring SMS Capabilities This session described how to provision a phone number, verify the number, and how to send/receive SMS messages. We also covered how to handle incoming messages with event grid listeners and code-based handlers. For a deeper dive on exactly what was covered, read about it here from Pranita herself! Episode 3: Maximizing Email Insights with Logs and Events Next, we dove into email analytics and telemetry setting up logs and events, understanding sender reputation, and using sample queries to gain insights. From basic sample queries to advanced Kusto Query Language (KQL) queries, this session covered everything you need to run a successful email marketing campaign with Azure Communication Services. Episode 4: Add Audio & Video Calling We wrapped the season with a demo-rich session on embedding calling features into your communications application using Azure Communication Services. Highlights included new AI-powered features like captions, noise suppression, grid views, and real-time translation. What’s Next? We’re already planning Season 2, launching later this year, with a focus on Azure Communication Services + AI. Expect deeper dives, new use cases, and more interactive demos. Want to stay in the loop? Sign up for season two updates to be the first to know when the new season launches and tell us what you want to learn about!
Getting Started with WhatsApp Messaging in Azure Communication Services feat. Microsoft Software Engineer Gloria Li
As part of our webinar Azure Communication Services Fundamentals series this month, we had the pleasure of walking developers through how to set up WhatsApp messaging using Azure Communication Services (ACS)—from the very first portal click to sending real messages to real users. Whether you're just experimenting or ready to go live, ACS makes it easier than ever to integrate WhatsApp into your applications. Check out what we covered in the live webinar below. And be sure to catch our next live webinar on ACS Fundamentals: Adding Audio Video Calling into Your Apps this Thursday, April 24 at 9 a.m. PT. Register here. How to Set Up WhatsApp Messaging using Azure Communication Services ✅ Create Your ACS Resource We kicked things off by creating an ACS resource in the Azure Portal—your gateway to adding voice, SMS, email, and messaging into your apps, backed by Microsoft Teams infrastructure. 🧪 Try WhatsApp Messaging in the Sandbox No Meta account yet? No problem. We explored the WhatsApp Sandbox experience, letting developers quickly test sending template and text messages via WhatsApp without setting up a business account. Just scan the QR code, send the unique connect message, and start experimenting—no cost, no strings attached. 🔌 Connect a WhatsApp Business Account For production readiness, we demonstrated how to register a WhatsApp Business Account via the Azure Portal: - Bring your own phone number or use an ACS-provided one. - Seamless embedded sign-up flow with Meta. - Register, verify, and configure—all in one place. 📩 Handle Inbound Messages with Event Grid We showed how to handle Advanced Messaging events using Event Grid. This enables your backend to handle real-time events using Azure Functions, Logic Apps, or Webhooks. Pro tip: Try out the Event Grid Viewer. 💬 Send Outbound WhatsApp Messages with SDKs Once your channel is set up, sending messages is simple using the SDKs for Advanced Messaging: - Supports .NET, JavaScript, Java, and Python. - We demoed a quick .NET console app to send freeform text messages. 💰 Understand the Pricing We broke down the two pricing components: Meta’s pricing is based on 24-hour conversation sessions. Azure Communication Services charges a flat per-message fee (inbound and outbound). Together, these offer a transparent, scalable pricing model. 🚀 Ready to Build By the end of the session, you’ll have everything you need to: - Create an Azure Communication Services resource. - Prototype with the sandbox. - Connect a verified WhatsApp channel. - Handle customer messages at scale. - Send replies from your app using trusted SDKs. 📺 Watch the full webinar on YouTube. 🛠️ Explore the full Learn collection here: 👉 Advanced Messaging for WhatsApp in Azure Communication Services 🛠️ Try it yourself in the Azure Portal For more tips on how to get started with WhatsApp Messaging in ACS, check out our Learn Documentation and follow Gloria Li on LinkedIn.The U.S. Small Business Administration integrates Azure Communication Services, sending critical information to millions of Americans more quickly
At the U.S. Small Business Administration (SBA), we faced significant challenges with our existing mass communications solution. Our organization is the only cabinet-level federal agency fully dedicated to small business counseling, capital, and contracting expertise. We also offer disaster assistance for homeowners, renters, nonprofits, and businesses of all sizes affected by disaster recovery-related assistance, so we needed a scalable platform that could efficiently send critical information to millions of Americans, especially during times of disaster recovery and urgent business support. We also required a platform that could do this quickly and was cost-effective, while also allowing us to further centralize operational capabilities within Dynamics 365, which would help teams internally. Centralizing capabilities means finding ways to consolidate information in one system, creating a single-pane-of-glass experience, so people here don't have to leave Dynamics 365 and can have everything they need without jumping from one system to another to get their work done. How the U.S. SBA implemented Azure Communication Services into their existing systems After evaluating various options, we decided to implement Microsoft Azure Communication Services for email mass communications. It was an easy decision for us because Azure Communication Services meets our needs in the following ways: Speed: We can send out one million emails in under 20 minutes. Cost: Azure Comms is significantly more affordable than competing solutions. Integration: Azure Comms seamlessly integrates into Dynamics 365 as a complementary feature. How SBA implemented Microsoft Azure Communication Services into Dynamics 365 To implement Azure Communication Services, we worked closely with the product group to integrate it into Dynamics 365, which we use to handle large campaigns. Dynamics 365 also centralizes knowledge base reference materials for agents and customer service representatives. With the Azure Comms integration into Dynamics 365, we focused on refining the system’s outbound email velocity over the course of several months. Within 30 days, we successfully transitioned over 50 campaign teams to Dynamics 365 while collaborating with Microsoft to integrate Azure Communications Services into the platform. Some challenges arose, but we collaborated with the product group to incorporate improvements and refine its capabilities over time. We are also leveraging AI for sentiment analysis and process automation to meet various internal and external requirements. We recently implemented Copilot in Dynamics 365, for example, which is safe, secure, and free within our D365 licensing. We've done so much work in bringing people’s knowledge base materials into the platform that employees have been able to immediately see the benefits of using Copilot: Ask feature on Copilot makes it easy to submit a prompt and get help with a specific task. Automatic summarization of a specific case, which helps our agents and service representatives get up to speed immediately and be more responsive. Email drafting is now much easier and faster with Copilot because it allows anyone to write an email from scratch in a fraction of the time. Language translation via Copilot allows us to toggle on the fly between different languages and translate a different language into English--and vice versa. A crucial skill set in our organization is language proficiency, particularly in Spanish. Historically, we've had to either hire representatives with Spanish language skills or rely on contracted services to meet this need. If we receive an email in Spanish, an English-speaking team member can translate it into English and respond in Spanish, helping to reduce our reliance on individuals with specialized language skills. "It’s difficult to predict when we’ll receive an email in a different language. But now, we can translate messages instantly and respond in the sender’s native language—removing barriers and streamlining communication like never before. It’s an incredible advantage—one that doesn’t get nearly enough attention. Microsoft Dynamics 365 Copilot’s translation capability isn’t just convenient; it’s a game-changer."" The results of implementing ACS into Dynamics 365 There are many benefits to using Azure Communication Services and integrating it into Dynamics 365. ACS has empowered our campaign teams to address time-sensitive requests efficiently, enabling us to send out critical communications within minutes or a few hours—rather than days. We have sent over 300 million emails through Azure Comms since its implementation in March 2023. Looking ahead We plan to extend the functionality of Azure Communication Services to support internal-facing communications in the near future, including: Exploring additional communication methods (e.g., SMS, phone) to provide omnichannel, preference-oriented campaigns. Leveraging virtual appointment options to enhance service delivery. Developing more refined metrics, such as open and click rates, for better insights into the impact of our mass communication efforts. Azure Communication Services has transformed our ability to respond promptly to diverse mass communication needs, even with minimal notice. Our journey with Azure Comms showcases the power of cloud-based communication solutions in enhancing organizational efficiency and effectiveness. The more people grasp the features of Dynamics 365 and its ease of integration with systems like Azure Communication Services, the better they understand its quantifiable benefits and true value. A common misconception is that adopting Dynamics 365 means being restricted to Microsoft-only systems—but that’s not the case. You can integrate a different phone system, leverage various databases, and customize the platform to suit your needs. Whether it's delivering crucial information to frontline workers or enabling responsive services, Dynamics 365 is far more than just a customer relationship management (CRM) system—it's a versatile, customizable solution designed to support a wide range of business requirements and operational needs.
Azure Communication Services technical documentation table of contents update
Technical documentation is like a map for using a platform—whether you're building services, solving problems, or learning new features, great documentation shows you the way to the solution you need. But what good is a map if it’s hard to read or confusing to follow? That’s why easy-to-navigate documentation is so important. It saves time, reduces frustration, and helps users focus on what they want to achieve. Azure Communication Services is a powerful platform, and powerful platforms require great documentation for both new and experienced developers. Our customers tell us consistently that our docs are a crucial part of their experience of using our platform. Some studies suggest that documentation and samples are the most important elements of a great developer experience. In this update, we’re excited to share how we’ve improved our technical documentation’s navigation to make it quicker and simpler than ever to find the information you need when you need it. Why did we change? In order for our content to be useful to you, it first needs to be findable. When we launched Azure Communication Services, the small number of articles on our site made it easy to navigate and find relevant content. As we’ve grown, though, our content became harder to find for users due to the quantity of articles they need to navigate. To refresh your memory, the table of contents on our docs site used to be structured with these base categories: Overview Quickstart Tutorials Samples Concepts Resources References These directory names describ e the type of content they contain. This structure is a very useful model for products with a clearly-defined set of use cases, where typically a customer’s job-to-be-done is more constrained, but it breaks down when used for complex, powerful platforms that support a broad range of use cases in the way that Azure Communication Services does. We tried a number of small-scale changes to address the problems people were having on our site, such as having certain directories default to open on page load, but as the site grew, we became concerned that our site navigation model was becoming confusing to users and having a negative impact on their experience with our product. We decided to test that hypothesis and consider different structures that might serve our content and our customers better. Our user research team interviewed 18 customers with varying levels of experience on our platform. The research uncovered several problems that customers were having with the way our docs navigation was structured. From confusing folder titles, to related topics being far away from each other in the nav model, to general confusion around what folder titles meant, to problems finding some of the most basic information about using our platform, and a host of other issues, our user research made it clear to us that we had a problem that we needed to fix for our users. What did we change in this release? To help address these issues, we made a few key changes to make our table of contents simpler and easier to navigate. The changes we made were strictly to site navigation, not page content, and they include: We've restructured the root-level navigation to be focused on communication modality and feature type, rather than content type, to better model our customers' jobs-to-be-done. Topics include All supported communication channels Horizontal features that span more than one channel Topics of special interest to our customers, like AI Basic needs, like troubleshooting and support This will allow customers to more easily find the content they need by focusing on the job they need to do, rather than on the content type. We've simplified the overview and fundamentals sections to make the site less overwhelming on first load. We've surfaced features that customers told us were difficult to find, such as UI Library, Teams interop, and Job router. We've organized the content within each directory to roughly follow a beginner->expert path to make content more linear, and to make it easier for a user to find the next step in completing their task. We've removed unnecessary layers in our nav, making content easier to find. We've added a link to pricing information to each primitive to address a common customer complaint, that pricing information is difficult to find and understand. We've combined quickstarts, samples, and tutorials into one directory per primitive, called "Samples and tutorials", to address a customer complaint that our category names were confusing. We added a directory to each primitive for Resources, to keep important information close by. We added root-level directories for Common Scenarios, Troubleshooting, and Help and support. We did a full pass across all TOC entries to ensure correct casing, and edited entries for readability and consistency with page content, as well as for length to adhere to Microsoft guidelines and improve readability. These changes have led us to a structure that we feel less taxing for the reader, especially on first visit, maps more closely to the customer’s mental model of the information by focusing on the job-to-be-done rather than content type, helps lead them through the content from easiest to hardest, helps make it easier for them to find the information they need when they need it, and helps remind them of all the different features we support. Here’s what the table of contents looks like on page load as of Feb 6: These changes are live now. You can see them on the Azure Communication Services Technical documentation site. What’s next: In the coming weeks we will continue to make refinements based on customer feedback and our assessment of usage metrics. Our content team will begin updating article content to improve readability and enhance learning. We will be monitoring our changes and seeking your feedback. How will we monitor the effectiveness of our changes? To track the effectiveness of our changes and to be sure we haven’t regressed, we’ll be tracking a few key metrics Bounce rates: We’ll be on the lookout for an increase in bounce rates, which would indicate that customers are frequently landing on pages that don’t meet their expectations. Page Views: We’ll be tracking the number of page views for our most-visited pages across different features. A decrease in page views for these pages will be an indicator that customers are not able to find pages that had previously been popular. Customer Interviews: We will be reaching out to some of you to get your impressions of the new structure of our content over the coming weeks. Customer Surveys: We've created a survey that you can use to give us your feedback. We'll also be adding this link to select pages to allow you to tell us what you think of our changes while you're using them! So, give our new site navigation a try, and please don’t hesitate to share your feedback either by filling out our survey or by sending an email to [email protected]. We look forward to hearing from you! ACopilot in Azure is now integrated in the Voice and Video Insights dashboard
We’re excited to announce the public preview of Copilot in Azure integrations in the Voice and Video Insights dashboard of your Azure Portal. This dashboard provides overview and debugging guidance on your Azure Communication Services calls. By integrating Copilot, we’ve expanded the capabilities of the existing data visualizations in this dashboard. This Copilot enhancement enables communication developers to chat with Copilot to better identify, understand, and improve calling issues that impact their end-users' calling experiences. By embedding Copilot in Azure in the Voice and Video Insights dashboard, communication developers get AI-driven, natural-language summaries of their calling insights, tailored to improve their end-user's call experience. These AI-driven conversations enable communication developers to quickly prioritize and focus their call improvement efforts. Instead of reviewing large volumes of public documentation to interpret their call logs, developers can chat with Copilot directly from the dashboard to get personalized guidance on their call issues. What to expect from your Voice and Video Insights Dashboard We understand the importance of having Copilot in Azure customized to your debugging workflow and visuals, which is why we’ve built multiple Copilot buttons in the Volume and Reliability tabs to give you starting points for your conversations with Copilot. Each Copilot button starts a conversation based on the visuals you’re reviewing in the dashboard. Since we know how important it is to quickly get inspiration and understand the types of questions Copilot can help you with as a developer, each Copilot button has a built-in, click-to-run starter prompt. Our customized Copilot experience can be accessed by clicking a Copilot button, which opens a chat panel for you to interact with. For example, in the Volume section, you can click on the “Ask Copilot about Azure Communication Services SDK” button, or in the User Facing Diagnostics (UFD) section, you can click Ask Copilot for help next to the UFD you want to focus on. Lastly, Copilot suggests follow-up prompts based on the context of your chat, to help you further investigate the issue at hand. Using Copilot in Azure embedded in Voice and Video Insights As communication developers, you’re focused on developing and maintaining high-quality calling experiences for your end users. We’re enhancing Copilot in Azure to directly support this core need by providing capabilities such as: Track your call usage and details Diagnose and reduce call issues By structuring the Voice and Video Insights features around these key areas, we aim to empower you with AI-driven insights and recommendations that accurately articulate the issues and solutions, streamlining workflows and enhancing your ability to improve your call performance more effectively. In our public preview, we’re announcing Copilot buttons that help communication developers in two key scenarios: Monitor your call composition and update your SDK with Copilot in Azure Troubleshoot in-call issues with Copilot in Azure Monitor your call composition and update your SDK with Copilot in Azure As a communication developer, you need tools that scale with your organization's growth and increasing solution complexity. The Copilot buttons in the Volume section of Voice and Video Insights offer high-level insights across all calls in your resource. To quickly understand your call volume and composition visuals you can click the Copilot buttons for explanations and guidance on best practices. You can also learn about recent improvements in the Calling SDK and how to update your SDK for optimal performance. Troubleshoot in-call issues with Copilot in Azure We recognize that even the best-designed calling solutions can encounter issues. The User Facing Diagnostics (UFD) section is designed to help you identify and address such issues. Given the multitude of variables that can impact users' call experiences, we provide detailed call data to help pinpoint potential root causes and solutions. The UFD tab enables you to use Copilot for specific insights about how different types of UFDs affect user experiences. Copilot interprets what each UFD signifies and offers practical advice on mitigating these issues to enhance user satisfaction. Moreover, Copilot can share code samples that help you surface in-call notifications, proactively alerting users to call issues, and suggesting measures like unmuting their microphone to mitigate impact. How to get started We are thrilled to bring you these advanced Copilot integrations, empowering you to achieve greater efficiency and effectiveness in managing and improving your communication services. Stay tuned for ongoing updates and enhancements that further elevate your development experience. Happy coding! Learn how to get started with Azure Copilot in Voice and Video Insights.Teams Phone extensibility powered by Azure Communication Services
Today, CCaaS providers are using Gen AI to improve customer experience by advising human agents and automating workflows. However, interactions are split between UCaaS like Teams and CCaaS systems, limiting Gen AI's potential. Brands want integrated solutions that unify telephony infrastructure for a complete view of customer interactions and robust data handling. Customizability is crucial for integration. On March 17th, 2025, we proudly announce the launch of Teams Phone extensibility powered by Azure Communication Services at Enterprise Connect, allowing CCaaS vendors to seamlessly integrate with Teams Phone to meet brand needs. Teams Phone with Azure Communication Services Azure Communication Services Call Automation and Calling SDK now enable CCaaS developers to extend Teams Phone capabilities into their application. Key benefits of Teams Phone extensibility include: Consolidate Telephony for UCaaS and CCaaS: Simplified setup with no need to configure and administer separate phone systems. Customers can leverage their Teams phone telephony investment for contact center deployments. Gen AI Integration:Utilize AI to streamline processes and empower customer service agents with automation capabilities, with direct access to Azure AI technology. Extend UCaaS Capabilities to CCaaS: Take advantage of Teams Phone enterprise features, including emergency calling, dial plan policies, wide geographic availability of Public Switched Telephone Network (PSTN) connectivity options and much more. Agent Notification handling: Allow data segregation between CCaaS persona and UCaaS persona with the choice of ringing either the Teams standard client or a CCaaS application. Cost Efficiency: Enable ISVs to build cost-effective solutions using existing Teams Phone plans, without adding Azure Communication Services numbers or Direct Routing. Key Features Provisioning for Seamless Integration:Our provisioning capabilities empower CCaaS providers to connect their ACS deployments with customers' Teams tenants, enabling the use of Teams telephony features for a cohesive and efficient communication experience. Emergency Calling: With Azure Communication Services Calling SDK, we bring enhanced emergency calling support for agents who can dial emergency services, provide their location, and receive callbacks from public safety answering points. Leverage Call Automation: Azure Communication Services Call Automation APIs enable CCaaS providers to build server-based and intelligent call flows. Call Routing and Mid-Call Controls: The solution includes advanced call routing capabilities, which enable more efficient call management and escalation to agents. Mid-call controls enable adding participants, redirecting calls, and transferring calls seamlessly. Convenience Recording: Azure Communication Services enables developers to integrate call recording capabilities into Microsoft Teams for CCaaS scenarios, allowing for customized recording processes controlled by CCaaS admins. Get to know more about Azure Communication Services Call Recording here. Conversational AI Integration: Developers can use Call Automation APIs to leverage AI-powered tools, play personalized greeting messages, recognize conversational voice inputs, and use sentiment analysis to improve customer service. Get started today with this QuickStart. ISV Ecosystem At launch, Teams Phone extensibility will be supported by a diverse range of ISV solutions, including Anywhere365, AudioCodes, ComputerTalk, Enghouse, IP Dynamics, Landis, and Luware. These partners expand the reach and functionality of Teams Phone, allowing you to select the best solution for your unique requirements. This flexibility means businesses can improve their telephony setup while experiencing the full power of Teams Phone and Azure Communication Services. Learn more Teams Phone extensibility will be in public preview and available for customers to try soon. To learn more about Teams Phone extensibility and how to develop your own solutions using Azure Communication Services, stay tuned for more resources and announcements at Build 2025 in May. Developers may join the Azure Communication Services TAP program with the following form: Azure Communication Services Technology Adoption Program Registration. We look forward to seeing how our partners and customers leverage these new capabilities to improve their contact center operations. Follow Sean Keegan on LinkedIn for more news about Azure Communication Services!Reliable, scalable, future-ready: AudioCodes extends the value of Teams for customer experiences with Azure Communication Services and Voca CIC
To improve customer experiences with cost-effective AI technology, AudioCodes fully onboarded their contact center from Microsoft Teams to Azure Communication Services.
Ignite 2024: Bidirectional real-time audio streaming with Azure Communication Services
Today at Microsoft Ignite, we are excited to announce the upcoming preview of bidirectional audio streaming for Azure Communication Services Call Automation SDK, which unlocks new possibilities for developers and businesses. This capability results in seamless, low-latency, real-time communication when integrated with services like Azure Open AI and the real-time voice APIs, significantly enhancing how businesses can build and deploy conversational AI solutions. With the advent of new AI technologies, companies are developing solutions to reduce customer wait times and improve the overall customer experience. To achieve this, many businesses are turning to AI-powered agents. These AI-based agents must be capable of having conversations with customers in a human-like manner while maintaining very low latencies to ensure smooth interactions. This is especially critical in the voice channel, where any delay can significantly impact the fluidity and natural feel of the conversation. With bidirectional streaming, businesses can now elevate their voice solutions to low-latency, human-like, interactive conversational AI agents. Our bidirectional streaming APIs enable developers to stream audio from an ongoing call on Azure Communication Services to their web server in real-time. On the server, powerful language models interpret the caller's query and stream the responses back to the caller. All this is accomplished while maintaining low latency, ensuring the caller feels like they are speaking to a human. One such example of this would be to take the audio streams and processing them through Azure Open AI’s real-time voice API and then streaming the responses back into the call. With the integration of bidirectional streaming into Azure Communication Services Call Automation SDK, developers have new tools to innovate: Leverage conversational AI Solutions: Develop sophisticated customer support virtual agents that can interact with customers in real-time, providing immediate responses and solutions. Personalized customer experiences: By harnessing real-time data, businesses can offer more personalized and dynamic customer interactions in real-time, leading to increased satisfaction and loyalty. Reduce wait times for customers: By using bidirectional audio streams in combination with Large Language Models (LLMs) you can build virtual agents that can be the first point of contact for customers reducing the need for customers waiting for a human agent being available. Integrating with real-time voice-based Large Language Models (LLMs) With the advancements in voice based LLMs, developers want to take advantage of services like bidirectional streaming and send audio directly between the caller and the LLM. Today we’ll show you how you can start audio streaming through Azure Communication Services. Developers can start bidirectional streaming at the time of answering the call by providing the WebSocket URL. //Answer call with bidirectional streaming websocketUri = appBaseUrl.Replace("https", "wss") + "/ws"; var options = new AnswerCallOptions(incomingCallContext, callbackUri) { MediaStreamingOptions = new MediaStreamingOptions( transportUri: new Uri(websocketUri), contentType: MediaStreamingContent.Audio, audioChannelType: MediaStreamingAudioChannel.Mixed, startMediaStreaming: true) { EnableBidirectional = true, AudioFormat = AudioFormat.Pcm24KMono } }; At the same time, you should open your connection with Azure Open AI real-time voice API. Once the WebSocket connection is setup, Azure Communication Services starts streaming audio to your webserver. From there you can relay the audio to Azure Open AI voice and vice versa. Once the LLM reasons over the content provided in the audio it streams audio to your service which you can stream back into the Azure Communication Services call. (More information about how to set this up will be made available after Ignite) //Receiving streaming data from Azure Communication Services over websocket private async Task StartReceivingFromAcsMediaWebSocket() { if (m_webSocket == null) return; try { while (m_webSocket.State == WebSocketState.Open || m_webSocket.State == WebSocketState.Closed) { byte[] receiveBuffer = new byte[2048]; WebSocketReceiveResult receiveResult = await m_webSocket.ReceiveAsync(new ArraySegment<byte>(receiveBuffer), m_cts.Token); if (receiveResult.MessageType == WebSocketMessageType.Close) continue; var data = Encoding.UTF8.GetString(receiveBuffer).TrimEnd('\0'); if(StreamingData.Parse(data) is AudioData audioData) { using var ms = new MemoryStream(audioData.Data); await m_aiServiceHandler.SendAudioToExternalAI(ms); } } } catch (Exception ex) { Console.WriteLine($"Exception -> {ex}"); } } Streaming audio data back into Azure Communication Services //create and serialize streaming data private void ConvertToAcsAudioPacketAndForward( byte[] audioData ) { var audio = new OutStreamingData(MediaKind.AudioData) { AudioData = new AudioData(audioData) }; // Serialize the JSON object to a string string jsonString = System.Text.Json.JsonSerializer.Serialize<OutStreamingData>(audio); // Queue the async operation for later execution try { m_channel.Writer.TryWrite(async () => await m_mediaStreaming.SendMessageAsync(data)); } catch (Exception ex) { Console.WriteLine($"\"Exception received on ReceiveAudioForOutBound {ex}"); } } //Send encoded data over the websocket to Azure Communication Services public async Task SendMessageAsync(string message) { if (m_webSocket?.State == WebSocketState.Open) { byte[] jsonBytes = Encoding.UTF8.GetBytes(message); // Send the PCM audio chunk over WebSocket await m_webSocket.SendAsync(new ArraySegment<byte>(jsonBytes), WebSocketMessageType.Text, endOfMessage: true, CancellationToken.None); } } To reduce developer overhead when integrating with voice-based LLMs, Azure Communication Services supports a new sample rate of 24Khz, eliminating the need for developers to resample audio data and helping preserve audio quality in the process Next steps The SDK and documentation will be available in the next few weeks after this announcement, offering tools and information to integrate bidirectional streaming and utilize voice-based LLMs in your applications. Stay tuned and check our blog for updates!
