AI brokers are reworking industries by automating workflows, enhancing productiveness, and enabling clever decision-making. Companies are leveraging AI brokers to course of insurance coverage claims, handle IT service desks, optimize provide chain logistics, and even help healthcare professionals in analyzing medical data. The potential is huge, and we’re excited to introduce two highly effective improvements in Azure AI Foundry.
AI brokers are reworking industries by automating workflows, enhancing productiveness, and enabling clever decision-making. Companies are leveraging AI brokers to course of insurance coverage claims, handle IT service desks, optimize provide chain logistics, and even help healthcare professionals in analyzing medical data. The potential is huge, and we’re excited to introduce two highly effective improvements in Azure AI Foundry:
- Responses API: A robust API enabling AI-powered purposes to retrieve data, course of knowledge, and take motion seamlessly.
- Pc-Utilizing Agent (CUA): A breakthrough AI mannequin that navigates software program interfaces, executes duties, and automates workflows.
Collectively, these capabilities empower companies to reimagine AI not simply as an assistant—however as an lively digital workforce. Enterprise clients will quickly achieve entry to those improvements driving automation, effectivity, and intelligence at scale.
Enhancing AI Brokers with the Responses API
The Responses API is the important thing to unlocking agentic AI in Azure AI Foundry, reworking how enterprises harness AI for real-world influence. It’s the new basis for leveraging Azure OpenAI Service’s highly effective built-in instruments, combining the simplicity of the Chat Completions API with the superior capabilities obtainable by means of Assistants API and Azure AI Agent Service. The Responses API permits seamless interplay with instruments like CUA, code interpreter, perform calling, and file search—all in a single API name. This API permits AI techniques to retrieve knowledge, course of data, and take actions—seamlessly connecting agentic AI with enterprise workflows.
How the Responses API Works
The Responses API gives a structured response format that enables AI to work together with a number of instruments whereas sustaining context throughout interactions. It helps:
- Software calling in a single easy API name: Now, builders can seamlessly combine AI instruments, making execution extra environment friendly.
- Pc use: Use the pc use software inside the Responses API to drive automation and execute software program interactions.
- File search: Work together with enterprise knowledge dynamically and extract related data.
- Code interpreter: Create and execute Python code effortlessly inside AI-powered purposes.
- Perform calling: Develop and invoke customized capabilities to reinforce AI capabilities.
- Chaining responses into conversations: Preserve observe of interactions by linking responses collectively utilizing distinctive response IDs, making certain continuity in AI-driven dialogues.
- Enterprise-grade knowledge privateness: Constructed with Azure’s trusted safety and compliance requirements, making certain knowledge safety for organizations.
By consolidating retrieval, reasoning, and motion execution right into a single API, the Responses API simplifies AI agent improvement, decreasing the complexity of orchestrating a number of AI instruments inside an automation pipeline.
This scalability makes it well-suited for enterprise use instances throughout industries similar to customer support, IT operations, finance, and provide chain administration, the place AI-powered automation can streamline workflows and enhance effectivity. For even larger flexibility and management, organizations can discover Azure AI Agent Service, which affords extra instruments and fashions for creating and scaling AI brokers. Azure AI Agent Service integrates with Semantic Kernel and AutoGen, enabling seamless multi-agent orchestration for extra advanced situations requiring a number of brokers to collaborate on duties.
Empowering AI Brokers with the Pc-Utilizing Agent
The Pc-Utilizing Agent (CUA) is a specialised AI mannequin in Azure OpenAI Service that enables AI to work together with graphical consumer interfaces (GUIs), navigate purposes, and automate multi-step duties—all by means of pure language directions. In contrast to conventional automation instruments that depend on predefined scripts or API-based integrations, CUA can interpret visible parts, adapt dynamically, and take motion based mostly on on-screen content material.
What makes the Pc-Utilizing Agent distinctive?
- Autonomous UI navigation: Can open purposes, click on buttons, fill out kinds, and navigate multi-page workflows.
- Dynamic adaptation: Interprets UI adjustments and adjusts actions accordingly, decreasing reliance on inflexible automation scripts.
- Cross-application job execution: Operates throughout web-based and desktop purposes, integrating disparate techniques with out API dependencies.
- Pure language command interface: Customers can describe a job in plain language, and CUA determines the proper UI interactions to execute.
With in the present day’s announcement, builders can begin constructing extra agentic capabilities straight away with CUA. As enterprises look to deploy this expertise at scale, we’re evaluating integration with Home windows 365 and Azure Digital Desktop to allow CUA automation to run seamlessly in a managed host setting on Cloud PCs or digital machines (VMs), making certain constant efficiency whereas sustaining enterprise compliance and safety requirements.
Guaranteeing safe and reliable AI automation
As AI techniques turn out to be extra autonomous, making certain safety, reliability, and alignment with human intent is important. The CUA mannequin is among the first agentic AI fashions able to instantly interacting with software program environments, bringing new challenges in misuse prevention, unintended actions, and adversarial dangers. To handle these, Microsoft and OpenAI have applied a multi-layered security strategy spanning the mannequin, system, and deployment ranges.
The CUA mannequin is developed with safeguards to refuse dangerous duties, reject unauthorized actions, and stop misuse. On the system stage, Microsoft implements enterprise-grade content material filtering and execution monitoring to assist detect and stop coverage violations. To attenuate unintended actions, CUA is designed to request consumer confirmations earlier than executing irreversible duties and to limit high-risk actions similar to monetary transactions.
Microsoft’s Reliable AI framework additional ensures real-time observability, logging, and compliance auditing for enterprise deployments. Automated and human-in-the-loop detection techniques monitor execution patterns, figuring out anomalous behaviors and imposing governance insurance policies. These safeguards are repeatedly refined based mostly on inner red-teaming, exterior audits, and real-world testing to strengthen safety towards immediate injections, adversarial manipulations, and unauthorized entry. Given the present reliability stage of the CUA mannequin—significantly in non-browser environments—human oversight stays strongly really useful for delicate operations.
As AI brokers evolve, Microsoft is dedicated to transparency, safety, and ongoing threat mitigation. By combining CUA’s built-in safeguards with Azure’s enterprise compliance and governance instruments, organizations can deploy AI-powered automation with confidence, making certain protected and accountable AI adoption at scale.
Getting began with CUA and Responses API
Azure AI Foundry continues to push the boundaries of AI-powered automation. Enterprise clients will achieve entry to the Responses API and CUA in Azure OpenAI Service within the coming weeks.
We’re excited to see how builders and companies innovate with these new capabilities.