An invoice-processing agent needs access to the supplier registry and finance API. Then a document includes an upload link, and the agent attempts to follow it. Reviewing the agent’s approved tool names does not answer whether its runtime can send data to that destination.

Outbound connectivity is a separate architectural boundary. It deserves a policy and a test of its own, alongside tool permissions, identity, and validation of the information being sent.

Evaluate the preview within its documented scope

Microsoft’s September 24 egress walkthrough introduces a preview for hosted-agent network controls. The article explicitly states that it has no preview SLA and is intended for development and evaluation rather than production use. Keep that limitation visible when assessing it.

The walkthrough describes separate Audit and Enforced modes for destination policy. It also warns that Transform and Rewrite actions still operate in Audit. Use non-sensitive test values and review action behaviour before treating an Audit trial as harmless observation.

Scope conclusions to the documented hosted-agent HTTP and HTTPS path. A successful experiment should not be generalized to other protocols, unrelated agent types, or every possible route out of an enterprise environment.

The July hosted-agent GA announcement describes the runtime milestone. That does not promote this later egress preview to GA. Runtime availability and network-control maturity are distinct facts in the architecture decision.

Explain what an outbound rule can establish

Consider a hypothetical invoice agent that checks approved finance records. A destination policy can constrain where its HTTP requests go. It does not establish that an allowed request contains the correct invoice or that the recipient should receive every field.

Separate destination permission, caller authorization, and payload validity. The identity used for the finance API needs appropriate access. The application also needs to validate the customer, invoice, and requested operation.

This separation makes investigation clearer. If a permitted endpoint receives another customer’s document, the problem is not solved by adding more hostnames to a deny list. The application failed to bind the payload to the authorized business task.

Treat redirection and helper behaviour as part of the test plan. An apparently simple request can cause a library or remote endpoint to initiate another request. Identify what the selected runtime and policy actually observe and control rather than assuming the first URL describes the whole operation.

Build a controlled experiment with endpoint evidence

Use one approved test endpoint and one deliberately unapproved endpoint under the team’s control. Neither should receive production data. Record requests at the destination so the test has independent evidence of receipt.

Attach an explicit test policy to a defined agent version and preserve its identifier. Record the runtime version, policy version, and test input together. A policy document in configuration storage is evidence of intent, not proof of enforcement.

In Audit, inspect the policy decision evidence and compare it with what both endpoints received. In Enforced, repeat the same calls and verify the approved path still works while the unapproved destination receives nothing.

Do not infer a successful policy denial from an arbitrary error. DNS failure, an expired certificate, or an endpoint’s own access rejection can all prevent a request for unrelated reasons. Correlate runtime policy evidence with the destination log before calling the experiment successful.

Keep TLS verification enabled and follow the platform’s documented certificate handling. Disabling verification to make a test work changes the security conditions and can hide a separate configuration problem.

These are recommended test steps. They are not results from a test performed for this article. Publish measured outcomes only after the team has captured them in its own environment.

Pair network controls with agent security tests

The 2024 AgentDojo evaluation demonstrates why utility and security need separate measurement. In its GPT-4o defense comparison, a tool filter reduced targeted attack success from 57.69% without defense to 6.84%, while benign utility was 69.0% and 73.13% respectively. Those are historical benchmark results under the paper’s tested attacks, not egress-control results or estimates for Foundry.

A filtered tool set and a destination boundary address different parts of the problem. The research supports testing both legitimate work and attacker goals after adding a control. Blocking everything can look secure while making the application unusable.

Microsoft’s AI Red Teaming Agent announcement describes preview assessment for prohibited actions and sensitive-data leakage. Use application-specific scenarios alongside those capabilities: an invoice containing a malicious URL, a misleading tool response, or a request to send data to an unrelated account.

Include allowed endpoints that receive invalid payloads. An attacker may not need an unapproved destination if a legitimate integration can be misused. Business authorization and output validation remain necessary.

Turn the experiment into an operating decision

Assign an owner for policy changes, an owner for application validation, and an operator for failed connections. The runbook should distinguish an expected denial from a service outage and a certificate problem.

Before changing a policy, replay required business calls and deliberate negative tests. Check each active agent version that is expected to use the change. Do not assume that editing a policy instantaneously changes every running session.

Retain a record of why each destination is needed and what data the agent is permitted to send there. Periodically remove unused destinations through the normal review process. An allow list that grows without ownership eventually loses much of its value.

For the current preview, keep the experiment in a development environment and continue using the enterprise’s established production network controls. Evaluate the capability’s maturity again before making it a production dependency.

The desired evidence is concrete: required finance lookups complete, unapproved test destinations receive no request under enforcement, and allowed calls still obey identity and payload rules. That is a useful architecture result. A statement from the agent that it followed policy is not equivalent evidence.

References

Microsoft: Hosted-agent egress preview walkthrough, September 24, 2026

Microsoft: Hosted-agent GA announcement, July 9, 2026

Debenedetti and colleagues: AgentDojo, 2024, defense comparison Table 5

Microsoft: AI Red Teaming Agent, November 2025

A daily operations brief is straightforward until the agent misses a run, receives the same work twice, or reports on the wrong reporting window. The challenge is no longer generating a useful answer on demand. It is deciding what one unattended run is allowed to do and how to establish that the work is complete.

A schedule supplies a starting condition. The application needs a run contract that defines inputs, authority, time boundaries, outputs, recovery, and ownership. Establish that contract before enabling recurring execution.

Choose a task with a verifiable end

Microsoft introduced Routines in public preview in the June Build recap. The July 7 Foundry digest continued to describe Routines as preview for scheduled and triggered execution. Treat that as the maturity basis for an evaluation at this stage, rather than assuming a production service commitment.

Begin with a task whose outcome is easy to inspect. A report summarizing completed maintenance work is a better initial candidate than an agent that can independently change maintenance schedules and notify customers. The first produces evidence for review; the second creates several external commitments.

For a hypothetical maintenance-reporting agent, define the approved data sources, business unit, reporting period, output location, and recipients. Make those application configuration. The model can summarize the permitted records without choosing a broader scope or a new audience.

Give the run an explicit result type: completed, completed with declared gaps, awaiting review, failed, or superseded. A fluent report should not be the only success signal. The operator needs to distinguish “nothing happened” from “the agent could not retrieve the records.”

Define time as business data

Choose a business timezone and use explicit period boundaries. “Yesterday” is ambiguous when the service, database, and users operate in different zones. Persist the intended window with the run so a retry uses the same interval.

Decide whether records are selected by event time, completion time, or last modification. A maintenance job completed late but entered the following morning can otherwise disappear between reports. Consider a correction or reconciliation policy for late-arriving data.

Keep the reporting window independent from the actual start time. If the task begins late, it should still process the intended period unless the run policy explicitly says otherwise. Moving the window during recovery makes successive reports difficult to compare.

A catch-up decision also belongs in the contract. After an outage, should the application create each missed report, produce a consolidated report, or notify an operator? None is universally correct. Choose according to the consumer’s needs and record the choice.

Make repetition safe

Assign a stable business key to each intended result, such as business unit, report type, and reporting window. A retry should reconcile that key before producing another externally visible result.

The February durable-orchestration description provides background on workflows that wait and resume. Persistence helps coordinate execution, but the application still needs to establish whether an external write completed. Resuming a workflow is not proof that its effects occurred only once.

For the reporting example, write an internal versioned artifact before delivery. Mark which artifact is approved for distribution. If delivery times out, inspect its authoritative result before resending. Avoid letting a language-model explanation substitute for delivery evidence.

The 2024 τ-bench study assesses task outcomes through final database state and repeated trials. That evaluation principle transfers well to unattended runs: inspect the report and delivery state after retries, rather than judging only the agent’s final message.

Repeated execution can also produce a different summary from the same records. Keep the source snapshot or record versions needed to explain the difference. If a corrected report supersedes an earlier one, preserve that relationship rather than pretending both are independent daily results.

Bound authority and failure handling

Use a dedicated identity with only the access required by the recurring task. Revalidate permission at execution time. A schedule created by an authorized person should not become permanent permission to read data after access changes.

Restrict destinations through application configuration. A malicious instruction in a service record must not redirect the report or add a recipient. Treat retrieved text as information about the task, not authority to redefine it.

Set resource and retry bounds. Decide how long a run may continue, how many recoverable attempts are permitted, and when an operator receives an actionable failure. An agent should not keep searching indefinitely because a source system is unavailable.

Give incomplete data an explicit handling rule. If one approved source fails, the application may permit a report that clearly identifies the missing section, or it may require withholding the report. The business owner should make that decision before a live failure forces an improvised answer.

For any write-capable extension, separate proposal from execution and preserve the required approval. A timer can authorize starting analysis without authorizing the agent to make every change it recommends.

Prove useful unattended operation

Test duplicate invocation, a delayed start, a restart after artifact creation, a timeout after delivery, revoked access, and late-arriving records. Verify both the intended output and the absence of duplicate external effects.

Add a failure rehearsal for operators. They should be able to find the relevant run, identify the missing or uncertain step, and recover it without manually rebuilding the whole conversation. Keep source and output identifiers in the operational record.

Measure useful completed runs, incomplete reports, delivery errors, duplicate results, correction effort, and the time required to resolve failures. Track total cost per accepted report, including review and operation. A higher run count is not automatically a better outcome.

For the pilot, compare the generated brief with the existing manual report over equivalent periods. Ask the report’s consumers whether it changes a decision or saves meaningful effort. If they must reconstruct the source analysis every day, the automation has not yet earned its operating cost.

Keep the preview evaluation separate from the production commitment. Once the task boundary, recovery path, and value are established, the team can reassess the service’s release status and deployment requirements. A reliable recurring agent begins with a precise definition of one successful run.

References

Microsoft: Foundry Build Edition, June 2, 2026

Microsoft: June digest, published July 7, 2026

Microsoft: Durable agent orchestration, February 2026

Yao and colleagues: τ-bench, 2024

An invoice extractor returns the supplier, account reference, and total in valid JSON. The total is wrong. A downstream payment workflow accepts the structure and creates a payable record. The extraction looked technically successful until the business consequence appeared.

Document automation needs a decision about which values can proceed, which need review, and which should be rejected. Confidence and grounding help supply evidence for that decision. They do not replace business validation.

Keep the production and preview paths distinct

Microsoft’s August 12 Content Understanding update distinguishes a refreshed CU 1.0 GA API from CU 2.0 public preview. The preview introduces capabilities including synchronous Read and Layout APIs and more advanced document processing. Choose the API path according to the required feature and its maturity.

Do not infer production suitability from a model family’s availability alone. The analyzer, API version, processing mode, and model deployment form a combined configuration. Record and evaluate that combination.

The accompanying model-selection guide describes comparing deployments while holding the analyzer and inputs constant. That is valuable when the goal is to understand whether a model change actually improves a document workflow.

Read the reported efficiency results carefully

In its August guide, Microsoft reports up to 28% fewer total inference tokens and up to 25% lower full-inference LLM cost in tested configurations. It also reports confidence-score and grounding improvements measured using AUROC and grounding exact match. These are vendor-reported internal results, not a guaranteed reduction in the total cost of processing an invoice.

The separate August update associates the token result with GPT-4.1 and GPT-5.2 evaluations. The public summaries do not provide enough detail to assume the same improvement for every document type, schema, or deployment. Keep the “up to” qualification and test the actual configuration.

Token savings affect only part of the business case. Review effort, document preparation, storage, retries, exception handling, and correction of downstream records can dominate the total.

Begin with a narrow business document

Consider a hypothetical accounts-payable workflow for invoices from a defined supplier group. Start with required fields and explicit validity rules: supplier identity, invoice number, currency, line totals, tax, and payable amount.

Preserve the original document and a stable identifier for each processing attempt. Connect each extracted field to available source evidence. When a reviewer challenges a value, the interface should show the relevant document area rather than ask them to search the full file.

Use authoritative business systems to validate supplier and purchase-order details. A document can accurately contain a fraudulent account number. Extraction correctness and transaction legitimacy are separate checks.

Calibrate review by field and consequence

A single document-level score is rarely enough. The total amount may need stricter review than an optional description. A field’s business impact should influence the release rule alongside extraction confidence.

Build a labelled validation set from representative documents. Compare score ranges with observed correctness for each important field and document class. Do not treat a confidence value as a universal probability of correctness unless that interpretation has been established for the configuration.

Recheck calibration after model, analyzer, schema, or input changes. A threshold that was useful for clean supplier PDFs may behave differently on photographed documents or a revised invoice layout.

Use grounding to accelerate the review

Grounding should make the reviewer faster and more accurate by connecting a proposed value with its origin. Test whether the highlighted evidence supports the exact value and context.

An amount beside a subtotal label is not evidence for the final payable amount. A repeated supplier name in a footer may not identify the invoicing entity. Include these distinctions in the evaluation rubric.

The original DocVQA research established document-image question answering as a task requiring understanding of document content and layout. It is useful background for evaluating source location and interpretation, rather than assuming plain text extraction captures every important relationship.

Define the exception queue before increasing volume

Route incomplete documents, conflicting values, unsupported formats, and failed business checks into explicit states. Give reviewers the reason for routing and the fields that require attention.

Avoid sending every uncertain document through repeated model calls without a stopping rule. Repeated extraction can increase cost while producing several plausible answers to the same ambiguous input.

Distinguish a review correction from a new source document. If the supplier sends a revised invoice, retain version relationships and prevent the older record from re-entering processing as a separate payable.

Measure accepted documents and missed errors

Report field accuracy, critical errors that escaped review, review rate, reviewer time, and correctly completed documents. Show results by supplier group and input quality rather than relying on a single average.

Calculate total cost per accepted document. Keep rejected and reprocessed documents in the cost numerator. Report model inference savings separately so leaders can see whether they changed end-to-end economics.

Microsoft’s model lifecycle guidance recommends workload-based comparison and controlled upgrades. Apply that discipline to the full extraction configuration, not just the language model name.

Keep extraction separate from business execution

The extractor should propose structured information with evidence. A deterministic validation layer should apply arithmetic and business rules. A review layer should resolve exceptions. A separate authorized operation should create or update the downstream record.

Use idempotency and reconciliation at the integration boundary. A timeout after creating a payable must not lead to an automatic duplicate submission. The document identifier and business record should provide a way to establish the confirmed outcome.

For a first release, favour a smaller set of documents with well-defined exception handling over broad format coverage with unclear review rules. The practical goal is reliable automation with a manageable review burden. Grounding and confidence become valuable when they help the team make and verify those decisions.

References

Microsoft: Content Understanding updates, August 12, 2026

Microsoft: GPT-5 series and confidence guide, August 12, 2026

Mathew, Karatzas, and Jawahar: DocVQA, 2020

Microsoft: Model lifecycle guidance, June 2, 2026

A cheaper model response can create a more expensive support case. If a customer needs another explanation, an engineer corrects the answer, or the agent retries several tools, the token saving may disappear before the task ends.

That is the test I would apply to Model Router in Microsoft Foundry. The relevant comparison is the total cost of an accepted business outcome at an agreed quality level. Selecting a model dynamically is an implementation choice whose value must be measured.

Establish the fixed-model baseline first

Microsoft’s May 19 router evaluation guide describes an open-source approach that measures quality, cost, latency, and selected-model distribution. It explicitly includes router input-prompt billing alongside underlying inference costs. That is a useful starting point for a defensible comparison.

Before running it, define the baseline deployment and workload. A baseline should reflect what the application would otherwise use, with the same retrieval sources, tools, and output requirements. Comparing a carefully configured router with an artificially weak baseline produces a promotional result rather than an engineering decision.

Microsoft’s June model lifecycle guide places selection inside a broader operating loop. Use that approach to separate model capability from application correctness. A good answer still needs valid citations, appropriate permissions, and compatible tool arguments.

What earlier routing research demonstrates

The FrugalGPT study, first released in 2023, evaluated learned cascades across tasks including news classification, reading comprehension, and scientific question answering. It reported matching the best individual model with up to 98% lower cost, or improving accuracy over GPT-4 by 4% at the same cost in its experiments.

Those are separate research comparisons under historical models, prices, and datasets. They are not Foundry Model Router results, and the reported accuracy percentage should not be relabelled as percentage points without checking the underlying comparison.

The research supports testing workload-dependent selection. It does not justify putting a 98% reduction into an enterprise budget forecast. Build the forecast from measured task mix, current pricing, and actual downstream work.

Build a dataset that resembles the queue

Imagine a hypothetical employee-support assistant handling routine policy questions, complex eligibility questions, and requests requiring a business-system lookup. The easy questions may favour a lower-cost path. The complex cases may require stronger reasoning or escalation.

Sample the actual queue by task class and include difficult cases deliberately. Preserve a separate holdout set for the final comparison. Do not repeatedly tune routing rules against the same small set and then call its result independent validation.

Include long prompts, missing evidence, contradictory sources, multilingual requests where relevant, and tool failures. The Lost in the Middle research showed why context use deserves testing rather than being inferred from maximum input size. A model accepting the prompt does not establish that it used the important passage.

Compare outcomes before averaging costs

For each request, retain the task identifier, configuration, selected model where available, accepted result, latency, and cost components. Distinguish an answer that was generated from one that met the review criteria.

Report quality by task class before reporting an overall average. A small improvement on frequent easy requests can hide a serious regression on infrequent eligibility decisions. The business owner should decide which classes have hard acceptance requirements.

Calculate cost per accepted task by dividing total processing and correction cost by accepted outcomes. Keep rejected responses in the numerator. Otherwise the most error-prone configuration may appear cheapest because its failures were removed from the calculation.

Account for routing and recovery

Include input and output inference, routing charges, retrieval, tool calls, retries, monitoring, evaluation, and human correction. Show which components were measured and which were estimated.

Track the distribution of selected models. A change in that distribution may explain a cost or latency movement even when the application code stayed the same. Preserve enough configuration detail to compare runs after the available model pool changes.

Test what happens when an otherwise eligible model is unavailable or lacks the required capability. The fallback path must preserve the task contract. A substitute that cannot produce the expected structured output is not a successful fallback.

Keep the evaluator from deciding the result by accident

The May guide describes dual-ordered pairwise scoring to reduce position bias in model-based judgement. Use that as a useful check, then calibrate judgement against domain reviewers on representative cases.

Blind reviewers to the candidate where practical. Give them explicit criteria for correctness, evidence, completeness, and policy compliance. Keep evaluator version and prompts with the result, because changing the judge can change the apparent winner.

For transactional tasks, inspect the final system state as well as the response. A judge may prefer a polished explanation that accompanies an incorrect tool action. Deterministic checks should assess facts that can be verified directly.

Make a decision by workload class

A reasonable result may be mixed: the router suits routine questions, while a fixed deployment remains preferable for a specific constrained workflow. There is no requirement that one serving policy handle the entire application.

Set release thresholds before the experiment. Include permitted quality regression, latency limits, cost objective, and any task classes where regression is unacceptable. Label these as business choices rather than universal platform recommendations.

Stage the rollout with a fallback to the known-good path. Monitor accepted task quality and cost distribution after release, because production traffic may differ from the offline dataset.

Keep the economics honest over time

Re-evaluate when pricing, model eligibility, prompts, retrieval, or the task mix changes. A successful test describes a configuration and workload at a point in time.

The operating report should show where savings came from and where review or retry work increased. That allows the sponsor to decide whether the trade-off remains worthwhile.

Model Router can reduce manual model-selection work and may improve application economics. The business case becomes credible when the team can demonstrate lower total cost for accepted tasks while preserving the quality and behaviour those tasks require.

References

Microsoft: Router evaluation guide, May 19, 2026

Microsoft: Model lifecycle guide, June 2, 2026

Chen, Zaharia, and Zou: FrugalGPT, 2023

Liu and colleagues: Lost in the Middle, 2023

The monitoring dashboard is green. The agent’s requests completed, latency is within target, and no service returned an error. Yet a support specialist reports that the assistant recommended the wrong maintenance procedure.

This is an application incident even if every API returned success. Investigating it requires evidence about the retrieved source, selected tool, model configuration, and resulting recommendation. Infrastructure health is one part of that evidence.

Trace the business task across components

Microsoft’s March Agent Service announcement describes evaluation and monitoring capabilities as part of the platform. The June observability announcement adds a more detailed map of the development loop, including preview extensions for multi-turn evaluation, trace replay, and turning traces into datasets.

Do not assume that a foundational GA capability gives every new extension the same status. Identify which tracing, evaluation, and retention surfaces the application uses and verify them separately.

My recommendation is to start with an application task identifier. Propagate it through orchestration, retrieval, model requests, tool calls, and review outcomes. That makes it possible to ask what happened to one customer request across several systems.

A successful response is not a successful task

The 2024 τ-bench research found less than 50% task success for its tested function-calling agents and retail pass^8 below 25%. Those historical results are not a scorecard for current Foundry deployments. They demonstrate why repeated task-level checks can reveal weaknesses that a plausible conversation hides.

For a support agent, a success label should depend on the correct recommendation and permitted final state. If the agent cites an obsolete procedure, record that as a quality failure even when retrieval and generation completed normally.

Separate task success, policy compliance, and service availability. Combining them into one green indicator makes operational decisions harder. Different failure classes require different owners and remedies.

Reconstruct an illustrative incident

Imagine a hypothetical industrial support assistant that advises replacing a component. A reviewer later discovers that the equipment variant uses a different procedure. The trace shows the assistant retrieved a manual for the correct product family but the wrong variant.

That evidence points first to retrieval and metadata filtering. Changing the model’s general instructions might suppress the symptom without fixing the source selection. The investigation should establish where the wrong variant entered the workflow.

Retain the source document identifier, revision, retrieval filter, and tool arguments where permitted. Store the model and prompt configuration alongside the task. Without those details, a later replay may use changed sources and produce a misleadingly correct answer.

Collect evidence deliberately

Content capture can expose customer information, proprietary documents, and credentials accidentally returned by tools. Decide what the investigation genuinely needs before enabling broad logging.

Use identifiers, timing, configuration versions, and outcome labels as the default operational layer. Capture sensitive content only through a defined process with access controls and retention appropriate to its purpose.

Test the telemetry pipeline with representative sensitive inputs. Verify redaction where intended and check error paths, because exceptions often contain more information than normal logs. A configuration that protects successful requests may still leak content when a dependency fails.

Monitor quality by meaningful slices

Aggregate accuracy can improve while a critical equipment class becomes worse. Group results by task type, language, source family, model configuration, and relevant tool path. Keep small samples visible rather than reporting unstable percentages with false precision.

Use human review for cases where domain judgement is essential. Calibrate any model-based evaluator against that review. Track disagreement so an evaluator change does not silently redefine what the dashboard calls success.

Compare the current release with a stable baseline. Alert on a meaningful change in important task outcomes, then inspect whether the cause is a new model, revised source, changed tool, or traffic shift.

Design an incident response sequence

First contain the consequence. Pause a problematic write path or direct affected requests to review while preserving safe read-only assistance where appropriate.

Next identify the affected versions and tasks. Use the task identifier and configuration record to find potentially incorrect recommendations. A model deployment name alone may not describe every relevant component of the release.

Then verify the business impact. Determine whether the recommendation was merely generated, accepted by a reviewer, or acted upon. Those are different incident scopes and may require different communication and correction.

Finally, reconcile unfinished work. A rollback can restore a prior configuration, but it cannot determine whether a previous tool call completed. Use authoritative records to resolve uncertain outcomes.

Turn the incident into a release test

Build a regression fixture from the failure with permitted data. Include the confusing variants and the correct procedure. Run it against the proposed correction and the baseline.

Add neighbouring cases rather than a single memorized example. If a metadata filter fixes one product but excludes valid sources for another, the correction is incomplete.

The July hosted-agent GA announcement is useful context for managed operations, while the earlier Agent Framework release description explains orchestration foundations. Neither replaces an application-specific incident test. The release decision should use evidence from the actual support workflow.

Connect observability to cost and accountability

Calculate evaluation and telemetry cost alongside model and runtime spend. Sample ordinary traffic where appropriate, while deliberately retaining evidence for important failures under the chosen policy.

Report time to detect a quality regression, time to identify its cause, unresolved affected tasks, and recurrence after correction. Pair those measures with accepted business outcomes and human correction effort.

Assign an incident owner who can coordinate data, application, model, and platform teams. The purpose of a trace is to shorten the path from a bad outcome to a verified correction. A richer dashboard is useful only when it helps that process.

For the first rollout, choose one task family and walk through a controlled wrong-answer incident from detection to regression test. If the team can explain the failure and prove the correction, observability is doing practical work.

References

Microsoft: Agent Service evaluation and monitoring, March 16, 2026

Microsoft: Observability and ROI, June 3, 2026

Yao and colleagues: τ-bench, 2024

Microsoft: Hosted-agent GA announcement, July 9, 2026

Microsoft: Agent Framework architecture, February 19, 2026

An enterprise may know who owns its customer database and still struggle to name the owner of an agent that reads it, calls a pricing service, and prepares a customer offer. The agent crosses several teams’ boundaries in one task. Governance has to connect those responsibilities.

The first control I would establish is a clear answer to three questions: who owns the outcome, whose authority the agent uses, and who can stop its actions. A product inventory helps, but an entry in a catalogue is not an operating agreement.

Production hosting changes the operating obligation

Microsoft’s July 9 Foundry announcement identifies hosted agents as generally available alongside new models and deployment capabilities. Microsoft also reports that more than 100,000 organizations are building on Foundry. That is a vendor-reported adoption figure, not a measure of successful deployments or proof of governance maturity.

A managed hosting milestone can remove infrastructure work. It does not remove the application owner’s responsibility for incorrect quotes, unauthorized access, or unfinished transactions. Review the agent as a business application with delegated authority.

The April developer announcement and June hosted-agent update provide background on runtimes, identity, and controls. Read feature-specific maturity and deployment requirements rather than carrying a general GA label across every related capability.

Define the business action before the tool

Consider a hypothetical sales-support agent that assembles a renewal proposal. Its read operations retrieve contract terms and approved price information. Its write operations might save an internal draft or submit an offer for review. Sending the offer to the customer is a separate business action.

These operations have different consequences and should have separate authorization rules. A broad tool called “manage renewal” makes it difficult to distinguish harmless preparation from an external commitment.

Create narrow operations with explicit targets and validated arguments. The application should resolve the customer account from authenticated context and verified records. An instruction embedded in a document must not select a different tenant, customer, recipient, or authority.

Establish an owner for each control

The business owner defines permitted outcomes, escalation rules, and materiality. The application owner implements transaction validation and recovery. Identity teams establish access and credential lifecycle. Platform teams own deployment, monitoring, and configuration distribution.

Security teams should assess abuse paths, but they should not become the default owner of every unresolved application decision. A policy requiring approval is ineffective unless an application team implements it and a business team supplies authorized reviewers.

Record who can change prompts, tools, model deployments, and policy configuration. Treat those changes as application releases. A small configuration update can increase authority without changing the repository’s main code path.

Use adversarial research to improve test design

The 2024 AgentDojo study created 97 realistic tasks and 629 security test cases for agents operating on untrusted tool data. Those counts describe benchmark coverage, not the probability that a Foundry agent will be compromised. The study’s value here is its stateful testing approach.

A renewal agent should encounter a supplier document containing instructions to send information elsewhere, a tool response claiming that approval is unnecessary, and a retrieved note that conflicts with current policy. Score whether the agent completes the legitimate task and whether the application blocks prohibited actions.

Microsoft’s AI Red Teaming Agent announcement describes preview testing for risks including sensitive-data leakage and prohibited actions. Treat such tools as part of a wider evaluation programme. Confirm that the chosen tests reflect the actual tools and permissions of the application.

Keep policy enforcement outside model persuasion

Prompts should explain expected behaviour. Deterministic controls should enforce resource scope, recipient restrictions, financial bounds, and required approvals. A model-generated justification must not grant an exception.

Where a tool accepts a record identifier, validate that the caller may act on that record. Where it accepts an external destination, restrict that destination according to application policy. These checks remain necessary even when the agent usually chooses correct parameters.

Do not give every tool the same credential. Narrow identities make the effect of a compromised component easier to contain and investigate. Confirm the selected hosting path’s actual identity behaviour before assuming a deployment supplies the intended separation.

Plan for changes in authority

An agent that begins with read-only access can accumulate privileges through normal product development. Review new tools against the original task boundary. Adding an API may create a new disclosure route or business commitment.

Maintain an inventory of active agent versions, tool definitions, policy versions, and approved deployment destinations. Include dependencies outside Foundry where they participate in the workflow. The governance boundary should follow the business action across services.

Test removal of access as well as addition. A revoked role or disabled connection should lead to a clear failure or escalation. It should not cause the agent to search for a broader credential or alternate route.

Make stopping and recovery operational

Define how to stop new work, prevent pending writes, and reconcile actions already submitted. A deployment rollback does not reverse an offer that was sent or a customer record that was changed.

The runbook should identify the authoritative transaction log and the operator who can determine what happened. Preserve operation identifiers so incident responders can distinguish an attempted action from a completed one.

Exercise this process with a controlled failure. The test should show how unfinished work is located, how the customer impact is assessed, and how service resumes. Record the evidence rather than accepting a diagram as proof.

Give leaders a useful governance report

Report agents by business owner and permitted action scope. Include failed authorization tests, overdue remediation, unresolved transaction outcomes, and changes that increased autonomy. Separate these from general traffic and token usage.

For the renewal example, pair technical measures with incorrectly scoped proposals, review corrections, and confirmed customer impact. A dashboard with many successful model responses can still conceal a serious authority problem.

Begin expansion only when the current boundary is understood and tested. Foundry can provide useful hosting and governance building blocks. Enterprise control comes from connecting those capabilities to named owners, enforceable authority, and evidence about the resulting business actions.

References

Microsoft: July 9, 2026 Foundry announcement

Microsoft: Developer journey and identity foundations, April 22, 2026

Microsoft: Hosted-agent controls, June 2, 2026

Microsoft: AI Red Teaming Agent, November 2025

Debenedetti and colleagues: AgentDojo, 2024

A caller interrupts an appointment assistant halfway through a confirmation: “Actually, make that next Thursday.” The assistant has already started checking availability. The important question is whether it updates the intended date before booking, rather than how natural its voice sounds.

Voice applications combine conversational timing with business transactions. A technically impressive speech demonstration can still create the wrong booking, repeat a payment request, or leave a caller unsure whether anything happened. Design the transaction and conversation together.

Choose the correct Voice Live integration

Microsoft’s March Agent Service announcement described the Voice Live integration in preview. The June Build recap distinguishes GA Voice Live prompt agents from preview hosted voice agents. Those paths provide different deployment choices and should not share a blanket maturity claim.

The June hosted-agent announcement describes real-time connectivity through WebSocket and WebRTC, including custom runtime and framework options. Select a path according to the control the application needs over orchestration, channel behavior, and recovery.

Before committing to a stack, demonstrate the specific transport, model, identity, and tool configuration together. A supported speech feature in one path does not prove equivalent behavior in another.

What speech research does and does not establish

The 2022 Whisper research paper reports training on 680,000 hours of multilingual, multitask supervision. Its best zero-shot model achieved a 2.5% word error rate on LibriSpeech test-clean. Against a supervised model with similar in-distribution performance, it reported a 55.2% average relative error reduction across other evaluated speech datasets.

Those historical results concern Whisper under the paper’s conditions. They do not measure Voice Live latency or guarantee its transcription quality. Their relevance is the gap between a clean benchmark and performance across different recording conditions.

For a service workflow, the transcript metric also has a business limitation. Mishearing a filler word and mishearing an appointment date may contribute similarly to word error rate while having very different consequences. Test critical entities separately.

A practical appointment workflow

Imagine a hypothetical maintenance company taking calls to arrange equipment visits. The agent identifies the service request, checks eligible appointment windows, reads back the selected date and address, and creates the booking only after the required confirmation.

Represent the appointment as structured state outside the transcript. The date, timezone, site address, service type, and customer identity should have explicit validation status. Do not infer that an earlier confirmation still applies after one of those values changes.

Keep the availability lookup read-only. Put the booking operation behind a separate application check that verifies the final slot, current availability, and confirmation evidence. The speech model can lead the conversation while deterministic code enforces the booking contract.

Handle interruption at the right layer

When the caller interrupts, stop or adjust the spoken response promptly, then decide which application work remains valid. Stopping audio does not necessarily cancel a running API request.

If a lookup is still relevant, keep its result. If a write may have completed, reconcile it before submitting another one. When the caller changes the appointment date, invalidate the old proposal rather than merely replacing a phrase in the conversation history.

Use an operation identifier for each booking attempt. If the connection drops after submission, a resumed session should query the booking record and explain the verified outcome. It should not create a second reservation because the first confirmation was never heard.

Budget latency across the whole interaction

Measure the interval from the caller finishing an utterance to useful spoken feedback. Break it into capture and endpointing, model processing, tool work, and playback. Report both typical and slow interactions.

Separate an acknowledgement from a completed answer. Saying “I’m checking that” may improve conversational clarity, but it does not reduce the time until confirmed availability. Track those moments independently so the team does not optimize an artificial response-time metric.

Test realistic devices, background noise, variable connections, accents, and supported languages needed by the service. Use permitted recordings or synthetic fixtures with known ground truth. Keep the same test inputs when comparing configurations.

Protect critical details without making every turn tedious

Read back the details that materially affect the transaction. A date, service address, or charged amount may require explicit confirmation; a casual preference about wording usually does not.

Set a defined clarification path when the agent is uncertain. Repeating a question in different words can help, but repeated failure should trigger a useful alternative such as a text confirmation or transfer where the business supports it.

The handoff should include the verified task state and unresolved question. Avoid forcing the customer to repeat the entire interaction merely because the agent reached a boundary. Do not tell the caller a transfer or follow-up has occurred until the corresponding operation is confirmed.

Measure service outcomes

For the maintenance example, score correct appointments, wrong-date bookings, duplicate bookings, abandoned calls, and successful recovery after disconnection. Include the time staff spend correcting agent-created records.

Calculate cost per correctly completed request using speech, model inference, tool infrastructure, monitoring, and correction effort. Longer calls and repeated confirmations affect economics even when the per-token model rate is attractive.

Run the pilot against an existing service baseline on equivalent requests. Human review should check both the conversation and the final booking. A pleasant interaction with a wrong record is a failed task.

Launch with a bounded service promise

Start with one service type and a clear escalation path. Publish an operational runbook for unavailable scheduling systems, uncertain identity, caller corrections, and interrupted writes. Assign an owner for both conversation quality and transaction reliability.

Review failure clusters before expanding supported tasks. Some problems require a better model; others require a clearer question, stronger input validation, or a more reliable booking API.

A successful voice agent reduces the effort needed to complete a service request while preserving the caller’s control over the result. Natural speech helps people use it. Correct state, clear confirmation, and reliable recovery make it useful.

References

Microsoft: Agent Service and Voice Live, March 16, 2026

Microsoft: Foundry Build Edition, June 2, 2026

Microsoft: Hosted-agent updates, June 2, 2026

Radford and colleagues: Robust Speech Recognition, 2022

An architecture review can absorb an entire release catalogue and still end without a decision. Build 2026 adds choices across agent development, hosting, tools, memory, and grounding. The useful question for an enterprise team is which change removes an observed obstacle in its own workload.

I would start with one accepted business task and trace where it fails today. Is the problem finding authoritative information, choosing an appropriate tool, recovering interrupted work, or proving that the result is correct? Each answer leads to a different investment.

Separate available capabilities from announced direction

The June 2 Foundry Build recap distinguishes stable developer components from preview services and future milestones. In particular, it describes Foundry Toolkit for VS Code as GA, memory and Toolboxes as preview, and hosted-agent GA as expected in early July. An expected date is a planning input, not a delivered production guarantee.

The Agent Service announcement presents a build, deploy, and operate approach. That is a useful way to organize responsibility, but it should not turn into a requirement to replace every component at once. Retain working parts of the application unless a change has a measurable benefit.

For each proposed capability, record the required feature, release status, deployment constraints, dependency owner, and acceptance evidence. A broad platform label cannot answer whether a specific preview integration is suitable for the application’s operating requirements.

Use the business bottleneck to choose a pilot

Consider a hypothetical equipment manufacturer whose support engineers prepare maintenance recommendations. The assistant currently retrieves manuals, checks equipment records, and drafts an explanation for an engineer. It struggles when manuals disagree and when an interrupted diagnostic sequence resumes.

If retrieval is the dominant failure, assess the grounding path first. Preserve document versions and test whether the correct evidence reaches the model. A richer agent runtime will not repair an index containing superseded maintenance instructions.

If failures arise after tool calls or restarts, assess orchestration and state recovery. If the assistant consistently knows the relevant information but omits required steps, procedural guidance or memory may deserve a controlled experiment.

This prioritization keeps the pilot small enough to interpret. Changing retrieval, memory, prompts, models, and hosting together can improve a score while leaving the team unable to explain why.

Read early memory results as research signals

The June 2 Agent Service post reports early Tau-bench absolute success-rate gains of 7–14 percentage points with procedural memory at near-baseline cost. This is Microsoft’s reported benchmark observation. It is not an enterprise-wide productivity estimate or a promise for the manufacturer’s support tasks.

The separate June 3 memory article reports about a 5% improvement in its STATE-Bench and Tau-Bench evaluations. It does not establish that this is the same configuration, denominator, or improvement measure as the earlier range. The two statements should not be combined into a single expected benefit.

That distinction matters in a business case. Use an external finding to justify testing a hypothesis, then measure the application under its own tasks, tools, and failure conditions. Approval for broad rollout should depend on that second result.

Define a comparison that can settle the decision

Freeze a baseline configuration and a representative task set. Include ordinary cases, ambiguous manuals, unavailable equipment records, and requests that should be escalated. Assign reviewers who understand the operational policy rather than asking a model to decide every question of correctness.

Compare the proposed capability against the baseline with equivalent input data and access. Keep failed and escalated tasks in the report. If an experiment excludes hard tasks, label that restriction and do not generalize the result to the whole service queue.

Measure accepted recommendations, missed mandatory checks, unsupported claims, elapsed resolution time, and cost per accepted result. Add a recovery test if the pilot changes runtime behavior. A successful happy-path demonstration cannot settle whether interrupted work is safe.

Make observability part of the pilot budget

The Build observability announcement distinguishes the tracing and evaluation foundation from new preview extensions, including multi-turn evaluation and trace replay. Record the specific surfaces used in the pilot and confirm their maturity separately.

My recommendation is to correlate each business task with model calls, retrieval, external actions, and the final review outcome. Capture enough configuration information to reproduce a failure, while keeping sensitive content out of routine telemetry unless it is deliberately required and controlled.

Include evaluation and human review in the cost calculation. A cheaper inference path can be more expensive overall if it causes additional review or repeat work. The business owner should see those costs alongside the quality result.

Keep the release decision reversible

A new dependency should have a fallback. For grounding, retain a known-good source configuration. For a model, retain the prior deployment while testing compatibility. For a workflow change, define how unfinished tasks are identified and completed without duplicate action.

Give the pilot an explicit exit condition. Examples include a material correctness improvement on a priority task class, a reduction in total handling effort, or recovery that meets an agreed operational objective. These are proposed acceptance categories, not claimed product outcomes.

If the experiment is inconclusive, keep the existing production path and revise the test. A release announcement does not create an obligation to adopt the feature immediately.

Put ownership beside each investment

The data owner should approve source quality and freshness. The application team should own workflow correctness. The platform team should own deployment and telemetry. Security and identity teams should verify access boundaries. The business sponsor should decide what improvement justifies the operating cost.

Ask each owner what evidence would make them comfortable with a wider release. Resolve incompatible expectations before the pilot starts. Otherwise a technically successful experiment can reach review with no agreed definition of production readiness.

For the next planning cycle, fund one constrained experiment with a baseline, accountable owners, and a rollback path. Foundry’s broader platform creates more ways to improve an application. The discipline is choosing the capability that addresses a demonstrated problem and proving its value before expanding.

References

Microsoft: Foundry Build Edition, June 2, 2026

Microsoft: Build and run agents at scale, June 2, 2026

Microsoft: Memory reliability and management, June 3, 2026

Microsoft: Observability and agent ROI, June 3, 2026

A returning customer says, “Use my usual delivery arrangement.” Remembering the preference could save a conversation. Remembering a previous customer’s arrangement could create a privacy incident. Remembering an outdated instruction could send an order to the wrong place.

Long-term memory makes an AI assistant more useful only when the application can distinguish a preference, a current fact, an old fact, and an authorization decision. The first implementation task is to define what each category means for the business.

Start with the feature that exists

Microsoft introduced memory in Foundry Agent Service in public preview in November 2025. The announcement describes extracting and retrieving user information and conversation summaries across interactions. That offers a managed foundation for continuity, with preview terms that teams need to evaluate before adopting it.

Keep the scope of that announcement clear. This design uses the documented memory concept and application controls; it does not assume that every retention, correction, or identity requirement is automatically solved by enabling memory.

The broader Agent Framework architecture provides orchestration and session building blocks. A workflow checkpoint, a conversation history, and a remembered customer preference serve different purposes. Treating them as one unstructured store makes retention and testing unnecessarily difficult.

The research argues for explicit evaluation

LongMemEval, introduced in 2024, uses 500 curated questions to examine long-term conversational memory. Its abstract reports an approximately 30% accuracy drop across sustained interactions in evaluated commercial assistants and long-context models. The benchmark tests extraction, cross-session reasoning, time, updates, and abstention. These findings describe its tested systems and conditions, not Foundry performance.

The older Lost in the Middle study also found that access to a long context did not ensure reliable use of relevant information, particularly when its position changed. A larger input window is therefore an option to evaluate, not proof that a memory problem has disappeared.

For an enterprise architect, the implication is practical. Test what the assistant recalls, when the remembered fact was true, and when it should decline to answer. Storage success alone is a weak acceptance criterion.

Separate business records from conversational memory

Imagine a hypothetical distributor using a support agent for repeat customers. The order-management system owns shipping addresses and account permissions. The memory layer might retain a preference for concise status summaries or a usual delivery-time discussion. It should not become an alternative system of record for the customer’s authorized receiving address.

For a new order, retrieve the current address from the authoritative system and ask for confirmation when the workflow requires it. The remembered preference can make the conversation smoother without bypassing the transaction’s normal validation.

This distinction also helps with data quality. If a customer says an address changed, the assistant can propose the appropriate account-update process. It should not silently write a conflicting address into memory and later present that as verified operational data.

Bind scope to authenticated identity

Derive memory scope from the application’s authenticated tenant and user identifiers. Never allow a natural-language request such as “use the profile for user123” to select a different person’s memory.

Decide how shared accounts work before launching. An account-level purchasing preference may be visible to a team, while a person’s accessibility preference should remain personal. Those are different scopes with different access rules.

Test isolation at retrieval and update boundaries. A successful test proves both that another user’s facts stay hidden and that the current user cannot overwrite them indirectly. Include tenant changes, reassigned accounts, and recycled application identifiers in the test plan.

Preserve origin and time

A useful memory item needs enough provenance to explain why it exists. My recommended record includes its source interaction, time recorded, applicable scope, fact type, and any validity period. Avoid retaining sensitive raw conversation content merely because it is convenient.

Represent corrections explicitly. If a customer first prefers phone calls and later requests email, retrieval should prioritize the current preference while respecting the chosen retention policy. A vague summary containing both statements leaves the model to guess which one governs.

Some memories should expire because the business meaning expires. A travel preference for a particular week is not a lifelong customer attribute. Where the chosen preview interface lacks a required lifecycle control, implement that control in the application or postpone that memory category.

Let users correct the assistant

Provide a practical way to inspect or challenge remembered information. “That is no longer correct” should trigger a defined correction path. The agent should acknowledge the change without promising deletion across every system unless the application has completed that operation.

Deletion needs its own verification. Check that removed information no longer appears through direct lookup, semantic retrieval, derived summaries, or copied application caches within the scope you control. Document any separately retained audit record and its purpose.

A support team also needs a way to investigate a disputed memory without unrestricted access to private conversations. Use narrowly scoped diagnostics and log metadata rather than exposing all remembered content by default.

Build a memory-specific test set

Create multi-session cases with known facts, deliberate corrections, similar customers, and questions with no answer in the history. Include temporal questions such as which instruction applied before a change.

Compare memory enabled and disabled on the same task set. Score correct recall, stale-fact use, unsupported recall, cross-user exposure, and appropriate abstention separately. An overall average can hide a serious isolation defect behind better preference recall.

Keep the retrieved memory items with the evaluation evidence when permitted. That lets engineers distinguish extraction failure from retrieval failure and reasoning failure. Changing the model when the wrong memory was retrieved may make the symptoms harder to reproduce.

Decide whether continuity is worth its cost

Measure avoided clarification turns and successful task completion alongside retrieval latency, storage, model calls, correction effort, and privacy operations. Use the business task as the denominator rather than the number of memories stored.

A sensible pilot begins with low-risk preferences in one authenticated application. Expand the scope after demonstrating useful recall, reliable correction, and effective separation between users.

The strongest memory design is selective. It remembers information with a clear purpose, retrieves it within the right identity boundary, and knows when a fresh business record or another question is more reliable.

References

Microsoft: Memory in Foundry Agent Service, November 25, 2025

Microsoft: Agent Framework architecture, February 19, 2026

Wu and colleagues: LongMemEval, 2024

Liu and colleagues: Lost in the Middle, 2023