Enterprise AI12 min read

Scaling Agentic AI: From Fragmented Data to Better Business Outcomes

Scaling Agentic AI: From Fragmented Data to Better Business Outcomes
Moving beyond AI agent pilots requires more than adding agents to existing systems. Discover how to connect trustworthy data, semantic layers, data products, and enterprise harnesses to achieve real business outcomes.

An AI agent can produce a convincing answer and still fail the business. It can retrieve the wrong definition of a customer, recommend a purchase without the relevant contract context, or save an analyst an hour while creating two hours of verification work elsewhere.

That is the real challenge of scaling agentic AI. The model is only one component. The harder work is connecting trustworthy data, business meaning, controlled execution, and people who can turn the output into a useful outcome.

Experiences discussed in a PwC-moderated panel involving Markel, Nokia, and a PwC delivery representative working with CareQuest point to a consistent lesson: moving beyond pilots requires more than adding agents to existing systems. It requires designing the environment in which those agents operate and, eventually, reconsidering the work itself.

💡

Core Principle: The destination is not maximum autonomy. It is a better business process with an appropriate level of autonomy.

Start with the outcome, not the agent

“Where can we use AI?” is an easy question to ask. It is also a weak starting point.

A stronger question is: Which outcome is constrained by the way work happens today?

In procurement, that might be the time needed to assemble information before an approval. In finance, it might be the effort required to reconcile reports before meaningful analysis can begin. In research, it might be the delay between receiving a question and producing a defensible answer.

These are different problems. They do not necessarily require the same architecture, model, or degree of automation.

An agentic system uses a model to select or coordinate steps toward a goal, often by retrieving information and invoking tools. A natural-language analytics interface may use some of these mechanisms without being an autonomous business operator. A conventional workflow can also automate substantial work without needing a model to decide every step.

Keeping these distinctions clear prevents a common mistake: treating every AI-assisted process as a candidate for unrestricted autonomy.

Ownership must connect delivery to value

The panel highlighted an organizational friction that is easy to miss: the team building the solution may not be the team accountable for realizing its value.

A technically successful deployment does not automatically change staffing priorities, approval practices, or how employees spend their time. Those decisions belong to the business operating model.

Before development begins, identify:

  • The outcome owner who will judge whether the process improved.
  • The practitioners who understand its daily exceptions and constraints.
  • The engineering team responsible for implementation.
  • The operational owner who will support it after launch.
  • The control owners who determine what the system may access and do.

These are responsibilities, not necessarily five separate teams. The important point is that none should appear for the first time at the production handover.

A solution should also make the user’s day easier. If it introduces another dashboard, another review queue, and another approval step without removing existing work, its apparent productivity gain may disappear in practice.

Imperfect data is not a reason to do nothing

Waiting for a perfect enterprise data estate can become an indefinite postponement strategy.

A bounded use case can begin with a limited set of sufficiently reliable sources. The requirement is not that every database in the organization be clean. It is that the information needed for the selected task be understood, accessible under the right permissions, and fit for that task.

Scaling changes the equation. More users, more datasets, and more consequential actions expose inconsistencies that a small pilot can hide.

The useful distinction is between local readiness and enterprise repeatability:

  • Local readiness means a particular use case has adequate data and controls.
  • Enterprise repeatability means additional use cases can reuse definitions, access patterns, quality checks, and operational capabilities instead of rebuilding them.

Start narrow, but avoid building a dead end.

Fragmentation is about meaning as well as location

The CareQuest discussion offered a deceptively simple example: what does a member’s age mean?

It could mean age today, age at enrollment, or age at the date of service. All three values can be correctly calculated. Only one may answer the question being asked.

Bringing records into the same platform does not resolve that ambiguity. Neither does giving a language model access to every table.

The same problem appears in finance when different reporting bases produce different legitimate results, or in procurement when a supplier record lacks the contract conditions that determine whether a purchase is appropriate.

Data availability answers “Can I retrieve it?” Business context answers “What does it mean here?”

Semantic disambiguation diagram contrasting raw data availability showing member_age branching into multiple valid interpretations versus a semantic filter resolving the exact business metric for the AI agent

Build a semantic layer, not just a larger data store

Three related concepts help explain how organizations can make business meaning reusable.

Semantic definitions

A semantic layer expresses agreed business meaning: measures, dimensions, relationships, and calculation rules.

For example, a utilization measure needs a defined population, time period, numerator, and denominator. A financial measure may need a reporting basis, currency treatment, and consolidation scope.

These definitions should not depend on the model improvising them from column names.

Ontologies

An ontology describes the important concepts in a domain and their relationships. In procurement, those concepts might include suppliers, contracts, purchase orders, invoices, approvals, and business units.

Its purpose is to make relationships explicit. A supplier can be approved for one category but not another. A contract can apply to a particular legal entity and period. Those relationships matter to decisions.

Knowledge graphs

A knowledge graph represents entities and relationships in a connected form. It can help a system navigate from a question to the relevant business objects and supporting evidence.

These concepts overlap, but they are not interchangeable. A semantic model does not automatically require a graph database, and a graph does not guarantee that its definitions are correct.

Think of the data platform as a library. The semantic layer explains what the books mean; the ontology describes the subjects and relationships; a knowledge graph can help navigate those connections. None of them makes a false statement true merely by cataloging it.

Capture the knowledge that currently lives in people

Markel’s panel contribution emphasized reporting knowledge held by particular teams and experienced individuals. That knowledge can determine which sources to use, which adjustments to apply, and how to interpret a result.

Making it explicit reduces dependency on a single expert. But capturing it requires more than indexing conversations or documents. Informal material may be outdated, incomplete, or inconsistent with an approved policy.

Useful context needs ownership, review, and versioning. A conversation can explain why a decision was made without becoming the authoritative rule for future decisions.

Better context can also reduce unnecessary retrieval and repeated model calls. That is an architectural possibility, not a guaranteed cost reduction: graph queries, retrieval services, and additional validation have their own overhead.

Data products turn access into a repeatable service

A data product is more than a dataset with a friendly name. It is a usable package of information with defined meaning, ownership, access conditions, and an intended purpose.

The panel described a PwC marketplace accelerator that supported publishing and discovering data products, managing access, monitoring usage, and tracking AI consumption. That description should not be confused with a universal Microsoft product or an independently verified list of deployed features.

The underlying pattern is nevertheless useful. Instead of making every agent negotiate separately with every source system, organizations can expose curated products with stable definitions and clear access boundaries.

CareQuest’s public Health Data Exchange provides a concrete example of discoverable products. Its AI Data Explorer supports plain-English exploration of dental and medical data, while its AI Research Explorer supports questions about CareQuest’s published research. These are distinct capabilities: querying structured health data is not the same task as searching research publications.

The exchange catalog also includes datasets and dashboards. That matters because not every user needs an agent. Sometimes a governed dashboard or a well-defined dataset is the right interface.

Trust needs an inspectable path to the answer

The CareQuest delivery representative described exposing a parallel SQL query so advanced users could check results directly. This is a reported implementation detail from the panel, not a feature independently established by the public product pages.

The principle is strong: users should be able to inspect how an answer was produced.

However, showing generated SQL is not sufficient proof. The query could use the wrong population, join, date field, or denominator. If the explanation and SQL share the same mistaken assumption, their agreement proves little.

A defensible analytical answer should connect the question to approved definitions, source data, query logic, and validation checks. Reproducibility is valuable, but reproducibly answering the wrong question is still a failure.

The panel also associated the explorer with water-fluoridation analysis. CareQuest’s catalog confirms a related dashboard, but that does not independently establish that a particular publication was generated through the AI tool or that the tool caused a policy outcome. Access to evidence, production of research, and real-world impact are separate claims.

Centralize the meaning without centralizing everything

The panel contrasted two approaches: using AI embedded in enterprise applications, and building agents over a shared data platform.

Neither is a complete answer on its own.

Native AI in an enterprise resource planning (ERP) or enterprise performance management (EPM) platform can benefit from proximity to its workflows and application context. A shared platform can support questions that cross application boundaries and require consistent analytical definitions.

A practical hybrid architecture separates responsibilities:

  1. Systems of record retain authoritative transactions and application controls.
  2. Governed data products expose information suitable for analysis and reuse.
  3. Semantic definitions connect business meaning across sources.
  4. Agents and workflows retrieve evidence, coordinate steps, and invoke approved tools.
  5. Application services validate and execute changes under their own authorization rules.

This is a conceptual pattern, not a reconstruction of any panelist’s complete deployment.

The critical boundary is between reading and acting. Permission to analyze a purchase order does not imply permission to approve it. A centralized analytical copy should not quietly become an alternative transaction system.

Hybrid enterprise AI architecture diagram showing the separation between the upper Analytical Read Plane and the lower Transactional Action Plane with a strict human approval gate boundary

Hybrid also does not mean duplicating every capability. Evaluate native functionality before building a custom service, and preserve the ability to replace interim components as platforms mature.

The enterprise harness is what makes agents operable

The panel used “enterprise harness” to describe the reusable environment around AI solutions: governance, access controls, evaluations, observability, and operational support.

A harness is not a single switch. It is a combination of platform capabilities, application design, and organizational responsibilities.

Its purpose is to answer questions such as:

  • Which identity is making this request?
  • Which sources and tools may it access?
  • Which actions require approval?
  • How do we know the result is acceptable?
  • What evidence is retained when something goes wrong?
  • Who can stop, correct, or roll back the process?

The panel reported using Azure and Foundry, with Markel describing a financial repository built using Azure and Databricks. Current Microsoft documentation uses the name Microsoft Foundry. Its evaluation overview describes evaluating datasets, model and agent responses, and production traces. Its tracing documentation describes visibility into tool usage, retries, latency, and other execution details.

Those capabilities support a harness. They do not automatically supply a complete enterprise operating model. Feature status and prerequisites must also be checked individually: Microsoft’s tracing setup guide distinguishes generally available tracing support from preview workflow-agent support.

Availability is not correctness

A conventional service can return a successful status code while an agent has chosen the wrong action or supplied the wrong tool argument.

Observability therefore needs several layers: technical health, execution behavior, policy compliance, output quality, and business outcome. Microsoft’s agent observability discussion illustrates this distinction with a refund scenario in which a technically successful call uses an incorrect amount.

Five-layer agent observability stack in Microsoft Foundry illustrating diagnostic layers from technical health and tool execution parameters up to policy compliance and business outcomes

Traces also create a responsibility. Prompts, retrieved passages, and tool arguments can contain sensitive information. Access, redaction, retention, and telemetry cost belong in the design, not in an afterthought.

Set acceptance criteria by consequence, not by a generic percentage

One panel comment suggested that approximate results might be acceptable for some internal uses. That is not a safe universal threshold.

“Internal” does not mean low risk. An internal financial control or procurement approval can have significant consequences. Nor does “90% accurate” mean much without defining the test population, scoring method, and severity of failures.

Acceptance criteria should match the task:

TaskWhat to validate
Exploratory summarySource support, relevance, omissions, and uncertainty
Analytical queryDefinitions, filters, joins, calculations, and reconciliation
RecommendationEvidence, applicable policy, exceptions, and reviewability
Transactional actionAuthorization, exact parameters, approval rules, and recovery behavior

Consequence-based acceptance matrix comparing four risk tiers from exploratory summaries to transactional actions and their required deterministic validation controls

Groundedness measures whether an answer is supported by supplied evidence. It is not identical to truth: the evidence itself may be stale or wrong. Task completion is also not enough if the task was completed through an unauthorized action.

The NIST Generative AI Profile provides a cross-sector risk-management reference for generative AI. It is a voluntary framework, not a certification that makes a deployment safe by association.

⚠️

No finite evaluation set proves that a generative system will never produce an unsupported answer. Use testing to characterize risk, and design the process to contain failures.

For consequential workflows, deterministic calculations and policy checks should remain deterministic where practical. Human review should be a meaningful decision point with adequate evidence, not a ceremonial click after an opaque recommendation.

Audit teams can help define required evidence early. Execution traces support investigation, but they do not by themselves prove that a control is effective or that an output is correct. The panel’s discussion of future audit work was a statement of intent, not evidence that those controls had already been implemented.

Measure value beyond adoption and token spend

Usage is useful telemetry. It is not the outcome.

An agent can attract many queries without improving decisions. Another can be used infrequently but remove a critical bottleneck.

Useful measures include cycle time, rework, exception handling, review effort, and the amount of previously deferred work now completed. Financial assessment should include inference, retrieval, infrastructure, telemetry, support, maintenance, and human verification, not just tokens.

The panel’s references to tenfold returns and percentage productivity gains were aspirations or informal claims, not documented benchmarks. They should not become planning assumptions without evidence.

Choose models by evaluated task performance

Model routing means selecting a model for a task according to factors such as capability, latency, cost, and data requirements. A smaller model may be adequate for classification; a more capable model may be justified for difficult synthesis.

The rule is not “always use the cheapest model.” It is use the least costly option that satisfies the task’s evaluated requirements.

A model inventory or registry helps record versions and approved uses, but it does not automatically implement runtime routing. Switching models requires regression testing, and a router introduces its own failure modes.

Similarly, the panel’s comparison of out-of-the-box Copilot and custom OpenAI calls was anecdotal. Without the specific product, configuration, workload, and evaluation method, it cannot establish that one is generally faster or better.

Open-weight models are another possible architectural choice, not an automatic compliance shortcut or a category inherently excluded from healthcare. Assess the license, deployment environment, data handling, evaluation results, and support model. Local hosting changes responsibilities; it does not remove them.

Move from automating steps to redesigning work

Nokia’s panel contribution described procurement as a useful starting point because it combines high transaction volumes, repeatable steps, existing data, and opportunities to improve decision preparation.

Crucially, the reported approach retained humans as active decision-makers. Four use cases were described as beyond the minimum viable product stage, but the discussion supplied no quantitative benchmark for their results.

The broader lesson is to distinguish two levels of change:

  • Improve the current process: remove slow preparation, retrieval, or summarization steps.
  • Redesign the process: reconsider sequencing, handoffs, and exception handling around the desired outcome.

For example, an illustrative procurement workflow might gather supplier information and prepare an approval summary. A redesigned workflow could perform independent checks in parallel and route only unresolved exceptions to specialists, while preserving mandatory approvals.

Workflow architecture comparison contrasting traditional serial step acceleration with an outcome-oriented parallel agentic triage and exception-routing pipeline

Standardized processes are easier to evaluate. Exceptions reveal where domain expertise and control design remain essential. Do not assume that an old process is optimal, but do not remove a step until you understand why it exists.

AI expands some forms of capacity. It does not eliminate human judgment, accountability, or review bottlenecks.

Microsoft’s 2025 Work Trend Index frames a “capacity gap” between business demands and employees’ available time and energy. That is useful context for redesign, not proof that a particular agent will close the gap.

Time saved becomes value only when the organization decides what to do with it: deeper analysis, faster response, more research, or better exception resolution.

Upskilling is part of the architecture

Giving employees access to AI tools is not the same as preparing them to use those tools well.

The Markel panel contribution described finance innovation showcases, focused training, a train-the-trainer approach, and a community of practice. These are reported practices, not independently measured proof of adoption gains.

Their value lies in connecting discovery to scale. Practitioners reveal real problems, peers see reusable ideas, and IT partners can identify technical blockers before promising experiments become unsupported services.

Upskilling must run in both directions. Business users need to understand verification, permissions, and model limitations. Engineers need to understand business definitions, exceptions, and the consequences of incorrect actions.

A useful showcase should demonstrate more than a polished answer. It should explain the problem, the evidence, the controls, the remaining manual work, and what would be required to support the solution more broadly.

That is how experimentation becomes organizational learning rather than a collection of isolated demos.

Scale the system around the agent

The strongest path beyond the pilot plateau is not to deploy more agents indiscriminately. It is to make successful patterns repeatable.

Begin with a bounded outcome. Make the relevant data trustworthy and its meaning explicit. Separate analysis from authorized action. Evaluate the entire process, including review effort and failure handling. Bring practitioners, engineers, operations, and control owners together early.

Then use what you learn to redesign the work, not merely accelerate its existing steps.

The agent may be the most visible part of the solution. The durable advantage comes from the system around it: shared context, clear accountability, inspectable evidence, and people equipped to use the resulting capacity well.

Discussion

Loading...