Enterprise AI pilots often succeed because they avoid the enterprise. They use a small, curated dataset, a simplified workflow, and enough computing capacity to make a demonstration feel effortless. Salesforce AI architect Franny Hsiao calls these environments “pristine islands.”
Production introduces fragmented data, variable volume, latency, permissions, unreliable connectivity, and actions that can affect customers. A model that looked intelligent in isolation can become slow or untrustworthy when connected to reality.
Scaling therefore begins with data architecture and governance, not a larger model.
Production Must Be Designed Into the Pilot
Hsiao identifies the failure to build production-grade data infrastructure with end-to-end governance as the most common architectural oversight. When teams postpone integration, normalisation, and transformation, they discover data gaps only after users depend on the system.
Observability and guardrails should cover the full lifecycle. Teams need to see what users ask, which data was retrieved, what tools an agent called, how long each step took, and where errors appeared.
Salesforce’s Session Tracing Data Model records questions, planner steps, tool calls, inputs, outputs, retrieved chunks, timing, and failures. That evidence supports adoption analytics, optimisation, and health monitoring while giving investigators a path through a disputed action.
Trace data requires protection too. It may contain customer information, prompts, and operational details. Retention and access should reflect the sensitivity of the underlying workflow.
Production tests should deliberately use imperfect records, competing definitions, peak volume, and unavailable dependencies. If the pilot never experiences the conditions most likely to break it, the evaluation has measured presentation quality rather than operational readiness.
Responsiveness and Offline Work Shape Trust
Reasoning models may take longer to produce an answer. Salesforce uses progressive streaming so users see output while heavier computation continues. Progress indicators and visible tool use can make latency feel deliberate rather than broken.
Perceived speed cannot conceal an unfinished or unsafe answer. Interfaces must show when content is partial and prevent users or downstream systems from acting before required checks are complete. Smaller models and response-length constraints can reduce both actual latency and cost when deep reasoning is unnecessary.
Field service adds another constraint: connectivity may disappear. An on-device model can identify a photographed part or error code and provide troubleshooting from cached knowledge, then synchronise when the connection returns.
Edge intelligence can improve latency, privacy, energy use, and resilience. It also needs version control, secure local storage, conflict resolution, and a way to handle knowledge that became outdated while the device was offline.
Field technicians should be able to see when guidance came from cached information and when synchronisation last occurred.
High-Stakes Actions Require Explicit Gateways
Salesforce defaults to human confirmation for creating, updating, or deleting records, contacting customers, making critical decisions, and actions vulnerable to prompt manipulation. Hsiao describes these checkpoints as architecture for accountability and continuous learning.
The threshold should follow consequence, reversibility, confidence, and context. A low-risk read operation may proceed automatically; changing a customer record or sending a message may require approval even when the agent is usually accurate.
Multi-agent systems add interoperability requirements. Salesforce supports Model Context Protocol and Agent-to-Agent standards for orchestration, while the Open Semantic Interchange effort aims to align meaning across systems. Shared transport is insufficient if two agents interpret “customer,” “revenue,” or “approved” differently.
Hsiao expects agent-ready data to become the next bottleneck. Searchable, contextual architectures must replace rigid pipelines that cannot provide the right information when an agent needs it.
The conclusion is less glamorous than another model release. Production AI depends on connected data, visible reasoning, responsive interfaces, offline resilience, semantic consistency, and human control at the consequential moments.
A pilot should test those conditions from the beginning. Otherwise, its success proves only that AI works on an island—and says nothing about whether it can survive the mainland.
