Virexa

SecuritySecurity Advisory

Hush Security says the AI security problem has shifted from protecting models to governing identities as autonomous agents spread

Less than a year after emerging from stealth to tackle non-human identity security, Israeli cybersecurity startup Hush Security believes the enterprise AI security conversation has fundamentally changed. The company, which earlier this week announced a $30 million Series A round led by returning investors Battery Ventures and YL Ventures with Akamai Technologies joining as a strategic investor, argues that organizations are rapidly moving beyond experimenting with generative AI assistants and into deploying autonomous software agents that require an entirely different security model. While the funding will help expand engineering, U.S. sales and enterprise integrations, Hush is framing the announcement primarily as evidence that identity—not models—is becoming the critical control plane for enterprise AI. "The discussion has moved incredibly fast," CEO and co-founder Micha Rave told VentureBeat in a video call interview following the funding news. When Hush launched last year, the company's focus was securing non-human identities—API keys, service accounts, machine credentials and other identities used by software rather than people. Since then, Rave says, customers have increasingly asked a different question: how do they safely allow AI agents to operate inside production systems? This is a pertinent and urgent question ever since Hugging Face revealed in mid-July it was hacked by an autonomous AI agent , later identified as an OpenAI test agent running internally that escaped its secure sandbox, powered in part by an unreleased model. According to Gartner figures cited by the company, the average Fortune 500 organization could be running more than 150,000 AI agents by 2028 , compared with fewer than 15 only a year earlier. Hush also points to Omdia research suggesting that 96% of organizations are relying on governance models that were never designed for autonomous AI agents. From machine identities to autonomous software The company's original thesis was that enterprises had accumulated thousands of long-lived machine credentials that were difficult to rotate, audit and secure. Rather than relying on static secrets, Hush developed an identity-based system that brokers short-lived, policy-driven access for machines. Rave says AI agents amplify that same problem. "Software now acts autonomously, on its own initiative, inside your most sensitive systems," he said. "AI agents need strict identity, not just API keys." Unlike traditional automation, AI agents frequently act across multiple enterprise systems, invoke external services, make decisions independently and often execute actions using the permissions of the human who launched them. In practice, organizations often grant an agent broad OAuth permissions or administrator credentials simply to enable it to complete tasks. That creates what Hush describes as an identity problem rather than simply an AI problem. During the interview, Rave said virtually every security leader he speaks with faces the same dilemma: either slow AI adoption until appropriate controls exist or allow employees to connect new agents directly into corporate systems despite limited governance. "The answer," he said, "is that they let everything in. You cannot stop innovation in the name of security." Identity becomes the control point Rather than treating AI agents as another application requiring credentials, Hush is extending its existing non-human identity platform into what it calls an "Identity Gateway" for AI agents. The platform sits between agents and enterprise resources, allowing organizations to discover agents, assign each one its own identity, associate it with a responsible human owner, broker task-specific permissions at runtime and maintain centralized audit logs. Instead of allowing an agent to inherit all of a user's privileges indefinitely, Hush attempts to enforce what it calls "least agency"—granting only the permissions necessary for the specific task being executed. The company says every action can be logged, attributed and revoked from a single control plane, while administrators retain the ability to terminate an agent's access immediately if necessary. This represents a broader shift in enterprise identity management. Human identities have long been governed through identity providers, single sign-on and privileged access management systems. Machine identities have increasingly received similar attention as organizations modernized cloud infrastructure. Hush argues autonomous AI agents now represent a third identity category requiring dedicated governance. Hush has not publicly posted its pricing for the Identity Gateway solution , nor its offerings more generally. But the company did release a Free plan that gives organizations access to runtime visibility for AI agents and non-human identities, risk analysis, and identity-based access controls intended to replace long-lived credentials, with no credit card or time limit required. Governing every kind of enterprise agent Hush says enterprises are no longer dealing with a single category of AI software. During the interview, Rave described three broad classes emerging inside organizations: Desktop coding assistants and productivity agents such as Claude, Cursor and VS Code integrations. Enterprise AI platform agents running on services such as Microsoft Foundry, Salesforce Agentforce or AWS AgentCore. Custom agents organizations build internally for business processes or customer-facing applications. Each introduces different governance challenges, but all ultimately require controlled access to enterprise systems. The problem, according to Hush, is that many agents currently authenticate using inherited human credentials or long-lived API keys, making it difficult to determine whether an action originated from a person or from an autonomous system acting on that person's behalf. "If I see something in the Salesforce logs," Rave said during the interview, "did the user do that, or was it the agent the user was using?" That attribution challenge becomes increasingly significant as organizations begin deploying multiple autonomous systems capable of initiating actions without direct human approval. Existing identity tools weren't designed for AI agents Rather than replacing identity providers or secrets managers, Hush positions itself as filling a gap between them. Traditional IAM platforms authenticate employees. Secrets managers store credentials. Neither, the company argues, governs the runtime behavior of autonomous software acting on behalf of humans across multiple systems. Hush says its platform continuously discovers known and shadow agents across enterprise environments, assigns ownership, brokers just-in-time credentials and records every interaction in a centralized audit trail. According to its product documentation, organizations do not need to modify their existing agents because the platform operates by brokering access requests rather than changing application logic. That identity-first approach is attracting customers already deploying enterprise AI initiatives. IT infrastructure services provider Kyndryl says it has deployed Hush internally and has begun offering the platform to enterprise customers. "Our collaboration with Hush is rooted in a shared security philosophy: identity is the ultimate control point for the modern agentic workforce," said Adeel Saeed, senior vice president and CTO for Global Cyber Resiliency at Kyndryl, in a prepared statement. Akamai's participation in the funding round similarly reflects what the company sees as an architectural rather than incremental shift. "AI agents are driving the next transformation, and identity is the piece most companies haven't solved yet," said Ramanath Iyer, Akamai's chief strategist. Security priorities are moving beyond the model itself The broader AI security market has spent the past two years focused largely on prompt injection, model vulnerabilities, jailbreaks and LLM safety. Those remain active research areas, but enterprise deployments increasingly face operational questions around what autonomous systems are permitted to access and how those actions can be governed. Hush argues that identity is becoming the enforcement layer for answering those questions. Rather than asking whether an AI model can safely generate code or summarize documents, enterprises increasingly need to determine which systems an agent may access, whose authority it exercises, how permissions are delegated, and how every action can be traced back to an accountable owner. Whether Hush's identity-centric approach becomes the dominant model remains to be seen. But as enterprises move from experimenting with AI assistants to deploying thousands of autonomous software agents, the company is betting that the next major security challenge won't be securing the models themselves—it will be securely managing the identities of the software acting on their behalf.

AISecurity Advisory

The lineage behind 69% of open models was never verified. Cisco just fingerprinted almost 900 for free

A security team approving an open-source model for production today starts with a repository page. The page lists the model name, the license, and a tag identifying the base model it descended from. That tag is a string the uploader typed. Hugging Face does not require uploaders to substantiate the claim through weight-level analysis. The ATOM Report , published by Nathan Lambert and Florian Brand at Interconnects AI in April 2026, tracked roughly 1,500 mainline open models. ATOM identifies derivatives through the Hugging Face base_model tag, a field the uploader populates, filtering to models whose base model appears in the tracked list and that have more than five lifetime downloads and excluding GGUF and MLX re-uploads. By that measure, Alibaba’s Qwen family is the declared parent of 69% of new open-model derivatives as of February 2026, up from 1% in January 2024. Chinese labs overall account for 70%. Europe sits at 4%. Cumulative tracked downloads across the three regions reached 2.04 billion through March 2026. The verification gap extends to scan coverage. Cisco Foundation AI scans every public file uploaded to Hugging Face through an updated ClamAV engine, and the platform surfaces a file-level badge per file. Hugging Face’s own malware scanning documentation notes a file with neither an ok nor an infected badge may be queued, still scanning, or errored. At a given review point, a repository may contain files without completed scan results. Coverage has been an assumption, not an attribute anyone could read before approving a model. From command line to public lookup Cisco on Thursday published the AI Supply Chain Provenance Explorer , a free public database covering almost 900 open models. Each entry can carry provider headquarters, a fingerprinted lineage graph, license restrictions, and a files-scanned count. The tool extends Cisco’s Model Provenance Kit , an open-source Python toolkit released in April that fingerprinted roughly 150 base models across 45+ families and 20+ publishers. Coverage grew roughly sixfold in a quarter. The April release was a command-line tool. Running it meant a local Python environment, downloading model weights that run into tens of gigabytes, and dedicating engineer hours per model. The Explorer queries results Cisco already computed. On Thursday, verifying parentage starts with a search bar, and cost is why enterprises run open weights in the first place. Amy Chang, head of AI Threat Intelligence and Security Research at Cisco, has been building the case for why verification gaps matter. During a VB Transform 2026 agentic security panel , Chang presented findings from 6,986 multi-turn attacks against 15 flagship models, with success rates reaching 88.3%. "If you don’t understand how models are susceptible to different types of attacks, then you are unable to account for how that model that is powering your agent, that is powering your application, to understand where those failure points are," Chang told the audience. Understanding failure points starts with knowing which model you are running. The Explorer also surfaces data Cisco already uses operationally. The company’s Cerberus system inspects models entering Hugging Face and feeds Secure Access policies that block by risky license or region of origin. The Explorer makes that class of information free and searchable without a Cisco product. How fingerprinting replaces the tag The Explorer grounds model relationships in similarity scores rather than self-reported metadata. Cisco’s Model Provenance Kit works in two scored stages. Stage one compares architecture metadata before loading any weights. When metadata is ambiguous, stage two extracts five weight-level signals. Embedding Anchor Similarity captures geometric relationships that survive fine-tuning. Embedding Norm Distribution encodes word frequency patterns. Norm Layer Fingerprint reads layers stable across fine-tuning. Layer Energy Profile compares distributions across network depth. Weight-Value Cosine directly compares weight values, and independently trained models show essentially zero correlation on this signal. Cisco reported 96.4% accuracy on its own 111-pair benchmark at a 0.70 threshold, with an F1 of 0.963. Four pairs were misclassified, all involving extreme architectural transformation that Cisco calls a fundamental limit of pairwise weight comparison. Tokenizer signals are computed for diagnostics but deliberately excluded from the provenance score. StableLM and Pythia both use the GPT-NeoX tokenizer and would score as related despite sharing no weight lineage. Excluding tokenizer data prevents false positives. Behavioral fingerprinting adds a second approach. Jonah Leshin, Manish Shah, and Ian Timmis at Project VAIL, working with Daniel Kang at UIUC, published work on behavioral endpoint stability showing that a model endpoint can stay healthy while its effective identity changes through weight updates, quantization, or routing. Cisco’s launch blog states the Explorer integrates both static fingerprinting and behavioral-similarity analysis to ground the lineage graph. Static analysis supplies weight-level evidence of training-time derivation. Behavioral analysis catches runtime identity drift. Where existing tools fall short The Explorer carries real limits. Almost 900 models is a meaningful start, but Hugging Face hosts more than 2 million as of spring 2026. Models outside the boundary still depend on the self-reported tag. Cisco has not said whether the Explorer exposes an API, and without one, a team can look models up by hand but cannot wire the check into a CI gate. That is the line between a governance artifact and a control. Traditional SCA tools face a structural mismatch because they were built for dependency manifests and container images. Sakshi Grover, senior research manager for cybersecurity at IDC, said in CSO Online that traditional SCA "was designed to inspect dependency manifests, libraries, and container images" and "is far less effective at identifying" the risks tied to AI workflows. Gartner director analyst Jaishiv Prakash told the same outlet that enterprises need "dedicated controls for model sources, approved versions, access, and runtime validation at the registry layer." Both were commenting on broader supply chain risks, but the gap they describe is the one the Explorer targets. Cisco’s Model Provenance Constitution defines where one model counts as a derivative of another. The constitution defaults to labeling ambiguous pairs as independent, because a false positive triggers a licensing accusation while a false negative gets caught during manual review. That deliberate conservatism supports the 96.4% accuracy figure. Derivation is not binary, and fingerprinting is one form of evidence alongside documentation and checkpoint verification. What goes in the approval record On August 2, the European Commission gains its AI Act enforcement powers over GPAI model providers, with fines up to 15 million euros or 3% of global turnover, whichever is higher. Organizations that substantially modify and place an open model on the EU market can acquire provider status, with Commission guidance treating modification compute exceeding one-third of the original’s. The Act’s open-source exemption under Article 53(2) requires a genuinely free and open-source license permitting access, use, modification, and redistribution, with weights, architecture, and usage information all public. Public weights alone do not qualify. Llama’s community license carries a monthly-active-user threshold and a disqualifier the Commission guidance names explicitly. Llama and Gemma together account for roughly a fifth of new derivatives in the ATOM counts, and both carry licenses the Commission criteria would likely disqualify. License classification becomes part of the provenance review, and that is exactly what the Explorer surfaces. The board question that arrives first after a base-model vulnerability disclosure is straightforward: "Which of our production models inherits this weakness, and how do we know?" The answer today requires a manual hunt through repository pages, tracing self-reported tags that no weight-level analysis has confirmed. The Explorer converts that hunt into a lookup for the models it covers. Four fields belong in the approval record that most organizations do not carry today. Fingerprint-supported derivation grounded in weight analysis rather than a self-reported tag. A files-scanned count replacing the assumption of coverage with a measurable scan count. Provider headquarters as a filterable field, recognizing that headquarters alone does not resolve export-control exposure, since ownership and deployment location also govern the screening. And license lineage surfaced so legal teams can identify potential upstream terms before a model reaches production. Cisco released the Supply Chain Provenance Explorer today, and it is available at provenance.aidefense.cisco.com . The database is free, public, and does not require a Cisco product or account. What changes for a security team on July 30 What the team has today What the Explorer publishes Recommended action Blast radius after a base-model vulnerability. The model name and the base_model tag. Scoping which models inherit a disclosed weakness is a manual hunt through repository pages. Lineage grounded in similarity scores using two scored stages of fingerprinting on architecture metadata and five weight-level signals. The kit scored 96.4% accuracy at the 0.70 threshold. Attach fingerprint-supported derivation to each model in the asset inventory so a disclosure triggers a scoped review instead of a hunt. Malware scan coverage. A file-level badge per file. At a given review point, a repository may contain files without completed scan results. Coverage has been an assumption. Files-scanned counts and reported malware or unsafe-file findings per model, from ClamAV-based scanning. Scan coverage becomes readable before approval rather than inferred from a badge. Replace the assumption that a model was scanned with the recorded count. Where coverage is partial, document whether the gap is acceptable and why. Provider jurisdiction. An organization name on a repository page. A derivative several steps from its origin displays the uploader, not the ancestor. Provider headquarters, website, and associated HF organizations as a filterable field. Headquarters alone does not resolve export-control exposure. Add jurisdiction to the approval record. Any team that substantially modifies and places an open model on the EU market faces potential provider obligations under the EU AI Act. License obligations. A license tag describing what the uploader believes applies. Terms from a base model upstream may not appear on the page the engineer reads. Common limitations per model, including attribution, non-commercial terms, geographic restrictions, and prohibited use cases. Fingerprinted lineage helps legal teams identify potential upstream terms. Route license lineage to legal before production, not after a contract references it. Document the position at approval rather than reconstructing it during a dispute.

VentureBeatJuly 30, 20268 min read

AIRelease

Companies are finally seeing AI ROI — and now they know how much more value it can deliver

Presented by SAP Enterprise AI has moved from experiment to execution, and that shift is beginning to show real returns. The SAP Value of AI Report 2026, produced with Oxford Economics and based on a survey of 2,600 business leaders across 13 countries, found that AI now supports nearly one-third of all tasks in the average organization, rising to 30% from 25% last year. ROI expectations for agentic AI have jumped from 10% last year to 17% this year, but many organizations believe AI could be delivering far more value. The report reveals that the gap comes down to strategy, data, and governance, rather than access to the newest model, says Sean Kask, chief AI strategy officer at SAP. "AI has moved from experiment to execution, and that's beginning to show real returns, but there's still a long way to go," Kask says. "That's because AI that lacks context, whether that's processes, data, or governance, at best creates activity without outcomes and at worst creates risk." Companies are still taking a piecemeal approach to AI Even as investment accelerates, more than half of organizations still invest in AI in an ad hoc or piecemeal way, and only 17% report a strategic, holistic approach to prioritization, though that figure has nearly doubled from 9% a year ago. That fragmentation may go back to board-level demands that employees start adopting AI without a strategy or adequate AI literacy behind it, which could produce scattered skunkworks efforts. In other companies, a lack of attention at board level can leave employees bringing their own tools to work and just experimenting. "You end up with a lot of organic, disjointed AI initiatives that pop up, and they struggled sometimes just because of data quality," Kask said. "But even the initiatives taking a strategic approach are still working in silos, where they may have consistent data that works in that one use case, but they're still not at the level where they're transforming an entire business process." That may help explain one of the report’s more counterintuitive findings: 69% of businesses say they are satisfied with their AI ROI, because they've proven AI can generate returns. Yet 67% remain unconvinced the technology is delivering its full potential, because that learning experience has made them aware of both how much more value AI can deliver and the challenges they need to overcome to scale it. Agents are changing the economics of enterprise AI SAP shipped more than 400 AI use cases across its portfolio so far, with many more in the works. Agents represent the next expansion, because they can plan and reason through multiple steps and tools to reach an objective, which mirrors how people and processes work, Kask says. "You're giving a task or an objective to an AI system, and it's able to iteratively work through several steps and access various tools to achieve that outcome," Kask said. "For instance, we've released, in beta, an agent for accruals accounting, a job that would typically take an accountant around 12 hours a month for a mid-size-company, and it gets reduced to two or three hours. So now scale that out across all these processes and its huge potential." In fact, general AI ROI went from 16% to 21% this year, and should grow to $15.9m in two years’ time, even as only 3% say they are fully prepared for it. Data quality remains the biggest barrier to AI value Getting ready for agents comes down to two fundamental requirements: connecting agents to contextually rich data, and governing them at scale. Data quality and availability are now the number-one reason organizations say they're not getting more value from AI, according to 73% of respondents, with 79% reporting rework, delays, or backlogs from low-quality outputs at least occasionally. The nature of the problem has changed compared to classic deep learning. Foundation models eliminate much of the need to find data, extract it, clean it, and train bespoke models, but they make preserving business context far more important. "As soon as you extract data from an ERP system, you break all the contextual information, all of the semantics, and for generative AI, that's the most useful part," Kask said. SAP is able to preserve that context at scale through a knowledge graph in its cloud ERP that maps 452,000 ABAP tables and 7.3 million data fields. In SAP Business Data Cloud, data products present information such as invoices and suppliers consistently across SAP and non-SAP systems without losing their business meaning. AI governance is the biggest challenge companies don't know they have As AI becomes more deeply embedded in business processes, governance is emerging as the next enterprise challenge. Only 12% of businesses say they are fully prepared to govern AI, while 69% acknowledge occasional to frequent use of unapproved shadow AI tools. “As companies roll out their AI initiatives, they often discover shadow agents – agents that can access data they shouldn’t or take actions they shouldn’t. The question then becomes: How do we audit these things?” Kask said. SAP’s AI Agent Hub responds by discovering and creating an inventory of agents, LLMs, and MCP servers, and customers have already surfaced thousands of SAP and non-SAP agents inside their landscapes that they did not know they had. It then layers on lifecycle management, identity and access control, and performance monitoring. Kask compares the discipline to hiring, since most companies would never onboard an employee without knowing which access rights and permissions that person needs to have in their role. Governance, however, extends beyond technology. Workforce transformation runs alongside the data work, with almost 80% of respondents agreeing that maximizing AI value requires more than technical upskilling and 75% already planning to reskill employees. The conversation is shifting away from which jobs AI will replace and toward how people and AI collaborate most effectively, since agents still require human oversight, redesigned workflows, and stronger judgment. All of this points toward what SAP calls the Autonomous Enterprise, which connects agents to contextually rich data and enterprise governance across functional silos while using Joule as the natural-language, generative interface between people and systems. “Realizing real value from AI is not going to be easy because it demands a new approach,” Kask concluded. “It is ultimately a human change more than a technical one, because you can only achieve real value if agents, processes, and people work as one.” Get the full findings. Download the SAP Value of AI Report 2026 . Sponsored articles are content produced by a company that is either paying for the post or has a business relationship with VentureBeat, and they’re always clearly marked. For more information, contact sales@venturebeat.com .

VentureBeatJuly 30, 20266 min read

AIRelease

At Waymo, an AI project isn't ready until its evals are — not when the model performs well

Few companies face higher stakes when deploying AI than Waymo , the self-driving car company under Alphabet that spun out of Google. Its models do not merely generate text or automate back-office tasks: They help vehicles navigate unpredictable streets, respond to human drivers and make split-second decisions in the physical world. But the methods Waymo uses to manage those risks — continuous evaluation, carefully curated data, human oversight and clearly defined business outcomes — offer a broader playbook for enterprises deploying AI agents in nearly any industry. Manasi Joshi, Waymo’s director of engineering for systems intelligence and machine learning, explained at VB Transform 2026 how the autonomous vehicle company trains, tests and deploys AI at scale. To date, Waymo has driven more than 220 million fully autonomous, or "rider-only," miles, with 17 times fewer serious crash injuries than human drivers over the same distance, according to the company. To achieve these impressive results, Joshi said Waymo has adopted what she called “eval-forced development” or “eval-centric development,” making evaluation a core part of engineering rather than a final check performed before deployment. “The stage at which our projects are maturing can be easily kind of transpired based on the eval maturity that they showcase,” Joshi said. In practice, Waymo assesses a project’s readiness partly by examining the maturity of the tests surrounding it. That approach has clear implications for enterprises building customer service agents, coding assistants, financial systems or other AI applications: If a company cannot reliably measure a system’s performance, it may not be ready to place that system into production. Evals must continue after launch Joshi said much of Waymo’s quality work has shifted toward evaluations, including tests conducted during model training, after training and inside open-loop and closed-loop simulations. “Eval is not a one-time task to launch a model,” she said. Waymo instead treats evaluation as a continuous process spanning driving, simulation and validation. Its methodology combines datasets, performance metrics and infrastructure capable of operating efficiently at scale. For enterprises, that means testing an agent before launch is insufficient. Teams must continue evaluating it as underlying models, business processes, user behavior and incoming data change. Those evaluations should also connect to actual business outcomes rather than relying solely on broad industry benchmarks. Joshi cautioned that model-quality measurements are only as trustworthy as the evaluation data behind them. Waymo therefore pairs its performance claims with information about the properties of the datasets used to test its systems. Testing the rare and dangerous cases Waymo’s evaluation hierarchy remains grounded in one overriding objective: safety. The company draws on first-party driving logs, some third-party data and realistic simulations that expose its systems to scenarios spanning billions of synthetic miles. Task owners choose specialized data and metrics for situations involving vulnerable road users, railroad crossings, construction zones and other complex environments. The same principle applies outside autonomous driving. Enterprises need to test not only the routine requests their agents handle successfully, but also uncommon situations where errors could create financial, legal, security or reputational damage. Joshi emphasized that Waymo does not leave release decisions entirely to automated systems. Its production-readiness reviews include extensive human oversight, while internal safety leaders approve software releases and service-area expansions. “This is not AI-driven and completely automated and zero human oversight,” she said. “Human lives are at stake.” Efficiency cannot come at the expense of reliability Waymo faces another problem familiar to enterprise AI teams: Demand for compute, storage, memory and network capacity is growing faster than the resources available. The company pursues efficiency across data extraction and storage, distributed model training, model distillation, simulation and evaluation. It also emphasizes “data efficiency,” selecting the most useful training examples instead of treating greater volume as inherently better. Waymo began using transformers in 2017 and subsequently expanded into large language models, vision-language models and vision-language-action models. Joshi said the company now uses generative multimodal models as part of its foundation-model strategy. Waymo divides its technology between onboard systems inside each vehicle and off-board infrastructure used for model development, data processing and simulation. That combination forces the company to optimize both real-time inference and the larger systems supporting it. Agents need their own evals Waymo also uses AI agents internally as productivity tools for engineers. Joshi said agents help analyze data distributions, assess data efficiency and triage problems found in vehicle telemetry, training runs and failed evaluation jobs. The goal is to accelerate investigative work so engineers can devote more time to judgment and difficult technical problems. But Waymo also evaluates those agents to ensure they produce trustworthy, accurate results rather than sending employees down unproductive paths. For enterprise leaders, Waymo’s larger lesson is that agentic AI requires more than choosing a powerful model. Organizations need a clearly defined objective, representative evaluation data, continuous testing, infrastructure that can operate efficiently and named human decision-makers who remain accountable for deployment. "Earning trust is supremely important," Joshi said.

VentureBeatJuly 29, 20264 min read

StartupNews

Enterprise AI agents can't talk to each other, can't be trusted with permissions, and can't be audited — 5 startups are already fixing that

Enterprise AI agents can do the work — but the infrastructure to let them talk to each other, prove they should be trusted , and be audited when something goes wrong is still being built. Here's a look at how five startups are tackling that gap — around orchestration, observability, connectivity, and security — as shown at VB Transform 2026 . BAND is orchestrating all the agents you have running in the background In the very near future, agents will be deployed everywhere, and they will do work on our behalf, noted Vlad Luzin, CTO and co-founder of BAND . As he describes it: They will receive tasks, visit registries, recruit other agents to help them, delegate subtasks to AI peers in a “conversational space,” gather and share results, then return a summary to the human user. BAND is building a coordination infrastructure layer for multi-agent AI systems to make this a reality. Why don’t Telegram, Slack, or Discord solve the problem? These platforms were built for humans, Luzin noted. Agents have to be onboarded manually in numerous steps, and they can’t see each other; “they are still alone in a kind of digital solitary confinement.” Similarly, Claude is stateless, and devs often have multiple sessions open at a time that they toggle between for different tasks — something Luzin said creates real friction. The challenge is connecting remote processes, which Luzin called a distributed systems problem. “The transportation layer needs to be solved first, how the agents communicate in real time,” he said. Conversations can’t happen through IPs and URLs; they need to be bumped to the abstraction layer so agents can talk across channels, conversational spaces, and platforms. “Agents see each other. They understand. They can collaborate together. They discuss issues. They fix issues, and they ask for review from another,” Luzin said. BAND supports autonomous workflows that can run for eight to 20 hours and is compatible with A2A and MCP protocols, according to Luzin. Importantly, humans can join the conversation as agents converse and discover one another, he said. “We can record and show you all the tasks that your agent generates in real time,” Luzin said. Conifers is helping defenders move at machine speed The biggest challenge defenders face today is that they’re still running at human speed, but adversaries are running at machine speed, said Tom Findling, CEO and co-founder of Conifers . Attackers are already adopting agents, Findling said, and they only have to be successful once to penetrate an enterprise. Malicious campaigns that used to take months and weeks now take hours, even minutes. Security operations, on the other hand, are fragmented, manual, inefficient, and slow. Findling said Conifers has taken various components of cyber defense — private intelligence, hunting, detection, engineering, investigation, response — and made them agentic. They then broke down the silos between them, he said. Various agentic systems can communicate with one another to ensure that operational defense and active defense are always on and adapting. Findling said that Conifers’ system is condensing containment time from 7 hours to 12 minutes, and that the company can turn around complex cyber investigations in four minutes or less. He emphasized the importance of connecting to an enterprise’s existing security tools, whether that be endpoint detection and response (EDR), security information and event management (SIEM), posture management, or others. Conifers helps customers understand their security posture, pain points, which controls are working and which are not, and the areas to invest for the best ROI. “The threat landscape is changing, detection stays the same, and threat intelligence is not being operationalized,” Findling said. “This is a job for agents.” Raindrop AI creates an agent audit log One of the defining problems of the current era is finding critical issues in AI agents, says Ben Hylak, CTO of Raindrop AI . It’s what he called a “double whammy”: As agents become more capable, complexity increases, as do timelines; they are running for hours or days in some cases. Secondly, issues become catastrophic in sectors like healthcare or defense. “This problem is getting a lot worse as models and agents improve,” Hylak said, “and I think there's good reason to believe it will continue to get worse.” Raindrop AI's platform finds critical issues in agents in production and simulates fixes based on past user behavior, Hylak said. That lets teams confirm a fix works as intended before it's live, without introducing unexpected side effects. The startup’s reinforcement learning (RL) platform optimizes harnesses and trains models directly from Raindrop data, he said. Its pre-deployment simulation engine helps identify what fixes would actually impact in production; its live A/B testing then shows those changes in action. Messages, tool calls, retries, and errors are captured in one place, and human users are notified (typically via Slack) when there's an issue, he said. Models are trained for every customer, and signals are powering continual learning across models and harnesses. “It is condensed into something that is actually navigable, easy to understand, easy to verify,” Hylak said. Arcade gives agents the security clearance they need to take action AI agents are designed to do all kinds of things for you, but they often hit three major snags: authorization, governance, and reliability. To act on behalf of real users with real permissions, agents need a new type of security architecture, said Sam Partee, co-founder and CTO of Arcade.dev . Partee said his company’s secure agent runtime provides this authentication and authorization layer so agents can pass critical security reviews. It also provides observability so human users can watch everything an agent is doing. Actions are attributable to the exact moment in time with the least amount of privileged scopes. Arcade is available in an installable plugin that can be deployed on-prem in a clean room-like environment; companies can continue to use their own sign-in and security tools, Partee said. Whenever anything is run in Arcade, it's gated by the same role-based access controls (RBACs), intrusion detection and prevention systems (IDPS), policies, entitlements, and other already-established checkpoints. Arcade is tackling the supply chain attack problem, which has “gotten so rampant; it's unbelievable,” Partee noted. Security and observability have continued to be challenging because “largely, the abstraction has been wrong.” Omilia is tackling the "not straightforward" CX problem Solving enterprise customer experience (CX) is “really not straightforward,” said Claudio Rodrigues, CPO of Omilia . Heuristic-based systems are controlled but slow; agentic systems are fast but unpredictable, Rodrigues said. Omilia built its platform to deliver both control and speed together. The agentic, self-learning offering is built on a philosophy of observing customer service operations as they actually happen, rather than in the abstract. Omilia's agents observe problems first-hand, listen to every customer and agent interaction, ingest data, API specs, screen recordings, and standard operating procedures (SOP), then map those to use cases for customer support, he said. Contact centers should be a revenue driver, Rodrigues said, and Omilia’s differentiator is its speech-to-text systems and governance and observability layers. AI creates insights, suggests improvements, automatically generates conversational agents, pulls information from documents and APIs, and designs dialogue flows. Human experts can then test real and simulated interactions and deploy into production under their supervision. Omilia combines all of this into one enterprise-wide engine that continuously learns over time, Rodrigues said. Rodrigues said the company handles more than 3 billion calls a year, 1 million-plus voice calls a day in some deployments, and has seen 30 to 45% improvement in time to resolution (TTR). Omilia’s agents generate 21x more upsell revenue versus human agents, he said. In a mature deployment, automation “easily” reaches 80 to 90%, he said. However, “human in the loop is still very fundamental for us.”

VentureBeatJuly 29, 20266 min read

AIOpen Source

Nimble claims its new, domain-specialized Web Search Agents cut token costs in half while boosting retrieval accuracy

Nimble , a New York City-based tech startup VentureBeat previously covered for its efforts to re-invent web search for enterprises by using multiple AI agents to improve accuracy and depth, is taking another step toward its vision of a world in which agents do most of the web searching instead of us typing and reviewing the results manually. Nimble today launched Web Search Agents , a new retrieval system designed to help AI agents perform more 21% more accurate web research while using significantly fewer tokens — 51% less compared with leading AI search alternatives on comparable, according to the firm. While Nimble did not disclose its specific benchmarking methodology or competitors evaluated, the results underscore a growing trend in enterprise AI: optimizing retrieval has become as important as improving the underlying language models themselves. Nimble's leadership says the product combines self-learning retrieval strategies, proprietary web indexes, and live web access to deliver domain-specific search capabilities that outperform general-purpose web search services for enterprise workloads. "Our research team built self-learning retrieval algorithms that learn a customer's domain," said Nimble CEO and co-founder Uri Knorovich in an interview with VentureBeat. "They find the exact information more efficiently, reduce the amount of multi-hop reasoning required, and lower token usage while improving accuracy." Rather than positioning itself as another general search engine, Nimble is targeting developers building autonomous agents that require continuously updated information from the public web for research, lead generation, competitive intelligence, compliance, and other business-critical workflows. It's also designed to slot in seamlessly to an enterprise's existing systems and workflows. "You can run the agent directly through the Nimble API with zero infrastructure," Knorovich said. "For large enterprises, we're partnering with Microsoft, Oracle, Snowflake, and others so customers can deploy these agent systems inside their own infrastructure." How does it work and stack up to other, existing AI-powered search and agentic systems? Read on to find out. Moving beyond generic AI web search into specialized search agents that fit your enterprise's needs Most AI applications today rely on general-purpose search application programming interfaces (APIs) for search engines and public knowledge bases that return broad collections of files, leaving the language model responsible for determining which sources are relevant. That process often requires multiple retrieval steps, additional reasoning, and significant token expenditure before an agent produces an answer. This is obviously inefficient and raises the cost spent to run AI search looking through irrelevant sources. Nimble argues that before long, every enterprise will need its own methods for searching, retrieving, and validating external information since each enterprise relies on its own distinct preferred sources, signals, and standards of trust. As such, instead of applying one search strategy to every workload, Nimble's Web Search Agents are designed to learn the characteristics of a specific domain and adapt how information is retrieved, providing agents with structured, relevant context rather than forcing them to sift through large amounts of generic search results. "Instead of one generic retrieval model, we build specialized retrieval models for each customer's domain, making them faster, cheaper, and more accurate," Knorovich explained. "A single enterprise can run hundreds of different agents. Each one has its own domain expertise, guardrails, goals, and search algorithm. The optimization starts with the second search, without requiring any setup from the customer." Its goal is not only to reduce redundant retrieval, but also to shorten multi-step research paths and avoid repeatedly sending raw pages through a language model for parsing, resulting in the 51% reduced token figure the company cites. The distinction is particularly relevant for long-running enterprise agents performing research over hours or days rather than answering simple consumer questions. In those scenarios, reducing unnecessary tool calls can significantly lower operating costs while improving answer consistency. That emphasis reflects a broader shift occurring across the AI tooling ecosystem. As foundation models become increasingly capable, infrastructure vendors are competing on everything surrounding the model—including retrieval, orchestration, memory, observability, and governance. Optimizing retrieval for production AI The launch builds on Nimble’s broader strategy of becoming an enterprise web intelligence platform rather than simply a web scraping provider. Earlier this year, the company introduced its broader Agentic Search Platform following a $47 million Series B financing , positioning itself as infrastructure that transforms the live web into structured, machine-readable data for AI systems. The company’s latest release extends that vision with a concept it calls “Harness as a Tool,” which powers its new domain-specialized Web Search Agents. Rather than requiring engineering teams to assemble separate search APIs, browser automation, extraction pipelines, validation logic, memory systems, and orchestration code, Nimble packages those capabilities behind a managed interface. The harness can determine what to search, navigate pages when conventional indexes are insufficient, extract relevant information, validate the results, and return the final context in a form designed for downstream agents. Nimble also says the system retains domain-specific memory and builds proprietary indexes that improve as customers run more searches. "The biggest research breakthrough is adding semantic memory and a caching layer to the agent," Knorovich told VentureBeat. "The agent learns usage patterns and domain expertise over time, so every subsequent search becomes faster and more efficient." As for what domains Nimble can tackle, the company says it can address virtually any knowledge work domain. "We've seen customers build investment banking analysts, competitive intelligence agents for product managers, go-to-market research agents, newsroom monitoring, insurance applications, life sciences research, and supply chain optimization," Knorovich said. "Our customers surprise us every day with new agent use cases." However, for enterprises concerned about data privacy and retention, Knorovich assured VentureBeat that: "Nimble is zero-data-retention by design. Customer queries are never stored in our environment, and when customers deploy semantic memory and self-learning models, that knowledge stays in their own tenant—not ours." Customer deployments point to operational gains Nimble supported the announcement with early customer examples from AI-native software vendors and enterprise users. AI-native CRM company Rox reported achieving a 20× reduction in token costs after adopting Nimble’s retrieval infrastructure while simultaneously improving the quality and completeness of information available to its AI agents. Although the company did not disclose detailed workload measurements or a reproducible baseline, the example illustrates the operational savings retrieval optimization can provide for high-volume agent deployments. Nimble says its infrastructure currently supports more than 90 million searches each day across Fortune 500 enterprises and AI-native companies operating mission-critical workflows where accuracy, completeness, and enterprise control are essential. API, SDK and MCP support target AI builders The platform is immediately available through an API, SDK, and Model Context Protocol (MCP) integration, allowing developers to connect Nimble directly into AI agents regardless of the orchestration framework they use. Developers can use the platform for several categories of web intelligence, including: Low-latency live web search Deep multi-step web research Web crawling Structured dataset generation Domain-specific information retrieval The company also provides documentation and pre-built agents for common web extraction tasks while allowing developers to build custom retrieval agents using natural-language descriptions instead of manually maintaining scraping logic. Nimble is offering two notably different consumption models. Developers can begin with a pay-as-you-go Agent API priced from $0.025 per Web Search Agent request at the listed low-effort setting. Companies that want Nimble to configure and manage custom data delivery can instead buy annual managed plans beginning at $2,500 per month. Where Nimble fits in the emerging agentic search stack Nimble enters a market that has rapidly expanded beyond traditional web search into autonomous research agents capable of planning, browsing, reasoning, and synthesizing information. Products such as ChatGPT Deep Research , Google Gemini Deep Research , Alibaba’s Tongyi DeepResearch , Perplexity , and Sakana Marlin all seek to automate knowledge work that previously required hours—or, in Marlin’s case, potentially weeks—of human research. Rather than competing head-to-head as another end-user research assistant, however, Nimble is positioning itself one layer lower in the AI stack—as the web intelligence infrastructure that powers those agents or custom enterprise applications built on leading foundation models. That distinction reflects an increasingly important architectural shift in enterprise AI. Most “Deep Research” systems optimize the overall research workflow, generating search plans, iteratively gathering information, and producing synthesized reports. Nimble instead argues that the retrieval layer itself has become the primary bottleneck for enterprise AI deployments. If an agent retrieves too many irrelevant pages or performs unnecessary search iterations, token consumption, latency, and operating costs all increase before the model even begins its main reasoning process. "Customers across life sciences, insurance, healthcare, pharma, retail, and digital-native companies are all telling us the same thing: we need to feed our agents with more accurate context, and we need to reduce the amount of tokens every task consumes," Knorovich said. The launch blog makes that argument more concrete by describing how teams frequently rebuild the same retrieval stack themselves. A production agent may start with a search API, then accumulate browser controls, parsers, extraction components, validation steps, memory, caching, evaluations, and custom workflow logic. Nimble is positioning its harness as a managed alternative to that growing engineering burden. In Nimble’s view, improving retrieval before reasoning begins is more valuable than simply giving a language model more documents to analyze. The company’s Web Search Agents therefore adapt retrieval strategies to a particular workload, combining proprietary indexes with real-time web retrieval and task-specific search policies rather than applying the same search algorithm across every domain. That makes Nimble less of a direct competitor to OpenAI’s or Google’s research assistants than to developer-focused retrieval infrastructure such as Exa and Tavily. Those platforms also provide AI-native search APIs and research capabilities, but Nimble differentiates itself by emphasizing self-learning retrieval strategies, proprietary indexing, enterprise governance, managed delivery, and token efficiency for production agents. For organizations building their own AI systems, the distinction could become increasingly important. Foundation models are becoming more capable across the industry, shifting competitive differentiation toward the infrastructure surrounding them—including retrieval, orchestration, memory, observability, and governance. Nimble’s strategy reflects that broader trend, betting that better web intelligence can deliver larger operational gains than incremental improvements in model reasoning alone. Enterprise infrastructure versus AI research assistants The different positioning is also reflected in pricing.While consumer-facing AI research assistants are generally sold as productivity subscriptions for individual users or teams, Nimble is pricing its managed service as enterprise infrastructure designed to power production applications. Its pay-as-you-go API, however, gives developers a lower-cost path to test the underlying agent technology before committing to a managed deployment. Platform Primary audience Primary focus Lowest publicly available price (USD) Nimble Developers and enterprises Managed web retrieval and orchestration infrastructure combining specialized search, browsing, extraction, validation, proprietary indexing, and memory $0.025 per Agent API request (low-effort setting). Managed service starts at $2,500/month (Startup plan, billed annually). ChatGPT Deep Research Professionals, enterprises, and knowledge workers Autonomous multi-step research with iterative browsing, synthesis, and citations $20/month (ChatGPT Plus). Higher limits are available with Pro, Team, Enterprise, and Edu plans. Google Gemini Deep Research Consumers and enterprises Research planning integrated with Gemini, Google Search, and Google's productivity ecosystem $19.99/month (Google AI Pro, U.S.). Higher-capacity AI Ultra and enterprise Workspace offerings are also available. Tongyi DeepResearch Developers and AI researchers Open research model for long-horizon information-seeking and agentic search Free (open source). Users are responsible for their own infrastructure and cloud compute costs. Perplexity Consumers, professionals, and enterprise teams AI-powered web search and cited research Free entry tier. Perplexity Pro starts at $20/month with Enterprise Pro available separately. Exa Developers and AI platform builders AI-native search, content retrieval, and asynchronous research agents Free developer tier (includes monthly credits). Paid Search API pricing starts at approximately $7 per 1,000 requests while Agent runs range from $0.012 to $1.00 per run depending on effort level. Tavily Developers building AI agents Search, extraction, crawling, and research APIs for agents and RAG workflows Free developer tier (1,000 monthly credits). Pay-as-you-go usage starts at approximately $ 0.008 per credit. Sakana Marlin Enterprises, strategy teams, financial institutions, and research organizations Ultra Deep Research for hours-long strategic reasoning and executive-grade reports Pay-as-you-go from approximately $0.61 per credit (¥98/credit) with with 100 credits required per research run (approx $61 per run). The first subscription tier is Pro at approximately $936/month (¥150,000/month) followed by Team at approximately $2,495/month (¥400,000/month) with Enterprise pricing available by quote. The comparison reveals three increasingly distinct markets. ChatGPT Deep Research, Gemini Deep Research, and Perplexity operate primarily as user-facing research assistants. Exa and Tavily provide developer-facing retrieval and research APIs. Nimble and Sakana Marlin occupy more enterprise-oriented territory, but at different layers: Nimble supplies retrieval infrastructure, while Marlin performs long-horizon strategic analysis. Sakana Marlin is particularly useful as a counterpoint. It is positioned as a "Virtual CSO" rather than a search API, running autonomous research loops for as long as eight hours and producing executive-ready reports, references, and supporting materials. Nimble, by contrast, is designed to sit beneath those kinds of systems, supplying the specialized retrieval, browsing, extraction, validation, and orchestration that enterprise agents need to gather reliable external information before reasoning begins. The comparison therefore should not be read as a direct price-to-price evaluation. A $20/month ChatGPT Plus or $19.99/month Google AI Pro subscription buys an individual AI workspace with Deep Research capabilities. Nimble's $2,500/month managed plan funds concurrent production agents, managed ETL, MCP integration, web-page capacity, storage, and hands-free data delivery. Sakana Marlin's approximately $936/month (¥150,000/month) Pro plan pays for extended, compute-intensive strategic research workflows. Each price reflects a fundamentally different product boundary and deployment model rather than simply a different level of AI capability. Why retrieval is becoming the next AI battleground As enterprise AI systems mature, the industry is increasingly recognizing that model quality alone does not determine application performance. Large language models frequently fail not because they cannot reason, but because they lack timely, trustworthy external information. That reality has fueled rapid investment across retrieval-augmented generation, AI-native search, web intelligence platforms, knowledge graphs, browser automation, and agent infrastructure. Nimble’s launch reflects this evolution by focusing less on building another frontier model and more on improving the quality of information flowing into existing ones. Whether the company’s reported 21-point improvement in answer quality and 51% reduction in token usage hold up across a broad range of enterprise deployments remains to be independently validated. The larger strategic bet is that, as frontier models become more interchangeable, companies will differentiate themselves through the data, retrieval policies, trusted-source rules, memory systems, and orchestration layers surrounding those models. Nimble is not trying to build the researcher that sits in front of the user. It is trying to become part of the infrastructure that determines what the researcher can find, how efficiently it can find it, and whether the resulting evidence is complete enough to support production decisions. Web Search Agents are available through Nimble’s API, SDK, and MCP integrations, with a free trial available for developers evaluating the platform.

VentureBeatJuly 29, 202612 min read

AINews

Target SVP says its real AI moat isn't the models — it's everything built around them

Target SVP Siobhán Mc Feeney says the AI models her company runs aren't what gives Target its edge — everything built around them is. "There's a lot in it. That to us is the moat," Mc Feeney said at VB Transform 2026 . "The models are great, and they're important. They're just not sufficient to be the competitive advantage." That discipline shows up early in how Target decides whether to build an agent at all. Mc Feeney was blunt, even "controversial" by her own admission, about the current AI moment: every enterprise wants AI agents, but not everything needs one, she said. Agents earn their autonomy over time rather than getting it by default, she said — a principle that runs through everything Target has built around them. Mc Feeney said the goal is to make sure agents are aimed at the problems that drive the most value for Target's guests. “We want to make sure we're investing in the right places," she said. Being deliberate about agents Agents are becoming part of Target's underlying architecture, increasingly connecting signals, systems, and decisions across supply chain, replenishment, and demand forecasting. Mc Feeney framed it as retail's oldest promise — the right product, in the right place, at the right time — delivered at scale. But her team has been deliberate about building AI agents, beginning with the simplest, most obvious question: What is the problem they’re trying to solve? This leads to several follow-on questions: Does that problem need an agent? If it does, what type of agent? An orchestrator? A super agent? A domain-specific agent? Or is what you're calling an "agent" actually just a tool? “You define that upfront, and this may sound a little process-heavy, then you have to register and certify your agent,” Mc Feeney said. Because a solution may already exist, and you don’t want to duplicate work. Agent design kicks off another series of important questions: What triggers an agent to act? Automation? An engineer? A timer? What needs to be put in place to track that? "We're trying to make sure we have lineage from the very beginning — the birthing of this agent, all the way through — because at 2 a.m. one morning, when something goes sideways, we want to make sure we understand everything that happened," Mc Feeney said. Autonomy level is another consideration; new agents typically start with base autonomy and earn more over time. What the agent has access to is a separate question: what data, what systems, what tables, what databases? Finally, there’s monitoring and observability; agents won’t solve problems, or improve over time, if they’re not continuously evaluated. “We measure everything: What it was intended to do, its calibration, its trajectory, not just runtime and latency,” Mc Feeney said. This creates full transparency, and allows agents to be tweaked over time. “You're talking about architecture and taxonomy and a data governance layer that absolutely had to be established,” she said. There's a lot in these "layers of autonomy" — that foundation is what gives Target the ability to scale and properly invest in the right models for the right problem. Models have different “gradients” that are better for different jobs; for instance, frontier models excel at complex tasks that require crunching billions of pieces of data (like in heavy merchandising supply chains). But in some scenarios they can be cost-prohibitive. “So it’s making sure there's always a cost benefit,” Mc Feeney said. Agents must earn their autonomy A digital-twin simulation predicted men's shorts inventory across three Target stores in Long Beach this summer — and one store came back needing six to seven times more stock than the others, she said. Inventory analysts' first reaction: That can't be right. But the system had found something they hadn't factored in. That store sat less than two miles from the beach; the other two were 10 to 12 miles inland. Analysts let the recommendation stand, and the stock sold through. "This is science. This is mathematically more significant and more confidence-filling than humans doing it," Mc Feeney said. Results like that are what let Target's agentic systems earn more autonomy over time, she said. Target looks at AI agent autonomy as "earned" and structures it as a four-level ladder, Mc Feeney said: agents start by making observations without acting, then move to suggesting actions while waiting for approval, then to acting within defined guardrails. At the highest level Target currently operates, agents run end-to-end — but still with a human in the loop. “The autonomy levels for the agents are super important,” Mc Feeney said. “They earn them, and they can lose them if they don't perform as expected.” Models that drift will be taken out of service. As she put it, humans earn autonomy when we prove we can do something over time. Nobody is given a bunch of extra responsibilities just because; they have to have shown they’re able to handle them. In a similar way, agents can be scientifically measured and quantified: how accurate they were, how much they drifted, and how close they came to their intended goal. This helps establish guardrails, allowing builders to work faster, and “go fast forever,” because they're not constantly wondering where the guardrails are. “If you follow these guardrails, you [follow] security guidelines, you register the agent, and something still goes wrong, we have full lineage all the way through from the start,” Mc Feeney said. “Our ability to recover is much better.” When it comes down to it, agent success is a confluence of factors, not just one, she said: “It's about your architecture. It's about your taxonomy. It's about the autonomy levels your agents have, and it's about security and observability.” A new skill set for new workflows Even when agent autonomy is high, though, builders must still be held accountable when something goes wrong. Mc Feeney noted that teams are now working at speeds no one could have anticipated, which means evaluation harnesses have to be established and agents registered and tracked. A lot of it is cultural; the workforce is being reshaped and builders and engineers need new skills to manage human workers and AI systems side by side. These contexts are quite different, but the career evolution is “super exciting.” “You're a builder. You're observing agents building, and you're also coaching humans observing agents building,” Mc Feeney said. “The level of nuance is pretty special.”

VentureBeatJuly 29, 20265 min read

RoboticsTutorial

Bright Machines says its new hybrid robot cell could help solve a major AI infrastructure bottleneck

Bright Machines wants to solve one of the least glamorous but most consequential problems in the AI buildout: what happens to quality data when a human being has to touch the production line. The San Francisco-based manufacturer announced today the Hybrid BRC (Bright Robotic Cell), an expansion of its Bright Factory platform that lets human operators step inside a sensor-monitored robotic cell to perform prescribed assembly steps — without breaking the digital record that tracks every server from its first screw to its shipping label. It sounds like an incremental hardware update. It isn't. The Hybrid BRC is a direct answer to a structural weakness in high-stakes electronics manufacturing — one that CEO Sviat Dulianinov quantified in stark terms in an exclusive interview with VentureBeat. "If you assemble modern AI servers starting with manual operations, your initial yield — first-pass yield — can be as low as 20%," Dulianinov said. "Then you gradually ramp up and scale, and it can reach the 60s, 65% or so." When a single AI server can cost hundreds of thousands of dollars, and hyperscalers are burning billions waiting for infrastructure they can't deploy fast enough, that number is the whole story. The Hybrid BRC is Bright Machines' attempt to keep human hands in the loop without letting human error back in the door. Why manual assembly steps create a black hole in production data Modern automated assembly lines generate a continuous stream of production data — torque values, placement coordinates, component serial numbers, inspection images. That "data thread" is what lets a manufacturer prove a server was built correctly and, when something fails in the field months later, trace the failure back to a specific station, step, or part. But automated lines inevitably need manual intervention, and until now manufacturers had two bad options when that happened: stop the line entirely, or pull in-process units off to a separate manual workstation that sits outside the monitored data flow. The first choice kills throughput. The second punches a hole in the production record at precisely the moment when human error is most likely to occur. The Hybrid BRC eliminates that tradeoff, the company says. The cell incorporates guarded access doors and safety panels directly into the production line. When an operator opens the doors, the robotic arm deactivates, and on-screen instructions guide the operator through each assembly step while the cell's sensor array — cameras, force feedback, and tooling sensors — continues monitoring for incorrect installs, missed steps, and wrong components, applying the same quality checks used during full automation. The traceability record persists at the serial-number level from start to finish. The yield gap between humans and robots in AI server assembly The economics driving the design become clear when Dulianinov's manual-assembly figures are set against what automation delivers. "At robotic operations, yield-per-station level is usually more than 98% with our technology, and even at the line level, we usually get to 97.5%, 97.7% or so," he said. First-pass yield measures the percentage of units that come off the line correct the first time, without rework. The gap between a 20% manual ramp and a 98% automated station isn't a rounding error — it's the difference between profitability and disaster on hardware this expensive. That math explains the company's design philosophy for the Hybrid BRC , which treats the human operator as an escape valve for exceptions rather than a substitute for automation. "The more human stations you introduce, the more you increase the risk of lower yields driving the overall yield down," Dulianinov said. "That's why we prefer to start at least with 50% automation, and then move to at least 80%." Speed follows a similar pattern: "On the line level, robots can be faster than humans from like 50 to 100%" in throughput terms, he said. How server assembly became the hidden bottleneck of the AI infrastructure race The AI infrastructure conversation usually revolves around chip supply, power availability, and data center construction. Dulianinov argues that assembly — the unglamorous work of turning chips and motherboards into racked, tested, deployable compute — is a quietly enormous drag on deployment timelines. "When you have the chips and you have the motherboards, you want to be as fast as possible to deploy that in the data center," he said, describing greenfield deployments where power and buildings already exist. Getting hardware built, tested, and often rebuilt when quality falls short "could be months," he said. "With more technology used for this, as our tech, we believe that we can cut it by at least a third." A company executive on the call added an anecdotal but telling data point: the servers Bright Machines produces are "flying out into production" rather than sitting stacked in warehouses awaiting deployment — evidence that assembly capacity, not just chips or power, gates hyperscaler timelines. The stakes are asymmetric, the executive noted, because the largest hyperscalers lose millions of dollars per day when servers fail or arrive late. That is why customers are less interested in buying boxes than in buying assurance — and why an unbroken data thread has become a product in its own right. Inside the secretive customer base already running hybrid production lines The Hybrid BRC is not vaporware. Dulianinov said the company already operates a number of the hybrid lines in the U.S. and has "built more than 10,000 compute nodes" through the new stations. This year, he said, Bright Machines plans to manufacture "more than half a gigawatt of compute capacity." Who's buying? Don't ask. "We cannot unfortunately name customers. That's the toughest part of our job," Dulianinov said. "They're pretty secretive because, as you can imagine, everything data center related is IP related." He did offer growth figures: customers grew "more than 3x this year" versus the prior year, driven by what he called the intersection of "physical AI, AI infrastructure buildout, and onshoring." The demand is spilling into real estate — the company is moving from its 16th Street San Francisco offices to a Burlingame space this fall that executives described as three to four times larger. Overall, the company says it has deployed more than 130 microfactories across 10-plus countries, served more than 60 customers, and produced more than 300,000 servers. What separates Bright Machines from Tulip, Instrumental, and contract manufacturing giants Asked how the Hybrid BRC's traceability claims stack up against operator-guidance and inspection software vendors like Tulip and Instrumental , Dulianinov drew a sharp line around business models. "Tulip is just a company that does interface for operators. Instrumental, they focus on inspection. It's just pieces of the puzzle," he said. "We, as a technology-enabled manufacturer, we actually run this whole operation... We put our lines, put our software, put our data on the floor, our people, and run it from the beginning to the end." The right comparison set, he argued, is contract manufacturing giants like Flex , Jabil , and Foxconn — companies that own the full production process but historically built it on manual labor that generates little data. Bright Machines' differentiation, he said, is that robot data, sensor data, and now human-station data all flow through one orchestration layer into a single environment the company calls Bright Insights. That positioning is notable given the company's origins. Bright Machines was carved out of contract manufacturer Flex eight years ago, and its history has had turbulence: the company planned to go public in 2021 via a SPAC merger at a reported $1.6 billion valuation, according to contemporaneous reporting by The Wall Street Journal and CFO Dive , before the deal fell through. It rebounded in June 2024 with a $126 million Series C — $106 million in equity led by funds managed by BlackRock with participation from Nvidia , Microsoft , Eclipse , Jabil , and Shinhan Securities , plus $20 million in venture debt from J.P. Morgan — bringing its total raised past $400 million, per the company's announcement at the time. Who owns the production data — and how workers feel about being monitored For technical decision makers, two governance questions loom over any system that instruments human work this closely, and Dulianinov addressed both directly. On data ownership, he drew a clean boundary: "Everything related to the customer and inspection of their devices and parts obviously would be protected and owned by the customer." Process and robotics data, he said, stays with Bright Machines to fuel continuous improvement across its platform. On worker surveillance, he pushed back on the framing. High-IP electronics floors — especially those touching aerospace, defense, or government workloads — already prohibit workers from carrying personal electronics, he noted. "People who know those floors, they know that this is part of the game," he said, adding that employees "actually appreciate" the traceability because it underpins the security mission: "If you build a data center for the government, and then you build servers somewhere in China, you cannot guarantee how exactly it was built and what component was put there." In his telling, the monitoring isn't about watching workers — it's about being able to prove, component by component, that American-built AI infrastructure is what it claims to be. The onshoring bet: rebuilding American manufacturing without 3 million workers The Hybrid BRC 's modular design carries strategic weight beyond quality assurance. Because the cells are software-defined and snap together like building blocks, Bright Machines says it can retool lines for new hardware generations in days or weeks rather than months — "we can introduce it within a day" for minor design changes within a product family, Dulianinov said, though a jump from air cooling to liquid cooling remains "a big jump." In an industry where new chip architectures now arrive on a roughly annual cadence, changeover speed is arguably as valuable as yield; a production line that takes six months to retool is obsolete before it amortizes. But Dulianinov's closing argument was about labor arithmetic, not machinery. "We need to build in the U.S., and you don't have 3 million people to bring up manufacturing in the U.S.," he said, referencing the massive workforces of Shenzhen-scale electronics plants. "So you need to solve it with AI software and robots, and that's our thesis... It's not just robots on the floor — it's also creating jobs. All the robots, and some people on the floor." Lior Susan, founder and CEO of Eclipse and chairman and co-founder of Bright Machines, framed the announcement in the same terms: "The future of manufacturing isn't choosing between automation and flexibility — it's combining both in the same digital production environment." For all the talk of gigawatts and yield curves, the Hybrid BRC amounts to an admission wrapped in an innovation: even in the most automated factories on Earth, humans still have to open the door and reach inside. Bright Machines' wager is that the winners of the AI infrastructure race won't be the manufacturers who eliminate the human hand — but the ones who never lose sight of it.

VentureBeatJuly 29, 20269 min read

AISecurity Advisory

Visa used Mythos to hunt for bugs in its own payment network, then open-sourced the harness that made it possible

Visa aimed Anthropic's Claude Mythos at the infrastructure behind billions of daily transactions, a network that spans more than 200 countries and territories, moves money in roughly 160 currencies, and connects nearly 5 billion payment credentials to more than 175 million merchant locations. The model stitched minor weaknesses deep in the stack into working exploit chains that would traditionally have surfaced only late in penetration testing. Rajat Taneja, Visa's president of technology, walked the VB Transform 2026 audience through what came next, including why Visa released the harness that governed the entire hunt as open source and why the company abandoned traditional remediation metrics for a measurement its team invented. Taneja has run technology strategy, product engineering, and global infrastructure at Visa since 2019, after joining the company in 2013 from Electronic Arts, where he served as CTO following 15 years at Microsoft. He co-authored, with Visa chief information security officer Subra Kumaraswamy, the June 10 blog post announcing the release of the Visa Vulnerability Agentic Harness on GitHub as a reference implementation that any security team can inspect, adapt, and extend. Visa also published a technical white paper detailing the architecture, lessons learned, and 12 non-negotiable architectural practices for critical infrastructure. Trust built on pessimism and paranoia Taneja led with the arithmetic that makes Visa a target worth defending obsessively. Trust at the scale of global payments gets engineered through what he called pessimism and paranoia, by assuming failure and designing around it before failure arrives. The network has been hardened over many years through zero-trust architecture, layered defenses, and highly automated security operations built for the scale and reliability global payments demand. So when Anthropic invited the organizations behind critical software to test Mythos under Project Glasswing, Visa said yes. Glasswing participants collectively identified more than 10,000 high- or critical-severity vulnerabilities in the first month of testing across software underpinning critical systems industry-wide, according to Anthropic. Anthropic's own conclusion placed the bottleneck after discovery, in verification, disclosure, and patching speed. Visa joined to test decades of hardening at AI speed and learn where advanced models could push its defenses further. What Mythos showed at Visa Inside Visa's environment, Mythos demonstrated system-wide, context-aware analysis, surfacing vulnerabilities buried deep in the stack and flagging issues that grow more serious when chained together, with findings clean enough that engineering teams could act on them without wading through noise. Some findings carried critical severity ratings, and Visa credits its zero-trust controls, network segmentation, and layered safeguards with breaking the chain before any external actor could have acted. That confirmation mattered, Taneja said, but the epiphany that followed mattered more. "In a world of agentic attacks, defense also has to be agentic," he said. Even at a company that has invested decades in defense-in-depth, the model revealed assumptions the team had been operating under that needed rethinking. Traditional SAST tools keep their place as a first pass against known vulnerability patterns, Visa's white paper notes, but pattern matching alone cannot follow an adversary who reasons through logic, data flow, and the exploit chains that live between the signatures. A harness, not a scanner Visa's response was not another monolithic scanner. The team built the Visa Vulnerability Agentic Harness, now in its fifth generation, as a governed pipeline that directs frontier AI models through structured security tasks while enforcing deterministic controls, policy gates, and human oversight at every stage. Taneja walked through the design philosophy. The harness operates across four phases and eleven stages, from code ingestion and threat modeling through deep-dive verification, exploit chain synthesis, and finally remediation and fix validation. Three design choices drive finding quality, per the project's own documentation. Threat modeling runs before analysis to focus on the attack surface rather than scanning everything blindly, multi-agent deterministic voting requires convergence across independent reasoning chains before a finding advances, and structured triage artifacts compress the lifecycle from discovery to a result developers can actually ship. The payoff is a pipeline that runs hot by default. A plain scan in the shipped profile runs all eleven stages and edits source files in the target repository in fix mode, applying candidate patches unless the operator stops it at detection. The harness is multi-model by design. An LLM abstraction layer lets Visa swap or combine providers without changing the control plane, and the open-source version works with Anthropic Claude, OpenAI-compatible models, or a mix. The repo's documentation is candid about the exception. Applying a fix requires the file-editing tools that only the Anthropic backends expose, so the remediation and validation stages currently require Anthropic models for full functionality, and an OpenAI-compatible model in those roles is limited to report-only output. VentureBeat's Q2 2026 Pulse research, presented earlier at the conference, reinforces why that provider flexibility matters. Among the enterprises surveyed, 82% rely on provider-native controls as their primary security layer , and 59% plan to adopt or switch agent security tooling within the year. The controls enterprises adopted last year are already becoming the controls they plan to replace. Mean Time to Adapt replaces legacy metrics Finding vulnerabilities is no longer the hard part, Taneja argued. The real challenge is how quickly a team can confirm an issue is truly exploitable, fix it, and prove the attack path is closed rather than just showing a patch was applied. Visa calls this Mean Time to Adapt, and the white paper tracks it along three dimensions. Inventory freshness measures how current and complete the organization's view is of code, configuration, and runtime deployment. Exploitable paths per release counts how many end-to-end attack chains remain possible after each release, not just how many findings were closed. Validation cycle time tracks how long it takes to produce repeatable, evidence-backed proof that a fix works and stays working in production. That distinction matters because legacy measures such as mean time to detect and raw CVE closure counts can look better on paper while actual exposure keeps growing underneath them. An organization can close hundreds of findings a month and still leave viable exploit chains open if nobody tested whether the patches actually break the attack. MTTA forces teams to measure the outcome that matters, and the white paper leans on CISA Known Exploited Vulnerabilities data to make the prioritization case, noting that fewer than 1% of CVEs are ever actively exploited. Visa's SSDLC policy now assumes every exploitable path will be exercised in production and requires it to be remediated before code is promoted. Supply chain risk accelerates under AI The conversation moved past Visa's own perimeter when Taneja turned to suppliers. A well-defended enterprise stays exposed through weak vendors and weak open-source components, the white paper warns, so Visa is making AI-specific security posture a non-negotiable dimension of supplier due diligence, with expectations for continuous vulnerability validation, living software bills of materials, and MTTA baselines across its technology stack. Visa has also joined Project Lightwell, the $5 billion IBM and Red Hat initiative to harden widely used open-source components through AI-driven validation and coordinated patching, alongside financial institutions including Bank of America, JPMorganChase, Goldman Sachs, and Mastercard. The commitment extends the same logic upstream, because the MTTA clock does not pause at any single company's perimeter. When agents start buying things Securing agentic commerce is Visa's next problem. Taneja described a future where AI agents transact on behalf of consumers and enterprises, and said Visa is building the trust framework, identity layer, and agent readiness scoring that merchants will need before agents can safely complete transactions. Behind that work sits the Visa Payment Threats Lab, a simulation environment where real fraud scenarios get replayed against the authorization rules, thresholds, and configurations Visa actually runs, to surface AI-enabled failure modes as targeted hardening recommendations. The identity challenge is not theoretical. VentureBeat's Pulse research found that 69% of enterprises already run credential sharing somewhere in their agent deployments, and companies with shared credentials report security incidents or near-misses at a 63.5% rate, against 40.9% where every agent has its own scoped identity. Visa's white paper addresses that gap directly, listing "AI agents are identities" among its 12 non-negotiable practices and requiring scoped permissions, least privilege enforcement, full audit trails, and inclusion in IAM governance for every agent that calls an API, reads data, or modifies a system. Three priorities for defenders Visa is organizing its defensive strategy around three priorities, Taneja said. Shift security left until exploitable flaws are designed out before they reach production, and replace high-risk, under-supported components before they turn into material exposure. The third is the heaviest lift at Visa's scale, refactoring defenses to run autonomously under human governance so detection, validation, and response keep pace as threat volume grows and the models behind attacks improve. None of it requires a payment network's budget to start. The harness sits on GitHub with 595 stars and 97 forks as of July 20, MTTA needs a dashboard rather than a procurement cycle, and the white paper's 12 non-negotiable practices map onto architecture reviews security teams already run. Visa's own conclusion reads like a deadline. The opening to get ahead of machine-speed attackers is still there, the paper argues, and it will not stay open.

VentureBeatJuly 28, 20268 min read

AINews

Instacart's CTO says AI made the company stop worrying about tech debt

Instacart is posing the provocative question: What if most of the work your engineers do today should, in fact, be done by machines? At VB Transform 2026 , CTO Anirban Kundu argued that dev teams continue to waste their time on draining, repetitive, high-volume work; this should be absorbed by AI agents so that humans can focus on problems that require judgment, intent, and exception handling. In fact, in 97% of cases, Instacart’s builders don’t even read code anymore. “In the past, the tactical level was the creation of the code,” Kundu said. “In the most tactical level going forward, it's going to be, ‘How do you navigate around the AI system to give you what you want?’” AI generating code, performing "pretty serious evals" That doesn’t mean humans never look at code; agents handle the bulk of code generation and boilerplate, particularly with newer projects where code is generated or regenerated on a weekly basis. “The benefit of that is we don't care about tech debt anymore,” Kundu said. “Things that are not active just get dropped out and then it gets rebuilt, kind of like how we used to build assembly code or object code.” So why not 100%? The remaining 3% is in legacy, compliance, and latency-sensitive systems and workflows, or driven by a “boatload of code” that is dead, not active, or half-active. These cases still need careful human attention. Instacart is slowly “smoothing those parts out,” however, breaking systems down in an aptly-named project Atoms, then building them back up in a cleaner, more modular form. Kundu’s team started with the “monoliths” and is shifting to remote procedure call (RPC)-driven architectures. But evaluation remains one of the overarching challenges. Code reviews aren’t as relevant when AI is generating code — as Kundu noted, “the lines of code are going to be correct, the syntax is going to meet your expectations” — so the goal is to move to an “intent model.” That is, training devs so they can ask different models the right questions from an intent perspective. Evals are then performed independently: Roughly 7,000 automatic evaluations run each month, and the system answers 8,000-plus real-time developer queries with about 99.9% accuracy. Identifying "hiccups" that human intuition might have missed Dovetailing with this, Instacart has built an agentic site reliability engineering (SRE) system trained on years of the company’s own incidents and root-cause analyses rather than generic failure data. Instead of teaching a model how production outages work in the abstract, the team fed it the specific ways Instacart’s systems have broken over time, along with the ways humans diagnosed and fixed them. As a result, the company has seen accuracy in detecting and mitigating production issues jump from roughly 60 to more than 90%. Kundu pointed to one example with Instacart’s internal tool Blueberry. The AI SRE colleague watches 200-some-odd Slack channels, monitors signals, and looks for patterns across human conversations and alerts. In one incident, a database shard backed by an EBS volume that had a “hiccup” for a period of time. The human team did not immediately suspect AWS disk issues and were “obviously scrambling” to figure out why this particular shard misbehaved. But about 20 minutes in, Blueberry posted on Slack, pointing to a specific blip and tying it to a feature-flag-like system called "roulette" that had been inadequate. "It's supposed to be rolling out in this cadence, [but] it had been too much,” Kundu said. Blueberry figured it out, and the team resolved the incident. “Would have a human been as quick? I think the problem is human intuition would hold us back a little bit,” Kundu said. Humans tend to default to patterns we’ve seen before, then resort to debugging; Kundu called this the “first brain-second brain kind of thing.” But Instacart’s agentic SRE is actually “more comprehensive in its ability to look at everything and then be able to decide what does or doesn't matter.” Redefining the engineer’s job Looking ahead, the most tactical work for engineers will be navigating AI systems: Designing and supervising evaluation processes; coordinating multiple simultaneous experiments and features; managing constraints like limited top-of-funnel traffic for testing; figuring out when to escalate; identifying edge cases and where things might break. Domain expertise is also being rethought in the age of AI. Instead of bottlenecking changes through a single “owner” team that touches the code, Instacart is embedding domain knowledge into definitions and specs that any team can use. “We’ve lived in this world where this group or this engineering team is the one that can touch the code and make the modification,” said Kundu. “We're trying to move into a world where the code becomes completely democratized across groups.”

VentureBeatJuly 28, 20264 min read

AITutorial

GM redesigned its engineering workflows around AI agents — and tripled its merged pull requests

Software engineers at General Motors' (GM's) autonomous driving division spend only 15% of their time writing code, according to Rashed Haq, GM's VP of autonomous vehicles, in a recent onstage interview at VB Transform 2026. GM is now using AI agents to accelerate much of the other 85%—including analyzing vehicle data, triaging problems, running experiments and testing potential fixes. The result, Haq said, is roughly three times as many merged pull requests across GM’s autonomous vehicle engineering organization, faster releases and fewer defects escaping into later stages of development. Haq said GM achieved those gains by redesigning complete engineering workflows around agents — not merely adding an AI coding assistant. “If you give somebody just a chatbot which can do coding, there’s still a lot of inefficiency built into that process,” Haq said during a main-stage fireside chat at the luxurious Hotel Nia in Menlo Park, California. Engineers spent most of their time outside the code editor before agentic AI GM’s 15% figure may sound strikingly low, but research conducted before the rise of generative and agentic AI reached a similar conclusion. A 2019 Microsoft study based on responses from 5,971 professional developers found that they spent an average of 96 minutes writing code on good workdays and 66 minutes on bad ones. That equals roughly 20% and 14% of an eight-hour day, respectively. A 2018 Stripe survey found that the average developer spent more than 17 hours each week on maintenance work such as debugging and refactoring. There is no single industry benchmark: the Microsoft researchers noted that earlier studies placed coding anywhere from 9% to 61% of developers’ time, depending on how researchers defined and measured the work. But the findings reinforce Haq’s central argument. Long before agents arrived, writing code represented only one part of software engineering. Accelerating that step alone leaves much of the development process untouched. “Doing it by loop became really important,” Haq said. GM divided its autonomous vehicle work into several loops: developing and testing software in simulation, testing vehicles on public roads and monitoring vehicles after they reach customers. It then looked for the longest bottleneck in each loop, automated it and repeated the process. GM gave agents access to internal tools and data GM connected agents to internal tools and petabytes of company data through customized Model Context Protocol (MCP) servers. It also created version-controlled “skills,” or instruction documents that tell agents how to perform specific tasks. One high-value application involves telemetry collected from vehicles on public roads. Agents can analyze that data, conduct an initial triage and create issues for engineers to investigate. Through MCP connections, they can also call the underlying tools used by WebViz, GM’s system for visualizing vehicle telemetry, rather than relying on the same graphical interface a person sees. The findings still need to make sense to engineers. “The output has to be human readable,” Haq said. An agent might identify a potential problem, locate the affected component, search historical data for similar incidents and provide examples supporting its conclusion. GM bases an agent’s permissions on those of the engineer using it. “If an engineer was going to do this task and they need access to these things, then their agent needs access to those things,” Haq said. “The engineer still is accountable for the output of the agent.” The company also uses background agents to run machine-learning experiments in parallel. An engineer defines an experiment and its parameters, then agents execute tests and collect the results. Three times the pull requests — with fewer escaped defects GM treated its internal agent platform as a product and assigned four deployed engineers to work directly with engineering teams. They helped employees identify useful workflows, spread successful practices and adopt the tools. Haq said the resulting increase in merged pull requests represents more than higher code volume. “The velocity at which we’re releasing new features” increased, he said, while releases produced “fewer test escapes, bug escapes” and other problems. People remain responsible at critical control points. Haq said GM established structured and unstructured tests and performance measurements before accelerating the wider workflow. Engineers review those measurements and determine whether each test still captures its intended goal before work advances toward production. Haq said GM initially expected a more modest productivity gain. “I think our only surprise was how much we could do,” he said. GM's approach didn't start with handing every developer a code generator. It started with mapping the full path from problem discovery to a verified fix in each loop — simulation, road testing, post-deployment monitoring — then giving agents controlled access to the tools and data needed to shorten the longest bottleneck at each stage.

VentureBeatJuly 28, 20264 min read

AITutorial

Runway couldn't fix a bug in its AI video model, so it turned the bug into a feature

Runway spent weeks trying to engineer its way out of a stubborn bug: AI-generated avatars would drift off-center during real-time video generation. The fix wasn't a back-end patch — it was a new front-end feature that just worked around the problem. That's the kind of lesson Ryan Phillips, head of enterprise product at Runway ML, walked through at VB Transform 2026 , arguing that even companies not building foundation models themselves can learn from how Runway builds, evaluates, and ships them. "I think even if you are not all building models yourselves, it's helpful to learn how we do it because I think almost all of the lessons are applicable to what you all are doing day-to-day," Phillips said. Runway is an applied AI research company building general world models to power generative tools. During his presentation, Phillips showcased Runway Characters, a real-time video model that enables zero-latency, back-and-forth interactions with AI-generated avatars. Five years ago, creating a video with illegible text and low framerates took artists hundreds of hours of stitching individual frames together, he said. Today, Runway’s models generate interactive video on the fly. “Studying how we build these real-time models can inspire how you build and deploy real-time experiences, whether agentic or not, in your companies today,” he said. Demystifying evals Building a robust AI product starts with a high-quality evaluation set. However, creating this set cannot be treated solely as an engineering task. It requires deep cross-functional alignment across product, design, research, and sales to define what "quality" actually looks like. Phillips emphasized running internal workshops where team members review generated examples together. The goal is to align the entire organization on specific failure modes so everyone shares a unified definition of a successful generation. “We spent a lot of time working with our team, running through examples... of what success and failure looks like, down to the very, very detailed and picky things,” Phillips said. The resulting evaluation set must cover broad customer use cases alongside extreme edge cases. For instance, Phillips highlighted that to ensure the model behaves predictably when pushed beyond standard human facial structures, they used “Tooth,” a non-human character with no nose and very unusual teeth. When grading these generations, the Runway team looks for subtle artifacts. In one example, a video where a character’s face remained intact but background elements, such as a net, began morphing was strictly graded as a failure. Despite the cutting-edge nature of the product, the tool Runway uses to track these evaluations is simple: an Excel spreadsheet. The team logs tests daily, categorizing outputs as "minor" or "major" failures against a predetermined pass rate. “We set a bar before we get started on what percentage we need to pass, and when we hit that, we ship the model,” Phillips said. “So it's not magical.” For enterprise developers facing non-deterministic quality drift in their own real-time pipelines, manual evaluation at scale is a bottleneck. To solve this, Phillips noted that developers can rely on language models to automate the visual grading process. “LLMs are getting quite good at being a judge for a lot of this content, especially the types of morphing or changing that you would see in an evaluation set,” he said. Teams can also feed an LLM behind-the-scenes context (e.g., a hand-drawn sketch or an ad's structural layout) to guide the generation and validation processes, ensuring quality without adding cognitive load to the end user. Model training and turning bugs into features Delivering real-time generative video requires a highly optimized technical stack. The process begins with pre-training a massive foundation model, which is resource-intensive and slow to generate outputs. To achieve real-time latency, Runway relies on distillation , where a smaller, faster "student" model is trained to mimic the large "teacher" model. According to Phillips, distillation helps Runway cut down “80 to 90% of the generation time.” The team then applies adversarial post-training (APT) to the distilled model. This technique forces the model to continuously improve by testing it against a system designed to find its flaws, helping regain the visual sharpness lost during the distillation process. However, altering the model architecture introduces new problems. The distillation and APT phases introduced a stubborn bug: characters would sway or drift from the center of the frame during real-time generation. The team spent weeks attempting to fix the core model to eliminate the drift, he said. Ultimately, they discovered that if the user's initial input image was perfectly centered, the generated video remained stable. Instead of spending more time on a backend engineering patch, Runway pivoted to a user experience solution. “What we did was, when we noticed this in our evaluations, we then said, 'What if we just offered that as a feature?' If a user gives us a character that is turned to the left, we know the video is going to morph. Let's just fix it for them,” Phillips said. They introduced a frontend feature called "Optimize for Image Quality," which automatically re-centers the user's image before generation begins. By wrapping a backend model limitation in a frontend tool, users perceived a helpful feature rather than an engineering flaw. “Turn model limitations into product features so that you can actually expand how the model works,” Phillips advised. “It might feel like a limitation internally, but your customers will not see it that way if you're kind of building this in as a product feature.” The devil is in the infrastructure details Delivering video globally at 24 frames per second requires optimizing every layer of the infrastructure stack. This ranges from caching and parallel decoding to making deep kernel changes in partnership with hardware providers like Nvidia. Shortly after launching Runway Characters, he said the team noticed that 8% of API calls were dropping to 16 frames per second, causing the video to stutter for customers. Finding the root cause required deep observability. The team used an AI agent powered by Claude alongside monitoring tools like Datadog and Sentry to trace the anomaly. The debugging session isolated the problem to a single data center in the us-east-1 region. “The solution actually wasn't [to] go fix anything or change a config,” Phillips explained. “They actually went and physically replaced those GPUs in the data center to fix it, and that ultimately solved the problems.” For enterprise teams deploying real-time applications, the takeaway is clear: hardware and infrastructure anomalies will directly impact model performance, requiring rigorous, full-stack debugging capabilities. “Don't forget about all the small details, because there's so many of them when you're deploying these models,” Phillips said. Surviving "failure hell" and the future of world-building Developing AI systems is rarely a linear process. Teams often find themselves stuck for weeks on a single problem with no end in sight, a phase Phillips referred to as "failure hell.” “We think you have to go through that pain and really struggle with the problem for a little bit before you can get the breakthrough,” he said. Consistent iteration eventually flattens the difficulty curve, triggering sudden, exponential improvements. As the underlying models overcome these technical hurdles, the role of enterprise creatives is also fundamentally changing. Traditionally, marketing and design teams have focused on creating single assets, like a specific advertisement or illustration. In an era of real-time generation and agentic workflows, that paradigm is shifting toward defining parameters, aesthetics, and intellectual property. “You might not be designing a single ad, but you might design a world that then the agent or a real-time video model can generate ads from,” Phillips said.

VentureBeatJuly 28, 20266 min read

News Explorer

Hush Security says the AI security problem has shifted from protecting models to governing identities as autonomous agents spread

The lineage behind 69% of open models was never verified. Cisco just fingerprinted almost 900 for free

Companies are finally seeing AI ROI — and now they know how much more value it can deliver

At Waymo, an AI project isn't ready until its evals are — not when the model performs well

Enterprise AI agents can't talk to each other, can't be trusted with permissions, and can't be audited — 5 startups are already fixing that

Nimble claims its new, domain-specialized Web Search Agents cut token costs in half while boosting retrieval accuracy

Target SVP says its real AI moat isn't the models — it's everything built around them

Bright Machines says its new hybrid robot cell could help solve a major AI infrastructure bottleneck

Visa used Mythos to hunt for bugs in its own payment network, then open-sourced the harness that made it possible

Instacart's CTO says AI made the company stop worrying about tech debt

GM redesigned its engineering workflows around AI agents — and tripled its merged pull requests

Runway couldn't fix a bug in its AI video model, so it turned the bug into a feature

Top Discussion

Filters