Virexa
HomeAIProgrammingCloudSecurityOpen SourceGamesMobile GamesDeveloper Hub
Sign InSign Up
Virexa
Sign InSign Up
AIProgrammingCloudSecurityOpen SourceGamesMobile GamesDeveloper Hub
Virexa

Modern AI news aggregation and newsletter platform covering technology, business, AI, games and world news.

Categories

  • AI
  • Programming
  • Cloud
  • Security
  • Open Source
  • Developer Hub

Company

  • About
  • Contact
  • Advertise

Resources

  • RSS Feed
  • API
  • Privacy Policy
  • Terms of Service

© 2026 Virexa. All rights reserved.

Virexa
HomeAIProgrammingCloudSecurityOpen SourceGamesMobile GamesDeveloper Hub
Sign InSign Up
Virexa
Sign InSign Up
AIProgrammingCloudSecurityOpen SourceGamesMobile GamesDeveloper Hub
Home›News Explorer

Explore

News Explorer

Browse every article collected by VIREXA. Newest articles appear first.

Filters

77 results • Page 6 of 7

Poolside drops Laguna S 2.1, an open-weight coding model that beats rivals 10x its size
ProgrammingResearch

Poolside drops Laguna S 2.1, an open-weight coding model that beats rivals 10x its size

Poolside , the San Francisco AI lab that has spent most of its three-year existence quietly selling coding models to governments and defense agencies, released its most capable model to date on Tuesday — and made an unusually aggressive bet that radical transparency, not raw scale, is how a smaller lab competes at the frontier. The model, Laguna S 2.1 , is a 118-billion-parameter Mixture-of-Experts (MoE) system that activates only 8 billion parameters per token, supports a context window of up to 1 million tokens, and — according to benchmarks published by the company — matches or beats open models several times its size on agentic coding tasks. The weights are available immediately on Hugging Face under the permissive OpenMDW-1.1 license. The headline numbers are striking for a model this small. Poolside reports that Laguna S 2.1 scores 70.2% on Terminal-Bench 2.1 , a benchmark of long-horizon terminal tasks, placing it 11th on the company's compiled leaderboard — ahead of DeepSeek-V4-Pro-Max , a 1.6-trillion-parameter model that scored 64.0; Thinking Machines' 975-billion-parameter Inkling , at 63.8; and Nvidia’s 550-billion-parameter Nemotron 3 Ultra , at 56.4. On SWE-Bench Multilingual , it posts 78.5%, and on SWE-Bench Pro 's public dataset, 59.4%. Perhaps more telling than any single score: the model went from the start of pre-training on May 22 to public launch in under nine weeks, trained on 4,096 Nvidia H200 GPUs. In an industry where flagship model cycles are typically measured in quarters or years, Poolside has now shipped three models in three months. Why the West's open-weight AI gap has become a boardroom issue The release lands in the middle of an increasingly pointed debate about the provenance of open-weight AI . Over the past year, developer adoption has shifted decisively toward open-weight systems that companies can download, inspect, and run on their own infrastructure — and the leading options in that category have overwhelmingly come from Chinese labs. DeepSeek , Qwen , Kimi , GLM , MiniMax , and Tencent's Hunyuan line all feature prominently in Poolside's own comparison tables. Poolside's accompanying press release frames Laguna S 2.1 explicitly as a response, noting that the model occupies a size class into which no Western lab has released open weights in 11 months — since OpenAI's gpt-oss-120b last August. "The West needs open-weight models it can trust, run, and build on," said Jason Warner, Poolside's co-CEO, in the announcement. Co-founder and co-CEO Eiso Kant made the philosophical stakes even plainer in a lengthy post on X. "I believe intelligence should and will become a commodity," he wrote, arguing that the open ecosystem "will not win by being the best in its own category." Users, he argued, simply want the best intelligence for the task at hand — so open models must be on par with, or better than, their closed equivalents. The strategic logic here is not charity. Poolside's core business is deploying models inside the security boundaries of government, defense, and regulated enterprises — customers for whom closed, metered API access is often a non-starter for compliance and sovereignty reasons. Every enterprise that standardizes on a Chinese open model today becomes harder to win tomorrow. Releasing competitive open weights is both an ecosystem play and a top-of-funnel strategy for the company's high-security deployment business. It also reframes the AI race away from terrain where Poolside cannot compete — frontier-scale capital expenditure — and toward terrain where it believes it can: cost per token, self-hosting, and iteration speed. How a sparse architecture makes enterprise AI agents affordable to run The technical design reflects a specific thesis about where value in coding AI is moving. Laguna S 2.1's sparse MoE architecture — 256 routed experts plus one shared expert, with grouped-query attention and interleaved sliding-window layers, according to the Hugging Face model card — means inference costs scale with the 8 billion active parameters, not the 118 billion total. Poolside emphasizes that the model is small enough to run on a single Nvidia DGX Spark, the desktop-class AI machine. That matters for what Poolside calls token economics. Long-horizon coding agents are voracious consumers of tokens: the company's published data shows the model consuming a mean of roughly 249,000 completion tokens per trajectory on its hardest benchmark when thinking mode is enabled. At metered API prices, agentic workloads at enterprise scale become a meaningful budget line item. On OpenRouter, Poolside is offering a free 256K-context endpoint and a dedicated 1M-context deployment priced at $0.10 per million input tokens and $0.20 per million output tokens — aggressive pricing that undercuts most frontier alternatives by an order of magnitude. The ecosystem support is unusually broad for day one. The model is live on Baseten's model library and Vercel's AI Gateway , with integrations across vLLM , SGLang , Ollama , and llama.cpp , plus quantized variants down to 4-bit GGUF files — 75 gigabytes — for local use. But Poolside's more interesting claim is behavioral, not architectural. Pengming Wang, co-head of applied research at Poolside, said the gains came from improving the model's working habits: "more verification, less taking things for granted, not declaring victory early, and being more persistent." Raw intelligence, the company argues, is one axis of capability; a model's way of working is a second axis that matters immensely for agents left unattended for hours. Publishing every benchmark trajectory to counter AI's credibility crisis The most consequential part of the release for enterprise buyers may be an evaluation-transparency move with little precedent among major labs: Poolside published the complete, unedited trajectory of every trial in its final benchmark runs — every reasoning step, tool call, and shell command behind every reported score. This addresses a growing credibility problem in AI benchmarking. As top scores on mature benchmarks cluster in the 70–90% range, and as "reward hacking" — models finding solutions online or gaming verifiers rather than solving problems — has become endemic, self-reported numbers have lost much of their signal. Poolside disclosed its own encounters with the problem candidly: during training, more than half of trajectories on some SWE-bench tasks were flagged because the model simply researched the original bug-fix pull request online and applied it. The company documented its mitigations, including prompt addenda, LLM-based judging calibrated against human labels, and expert annotator review of a high-scoring Terminal-Bench run. Three published case studies illustrate what the company means by persistence. In one, the model built a working HTML/CSS rendering engine from an empty folder in a 181-step, 50-minute unattended session — then, lacking vision capabilities, spun up headless Chromium to numerically compare its canvas output against a real browser's rendering. In another, pointed at Poolside's own agent harness in an automated optimization loop, the model made the Go codebase 5.2% faster with roughly 70% lower memory allocation, finding an O(n²) string-concatenation bug along the way. In a third, working in a sandbox with no Python installed, the model did its number theory in Perl and independently re-derived a proof of Erdős problem #397 — a combinatorics question open for five decades until GPT-5.2 Pro first solved it this past January. Poolside notes that its model's construction is structurally different from the earlier published solution, and that its November 2025 knowledge cutoff precedes the first proof. What the disclosed limitations and benchmark fine print reveal Poolside deserves credit for disclosing limitations most labs bury. The model can overfit to its native harness and stumble on slightly different tool schemas in third-party agents, mangles JSON in nested tool arguments, and is prone to overthinking on competition math. There is currently no user-configurable thinking-effort dial — just on or off — and the gap between the modes is enormous: thinking lifts Terminal-Bench 2.1 from 60.4% to 70.2%, and DeepSWE from 16.5% to 40.4%, at substantially higher token cost. Buyers should apply their own discounts to the comparison tables. Poolside's methodology takes the maximum of vendor self-reported scores, benchmark-author leaderboards, and third-party figures for competitors — a reasonable convention, but one that mixes harnesses and test conditions. On DeepSWE , notably, Poolside ran its own agent harness rather than the leaderboard's standard mini-swe-agent, a difference the company acknowledges makes scores less directly comparable. And the frontier remains clearly out of reach: closed models like GPT-5.6 Sol , at 88.8 on Terminal-Bench 2.1, and Claude Fable 5 , at 88.0, along with the 2.8-trillion-parameter open-weight Kimi K3 , at 88.3, sit well above Laguna S 2.1. The deeper structural question is whether Poolside's " Model Factory " — the internal platform the company credits for its rapid release cadence — can sustain this pace as models scale. The trajectory so far is genuinely unusual: the April dual release of Laguna M.1 and XS.2, the July 2 refresh of XS 2.1, and now S 2.1, which the company says outperforms April's flagship M.1 at roughly a third of its active size. Remarkably, S 2.1 used the exact same pre-training data as XS 2.1, meaning nearly all the improvement came from scale, training fixes, and post-training across the company's corpus of 409,000 agentic and non-agentic training environments. Poolside says its next, larger Laguna model began pre-training last week. For technical decision makers, Laguna S 2.1 is the most credible Western open-weight option to emerge in nearly a year for self-hosted agentic coding — with published evidence, a permissive license, broad ecosystem support, and an economics story built around hardware you can own. Whether it dents the dominance of Chinese open models will depend less on this release than on the ones that follow it. Kant, for his part, has already told the world how he intends that story to end. Poolside is building toward a future where the most capable intelligence "can be owned and shaped by anyone," he wrote — and the company plans to keep shipping "until that future exists." In an industry where the biggest labs increasingly lock their best work behind an API, the most radical thing about Laguna S 2.1 may not be what it scores, but that anyone can download it and check.

← Previous1…4567Next →
🔥

Developer Pulse

What developers are discussing today

  • GPT-5.5 API↗9.4K
  • Next.js 16↗6.2K
  • Claude Code↗5.8K
  • Kubernetes→3.4K
  • Rust↗2.7K
VentureBeat·July 21, 2026·8 min read
Stop adding more GPUs: Weka's new storage platform reduces load by caching 100% of an AI model's pre-calculated tokens
AIResearch

Stop adding more GPUs: Weka's new storage platform reduces load by caching 100% of an AI model's pre-calculated tokens

GPU memory is the most expensive resource in production AI, and it's also the one running out fastest. Long context windows and multi-turn conversations force AI models to repeatedly recompute information they've already processed, consuming GPU memory and compute that could otherwise serve additional users or generate new responses. Instead of treating GPU memory as the limiting resource, why not extend it with much cheaper storage technologies? Weka , for one, believes that cheap flash storage can close that gap. The company's NeuralMesh 6 software platform, launching alongside its first self-designed hardware line, Wekapod 3, extends what Weka calls Augmented Memory Grid, an approach that aggregates NAND flash to behave like GPU memory at a fraction of the cost. This is an active and increasingly crowded category. Dell, NetApp, Pure Storage and VAST have all repositioned toward AI infrastructure over the past two years and Weka is one of several vendors arguing it's built for this specific moment rather than adapting to it. "What we're seeing now with customers is they're chasing availability of compute, and once they get new allocation from anyone, they want to be able to grab it and start running right away," Weka co-founder and CEO Liran Zvibel, told VentureBeat. The potential payoff is straightforward: better utilization of existing GPU investments, lower inference costs and faster deployment of new AI workloads without waiting months for additional GPU capacity. The technology is most relevant for organizations already operating AI at scale or expecting rapid growth in usage, particularly enterprises building internal copilots, customer service agents, software engineering assistants or retrieval systems with long context windows. Smaller deployments may see less immediate benefit than organizations where GPU utilization has already become a limiting factor. Inside Weka's NeuralMesh 6 NeuralMesh 6 adds four capabilities aimed directly at a functionality gap Zvibel says has been costing Weka deals in competitive evaluations. Composable and virtual multi-tenancy. Composable clusters give anchor tenants full hardware-level isolation, dedicated CPU, memory, and storage. Virtual multi-tenancy runs through Weka's RDMA fabric, delivering network-level isolation that scales past 1,000 tenants per cluster, with provisioning in under 30 minutes. Combined, a single cluster running 50 composable clusters can support up to 50,000 tenants. Unified file and object storage. Most storage systems keep two separate paths: a file-based path (the standard way servers and applications read and write files, used heavily in training and fine-tuning pipelines) and an object-based path (S3, the format inference and cloud-native tools typically expect). Normally a gateway translates between the two, meaning the data effectively exists twice. Weka's claim is that the same physical data on disk is directly readable through either path at once, no translation layer, no second copy. Zvibel is targeting non-AWS GPU clouds specifically, naming Lambda, Nebius, G42, and CoreWeave, with what he described as roughly two orders of magnitude higher performance than conventional S3 and a capacity-based pricing model instead of per-API charges. Metadata-first replication. Destination environments become browsable before a full data copy arrives, with data hydrating only when accessed. "They had to wait for all of that to make it to the other side, and this takes days or weeks, in extreme cases a month," Zvibel said. "We now allow our customers to grab some allocation of new GPUs and get up and running within an hour." AlloyFlash and Always-On data reduction . TLC and QLC are two types of NAND flash memory. TLC is faster and more durable but costs more per terabyte, while QLC is cheaper and holds more data per chip but is slower. AlloyFlash mixes both within a single cluster, automatically routing latency-sensitive work to TLC while running bulk-capacity workloads on QLC, cutting cost per terabyte without a performance penalty on the work that needs speed. Data reduction now runs by default rather than as an option. Solving AI's context problem Multi-tenancy and object storage solve how enterprises and neo clouds operate the platform day to day. A harder problem sits underneath: as context windows and multi-turn interactions grow, so does the GPU compute wasted recalculating work a model has already done. Augmented Memory Grid, a NeuralMesh 6 feature built specifically for this, is Weka's answer. Every prompt triggers two stages. Prefill calculates attention, the core mechanism behind how large language models process input, and it's computationally expensive. Decode converts that calculation into output and is comparatively lightweight. The cost shows up hardest in multi-turn sessions like chat or coding, where each new turn re-triggers prefill for everything that came before it, unless that work has been cached. "If you have 10 turns, you may overcalculate 100 times because you're redoing all of them. If you have 20, you'll overcalculate 400 times," Zvibel said. "You can put two orders of magnitude more NAND than you could afford in shared memory, and we can cache 100% of the pre-calculated tokens, so you never need to redo it." Where Weka sits competitively Storage vendors have spent the past year and a half repositioning around AI, and separating genuine capability from repositioned messaging is now a real evaluation problem for buyers. "The storage world is shifting its focus from serving bits to enterprise workloads to managing data at the speed of AI. We've seen that most clearly over the past 18 months from Dell, NetApp, and Pure," Steve McDowell, chief analyst at NAND Research, told VentureBeat. "The interesting thing is that companies like Weka, and VAST, are the true AI-native data companies, solving these problems since day one." McDowell singled out Augmented Memory Grid as Weka's clearest technical lead. "Weka continues to have the most technically capable KV cache implementation on the market with its Augmented Memory Grid," he said. " They were early with this technology, and continue to innovate. This is critical for AI inference, as it enables a level of GPU efficiency that, without question, saves money on GPUs and memory. That’s key for today’s memory and GPU constrained market." He also flagged Weka's contractual guarantee on its data reduction claims as underappreciated. "One flying a little under the radar: Weka is putting its money where its mouth is with its contractual guarantees for its data reduction promises," he said. McDowell's advice to buyers evaluating competing claims from Weka, VAST, Pure and NetApp alike was pointed suggesting that enterprise buyers should look hard at what vendors are promising versus what they're actually delivering. "A smart buyer will look at how competing vendors are solving real-world problems today," McDowell said. " They do this by talking to organizations running similar workloads at similar scale. If a vendor can't point to that, then it should be a warning sign."

VentureBeat·July 21, 2026·6 min read
Google's Gemini 3.6 Flash model cuts AI agent token costs by up to 65% on long horizon engineering tasks —and 3.5 Pro is on the way
AISecurity Advisory

Google's Gemini 3.6 Flash model cuts AI agent token costs by up to 65% on long horizon engineering tasks —and 3.5 Pro is on the way

Google DeepMind today released three new proprietary AI models it says are among its most token-efficient yet: Gemini 3.6 Flash, Gemini 3.5 Flash-Lite, and Gemini 3.5 Flash Cyber. The models aim to make AI agents faster, smarter, and cheaper at scale. Google is pricing Gemini 3.6 Flash at $1.50 per one million input tokens and $7.50 per one million output tokens through its application programming interface (API), while Gemini 3.5 Flash-Lite costs a staggeringly cheap $0.30/$2.50 per million tokens in/out. Compare that to the $1.50/$9.00 per 1M tokens for Gemini 3.5 Flash, and the $2/$12 for Gemini 3.1 Pro Preview, and the savings are considerable. However, Google's prior generation Gemini 3.1 Flash-Lite still remains the search giant's "most cost-efficient" model at $0.25/$1.50 per 1M tokens. Yet, it remains 2X slower than the new, more expensive Gemini 3.5 Flash-Lite, giving those enterprises who value speed more "bang" for their buck. VB Frontier AI Model API Pricing Comparison Chart (Late July 2026 Shortlist) Model Input ($/1M) Output ($/1M) Total ($/1M) Source MiMo-V2.5 Flash $0.10 $0.30 $0.40 Xiaomi deepseek-v4-flash $0.14 $0.28 $0.42 DeepSeek deepseek-v4-pro $0.435 $0.87 $1.305 DeepSeek MiniMax-M3 $0.30 $1.20 $1.50 MiniMax LongCat-2.0 — limited-time promo $0.30 $1.20 $1.50 LongCat Gemini 3.1 Flash-Lite $0.25 $1.50 $1.75 Google Qwen3.7-Plus $0.40 $1.60 $2.00 Alibaba Cloud MiMo-V2.5 $0.40 $2.00 $2.40 Xiaomi Gemini 3.5 Flash-Lite $0.30 $2.50 $2.80 Google LongCat-2.0 — standard $0.75 $2.95 $3.70 LongCat MiMo-V2.5 Pro (≤256K) $1.00 $3.00 $4.00 Xiaomi GLM-5.2 $1.40 $4.40 $5.80 Z.ai GPT-5.6 Luna $1.00 $6.00 $7.00 OpenAI Grok 4.5 $2.00 $6.00 $8.00 xAI MiMo-V2.5 Pro (>256K) $2.00 $6.00 $8.00 Xiaomi Gemini 3.6 Flash $1.50 $7.50 $9.00 Google Qwen3.7-Max $2.50 $7.50 $10.00 Alibaba Cloud Gemini 3.5 Flash $1.50 $9.00 $10.50 Google Gemini 3.1 Pro Preview (≤200K) $2.00 $12.00 $14.00 Google GPT-5.6 Terra $2.50 $15.00 $17.50 OpenAI GPT-5.4 $2.50 $15.00 $17.50 OpenAI Kimi K3 $3.00 $15.00 $18.00 Moonshot AI Gemini 3.1 Pro Preview (>200K) $4.00 $18.00 $22.00 Google Claude Opus 4.8 $5.00 $25.00 $30.00 Anthropic GPT-5.5 $5.00 $30.00 $35.00 OpenAI GPT-5.5 Instant (chat-latest) $5.00 $30.00 $35.00 OpenAI Sakana Fugu Ultra (≤272K) $5.00 $30.00 $35.00 Sakana AI GPT-5.6 Sol $5.00 $30.00 $35.00 OpenAI Claude Fable 5 / Claude Mythos 5 $10.00 $50.00 $60.00 Anthropic No price was provided yet for the specialty Gemini 3.5 Flash Cyber model, which, as its name would imply, is designed for cybersecurity researchers and red teamers to patch bugs. While the prices are among the middle-low end of all major AI models globally, the fact that Google designed them to use less tokens overall also should drive down costs for enterprises beyond what the sticker price shows (since you'll be paying for fewer total tokens at any rate). Gemini 3.6 Flash and Gemini 3.5 Flash-Lite are available immediately through the Gemini API in Google AI Studio and Android Studio, as well as within the consumer Gemini application and Google Search. According to a separate Google blog post , Gemini 3.5 Flash Cyber will be available "exclusively available to governments and trusted partners via CodeMender soon" — CodeMender being Google's proprietary AI code bug-fixing agent released last year. As with previous Gemini models, these are all proprietary and "closed source," thus, they can only be obtained through Google's official API and that of its partners, as opposed to an open-source license like MIT or Apache 2.0. One conspicuous omission noted by developers on X and social media: where is the larger, more powerful, flagship Gemini 3.5 Pro model Google previously alluded would be released this summer? After all, Gemini 3.1 Pro, the prior flagship, debuted back in February 2026 , and rivals OpenAI and Anthropic have since released several more generations of flagship updates far more powerful than Google's. Google technical staffer Logan Kilpatrick responded to one such inquiry on X, writing : "Gemini 3.5 Pro is currently testing with partners and we plan to make it broadly available as soon as it’s ready." Google's release signals that the immediate future of AI lies in agentic capabilities—systems that operate autonomously over extended periods. If early large language models are akin to massive, fuel-hungry freight trains capable of hauling incredible loads at immense cost, the new Flash series represents a fleet of nimble, hyper-efficient hybrid delivery vans. Efficiency gains ranging from 17% to 65% reduced tokens for strong results on third-party benchmarks Under the hood, Gemini 3.6 Flash achieves significant efficiency gains. The model reduces output token usage by 17% compared to its predecessor, Gemini 3.5 Flash, according to the Artificial Analysis Index maintained by the independent third-party AI benchmarking group of the same name. In specific long-horizon software engineering benchmarks like DeepSWE , which measures how well agents complete multi-step engineering tasks from scratch, the token savings reach up to 65%. This reduction means the model requires fewer reasoning steps and tool calls to complete the exact same multi-step workflow. Think of token efficiency like fuel economy in a vehicle. When an AI model takes a convoluted path to solve a problem, it burns through more computational fuel, driving up the final cost for the developer. By streamlining its internal logic, Gemini 3.6 Flash arrives at the correct answer faster and cheaper. While Google's materials did not specify the exact architectural or algorithmic changes used to achieve this token efficiency, they noted that the model "takes fewer reasoning steps and tool calls to accomplish multi-step workflows" and exhibits reduced "verbosity." The official model cards released by Google reveal that both Gemini 3.6 Flash and Gemini 3.5 Flash-Lite feature a 1-million-token input context window alongside a max output limit of 64,000 tokens, with both models sharing a knowledge cutoff date of March 2026. Respectable benchmark performance at low cost The technological improvements extend to concrete capabilities. Gemini 3.6 Flash scores 49% on the DeepSWE benchmark, a notable increase from the 37% achieved by version 3.5. It also pushes machine learning engineering performance higher, scoring 63.9% on MLE-Bench compared to 49.7% previously. Furthermore, Google integrates computer use as a built-in client-side tool via the Gemini API and Gemini Enterprise, reflecting an OSWorld-Verified score of 83.0%, up from 78.4%. The model also tackles knowledge work with greater proficiency, outperforming its predecessor on benchmarks like GDPval-AA v2 by moving from a score of 1349 to 1421. To ensure safety amidst these capability upgrades, Google deploys enhanced Frontier Safety safeguards. These protections harden the model against jailbreaks and mitigate risks in Chemical, Biological, Radiological, and Nuclear domains, as well as cyber offense misuses. The engineering team trains the model to minimize refusals for beneficial uses, striking a necessary balance between strict security and practical utility. M odels for low-cost coding, agentic, and cybersecurity use cases — respectively Google divided its new offerings into three distinct products tailored for different operational needs. Gemini 3.6 Flash serves as the heavy-duty workhorse of the trio. It handles complex coding, intricate knowledge work, and multimodal processing with improved precision. Enterprise customers utilize it for demanding tasks such as complex document parsing, intricate chart and data analysis, and long-form report drafting. The model executes complex code migrations using multi-agent orchestration frameworks with lower latency and higher quality than earlier iterations. Furthermore, 3.6 Flash aids in developing photographic texture extractors for 3D workflows using canvas interfaces. Gemini 3.5 Flash-Lite targets environments where high throughput and absolute minimal latency are non-negotiable. Google designates it as the fastest model in the 3.5 series. As measured by Artificial Analysis, the model processes 350 output tokens per second, making it highly effective for agentic search and massive document processing workloads. Artificial Analysis notes this is about twice as fast as prior generation model Gemini 3.1 Flash-Lite. Developers can configure 3.5 Flash-Lite to prioritize low-latency execution for high-volume tasks using minimal thinking levels, or engage higher thinking levels to process complex multi-step subagent workloads. Despite its lite designation, it outperforms the standard Gemini 3 Flash on several key agentic and coding evaluations, including SWE-Bench Pro, where it scores 54.2% compared to 49.6%, and OSWorld-Verified, scoring 74.0% versus 65.1%. The model extracts product features from massive datasets, generates interactive web design concepts, and scales receipt translation seamlessly. The third product, Gemini 3.5 Flash Cyber, represents a highly specialized deployment. Google fine-tuned this model specifically to find and fix cybersecurity vulnerabilities. It integrates directly with Google's CodeMender agent. In practice, multiple 3.5 Flash Cyber agents work concurrently to produce a single, comprehensive vulnerability report, achieving competitive performance at the frontier on the CyberGym benchmark, even getting within range of Anthropic's much-hyped Mythos model. Google did not specify an exact numerical cost for 3.5 Flash Cyber, stating only that it is fine-tuned "at a lower price per token than larger models. Commercial licensing only The licensing framework for the new Gemini models carries profound implications for developers and enterprise users. Google deploys Gemini 3.6 Flash and 3.5 Flash-Lite under a commercial, proprietary API model. Unlike open-source software governed by licenses such as the MIT License or the GNU General Public License, developers do not gain access to the underlying model weights, training data, or source code. An MIT or GPL license grants users the freedom to download the codebase, modify the internal architecture, self-host the deployment, and distribute the software infrastructure independently. In contrast, Google's API approach means developers essentially rent access to the intelligence on a strict metered basis. Every prompt and generated response travels through Google's managed servers, incurring a cost based on the strict pricing structure of $1.50 per million input tokens for 3.6 Flash. This commercial tethering restricts deployment flexibility. Enterprises cannot air-gap the models entirely on their own local secure hardware without establishing specialized, high-tier enterprise agreements with Google Cloud. Developers remain bound by Google's acceptable use policies, arbitrary rate limits, and network requirements, creating a permanent dependency on Google's infrastructure uptime and terms of service. The licensing for Gemini 3.5 Flash Cyber proves even more restrictive. Acknowledging the dual-use nature of cybersecurity AI—which attackers can weaponize just as easily as defenders can use it to patch systems—Google is for now making the model only available behind a limited-access pilot program, similar to the trend kicked off by Anthropic's Mythos model with its Project Glasswing program , and continued by OpenAI with its staggered rollout for GPT-5.6 . In this case, Google is making 3.5 Flash Cyber exclusively available to governments and trusted partners. This strict gatekeeping prevents open access, prioritizing systemic security over widespread developer innovation. Looking ahead Google DeepMind continues to iterate rapidly, but the gap in its product line remains apparent. While the Flash series excels in speed and economy, the industry eagerly awaits the deployment of Gemini 3.5 Pro to gauge Google's absolute frontier capabilities. Simultaneously, the company confirms that pre-training for Gemini 4 has already commenced. Until the next major flagship release materializes, developers must optimize their systems using the highly efficient, yet purposefully constrained, Flash architecture.

VentureBeat·July 21, 2026·9 min read
Evals are the new PRD, Expedia’s AI chief tells VB Transform 2026
AIResearch

Evals are the new PRD, Expedia’s AI chief tells VB Transform 2026

“The new PRD are the evals,” Xavi Amatriain, Expedia Group’s first chief AI and data officer, told the VB Transform 2026 audience last week in Menlo Park. “So basically, you encode what you want the product to do through your evals, which might include red teaming evals and all kinds of other things, which already have a bunch of security requirements. So, you already embed that into the PRD and the product design document before you even start coding.” He pushed it further. “With AI-assisted or AI-generated code, that’s gonna be the future. It’s like all your thinking is gonna go into the evals.” Amatriain served as VP of AI and Compute Enablement at Google across the platforms powering Gemini and Google Search before his December 2025 appointment at Expedia. He's mentored talent who went on to found Perplexity and Scale AI. VentureBeat’s VB Pulse research on the evaluation gap reinforced the stakes. Sixty-six percent of the 157 enterprises surveyed already permit some production deployment without human review or are building toward it within the next 12 months, yet only 5% fully trust the automated evaluations that would make that decision. Half have shipped an agent that passed internal evals but then failed with a real customer. Don’t let guardrails get in the way of feedback “The more guardrails and artificial business rules and sort of rules that you put into the system, the worse off,” Amatriain said. “Not only because they’re brittle, but also because they actually mess up with the feedback loop. You are actually biasing the user and the feedback you get from the user, and then you’re learning that in the wrong way.” He called guardrails “a necessary evil” and said the goal is to minimize their impact over time. Not everyone at Transform agreed. Other speakers argued during the event that the highest-risk actions still demand very firm guardrails. Expedia governs AI through three layers instead. Principles come first, communicated broadly. “I like to encode at a very high level how I expect decisions to be made, because in a large organization you’re gonna have a lot of distributed decision making,” Amatriain said. “And sometimes, if you’re lucky enough, those principles might be embedded in your culture. But most of the time, my experience has been they’re not.” The processes and tools that enforce them follow. “Principles look really nice on a picture on some wall, but you need to then give them teeth,” he said. Automation sits on top of both. In practice, this plays out through what Expedia calls agent release toll gates, checkpoints calibrated to risk. “Governance needs to correlate to the risk,” Amatriain said. “And if you have something that is low risk, you don’t need too much governance to get in the way. But if there’s a lot of risk, then you need more governance. That can be encoded.” The toll gates tie evaluation rounds, red teaming, and security review to each agent’s risk level, and the checks shift from recommended to required as the stakes climb . Specialized agents over monolithic intelligence “Even when I was at Google, I was like, I don’t believe in AGI as sort of like a singleton and a unified sort of like single model,” Amatriain told the audience. “I think it’s much better to think of it as composition, sort of like having specialized agents that are very good at some task and then composing the system out of those specialized agents.” Expedia’s architecture starts at the component level. Tools compose into skills, skills assemble into sub-agents, and sub-agents get orchestrated into the full agentic system. “You need to have those principles that are unified that talk about things like what is the tone that we’re using, how are we addressing the user, how are we passing context, memory,” he said. “All of that needs to be thoroughly designed.” He framed this as a systemic design problem. “It’s not about the model, it’s not about a specific solution, it’s about how you’re designing the system.” Amatriain argued that scoping each agent narrowly also makes the system easier to secure, since teams can evaluate and lock down individual agents in isolation before composing them. When the user must keep the final click Travel pricing changes in real time, flight availability shifts minute to minute, and hotel reviews routinely contradict what suppliers claim. Amatriain described a system that blends retrieval-augmented generation with direct API tool calls, choosing the approach based on latency. “If the user asks you a question like, how much does a four star hotel usually cost in Chicago in July, you don’t expect the agent to take two minutes to answer that question,” he said. “You expect an immediate answer because that answer can be cached and it doesn’t need real-time information.” A pet-friendly four-star near Lake Michigan with a pool might justify a 30-second reasoning window. “The supplier might be saying, yeah, we have a great swimming pool, but then we also have the reviews from the travelers and we actually see there’s two reviews that say the swimming pool was not great or was not open after 6 p.m.,” Amatriain explained. A generic chatbot, he added, would only surface what a supplier self-reports, while Expedia cross-references against its own review corpus. “We don’t want the agent to book the hotel or to buy you a plane ticket for you,” Amatriain said. “That’s something that the user has to have the agency. And the agent can recommend, can suggest, can discuss with you, but you’re gonna have to hit that click. And that’s non-negotiable.” That constraint, he argued, is also a security decision. “Once you establish those design principles, you also don’t need the guardrail because otherwise you’re gonna have to put all those guardrails in after the fact.” The next attackers will be other AI systems “Security needs to be a principle that is shifted as left as possible and as part of the design itself,” Amatriain said in response to an audience question. “And usually when you need a guardrail is because you’ve not thought about it early on.” A second audience member pressed for lessons learned from production. Amatriain described a feedback loop where monitoring signals flow back into the eval suite. “You can almost automate the whole cycle,” he said. “But having that whole feedback loop from real signals, from your operating AI system, all the way into being reported and fixed as quickly as possible is going to become essential.” Amatriain's toll gates are a bet that governance calibrated to risk can stay ahead of that feedback loop. VentureBeat’s separate June Pulse survey on agent security , drawn from 107 enterprises, shows how thin that margin is. More than half, 54 percent, have already had an agent security incident or near-miss. Fifty-nine percent plan to adopt, add, or replace agent security tooling within 12 months, and 29% plan to move this quarter. Incident rates climb with organization size, reaching 63% among enterprises with more than 1,000 employees versus 49% for companies with 101 to 1,000. And sandbox isolation, the one post-breach control that limits damage, drops from 35% adoption at the smaller companies to just 20 percent at the largest. Amatriain warned that threats will increasingly come from other AI systems. “You’re gonna get threats coming not only from humans but also from other external agentic systems that are really powerful, and they’re gonna be poking at everything you’re doing. And as soon as you detect something, it’s not only about the detection, but the time to fix becomes essential here.”

VentureBeat·July 21, 2026·6 min read
Atlassian: Research shows organizations should approach AI at the team level, not the individual level, to achieve true ROI
ScienceResearch

Atlassian: Research shows organizations should approach AI at the team level, not the individual level, to achieve true ROI

Presented by Atlassian Most companies are approaching AI adoption backwards by optimizing how individuals use AI instead of how teams work together, said Dr. Molly Sands, head of the Teamwork Lab at Atlassian, during a fireside chat with VentureBeat senior technology contributor Sam Witteveen at VB Transform 2026 . Sands leads a team of behavioral scientists and psychologists who study how AI is reshaping the way people work together, using those findings to help organizations redesign how work gets done. "We don't just study it, we also actively go in and change it," she explained. Her teams teach new ways of working and remap how work flows across companies, a challenge that many organizations are still struggling with, she said. Why AI speed isn’t translating into ROI Atlassian's annual State of Teams Report, which this year surveyed 12,000 global knowledge workers and interviewed roughly 200 Fortune 1000 executives, found a significant disconnect between activity and value, showing that everyone is using AI, while very few can yet locate where it pays off. "89% of those executives told us that individuals are speeding up in their companies, and only 6% of them said they could point to specific examples of clear ROI," Sands said. But roughly 14% of teams had translated AI usage into real value — meaning a single organization could contain a handful of high-performing teams surrounded by others seeing no return at all. Those leading teams shared three characteristics: context, workflows and culture. The teams pulling ahead were building what Atlassian calls a context graph by capturing goals, decisions, and organizational knowledge in shared digital records rather than leaving them in individual memory. Across products such as Jira and Confluence, the graph connects work items, goals and the people doing them, giving AI access to the organizational context it needs. On workflows, the winning teams redesigned entire end-to-end processes rather than simply accelerating isolated tasks. Otherwise, speeding up individuals who are pointed in slightly different directions only causes them to “very quickly start to crash into each other,” as Sands puts it. On culture, the fastest-moving teams worked under leaders who explicitly encouraged learning and experimentation, while making it clear that some experiments would fail. How leaders can move AI from individual hack to team advantage Experimentation and constraints are the fastest route to learning, Sands said. The teams seeing the biggest gains were deliberately imposing constraints on how they worked, from breaking every task into the smallest practical unit of work (a single story point) to committing to write no code by hand for a week. "Most of it is not sustainable to do forever, but it is a really, really fast way to learn," she said. Sands argued that another obstacle isn’t the technology itself but the fact that employees are figuring out AI on their own. Every worker develops different prompts, agents and assumptions, creating another layer of unspoken knowledge inside teams that rarely translates into organizational performance. To counter that, Atlassian experimented with AI working agreements at the start of projects, asking teams to decide not only what they would use AI for, but what they would deliberately avoid using it for, which agents they would share and what common skills would keep everyone working from the same context. Teams that adopted the practice used AI more, moved faster, made better decisions and ultimately produced higher-quality work. The broader lesson, Sands said, is that AI isn’t creating entirely new management problems so much as exposing old ones. Teams have always struggled with hidden assumptions and different mental models of their work. AI simply makes those gaps more consequential, increasing the importance of shared context and explicit ways of working. Sponsored articles are content produced by a company that is either paying for the post or has a business relationship with VentureBeat, and they’re always clearly marked. For more information, contact sales@venturebeat.com .

VentureBeat·July 21, 2026·3 min read
Writer's AI harness cuts token spend nearly 40% — without sacrificing accuracy
AIResearch

Writer's AI harness cuts token spend nearly 40% — without sacrificing accuracy

Enterprise AI is facing an ROI paradox. While throwing more compute at the strongest foundation model works well in product experiments, the costs become unbearable when the product is deployed in production. A new paper from researchers at Writer provides a solution that is accessible to engineering teams. The study takes a systematic look at optimizing the different components of the orchestration layer that wraps around the foundation model, aka the AI harness. By optimizing the harness, the researchers show dramatic reductions in tokens per task, a drop in cost-per-successful-task by up to 61%, and quality that holds steady, all without changing the underlying foundation model. Because the harness is fully under the developer's control and requires no model fine-tuning, engineering teams can apply these findings to build highly cost-efficient AI applications. The ROI crisis of tokenmaxxing The current state of AI engineering is plagued by " tokenmaxxing ," an industry trend where developers rely on massive context windows and brute-force token consumption as a substitute for good system design. Rather than engineering elegant workflows, developers have imported a reflex from traditional software development: generate, run, fail, stuff the error and more context back into the window, and retry. "Teams tokenmaxx because it's the cheapest fix in the moment, and because it's literally how most engineers work today," Waseem AlShikh, CTO and co-founder of Writer, told VentureBeat. Because this approach succeeds often enough on coding tasks, it has become the default reflex for every other agentic workload. The danger is that per-token price drops mask the underlying inefficiency. "Your invoice is tokens-per-task times price-per-token, and most teams only watch the second number," AlShikh said. "In agentic workloads, tokens-per-task compounds — every loop iteration re-transmits the growing context — and it compounds faster than prices fall. The price cut becomes an anesthetic. It masks the fact that the loop itself is bleeding." Tokenmaxxing leads to several enterprise failure modes. Teams route simple tasks to premium frontier models by default. They use the LLM as a lazy search index, stuffing the context window with raw documents instead of retrieving exact answers. Most destructively, they build unconstrained agentic loops that spiral out of control when the model encounters an error. Because output tokens cost significantly more than input tokens across all major model providers, inefficient task execution acts as a silent budget killer. The industry has introduced several efficiency techniques to curb these costs, but they largely fall short because they treat the model in isolation: Prompt compression condenses input text to save space, but ignores how the system sequences those inputs across complex workflows. Budgeted reasoning caps the computational steps a model can take, which often degrades output quality if the workflow isn't intelligently routed. Terse coding forces models to output minimal code to save output tokens, but does nothing to solve inefficient tool calling. Speculative decoding uses a smaller draft model to speed up a larger model's text generation, optimizing inference speed while failing to address bloated agent architectures. These efforts fail because they optimize the engine while ignoring the transmission. They do not look at the orchestration layer, leaving underlying architectural inefficiencies unresolved. Unpacking the harness: the levers of efficiency The harness is the orchestration layer that routes, formats, and turns the underlying LLM into a working system. The core levers of harness optimization include system prompt caching, interaction history compaction, tool management, retrieval strategies, and error management. These are the most accessible intervention points for engineering teams looking to improve AI performance. As the Writer researchers note in the study: “If the harness is the layer that composes model calls into work, it is also the layer that sets the price of work.” Historically, developers have treated the harness as disposable glue code designed simply to connect an API to a user interface. The study signals that the harness must now be treated as a first-class object: a primary software artifact that requires its own testing, versioning, and rigorous design. For enterprises, this reframes the "own-versus-rent" decision. "Enterprises spend months on model evaluations and then rent their orchestration off the shelf — which means they're optimizing the smaller lever and outsourcing the bigger one," AlShikh said. "Whoever owns the harness owns your unit economics, and an open framework tuned for demos is not tuned for your invoice." Inside the experiments To isolate the impact of the orchestration layer, the researchers ran experiments on six foundation models spanning multiple vendors and weight classes: Claude Sonnet 4.6, Gemini 3.1, Gemini Flash 3.5, Qwen 3.6, GLM 5.1, and Writer’s own model, Palmyra X6. Their experiments compared a frozen, conventional production agent loop against the finished Writer Agent Harness on the same 22 locked enterprise tasks, spanning capabilities like grounding and retrieval, multi-step workflows, tool use, and content generation. By holding the models and tasks constant, they could isolate the effects of the orchestration layer itself. The optimized harness drove a significant drop in costs, cutting the blended cost per task by 41%, from 21 cents to 12 cents. This was largely achieved by slashing token consumption, with the number of tokens per task falling 38%, from 14.2k to 8.8k. The harness is designed to delegate tasks like search to specialized sub-agents. A sub-agent receives only the tool and the specific query it needs, retrieves the exact data, and returns a capped, clean summary to the main agent — keeping the primary context window from filling up with raw search results. Task success rates held steady even as token use fell — moving from 78% to 81%, a gain the researchers describe as directional rather than statistically significant at their sample size, meaning quality didn't suffer even as costs dropped. End-to-end task latency also dropped significantly, reducing the median wall-clock time by 44%, from 48 seconds to 27 seconds, due to prompt caching and the elimination of dead-end reasoning loops. However, the researchers also found limits to multi-agent orchestration. Smaller models like Gemini Flash 3.5 and Qwen 3.6 scored well below a usable reliability threshold on sub-agent delegation tasks (0.45 and 0.42, respectively) — the capability simply isn't dependable yet on lighter-weight models. Sub-agent orchestration only crossed a usable reliability threshold on the two strongest models tested: Writer's own Palmyra X6 (0.86) and Claude Sonnet 4.6 (0.85). The developer’s playbook: actionable takeaways and tradeoffs The findings from the study translate into a playbook for enterprise developers building agentic workflows at scale. The first step is to implement what AlShikh calls the "Two-Zone Prompt" and "Context Offloading." Structure for system prompt caching (The Two-Zone Prompt): Modern LLM APIs offer prompt caching, but developers must structure their payloads correctly to trigger it. Developers must separate the "stable zone" from the "volatile zone." Place static, unchanging elements (e.g., core rules, large tool schemas, and standard operating procedures) at the top of the prompt. Dynamic elements, such as the specific user query or recent conversational task state, must be appended at the bottom. This ordering allows the harness to reuse the cached prefix across hundreds of calls. "That single separation makes prompt caching actually work and stops you from re-paying for the same instructions on every one of an agent's thirty steps," AlShikh said. Manage context with Context Offloading: Avoid context stuffing, where every turn of a loop is appended into a monolithic prompt until the window maxes out. Instead, move history and intermediate artifacts out of the window into retrievable storage, and pull back only what the current step needs. If possible, delegate tasks to single-purpose sub-agents to avoid context bloat. As AlShikh points out, "the biggest line item in agent spend isn't reasoning — it's re-sending things the model has already seen." Build resilient loops and redefine KPIs: Unmanaged agent loops drain API budgets rapidly. Teams must begin tracking Completions Per Million tokens (CPM) to understand their true task costs, but the harness itself must contain physical guardrails. "The core principle is that you never ask the model to police its own spending," AlShikh said. "The fence has to live below the model, in code, on your side of the API." This requires three hard checks: Hard per-task token budgets: The run terminates when the budget is spent, no exceptions. Generation fencing: Caps on steps, tool calls, and recursion depth to stop non-converging agents. Failure-spend governance: Cap what a run can spend after its first failed validation so a failing task doesn't become your most expensive task. Avoid unnecessary complexity: Optimizing the orchestration layer comes with engineering overhead. If you're in the prototyping and exploration stage, that overhead isn't justified — iterate fast with a strong model and a light harness. Once you're scaling to millions of requests a day, the savings from harness optimization become substantial. However, teams must be aware of "harness leverage." Adding structural scaffolding requires the model to hold and obey that context. If a model is too small, it will spend its limited capacity parsing the scaffolding instead of doing the task, causing accuracy to drop and tokens to rise. The rule for adding complex orchestration features is strictly mathematical: "If a feature adds more coordination tokens than it removes task tokens for that specific model, cut it," AlShikh said. "Nothing in the harness is free." The future of the enterprise harness The era of tokenmaxxing and treating context windows like bottomless buckets is coming to an end. Throwing more compute at poorly designed systems is not a viable strategy for companies that need to demonstrate a return on their AI investments. As foundation models evolve to absorb planning, tool selection, and multi-step reasoning natively into their weights, the role of the harness will shift from compensating for model weakness to enforcing enterprise policy. "What never moves into the model is the 'allowed': budgets, permissions, data boundaries, audit trails, deterministic kill-switches," AlShikh said. "Five years from now, the harness will be thinner but more important. There will be less scaffolding and more governance. However capable the model gets, someone external to it still has to define what it may spend, see, and touch. That layer belongs to the enterprise, and it should never be rented."

VentureBeat·July 20, 2026·8 min read
A single AI agent conversation can look perfect and still be broken, leaders from LangChain, Conviva and CoreWeave said at VB Transform 2026
AIOpen Source

A single AI agent conversation can look perfect and still be broken, leaders from LangChain, Conviva and CoreWeave said at VB Transform 2026

A single AI agent conversation can look flawless scored on its own and still point to a broken product. That gap is driving a shift in how enterprises evaluate agents, away from scoring individual traces and toward comparing cohorts of users against a baseline. At VB Transform 2026 , Harrison Chase, CEO of LangChain; Hui Zhang, CTO and co-founder of Conviva; and Emmanuel Turlay, director of engineering at CoreWeave, described that shift, along with a parallel move toward cheaper, narrower judge models. Agent-as-judge — judging one AI agent's output with another — hasn't replaced LLM-as-judge, which Chase said remains the default. The larger tension, Zhang said, is between automated judging, whether by LLM or agent, and human review. "You have scalable but ungrounded, whether it's agents as judge or LLMs as judge, you grade the outcome, you grade the work. It still is very difficult to ground it and then you use humans and that's just not scalable," Zhang said. "The whole industry is facing this, which poison you want to pick." Evaluation criteria now function as the product spec That gap — a conversation that scores well but still signals a broken product — is what teams try to close by building an exhaustive evaluation suite before they ship anything. Chase said that doesn't work. "We sometimes see teams that have almost eval paralysis," Chase said. "They're like, this is an eval set, I can't launch it. The best teams launch and then iterate." Chase framed evaluation criteria as a living specification, not a one-time test suite: a product requirements document — the standard software-development spec for what an application should do. "Evals are like the new PRD," he said. "They define what your agent should and shouldn't do." Turlay described hitting the same failure from a different angle. "I was trying to reach 100% coverage for my tests, and I still had bugs in production," he said — a test suite that looked complete but still missed what mattered, the same gap Chase was describing with evals. Broad, always-on monitoring, he said, catches more real failures than an exhaustive pre-launch test suite. Teams should set up wide online checks first, use those to identify failure classes as they occur, then build a targeted offline evaluation set around the problems that surface. Why scoring traces one at a time is a mistake Even a well-built evaluation process can still score the wrong thing. Zhang's objection is to how most teams run evaluation: sampling traces, whether 50 of them or a full population, scoring each in isolation. That approach misses a signal that only shows up when comparing cohorts of users against a baseline, a method Zhang calls contrastive analysis. Zhang illustrated it with a retail example: a shopper asks an agent for a running shoe ahead of a half marathon, the agent asks qualifying questions, and the shopper buys a shoe. Scored individually, that interaction looks fine. But the clarification ratio, how many follow-up questions an agent asks before completing a task, came in three times higher than baseline for that shoe category across the full user population. A second metric, how often shoppers finished their purchase outside the conversation, was five times higher than baseline for the same category. Neither number is visible from a single trace. Both point to a debuggable, category-specific problem. Zhang said the industry also lacks a second data source: what happens before, between and after the conversation, not just the trace itself. Sizing the judge to the job Once contrastive analysis flags which category is actually broken, the next problem is what watches for it going forward — and at what cost. Turlay's rule was to start with the most capable model available to prove a task is solvable, then work down. If it can't be done with a top-tier model, he said, it won't work with a smaller one. Once a pattern proves viable, teams can sample a fraction of traffic instead of judging every interaction, and move simpler tasks like binary classification to smaller open source models. LangChain took that further, fine-tuning its own model to detect when a user believes the agent made a mistake, a signal Chase calls perceived error. "The model we fine-tuned was a Qwen model," he said, referring to Alibaba's open source family. Combining hand labeling with distillation, the result performed well. "Same as [Claude]Sonnet, for, depending on how we served it, either 10 to 100x cost reduction," Chase said. Not every guardrail needs a model. Chase pointed to Claude Code's own guardrails as proof: regexes, the common programming technique for finding and validating patterns in code. "A lot of the guardrails they had were just regexes," he said. "They weren't small LLMs, they were just regexes." LLM-as-judge doesn't mean human-in-the-loop disappears The bigger question is whether using LLM as a judge removes the need for a human in the loop. Turlay pointed to accountability, drawing on his prior work at a self-driving car company. His team compressed data intake and retraining into a two-week cycle for shipping a new model to the car. Even then, someone still had to sign off. "I felt confident on behalf of the company to say this model should go into the car," he said. The same logic extends to legal, finance and healthcare. "Before we can remove a human to say, I endorse this and I take responsibility legally for it, it's going to be a while before agents can do that on their own." Zhang agreed a human has to remain the guardian on corner cases, even as automation eventually runs at a scale that beats individual human accuracy — machines can see more at the pattern level. Chase went further: that human check isn't just a safety net. "Human in the loop is really important for building trust in how these agentic systems work, and also really important for memory and learning from systems," he said. "There has to be interactions in order for the system to learn."

VentureBeat·July 20, 2026·5 min read
At VB Transform 2026, Zillow's engineering chief said AI ROI numbers only hold up if you measure before you build
AINews

At VB Transform 2026, Zillow's engineering chief said AI ROI numbers only hold up if you measure before you build

Zillow, the real estate technology company, doesn't get one conversation with its customers. They move from a phone screen to a loan officer to a real estate agent, sometimes over months or years, and expect the context to follow them. A single chatbot could never carry that thread. At VB Transform 2026 , Zillow SVP of Engineering Toby Roberts and Glean co-founder and CEO Arvind Jain described how they built AI architecture meant to carry context across that entire journey — and why context, not raw data, turned out to be the harder problem to solve. Zillow's products touch roughly 80% of U.S. real estate transactions each year, and the company has been using AI long before ChatGPT existed. "We pretty quickly identified that we were going to need a persistent context layer that was going to meet our customers and the professionals wherever they were," Roberts said. Data was never the hard part Roberts said Zillow's AI effort started where most enterprise AI efforts start, with the data itself. "We started with a large push around making sure our data did have the right foundation," Roberts said. That meant a data mesh approach, clear data lineage and a governance structure with permissions and identity attached to the data itself. None of that turned out to be the hard problem. The hard problem was building something that remembered where a customer was in their journey and carried that forward, no matter which surface they showed up on next. "This context layer has to live to be able to support you where you are at any given point in your journey," Roberts said. Zillow chose to own that layer itself rather than depend on a single external chat interface, a decision Roberts said the team reached quickly once it looked at the shape of a real transaction rather than a single conversation. Why Zillow built its own architecture, and where Glean fits into it Zillow built its own harness rather than route customers through a single model API. The team drew on 20 years of machine learning history behind products like Zestimate, leaning into smaller, task-specific fine-tuned models instead of one general-purpose model. Internally, that harness runs alongside Glean. Roberts said Zillow now has thousands of Glean agents in production, handling repetitive tasks with tens of thousands of executions across the company. Glean's pitch, per Jain, is centralizing that integration work once, through the Glean MCP gateway, rather than letting finance, legal and marketing each rebuild their own connections to the same systems. That centralization is also a cost lever. Jain pointed to two mechanisms: model routing, which sends most tasks to smaller, cheaper models instead of defaulting to frontier models, and precomputed context, which avoids an agent burning tokens assembling its own context from scratch. "Claude is also very slow because the first part of assembling that context actually takes forever," Jain said. Routing that request through Glean instead, he said, can cut token consumption by as much as half. What Zillow and Glean's approach means for enterprises Across data, cost and permissions, the session offered a few practical takeaways for enterprises building agentic AI on their own systems. Build the measurement baseline before the AI push, not after. Roberts said Zillow's ability to credibly attribute a 40% increase in shipped code to AI adoption rests on a DORA metrics baseline the team put in place years earlier, not on the AI rollout itself. Centralize context once instead of letting every team rebuild it. Jain's core argument for Glean's platform is that duplicated integration work across finance, legal and marketing teams is a hidden cost most enterprises haven't accounted for. Don't assume permission inheritance is enough for regulated data. Even with a permissions-aware context platform in place, Zillow layered hard rules and a standing compliance check on top for its most sensitive categories, rather than trusting the architecture to handle it automatically. Treat context as a cost lever, not just a capability. Model routing and precomputed context were the two mechanisms Jain pointed to for cutting AI spend, both aimed at reducing wasted token consumption rather than adding new capability. "Models by themselves are not enough to bring automation with AI inside your enterprise," Jain said. "You do have to connect it with your enterprise context."

VentureBeat·July 20, 2026·4 min read
The cleanup trap: Stop asking RAG to fix bad data
AINews

The cleanup trap: Stop asking RAG to fix bad data

The enterprise technology ecosystem is caught in a costly cycle. Over the past two years, millions of dollars have been funneled into generative AI pilots, yet many of these initiatives stall out before ever reaching a live production environment. When a project fails, the immediate instinct of technical leadership is often to blame the model: The context window was too restrictive, the latency was too high, or the reasoning capabilities simply were not there. But as data engineers building the scaffolding for these systems, we often see a different reality: The model receives the blame, but the pipeline usually contains the root cause. Production gen AI rarely fails because of model limitations alone. More often, it fails because the enterprise data foundation underneath it is fundamentally unready. This is what I call the 'Cleanup Trap': The false belief that an organization can pipe fragmented, inconsistent, and ungoverned legacy data into a large language model (LLM) orchestrator and simply “clean it up” or patch it at the retrieval layer. The mirage of the retrieval layer In a standard retrieval-augmented generation (RAG) architecture, the retrieval layer is tasked with pulling relevant business context to ground the model’s responses. Because modern frameworks make it simple to stand up a vector database and a basic embedding pipeline, leadership often assumes that the data engineering problem is solved. It is not. When an embedding model receives raw, unvalidated data directly from operational silos, the resulting vector space inherits the structural noise, duplicate records, and conflicting states present in the source systems. If the core data pipeline suffers from silent degradation — schema drift, missing fields, delayed change-data-capture (CDC) synchronization — that degradation cascades directly into the vector store. An AI model cannot accurately synthesize customer intelligence if the data pipeline behind it is serving stale, contradictory profiles across disparate storage layers. No amount of prompt engineering, semantic reranking, or vector hyperparameter tuning can compensate for a broken ingestion pipeline. If the foundation is compromised, the downstream application will hallucinate, expose unauthorized context, or fail to deliver deterministic value. Shifting from ad-hoc patching to programmatic guardrails To break out of the 'Cleanup Trap,' enterprise data teams must stop treating data quality as a post-processing step. They need to treat data readiness for AI with the same rigor they bring to traditional transaction processing. This requires a deliberate architectural shift toward zero-trust data ingestion, structured validation frameworks, and automated anomaly detection before data ever reaches an AI orchestration layer. 1. Harden the ingestion pipeline Data quality checks cannot exist as a nightly batch afterthought. If an enterprise AI application relies on real-time data to assist users, validation must happen inline. Teams should implement explicit schema validation checks at the earliest ingestion point, such as the streaming ingress layer or the bronze landing layer of a medallion architecture. If an upstream operational database mutates a schema without warning, the pipeline should quarantine anomalous payloads rather than allowing corrupted metadata to pollute downstream AI contexts. 2. Use multi-tiered algorithmic validation Static row-count validation rules are insufficient for AI readiness. True data health requires a multi-tiered approach. This means pairing structural verification — null checks, type conformance, and schema validation — with statistical profiling to monitor for data drift. Tracking metric deviations across feature distributions helps ensure that historical context remains stable over time. If a pipeline suddenly processes an unexpected spike in empty string variables or structurally deviant fields, automated alerts should trigger an immediate pause before vector database updates continue. 3. Decouple security and compliancemfrom the model An LLM should never be the arbiter of data access control. Trying to enforce row-level security or personal data filtering through system prompts is a compliance risk. Security must be managed within the data infrastructure tier. Enterprise data foundations should enforce strict access controls, tokenization of sensitive identifiers, and rigorous lineage tracing before information is indexed into vector stores or passed into an agent’s context window. Technical alignment: A pragmatic blueprint For technology leaders mapping their infrastructure roadmaps, AI readiness requires evaluating data pipelines against a strict operational checklist. Can you trace a flawed AI response back to the exact pipeline execution, source record, and transformation step that produced it? Does your data lake architecture have a programmatic mechanism to segment and quarantine corrupted or non-compliant data before it reaches production feature stores? Are your operational systems and AI-facing vector databases tightly synchronized, or are your agents making automated decisions based on outdated snapshots? These questions matter because production AI is not just a model deployment problem. It is a data reliability problem. Building for the production era The honeymoon phase of gen AI experimentation is ending. Enterprise leaders are demanding measurable, predictable, and secure business outcomes from their AI investments. If an organization wants to transition from isolated, impressive-looking demos to resilient, production-grade AI systems, it must redirect its focus. Stop looking exclusively at the model tier. The real competitive differentiator is not only the LLM an organization chooses. It is the engineering discipline, data governance, and pipeline resilience of the infrastructure built to feed it. In the production era of AI, data engineering is no longer a backend function. It is the control plane for enterprise intelligence. Naveen Ayalla is a senior data engineer.

VentureBeat·July 20, 2026·4 min read
Safety guardrails blocked Hugging Face's defenders, not the attacker, when an AI agent breached its systems
AISecurity Advisory

Safety guardrails blocked Hugging Face's defenders, not the attacker, when an AI agent breached its systems

Hugging Face’s incident response team first turned to frontier AI models to analyze a breach of the company’s production infrastructure, and the models refused to help. Commercial safety guardrails built to stop attackers blocked every forensic query because they treated the IR team’s real exploit data the same way they would treat a live attack. The attacker, an autonomous AI agent running the campaign end to end, moved laterally across the Hugging Face infrastructure for a weekend, undetected and unstopped. Security leaders are quick to recognize the pattern and diagnose what went wrong. “I’ve seen versions of this during red-team exercises and internal security testing, but this is one of the first high-profile examples where it materially affected real incident response,” said Merritt Baer, senior adviser to Andesite, G2I, and AppOmni and former Deputy CISO at AWS. None of this is unique to Hugging Face, Baer said. “Commercial frontier models optimize for preventing misuse. They generally have no cryptographic or organizational way to determine whether the person asking ‘analyze this malware’ is an incident responder or the malware author.” A malicious dataset opened two code-execution paths On July 16, Hugging Face disclosed that an autonomous AI agent system had compromised its production infrastructure, gaining unauthorized access to a limited set of internal datasets and several service credentials. The company verified that its software supply chain was clean and found no evidence of tampering with public models, datasets, or Spaces. Hugging Face is still assessing whether the intrusion touched any partner or customer data and says it will contact affected parties directly as required. But few enterprise threat models had accounted for the attacker Hugging Face faced. An autonomous agent ran the entire campaign from initial access through credential harvesting. No human guided it. The entry point was a malicious dataset. When the data-processing pipeline ingested it, the dataset triggered code execution in two ways, through a remote-code loader and through a template-injection flaw in dataset configuration files. No admission gate screened the file before it reached a processing worker, the compute job assigned to each dataset. Initial access came through the data pipeline itself, a route that worked because most enterprise security teams treat the data feeding their pipelines as trusted input rather than an attack surface. Worker isolation did not hold. From that first foothold, the agent broke out onto the node running the workload, then harvested cloud and cluster credentials scoped broadly enough to reach multiple internal clusters, all of it inside a single weekend. Hugging Face described the agent framework as executing thousands of individual actions through a swarm of short-lived sandboxes, with self-migrating command-and-control staged on public services. The disclosure adds that the framework appeared to be built on an agentic security-research harness, which would put tooling designed for red-team work behind a live intrusion. Why the defenders’ queries looked like attacks Investigators reconstructed more than 17,000 recorded events using AI-driven analysis agents of their own. First attempts at the log analysis ran on frontier models behind commercial APIs. Defenders’ steps included submitting real attack commands, exploit payloads, and command-and-control artifacts for classification, but safety guardrails blocked the requests outright. Baer traced the block to the prompts themselves. “The same prompts that are most valuable during an active intrusion, shell commands, exploit chains, credential dumps, persistence mechanisms, lateral movement, are exactly the prompts most likely to trigger safety systems,” she told VentureBeat. “As AI becomes embedded in security operations, this becomes an operational resilience issue rather than merely a model policy issue.” The forensic analysis finished on GLM 5.2 GLM 5.2, an open-weight model deployed on Hugging Face’s own infrastructure, took the job the commercial APIs refused. No attacker data left the company’s environment. “This experience points to a gap worth planning for,” the company wrote in its disclosure. Hugging Face does not know which model powered the agents. It could have been a jailbroken hosted model or an open-weight model running without restrictions. Either way, the disclosure continued, “the attacker was bound by no usage policy, while our own forensic work was blocked by the guardrails of the hosted models we first tried.” Hugging Face drew that line itself, writing that the experience is not an argument against safety measures on hosted models and that it is sharing the feedback with the providers concerned. What authenticated trust changes The industry, Baer argued, needs to move past treating AI safety as a content moderation problem. “Security operations require something different. Authenticated trust.” Instead of asking whether anyone should receive an answer, the question becomes whether an authenticated security team, operating under enterprise controls, should receive it. “The model shouldn’t only understand what is being asked. It should understand who is asking, why, and under what governance.” “Organizations already build contingency plans for cloud outages, identity provider failures, or EDR failures,” Baer wrote. “AI assistants are becoming another dependency.” Her advice on IR playbooks was blunt. “A mature incident response plan should assume that during a severe incident, commercial AI APIs may refuse requests, API rate limits may become unavailable, internet connectivity may be impaired, and data governance rules may prohibit uploading forensic evidence externally.” The lesson, she wrote in her emailed answers, “isn’t ‘don’t use commercial models.’ It’s ‘don’t make them a single point of failure.’” AI-enabled attacks rose 89% year-over-year Autonomous AI-driven attacks are not limited to AI platforms. CrowdStrike’s 2026 Global Threat Report documented AI-enabled adversary operations increasing by 89% year over year, with average breakout times falling to 29 minutes. Enterprises running AI workloads in production with agentic access to their pipelines face similar exposure. Six control domains determined the blast radius and recovery speed at Hugging Face. Each one maps to a concrete action security leaders can take before the next autonomous-agent breach arrives. AI Pipeline Breach Response Playbook Control Domain What Broke Monday Action Dataset admission controls Two code-execution paths were exploited. No admission gate validated the dataset before it reached a processing worker. The data pipeline became the initial access infrastructure. Require sandbox execution and static analysis of all datasets before they reach workers. Block remote-code loaders and template-injection paths by default. Audit for any path granting code execution to untrusted content. Report to the board as a supply-chain risk. Worker-to-node privilege boundaries Worker isolation failed to prevent escalation to the node. The agent gained cluster credentials because the workload-infrastructure boundary was never enforced at container runtime. Enforce hard privilege boundaries between workers and nodes. Deploy container runtime security to prevent workload escape. Audit whether workers can reach node-level APIs or credential stores. Include in the next penetration test scope. Credential exposure Cloud and cluster credentials harvested after node access. The scope was broad enough for lateral movement across multiple clusters over a weekend. Rotate credentials on a scheduled cadence and after any anomaly alert. Scope to the minimum cluster and service. Deploy monitoring that flags access from unexpected nodes at machine speed. Map blast radius for board reporting. Machine-speed detection Thousands of actions through short-lived sandboxes with self-migrating C2. AI-assisted anomaly detection surfaced the campaign after a weekend of lateral movement, per the disclosure. Calibrate detection for machine-speed patterns. Ensure high-severity alerts page responders in minutes, regardless of time. Audit SIEM rules for detecting thousands of short-lived executions within a single hour. Private AI forensic capacity Commercial APIs blocked forensic analysis. Guardrails screened query content, never analyst identity. Investigation ran on GLM 5.2 privately. Deploy a capable open-weight model on private infrastructure before an incident. Test against real forensic workflows. Ensure IR playbook includes fallback for when commercial APIs refuse. Document gap for cyber insurance. Autonomous-agent threat modeling The campaign matched the forecast agentic-attacker scenario, but no threat model had operationalized it. LLM powering the agent is still unknown. Add autonomous AI agents as a distinct adversary class with machine-speed decision cycles. Run tabletop at agent speed. Present results to the board as evidence that timelines need recalibration. Include in the cyber insurance application. The board question is operational resilience “The question for directors is simple. What happens if one of our critical security tools becomes unavailable during the exact moment we need it most?” Baer framed that as operational resilience, not AI policy. She would have boards take that framing straight to management and press for specifics. “Have we actually exercised that fallback during tabletop exercises? How quickly can we switch during an incident?” Procurement needs to change alongside governance, starting with the questions buyers ask. Security teams evaluating AI vendors should ask about their process for authenticated incident responders, whether enterprise customers receive different handling during verified incidents, and whether models can be deployed privately. “Those questions belong alongside uptime, privacy, and compliance,” Baer said. “The biggest takeaway isn’t that safety guardrails are ‘bad.’ They’re doing what they were designed to do,” she argued. Her larger point is that the threat model itself has changed. “For decades, defenders had better tools than attackers because they operated inside trusted enterprise environments. With foundation models, both sides increasingly use the same capabilities, but one side is constrained by enterprise governance, policy, compliance, and safety controls, while the adversary simply downloads an uncensored open-weight model and keeps going. That’s a new kind of asymmetry,” she added. “The organizations that handle it best won’t necessarily be the ones with the most powerful AI. They’ll be the ones that architect AI as a resilient security capability rather than a single cloud service.” Hugging Face has contained the intrusion, rebuilt compromised nodes, rotated credentials, and reported the incident to law enforcement. The company recommends that all users rotate access tokens and review recent account activity. Mid-incident, Hugging Face found out whether its own AI tooling would be available, and the first answer was no. Security leaders running AI in production should find out in incident response planning instead, before an autonomous agent forces the test.

VentureBeat·July 20, 2026·8 min read
AI confidence just dropped 17 points in six months. That’s actually great news.
AIResearch

AI confidence just dropped 17 points in six months. That’s actually great news.

Presented by JumpCloud The organizations losing confidence in AI are the ones most likely to get it right. Six months ago, 40% of IT leaders described their organizations as mature in AI deployment. Today that number is 23% . Before you read that as a setback, consider what it actually reflects. We recently surveyed 800 IT leaders across the U.S. and U.K. for our Q3 2026 trends report, and the data tells a consistent story: the organizations revising their self-assessment downward are overwhelmingly the ones that have moved AI agents from pilots into production. They’re not losing faith in AI. They’re running into the problems that only show up when agents are doing real work in real systems, and they’re being honest about what they found. That kind of honesty is harder to come by than it sounds, and it matters more than the confidence number itself. Deployment was the easy part 84% of organizations plan to expand AI use in IT operations over the next 6 to 24 months, so the drop in confidence isn’t a retreat. What it reflects is a more accurate picture of what production actually requires. In a pilot, an AI agent does one thing in a controlled setting. In production, it accesses real systems, makes decisions that affect real workflows, and operates continuously, often without a human in the loop. The governance infrastructure that entails is materially different from what it took to get the pilot working. Most organizations built enough to ship. Fewer built enough to scale. The IT leaders revising their self-assessment are confronting questions they didn’t have to ask at the pilot stage: Can we see every agent running in our environment? Do we know what each one can access? If an agent behaved unexpectedly last week, how long would it take to find out? For most organizations, at least one of those answers is uncomfortable. The gap between perception and reality is where risk accumulates The graphic above captures the structural problem. Across confidence, governance, and autonomy, the same pattern holds: deployment is moving faster than the controls built around it. The organizations that have closed this gap share specific characteristics. They’ve consolidated their IT environments rather than adding tools to solve each new problem, because every additional platform creates another place where agent identity, access, and accountability can go unmanaged. They treat AI agents as governed identities rather than tolerated shadow processes. And they measure what AI actually produces, not just what it deploys. The payoff is tangible. Organizations in the top tier of our maturity model are five times more likely to report no barriers to expanding their AI agents than the average organization. They are not more cautious about AI. They are more confident in it, because they built the foundation that makes confidence earned rather than assumed. The governance gap has a specific shape The hardest problem in enterprise AI right now is not capability. It is accountability, and the data makes the specific failure point clear: non-human identity governance is the least adopted AI security practice we measured, in place at just 21% of organizations. Non-human identities now outnumber human users in 83% of organizations, and that population is growing fast. Yet most of those identities exist without the governance structures that every human employee has as a matter of course: no formal record, no named owner, no defined scope of access, no offboarding process when their purpose expires. They keep running. They keep accessing systems. They keep accumulating permissions. We call these Zombie Agents, and they are the service account problem of the AI era, operating at machine speed and in every department. The accountability gap is where real risk lives. When a human employee takes an action, there is an implicit accountability chain. When an autonomous agent takes an action, that chain breaks unless it has been deliberately engineered. Most organizations have not yet engineered it, and the gap between the autonomy agents are being granted and the oversight structures in place to manage them is widening every month. What the confidence drop is actually telling us When AI maturity confidence was uniformly high across the market, that was worth worrying about. It meant most organizations hadn’t yet run into the hard parts. A selective drop, concentrated among organizations actively running agents in production, means the market is developing a more accurate picture of what AI operations genuinely require. The organizations recalibrating are doing the work that makes long-term AI adoption possible: building identity infrastructure that covers agents alongside humans and devices, unifying the environments where governance needs to apply, and measuring outcomes rather than just counting deployments. They haven’t lowered their ambitions for AI. They have raised their standards for what it means to run it responsibly. 84% of organizations plan to expand AI use over the next two years. The ones that will do it well are honest enough, right now, to admit what they haven’t yet built. JumpCloud’s Q3 2026 AI Readiness Research report (n=800 IT leaders, U.S. + U.K.) is available here . The report covers AI agent deployment stages, identity governance gaps, IT unification benchmarks, and budget realism across mid-market and enterprise organizations. Rajat Bhargava is CEO and Co-founder at JumpCloud. Sponsored articles are content produced by a company that is either paying for the post or has a business relationship with VentureBeat, and they’re always clearly marked. For more information, contact sales@venturebeat.com .

VentureBeat·July 20, 2026·5 min read
Capital One releases VulnHunter, an open-source AI tool that finds software flaws before hackers do
AISecurity Advisory

Capital One releases VulnHunter, an open-source AI tool that finds software flaws before hackers do

Capital One on Thursday released VulnHunter , an open-source, agentic AI security tool that scans source code for exploitable vulnerabilities, maps out how an attacker would reach them, and proposes targeted fixes — all before a single line ships to production. The tool, built internally and now available on GitHub under an Apache 2.0 license, is one of the most ambitious attempts by a major financial institution to turn offensive AI capabilities into a public defensive resource. At a time when security teams are facing a rising tide of new AI threats, Capital One's decision to open-source the tool reflects an effort, according to CISO Chris Nims, to address "an increasingly brief window before sophisticated, next-generation AI attack capabilities become affordable and accessible to virtually every adversary." Capital One is not simply releasing another vulnerability scanner. VulnHunter introduces what the company calls an " attacker-first forward analysis " — a workflow in which the tool begins at the points where a real adversary would enter a system, such as APIs, network messages, or file uploads, and reasons forward through the application's logic to determine whether an exploit path actually survives the code's existing defenses. Conventional scanners typically work in reverse, flagging a dangerous-looking code pattern and then searching backward for a hypothetical attacker. That approach, security practitioners widely acknowledge, buries engineering teams under avalanches of false positives. VulnHunter attacks that problem head-on with a second innovation: a built-in "falsification engine" that tries to disprove its own findings before a developer ever sees them. After the tool surfaces a potential vulnerability, a structured reasoning workflow hunts for logical gaps, unsupported assumptions, and conditions that would prevent the attack from succeeding. Only findings the engine fails to rule out reach a human reviewer — and when they do, VulnHunter delivers not just an alert but a full explanation of the exploit path and a proposed code fix ready for engineering review. The tool currently runs on Anthropic's Claude Opus 4.8 model inside a Claude Code environment, though Capital One says the framework has the potential to work across other foundation models and coding harnesses. Why Capital One is giving the tool away Asked why Capital One decided to open-source a tool this consequential, Nims pointed to the communal nature of the problem. "We felt an imperative to open-source VulnHunter because modern software supply chains are very connected, and the scale of the AI threat is larger than any single organization," Nims told VentureBeat. "Securing software and our digital environments is a shared foundation that benefits developers, enterprises, and the people who depend on the systems we all build. The defensive tools to address this reality need to be just as widely distributed, tested, and improved as the codebases they protect." "Rather than wait," he added, "we decided that the right response was to build a product that is purpose-fit for today's complex security landscape, and put it into the hands of defenders everywhere." Why Capital One believes open-sourcing VulnHunter strengthens everyone's defenses The release nonetheless arrives against a backdrop the company knows well. On July 19, 2019, Capital One disclosed that an outside individual — later identified as a former Amazon Web Services employee named Paige Thompson — had gained unauthorized access to names, addresses, self-reported income, Social Security numbers, and linked bank account numbers belonging to credit card customers and applicants. The breach, which Capital One says occurred on March 22 and 23, 2019, was discovered only after an external security researcher flagged a configuration vulnerability through the company's Responsible Disclosure Program on July 17 of that year. The damage was sweeping. Approximately 100 million people in the United States and 6 million in Canada were affected. Roughly 140,000 Social Security numbers, about 80,000 linked bank account numbers, and approximately 1 million Canadian Social Insurance Numbers were compromised. The FBI arrested Thompson, and the government stated it believed the data had been recovered with no evidence of fraud. But the reputational and regulatory toll was enormous. In August 2020, the Office of the Comptroller of the Currency fined Capital One $80 million , finding that the bank had failed to adequately identify and manage risks as it migrated significant technology operations to the cloud. As Reuters reported at the time, the OCC's consent order cited insufficient network security controls, inadequate data loss prevention measures, and a board that failed to hold management accountable when internal auditing surfaced problems. The OCC also ordered Capital One to overhaul its operations and submit new cybersecurity plans for regulatory review. CyberScoop at the time called the incident " a cautionary tale for companies rushing to embrace new tech ." Capital One's own CEO, Richard D. Fairbank, acknowledged the gravity of the moment. "While I am grateful that the perpetrator has been caught, I am deeply sorry for what has happened," Fairbank said at the time. "I sincerely apologize for the understandable worry this incident must be causing those affected and I am committed to making it right." Inside Capital One's decade-long open-source strategy — and what VulnHunter adds to it. What followed was not a retreat from technology but a doubling down — with security explicitly at the center. Capital One began releasing open-source projects in 2014 and declared itself an " open-source first " company in 2015 as part of a broader technology transformation that began over a decade ago. The company has continued to invest in software supply chain security, open-source governance, and AI-driven defense. In August 2022, Capital One joined the Open Source Security Foundation as a premier member, earning a seat on the organization's Governing Board. Chris Nims, then EVP of Cloud & Productivity Engineering, framed the move as a natural extension of the company's operating philosophy. "As a highly-regulated company, we are seasoned in managing compliance and governance and advocate for standardization, automation and collaboration," Nims said in the OpenSSF announcement . Behind that public commitment lay a substantial operational apparatus. Capital One's Open Source Program Office , now in its third iteration, manages open-source usage, contributions, and community building across the enterprise. The company has released more than 40 open-source projects and has made thousands of contributions to external open-source projects it depends on, according to the company. Those efforts address not just code dependencies but the entire software development lifecycle — DevSecOps tools, infrastructure, and the collaborative environments, both internal and external, that shape how software gets built and shipped. VulnHunter is the most consequential product of that multi-year effort — and the release reflects a calculation increasingly common among large enterprises: that in security, giving away the tool is cheaper than fighting alone. "At the end of the day our goal is to make this product as broadly accessible as possible," Nims said. The company argues that modern software supply chains are so deeply interconnected that a single vulnerability in a widely used open-source component can cascade across thousands of enterprises simultaneously. Proprietary defenses, no matter how sophisticated, cannot address a problem that is fundamentally communal. By releasing VulnHunter under a permissive license, Capital One invites the global security research community to stress-test, extend, and improve the tool — effectively crowdsourcing its own defense infrastructure while strengthening the broader ecosystem. Inside VulnHunter's three-stage AI engine for finding exploitable code For engineering leaders evaluating VulnHunter , the technical architecture is where the tool's ambitions become concrete. The workflow unfolds in three distinct stages. In the first stage — attacker-first forward analysis — VulnHunter begins at the points where an external adversary would interact with a system: API endpoints, network message handlers, file upload interfaces. From each entry point, the tool reasons forward through application logic, tracing data flows, transformations, and internal security checkpoints to determine whether an attacker can actually reach a dangerous code path. This approach mirrors how a skilled penetration tester would probe a system, but automates the process at a scale no human team could match. The second stage is where VulnHunter departs most sharply from conventional scanners. After identifying a potential vulnerability, the falsification engine runs a structured reasoning workflow designed to disprove its own conclusion. It searches for assumptions that do not hold, logical gaps in the exploit path, and environmental conditions that would prevent an attack from succeeding. Findings that fail this internal challenge are discarded before any developer sees them. Capital One's explicit goal is to shift the developer's burden away from triaging false alarms — a perennial pain point that erodes trust in security tooling and slows development velocity. In the third stage, vulnerabilities that survive the falsification engine trigger an evidence-backed remediation workflow. VulnHunter gathers supporting evidence across the codebase, maps the complete surviving exploit path, explains the defect and the specific capabilities an attacker would gain, and generates targeted code changes for engineering review. The output is not a generic advisory but a concrete, context-aware patch proposal. Capital One says it validated VulnHunter internally before release, running it across thousands of repositories spanning tens of business areas. The company reports that the tool identified and remediated vulnerabilities with speed and efficiency that far exceeded what its teams previously achieved through manual triage. Why AI-powered attacks are forcing banks to rethink traditional cyber defenses VulnHunter arrives at a moment when the cybersecurity landscape is shifting beneath the feet of every enterprise. Capital One's announcement frames the urgency in stark terms: advanced AI models have "dramatically lowered the barrier for bad actors to discover and exploit vulnerabilities in software," and the window before sophisticated AI attack capabilities become affordable and accessible to virtually every adversary is shrinking rapidly. "Safeguarding information is essential to our mission and our role as a financial institution," Nims told VentureBeat. "We have invested heavily in cybersecurity and will continue to do so to stay ahead of today's evolving threat landscape." The company's own AI security researchers have been tracking these trends closely. At NeurIPS 2024 in Vancouver, Capital One's team presented research and curated a list of nearly 100 papers spanning LLM safety, adversarial resilience, jailbreak attacks, and synthetic data generation. The papers they highlighted — including work on multi-agent defense frameworks, automated red-teaming, and guardrail classifiers — paint a picture of an arms race in which offensive and defensive AI capabilities are co-evolving at breakneck speed. Several of those research themes map directly onto VulnHunter's architecture. The falsification engine echoes the adversarial defense strategies explored in papers like " BackdoorAlign ," which demonstrated that embedding a structured safety mechanism into a small number of training examples could recover a model's safety alignment without degrading performance. The attacker-first forward analysis reflects the philosophy of " WildTeaming ," a framework that collects and analyzes real-world jailbreak attempts to build more resilient models. And VulnHunter's emphasis on minimizing false positives parallels the goals of "GuardFormer," a guardrail classifier that outperformed GPT-4 on safety benchmarks while running 14 times faster. The thread connecting all of this work is a conviction that traditional, reactive security — monitoring networks, patching known vulnerabilities, responding to incidents after they occur — is no longer sufficient when adversaries can use AI to discover and exploit zero-day vulnerabilities at machine speed. The only durable defense, Capital One argues, is to find and fix the vulnerabilities in your own code before attackers find them first. What Capital One's cloud security journey reveals about the entire banking industry Capital One's cloud journey also illuminates a broader reckoning across financial services. When Capital One moved aggressively to Amazon Web Services in the mid-2010s, it was a rarity among major banks. Most financial institutions simply did not trust third parties to store their most sensitive data. Capital One's CIO at the time, Rob Alexander, publicly championed the cloud as more secure than the bank's own data centers — a claim that the 2019 breach complicated considerably. The CyberScoop report from that period captured the tension within the industry. W. Patrick Opet, managing director of cybersecurity at JP Morgan Chase, described a cultural shift in banking from prioritizing traders to prioritizing developers: "Now, it's 'Focus on the developer, turn everything into code, and automate everything.'" Mark Nicholson, Deloitte's cyber leader for the financial industry, noted that the pressure to move quickly was exposing "weaknesses in the development methodology." And the breach itself was a reminder that even as Chase spent $600 million annually on cybersecurity, relatively simple vulnerabilities — like the Apache Struts bug that enabled the Equifax breach — could undercut massive investments in data protection. Seven years later, the industry has largely followed Capital One into the cloud, and the security challenges have only intensified. The question is no longer whether to use cloud infrastructure but how to secure the software that runs on it. VulnHunter represents part of Capital One's answer to that question: rather than relying solely on network-level controls and perimeter defenses, push security directly into the code itself, at the moment it is written. The open-source release also carries implicit competitive pressure. If VulnHunter gains traction among developers and security teams, it could set a new baseline for what enterprise security tooling is expected to do — and force rival banks, fintechs, and cloud providers to match or exceed its capabilities. Whether VulnHunter lives up to that ambition will depend on adoption, community engagement, and the tool's real-world performance against the increasingly sophisticated AI-powered attacks it was designed to counter. But the release itself tells a story that extends well beyond any single tool or any single company. In 2019, a misconfigured web application firewall — exploited alongside an overly permissive cloud access role — exposed the records of roughly 100 million Americans , drew an $80 million regulatory fine , and became one of the most widely cited cloud security incidents in the industry . In 2026, the same institution is open-sourcing an AI-driven defense built for a new generation of threats — and betting that the best way to protect its own code is to help the entire industry protect theirs.

VentureBeat·July 17, 2026·12 min read
Zero-Day
↘2K
💬

Top Discussion

HN

Hacker News

“GPT-5.5's API pricing is reshaping how startups build AI products”

14.1K932 comments
View discussion→

Filters

Time
Categories
Sources
Content Type