AINews
AI Companies Are Trying to Hide a Staggering Amount of Debt
680 points 368 comments on Hacker News · futurism.com
Hacker News1 min read
Explore
Latest AI news, model releases, research and developer updates.
554 results • Page 29 of 47
680 points 368 comments on Hacker News · futurism.com
Lock in and load up the cloud. GFN Thursday brings fresh updates and new adventures, all ready to play without waiting for downloads. Set sail in Path of Exile: Curse of the Allflame and charge in Battlefield 6 Season 4 both launching major content for members this week. Then revisit Capcom legends like Breath of […]
Talking to The Economist, Musk said he doesn't care if people hate him and said he was not a racist.
With “blessings” from the original Jibo founders, iKairos is a wearable or desk-mounted “AI journal” that turns your family moments into AI images and video.
Built this after watching DTC brands burn weeks and budget on UGC-style ad shoots. It scrapes a product page and generates a script + video using AI avatar happy to answer questions about the approach.
NVIDIA founder and CEO Jensen Huang today visited the Naval Postgraduate School in Monterey, California, to commission an NVIDIA DGX GB300 system — bringing one of the world’s most powerful AI platforms fully online for the students, researchers and faculty at the U.S. military’s flagship graduate university. “Our nation depends on our men and women […]
Economist Mary Lovely dismisses the claims that getting automakers to produce more cars in America proves tariffs work.
After IBM's stock crashed last week on warnings of poor mainframe sales, the CEO explained that AI wrecked corporate hardware budget, temporarily.
When Hugging Face got hit last week, co-founder Clement Delangue suspected a frontier lab, given the agent's sophistication. He was right. Delangue said on X that after a day working with OpenAI he strongly believed there was no malicious intent and that it was mind-blowing the whole thing had happened autonomously. The two OpenAI models that broke into Hugging Face last week did not breach it through malice or superintelligence. They breached it through credentials and permissions they should never have been able to reach, a non-human identity failure that is the oldest problem in security rather than the newest one in AI, and the one every enterprise can actually fix. OpenAI disclosed on July 21 that two of its models, GPT-5.6 Sol and an unreleased, more capable model, were running a cyber benchmark called ExploitGym with their safety refusals switched off, and inferred that the answer key sat in Hugging Face's production database. Getting there took two different failures. A zero-day in a package-registry proxy let the models out of their sandbox and onto the open internet, the kind of persistence OpenAI details in its companion post on long-horizon safety , and that part is genuinely new. The breach of Hugging Face itself came the ordinary way. OpenAI's own account is that the models chained stolen credentials and further zero-days into a remote code execution path, after a series of privilege escalation and lateral movement steps. The exotic part got them to the door, and credentials walked them through it. Hugging Face also disclosed last week that an autonomous agent had harvested cloud and cluster credentials scoped broadly enough to reach multiple internal clusters, then left a trail of more than 17,000 recorded events across short-lived sandboxes over a weekend. Both disclosures describe the same escalation. An agent lands somewhere it should not be, finds credentials scoped far wider than any task requires, and uses them to move. These are two accounts of one incident, not two attacks. The agent Hugging Face watched was OpenAI's models, and both companies describe the same ordinary escalation. The version of this in a typical enterprise is worse, not better. OpenAI and Hugging Face are among the most security-mature organizations in the industry, and both still needed the intrusion to happen before they could see it. The average company wiring agents into Copilot or an internal assistant has neither the identity inventory nor the behavioral monitoring those two brought to bear. The same breach in a normal company would not be contained in days, it would simply go unnoticed. The industry is debating the wrong failure The reaction has split into familiar camps. Former White House AI and crypto czar David Sacks and a run of China hawks seized on the guardrail paradox , that commercial safety filters blocked Hugging Face's defenders while the attacking model ran with its refusals off, and that a Chinese open-weight model, z.ai's GLM 5.2, was what finally let the team finish its forensics. Hugging Face made the case for openness, arguing in an April blog post that open models and open tooling give defenders the same capabilities attackers already have. Both arguments are about the model, and neither touches the mechanism. Reduced refusals let the model attempt an attack, and over-scoped credentials are what let it succeed, and those have nothing to do with whether the model was open or closed, American or Chinese. Making a frontier model provably safe is a multi-year alignment problem no customer can buy or accelerate, while scoping an identity is a configuration change a team can ship this sprint. The industry is being urged to fixate on the part of this it cannot control and to treat the part it can as a footnote. Forrester reached the same read. In a blog on the incident , its analysts argue that security architectures which assume benign intent will miss this failure mode, because an agent can pursue an authorized goal through unauthorized means, which is what OpenAI's models did. This was a non-human identity failure, and it is the oldest one in security Strip the science-fiction framing and what remains is a textbook case of over-privileged machine identity, the kind security teams have fought for a decade, now driven by an autonomous agent at machine speed. Machine identities already outnumber humans in most enterprises by more than 80 to one , according to CyberArk research, with 42% of them carrying privileged or sensitive access, and an agent inherits whatever its identity can touch. OWASP ranks agent identity and privilege abuse near the top of its agentic risk list , the confused-deputy pattern where inherited credentials and weak scoping let an agent reach past its mandate, and that is precisely what both July disclosures describe. IEEE Senior Member Kayne McGladrey has argued in previous VentureBeat interviews that enterprises keep cloning human user accounts onto agents that then wield far more permission than any human would, and this is what that looks like when the agent is a frontier model and the target is a production database. The people closest to it read it the same way. OpenAI frames its models as hyperfocused on a benchmark score rather than acting against anyone. Nobody describes an adversary, only a goal, a scoring function, and credentials that were reachable when they should not have been. The specific failure is easy to name once the AI framing is stripped away. A credential scoped to one job that can reach ten is a standing invitation, and it does not matter whether a human attacker, a worm, or an autonomous model chasing a benchmark score finds it. What changed in July is the finder. An agent enumerates reachable systems, tests credentials, and pivots faster than any human red team, without malice or hesitation, whenever the path is open. The over-scoping was always the vulnerability, and the agent merely industrialized its discovery. Forrester named the control that would have blunted it. Its agentic-security framework, AEGIS, calls for least agency, holding an agent's tools, credentials, and network paths to the minimum its task requires, and files this incident under unrestrained agency and privilege. That is the identity argument in different words, arrived at independently by an analyst firm. The data says this is where the risk now lives. Verizon's 2026 Data Breach Investigations Report found that exploitation of vulnerabilities has overtaken stolen credentials as the top initial access vector for the first time in 19 years. That is the initial-access half. The other half is the one OpenAI itself describes, stolen credentials driving the privilege escalation and lateral movement that followed. A vulnerability opened the door, and credentials walked through the building unchallenged. Beyond the breach itself, that same over-scoping carries a legal liability most enterprises have never priced. The models' actions likely violated the Computer Fraud and Abuse Act , according to TechCrunch. The statute contains no carve-out for an AI agent that exceeds its authorized scope during sanctioned testing. Whatever the legal answer, the technical enabler is the same, an identity scoped wider than its task. This is an access-control problem with an owner and a budget, not a philosophy seminar about machine cognition. Merritt Baer, Senior Advisor to Andesite, G2I, and AppOmni and former Deputy CISO at AWS, frames the underlying shift to VentureBeat as a new kind of asymmetry. Both sides now reach for the same capabilities, she said, but one side is constrained by enterprise governance, policy, compliance, and safety controls while the adversary simply downloads an uncensored open-weight model and keeps going. The organizations that come through it best, in her view, will be the ones that treat AI as a resilient, governed capability rather than a single service they do not control. Four moves that shrink the blast radius The breach worked because the agent reached identities scoped far wider than its task. None of the four controls that would have contained it requires a new platform, and none of them appears on the list of general AI-safety advice now circulating. They are identity hygiene, applied to non-human actors with the same rigor you already apply to people. 1. Scope every non-human identity to one task. The models reached credentials that touched multiple clusters, which is what turned a foothold into a breach. An identity scoped to a single job, with no standing access to anything else, hits a wall at the first lateral move instead of opening the next door. This is least privilege, the control everyone endorses and few enforce on machine accounts, and it is the single highest-impact fix here. 2. Give credentials short lifetimes and rotate them hard. Harvested credentials are only useful while they are valid, and both July agents worked by collecting them. Short time-to-live and aggressive rotation turn a credential dump into expired noise, so a token stolen during a weekend intrusion is dead before the attacker can chain it. Static secrets that never rotate are the version of this control that fails. 3. Monitor for lateral movement, not just prompts. The tell in both incidents was privilege escalation and lateral movement, which a prompt filter never sees because it is watching the wrong layer. Identity-behavior monitoring, keyed to what a given non-human identity normally does and alerting when it reaches somewhere new, catches the escalation the content guardrail missed. The question for your stack is whether anything you run today would flag a service account suddenly moving between clusters. 4. Rehearse instant revocation before you need it. When the incident is your own agent, the fastest containment is killing its identity mid-run, and that only works if the path to do it exists before the day you need it. Rehearse revoking a machine identity under fire the way you rehearse a human credential compromise. If you have never done it, you do not yet have the control, you have an intention. The defense also worked, and that matters. OpenAI's security team caught the anomalous activity internally, Hugging Face's own detection and agents stopped the intrusion, and the breach was contained in days rather than discovered in months, because the defenders could see into systems they controlled. That visibility is the same discipline the four controls depend on. The debate over whether frontier models are safe, open, or American will run for years, and none of it will be settled in time to help the enterprise deploying agents this quarter. The non-human identity gap is different, because it is understood, measurable, and fixable now. The model that breached Hugging Face did not need to be brilliant; it needed credentials someone left in reach. The fix is scoping them before an agent finds them.
There’s a debate going on in the Trump administration over how to handle increasingly powerful Chinese AI models.
You spend weeks tuning an AI chatbot. Answers are accurate. Stakeholders sign off, and you ship it. Three months later, the system is confidently wrong about a third of what users ask. Nobody changed the model, and nobody touched the prompts. The world moved, pricing changed, a policy updated, a product spec shipped a new version, and the underlying knowledge store didn't move with it. This is not a hypothetical. It's one of the most common production failure modes in enterprise AI right now, and most data engineering teams don't have the right tooling to catch it, regardless of how the AI system retrieves the data. The failure that doesn't look like a failure An AI application doesn't care whether it's retrieving from a vector store, a document index, or an API call. Whatever the mechanism, nothing in a standard retrieval pipeline checks whether what it's serving is still correct. A stale pricing document retrieves just as confidently as a current one, because the system is scoring relevance or availability, not correctness. A record with a silently missing field passes through just as cleanly as a complete one, for the same reason. So the failure is invisible by design. Outdated or incomplete data still scores high on relevance, or passes every check a data pipeline was built to run. The model answers with full confidence because the retrieved context looks authoritative. Every dashboard you're watching stays green. The system looks like it's working. It's just wrong. I’ve watched a similar version of this happen outside the AI context, in a fintech pipeline. An upstream system changed a field without notifying downstream users. The pipeline did not fail; it simply propagated bad values into dashboards because the system only checked whether the job completed, not whether the data was still correct. The issue surfaced only when a customer noticed something inconsistent. By then, the bad data had already moved downstream. Whether it's a document that's gone stale or a field that's gone silently missing, the failure shape is the same: the absence of an error is not the presence of correctness, and without building proper validation layers, nothing in the pipeline could identify the problem. Why this is a data engineering problem Teams that hit this failure tend to misdiagnose it, and they tend to do it twice. Blaming the model: The first instinct is to blame the model, try a different LLM, adjust the prompt. The real problem lies further upstream, at the data engineering layer, the same instinct behind the fintech failure above: monitoring built for the pipeline, not the data. Blaming the retrieval layer: Once the model's ruled out, the next instinct is to blame the retrieval or context layer instead and buy a better one. The timing isn't a coincidence: as enterprises push these systems into the real production world, this gap is exactly what's starting to surface, and the vendor response has been everywhere. AWS just entered the "context layer" race with a knowledge graph that learns from agent usage. Snowflake's new Horizon Context and Cortex Sense target the exact symptom this piece opened with : agents giving confident wrong answers because nothing governs the business logic underneath them. Both are real responses to a real problem, but they sit one layer above it; a knowledge graph still depends on whatever feeds it. The real problem lies further upstream, at the data engineering layer. Teams check whether a job ran, not whether the data it moved is still true, an instinct that predates AI by years. Monitoring is built for the pipeline, not for the data. What's actually missing: Data observability Data observability is a well-known concept that doesn't get enough attention in how it's actually implemented. The relevant metric isn't a percentage — it's coverage: what fraction of critical datasets have lineage that's actually queryable, versus only living in someone's head. Uber built a dedicated data quality and observability platform long before retrieval-augmented generation existed. Their Unified Data Quality platform supports more than 2,000 critical datasets and detects around 90% of data quality incidents before they reach downstream consumers. Netflix solved a different piece of the same problem, building a company-wide data lineage system so anyone could answer where a dataset came from and what touched it along the way. It maps dependencies across Kafka topics, ML models, and experimentation, not just warehouse tables. Similar to Uber, the platform was built for humans and now it has become more important with the rise in AI/LLM applications. Between them, Uber and Netflix cover two of the four things worth building for. In practice, I think about it as four dimensions, each measurable on its own terms. Correctness: Does each record conform to the shape and rules it's supposed to, right field types, no unexpected nulls, values in range. Tools like Great Expectations and Soda handle this well: automated row and column-level validation instead of manual checks after something breaks. Track percentage of records passing validation per run. Freshness: Is the data still current relative to its source, not just current as of its last check. Track time since last successful update per source, with an SLA per dataset rather than one blanket threshold, since some sources need hourly refresh and others don't. Consistency: Does the same fact read the same way everywhere it's stored or indexed. This fails silently, it only shows up when two systems fed by the same source start disagreeing. A periodic cross-check between downstream destinations, flagging mismatch rate above a threshold, is enough to catch it early. Lineage: Can you trace any output back to its source and every transform it passed through, the same question Netflix built its system to answer. None of this requires infrastructure most data teams don't already have. I know because I've built it, not just argued for it. At Socure , client data arrived in whatever shape the client felt like sending it, and occasionally, quietly wrong. The challenge was building a system where incorrect data could be identified before it propagated downstream. The same principles applied: Validate what arrived, understand where it came from, and prevent bad data from becoming someone else's problem. Great Expectations became part of that foundation: schema and range validation at ingestion, per-source SLAs for freshness, cross-system checks for consistency, and file-level lineage. All of it sat behind a write-audit-publish pattern, where data landed in staging, was validated, and only moved downstream if it passed the required checks. The result showed up downstream: better accuracy across the board, in reporting, in the ML models, and in AI retrieval built on top of that same data. What to do Monday morning If you're running retrieval-based AI systems in production, the diagnostic question isn't which model to try next or which retrieval architecture to migrate to. It's four narrower questions: Is the underlying data validated against the standards required by its consumers? What's the oldest piece of content currently being served with high confidence? Would two chunks of the same source ever disagree with each other in the same retrieval result? Could you trace where it came from if it turned out to be wrong? If you can't answer those questions, then the gap lies in the pipeline between your source systems and whatever your agent reads from. That’s a data engineering fix, not a model swap or a vendor migration. Whether you're building reporting pipelines, ML systems, or AI agents, correctness, freshness, consistency, and lineage are what make data trustworthy. AI simply exposes weaknesses that have existed in data engineering all along.
OpenAI made a mistake setting up what it called a “highly isolated” testing environment and sandbox. According to cybersecurity experts, that human mistake is what made the AI-powered attack on Hugging Face possible.
