I deleted my Reddit account a long time around the beginning of the hostilities, but still occasionally visit via search results. In that case, I usually try to switch to `old.reddit.com` because the "ugly" UI is clearly better. Anyway, today, I got a modal dialog demanding account creation. I was not aware of this and I hadn't seen it mentioned anywhere. FYI.
Hey HN, Henry & Roman here from Cactus. A small, on-device model is fast and private, but sometimes wrong, but frontier models are getting expensive pretty fast. So, we post-trained Gemma 4 E2B post-trained to know when it's wrong. Every response comes with a confidence score between 0 and 1. Developers can accept the on-device when it's high, hand off to a bigger cloud model when it's low. By routing only 15-35% of queries to Gemini 3.1 Flash-Lite, Gemma-4-E2B matches Gemini 3.1 Flash-Lite on most benchmarks. - ChartQA: 15-20% - LibriSpeech: 25-30% - MMBench, GigaSpeech, MMAU: 30-35% - MMLU-Pro: 45-55% We were always frustrated by the routing signals hybrid apps rely on: asking the model to rate itself in text (unreliable, and you're parsing prose), or token entropy heuristics (barely better than a coin flip in our tests). So we did mechanistic studies on small models, Gemma 4 particularly, and found the hidden state for different layers carry meaningful self-awareness signal for various situations. SO we extended the model with a 68k params probe layer (LayerNorm, low-rank projection, attention pooling, small MLP head) reads one intermediate layer during decoding and predicts p(wrong); confidence = 1 - p(wrong), returned as structured data, never parsed out of the answer text. Across 12 hold-out benchmarks spanning text, vision and audio, the probe averages 0.814 AUROC vs 0.549 for token entropy. The result that convinced us this is real: the probe was trained on zero audio data, yet scores 0.79-0.88 AUROC on four audio benchmarks where entropy is near-random or worse (0.32-0.52). It's reading a modality-independent correctness signal from the hidden state, not memorizing patterns from its training data. We published all weights on HuggingFace and provide copy-pase codes to run it on Transformers, MLX, Llama.cpp or Cactus. With Ollama, vLLM, SGLang etc in the works. For llama.cpp we ship a patch series you compile in once (upstreaming is planned). The code is MIT licensed; Gemma model use remains subject to the Gemma terms. GitHub: https://github.com/cactus-compute/cactus-hybrid Weights: https://huggingface.co/collections/Cactus-Compute/cactus-hyb... Some caveats: - The probe scores single-sequence decoding only, up to the first 1024 generated tokens. - Handoff works best when routing per task in a multi-step process, not per step. - Hierarchical routing is still in the works: try on-device, then DeepSeek v4 Flash, before Fable/GPT5.5/Gemini/Muse/Grok. - The technique is boutique for each model, we will share each weights as they roll out. These issues are currently being tackled at Cactus and updated weights will be shipped directly into the HuggingFace collection and GitHub repository straight up. Please let us know your thoughts, it helps us find ways to improve the design progressively. Thanks a million!
Posted by Zoe Lopez-Latorre, Senior Developer Relations Engineer, Android Back in March, we introduced Android Bench —our LLM leaderboard for real-world Android development tasks. Our goal was to provide transparency around model capabilities in Android development and to encourage model improvements, to give you more helpful AI options for your everyday workflow. Since then, we have enhanced the benchmark based on your feedback, including evaluating open-weight models and adding cost and efficiency dimensions to the leaderboard. But AI capabilities are ever-evolving, and measurement needs to follow suit. As part of our July release, we have adopted the Harbor framework , which includes an updated version of the benchmarking agent used to evaluate models. Along with this change to our evaluation, in this July release we’re adding 8 new models ( Claude Fable 5, Claude Sonnet 5, Claude Opus 4.8, GLM 5.2, Kimi K2.7 Code, MiniMax M3, Qwen 3.7 Plus and Qwen 3.7 Max ) to the leaderboard. We’re also sharing opportunities for you, the Android developer community, to contribute to the benchmark. Upgrading our methodology with the Harbor framework When we designed Android Bench, we anchored our methodology on leading industry standards available at the time. We used mini-swe-agent v1, a general-purpose benchmarking agent, and adapted it to the nuances of Android development to provide a baseline measurement for the capabilities of models for common Android development tasks. To continue providing you with state-of-the-art evaluations that accurately measure the latest model capabilities on Android development, we are standardizing our benchmark to the Harbor framework . Harbor defines standards and integrations that make it easy for anyone to run the benchmark, evaluate their preferred set-up, or share results – providing you with additional transparency and visibility. This upgrade enables us to more rigorously evaluate models and their capabilities, and we re-ran the benchmark on all models to establish an updated baseline. This means there is a minor shift in scoring, but you will still be able to view historical scores within the archive on our website. We want to ensure Android Bench is helpful for you, so we will continuously update it as our evaluations and the industry mature. Expanding the leaderboard with 8 new models As part of our commitment to keeping the leaderboard fresh, we have added Claude Fable 5, Claude Sonnet 5, Claude Opus 4.8, GLM 5.2, Kimi K2.7 Code, MiniMax M3, Qwen 3.7 Plus and Qwen 3.7 Max to the Android Bench leaderboard. You will see that Claude Fable 5 is at the top of the leaderboard with a score of 84.5, followed by GPT 5.5 with 80.2, with Claude Sonnet 5 in 3rd with a score of 76.2. When just comparing Open-weight models, GLM 5.2 is at the top with 72.2, followed by Kimi K2.7 Code with a score of 70.4. You can check out model performance and efficiency metrics on the updated leaderboard to see how these new and previous models navigate Android-specific challenges like Jetpack Compose migrations, wearable networking, and platform API updates. Opening Android Bench to community contributions From the beginning, we’ve valued an open and transparent approach, which is why we made our original methodology and test harness publicly available on GitHub. You’ve asked for a way to provide feedback on our dataset, so now we’re taking collaboration a step further by giving you, the Android developer community, a chance to shape Android Bench. Starting today, you can contribute to Android Bench in two ways: Design and submit your own Android development tasks to evaluate how models handle the scenarios that matter to you. Run and share benchmark evaluations firsthand, testing your preferred models against our dataset or your own custom tasks. We will be reviewing the submitted tasks and will be assessing if they get added to the benchmark. We hope to build a benchmark that truly reflects the diverse, day-to-day realities of the global Android developer community. Looking ahead With more and more options for agentic development, maintaining a cutting-edge benchmark ensures that the AI assistance you rely on keeps getting smarter, more helpful, and more effective. Head over to our GitHub repository to check out the tasks. We invite you to submit a task to our team for review, and you can check out Harbor Hub to explore the dataset or submit evaluations. As always, you can find the updated leaderboard , or read the methodology on our website. Android Bench, LLM leaderboard, Harbor framework, Android development, Claude Fable 5, GPT 5.5, Claude Sonnet 5, GLM 5.2, Kimi K2.7 Code, MiniMax M3, Qwen 3.7 Plus, Qwen 3.7 Max, AI benchmarking, Jetpack Compose migration, wearable networking, mobile AI agent, Zoe Lopez-Latorre, model evaluation, open-weight models, developer community contributions.
Well here it is, friends. This will be the last of my retro game eShop lists, mostly because I am … Continue reading "The 10 Best PlayStation 1 Games on Nintendo Switch – SwitchArcade Special"
Posted by Jolanda Verhoef, Senior Developer Relations Engineer, Android Developer Relations Building GenAI features in your app usually means navigating through various models, APIs and architecture choices: Execution location: Where does your model run? On device, in the cloud, or both? Complexity: How complex is your setup? Are you doing a single inference call or do you need a more agentic flow? In-app or Android System: Should your feature be built into your Android app or does it fit better as an Android system integration? In this blog post series we'll navigate these choices with you. We will take you along on a journey, starting with a basic mobile app and transforming it into a personalized , intelligent , and agentic experience. Jetpacker: a demo travel app Jetpacker is a technical showcase app that our team built from the ground up for this year's Google I/O (built using Antigravity). At its core, Jetpacker helps users plan, explore, and enjoy their next big adventure. It shows an overview of your trips, the itinerary of each trip, and details of each event on that trip. Of course following all best practices of Android development, including a beautifully expressive Material UI design. And best of all? It's fully open source ! Today we are publishing a series of technical blog posts diving deep into each of these features. We’ll provide detailed implementation steps, code snippets, and architectural insights to help you build your own intelligent Android applications. On-device intelligence On-device features in Jetpacker: Summarizing trip itineraries, managing expenses, and voice notes Using an on-device model comes with no additional cloud inference costs, means you don't have to worry about internet connectivity , and lets users be confident that private information will be processed locally , on the device, without any of their data being sent to the cloud. In Jetpacker, we chose on-device inference for three of our features: The trip overview feature transforms a messy, multi-day itinerary into a concise, actionable summary. It leverages Gemini Nano through the ML Kit GenAI APIs to process data locally on the device. We consider this a nice-to-have feature where we don't want to incur extra cloud costs, making on-device inference the right choice. The expense tracker automatically extracts structured data from receipt images to help users track their travel spending. It uses the multimodal capabilities of Gemini Nano 4 through the ML Kit GenAI APIs. We choose an on-device solution so that any privacy-sensitive information on the receipt images never leaves the user's device. The audio diary records, transcribes, and categorizes voice notes into relevant trip activities. It is powered by the ML Kit Speech Recognition and GenAI Prompt APIs . We chose an on-device solution for privacy and connectivity reasons. Cloud & hybrid inference Cloud and hybrid features in Jetpacker: Museum assistant with web grounding, hybrid restaurant review drafting, and hotel support chat featuring custom-routed live translation. Sometimes your use-case requires AI models with greater world knowledge or a much larger context window and with greater ability in handling complex tasks . In that case, we can switch from running an on-device model to using a cloud model instead. Or, if you want to get the best of both worlds, you can use hybrid inference to dynamically choose either a cloud or on-device model at runtime. This allows us to lower costs by moving inference to the device when it is available, but at the same time support all Android devices running the app. In Jetpacker, we implemented several features using cloud or hybrid inference: The place Q&A feature answers user questions about specific locations by grounding responses in real-world data. It uses Firebase AI Logic integrated with Google Maps and web context . Using a cloud model is necessary here for its greater world knowledge. The review drafting feature helps users compose detailed reviews for the places they have visited. It leverages both on-device and cloud models through Firebase AI Logic's new Hybrid inference API . This is a feature we wanted to make available to all app users, so we're using a cloud model as a fallback when an on-device model is unavailable. The automatic chat translation dynamically translates chat messages in real time to facilitate seamless communication, demonstrating custom hybrid inference logic. Again, we want this feature to be available to all app users, but at the same time have some specific considerations on when to choose on-device versus cloud. System integration While not a feature you see in the app itself, the Android system integration opens up the app's core capabilities directly to the Android operating system. It uses the AppFunctions API to integrate with system-level intelligence. In-app agentic workflows (coming soon!) The booking assistant shows several in-progress flight bookings, asking the user for input before making a final booking. Agenticness introduces a higher level of autonomy , enabling models to act as agents. Instead of a single inference call, an agent works towards a specific goal via an orchestration loop that allows it to reason , use tools , and adapt its path. Depending on your requirements, these intelligent agents can run either in the cloud, directly on-device, or in a hybrid setup. For Jetpacker we added a booking assistant that automates end-to-end booking workflows directly within the application to streamline reservations. It is built using A2UI and ADK running in the cloud. The Android app functions as a front-end to the multi-agentic system running in the cloud. Learn more Check out the other parts of this blog post series: Part 1 (this post!): Introduction of the app and a high-level overview. Part 2: On-device intelligence. Deep-dive into ML Kit’s GenAI APIs and Gemini Nano to build privacy-first features like itinerary summarization, receipt parsing, and local audio processing. Part 3: Hybrid and cloud reasoning. Explore how to use Firebase AI Logic to ground LLM answers in real-world data like Google Maps and web context. Part 4: System integration. Integrating with the Android intelligence system using AppFunctions. Part 5 (coming soon): In-app agentic workflows. Extend the app with an end-to-end booking assistant powered by A2UI and ADK. Interested in more on Android Development? Follow Android Developers on YouTube or LinkedIn !
Android Developers Blog··5 min read
Zero-Day
↘2K
💬
Top Discussion
HN
Hacker News
“GPT-5.5's API pricing is reshaping how startups build AI products”