Search Results

Show HN: I built a hypervisor and client for inference on consumer compute

I'm the founder of Scalattice, this is my second company, third total product. I'm a 2x founder building some challenging software, some easy software, and some curiosity based tools that I've just always wanted to be a part of! So here is Scalattice. Scalattice is an OpenAI-compatible inference API. You keep the OpenAI SDK, swap base_url + API key, and call open models (Qwen3, Llama 3.3 70B, Gemma 3, DeepSeek R1, etc.). Inference is performed by a provider node on our distributed network of consumer hosted inference machines, we run an open source agent ( https://github.com/scalattice/scalattice-agent ) which works in Rust to run the inference and get the provider paid. In short, anyone can become a provider, offer up their machine with our Windows/Linux Rust agent, and earn some extra cash, or start a farm of machines to make big bucks. I decided to build some innovative behavioural traits to the API for higher performance/security: 1. output vetting - Scalattice Cloud vets the response against replica responses requested on the API header N times. 2. security tiers - Using some split inference, the job is completed in chunks by multiple providers and in part by Scalattice Cloud hypervisor for added security/privacy (at an additional cost) 3. regional policy - I know how important data residency is to developer clients (from my experience with Digital ID Infrastructure) and so I built the platform to let developers specify the region for inference. We pay our providers a majority share (Currently 80%) of every token spent by the developer. We currently back the network with a failover of our own company machines which ensures we never drop a request. If you want to give it a try in a couple of minutes, we are currently running a "Top Up $10 for UNLIMITED QWEN3" for an entire month of API calling, (terms and conditions apply). If you want to give it a try in a couple of minutes: 1. Create a key at https://scalattice.cloud/developers 2. Use the copy-paste snippet on the linked post 3. Live rates: https://scalattice.com/pricing Or, if you want to be a provider (We are really keen to onboard people and get them earning): 1. Create a machine at https://scalattice.cloud/providers 2. Download our Windows app, or use our on-page instructions to curl the agent installation script for Linux. 3. Attach a provider token to the app or Linux agent from the Machine created on the website. 4. Control the machine from the website, select which models you want to offer, and the physical hardware components to use for inference and the hours of operation. 5. Earn! Happy to take feedback - especially on DX, pricing clarity, and what would make you try this instead of Together / Fireworks / OpenRouter / Salad etc. This is my first time posting on HN, please let me know if you think the product has potential, there are obviously trade-offs with non-datacentre inference (latency, capability of cards), but I think this could serve a decent amount of developers very well, because not everyone needs H100 cards for their inference. Thanks again.