Serverless containers & batch jobs
Can I host an open-source model that isn't offered on the platform?
Yes - deploy it yourself in a serverless container (for example with vLLM) or on a GPU instance. You have a wide GPU selection in the console, and for...
My deployment shows "running" but the endpoint returns a 404
A few common causes: you're using the wrong auth (containers need an Inference API key , not a management key); a misconfigured health-check path or ...
Should I use spot or on-demand for a serverless deployment?
On spot, GPUs go to on-demand users first, so a spot replica may not get capacity back after scaling to zero. For production where availability matter...
How is serverless billing calculated?
Your balance is deducted in advance in 10-minute increments for active replica-time (GPU hourly rate x active replicas x duration), including spin-u...
How does scale-to-zero work, and how do I avoid slow cold starts?
Containers scale automatically with traffic and can drop to zero when idle, so you only pay for GPU time when a request comes in. Scaling back up from...