Vertex AI is Google Cloud’s umbrella for machine learning, and it spans two workflows that have little in common operationally. Confusing them leads to buying the wrong account tier.
Two different jobs
Calling managed models. Model Garden gives access to Gemini and a catalogue of other foundation models through an API. You send requests and pay per token or per request. There is no cluster to size and no GPU quota to negotiate. Your constraint is request throughput.
Training and serving your own. Custom training jobs and endpoints run on machines you specify, including GPU and TPU. Here your constraints are accelerator quota, region availability and cost per hour, and they are substantial.
Most teams starting out only need the first. Buying capacity for the second before you need it is the most common overspend.
What Vertex AI includes
- Model Garden for managed foundation models.
- Custom training for your own models on managed infrastructure.
- Endpoints for serving models behind a managed API.
- Pipelines for orchestrating repeatable ML workflows.
- Feature Store and Model Registry for managing features and versions.
- Workbench, managed notebooks for development.
The quota reality
Accelerator quota is the practical gate on custom training. New projects frequently start with zero GPU quota in a given region, and increases are requested and reviewed. Availability also varies by region and accelerator type, so quota alone does not guarantee capacity.
If your plan involves training, confirm the account’s accelerator quota in your intended region before committing to a timeline. Credit balance does not affect this: quotas and credits are separate systems, as covered in cloud credits explained.
Cost shape
- Managed model calls bill per token or request. Cost tracks usage smoothly and is easy to forecast once you know your call volume.
- Training bills per accelerator hour and dominates everything else when it is running.
- Endpoints bill for the machine while deployed, whether or not it receives traffic. Undeployed endpoints left running are a classic silent cost.
- Data stored and moved is billed separately, and BigQuery scanning during feature preparation can be significant. See BigQuery cost control.
How it compares
Vertex AI’s advantage is integration: if your data already lives in BigQuery, the path from warehouse to trained model is short. AWS offers Bedrock for managed models and SageMaker for the full lifecycle with more control and more assembly. Azure AI leads where Microsoft identity and compliance tooling already govern the organisation. Compare directly in AWS vs Google Cloud.
Our Google Cloud accounts include credit-backed tiers that suit funded AI projects, where the balance is consumed inside its validity window rather than expiring unused.
Common questions
Do I need GPU quota to use Model Garden?
No. Managed model calls do not consume your accelerator quota.
Why did my endpoint keep billing with no traffic?
A deployed endpoint reserves its machine. Undeploy it when idle.
Which region should I use?
The one where the accelerators and models you need are actually offered, then latency. Availability varies more than people expect.