Almost every data-driven enterprise runs the same workflow. Data leaves the operational database, travels through an ETL (Extract, Transform, Load) pipeline, lands in a data warehouse, and finally reaches an AI model. The whole journey takes hours. Sometimes days. Staleness is a feature, not a bug, because that is how the architecture works.
This design made sense when AI models lived somewhere else. It makes considerably less sense when the AI model lives inside your data warehouse and you can call it directly from a SQL query.
What BigQuery AI Functions Actually Do
BigQuery now ships with native AI functions: AI.GENERATE for text generation, AI.EMBED for creating vector embeddings, and AI.SIMILARITY for semantic similarity comparisons. These are not connectors to an external API. They run inside BigQuery, on your data, governed by your access controls, with no data leaving your Google Cloud environment.
In practice, you can write a SQL query that classifies every support ticket in your database by sentiment and category, generates a summary of each one, and returns the results in a single statement. No Python. No pipeline. No intermediate storage layer. No data engineering sprint before the data science team can start working. The operational tax of stitching together separate systems for storage and AI processing effectively disappears.
For teams whose job descriptions include “manage the pipeline,” this is either a relief or a mild existential crisis.
Why This Is a Product Architecture Question, Not Just an Infrastructure One
The more interesting question for ISVs is not “can we reduce infrastructure costs.” It is “what products become possible when AI is a function you call at query time instead of a service you route data to.”
Real-time personalization that responds to what a user just did, rather than a batch job from last night. Fraud detection that runs on every transaction as it inserts, not on a sample reviewed after the fact. Customer-facing analytics that generate natural language explanations alongside the numbers, without a separate service to maintain. None of these are new ideas. They have simply been expensive and complex enough to stay on roadmaps instead of shipping.
When AI becomes a SQL function, the distance between “we have the data” and “we are doing something intelligent with it” shrinks dramatically. Competitors still moving data before touching it with AI are operating at a structural disadvantage, and that gap tends to widen as product velocity compounds.
A Few Questions Worth Sitting With
How much engineering time does your current architecture spend moving data to where the AI is, rather than running the AI where the data already lives? Which features sit in “Phase 2” because the pipeline work to support them is too heavy a lift for Phase 1? And if AI inference could happen at query time on your existing warehouse, which architectural assumptions would you revisit?
That last one is worth a few minutes of honest thought. The most interesting product decisions often come from questioning infrastructure choices made when the options looked very different.
Want to go deeper?
- Google Cloud: BigQuery AI.GENERATE documentation — full reference for running generative AI directly inside SQL queries.
- GCP Blog: BigQuery AI Functions announcement — the original launch post with context on how native AI functions fit into the broader BigQuery roadmap.
- Google Cloud: BigQuery vector search — how AI.EMBED and AI.SIMILARITY connect to vector search for semantic retrieval at warehouse scale.