I tested Databricks AI_DECIDE() against 20 LLMs
AI_DECIDE is currently free in beta, so I tried it on a fairly simple task: deciding whether to approve or reject a post. Managed inference is free, but running the notebook still costs compute.
To compare accuracy, speed, and cost, I put it against the 20 models available via the Databricks AI Gateway (Llama, Qwen, DeepSeek, GPT-6, Claude, Gemini, Grok, and OpenJev).
What is AI_DECIDE()?
AI_DECIDE takes natural-language instructions (text or structured data) and returns a result you can use directly in SQL. The result can be one of your labels, a probability between 0 and 1, or a score on a scale that you define.
You can use it to categorize a travel expense, check a hotel's pet policy, or score a guest review. In this case, I'm using its choice output for moderation (approved or refused).
What did I test?
I limited the sample to five posts: a standard SQL question, a post criticizing a product release, a money scam, a direct threat, and a safety report that contained a threat. The expected decisions were approve, approve, refuse, refuse, and approve so that the test could compare model outputs against a fixed benchmark.
Simple calls
Here's the mini version of the example. I've pulled the product review from the benchmark: "This release is terrible. The search box crashes and the new pricing is absolutely terrible value." That should be approved, as it's criticism.
With AI_DECIDE, I define the two allowed choices and select only the choice field from the response:
For the LLM version, ai_query sends the same post and rule to Llama 3.3. The prompt asks it to return only approved or refused:
Both cells were running on serverless compute.
Because I only required one label, I used very small output limits (e.g., 16 tokens for Llama, 256 for GPT-6 and Gemini). For the reasoning models, I used low effort where available and turned off reasoning for Deepseek. The cost table below uses actual billed usage where available, with token-based estimates as appropriate.
Temperature=0 means no sampling randomness; I used it where available. AI_DECIDE and OpenJev both used a choice scheme without any of the above controls, and the linked code has the exact per-model settings.
Did all the models get it right?
Most models returned 15/15 correct, properly formatted answers. Llama 3.1 8B added SQL advice to three responses, leaving 12/15 usable. Gemma 3 12B incorrectly refused the safety-report post in all three repeats, also scoring 12/15. Every other model, including Grok 4.7, returned 15/15 usable answers.
Decision speed
How much did it cost?
AI Decide inference is free during beta, so the next cost figures are for recorded inference and compute.
Beyond the inference costs we benchmarked here, you need to add compute costs.
The full serverless job for the five-post AI_DECIDE test has a provisional recorded compute cost of approximately $0.07. AI_query used for other models has the same cost structure, with compute cost dependent on workload and runtime. You can also run SQL directly on the SQL warehouse and pay for the capacity it uses; model inference is a separate cost.
My takeaway
AI_DECIDE felt like the simplest fit for this task: give it a rule and two choices, then use the answer in SQL. The larger models didn't improve decisions on these five posts. It is the fastest; we will need to check the price again once inference will be not for free, but I guess databricks will like to keep the cost similar to OpenJev.
TL;DR
AI_DECIDE returned 15/15 usable, correct answers, with the fastest median response time: 0.976 seconds per post.
A separate serverless SQL batch processed all five posts correctly in 4.481 seconds of query execution.
Across 315 comparison responses, 309 were usable and correct. Llama 3.1 8B had three format errors; Gemma 3 12B had three wrong decisions.
