The vast majority of enterprise data is tabular (it lives in data warehouses, CRMs, and financial ledgers), but building a reliable model from it still means training a new one from scratch for each data set, then maintaining cycles of hyperparameter tuning, feature engineering, and retraining pipelines to combat data drift. Google Research proposes a way to avoid this: a new basic model called TabFM which treats tabular prediction as a learning problem in context.
You can generate predictions for a new, invisible table in a single step forward. For enterprise developers and AI engineers, this reduces production time from weeks of pipeline engineering to a single API call.
The challenge of traditional ML
To extract reliable predictions from a gradient-driven tree, data scientists must build and maintain complex data pipelines. They have to clean up messy inputs, impute missing values, encode categorical variables into numerical formats, and design custom feature crossovers.
Once the data is ready, they must run repetitive loops of hyperparameter optimization, searching through learning rates, tree depths, subsampling ratios, and regularization grids to find the best configuration.
Once implemented, these traditional models "incur ongoing operational debt through monitoring data drift and retraining channels to stay accurate," Weihao Kong, a research scientist at Google Research, told VentureBeat.
Meanwhile, the rest of the AI industry has moved on. Generative AI models for text and computer vision have seamlessly transitioned to zero-shot inference, where a model can perform an entirely new task simply by receiving context cues.
Large language models (LLM) already stand out in learning in contextSo why can’t we just plug tables into a commercially available LLM?
Because LLMs are trained in natural language rather than structured data, they have difficulty processing tables directly. First, your context limits are quickly exhausted in medium-sized tables containing only a few thousand rows and hundreds of columns. Second, LLMs suffer from inefficiency in tokenization, as they awkwardly split numerical values and destroy mathematical precision. Finally, they suffer from structural blindness. When a 2D table is serialized as a 1D text string, LLMs lose track of which value belongs to which row and column as the table grows.
"That’s why, nowadays, it’s much more effective to use an LLM to write the code that handles feature engineering and calls XGBoost than it is to ask the LLM to read the table itself." Kong said.
What is TabFM?
To run inference with TabFM, you do not update any model weights. Instead, it takes your historical examples (the training rows with their known labels) and your target rows (the new data you want to predict) and passes them to the model as a single, unified message. The model learns to interpret relationships between columns and rows directly from this context at run time.
For example, consider a business analyst trying to predict customer churn. Instead of creating a custom data pipeline and training an XGBoost model, they can simply pass a sample of historical user session data along with a new, active session to TabFM. In a step forward, the model returns an instantaneous churn probability.
TabFM overcomes the limitations of LLMs by treating data as a grid, preserving its structural integrity without forcing it into a one-dimensional text string.
To efficiently process diverse tabular structures while enabling scalable zero-shot prediction, TabFM synthesizes the strengths of previous experimental architectures, TabPFN and TabICL. PFN Tabdeveloped by Prior Labs, demonstrated for the first time that a transformer architecture could perform zero-shot sorting on small tables, although it had difficulty scaling computationally to larger data sets.
Later, ICL Tabdeveloped by France’s National Institute for Research in Digital Science and Technology, addressed this bottleneck by introducing row compression, enabling in-context learning to efficiently process much larger tables.
TabFM combines the deep feature contextualization of TabPFN with the efficient compression of TabICL in a novel hybrid design built on three key mechanisms:
1. Alternate attention to rows and columns: The raw table is first processed through a multi-layer attention module that alternates columns (features) and rows (examples). By paying continuous attention to these two dimensions, the model natively captures complex feature interactions. This deep contextualization does the heavy lifting that would normally require tedious manual feature construction by data scientists.
2. Row compression: After this contextualization, the cross information for each row is compressed into a single, dense vector representation. TabICL pioneered this by using CLS tokens to compress the rich information of a row into a vector. "Unlike TabPFN v2, v2.5 and v2.6, which serve over the entire cellular network across the entire network," Kong explained. This dramatically reduces the computational footprint.
3. Learning in context (ICL): A causal transformer then operates on this sequence of compressed embeddings. This Transformer model uses TabICL’s attention mechanism to serve these dense row vectors, dramatically reducing computation cost and allowing the model to process large data sets efficiently.
A major selling point of TabFM is its pre-workout recipe. The model was trained entirely on hundreds of millions of synthetic data sets. These data sets were dynamically generated using structural causal models (SCM) that incorporate a wide variety of random functions. By training exclusively on synthetic SCMs, TabFM learned the fundamental mathematical aspects of how tabular features interact without ingesting sensitive real-world CSV files.
TabFM in action
To test the model’s capabilities, Google researchers benchmarked TabFM on TabArena, a comprehensive evaluation suite spanning 51 diverse tabular data sets across 38 classification and 13 regression tasks.
On these public benchmarks, TabFM’s zero predictions already match or beat the strongly tuned monitored baselines. However, Google is careful to point out that this doesn’t automatically mean that TabFM will universally dethrone custom, hyper-optimized production models in every enterprise workload.
"Rather than replacing hyper-optimized production models, the real practical business value it unlocks for lean engineering teams is speed," Kong said. "It enables data analysts and backend engineers to instantly create high-quality reference models without a dedicated data science team managing a complex lifecycle."
For advanced professionals looking to obtain maximum precision, the research team also introduced a "TabFM Set" configuration. By running the model through 32 different variations and combining the results, TabFM takes performance even further.
Getting started, trade-offs, and the future of the cloud
The shift to in-context learning for tables introduces a new economic trade-off that engineering teams must consider.
With traditional algorithms, training is slow and expensive, but inference is lightning fast and cheap. TabFM changes this dynamic. As training time drops to zero, inference becomes significantly heavier. Because the model must process the entire historical data set as context during each prediction, it requires more computation and memory at runtime.
In this new paradigm, "traditional machine learning training becomes the “pre-fill” (KV caching) phase in the context window," Kong said. While this prefill cost is high, you pay only once per table and the cache is reused in subsequent queries. "The problem is prediction latency, which no caching eliminates," Kong added. Each new prediction requires going through a great transformer. "Any production API that requires response times in the single digits and milliseconds cannot tolerate the passthrough overhead of TabFM."
For developers looking to evaluate the model today, the barrier to entry is low. Google designed TabFM as a drop-in replacement for traditional ML workflows, offering a scikit-learn compatible API (TabFMClassifier and TabFMRegressor). It natively handles mixed numeric and categorical columns, works directly with pandas DataFrames, and requires no manual ordinal encoders or numeric scalers. The library supports JAX and PyTorch backends.
However, enterprise teams should be aware of current licensing limitations and restrictions. The model architecture has a hard limit of 10 output classes for classification tasks and is optimized for tables with up to 500 features. More importantly, while Google launched the underlying code base Under the permissive Apache 2.0 license, pretrained model weights are published on hugging face under a strict tabfm-non-commercial-v1.0 license. Developers can evaluate the model internally, but it cannot yet be implemented in commercial products.
Looking ahead, Google is addressing frictions in commercial deployment through its cloud ecosystem. TabFM is being integrated directly into Google BigQuery, allowing analysts to run zero predictions natively using an “AI.PREDICT” command. By placing basic model inference right next to the data warehouse, TabFM could soon make complex tabular machine learning as accessible as a basic database query.
In practice, TabFM shines in rapid prototyping, high data drift environments, and small to medium data sets of less than 100,000 rows. Instead, teams should stick to traditional models for strict ultra-low latency APIs or massive tables exceeding one million rows, which currently require aggressive row sampling that degrades the competitive advantage of the base model.
