I recently gave a presentation exploring a question that sits at the heart of our work at Fundamental:

If large language models can write code, explain statistics, solve mathematical problems, and assist with increasingly sophisticated analytical tasks, why aren't they equally good at making predictions?

It's a reasonable assumption. In many ways, it feels like prediction should be a natural extension of everything we've seen LLMs achieve over the last few years.

But when you look closely at the evidence, a different picture begins to emerge.

Watch the full presentation below.

In This Presentation

• Why an LLM achieves 98% accuracy on a classic machine learning benchmark, then falls below a simple baseline on a slightly harder task

• How benchmark memorization can create misleading results

• Why tabular prediction is fundamentally different from language generation

• The limitations of applying LLMs directly to structured data

• Why a new generation of tabular foundation models is emerging

Prediction Is Not Generation

One of the biggest misconceptions in AI today is that language generation and tabular prediction are fundamentally the same problem.

They are not.

When we think about the breakthroughs that have captured public attention, we're usually talking about language models (like Gemini, ChatGPT, Claude, …) image and video generators, and coding assistants. These systems are trained on unstructured data. Text, images, video, and code all fall into this category.

These systems have become remarkably capable because they were specifically designed and trained for those tasks.

Tabular prediction lives in a different world.

The data that powers forecasting, risk assessment, supply chains, healthcare, manufacturing, scientific research, and enterprise operations is typically structured. It exists in rows and columns within tables. Relationships between variables matter. Patterns emerge across many dimensions simultaneously.

While both language generation and tabular prediction involve learning from data, they place very different demands on a model.

The question is whether a model optimized for one can automatically excel at the other.

When Strong Results Turn Out To Be Misleading

At first glance, the answer appears to be yes.

In the presentation, I begin with one of the most famous datasets in machine learning (and I might be showing my age here): the Iris dataset. This incredibly small, simple dataset has often been used as a sanity check for new algorithms in the past. One might be tempted argue we have long moved on from data sets of this size, but ironically this does not seem to be the case yet.

When asked to perform a classification task on this dataset, a leading LLM achieves an impressive 98% accuracy.

If we stopped there, we might conclude that language models are already excellent predictive systems.

But then we make the problem only slightly more difficult.

Moving to a very marginally larger dataset, the breast cancer dataset, the results change dramatically. Traditional machine learning models continue to achieve over 90% accuracy. The LLM falls to 69%, performing worse than a simple baseline that predicts the most common class.

What's striking isn't that performance drops.

It's how quickly it drops.

A model that appeared to have mastered prediction suddenly struggles on a task that remains extremely basic and still very much within the ‘toy dataset’ category for decades-old machine learning techniques.

That suggests something important: the impressive performance we sometimes observe may not be evidence of a general predictive capability.

Why Benchmarks Don't Tell The Whole Story

This leads to a second question.

If LLMs aren't naturally strong predictive models, why do benchmark results sometimes make them appear that way?

Part of the answer is that many popular benchmarks are public.

Modern language models have been trained on enormous portions of the internet. Datasets, documentation, tutorials, papers, and examples often exist online in various forms.

In the presentation, I walk through a series of experiments designed to test whether benchmark familiarity is influencing performance. For example, in a small experiment I test whether the models can predict classes that they have never seen by leaving out an entire class during training. 

The results are revealing: Traditional methods unsurprisingly fail - this is an impossible task after all. LLMs, however, achieve the impossible and are able to correctly predict classes that it should not know exist. This very clearly indicates that they are not actually carrying out prediction tasks but rather relying on prior exposure to these data sets.

This doesn't mean the model is intentionally cheating. It's simply leveraging information it has encountered during training.

The problem is that real-world prediction tasks rarely look like benchmark problems.

Enterprise data is proprietary.

Scientific data can often be proprietary.

Operational data is proprietary.

The model cannot rely on familiarity when it comes to these because it has never seen the data before.

And that's where true predictive ability matters.

The Difference Between Doing Analysis and Generating Analysis

Another source of confusion comes from the fact that LLMs are undeniably useful for analytical work.

They can generate SQL queries, write Python code, summarize findings, automate workflows, and help researchers move faster.

These are valuable capabilities.

But there is an important distinction between generating analysis and performing analysis.

Many examples presented as evidence that LLMs are excellent predictive models actually involve the model generating code that then uses traditional machine learning methods.

The LLM helps build the tool. The tool performs the prediction. However, assuming a non-extraordinary amount of available cycles for agents to develop said tools for every prediction, it’s safe to assume they will resort to existing methods and libraries. They are not going to be able to develop something that is fundamentally new, of the scale that a research lab can produce over months and years. 

A Different Problem Requires Different Models

None of this should be interpreted as criticism of LLMs.

On the contrary.

Language models have transformed AI because they were designed for language. Their success is a consequence of being exceptionally well matched to the problem they were built to solve.

The mistake is assuming that every data problem is ultimately a language problem.

Tabular prediction remains one of the most important challenges in AI.

Businesses need to forecast demand. Financial institutions need to assess risk. Healthcare systems need to anticipate patient outcomes. Supply chains need to predict disruptions.

These decisions depend on structured data.

And structured data behaves differently from language.

This is why a new field has emerged around Large Tabular Models which are designed specifically for prediction rather than generation.

The goal isn't to replace LLMs at what they do.

The goal is to bring the same scale, flexibility, and generalization capabilities that transformed language to the world of structured prediction.

At Fundamental, this is the challenge we spend our time working on.

Because while language generation and prediction may appear similar from a distance, they are fundamentally different problems.

And fundamentally different problems require fundamentally different models.