The Model Was Never the Problem

Artificial Intelligence has become remarkably good at selling itself.

A few years ago, companies proudly announced that they were “moving to the cloud.” Today, those same companies proudly announce that they are “implementing AI.” The vocabulary has changed, but the pattern remains almost identical. Executive boards see competitors discussing machine learning, investors ask questions about AI strategy during quarterly meetings, and suddenly every organization feels an urgent need to have an artificial intelligence initiative.

The pressure is understandable.

Nobody wants to become the company that ignored the next technological revolution.

Unfortunately, urgency has a habit of skipping the most important question.

What exactly are we trying to improve?

One financial services company believed it already knew the answer.

Its objective looked perfectly reasonable. The business wanted to predict which loan applications were most likely to become funded customers. Better predictions would improve marketing efficiency, reduce acquisition costs, and help sales teams prioritize high-value leads. The expected financial impact exceeded several million euros annually, making the project one of the company’s strategic initiatives.

The executive presentation was impressive.

Google Vertex AI would train predictive models using historical data stored in BigQuery. Marketing events from Google Analytics 4 and AppsFlyer would be combined with CRM information, transactional history, and customer behavior. The engineering roadmap included automated retraining, model monitoring, A/B testing, and continuous deployment. The architecture followed Google’s recommended patterns almost perfectly.

Funding was approved within two weeks.

The data science team started preparing the first training dataset.

That was the moment when the project quietly stopped being an AI initiative and became a data investigation.

The first request sounded deceptively simple.

“Please provide three years of historical customer data with the final business outcome.”

Nobody expected that request to become controversial.

After all, the company had invested heavily in its analytical platform. BigQuery contained billions of records. Every department had dashboards. Executives regularly praised the quality of reporting during board meetings.

Surely preparing a machine learning dataset would be straightforward.

It wasn’t.

The first inconsistency appeared less than a day later.

Marketing identified approximately 2.8 million customer acquisition records.

CRM contained 2.6 million customer profiles.

The finance department reported 2.5 million completed applications.

None of the numbers matched.

Initially, the engineering team assumed that different business filters explained the discrepancy. That would have been entirely normal. Different departments often work with different operational definitions.

Unfortunately, that explanation survived only a few hours.

As analysts continued comparing datasets, the inconsistencies became increasingly difficult to ignore.

Thousands of customers appeared multiple times under different identifiers.

Historical campaign parameters had changed without preserving previous values.

Several source systems stored timestamps in different time zones.

Some applications existed in transactional databases but had never appeared in analytical exports.

More concerning still, the target variable—the very outcome the model was supposed to predict—was defined differently by three business departments.

From a reporting perspective, none of these issues had caused major problems.

Dashboards continued functioning.

Financial reports remained accurate.

Marketing campaigns continued operating successfully.

Machine learning, however, has remarkably little tolerance for ambiguity.

A dashboard can survive imperfect data because humans naturally compensate for inconsistencies.

A predictive model cannot.

It faithfully learns whatever relationships exist inside the data, including mistakes.

One of the senior data scientists summarized the situation during the weekly steering committee.

“We don’t have an AI problem.”

He paused before continuing.

“We don’t even have a data quality problem.”

The executives looked confused.

“What we have,” he explained, “is a data definition problem.”

That distinction would eventually save the company almost a year of engineering effort.

The following week, the AI project was officially paused.

Several executives worried that the initiative had failed.

In reality, it had just avoided becoming an extremely expensive success.

Because a machine learning model trained on inconsistent business definitions is entirely capable of producing accurate predictions.

They are simply predictions about the wrong reality.

Data Quality Wasn’t the Problem. Business Definitions Were.

The steering committee expected the data science team to present the first predictive model within six weeks. Instead, they received a recommendation that surprised everyone in the room.

Do not train the model.

For a few moments, nobody spoke.

The project had already received executive approval. Budget had been allocated. Google Vertex AI environments were configured, data pipelines had been prepared, and the business had started discussing how predictive scoring would improve conversion rates and reduce acquisition costs.

From the board’s perspective, the project was moving exactly as planned.

From the data scientists’ perspective, it had not actually started.

The lead data scientist projected a single slide onto the screen. It contained no diagrams, no machine learning architecture, and no accuracy metrics. Instead, it showed one customer journey represented by three different systems.

Marketing considered the customer converted after the application was submitted.

Sales considered the same customer converted after a sales representative completed verification.

Finance recognized the customer only after the first successful payment.

Each department had valid reasons for its definition. Marketing measured campaign effectiveness. Sales measured operational performance. Finance measured realized revenue.

The problem was that the machine learning team had been asked to predict one outcome.

Unfortunately, the organization had three different versions of that outcome.

The discussion immediately shifted away from algorithms.

One executive asked whether the data simply needed additional cleansing.

The answer was no.

Cleaning data removes errors.

It does not resolve disagreements.

The datasets contained remarkably few technical defects. BigQuery tables were complete, ingestion pipelines were stable, and Dataform transformations consistently passed quality tests. Missing values represented less than one percent of observations. Duplicate records had already been addressed through existing reconciliation processes.

From an engineering perspective, the platform was healthy.

From a business perspective, it lacked a shared definition of success.

That distinction is one of the least understood aspects of machine learning projects.

Traditional reporting tolerates semantic inconsistencies surprisingly well because humans naturally interpret context. An experienced executive understands that marketing revenue and accounting revenue are different concepts. A machine learning model does not. It simply learns statistical relationships from the data it receives. If the target variable represents three different business meanings, the model will faithfully optimize against all three simultaneously.

The result may produce impressive technical metrics while delivering disappointing business outcomes.

To demonstrate the issue, the team performed a simple experiment.

They trained three identical models using exactly the same features.

The only difference was the target variable.

The first model predicted marketing conversion.

The second predicted completed customer onboarding.

The third predicted first successful payment.

All three models achieved similar validation accuracy.

None of them selected the same customers.

That result completely changed the executive discussion.

The problem was never model performance.

The problem was deciding what the business actually wanted the model to optimize.

Instead of continuing development, the CTO paused the implementation and created a cross-functional working group involving Marketing, Sales, Finance, Risk, and Data Engineering.

For nearly three weeks, nobody discussed algorithms.

They discussed definitions.

By the end of the workshops, the company had produced something far more valuable than an AI model.

It had created a formal business glossary defining every critical customer state, ownership for each metric, acceptable data sources, and the conditions under which those definitions could change.

Only then did the machine learning project resume.

The second implementation looked surprisingly modest compared to the original proposal.

No additional cloud services were introduced.

No expensive infrastructure upgrades were required.

The engineering team simply rebuilt the training dataset using the newly approved business definitions. Vertex AI trained the model exactly as originally planned, but this time every prediction referred to a single, universally accepted business outcome.

Six months after deployment, the model increased funded customer conversions by 11 percent, while marketing acquisition costs fell by approximately 8 percent. More importantly, no department questioned the predictions because everyone had participated in defining what success actually meant.

Looking back, the data science team reached an interesting conclusion.

The project had not been delayed by artificial intelligence.

It had been delayed by organizational ambiguity.

Ironically, the most valuable deliverable was not the machine learning model.

It was the agreement that made the model trustworthy.

Executive Takeaways

• Successful AI projects begin with business definitions, not algorithms.
• High-quality data cannot compensate for inconsistent business terminology.
• Machine learning optimizes whatever target it receives—even if that target is poorly defined.
• A business glossary is often a higher-return investment than a more sophisticated model.
• Cross-functional agreement is a technical dependency, not merely a management exercise.

One Question Every CTO Should Ask

“If we asked five department heads to define the outcome our AI model should predict, would we receive one answer—or five?”