Estimate project
← All posts

The 6 Biggest Business Misconceptions About ML Projects

In this article, I try to show what a machine learning project really looks like through an engineer's eyes, and which problems clients typically run into. It looks at those problems, at what goes wrong, what catches us off guard, where the genuine risks lie, and how to lower them.

18 min read
Machine LearningML projectML project management

Here is how I picture the thinking of some clients I have worked with:

“We’ve got the data, we understand the problem – just build us a model. We could even use AutoML. Once the model is there, the job’s finished. We’ll push it to production and it’ll run by itself for the next few years.”

Sadly, things are more complicated than that.

Machine learning is not an Excel plugin. It is not a plug-and-play part you buy, install, and then use for years with no effort. It is a living system that depends on data, shifts over time, and needs continuous improvement and updating. It calls for patience, experimentation, and a healthy dose of humility about what we (still) don’t know.

In this article, I try to show what a machine learning project really looks like through an engineer’s eyes, and which problems clients typically run into. It looks at those problems, at what goes wrong, what catches us off guard, where the genuine risks lie, and how to lower them.

If you are thinking about bringing AI into your product or planning to work with an ML team, this article may help you steer clear of typical mistakes. Or at least help you see why the model did not appear by magic after a week.

Myth 1: AI can solve anything

Interest in AI tools has exploded with the arrival of systems such as ChatGPT and Midjourney. They are fast, often produce impressive results, and feel almost like magic. Small wonder that some business teams now treat AI as a “one-size-fits-all solution.”

As a machine learning engineer, though, I regularly meet clients with unrealistic expectations. They want a model that predicts everything, never errs, responds instantly, copes with every scenario, and adapts by itself. The trouble is - models like that do not exist.

A machine learning team’s job is not only to build a model, but also to educate clients about where machine learning makes sense and where it does not: where it can bring real value, and where classic data analysis or even a simple rule in the system would serve better.

Knowing these limits is part of what makes an ML project succeed. As technical partners, we are responsible for pointing out those boundaries and helping to find the best solution available, even if it is less spectacular but far more realistic.

Myth 2: Just dump the data in and ML takes care of the rest

One of the most widespread beliefs about machine learning sounds like this:

“We’ve got data, so we put it into a model, and the model will learn and solve the problem.”

Seems reasonable, doesn’t it? Data is the “fuel” of AI, after all - so the faster we pour it in, the faster the engine starts. But to stay with the engine comparison: if we use poor or contaminated fuel, the engine breaks down and we go nowhere. This is why “garbage in, garbage out” is such a common saying in ML. Train a model on low-quality data and you cannot expect good results.

Where does this myth come from?

In conventional IT projects, data tends to be clean and structured: forms and databases with a defined format and source. In machine learning, however, data is not only about correct values in cells - it is also about meaning, logic, and how the data came to be. The model learns straight from what it receives, so the data must genuinely reflect the process or problem we want to model.

So what should you do instead of “just dumping the data in”?

  1. Begin with a simple question: What precisely should the model predict? Can it be measured? Do we have historical examples of it?
  2. Pull a small sample (50 to 100 records) and go through it by hand. Are the labels consistent? Do the columns make sense? Do values contradict each other? Do we actually have this kind of data? Do we have the variable we want to predict?
  3. Schedule a small data audit before the project kicks off. The machine learning team can supply a checklist: value distributions, missing data, correlations, and label quality.
  4. Bring in business experts to explain the data. Only people who know the project can tell you things like: “This column only matters for Region A”, or “these records predate our system change.” Without that context, understanding and making good use of the data becomes much harder.

Myth 3: More data = a better model

Once we accept that data matters, it is tempting to take shortcuts:

“We’ve got loads of data, millions of records. Okay, some of it may be messy, but the model will sort it out. More data is always better. Isn’t it?”

No. In machine learning, piling on more poor-quality data does not make your model better. It can actually make it worse.

Machine learning models search for patterns in data. If the data is noisy, inconsistent, or wrongly labelled, then however large your dataset is, it will simply be a huge pile of useless examples.

In two of my recent projects, clients proudly handed over enormous datasets meant for training the model. We soon found that the data was inconsistent, riddled with wrong labels, and in the end unusable without serious cleaning.

Data: the greatest risk in any ML project

From what we have seen, most of the time on machine learning projects goes not into building models but into understanding and repairing the data. A survey among ML experts found that this stage can account for up to 80% of the total effort. It is also where most misunderstandings arise.

Typical data problems we run into:

  • Incomplete records - e.g., 40% lack key information such as customer decisions.
  • Inconsistencies - one column reads “status = canceled,” another “status = closed.”
  • Misleading labels - a “success” label could mean “offer sent” rather than “product sold.”
  • Missing context - nobody remembers why business rule X was introduced, so 1/3 of the data is skewed.

So which matters more: quantity or quality?

People often ask:

“Which is more important: the quantity or the quality of data?”

My advice is to put quality first, although quantity matters as well. Here is a real example.

In one project, a client supplied a large dataset for training a predictive model. On the surface it looked excellent: plenty of records, lots of variety, reasonable coverage. A quick audit, however, revealed missing values and poorly defined labels.

Instead of a lengthy debate about why this was a problem, I chose to demonstrate it. We picked a small, carefully chosen subset, under 1% of the data, and checked and fixed it by hand. Then we built two models:

  1. One trained on the entire, messy dataset
  2. One trained on the small, cleaned subset

We evaluated both on the same cleaned test set. The outcome? The model trained on clean data clearly beat the bigger one.

This little experiment left a strong impression on the client. It proved an essential point: better data wins over more data.

This matches what AI expert Andrew Ng says: in many use cases, only a few dozen high-quality examples may be enough to build an effective model. That is the core of data-centric AI - putting effort into better data, not just larger models, frequently gives the best outcome.

How data quality relates to drops in machine learning algorithm performance has been examined in depth in the scientific paper “The Effects of Data Quality on Machine Learning Performance on Tabular Data.” Another study, on training small language models, shows that the quality of data, not its quantity, is what drives strong results.

What poor data quality costs in practice

When nobody pays attention to data quality:

  • The model picks up misleading correlations - such as “customers whose IDs end in 7 convert more often.”
  • You lose time and money - training runs longer and debugging gets harder.
  • Time-to-market grows - fixing problems later means doing work twice.

Why is this myth so widespread?

  • In traditional analytics, more data often really does help. For instance, four times as much training data in linear regression can halve the error - but only when the data satisfies the model’s assumptions.
  • People forget that marketing examples often rely on carefully selected, cleaned datasets (e.g., for training large models such as ChatGPT or Falcon).
  • Collecting more data is usually easier than reviewing it, cleaning it, and really understanding it.

What does a modern, data-centric approach involve?

The best machine learning teams today follow data-centric AI. In practice:

  • A simple model can perform well when the data is high quality.
  • Rather than collecting more data, we improve the data we already have - removing noise, correcting labels, and aligning definitions.
  • Cleaning data beats hoarding it - a smaller set of cleaner examples often does better than a bigger but flawed dataset.

Myth 4: ML works the same way as classic IT

Lots of clients come to machine learning projects thinking in terms of traditional software development. That makes sense - we have been building software for years with repeatable frameworks:

“We write the requirements, split the work into sprints, plan the releases, run a demo, and it works.”

That works very well for building an API, a dashboard, or a customer support system. Once an ML project starts, though, this approach soon breaks down. Why?

Because machine learning is not about writing logic, it is about finding patterns in data. And that is a completely different game.

ML is research, not merely implementation

In machine learning projects:

  • We cannot know beforehand which model will work - we need to experiment and understand what the data is like.
  • We cannot estimate the effort precisely - results depend on the quality of the data, not only on clean code.
  • We do not implement the logic - we train the model to learn it. And the model may learn it incorrectly, even when the code is flawless.

I once ran a project where the client expected “a prediction demo in two weeks.” Two weeks in, we had five hypotheses explaining inconsistencies in the data, three preprocessing ideas, and no working model. Not because the team was inefficient or lacked skills, but because in machine learning, before anything can work, you have to understand what you are dealing with.

Where traditional IT approaches fall short

Success is not guaranteed

A machine learning project will not always succeed, particularly on the first attempt. When the problem is complex, the starting assumptions may be wrong. The data gathered may be insufficient, inconsistent, or just too limited for machine learning.

Sometimes it is enough to reformulate the problem, re-examine the assumptions, prepare the data more carefully, and try again. In other cases, the problem is too complex for ML to handle efficiently.

Most of the time goes into data quality and processing

Clients usually want a working machine learning model as soon as possible - understandably, since time is money. But skipping the phase of checking data quality and understanding the data reduces the odds of success.

Refactoring code is a normal part of a software development project’s lifecycle. So why is “data refactoring” – analysing and improving data quality – still seen as optional? Because people do not grasp how complex machine learning development is or how much the project’s success depends on the data.

Clients arrive saying they have data and expect us to simply “train the model and finish.” That is not how it works.

Skipping a thorough data analysis and going straight to model training results in poor performance, then debugging, which eventually leads back to, as you might guess, the data. It pays to invest early in validating and understanding the data. If serious problems come up, we can pause the project, fix the data, and then continue with confidence.

Other project risks

In classic software projects, risks can usually be spotted and reduced early, during requirements gathering and prototyping, because e-verything centres on functionality and use cases. In ML projects, by contrast, the main risk is the data. Until an engineer has seen and analysed it, they are working on assumptions.

Unpredictability

Estimating how long it will take to “fix” the data or tune hyperparameters is hard. Even after an encouraging data analysis, the model may fail because the data does not cover enough cases or has little predictive power. That frequently means going back to data collection and labelling and starting over.

Data can turn out to be a real Pandora’s box. It may look fine at first sight, but a closer analysis can uncover gaps and errors. This is a serious project risk - and one that is hard to foresee without a proper deep dive.

Serious data problems can sharply lower the odds of project success, or even rule out building a model at all. The project may have to be put on hold or halted completely until the data is fixed, and it is often impossible to say how long that will take.

Iteration in R&D style

Rather than the classic “to do → in progress → done,” ML projects tend to look like this: “Tried → didn’t work → trying again another way.”

As I said earlier, data that looked solid may prove faulty, or the model architecture may turn out to be a poor fit. This results in a loop of experimenting and correcting.

Even hyperparameter tuning is an iterative process that is tightly linked to the training dataset.

No conclusive tests

ML has no unit test that declares: “The model works.” There are statistical metrics, and they need to be interpreted.

A different cost curve

In software projects, costs peak during development and fall after deployment,and only maintenance is left.

Blog%20post%20graph%2001

ML project costs do not follow that curve. They frequently stay high or even grow over time, because:

  1. Models deteriorate and have to be retrained,
  2. Data shifts and has to be analysed again,
  3. New models have to be built to match a changing reality.

So what can you do?

Approach ML as an exploratory project

Think in cycles of: hypothesis → experiment → analysis → decision – exactly as in R&D.

Manage uncertainty, not only tasks

Ask yourself: What are we not sure about? Where do we have to validate the data? What might go wrong?

Plan a realistic timeline

Build in phases such as data exploration, experimentation, validation, tuning, and integration.

Add a buffer. Then add one more.

Manage expectations

Report progress clearly and frequently:

“The model isn’t performing yet because we found issue X in the data. We’re working on it.” Rather than vague promises such as: “The MVP is nearly finished.”

Myth 5: ML model = ML product

Plenty of clients, including very technical ones, believe that once a machine learning model performs well, the problem is solved. Their question is:

“We’ve got a model and it performs well. When can we put it into production?”

At first sight this seems logical: if the model does well on the test set, we should be able to “plug it into the system” and be finished. Sadly, it does not work that way.

An ML model is the heart of the solution, but on its own it is not a product. It has no lungs, no nervous system, and no resilience to changes in the dynamic environment around it.

What turns it into a real ML product?

Picture the ML model as an engine. Even if it runs flawlessly on the test bench, it will not take you anywhere without other parts. For a car, those would be the gearbox, wheels, brakes, and so on. For an ML product, they are:

  • A production environment (where the model is deployed and can be reached),
  • Input and output interfaces (how data gets into and out of the model),
  • Business logic around it (e.g., when the model should or should not be called),
  • Real-time monitoring (is it performing well today, not just last week?),
  • Re-training mechanisms (since data shifts and models go stale),
  • Version control for data and models (what was trained, and on what data?),
  • Deployment infrastructure (CI/CD, APIs, availability, security).

Without all of these tools, the model stays a proof of concept rather than a production-ready solution.

What happens when this myth survives?

  • The model goes live but stops working properly - because the data shifted and nobody was watching.
  • Users cannot make sense of the predictions - because interpretability was never addressed.
  • The team does not know how to retrain it - because there is no version control for the data and the training code.
  • The model breaks on real-world inputs - because production APIs differ from the test data, or the data has drifted and the model is out of date.

How do you get it right?

  • Plan the ML product as a whole - starting on day one

Don’t only ask: How will we train the model?
Ask as well: How will we deliver it, integrate it, maintain it, and improve it?

  • Settle who owns the production phase

The ML team? The DevOps team? The client’s IT department?
This matters more than you might think.

  • Make sure monitoring and interpretation are in place

What does a prediction of 0.87 mean? Is it confidence? Has the model deteriorated?

  • Accept that the model is just one component

Even the finest algorithm is worthless if it cannot be used in a real-world process.

Myth 6: Adding new functionality will be easy

Now and then, in the middle of an ML project, a client may say:
“Hmm, what if instead of predicting X we tried classifying Y?” Or: “Let’s change the model, maybe a neural network instead of XGBoost?”

In traditional software projects, a change like this is no big deal. You adjust the logic, modify an endpoint, add a condition, and that’s it.

In machine learning projects, however, even a minor change to how the problem is defined can mean rebuilding half the solution from the ground up.

Why a change in ML = starting over

Machine learning depends on data and a clearly defined target variable. If you change:

  • what you are trying to predict,
  • what counts as a “positive” class,
  • which features affect the decision,

…you are not just changing the method - you are changing the problem itself. That brings:

  • new data (or a different way of processing existing data),
  • new feature engineering,
  • new labels,
  • a new model architecture,
  • a new evaluation method,
  • new risks and success metrics.

It is like a client deciding they want an open-plan kitchen once the cabinets are already fitted. Can it be done? Of course. Is it easy? Not in the least.

An example from a project

In one project, the aim was to predict how aesthetically appealing an image was. Halfway through, the client added a requirement: predictions had to come with explanations so that users could improve their image based on what the model told them.

At first sight, this looked like a minor change - perhaps just one more module for explainability. In reality, it required:

  • choosing and designing model features that users could make sense of,
  • rebuilding the training dataset,
  • reworking the preprocessing and validation logic,
  • retraining and retesting the models,
  • and creating a new mechanism for generating suggestions.

The outcome? Three additional months of work and a step back to the data analysis phase.

Why does this myth hang on?

  • In traditional projects, a change frequently comes down to adding a condition to the code.
  • Managing ML projects is not intuitive. Adding model explainability sounds like a small feature, but it often amounts to redefining the problem.
  • Seen from outside, nothing changes: input, output, charts.

Under the hood, however, it is a completely different story.

How do you push back against it?

  1. Define the problem and the success metric clearly at the outset. What exactly are we predicting? How do we judge a “good” result? What matters most?
  2. Lock the problem definition during the experimental phase. New ideas are welcome, but they should wait for the next iteration. Keep a backlog and come back to it when the current phase is finished.
  3. Treat major changes as new projects. Even if it is “just a new metric,” check how it affects the whole pipeline.
  4. Bring the ML team into decisions. Not every pivot pays off. Sometimes changing the goal wipes out the value of everything built so far. The ML team can help evaluate ideas quickly and estimate the effort involved.

Managing an ML project - what should you be ready for?

Machine learning sounds attractive: automation, predictions, optimisation, “smart decisions.” And a well-built model really can deliver substantial business value. Before it does, though, there are a few things worth understanding.

An ML project is not a sprint; it is a trip into unmapped territory. Some things will go as planned. Others will need adjustments, experiments, and a different approach. Below are the things you should be prepared for - not to put you off, but to spare you surprises.

Not everything will work straight away

  • The first model version is seldom the last.
  • Experiments may make results worse before they make them better.
  • Sometimes it takes several iterations just to find out whether the problem can be solved at all.

ML is a scientific process, not just putting together ready-made pieces.

Preparing data takes time

  • Understanding, cleaning, and organising data will take more time than you expect.
  • You will need help from people who know the business context of the data - often from operational teams.

This is not only the ML team’s task; it is a vital part of the project.

ML might not work out

  • Sometimes the data is too sparse, too inconsistent, or simply not enough.
  • Sometimes there is no clear pattern in the problem, and a simple rule-based approach may do better than a model.

Machine learning does not solve every problem. And it is better to realise that early.

You will have to adapt

  • Changes to the goal definition, the data, or the project scope may mean going back several steps.
  • Even minor changes may require retraining the model, adjusting the infrastructure, or redefining how success is measured.

ML projects react strongly to change. That is not a defect; it is simply their nature.

Continuous supervision is a must

  • A model is not something you “set and forget.”
  • It has to be monitored, evaluated, and updated - because your data and business processes will keep changing.

ML grows along with your company. It must be treated as part of your system, not as a standalone “gadget.”

Wrap-up

Machine learning holds huge potential. It can support decisions, automate processes, and reveal patterns people cannot see. To actually reap that potential, though, you have to understand that ML projects are not like traditional IT projects.

Here is what I hope you remember:

  • A model is not a product. It only delivers value as part of a well-designed process.
  • Data is the foundation. Most ML challenges are not technical - they stem from the quality, availability, and understanding of data.
  • Not everything is predictable. ML projects are iterative and experimental and often call for revising assumptions along the way.
  • Working with an ML team is a partnership. You know your business, we know the tools - together we can create something genuinely valuable.
  • The best outcomes come from clarity and preparation. The most successful projects begin with a clear purpose and a readiness to take part in the process.

If you are planning a machine learning project, you do not need every answer upfront. It does help, however, to arrive with an open mind, a clear goal, and a willingness to collaborate.

Because ML is not magic; it is collaboration, logic, data, and some patience. And a well-run ML project can genuinely change the way your business works.