Estimate project
← All posts

How can you check whether your machine learning use case idea is viable?

This post explains how we can join forces to validate your machine learning use case with a holistic, scientific approach, so you can be sure the available resources and technology are put to the best possible use.

10 min read
Machine LearningArtificial IntelligenceMachine Learning Workshop

Every innovation effort and every production rollout calls for risk mitigation, and Machine Learning projects bring their own particular difficulties in this respect. Classical software engineering can be demanding, yet it is usually more predictable and easier to plan and execute; machine learning, by contrast, comes with many factors that are simply unknown.

Projects in machine learning tend to be more complex, carry more uncertainty and are, on the whole, riskier. Yet when they succeed, the payoff can be significant, fuelling innovation in many different industries.

Suppose you are a visionary entrepreneur about to start a company that, in your view, would gain from AI and machine learning technology – today’s limitations should not put you off. Rather, put your energy into figuring out how to make your ideas work. Keep in mind, though, that machine learning project ideas frequently fail not because of errors in data processing, ML algorithms or the choice of model, but because the right resources, expertise or strategy were missing from day one.

A visionary enterpreneur by Midjourney

Starting a project the right way is critical to whether it succeeds. When machine learning experts help you establish whether your project can really be delivered with machine learning techniques, you gain the confidence required to go ahead.

This post explains how we can join forces to validate your machine learning use case with a holistic, scientific approach, so you can be sure the available resources and technology are put to the best possible use.

What is the point of validating your ideas?

Tech startups all over the world are applying machine learning and artificial intelligence to problems nobody had solved before. AI projects are attractive, and early movers holding high-quality training data that allows accurate predictions stand a good chance of success – although at the outset this is not always obvious.

That is why the first step is asking: are we using ML for the right problem?

Whatever software you build, you want to stay lean and cut risks as early as possible. You know better than anyone that development takes both money and time. The last thing you want is to commit to a full-scale software project and discover only at the end that the investment should have been made in another way.

In classical software engineering, an idea that comes to mind can be assessed for feasibility against the current state of the art. You break it down into architecture and system components, then check how it could be built. Machine learning is a different story, since it involves many unknowns. The outcome hinges on a range of factors, such as the quality of the data and the concept underlying the model. It may turn out that the suggested solution just won’t work – because there is not yet any technical way to obtain data with strong predictive power, because the problem is so complex that not even a neural network can help, or because it would demand computing capacity that cannot be obtained.

This is why business owners frequently come to us to have their ideas checked and to get answers. An immediate answer is not always possible. At times research or an experiment has to be carried out. Our aim, therefore, is to bridge the scientific and business worlds in ML projects and build a roadmap for the project’s viability. To that end, we have defined three concrete milestones:

  1. Machine Learning Workshop
  2. Machine Learning PoC
  3. Machine Learning MVP

Below we go through each of these steps and explain what it is meant to achieve.

Machine Learning Workshop

Is your business idea promising from the perspective of AI and machine learning? In reality, ML is not the best answer to every problem, and if you can reach your goals in a better, simpler way, we will let you know.

In AI and machine learning it is tempting to get carried away by enthusiasm for the newest, most advanced algorithms and models. But however impressive a technology is, it will not deliver if it is aimed at the wrong problem.

Over a 3-to-5 day workshop we make room for business and scientific ideas to meet, so we can find the most suitable route to results for you. The aim is to align the problem with a solution and determine whether your idea is ready for a business PoC, or whether it needs more research or could be handled with more conventional technology.

What the workshop is like

The workshop is fairly intense and resembles an interview. Science is familiar territory for our engineers – every one of them either holds or is pursuing specialized education in a scientific discipline.

Our research-driven approach rests on the conviction that understanding our clients is the key to success. The workshop is built to help everyone get to the very root of the problem. We achieve this by:

  • posing the right questions about where your idea came from,
  • getting to know your company and its position in the market,
  • pairing your needs with what science and machine learning currently have to offer.

At the end of day one, you will get a list of research problems drawn up from our findings. It becomes the starting point for further investigation. Based on that list we may go on to talk about areas of concern, aspects to improve, obstacles to remove, or difficult questions present in scientific theory or in practice.

With a clear view of the problems in front of us, we plunge straight into analysis. We tackle the problems one by one and jointly work out ways to overcome them. This is also when our machine learning engineers pass on their extensive expertise, giving you insight into the world of machine learning. As a result, each side gains a better grasp of the other’s point of view and together we arrive at a stronger solution.

Who runs the workshop

The workshop is run by a team of seasoned machine learning engineers who are experienced in doing research, testing hypotheses and drawing empirical conclusions. Their know-how spans both academia and industry.

A workshop session with one of our clients

The specialists for the machine learning role are chosen according to the domain and the technical skills required. If your case involves image recognition, for instance, a computer vision specialist will take part, and if it involves text, an NLP specialist will.

Our team will walk you through the fundamental concepts of machine learning, for example: how machine learning differs from deep learning, what artificial intelligence is, what machine learning algorithms and machine learning models are, and what computer vision or natural language processing (nlp) means. All of this helps turn your idea into the language of process requirements based on machine learning.

What the workshop aims to achieve

At the end of the 5 days you will know whether machine learning is right for you, or whether a better, simpler path to your goals exists.

You will be given a Workshop Report containing:

  • A description of your project in light of our findings
  • The engineering competences the project requires
  • A concept, plan and time estimate for the Proof of Concept stage
  • An initial machine learning architecture for your project

We aim for complete transparency and share the results as soon as they emerge, so that the earlier approach can be adjusted in various ways. Incremental development that tests whether an idea is worth pursuing matters in any software project, but in machine learning applications it is absolutely essential.

If machine learning is not the right fit for your project, we will say so.

Machine Learning PoC

Once the Workshop confirms that your idea suits a machine learning implementation, the next step is to test it by training a model. A successful proof-of-concept stage depends on having the right skills and data together with a meaningful conversation between the business and the engineers. That is all it takes – and everything it takes – to keep models accurate over time, so that a PoC turns into real solutions running in production.

What the PoC is like

The Machine Learning PoC stage takes between three up to six months. Throughout this period we meet every three weeks, and our engineers present the progress achieved since the previous meeting. Working this way allows us to focus on reaching the right conclusions for more accurate predictions and on a shared understanding of the progress. Thanks to open and regular communication, we can change direction quickly and decisively whenever it is needed.

To give you real visibility into our work, we rely on experiment tracking tools such as Neptune.ai or Weights and Biases. These excellent tools keep everyone informed while ML experiments are running. Picture it as looking through an oven door to see how a tasty pie is baking. Neptune works in just that way – it keeps you updated and lets you watch what is cooking in our ML model. We want you to be able to look at our work whenever you like and on your own terms.

A view from neptune.ai -> see the quickstart docs

Which data is used during the PoC stage

That varies, and it is settled at the workshop stage. Usually we are aware of existing sample data sets suitable for our experiment and can recommend them, but sometimes data has to be gathered. In that case we include this work in the PoC stage and help you collect it properly by supplying a suitable collection methodology, setting up supporting software and even putting together an annotation team.

What the PoC aims to achieve

Our goal is to identify the right data – in sufficient quantity. In choosing the data sample for our proof-of-concept (POC), we concentrate on a fairly small yet sufficient data set that lets us run into and examine as many problems as we can.

This approach gives you confidence in the quality of the data and of all the other data science and ML work for your specific use case. You will also gain experience in talking with ML/AI teams, along with the ability to set requirements and check their work.

Once the proof of concept is complete, we will determine whether applying machine learning tools and techniques in your case is worth enough to justify what the whole project will cost.

You will be given a Roadmap containing:

  • An estimate of the cost and time needed to solve the problem in production
  • The engineering competences the project requires

Machine Learning MVP

If your idea has made it this far with us, it is time to get to work and build the software. For the MVP we apply a fixed-price model and keep things budget-friendly, so you know in advance what your minimum viable product will cost.

What the MVP is like

We define the modules of your system and the competences the project calls for, and we set out a roadmap and the costs. Our machine learning engineers bring the required know-how and blend smoothly into your team. Since we also have experience as software engineers – our parent company is a software house – we can either support a software development team or build the whole software ourselves.

Our Ops experts can boost developer productivity with carefully chosen infrastructure automation tools. With a multidisciplinary team like this behind you, you can concentrate on marketing your business and welcoming your first clients.

What the MVP aims to achieve

A successful MVP lets you move on with confidence to building a full product.

Summary

Testing your idea is worth it, particularly in a field as innovative as machine learning, where the flood of new algorithms can feel overwhelming. We can help you find your way through this landscape and make well-informed technology decisions. Through our work with clients we have built an effective process in which we provide a place where business and scientific ideas meet so that machine learning technology can be defined or deployed.

Would you like to try our workshops, Proof of Concept and MVP steps in your project? Get in touch and we will gladly apply science together with you!