Estimate project
← All posts

Getting better at discussing ML with people - excerpt from an interview with one of our ML experts

Several months back, one of our machine learning experts was invited as a guest to a machine learning podcast.

11 min read
Machine LearningPodcasts

Several months back, one of our machine learning experts was invited as a guest to a machine learning podcast. In the episode, he talked about how to discuss machine learning with others and gave advice on how to get better at it.

We have prepared an English excerpt of the conversation, which you can read below. Enjoy!

‍
Hi! My guest today is a machine learning engineer. Let me begin by asking what your role in your company is, what vision you have for it, and what interesting work you do there.

Hello, it’s a pleasure. Incidentally, I also lead a machine learning company that grew out of a larger software house. There’s a long story behind how this company came about and what the idea behind it was. It started as a spin-off, set up by a few software developers. Today we’re a small part of the company with a research mission. The main concept is to carry out interesting research and take part in projects that use machine learning while also being connected to different kinds of primary sciences. Naturally, on top of that, you have to mix in some commercial projects to finance it properly. I like to say that I want to seek out projects that really change the world around me. And I’m on the lookout for good projects that I believe are genuinely needed.

So we mean a GUI along the lines of saving elephants, redirecting rivers and so on?

But would rerouting a river actually be a good idea? No, most likely not. I have personal interests connected with remote sensing - examining the Earth through satellite imagery. Those are the kinds of nature-related topics I’m after in my work. I won’t claim there are lots of such AI projects, though we’re fortunate to have some and to keep working on them. Starting from the initial idea when we launched a medical project, up to now when we’re also doing a pro-ecological project, it’s achievable. But I’ll be honest, it’s extremely hard.

Projects on detecting cancer sound fantastic. Could you say a bit more about them? Up to now, I haven’t heard of anything that went all the way from research and model building to an actual computer running in a hospital and doing the job. It seems really hard to get past that barrier.

Because of NDA agreements, all I can say is that the project rests on a strong scientific foundation, and its core aim was to get through the FDA audit in the United States – which it did. That means it’s a medical device that’s already in production. The company had worked on this kind of thing before and funded it. They had excellent groundwork for building such models and amazing infrastructure. The research problem was well defined, and a team of specific experts had been assembled. Our job was simply to find a smart solution to that problem. Which is exactly what we at our company are best at and enjoy the most.

‍It must be very convenient to build only the part you enjoy and understand, or to get to know one another as you go, without sitting through 10,000 meetings with people who are either explaining things to you or trying to figure out what they want from you.

This stage – let’s call it incubation – is the one I love. And I wouldn’t say I’m a big fan of meetings. That said, there are also intense, inspiring meetings meant to work through the problem and come up with hypotheses. At our company, that’s one part of the overall process. But in general, when we begin talking with clients, we propose meeting them for roughly a week before we do any group concept. We call this workshops. Over that week, we discuss, go through the literature, and consider what the current state of knowledge on the subject is. This approach hasn’t let me down so far; for as long as I’ve been in this industry, the workshops have really worked. Once people start talking to one another, things begin to make sense, and from there it gets easier. For instance, during such a meeting they build a shared vocabulary. Everyone knows the week will be very intense, and they come prepared. Quite often, clients show up to the workshops having already read the literature. That’s the ideal scenario. The workshops are extremely effective.

‍ All right, but there are also meetings where lots of people from different levels of the organization try to settle what they actually want to commission from this Data Lab, or whatever we want to call it. I wonder what drives the client. Is it a rate high enough that it pushes them to read the literature, knowing this week costs X, Y, Z, so they’d better move quickly? Or is there some other reason, and the process really kicks off once you’re there?

Money clearly matters here. But right at the start, whoever we’re incubating the project with, we first have to infect them with the fascination we feel – so that they share the same motivation. Take me: I’m very much into scientific research. When I know I’ll be running a workshop with someone and we’ll be tackling some extremely hard problems, often in a field I knew nothing about beforehand, and that I’ll have plenty to learn, I’m already hugely motivated. My sense is that people catch this motivation, and that’s what matters most – people come in thinking, okay, this person is on his way, and now it’s going to be tough. He won’t let up; even though we’re his clients, he’ll ask a lot of us. On top of that, we know it’s a week with a fitting rate. It’s not over the top, but people feel it’s a week they’re investing in. And I’m glad everyone prepares really well, because then I feel I need to prepare even better. It’s a spiral of preparation. And then the discussion is a good one. I once ran a fantastic workshop with an expert in his field, also in medical diagnostics. He conveyed his passion in the way he spoke and was so well prepared that the discussion was really strong. We had to be pushed out of the room. But something good came out of it.

‍Could you tell me more about that? I’m curious where you start and at what point you say, okay, we’ve got it. Do you teach people what machine learning is? How does it go?

Right at the beginning, we do the usual business work with the client and discuss the basic terms. Then, when I arrive at the workshops, I split them into suitable sections depending on what kind of client it is. If it’s a situation where nobody knows whether ML is the right choice, the workshop opens with a discussion of what machine learning is, how it’s different from other approaches, and whether the client really needs it. Sometimes they don’t, and it’s our duty to say so plainly, because not every problem has to be tackled with machine learning algorithms. That’s usually day one. Day two is for asking questions. I script that conversation so I can extract as much information as possible from the potential client and understand their domain. I also prepare beforehand - I go through the literature and see what solutions exist right now. After that conversation, I have enough information to decide whether a Proof of Concept is necessary. Next, I show the client the first possible solution, the simplest one. After that first part, I may say - okay, this is well-known territory; there’s no need for a Proof of Concept. That’s a comfortable position to be in. If you’d like to run a project, we can begin right away and put a team together. But at times a Proof of Concept is necessary, because the knowledge hasn’t been widely covered in science and it’s unclear whether the thing can be implemented at all - and I tell the client that plainly.

‍This magic Proof of Concept – ask 30 people in our industry and you’ll get 30 entirely different definitions. That’s what I’d like to get to the bottom of. And those questions you mentioned, what do they cover?

My approach is to look into the areas the company I’ll be working for is active in, but also what sort of team I’ll be working with, who can help me with the task, who’s involved, who’s already an expert, what knowledge exists, what the background of the project is, and where the idea originated. To work together well, you have to fully understand the person you’re working for and what they intend – why they want to build this in particular and not something else. It definitely makes later conversations easier, and you can quickly put together a glossary of terms. As you say, we each understand Proof of Concept differently, so we have to agree on the terminology. As for what a PoC means to me, I treat it as a scientific process: it defines a research problem and verifies hypotheses. So I give the client the research problem, the hypotheses behind it, which ones were confirmed and which rejected, and then what we’ll do with it next.

‍And that’s exactly why I wanted to pin it down. People often mix up PoC and MVP, and there’s plenty of confusion around it. Some treat the two abbreviations as interchangeable, but if you think it through, they aren’t.

Precisely, and I think it’s also crucial to define what we’re going to do and walk through the whole process. That’s one of the ideas we bring to the workshops. On day one, we really do talk about how we work and what we’re going to do next. What does PoC mean? What does MVP mean? What does working remotely mean? That’s also a very, very interesting idea nowadays. The key thing is that everyone knows this glossary, so collaboration goes more smoothly.

‍You mean teaching people about machine learning? What’s your experience there? Because it’s hard to explain to someone in a few hours how this field works – to move them from the view that everything can be planned with a known duration to the view that after some time I’ll tell you whether it can be done, and only then will we build it. Many people don’t find that obvious, or at least that’s how it seems to me.

I think you’ve made a really good point. The first step is to draw a clear line right away between classic development and machine learning development. In the former, we can roughly predict when the project will be done. All the building blocks of a given solution are fairly well understood. In ML, however, even though the blocks are known, the “car” isn’t - nobody knows what the outcome will be. That’s the stage where you need to tell the story over and over, and the simplest way is through an example to be built. That’s what these workshops are for me – afterwards I roughly know what we have to do, and I tell them what data we’ll need. Admittedly, I may be downplaying modeling or writing the machine learning model a little, because I’m simply saying - here’s the data we have, this data contains some noise; you also have to understand that noise, because it will carry over into further solutions. And it still might not work, because I can’t promise you otherwise. And of course the first question is always: how much data do I need? This is machine learning; this is research. I can design a research project so that at some point I’ll be able to tell how much data we need. But not right at the start; that’s often the case. The more open you are about how things work, the better - that way there are no surprises down the line. We run all PoCs and MVPs at a fixed price, because it makes planning easier, and I believe that’s also extremely important. We don’t say we’ll finish this ML at some unspecified point; instead we set an exact period, say half a year, during which we’ll work. Then, after those six months, I show the client the outcome, but it may differ from what they hoped for. It’s still research, and we have to make sure they understand that. On the other hand, following the fail-fast principle, let’s move quickly and communicate a lot so we know where everything is going. Yes, I believe being honest is a key part of teaching people about machine learning.