Counting colony forming units: an AI use case in life sciences
Life sciences make an ideal testing ground for edge Artificial Intelligence. To show this, we take on a very common task: counting bacterial colonies once an agar plate has been incubated. This post describes our straightforward experiment and shares a few of its results.
Introduction
These days it would be hard to overstate how much machine learning (ML) and artificial intelligence (AI) add to scientific research. Many fresh ideas are being actively investigated right now to make ML models more precise, more robust, and easier to use. Tailoring ML methods and techniques to the particular needs of individual scientific fields takes time, and plenty of research teams are interested in doing it. As researchers, we can take advantage of the most groundbreaking AI innovations to automate dull, repetitive work, handle much larger volumes of data, and take on more ambitious research.
Don’t misunderstand us. ML is hardly a newcomer to the scientific world. What has changed is that it is now simpler to use it to its full extent. For that we can thank the wide range of open-source software libraries, publicly available training datasets, great documentation and, above all, the ability to explain what an ML model produces.
Researchers like us are getting better and better at using information technology. A growing number of them can write good code. As a result, mastering computer-aided data analysis is now within the grasp of more research groups. Code and programming no longer stand in the way of carrying out interesting research.
ML has huge potential as a helper in everyday scientific work. Chances are you already rely on sophisticated systems built around learning from data. This is becoming easier thanks to the shrinking size of hardware that can run ML models. Of the options on offer, small devices such as microcontrollers are especially intriguing.
Computer vision (CV) is a perfect illustration. The ability to run even an elaborate vision model on a board no bigger than a matchbox opens up a lot of interesting possibilities. Add to that easily available hardware components, the backing of great DIY communities happy to pass on their know-how, and approachable programming ecosystems, and researchers have everything they need to build a sophisticated prototyping playground.
Life sciences make an ideal testing ground for edge Artificial Intelligence. To show this, we take on a very common task: counting bacterial colonies once an agar plate has been incubated. This post describes our straightforward experiment and shares a few of its results. You will also find a brief discussion section.
So grab a coffee, some Petri dishes, and a microcontroller. It’s a great day for science!
This post was written by a wonderful brother-and-sister duo – an assistant professor and a fellow researcher, both holding a Ph.D. – with help from their skilled young assistants.
Calculating bacterial number
Table 1. Kinds of solid media.
Anyone who has bought a probiotic drink has probably spotted “CFU” on the label. Counting Colony Forming Units (CFU) is the commonly used gold-standard method for establishing how many bacterial cells there are. A CFU is simply a single colony of bacteria, fungi, or yeast, and the count can be given per examined area or per volume, depending on what your experiment is for. In our case, we measured how contaminated various everyday objects were. We just pressed a chosen object onto the surface of solid media, holding it gently for around 10 sec. The plate surface works like a canvas for counting. After 24h - 36h of incubation at 25°C - 30°C (77°F - 86°F), colonies appear that differ in size, shape, and color – in other words, in their morphology.
Fig 1. Bacterial colonies after incubation; starting in the top-left corner: fingerprint, coin, mobile phone, air sample.
Planning
Our experiment was planned in several stages, made up of these tasks:
- Build a Do-It-Yourself monitoring platform on top of Arduino and Raspberry Pi – this step called for a fair amount of hardware wiring and additive manufacturing.
- Prepare agar plates and grow various kinds of bacteria – we collected the bacteria samples from ordinary household objects. Not for the squeamish 😅.
- Collect and label training data – taking photos and annotating bacterial clusters went without a hitch and, to our surprise, did not take much time.
- Train a machine learning object detection model and test it.
- Build a data analysis layer – we really wanted to demonstrate how object detection output can be put to use in later steps of a research process.
- Analyze bacterial growth using time series of images – this took a lot of time, but it was a rewarding part of the experiment.
- Prepare video material showing what we found.

Fig 2. Our experiment plan.
Methods
Monitoring device
We are big fans of what 3D printing offers for fast prototyping. After a few iterations, we settled on a plain semi-transparent jar as our monitoring device. It was easy to store and allowed us to try out different diffused light sources to give the camera sensor the best possible conditions. A few drilled holes kept the air flowing and the bacteria growing.
The solid lid was there to conceal a microcontroller and a camera. To make the cover sturdier, we printed it with graphene filament. Among the vision modules available, Arduino Portenta paired with a LoRa vision shield and the trusty Raspberry Pi with a camera turned out to be the best options, as they are small yet powerful enough to run a machine vision model. We used two different sensors to find out which approach would work better in a full-scale experiment.
Fig. 3a. The monitoring device – Arduino Portenta.
Fig. 3b. The monitoring device – Raspberry Pi.
Data collection and preprocessing
Fig. 4. How the data was prepared.
Preparing samples and collecting data was straightforward. We transferred bacteria onto the selected medium by pressing the object against the agar plate. Sometimes we used the object itself. Other times we wiped the object with a cotton swab and moved the bacteria to the medium that way. Next, the Petri dishes went into our monitoring containers.
Table 2. Description of the dataset. Ad 1. Collected from my daughter’s hands after a day in kindergarten – 100% biohazard 😆
The devices were set to snap a photo every 5 minutes and upload it to storage. The Ardunio jar recorded 320px x 320px grayscale images. The RGB images taken with the Rasberry Pi had a resolution of 2048px x 2048px.
Working in a home lab let us keep temperature and humidity steady. Getting the right lighting inside the containers required some extra setup. A mix of the built-in light sources and a shadeless photo tent solved that. It is something to keep in mind for the container’s next version.
While collecting data, we progressively disposed of older samples by covering them with bleach. Between capturing successive samples, we sterilized both the container and the lab with a UV lamp.
Once every sample was collected, we preprocessed the data to suit the machine learning training stage. First, we cropped the images with a fixed-size circle so that only the working area remained. Next, we turned the RGB images into grayscale. Color information is not needed in this experiment, and dropping it keeps the model simpler.
Finally, we ended up with two separate datasets: high-resolution time-lapses from the Raspberry Pi and low-resolution grayscale time-lapses from the Arduino Portenta.
Using annotation software, we also placed object markers (points). We began with the most basic scenario possible, marking only bacterial clusters and making no attempt to tell particular types or species apart. That can wait for more in-depth research backed by microscopic analysis and a spectral camera.
Model
Training a machine learning model for an embedded device was the stage of the project where the magic happened. Luckily, with a well-curated dataset and a dedicated platform, designing and deploying a model is easy. You can either pick a ready-made model offered by one of many vendors or build your own. I would suggest the second route only to those who know TensorFlow or PyTorch really well.
We chose to frame bacteria strain classification as object detection and instance segmentation computer vision problems.

Fig. 5. Architecture of the model.
Object detection models locate regions of an image that correspond to particular classes of things, which makes them excellent for counting. Bacterial colonies, of course, are not well-defined objects. They are less like things and more like shapeless stuff that we need to locate and label. For that reason we went with a point detection model, which relies on heat maps during detection instead of the familiar bounding boxes. We tried two approaches: the first on grayscale images from the Arduino Portenta, the second on RGB images taken with the Raspberry Pi.
Incidentally, for a task this simple an ML model is not strictly necessary. Good enough results could be obtained by thresholding the image correctly and running watershed, or by plain blob detection.
Still, when experimenting, testing several algorithms is always a sensible idea.
Results
To be honest, the images from the Arduino Portenta were a spectacular failure. We did get some results, but because the image resolution (320px x 320px) was too low, the model picked up only the biggest objects, which showed up halfway through incubation. The vision shield sensor simply isn’t built for this kind of use. We had anticipated this possibility, and now we know for certain that a device with different specs is required. The experiment failed successfully 🙂
Luckily, the Rasberry Pi images were of high enough quality for the detection process. We ran one of the test samples through the algorithm: the output is still somewhat noisy, but far quicker than counting every instance manually. Much of that noise could be eliminated by building a less dusty monitoring device with an extra mechanism for wicking away moisture.
Discussion
How can Machine Learning contribute to life sciences?
Machine learning has already shown itself to be a very promising way of solving problems in many branches of industry, and only fairly recently has it begun to make its way into life science, offering a powerful means of surveying and classifying biological data [^1,2,3]. The life science industry is steadily gaining from bringing artificial intelligence and machine learning into its research agendas, particularly in genomics, chemistry, biophysics, microscopy, medical analysis, and many other areas. It is becoming ever more important in healthcare, for example in cancer research, the study of brain cells, and the development of therapeutics. Many studies demonstrate that machine learning algorithms explore data in ways people cannot and can spot features that would otherwise be impossible to notice, for instance when classifying cellular images or finding genomic connections [^1,3,4]. Machine learning tools are developing quickly, and to benefit from them or exploit their full potential, life science labs will soon require dedicated computational expertise, partnerships, or both.
Limitations of Machine Learning
Even though machine learning holds the promise of a breakthrough for science, it comes with some limitations that deserve a mention:
- Shortage of large-scale, high-quality data – how do you train a model on less data, which is a common situation in life science;
- Data quality – a model is only as good as the data it is fed;
- Striking a balance between an accurate algorithm and a result that can be explained biologically remains hard;
- Absence of sound, biologically meaningful judgment – computers cannot completely take the place of scientists [^1,5].
Summary
It goes without saying that we really enjoyed this project! Planning the experiment, designing and assembling a monitoring device, growing bacteria, and training a machine learning model – all in a single, short life science project.
Above all, we learned how machine learning can be applied even in the wildest scenarios. As you have likely seen, embedded artificial intelligence has many uses. Counting bacterial colonies is only the start.
You may be curious what we will do with our data next. We intend to grow the dataset. Incubation is a lot of work. We might add microscope images and attempt a multimodal approach. Who knows? 🙂 We will certainly tell you once we release the collection publicly.
We also need a new version of our device. The existing one works well but can only handle a single sample at a time.
Thank you for reading, and best of luck with your own experiments!
References
[^1] Webb S. (2018) Deep learning for biology. Nature 554, 555-557.
[^2] Pugliese R. et al. (2021) Machine learning-based approach: global trends, research directions, and regulatory standpoints. Data Science and Management 4, 19-29.
[^3] Bajorath J. (2022) Artificial intelligence in interdisciplinary life science and drug discovery research. Future Science OA 8, FSO792.
[^4] Ghosh, S., Dasgupta, R. (2022). A Brief Overview of Applications of Machine Learning in Life Sciences. In: Machine Learning in Biological Sciences. Springer.
[^5] Ching T. et al. (2018) Opportunities and obstacles for deep learning in biology and medicine. Journal of the Royal Society Interface 15, 20170387.