Five highlights that stood out to me at ECCV 2024
Five takeaways I found the most interesting and unexpected, from both the research and the applied point of view.
Last week I had a great time at ECCV 2024, which offered engaging workshops on applied computer vision, oral presentations in which computer vision experts shared their cutting-edge research, and inspiring keynotes from industry giants.
In this brief post, I go over five takeaways that I found the most interesting and unexpected, from both the research and the applied point of view.
What ECCV is
The European Conference in Computer Vision (ECCV) ranks among the biggest and most prestigious Computer Vision conferences (alongside CVPR, ICCV, and NeurIPS) and has given us well-known papers such as NERF and Group Normalization. The conference lasts 6 days overall: 2 days of workshops and then 4 days of the main event. Given the number of tracks, papers, and workshops (12 tracks, 2387 papers, and 51 workshops), there is no way to look at and properly grasp everything, since the subjects span from Video Generation and 3D Depth Estimation to applications for automatically spotting defects in manufacturing or diseases in agriculture. I did my best to keep up with the topics that interest me most.
Conference highlights
Below, I try to pull out the most interesting takeaways from the conference across several fields, from avatars to GenAI, Explainable AI, and agriculture.
Create your own avatar with Synthesia
During their keynote, two lead researchers (Vittorio Ferrari and Lourdes Agapito) walked through how the field has evolved over the last 6 years, covering a range of breakthroughs and the research behind them. The company shares what it learns in peer-reviewed papers at conferences such as ECCV, and also in appearances aimed at a broader audience, for example the well-known „Malaria Must Die” ad featuring David Beckham.
I already knew about some of the progress in this area: most avatars were either low quality (when personalised), showed serious artifacts (usually around the lips), or simply were not very engaging.
Over the last 2 years, they have fixed many of these problems (avatars are now more engaging and of higher quality, with experimental hand gesture features). That seems to have been the state in 2023, whereas over the last 2 years the field has moved forward considerably, with more accurate lip sync in 150+ languages, more engaging voices, and even hand gestures).
Soon, Synthesia also plans to add interactions with objects to make the avatars even more compelling; natural object grasping, however, is a well-known problem where computer vision meets robotics.
Fair, transparent and accountable AI

Bias preservation checklist taken from Bias Preservation in Machine Learning: The Legality of Fairness Metrics Under EU Non-Discrimination Law
Another engaging keynote, titled „Fair, transparent and accountable AI: What is legally required, what is ethically desired, and what is technically feasible”, was given by Prof. Sandra Wachter of the University of Oxford. It was especially interesting because it took a critical look at the EU AI act and the current legal system, pointing out major loopholes and vague wording (see the speaker’s related report: Limitations and loopholes of EU AI act) and the absence of clear guidelines. What I had not known before is that more domain-specific guidelines are now being drafted to complement the EU AI act.
The lecture also covered how biases are classified and what effects they have (e.g. Facebook used to let advertisers exclude users by race). One striking bias came from the Netherlands, where applicants of certain races and applicants from certain neighbourhoods had limited access to mortgages. This was partly because credit scoring systems labelled some areas as high-risk and others as low-risk.
Finally, Prof. Sandra Watcher presented a remedy for this, called Conditional Demographic Disparity -CDD, which was added to the AWS ecosystem in Amazon Clarify, a bias and interpretability tool.
Model-centric and data-centric Explainable AI
How different heads in the CLIP architecture activate, Source: Interpreting the Second-Order Effects of Neurons in CLIP
The „Explainable Computer Vision” workshop was another highlight, featuring a two-part lecture from BAIR presenting two distinct approaches to Explainable AI. The first part came from Yossi Gandelsman, who described his work on understanding how individual attention modules in CLIP (which underpins most of today’s computer vision GenAI applications) contribute. It was groundbreaking, as his research identified which attention heads are responsible for features such as:
- Geolocation
- Colors
- Textures
- Animal
- And plenty more.
The second part of the lecture focused on the data perspective (drawing parallels to EigenFaces, built on the idea that every face in the world can be expressed as a weighted combination of roughly 100 faces). Here, Prof. Alexei A. Efros argued that rather than concentrating on explaining how different algorithms behave, we should shift from a model-centric approach („which part of the model made this decision”) to a data-centric one („which data led to this”). His research suggests this is achievable on smaller, controlled datasets, with encouraging prospects for larger and more varied datasets (e.g., ImageNet or LAION-4b).
Clay filters and taming an unexpectedly huge AWS bill
Viral Clay filter from Bending Spoons, source recodechinaai.substack.com
This talk came from Bending Spoon, one of the sponsors, who described the successive waves of interest in their filter generation product. In Europe they are known for an app that turns casual photos into corporate pictures for LinkedIn. In Asia, their Clay filter feature is what made them popular.
They explained how progress in Stable Diffusion let them build appealing filters (e.g. Corporate Headshots) and which countries and platforms showed the most interest (Asian countries leading the way).
The latest and biggest wave of hype (around clay filter generation) was so large that in a single day they ran up a 300k$ bill for that day alone. They had to act quickly and cut their cloud usage fast, while preserving . Their solution was actually quite simple: they accepted and generated smaller images, switched to smaller and cheaper GPUs, and finally cut training time considerably).
This brought their costs down 10x, which shows that small tweaks can make a big difference.
Progress in agriculture and phenotyping
Segmentation masks of leaf diseases. Source: A new Large Dataset and a Transfer Learning Methodology for Plant Phenotyping in Vertical Farms
Since agriculture is a big interest of mine, I attended the talks of the „Computer Vision in Plant Phenotyping and Agriculture” (CVPPA) workshop.
The talk that stood out most to me in this workshop was given by Dr Fiora Pirri from DeepPlants, who presented a project building a disease detection dataset, published on HuggingFace. The biggest surprise was how complex the pipeline is (at least 5 different architectures and 2-3 modalities: RGB, depth etc). The dataset can help push forward early disease detection (from mobile phone camera videos) in disadvantaged countries and reduce the damage diseases cause. It was also interesting to hear about various ideas that build on progress in Computer Vision (e.g., using Stable Diffusion to generate samples of particular diseases to improve how they are classified).
Conclusions
In this brief post, I went over the state of the art in five different areas – from avatars, through progress in applied fields such as agriculture, to understanding models and data and spotting bias in your data.
ECCV was an excellent chance to see where SOTA currently stands, how progress in one field affects others (e.g., agriculture or the medical industry), and to experience the cutting edge of the field in practice.