Thanks to deep learning, robotic systems in the agricultural sector are improving in the recognition and processing of crop photos. To properly train the systems, hundreds to thousands of images are needed of the culture for which they are used. These images are largely processed by humans. First they find out which photos are relevant, then they have to annotate them: draw in pixels and label them. Based on this information, a system learns to recognize the patterns it will encounter in the greenhouse.
Image selection and annotation by experts is expensive and laborious work that can involve human error. Pieter Blok, a Ph.D. student at Wageningen University & Research, set out to find a method that could simplify image selection and annotation. He designed an algorithm that uses active learning, an intelligent sampling technique. The algorithm is able to select the most interesting images from a dataset itself. These are the images the system struggles with and can therefore get the most out of.
Experiment with broccoli
To test his algorithm, Blok got to work with photos of broccoli plants in the field. The image below shows how. Three different attempts by the system to process broccoli produced three different outputs on the same image: diseased (left), overripe (center), and healthy (right). The uncertainty factor was therefore very large and the algorithm concluded that it was an interesting image for humans to label. The tagged image was then sent back to the system for it to learn. Blok: “If the algorithm is uncertain about something, there’s probably a lot of room to improve its performance.”
By leaving only the “uncertain” images to humans, Blok’s method saved a lot of time. With random sampling, people should have tagged 2,300 images, but thanks to its smart sampling, that number was reduced to just 900 images. That is a saving of 1,400 images. Blok does a quick math: “Annotating an image takes about 3-5 minutes. If you save 1400 frames, like in my dataset, you’re talking about a time saving of maybe 7000 minutes in total. That’s over 116 hours.
According to Blok, all companies in the agricultural sector are faced with the fact that annotating images takes a lot of time. He therefore hopes that his innovative method will be widely used. Its software, called MaskAL, is free to download.
For more information:
Wageningen University and Research