Publishing the „Carbonaro“ dataset from FRE 2022
Last year, we published our dataset and models for image recognition during the online Field Robot Event 2021. Our blog post ended with a call for cooperation in creating a dataset and AI models for this year’s event, which has since taken place at DLG Feldtage in June.
Successful cooperation
Team Carbonite from the Überlingen Students‘ Research Center (SFZ Überlingen) approached and we subsequently worked together on building a realistic image dataset. As a nod to the two teamnames, this dataset shall henceforth be known by the name “Carbonaro“.
Real-world and simulated datasets
In line with this year’s detection task, the dataset contains coarse semantic segmentation labels for (artificial) dandelion flowers and beverage cans.
Each task of FRE 2022 was held both on the real field and in a simulation, which was similar to last year’s. Therefore, we publish raw data and trained models for both of these environments.
The simulated dataset is largely based on its predecessor from 2021, but contains additional images, since dandelions were a new object class this year. Due to a lack of neccessity and time, and to correspond more closely to the real-world dataset, maize plants are not labeled in this year’s dataset, and are therefore part of the background class.
What is available?
Team Carbonite opted for a bounding-box detection model based on Yolo v3. Kamaro used a coarse semantic segmentation model with an EfficientNetV2 backbone. We also tried instance segmentation with Mask-R-CNN.
The trained models, as well as documentation and software for ROS integration, are available on Github:
The datasets for training your own models are available on the Kamaro Nextcloud in multiple formats.
- Numpy gzip compressed format X and y
The labels are provided as PNG files where the three color channels correspond to the three classes:- red (
#ff0000) for background/maize - green (
#00ff00) for beverage cans - blue (
#0000ff) for dandelions
- red (
- COCO json format
- COCO txt format
- PNG image and one mask for every instance
- Original images and masks
Guides
For training your own models on the dataset, we provide guides and supplementary materials:
Supporting the shift to deep learning at FRE
Like last year, we are again publishing the dataset, trained models and supplementary materials to aid other teams in the transition to deep learning based, data driven perception methods. Much of the agricultural machinery industry is already shifting to advanced camera-based perception methods. We strongly believe that supporting this development at FRE is necessary to keep the event relevant and attractive in the future. We therefore again invite all other teams to cooperate and pool resources for next year’s dataset.
Have fun exploring the available materials, and please drop us an e-mail if you are interested joining the deep learning hype train and contributing to the FRE 2023 dataset!