Abstract
Field trials are vital in herbicide research and development to assess effects on crops and weeds under varied conditions. Traditionally, evaluations rely on manual visual assessments, which are time-consuming, labor-intensive, and subjective. Automating species and damage identification is challenging due to subtle visual differences, but it can greatly enhance efficiency and consistency. We present an improved segmentation model combining a general-purpose self-supervised visual model with hierarchical inference based on botanical taxonomy. Trained on a multi-year dataset (2018-2020) from Germany and Spain using digital and mobile cameras, the model was tested on digital camera data (year 2023) and drone imagery from the United States, Germany, and Spain (year 2024) to evaluate robustness under domain shift. This cross-device evaluation marks a key step in assessing generalization across platforms of the model. Our model significantly improved species identification (F1-score: 0.52 to 0.85, R-squared: 0.75 to 0.98) and damage classification (F1-score: 0.28 to 0.44, R-squared: 0.71 to 0.87) over prior methods. Under domain shift (drone images), it maintained strong performance with moderate degradation (species: F1-score 0.60, R-squared 0.80; damage: F1-score 0.41, R-squared 0.62), where earlier models failed. These results confirm the model's robustness and real-world applicability. It is now deployed in BASF's phenotyping pipeline, enabling large-scale, automated crop and weed monitoring across diverse geographies.
Abstract (translated)
田间试验在除草剂的研究和开发中至关重要,用于评估作物和杂草在不同条件下的影响。传统上,评估依赖于手动视觉评估,这既耗时又费力,并且具有主观性。由于细微的视觉差异,自动物种识别和损伤识别颇具挑战性,但可以大大提高效率和一致性。我们提出了一种改进的分割模型,结合了通用自监督视觉模型与基于植物分类学的层次推理方法。该模型在来自德国和西班牙(2018-2020年)多年度数据集上使用数字和移动相机训练,并通过美国、德国和西班牙(2023年和2024年)采集的数字相机数据和无人机图像进行测试,以评估域转移情况下的鲁棒性。这种跨设备评估标志着模型在不同平台上的泛化能力的关键步骤。 我们的模型在物种识别(F1分数:从0.52提高到0.85,R平方值:从0.75提升至0.98)和损伤分类(F1分数:从0.28增至0.44,R平方值:从0.71升至0.87)方面相对于先前的方法有了显著改进。在域转移条件下(无人机图像),尽管性能略有下降(物种:F1分数 0.60,R平方值 0.80;损伤:F1分数 0.41,R平方值 0.62),该模型仍保持了强大的表现能力,而早期的模型在此情况下则完全失效。这些结果证实了我们模型的强大鲁棒性和现实应用价值。 目前,该模型已被部署在巴斯夫公司的表型分析管道中,在广泛的地理区域内实现了大规模、自动化的作物和杂草监测。
URL
https://arxiv.org/abs/2508.07514