Abstract
Information on trees at the individual level is crucial for monitoring forest ecosystems and planning forest management. Current monitoring methods involve ground measurements, requiring extensive cost, time and labor. Advances in drone remote sensing and computer vision offer great potential for mapping individual trees from aerial imagery at broad-scale. Large pre-trained vision models, such as the Segment Anything Model (SAM), represent a particularly compelling choice given limited labeled data. In this work, we compare methods leveraging SAM for the task of automatic tree crown instance segmentation in high resolution drone imagery in three use cases: 1) boreal plantations, 2) temperate forests and 3) tropical forests. We also study the integration of elevation data into models, in the form of Digital Surface Model (DSM) information, which can readily be obtained at no additional cost from RGB drone imagery. We present BalSAM, a model leveraging SAM and DSM information, which shows potential over other methods, particularly in the context of plantations. We find that methods using SAM out-of-the-box do not outperform a custom Mask R-CNN, even with well-designed prompts. However, efficiently tuning SAM end-to-end and integrating DSM information are both promising avenues for tree crown instance segmentation models.
Abstract (translated)
树木的个体信息对于监测森林生态系统和规划林业管理至关重要。目前,地面测量方法耗时、费力且成本高昂。无人机遥感技术和计算机视觉的进步为从高分辨率航拍图像中绘制单个树木提供了巨大潜力。特别地,在标注数据有限的情况下,大型预训练视觉模型(如Segment Anything Model (SAM))成为了一个极具吸引力的选择。在本研究中,我们对比了三种应用场景下使用SAM进行自动树冠实例分割的方法:1)针叶林人工林;2)温带森林;3)热带森林。此外,我们还探讨了如何将高程数据(以数字表面模型(DSM)的形式获取,可以从RGB无人机图像免费获得)集成到模型中。我们提出了一种名为BalSAM的模型,该模型结合了SAM和DSM信息,并在人工林场景中表现出比其他方法更优的潜力。然而,我们也发现直接使用未经调整的SAM并不能优于定制化的Mask R-CNN模型,即使使用精心设计的提示也是如此。不过,对SAM进行端到端的有效微调以及整合DSM数据均为树冠实例分割模型提供了有前景的发展方向。
URL
https://arxiv.org/abs/2506.04970