Paper Reading AI Learner

Bringing SAM to new heights: Leveraging elevation data for tree crown segmentation from drone imagery

2025-06-05 12:43:11
M\'elisande Teng, Arthur Ouaknine, Etienne Lalibert\'e, Yoshua Bengio, David Rolnick, Hugo Larochelle

Abstract

Information on trees at the individual level is crucial for monitoring forest ecosystems and planning forest management. Current monitoring methods involve ground measurements, requiring extensive cost, time and labor. Advances in drone remote sensing and computer vision offer great potential for mapping individual trees from aerial imagery at broad-scale. Large pre-trained vision models, such as the Segment Anything Model (SAM), represent a particularly compelling choice given limited labeled data. In this work, we compare methods leveraging SAM for the task of automatic tree crown instance segmentation in high resolution drone imagery in three use cases: 1) boreal plantations, 2) temperate forests and 3) tropical forests. We also study the integration of elevation data into models, in the form of Digital Surface Model (DSM) information, which can readily be obtained at no additional cost from RGB drone imagery. We present BalSAM, a model leveraging SAM and DSM information, which shows potential over other methods, particularly in the context of plantations. We find that methods using SAM out-of-the-box do not outperform a custom Mask R-CNN, even with well-designed prompts. However, efficiently tuning SAM end-to-end and integrating DSM information are both promising avenues for tree crown instance segmentation models.

Abstract (translated)

树木的个体信息对于监测森林生态系统和规划林业管理至关重要。目前,地面测量方法耗时、费力且成本高昂。无人机遥感技术和计算机视觉的进步为从高分辨率航拍图像中绘制单个树木提供了巨大潜力。特别地,在标注数据有限的情况下,大型预训练视觉模型(如Segment Anything Model (SAM))成为了一个极具吸引力的选择。在本研究中,我们对比了三种应用场景下使用SAM进行自动树冠实例分割的方法:1)针叶林人工林;2)温带森林;3)热带森林。此外,我们还探讨了如何将高程数据(以数字表面模型(DSM)的形式获取,可以从RGB无人机图像免费获得)集成到模型中。我们提出了一种名为BalSAM的模型,该模型结合了SAM和DSM信息,并在人工林场景中表现出比其他方法更优的潜力。然而,我们也发现直接使用未经调整的SAM并不能优于定制化的Mask R-CNN模型,即使使用精心设计的提示也是如此。不过,对SAM进行端到端的有效微调以及整合DSM数据均为树冠实例分割模型提供了有前景的发展方向。

URL

https://arxiv.org/abs/2506.04970

PDF

https://arxiv.org/pdf/2506.04970.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Time_Series Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot