Paper Reading AI Learner

Using Vision Language Foundation Models to Generate Plant Simulation Configurations via In-Context Learning

2026-03-09 20:58:43
Heesup Yun, Isaac Kazuo Uyehara, Earl Ranario, Lars Lundqvist, Christine H. Diepenbrock, Brian N. Bailey, J. Mason Earles

Abstract

This paper introduces a synthetic benchmark to evaluate the performance of vision language models (VLMs) in generating plant simulation configurations for digital twins. While functional-structural plant models (FSPMs) are useful tools for simulating biophysical processes in agricultural environments, their high complexity and low throughput create bottlenecks for deployment at scale. We propose a novel approach that leverages state-of-the-art open-source VLMs -- Gemma 3 and Qwen3-VL -- to directly generate simulation parameters in JSON format from drone-based remote sensing images. Using a synthetic cowpea plot dataset generated via the Helios 3D procedural plant generation library, we tested five in-context learning methods and evaluated the models across three categories: JSON integrity, geometric evaluations, and biophysical evaluations. Our results show that while VLMs can interpret structural metadata and estimate parameters like plant count and sun azimuth, they often exhibit performance degradation due to contextual bias or rely on dataset means when visual cues are insufficient. Validation on a real-world drone orthophoto dataset and an ablation study using a blind baseline further characterize the models' reasoning capabilities versus their reliance on contextual priors. To the best of our knowledge, this is the first study to utilize VLMs to generate structural JSON configurations for plant simulations, providing a scalable framework for reconstruction 3D plots for digital twin in agriculture.

Abstract (translated)

这篇论文介绍了一个合成基准,用于评估视觉语言模型(VLM)在为数字孪生生成植物仿真配置方面的性能。尽管功能性-结构性植株模型(FSPM)是模拟农业环境中生物物理过程的有用工具,但它们的高度复杂性和低吞吐量限制了其大规模部署的能力。我们提出了一种新颖的方法,利用最先进的开源视觉语言模型——Gemma 3 和 Qwen3-VL,直接从基于无人机的遥感图像生成JSON格式的仿真参数。通过使用Helios 3D程序化植物生成库生成的合成豇豆地块数据集,我们测试了五种上下文学习方法,并在三个类别中评估了这些模型:JSON完整性、几何评价和生物物理评价。我们的研究结果表明,虽然VLM可以解释结构元数据并估计植株数量和太阳方位角等参数,但在视觉线索不足的情况下,它们往往会因为上下文偏差而表现不佳或依赖于数据集平均值。 在真实世界的无人机正射影像数据集上进行验证以及使用盲基线的消融研究进一步刻画了模型推理能力与其对上下文先验依赖之间的关系。据我们所知,这是首次利用VLM生成植物仿真的结构化JSON配置的研究,为农业中数字孪生体3D地块的重建提供了一个可扩展框架。

URL

https://arxiv.org/abs/2603.08930

PDF

https://arxiv.org/pdf/2603.08930.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model LLM Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Robot Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Time_Series Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot