Abstract
Unsupervised Domain Adaptation (UDA) is a critical challenge in real-world vision systems, especially in resource-constrained environments like drones, where memory and computation are limited. Existing prompt-driven UDA methods typically rely on large vision-language models and require full access to source-domain data during adaptation, limiting their applicability. In this work, we propose Prmpt2Adpt, a lightweight and efficient zero-shot domain adaptation framework built around a teacher-student paradigm guided by prompt-based feature alignment. At the core of our method is a distilled and fine-tuned CLIP model, used as the frozen backbone of a Faster R-CNN teacher. A small set of low-level source features is aligned to the target domain semantics-specified only through a natural language prompt-via Prompt-driven Instance Normalization (PIN). These semantically steered features are used to briefly fine-tune the detection head of the teacher model. The adapted teacher then generates high-quality pseudo-labels, which guide the on-the-fly adaptation of a compact student model. Experiments on the MDS-A dataset demonstrate that Prmpt2Adpt achieves competitive detection performance compared to state-of-the-art methods, while delivering up to 7x faster adaptation and 5x faster inference speed using few source images-making it a practical and scalable solution for real-time adaptation in low-resource domains.
Abstract (translated)
无监督领域适应(UDA)是现实世界视觉系统中的一个关键挑战,特别是在资源受限的环境中如无人机中,因为这些环境具有有限的记忆和计算能力。现有的基于提示驱动的UDA方法通常依赖于大型视觉-语言模型,并且在适应过程中需要完全访问源域数据,这限制了它们的应用范围。在这项工作中,我们提出了Prmpt2Adpt,这是一种轻量级且高效的零样本领域自适应框架,围绕由基于提示的功能对齐指导的教师-学生范式构建。我们的方法的核心是一个经过蒸馏和微调的CLIP模型,用作冻结后的Faster R-CNN教师模型的骨干网络。一组小规模的低级别源特征通过Prompt驱动实例归一化(PIN)与仅通过自然语言提示指定的目标域语义对齐。这些被语义引导的功能用于短时间微调教师模型的检测头。适应后的教师然后生成高质量的伪标签,指导紧凑的学生模型进行在飞行自适应。在MDS-A数据集上的实验表明,Prmpt2Adpt实现了与最先进的方法相比具有竞争力的目标检测性能,同时使用少量源图像时提供高达7倍更快的自适应速度和5倍更快的推理速度,使其成为低资源领域中实时自适应的实际且可扩展解决方案。
URL
https://arxiv.org/abs/2506.16994