Abstract
With large language models demonstrating significant potential in code generation tasks, their application to onboard control of resource-constrained Unmanned Aerial Vehicles has emerged as an important research direction. However, a notable contradiction exists between the high resource consumption of large models and the real-time, lightweight requirements of UAV platforms. This paper proposes an integrated approach that combines knowledge distillation, chain-of-thought guidance, and supervised fine-tuning for UAV multi-SDK control tasks, aiming to efficiently transfer complex reasoning and code generation capabilities to smaller models. Firstly, a high-quality dataset covering various mainstream UAV SDKs is constructed, featuring instruction-code-reasoning chains, and incorporates counterfactual negative samples for data augmentation, guiding the model to learn the end-to-end logic from instruction parsing to code generation. Secondly, leveraging DeepSeek-Coder-V2-Lite quantized via QLoRA as the teacher model, and based on a hybrid black-box and white-box distillation strategy, high-quality chain-of-thought soft labels are generated. These are combined with a weighted cross-entropy loss using hard labels to transfer complex reasoning capabilities to the smaller student model. Finally, through prompt tuning engineering optimized for the UAV control scenario, the model performance on core tasks such as SDK type recognition and function call matching is enhanced. Experimental results indicate that the distilled lightweight model maintains high code generation accuracy while achieving significant improvements in deployment and inference efficiency, effectively demonstrating the feasibility and superiority of our approach in achieving precise and lightweight intelligent control for UAVs
Abstract (translated)
随着大型语言模型在代码生成任务中展现出巨大潜力，它们被应用于资源受限的无人驾驶飞行器（UAV）的在线控制，成为了一个重要的研究方向。然而，大型模型高资源消耗与无人机平台实时、轻量级需求之间存在明显的矛盾。本文提出了一种综合方法，结合知识蒸馏、链式思维引导和监督微调技术，以实现将复杂推理和代码生成能力高效转移至小型模型的目标。具体而言： 首先，构建了一个高质量的数据集，涵盖了各种主流UAV软件开发工具包（SDK），该数据集中包含了指令-代码-推理链条，并且引入了反事实负样本进行数据增强，引导模型从指令解析到代码生成学习端到端逻辑。 其次，利用通过QLoRA量化后的DeepSeek-Coder-V2-Lite作为教师模型，并基于混合黑盒和白盒蒸馏策略生成高质量的链式思维软标签。这些软标签与硬标签结合使用加权交叉熵损失函数，以将复杂推理能力转移至较小的学生模型。 最后，通过针对UAV控制场景优化提示调优工程，增强了模型在SDK类型识别、功能调用匹配等核心任务上的性能表现。 实验结果表明，蒸馏后的轻量级模型能够保持高代码生成准确性的同时，在部署和推断效率上实现了显著改进。这有效地展示了我们的方法在实现精确且轻量级的UAV智能控制方面的可行性和优越性。
URL
https://arxiv.org/abs/2601.08412
