Style_Transfer
-
SceneExpander: Expanding 3D Scenes with Free-Form Inserted Views
2026-03-28 02:04:48Zijian He, enjie Liu, Yihao Wang, Weizhi Zhong, Huan Yuan, Kun Gai, Guangrun Wang, Guanbin LiAbstract
World building with 3D scene representations is increasingly important for content creation, simulation, and interactive experiences, yet real workflows are inherently iterative: creators must repeatedly extend an existing scene under user control. Motivated by this research gap, we study 3D scene expansion in a user-centric workflow: starting from a real scene captured by multi-view images, we extend its coverage by inserting an additional view synthesized by a generative model. Unlike simple object editing or style transfer in a fixed scene, the inserted view is often 3D-misaligned with the original reconstruction, introducing geometry shifts, hallucinated content, or view-dependent artifacts that break global multi-view consistency. To address the challenge, we propose SceneExpander, which applies test-time adaptation to a parametric feed-forward 3D reconstruction model with two complementary distillation signals: anchor distillation stabilizes the original scene by distilling geometric cues from the captured views, while inserted-view self-distillation preserves observation-supported predictions yet adapts latent geometry and appearance to accommodate the misaligned inserted view. Experiments on ETH scenes and online data demonstrate improved expansion behavior and reconstruction quality under misalignment.
Abstract (translated)
基于3D场景表征的世界构建对于内容创作、仿真和交互体验愈发重要，但实际工作流本质上是迭代的：创作者必须在用户控制下反复扩展现有场景。受此研究空白驱动，我们在以用户为中心的工作流中研究3D场景扩展：从多视角图像捕捉的真实场景出发，通过插入由生成模型合成的额外视图来扩展其覆盖范围。与固定场景中的简单对象编辑或风格迁移不同，插入的视图通常与原始重建存在3D错位，引入几何偏移、幻觉内容或视角相关伪影，从而破坏全局多视角一致性。为应对这一挑战，我们提出SceneExpander，该方法对参数化前馈3D重建模型应用测试时适应，并利用两种互补的蒸馏信号：锚点蒸馏通过从捕捉的视图中蒸馏几何线索来稳定原始场景，而插入视图自蒸馏则保留有观测支持的预测，同时适应潜在几何和外观以容纳错位的插入视图。在ETH场景和在线数据上的实验表明，该方法在错位条件下改善了扩展行为和重建质量。
URL
https://arxiv.org/abs/2603.27084
PDF
-
Adapting Frozen Mono-modal Backbones for Multi-modal Registration via Contrast-Agnostic Instance Optimization
2026-03-27 13:21:19Yi Zhang, Yidong Zhao, Qian TaoAbstract
Deformable image registration remains a central challenge in medical image analysis, particularly under multi-modal scenarios where intensity distributions vary significantly across scans. While deep learning methods provide efficient feed-forward predictions, they often fail to generalize robustly under distribution shifts at test time. A straightforward remedy is full network fine-tuning, yet for modern architectures such as Transformers or deep U-Nets, this adaptation is prohibitively expensive in both memory and runtime when operating in 3D. Meanwhile, the naive fine-tuning struggles more with potential degradation in performance in the existence of drastic domain shifts. In this work, we propose a registration framework that integrates a frozen pretrained \textbf{mono-modal} registration model with a lightweight adaptation pipeline for \textbf{multi-modal} image registration. Specifically, we employ style transfer based on contrast-agnostic representation generation and refinement modules to bridge modality and domain gaps with instance optimization at test time. This design is orthogonal to the choice of backbone mono-modal model, thus avoids the computational burden of full fine-tuning while retaining the flexibility to adapt to unseen domains. We evaluate our approach on the Learn2Reg 2025 LUMIR validation set and observe consistent improvements over the pretrained state-of-the-art mono-modal backbone. In particular, the method ranks second on the multi-modal subset, third on the out-of-domain subset, and achieves fourth place overall in Dice score. These results demonstrate that combining frozen mono-modal models with modality adaptation and lightweight instance optimization offers an effective and practical pathway toward robust multi-modal registration.
Abstract (translated)
URL
https://arxiv.org/abs/2603.26393
PDF
-
HAM: A Training-Free Style Transfer Approach via Heterogeneous Attention Modulation for Diffusion Models
2026-03-25 07:52:30Yeqi He, Liang Li, Zhiwen Yang, Xichun Sheng, Zhidong Zhao, Chenggang YanAbstract
Diffusion models have demonstrated remarkable performance in image generation, particularly within the domain of style transfer. Prevailing style transfer approaches typically leverage pre-trained diffusion models' robust feature extraction capabilities alongside external modular control pathways to explicitly impose style guidance signals. However, these methods often fail to capture complex style reference or retain the identity of user-provided content images, thus falling into the trap of style-content balance. Thus, we propose a training-free style transfer approach via $\textbf{h}$eterogeneous $\textbf{a}$ttention $\textbf{m}$odulation ($\textbf{HAM}$) to protect identity information during image/text-guided style reference transfer, thereby addressing the style-content trade-off challenge. Specifically, we first introduces style noise initialization to initialize latent noise for diffusion. Then, during the diffusion process, it innovatively employs HAM for different attention mechanisms, including Global Attention Regulation (GAR) and Local Attention Transplantation (LAT), which better preserving the details of the content image while capturing complex style references. Our approach is validated through a series of qualitative and quantitative experiments, achieving state-of-the-art performance on multiple quantitative metrics.
Abstract (translated)
扩散模型在图像生成领域展现出卓越性能，尤其在风格迁移任务中表现突出。现有风格迁移方法通常利用预训练扩散模型强大的特征提取能力，结合外部模块化控制路径来显式施加风格引导信号。然而，这些方法往往难以捕捉复杂风格参考或保留用户提供内容图像的身份特征，从而陷入风格-内容平衡的困境。因此，我们提出一种无需训练的异质注意力调制（HAM）风格迁移方法，以在图像/文本引导的风格参考迁移过程中保护身份信息，从而解决风格-内容权衡挑战。具体而言，我们首先引入风格噪声初始化来为扩散过程初始化潜在噪声。随后在扩散过程中，创新性地对不同的注意力机制采用HAM，包括全局注意力调控（GAR）与局部注意力移植（LAT），在捕捉复杂风格参考的同时更好地保留内容图像的细节。通过一系列定性与定量实验验证了本方法，在多项定量指标上达到了最先进性能。
URL
https://arxiv.org/abs/2603.24043
PDF
-
FHAvatar: Fast and High-Fidelity Reconstruction of Face-and-Hair Composable 3D Head Avatar from Few Casual Captures
2026-03-24 15:43:54Yujie Sun, Zhuoqiang Cai, Chaoyue Niu, Jianchuan Chen, Zhiwen Chen, Chengfei Lv, Fan WuAbstract
We present FHAvatar, a novel framework for reconstructing 3D Gaussian avatars with composable face and hair components from an arbitrary number of views. Unlike previous approaches that couple facial and hair representations within a unified modeling process, we explicitly decouple two components in texture space by representing the face with planar Gaussians and the hair with strand-based Gaussians. To overcome the limitations of existing methods that rely on dense multi-view captures or costly per-identity optimization, we propose an aggregated transformer backbone to learn geometry-aware cross-view priors and head-hair structural coherence from multi-view datasets, enabling effective and efficient feature extraction and fusion from few casual captures. Extensive quantitative and qualitative experiments demonstrate that FHAvatar achieves state-of-the-art reconstruction quality from only a few observations of new identities within minutes, while supporting real-time animation, convenient hairstyle transfer, and stylized editing, broadening the accessibility and applicability of digital avatar creation.
Abstract (translated)
我们提出了FHAvatar，一种新颖的框架，可从任意数量的视角中重建具有可组合面部与发丝组件的三维高斯虚拟形象。与以往在统一建模过程中耦合面部与发丝表征的方法不同，我们通过在纹理空间中用平面高斯表示面部、用基于发丝的高斯表示发丝，显式解耦这两个组件。为克服现有方法依赖密集多视角采集或昂贵逐身份优化的局限，我们提出聚合Transformer主干网络，从多视角数据集中学习几何感知的跨视角先验与头-发结构一致性，从而实现对少量 casually captured 数据的高效特征提取与融合。大量定量与定性实验表明，FHAvatar在数分钟内即可从新身份的少量观测中实现最先进的重建质量，同时支持实时动画、便捷发型迁移与风格化编辑，拓宽了数字虚拟形象创建的普及性与应用范围。
URL
https://arxiv.org/abs/2603.23345
PDF
-
AuthorMix: Modular Authorship Style Transfer via Layer-wise Adapter Mixing
2026-03-24 11:07:00Sarubi Thillainathan, Ji-Ung Lee, Michael Sullivan, Alexander KollerAbstract
The task of authorship style transfer involves rewriting text in the style of a target author while preserving the meaning of the original text. Existing style transfer methods train a single model on large corpora to model all target styles at once: this high-cost approach offers limited flexibility for target-specific adaptation, and often sacrifices meaning preservation for style transfer. In this paper, we propose AuthorMix: a lightweight, modular, and interpretable style transfer framework. We train individual, style-specific LoRA adapters on a small set of high-resource authors, allowing the rapid training of specialized adaptation models for each new target via learned, layer-wise adapter mixing, using only a handful of target style training examples. AuthorMix outperforms existing, SoTA style-transfer baselines -- as well as GPT-5.1 -- for low-resource targets, achieving the highest overall score and substantially improving meaning preservation.
Abstract (translated)
作者风格迁移的任务旨在在保持原文语义的前提下，将文本重写为目标作者的风格。现有风格迁移方法通常在大规模语料上训练单一模型以同时建模所有目标风格：这种高成本方法对目标特定适应的灵活性有限，且常以牺牲语义保持为代价来实现风格迁移。本文提出AuthorMix：一个轻量、模块化且可解释的风格迁移框架。我们在少量高资源作者的数据上训练独立的、风格特定的LoRA适配器，通过学习的逐层适配器混合，仅需少量目标风格训练样本即可快速训练针对每个新目标的专用适应模型。AuthorMix在低资源目标上超越了现有最先进风格迁移基线以及GPT-5.1，取得了最高总分并显著提升了语义保持效果。
URL
https://arxiv.org/abs/2603.23069
PDF
-
SelfTTS: cross-speaker style transfer through explicit embedding disentanglement and self-refinement using self-augmentation
2026-03-23 17:45:03Lucas H. Ueda, Jo\~ao G. T. Lima, Pedro R. Corr\^ea, Fl\'avio O. Sim\~oes, M\'ario U. Neto, Paula D. P. CostaAbstract
This paper presents SelfTTS, a text-to-speech (TTS) model designed for cross-speaker style transfer that eliminates the need for external pre-trained speaker or emotion encoders. The architecture achieves emotional expressivity in neutral speakers through an explicit disentanglement strategy utilizing Gradient Reversal Layers (GRL) combined with cosine similarity loss to decouple speaker and emotion information. We introduce Multi Positive Contrastive Learning (MPCL) to induce clustered representations of speaker and emotion embeddings based on their respective labels. Furthermore, SelfTTS employs a self-refinement strategy via Self-Augmentation, exploiting the model's voice conversion capabilities to enhance the naturalness of synthesized speech. Experimental results demonstrate that SelfTTS achieves superior emotional naturalness (eMOS) and robust stability in target timbre and emotion compared to state-of-the-art baselines.
Abstract (translated)
本文提出SelfTTS，一种旨在实现跨说话人风格迁移的文本转语音（TTS）模型，该模型无需依赖外部预训练的说话人或情感编码器。其架构通过结合梯度反转层（GRL）与余弦相似度损失的显式解耦策略，将说话人与情感信息分离，从而在中性说话人中实现情感表达性。我们引入多正例对比学习（MPCL），基于说话人与情感的标签诱导其嵌入表示的聚类。此外，SelfTTS采用通过自增强（Self-Augmentation）实现的自我精炼策略，利用模型的语音转换能力提升合成语音的自然度。实验结果表明，与最先进的基线方法相比，SelfTTS在目标音色与情感控制上具有更优的情感自然度（eMOS）和稳健稳定性。
URL
https://arxiv.org/abs/2603.22252
PDF
-
3D Multi-View Stylization with Pose-Free Correspondences Matching for Robust 3D Geometry Preservation
2026-03-22 01:23:34Shirsha BoseAbstract
Artistic style transfer is well studied for images and videos, but extending it to multi-view 3D scenes remains difficult because stylization can disrupt correspondences needed by geometry-aware pipelines. Independent per-view stylization often causes texture drift, warped edges, and inconsistent shading, degrading SLAM, depth prediction, and multi-view reconstruction. This thesis addresses multi-view stylization that remains usable for downstream 3D tasks without assuming camera poses or an explicit 3D representation during training. We introduce a feed-forward stylization network trained with per-scene test-time optimization under a composite objective coupling appearance transfer with geometry preservation. Stylization is driven by an AdaIN-inspired loss from a frozen VGG-19 encoder, matching channel-wise moments to a style image. To stabilize structure across viewpoints, we propose a correspondence-based consistency loss using SuperPoint and SuperGlue, constraining descriptors from a stylized anchor view to remain consistent with matched descriptors from the original multi-view set. We also impose a depth-preservation loss using MiDaS/DPT and use global color alignment to reduce depth-model domain shift. A staged weight schedule introduces geometry and depth constraints. We evaluate on Tanks and Temples and Mip-NeRF 360 using image and reconstruction metrics. Style adherence and structure retention are measured by Color Histogram Distance (CHD) and Structure Distance (DSD). For 3D consistency, we use monocular DROID-SLAM trajectories and symmetric Chamfer distance on back-projected point clouds. Across ablations, correspondence and depth regularization reduce structural distortion and improve SLAM stability and reconstructed geometry; on scenes with MuVieCAST baselines, our method yields stronger trajectory and point-cloud consistency while maintaining competitive stylization.
Abstract (translated)
URL
https://arxiv.org/abs/2604.09639
PDF
-
X-World: Controllable Ego-Centric Multi-Camera World Models for Scalable End-to-End Driving
2026-03-20 14:23:32Chaoda Zheng, Sean Li, Jinhao Deng, Zhennan Wang, Shijia Chen, Liqiang Xiao, Ziheng Chi, Hongbin Lin, Kangjie Chen, Boyang Wang, Yu Zhang, Xianming LiuAbstract
Scalable and reliable evaluation is increasingly critical in the end-to-end era of autonomous driving, where vision--language--action (VLA) policies directly map raw sensor streams to driving actions. Yet, current evaluation pipelines still rely heavily on real-world road testing, which is costly, biased toward limited scenario coverage, and difficult to reproduce. These challenges motivate a real-world simulator that can generate realistic future observations under proposed actions, while remaining controllable and stable over long horizons. We present X-World, an action-conditioned multi-camera generative world model that simulates future observations directly in video space. Given synchronized multi-view camera history and a future action sequence, X-World generates future multi-camera video streams that follow the commanded actions. To ensure reproducible and editable scene rollouts, X-World further supports optional controls over dynamic traffic agents and static road elements, and retains a text-prompt interface for appearance-level control (e.g., weather and time of day). Beyond world simulation, X-World also enables video style transfer by conditioning on appearance prompts while preserving the underlying action and scene dynamics. At the core of X-World is a multi-view latent video generator designed to explicitly encourage cross-view geometric consistency and temporal coherence under diverse control signals. Experiments show that X-World achieves high-quality multi-view video generation with (i) strong view consistency across cameras, (ii) stable temporal dynamics over long rollouts, and (iii) high controllability with strict action following and faithful adherence to optional scene controls. These properties make X-World a practical foundation for scalable and reproducible evaluation.
Abstract (translated)
可扩展且可靠的评估在自动驾驶端到端时代日益关键，其中视觉-语言-动作（VLA）策略直接将原始传感器流映射为驾驶动作。然而当前评估流程仍高度依赖真实路测，该方法成本高昂、场景覆盖有限且难以复现。这些挑战催生了对真实世界模拟器的需求——它需能在提议动作下生成逼真的未来观测，同时保持长期可控性与稳定性。我们提出X-World，一个动作条件的多摄像头生成式世界模型，直接在视频空间中模拟未来观测。给定同步多视角摄像头历史与未来动作序列，X-World可生成遵循指令动作的未来多摄像头视频流。为确保场景推演的可复现与可编辑性，X-World进一步支持对动态交通参与者与静态道路元素的可选控制，并保留文本提示接口以实现外观级控制（如天气与时段）。除世界模拟外，X-World还能通过外观提示条件实现视频风格迁移，同时保持底层动作与场景动态。X-World的核心是一个多视角隐空间视频生成器，专为在不同控制信号下显式促进跨视角几何一致性与时间连贯性而设计。实验表明，X-World实现了高质量多视角视频生成，具备：（i）摄像头间强视角一致性；（ii）长期推演中稳定的时间动态；（iii）通过严格动作跟随与忠实执行可选场景控制实现的高可控性。这些特性使X-World成为可扩展、可复现评估的实用基础。
URL
https://arxiv.org/abs/2603.19979
PDF
-
SparkVSR: Interactive Video Super-Resolution via Sparse Keyframe Propagation
2026-03-17 17:59:30Jiongze Yu, Xiangbo Gao, Pooja Verlani, Akshay Gadde, Yilin Wang, Balu Adsumilli, Zhengzhong TuAbstract
Video Super-Resolution (VSR) aims to restore high-quality video frames from low-resolution (LR) estimates, yet most existing VSR approaches behave like black boxes at inference time: users cannot reliably correct unexpected artifacts, but instead can only accept whatever the model produces. In this paper, we propose a novel interactive VSR framework dubbed SparkVSR that makes sparse keyframes a simple and expressive control signal. Specifically, users can first super-resolve or optionally a small set of keyframes using any off-the-shelf image super-resolution (ISR) model, then SparkVSR propagates the keyframe priors to the entire video sequence while remaining grounded by the original LR video motion. Concretely, we introduce a keyframe-conditioned latent-pixel two-stage training pipeline that fuses LR video latents with sparsely encoded HR keyframe latents to learn robust cross-space propagation and refine perceptual details. At inference time, SparkVSR supports flexible keyframe selection (manual specification, codec I-frame extraction, or random sampling) and a reference-free guidance mechanism that continuously balances keyframe adherence and blind restoration, ensuring robust performance even when reference keyframes are absent or imperfect. Experiments on multiple VSR benchmarks demonstrate improved temporal consistency and strong restoration quality, surpassing baselines by up to 24.6%, 21.8%, and 5.6% on CLIP-IQA, DOVER, and MUSIQ, respectively, enabling controllable, keyframe-driven video super-resolution. Moreover, we demonstrate that SparkVSR is a generic interactive, keyframe-conditioned video processing framework as it can be applied out of the box to unseen tasks such as old-film restoration and video style transfer. Our project page is available at: this https URL
Abstract (translated)
URL
https://arxiv.org/abs/2603.16864
PDF
-
StyleGallery: Training-free and Semantic-aware Personalized Style Transfer from Arbitrary Image References
2026-03-11 03:05:02Boyu He (College of Computer Science and Technology, National University of Defense Technology), Yunfan Ye (School of Design, Hunan University), Chang Liu (College of Computer Science and Technology, National University of Defense Technology), Weishang Wu (College of Computer Science and Technology, National University of Defense Technology), Fang Liu (School of Design, Hunan University), Zhiping Cai (College of Computer Science and Technology, National University of Defense Technology)Abstract
Despite the advancements in diffusion-based image style transfer, existing methods are commonly limited by 1) semantic gap: the style reference could miss proper content semantics, causing uncontrollable stylization; 2) reliance on extra constraints (e.g., semantic masks) restricting applicability; 3) rigid feature associations lacking adaptive global-local alignment, failing to balance fine-grained stylization and global content preservation. These limitations, particularly the inability to flexibly leverage style inputs, fundamentally restrict style transfer in terms of personalization, accuracy, and adaptability. To address these, we propose StyleGallery, a training-free and semantic-aware framework that supports arbitrary reference images as input and enables effective personalized customization. It comprises three core stages: semantic region segmentation (adaptive clustering on latent diffusion features to divide regions without extra inputs); clustered region matching (block filtering on extracted features for precise alignment); and style transfer optimization (energy function-guided diffusion sampling with regional style loss to optimize stylization). Experiments on our introduced benchmark demonstrate that StyleGallery outperforms state-of-the-art methods in content structure preservation, regional stylization, interpretability, and personalized customization, particularly when leveraging multiple style references.
Abstract (translated)
尽管基于扩散的图像风格迁移技术取得了进步，现有的方法通常受限于以下几点：1)语义差距：风格参考可能缺少适当的内容语义，导致无法控制的风格化；2)对额外约束（如语义蒙版）的依赖，限制了适用性；3)刚性的特征关联缺乏自适应全局-局部对齐能力，在精细风格化和全局内容保留之间难以取得平衡。这些局限性，特别是不能灵活利用风格输入的问题，从根本上限制了风格迁移在个性化、准确性和适应性方面的表现。 为了解决这些问题，我们提出了StyleGallery，这是一个无需训练且具有语义感知的框架，支持任意参考图像作为输入，并能够实现有效的个性化定制。它包含三个核心阶段：语义区域分割（通过隐式扩散特征自适应聚类来划分区域，不依赖额外输入）；集群区域匹配（在提取的特征上执行块过滤以实现精确对齐）；以及风格迁移优化（使用能量函数指导下的扩散采样和区域样式损失来优化风格化）。我们在引入的基准测试上的实验表明，StyleGallery 在内容结构保留、区域风格化、可解释性和个性化定制方面优于现有最先进的方法，尤其是在利用多个风格参考时表现尤为突出。
URL
https://arxiv.org/abs/2603.10354
PDF
-
CARE-Edit: Condition-Aware Routing of Experts for Contextual Image Editing
2026-03-09 16:40:47Yucheng Wang, Zedong Wang, Yuetong Wu, Yue Ma, Dan XuAbstract
Unified diffusion editors often rely on a fixed, shared backbone for diverse tasks, suffering from task interference and poor adaptation to heterogeneous demands (e.g., local vs global, semantic vs photometric). In particular, prevalent ControlNet and OmniControl variants combine multiple conditioning signals (e.g., text, mask, reference) via static concatenation or additive adapters which cannot dynamically prioritize or suppress conflicting modalities, thus resulting in artifacts like color bleeding across mask boundaries, identity or style drift, and unpredictable behavior under multi-condition inputs. To address this, we propose Condition-Aware Routing of Experts (CARE-Edit) that aligns model computation with specific editing competencies. At its core, a lightweight latent-attention router assigns encoded diffusion tokens to four specialized experts--Text, Mask, Reference, and Base--based on multi-modal conditions and diffusion timesteps: (i) a Mask Repaint module first refines coarse user-defined masks for precise spatial guidance; (ii) the router applies sparse top-K selection to dynamically allocate computation to the most relevant experts; (iii) a Latent Mixture module subsequently fuses expert outputs, coherently integrating semantic, spatial, and stylistic information to the base images. Experiments validate CARE-Edit's strong performance on contextual editing tasks, including erasure, replacement, text-driven edits, and style transfer. Empirical analysis further reveals task-specific behavior of specialized experts, showcasing the importance of dynamic, condition-aware processing to mitigate multi-condition conflicts.
Abstract (translated)
统一扩散编辑器通常依赖于一个固定的、共享的骨干网络来执行各种任务，这会导致任务干扰和对异构需求（例如局部与全局、语义与光度）适应性差的问题。特别是流行的ControlNet和OmniControl变体通过静态拼接或加法适配器结合多种条件信号（如文本、掩码、参考图像），但这些方法无法动态地优先处理或抑制冲突模态，因此在多条件输入下会出现诸如跨掩码边界的颜色溢出、身份或风格漂移以及行为不可预测等问题。为解决这些问题，我们提出了Condition-Aware Routing of Experts (CARE-Edit)，该方法使模型计算与特定编辑任务的能力相匹配。核心思想是一个轻量级的潜在注意力路由器根据多模态条件和扩散时间步骤将编码的扩散令牌分配给四个专门化的专家——文本、掩码、参考图像和基础图像： 1. 掩码重绘模块首先对粗略的用户定义掩码进行细化，以提供精确的空间指导。 2. 路由器应用稀疏的Top-K选择来动态地将计算资源分配给最相关的专家。 3. 潜在混合模块随后融合各个专家的输出，将语义、空间和风格信息连贯地整合到基础图像中。 实验验证了CARE-Edit在上下文编辑任务（包括擦除、替换、文本驱动的编辑和风格转移）中的强大性能。实证分析进一步揭示了专门化专家的任务特定行为，展示了动态、条件感知处理的重要性以减轻多条件冲突的影响。
URL
https://arxiv.org/abs/2603.08589
PDF
-
Real-Time Monocular Scene Analysis for UAV in Outdoor Environments
2026-03-09 14:08:50Yara AlaaEldinAbstract
In this thesis, we leverage monocular cameras on aerial robots to predict depth and semantic maps in low-altitude unstructured environments. We propose a joint deep-learning architecture, named Co-SemDepth, that can perform the two tasks accurately and rapidly, and validate its effectiveness on a variety of datasets. The training of neural networks requires an abundance of annotated data, and in the UAV field, the availability of such data is limited. We introduce a new synthetic dataset in this thesis, TopAir that contains images captured with a nadir view in outdoor environments at different altitudes, helping to fill the gap. While using synthetic data for the training is convenient, it raises issues when shifting to the real domain for testing. We conduct an extensive analytical study to assess the effect of several factors on the synthetic-to-real generalization. Co-SemDepth and TaskPrompter models are used for comparison in this study. The results reveal a superior generalization performance for Co-SemDepth in depth estimation and for TaskPrompter in semantic segmentation. Also, our analysis allows us to determine which training datasets lead to a better generalization. Moreover, to help attenuate the gap between the synthetic and real domains, image style transfer techniques are explored on aerial images to convert from the synthetic to the realistic style. Cycle-GAN and Diffusion models are employed. The results reveal that diffusion models are better in the synthetic to real style transfer. In the end, we focus on the marine domain and address its challenges. Co-SemDepth is trained on a collected synthetic marine data, called MidSea, and tested on both synthetic and real data. The results reveal good generalization performance of Co-SemDepth when tested on real data from the SMD dataset while further enhancement is needed on the MIT dataset.
Abstract (translated)
URL
https://arxiv.org/abs/2603.13368
PDF
-
Parameterized Brushstroke Style Transfer
Abstract
Computer Vision-based Style Transfer techniques have been used for many years to represent artistic style. However, most contemporary methods have been restricted to the pixel domain; in other words, the style transfer approach has been modifying the image pixels to incorporate artistic style. However, real artistic work is made of brush strokes with different colors on a canvas. Pixel-based approaches are unnatural for representing these images. Hence, this paper discusses a style transfer method that represents the image in the brush stroke domain instead of the RGB domain, which has better visual improvement over pixel-based methods.
Abstract (translated)
基于计算机视觉的风格迁移技术多年来一直被用于表现艺术风格。然而，大多数现代方法都局限于像素领域；换句话说，风格迁移的方法主要是通过修改图像像素来融入艺术风格。但是，真正的艺术品是由不同颜色的笔触在画布上构成的。基于像素的方法对于表示这些图片来说是不自然的。因此，本文提出了一种新的风格迁移方法，它不是在RGB域中表现图像，而是在笔刷痕迹域中进行，这种方法在视觉效果上的改进要优于传统的基于像素的方法。
URL
https://arxiv.org/abs/2603.07776
PDF
-
QdaVPR: A novel query-based domain-agnostic model for visual place recognition
2026-03-08 02:01:20Shanshan Wan, Lai Kang, Yingmei Wei, Tianrui Shen, Haixuan Wang, Chao ZuoAbstract
Visual place recognition (VPR) aiming at predicting the location of an image based solely on its visual features is a fundamental task in robotics and autonomous systems. Domain variation remains one of the main challenges in VPR and is relatively unexplored. Existing VPR models attempt to achieve domain agnosticism either by training on large-scale datasets that inherently contain some domain variations, or by being specifically adapted to particular target domains. In practice, the former lacks explicit domain supervision, while the latter generalizes poorly to unseen domain shifts. This paper proposes a novel query-based domain-agnostic VPR model called QdaVPR. First, a dual-level adversarial learning framework is designed to encourage domain invariance for both the query features forming the global descriptor and the image features from which these query features are derived. Then, a triplet supervision based on query combinations is designed to enhance the discriminative power of the global descriptors. To support the learning process, we augment a large-scale VPR dataset using style transfer methods, generating various synthetic domains with corresponding domain labels as auxiliary supervision. Extensive experiments show that QdaVPR achieves state-of-the-art performance on multiple VPR benchmarks with significant domain variations. Specifically, it attains the best Recall@1 and Recall@10 on nearly all test scenarios: 93.5%/98.6% on Nordland (seasonal changes), 97.5%/99.0% on Tokyo24/7 (day-night transitions), and the highest Recall@1 across almost all weather conditions on the SVOX dataset. Our code will be released at this https URL.
Abstract (translated)
视觉位置识别（VPR）旨在仅基于图像的视觉特征预测其位置，这是机器人技术和自主系统中的一个基本任务。领域变化仍然是VPR的主要挑战之一，并且研究相对较少。现有的VPR模型试图通过在大规模数据集上进行训练来实现对领域的无感知性，这些数据集中固有地包含了一些领域变化，或者通过专门适应特定的目标域来进行这种尝试。实践中，前者缺乏明确的领域监督，而后者则难以泛化到未见的领域偏移中。 本文提出了一种新颖的查询基、领域无关型VPR模型——QdaVPR。首先设计了一个双层对抗性学习框架，该框架鼓励全局描述符中的查询特征以及这些查询特征所衍生出的图像特征达到领域不变性。其次，基于查询组合设计了三元监督机制，以增强全局描述符的区分能力。为了支持学习过程，我们通过使用风格迁移方法扩展了一大规模VPR数据集，生成多种合成域，并提供相应的领域标签作为辅助监督。 广泛的实验表明，QdaVPR在多个具有显著领域变化的VPR基准测试上实现了最佳性能。具体而言，在几乎所有的测试场景中，它达到了最好的Recall@1和Recall@10：Nordland（季节变化）为93.5%/98.6%，Tokyo24/7（昼夜转换）为97.5%/99.0%；在SVOX数据集的几乎所有天气条件下，它也取得了最高的Recall@1。我们的代码将在[此处发布](此链接应替换为实际提供的URL)。
URL
https://arxiv.org/abs/2603.07414
PDF
-
The Art That Poses Back: Assessing AI Pastiches after Contemporary Artworks
2026-03-06 14:25:55Anca Dinu, Andreiana Mihail, Andra-Maria Florescu, Claudiu CreangaAbstract
This study explores artificial visual creativity, focusing on ChatGPT's ability to generate new images intentionally pastiching original artworks such as paintings, drawings, sculptures and installations. The process involved twelve artists from Romania, Bulgaria, France, Austria, and the United Kingdom, each invited to contribute with three of their artworks and to grade and comment on the AI-generated versions. The analysis combines human evaluation with computational methods aimed at detecting visual and stylistic similarities or divergences between the original works and their AI-produced renditions. The results point to a significant gap between color and texture-based similarity and compositional, conceptual, and perceptual one. Consequently, we advocate for the use of a "style transfer dashboard" of complementary metrics to evaluate the similarity between pastiches and originals, rather than using a single style metric. The artists' comments revealed limitations of ChatGPT's pastiches after contemporary artworks, which were perceived by the authors of the originals as lacking dimensionality, context, and intentional sense, and seeming more of a paraphrase or an approximate quotation rather than as a valuable, emotion-evoking artwork.
Abstract (translated)
这项研究探讨了人工智能的视觉创造力，重点关注ChatGPT生成新图像的能力，这些新图像是对原作如绘画、素描、雕塑和装置艺术进行有意模仿的结果。该过程邀请来自罗马尼亚、保加利亚、法国、奥地利和英国的十二位艺术家参与，每位艺术家贡献三件自己的作品，并对其AI生成版本进行评分和评论。分析结合了人类评价与计算方法，旨在检测原始作品与其人工智能生成版本之间的视觉和风格上的相似性或差异。 研究结果表明，在颜色和纹理基础上的相似性和构成、概念及感知层面的相似性之间存在显著差距。因此，我们倡导使用一个“风格转换仪表板”，采用互补指标来评估模仿作品与原作之间的相似度，而不是依赖单一的风格指标。 艺术家们的评论揭示了ChatGPT在对当代艺术品进行模仿时存在的局限性。原作者认为这些模仿缺乏深度、背景信息和有意图的感觉，并且看起来更像是引用或近似复制，而非具有情感价值的艺术品。
URL
https://arxiv.org/abs/2603.06324
PDF
-
Machine Pareidolia: Protecting Facial Image with Emotional Editing
2026-03-04 02:39:26Binh M. Le, Simon S. WooAbstract
The proliferation of facial recognition (FR) systems has raised privacy concerns in the digital realm, as malicious uses of FR models pose a significant threat. Traditional countermeasures, such as makeup style transfer, have suffered from low transferability in black-box settings and limited applicability across various demographic groups, including males and individuals with darker skin tones. To address these challenges, we introduce a novel facial privacy protection method, dubbed \textbf{MAP}, a pioneering approach that employs human emotion modifications to disguise original identities as target identities in facial images. Our method uniquely fine-tunes a score network to learn dual objectives, target identity and human expression, which are jointly optimized through gradient projection to ensure convergence at a shared local optimum. Additionally, we enhance the perceptual quality of protected images by applying local smoothness regularization and optimizing the score matching loss within our network. Empirical experiments demonstrate that our innovative approach surpasses previous baselines, including noise-based, makeup-based, and freeform attribute methods, in both qualitative fidelity and quantitative metrics. Furthermore, MAP proves its effectiveness against an online FR API and shows advanced adaptability in uncommon photographic scenarios.
Abstract (translated)
面部识别（FR）系统的普及在数字领域引发了隐私问题，因为恶意使用这些系统对个人安全构成了重大威胁。传统的对策，如化妆风格转移，在黑盒设置中传输效果不佳，并且在不同的人群，包括男性和肤色较深的个体之间适用性有限。为了解决这些问题，我们提出了一种新的面部隐私保护方法——称为**MAP**的方法，它采用人类情感修改技术来将原始身份伪装成目标身份。我们的方法独特地调整了一个评分网络以同时学习两个目标：目标身份和人类表情，并通过梯度投影联合优化这两个目标，确保在共享局部最优中收敛。 此外，我们通过应用局部平滑正则化并优化评分匹配损失来提高保护图像的感知质量。实验证明，我们的创新方法在定性和定量指标上都超越了以前的基础线方法，包括基于噪声、化妆和自由形式属性的方法。此外，MAP证明了其对抗在线面部识别API的有效性，并展示了在不常见摄影场景中的先进适应能力。
URL
https://arxiv.org/abs/2603.03665
PDF
-
DiffusionXRay: A Diffusion and GAN-Based Approach for Enhancing Digitally Reconstructed Chest Radiographs
2026-03-02 10:14:50Aryan Goyal, Ashish Mittal, Pranav Rao, Manoj Tadepalli, Preetham PuthaAbstract
Deep learning-based automated diagnosis of lung cancer has emerged as a crucial advancement that enables healthcare professionals to detect and initiate treatment earlier. However, these models require extensive training datasets with diverse case-specific properties. High-quality annotated data is particularly challenging to obtain, especially for cases with subtle pulmonary nodules that are difficult to detect even for experienced radiologists. This scarcity of well-labeled datasets can limit model performance and generalization across different patient populations. Digitally reconstructed radiographs (DRR) using CT-Scan to generate synthetic frontal chest X-rays with artificially inserted lung nodules offers one potential solution. However, this approach suffers from significant image quality degradation, particularly in the form of blurred anatomical features and loss of fine lung field structures. To overcome this, we introduce DiffusionXRay, a novel image restoration pipeline for Chest X-ray images that synergistically leverages denoising diffusion probabilistic models (DDPMs) and generative adversarial networks (GANs). DiffusionXRay incorporates a unique two-stage training process: First, we investigate two independent approaches, DDPM-LQ and GAN-based MUNIT-LQ, to generate low-quality CXRs, addressing the challenge of training data scarcity, posing this as a style transfer problem. Subsequently, we train a DDPM-based model on paired low-quality and high-quality images, enabling it to learn the nuances of X-ray image restoration. Our method demonstrates promising results in enhancing image clarity, contrast, and overall diagnostic value of chest X-rays while preserving subtle yet clinically significant artifacts, validated by both quantitative metrics and expert radiological assessment.
Abstract (translated)
基于深度学习的肺癌自动诊断技术已经成为一项重要的进展，使医疗专业人员能够更早地检测和启动治疗。然而，这些模型需要大量的训练数据集，并且这些数据集中应包含各种特定病例的特点。特别是高质量的标注数据对于细微肺结节（即使是经验丰富的放射科医生也难以发现）的情况来说尤其难以获取。这种良好标记的数据缺乏可能会限制模型在不同患者群体中的表现和泛化能力。利用CT扫描生成合成胸部正位X光片，这种方法通过数字重建射线照片(DRR)并在图像中人工插入肺结节来提供了一种可能的解决方案。然而，这种方法会导致显著的图像质量下降，特别是在解剖特征模糊和细微肺野结构丢失方面。 为了解决这些问题，我们引入了DiffusionXRay，这是一种新的用于胸部X光片图像恢复的技术流程，它结合了去噪扩散概率模型（DDPM）和生成对抗网络（GAN）。DiffusionXRay采用了一种独特的两阶段训练过程：首先，我们研究两种独立的方法——DDPM-LQ 和 GAN-based MUNIT-LQ——来生成低质量的胸部X光片（CXRs），以此应对训练数据不足的问题，并将其视为一种风格迁移问题。然后，我们使用一对低质量和高质量图像对基于DDPM的模型进行训练，使其能够学习到X射线图像恢复中的细微差别。 我们的方法在提高胸部X光片的清晰度、对比度以及整体诊断价值方面显示出有希望的结果，并且验证了该方法能够在保留临床意义的微妙结构的同时保持良好的定性和专家放射学评估结果。
URL
https://arxiv.org/abs/2603.01686
PDF
-
Benchmarking Semantic Segmentation Models via Appearance and Geometry Attribute Editing
2026-03-02 07:05:37Zijin Yin, Bing Li, Kongming Liang, Hao Sun, Zhongjiang He, Zhanyu Ma, Jun GuoAbstract
Semantic segmentation takes pivotal roles in various applications such as autonomous driving and medical image analysis. When deploying segmentation models in practice, it is critical to test their behaviors in varied and complex scenes in advance. In this paper, we construct an automatic data generation pipeline Gen4Seg to stress-test semantic segmentation models by generating various challenging samples with different attribute changes. Beyond previous evaluation paradigms focusing solely on global weather and style transfer, we investigate variations in both appearance and geometry attributes at the object and image level. These include object color, material, size, position, as well as image-level variations such as weather and style. To achieve this, we propose to edit visual attributes of existing real images with precise control of structural information, empowered by diffusion models. In this way, the existing segmentation labels can be reused for the edited images, which greatly reduces the labor costs. Using our pipeline, we construct two new benchmarks, Pascal-EA and COCO-EA. We benchmark a wide variety of semantic segmentation models, spanning from closed-set models to open-vocabulary large models. We have several key findings: 1) advanced open-vocabulary models do not exhibit greater robustness compared to closed-set methods under geometric variations; 2) data augmentation techniques, such as CutOut and CutMix, are limited in enhancing robustness against appearance variations; 3) our pipeline can also be employed as a data augmentation tool and improve both in-distribution and out-of-distribution performances. Our work suggests the potential of generative models as effective tools for automatically analyzing segmentation models, and we hope our findings will assist practitioners and researchers in developing more robust and reliable segmentation models.
Abstract (translated)
语义分割在自动驾驶和医学图像分析等多种应用中扮演着关键角色。当将分割模型部署到实际场景时，提前测试它们在各种复杂环境中的表现是至关重要的。在这篇论文中，我们构建了一个自动数据生成管道Gen4Seg，通过生成具有不同属性变化的各种挑战性样本来压力测试语义分割模型。与以往只关注全局天气和风格转换的评估范式不同，我们研究了对象级和图像级别上外观和几何属性的变化情况。这些变化包括物体的颜色、材质、尺寸、位置以及图像级别的天气和风格等。为了实现这一目标，我们提出利用扩散模型对现有真实图像的视觉属性进行编辑，并且能够精确控制结构信息。这样可以重用原有的分割标签来编辑后的图片，大大减少了人工劳动成本。 使用我们的管道，我们构建了两个新的基准测试集：Pascal-EA和COCO-EA。我们将各种语义分割模型进行了广泛的评估，这些模型涵盖了从封闭集合模型到开放词汇大模型的不同类型。我们有以下几个主要发现： 1. 在几何变化的情况下，先进的开放词汇模型并不比封闭集合方法表现出更强的鲁棒性。 2. 数据增强技术（如CutOut和CutMix）在对抗外观变化时有限于提升模型的鲁棒性。 3. 我们的管道也可以用作数据增强工具，并且能够改善分布内和分布外的表现。 我们的工作展示了生成模型作为自动分析分割模型的有效工具的巨大潜力，希望我们的研究发现能帮助从业者和研究人员开发出更加稳健可靠的分割模型。
URL
https://arxiv.org/abs/2603.01535
PDF
-
LoR-LUT: Learning Compact 3D Lookup Tables via Low-Rank Residuals
2026-02-26 04:28:35Ziqi Zhao, Abhijit Mishra, Shounak RoychowdhuryAbstract
We present LoR-LUT, a unified low-rank formulation for compact and interpretable 3D lookup table (LUT) generation. Unlike conventional 3D-LUT-based techniques that rely on fusion of basis LUTs, which are usually dense tensors, our unified approach extends the current framework by jointly using residual corrections, which are in fact low-rank tensors, together with a set of basis LUTs. The approach described here improves the existing perceptual quality of an image, which is primarily due to the technique's novel use of residual corrections. At the same time, we achieve the same level of trilinear interpolation complexity, using a significantly smaller number of network, residual corrections, and LUT parameters. The experimental results obtained from LoR-LUT, which is trained on the MIT-Adobe FiveK dataset, reproduce expert-level retouching characteristics with high perceptual fidelity and a sub-megabyte model size. Furthermore, we introduce an interactive visualization tool, termed LoR-LUT Viewer, which transforms an input image into the LUT-adjusted output image, via a number of slidebars that control different parameters. The tool provides an effective way to enhance interpretability and user confidence in the visual results. Overall, our proposed formulation offers a compact, interpretable, and efficient direction for future LUT-based image enhancement and style transfer.
Abstract (translated)
我们提出了LoR-LUT，这是一种用于生成紧凑且可解释的3D查找表（LUT）的统一低秩表示方法。与传统的基于3D-LUT的方法不同，后者依赖于基LUT的融合，而这些基LUT通常是密集张量，我们的统一方法通过同时使用残差校正来扩展当前框架，这些残差实际上是低秩张量，并与一组基LUT结合使用。这里描述的方法主要由于其对残差校正的新颖应用，提高了图像现有的感知质量。与此同时，我们利用了显著较少的网络、残差校正和LUT参数，在保持相同的三线性插值复杂度的情况下实现了这一目标。 在MIT-Adobe FiveK数据集上训练得到的LoR-LUT模型能够以不到1MB的大小高度逼真地复现专家级图像修饰特性，具有良好的感知保真度。此外，我们还引入了一个交互式可视化工具，名为LoR-LUT Viewer，通过调整多个滑块来控制不同参数，该工具可以将输入图片转换为经过LUT校准后的输出图片。此工具提供了一种有效的方法以增强视觉结果的可解释性和用户信心。 总体而言，我们的方法提供了未来基于LUT的图像增强和风格迁移的一种紧凑、可解释且高效的途径。
URL
https://arxiv.org/abs/2602.22607
PDF
-
CleanStyle: Plug-and-Play Style Conditioning Purification for Text-to-Image Stylization
2026-02-24 09:33:05Xiaoman Feng, Mingkun Lei, Yang Wang, Dingwen Fu, Chi ZhangAbstract
Style transfer in diffusion models enables controllable visual generation by injecting the style of a reference image. However, recent encoder-based methods, while efficient and tuning-free, often suffer from content leakage, where semantic elements from the style image undesirably appear in the output, impairing prompt fidelity and stylistic consistency. In this work, we introduce CleanStyle, a plug-and-play framework that filters out content-related noise from the style embedding without retraining. Motivated by empirical analysis, we observe that such leakage predominantly stems from the tail components of the style embedding, which are isolated via Singular Value Decomposition (SVD). To address this, we propose CleanStyleSVD (CS-SVD), which dynamically suppresses tail components using a time-aware exponential schedule, providing clean, style-preserving conditional embeddings throughout the denoising process. Furthermore, we present Style-Specific Classifier-Free Guidance (SS-CFG), which reuses the suppressed tail components to construct style-aware unconditional inputs. Unlike conventional methods that use generic negative embeddings (e.g., zero vectors), SS-CFG introduces targeted negative signals that reflect style-specific but prompt-irrelevant visual elements. This enables the model to effectively suppress these distracting patterns during generation, thereby improving prompt fidelity and enhancing the overall visual quality of stylized outputs. Our approach is lightweight, interpretable, and can be seamlessly integrated into existing encoder-based diffusion models without retraining. Extensive experiments demonstrate that CleanStyle substantially reduces content leakage, improves stylization quality and improves prompt alignment across a wide range of style references and prompts.
Abstract (translated)
在扩散模型中，风格转换技术通过注入参考图像的风格实现可控的视觉生成。然而，最近基于编码器的方法尽管高效且无需调整参数，但常常会遭受内容渗漏的问题：即从风格图提取出的一些语义元素不希望地出现在输出结果中，从而损害了提示准确性和风格一致性。为此，在这项工作中我们引入了一种名为CleanStyle的插件式框架，该框架能够在无需重新训练的情况下过滤掉与内容相关的噪声。 基于实证分析，我们观察到这种渗漏主要来源于样式嵌入中的尾部成分（通过奇异值分解SVD来隔离）。为解决这一问题，我们提出了CleanStyleSVD (CS-SVD)，一种使用时间感知指数调度动态抑制尾部组件的方法。这在整个去噪过程中提供了既干净又保持风格的条件嵌入。 此外，我们还提出了一种风格特定的无分类器引导（Style-Specific Classifier-Free Guidance, SS-CFG）方法，该方法重用被压制的尾部成分来构建具有感知到的风格但与提示无关的无条件输入。不同于传统的方法使用通用负向嵌入（如零向量），SS-CFG引入了针对特定风格但与提示无关的视觉元素的目标化负面信号。这使模型能够在生成过程中有效抑制这些干扰模式，从而提高了提示准确性并提升了整体视觉质量。 我们的方法重量级轻、解释性强，并且可以无缝地集成到现有的基于编码器的扩散模型中而无需重新训练。大量的实验表明，CleanStyle大幅减少了内容渗漏，改进了风格化质量，增强了在广泛风格参考和提示下的指令对齐度。
URL
https://arxiv.org/abs/2602.20721
PDF
