Face_Detection
-
Glass Surface Detection: Leveraging Reflection Dynamics in Flash/No-flash Imagery
2025-11-21 02:00:17Tao Yan, Hao Huang, Yiwei Lu, Zeyu Wang, Ke Xu, Yinghui Wang, Xiaojun Chang, Rynson W. H. LauAbstract
Glass surfaces are ubiquitous in daily life, typically appearing colorless, transparent, and lacking distinctive features. These characteristics make glass surface detection a challenging computer vision task. Existing glass surface detection methods always rely on boundary cues (e.g., window and door frames) or reflection cues to locate glass surfaces, but they fail to fully exploit the intrinsic properties of the glass itself for accurate localization. We observed that in most real-world scenes, the illumination intensity in front of the glass surface differs from that behind it, which results in variations in the reflections visible on the glass surface. Specifically, when standing on the brighter side of the glass and applying a flash towards the darker side, existing reflections on the glass surface tend to disappear. Conversely, while standing on the darker side and applying a flash towards the brighter side, distinct reflections will appear on the glass surface. Based on this phenomenon, we propose NFGlassNet, a novel method for glass surface detection that leverages the reflection dynamics present in flash/no-flash imagery. Specifically, we propose a Reflection Contrast Mining Module (RCMM) for extracting reflections, and a Reflection Guided Attention Module (RGAM) for fusing features from reflection and glass surface for accurate glass surface detection. For learning our network, we also construct a dataset consisting of 3.3K no-flash and flash image pairs captured from various scenes with corresponding ground truth annotations. Extensive experiments demonstrate that our method outperforms the state-of-the-art methods. Our code, model, and dataset will be available upon acceptance of the manuscript.
Abstract (translated)
URL
https://arxiv.org/abs/2511.16887
PDF
-
Quality-Controlled Multimodal Emotion Recognition in Conversations with Identity-Based Transfer Learning and MAMBA Fusion
2025-11-18 23:24:27Zanxu Wang, Homayoon BeigiAbstract
This paper addresses data quality issues in multimodal emotion recognition in conversation (MERC) through systematic quality control and multi-stage transfer learning. We implement a quality control pipeline for MELD and IEMOCAP datasets that validates speaker identity, audio-text alignment, and face detection. We leverage transfer learning from speaker and face recognition, assuming that identity-discriminative embeddings capture not only stable acoustic and Facial traits but also person-specific patterns of emotional expression. We employ RecoMadeEasy(R) engines for extracting 512-dimensional speaker and face embeddings, fine-tune MPNet-v2 for emotion-aware text representations, and adapt these features through emotion-specific MLPs trained on unimodal datasets. MAMBA-based trimodal fusion achieves 64.8% accuracy on MELD and 74.3% on IEMOCAP. These results show that combining identity-based audio and visual embeddings with emotion-tuned text representations on a quality-controlled subset of data yields consistent competitive performance for multimodal emotion recognition in conversation and provides a basis for further improvement on challenging, low-frequency emotion classes.
Abstract (translated)
URL
https://arxiv.org/abs/2511.14969
PDF
-
Autobiasing Event Cameras for Flickering Mitigation
2025-11-04 01:49:08Mehdi Sefidgar Dilmaghani, Waseem Shariff, Cian Ryan, Joe Lemley, Peter CorcoranAbstract
Understanding and mitigating flicker effects caused by rapid variations in light intensity is critical for enhancing the performance of event cameras in diverse environments. This paper introduces an innovative autonomous mechanism for tuning the biases of event cameras, effectively addressing flicker across a wide frequency range -25 Hz to 500 Hz. Unlike traditional methods that rely on additional hardware or software for flicker filtering, our approach leverages the event cameras inherent bias settings. Utilizing a simple Convolutional Neural Networks -CNNs, the system identifies instances of flicker in a spatial space and dynamically adjusts specific biases to minimize its impact. The efficacy of this autobiasing system was robustly tested using a face detector framework under both well-lit and low-light conditions, as well as across various frequencies. The results demonstrated significant improvements: enhanced YOLO confidence metrics for face detection, and an increased percentage of frames capturing detected faces. Moreover, the average gradient, which serves as an indicator of flicker presence through edge detection, decreased by 38.2 percent in well-lit conditions and by 53.6 percent in low-light conditions. These findings underscore the potential of our approach to significantly improve the functionality of event cameras in a range of adverse lighting scenarios.
Abstract (translated)
URL
https://arxiv.org/abs/2511.02180
PDF
-
Investigating Adversarial Robustness against Preprocessing used in Blackbox Face Recognition
2025-10-20 05:19:35Roland Croft, Brian Du, Darcy Joseph, Sharath KumarAbstract
Face Recognition (FR) models have been shown to be vulnerable to adversarial examples that subtly alter benign facial images, exposing blind spots in these systems, as well as protecting user privacy. End-to-end FR systems first obtain preprocessed faces from diverse facial imagery prior to computing the similarity of the deep feature embeddings. Whilst face preprocessing is a critical component of FR systems, and hence adversarial attacks against them, we observe that this preprocessing is often overlooked in blackbox settings. Our study seeks to investigate the transferability of several out-of-the-box state-of-the-art adversarial attacks against FR when applied against different preprocessing techniques used in a blackbox setting. We observe that the choice of face detection model can degrade the attack success rate by up to 78%, whereas choice of interpolation method during downsampling has relatively minimal impacts. Furthermore, we find that the requirement for facial preprocessing even degrades attack strength in a whitebox setting, due to the unintended interaction of produced noise vectors against face detection models. Based on these findings, we propose a preprocessing-invariant method using input transformations that improves the transferability of the studied attacks by up to 27%. Our findings highlight the importance of preprocessing in FR systems, and the need for its consideration towards improving the adversarial generalisation of facial adversarial examples.
Abstract (translated)
面部识别（FR）模型已被证明容易受到微小改变的良性面部图像攻击，这些攻击揭示了系统的漏洞，并保护了用户隐私。端到端的面部识别系统在计算深度特征嵌入相似性之前，会从各种面部图像中获取预处理过的脸部。虽然面部预处理是面部识别系统的一个关键组成部分，因此也是对抗攻击的一部分，但我们观察到，在黑盒设置下，这一过程往往被忽视了。我们的研究旨在探讨几种现成的最先进的对抗攻击在应用于不同黑盒设置中的预处理技术时的效果转移性。 我们发现，选择不同的面部检测模型可以降低78%的攻击成功率，而降采样期间使用的插值方法的影响相对较小。此外，我们还发现对预处理的需求甚至会削弱白盒环境下的攻击力度，这是由于生成的噪声向量与面部检测模型之间的意外相互作用所致。 基于这些发现，我们提出了一种使用输入变换的方法来提高所研究攻击的效果转移性，这种方法不受预处理过程的影响，并将效果提高了多达27%。我们的研究表明，在面部识别系统中，预处理的重要性不容忽视，并且为了改善面部对抗样本的泛化能力，需要考虑这一因素。
URL
https://arxiv.org/abs/2510.17169
PDF
-
Extreme Amodal Face Detection
2025-10-08 09:22:03Changlin Song, Yunzhong Hou, Michael Randall Barnes, Rahul Shome, Dylan CampbellAbstract
Extreme amodal detection is the task of inferring the 2D location of objects that are not fully visible in the input image but are visible within an expanded field-of-view. This differs from amodal detection, where the object is partially visible within the input image, but is occluded. In this paper, we consider the sub-problem of face detection, since this class provides motivating applications involving safety and privacy, but do not tailor our method specifically to this class. Existing approaches rely on image sequences so that missing detections may be interpolated from surrounding frames or make use of generative models to sample possible completions. In contrast, we consider the single-image task and propose a more efficient, sample-free approach that makes use of the contextual cues from the image to infer the presence of unseen faces. We design a heatmap-based extreme amodal object detector that addresses the problem of efficiently predicting a lot (the out-of-frame region) from a little (the image) with a selective coarse-to-fine decoder. Our method establishes strong results for this new task, even outperforming less efficient generative approaches.
Abstract (translated)
极端模态检测（Extreme Amodal Detection）的任务是推断出输入图像中未完全可见但可以在扩展视野范围内看到的对象的二维位置。这与一般的模态检测不同，后者是指对象在输入图像中部分可见但被遮挡的情况。本文着重探讨了面部检测这一子问题，因为此类应用涉及安全和隐私领域的重要场景，但我们并未将方法专门针对这个类别进行设计。现有的方法依赖于图序列，以便从周围的帧中插值缺失的检测或利用生成模型来采样可能的完成情况。相比之下，我们考虑单一图像的任务，并提出了一种更高效的无需样本的方法，该方法利用来自图像的情境线索来推断未见人脸的存在。 为了应对这个任务，我们设计了一个基于热图（heatmap）的极端模态对象检测器，它解决了从少量信息高效预测大量数据的问题（即预测超出画面范围的部分），通过选择性地采用粗到细的解码器。我们的方法在这一新任务上取得了显著成果，并且优于其他更不高效的生成式方法。
URL
https://arxiv.org/abs/2510.06791
PDF
-
A Real-Time Multi-Model Parametric Representation of Point Clouds
2025-09-18 09:23:51Yuan Gao, Wei DongAbstract
In recent years, parametric representations of point clouds have been widely applied in tasks such as memory-efficient mapping and multi-robot collaboration. Highly adaptive models, like spline surfaces or quadrics, are computationally expensive in detection or fitting. In contrast, real-time methods, such as Gaussian mixture models or planes, have low degrees of freedom, making high accuracy with few primitives difficult. To tackle this problem, a multi-model parametric representation with real-time surface detection and fitting is proposed. Specifically, the Gaussian mixture model is first employed to segment the point cloud into multiple clusters. Then, flat clusters are selected and merged into planes or curved surfaces. Planes can be easily fitted and delimited by a 2D voxel-based boundary description method. Surfaces with curvature are fitted by B-spline surfaces and the same boundary description method is employed. Through evaluations on multiple public datasets, the proposed surface detection exhibits greater robustness than the state-of-the-art approach, with 3.78 times improvement in efficiency. Meanwhile, this representation achieves a 2-fold gain in accuracy over Gaussian mixture models, operating at 36.4 fps on a low-power onboard computer.
Abstract (translated)
近年来，点云的参数表示在诸如记忆高效地图构建和多机器人协作等任务中得到了广泛应用。高度自适应模型（如样条曲面或二次曲面）在检测或拟合过程中计算成本较高。相比之下，实时方法（例如高斯混合模型或平面模型），虽然自由度较低，但有助于提高效率，不过用少数几个基本元素实现高精度较为困难。为解决这一问题，提出了一种结合了多种模型的参数表示法，实现了实时表面检测和拟合。 具体来说，首先采用高斯混合模型将点云分割成多个簇。然后选择平面簇并将其合并成平面或曲面。平面可以通过基于二维体素边界描述的方法轻松拟合和限定。对于带有曲率的表面，则使用B样条曲面进行拟合，并且应用相同边界的描述方法。 通过在多个公开数据集上的评估，所提出的表面检测方法比现有最佳方法显示出更强的鲁棒性，在效率方面提高了3.78倍。同时，该表示法在准确性上较高斯混合模型提升了两倍，在低功耗车载计算机上的运行速度达到每秒36.4帧。
URL
https://arxiv.org/abs/2509.14773
PDF
-
Bridging Thoughts and Words: Graph-Based Intent-Semantic Joint Learning for Fake News Detection
2025-09-01 17:59:18Zhengjia Wang, Qiang Sheng, Danding Wang, Beizhe Hu, Juan CaoAbstract
Fake news detection is an important and challenging task for defending online information integrity. Existing state-of-the-art approaches typically extract news semantic clues, such as writing patterns that include emotional words, stylistic features, etc. However, detectors tuned solely to such semantic clues can easily fall into surface detection patterns, which can shift rapidly in dynamic environments, leading to limited performance in the evolving news landscape. To address this issue, this paper investigates a novel perspective by incorporating news intent into fake news detection, bridging intents and semantics together. The core insight is that by considering news intents, one can deeply understand the inherent thoughts behind news deception, rather than the surface patterns within words alone. To achieve this goal, we propose Graph-based Intent-Semantic Joint Modeling (InSide) for fake news detection, which models deception clues from both semantic and intent signals via graph-based joint learning. Specifically, InSide reformulates news semantic and intent signals into heterogeneous graph structures, enabling long-range context interaction through entity guidance and capturing both holistic and implementation-level intent via coarse-to-fine intent modeling. To achieve better alignment between semantics and intents, we further develop a dynamic pathway-based graph alignment strategy for effective message passing and aggregation across these signals by establishing a common space. Extensive experiments on four benchmark datasets demonstrate the superiority of the proposed InSide compared to state-of-the-art methods.
Abstract (translated)
虚假新闻检测是一项重要且具有挑战性的任务，旨在保护在线信息的完整性。现有的最先进的方法通常会提取诸如包含情感词汇的写作模式和风格特征等新闻语义线索。然而，仅针对此类语义线索调优的检测器容易陷入表面检测模式，在动态环境中这些模式变化迅速，从而在不断演变的新闻景观中表现有限。为了解决这个问题，本文从一个新颖的角度出发，通过将新闻意图纳入虚假新闻检测来探讨这一问题，旨在连接意图和语义之间的联系。核心见解在于：考虑新闻意图可以深入了解其背后的内在思维，而不仅仅是表面的文字模式。 为了实现这一目标，我们提出了一种基于图的意图-语义联合建模方法（InSide），用于假新闻检测。该模型通过基于图的联合学习方式从语义和意图信号中提取欺骗线索。具体而言，InSide将新闻的语义和意图信号重新表述为异构图结构，这使长距离上下文交互成为可能，并且能够通过粗到细级别的意图建模捕捉全面及实施层面的意图。 为了更好地实现语义与意图之间的对齐，我们进一步开发了一种基于动态路径图对齐策略的方法，以有效进行跨信号的消息传递和聚合，在此过程中建立了一个公共空间。在四个基准数据集上的广泛实验证明了所提出的InSide方法优于现有最先进的方法。
URL
https://arxiv.org/abs/2509.01660
PDF
-
SIExVulTS: Sensitive Information Exposure Vulnerability Detection System using Transformer Models and Static Analysis
2025-08-26 23:23:35Kyler Katz, Sara Moshtari, Ibrahim Mujhid, Mehdi Mirakhorli, Derek GarciaAbstract
Sensitive Information Exposure (SIEx) vulnerabilities (CWE-200) remain a persistent and under-addressed threat across software systems, often leading to serious security breaches. Existing detection tools rarely target the diverse subcategories of CWE-200 or provide context-aware analysis of code-level data flows. Aims: This paper aims to present SIExVulTS, a novel vulnerability detection system that integrates transformer-based models with static analysis to identify and verify sensitive information exposure in Java applications. Method: SIExVulTS employs a three-stage architecture: (1) an Attack Surface Detection Engine that uses sentence embeddings to identify sensitive variables, strings, comments, and sinks; (2) an Exposure Analysis Engine that instantiates CodeQL queries aligned with the CWE-200 hierarchy; and (3) a Flow Verification Engine that leverages GraphCodeBERT to semantically validate source-to-sink flows. We evaluate SIExVulTS using three curated datasets, including real-world CVEs, a benchmark set of synthetic CWE-200 examples, and labeled flows from 31 open-source projects. Results: The Attack Surface Detection Engine achieved an average F1 score greater than 93\%, the Exposure Analysis Engine achieved an F1 score of 85.71\%, and the Flow Verification Engine increased precision from 22.61\% to 87.23\%. Moreover, SIExVulTS successfully uncovered six previously unknown CVEs in major Apache projects. Conclusions: The results demonstrate that SIExVulTS is effective and practical for improving software security against sensitive data exposure, addressing limitations of existing tools in detecting and verifying CWE-200 vulnerabilities.
Abstract (translated)
敏感信息泄露（SIEx）漏洞（CWE-200）仍然是软件系统中持续存在的、未充分解决的安全威胁，经常导致严重的安全漏洞。现有的检测工具很少针对CWE-200的各种子类别进行目标定位或提供代码级数据流的上下文感知分析。本论文旨在介绍一种名为SIExVulTS的新颖漏洞检测系统，该系统整合了基于Transformer的模型与静态分析技术，以识别和验证Java应用程序中的敏感信息泄露。 **方法：** SIExVulTS采用三阶段架构： 1. **攻击面检测引擎**：使用句子嵌入来识别敏感变量、字符串、注释及接收点。 2. **暴露分析引擎**：实例化与CWE-200层级相匹配的CodeQL查询。 3. **流验证引擎**：利用GraphCodeBERT进行源到接收点数据流的语义验证。 我们使用三个经过整理的数据集来评估SIExVulTS，包括现实世界的CVE、一组合成的CWE-200示例以及来自31个开源项目标记的数据流。 **结果：** 攻击面检测引擎实现了平均F1分数超过93%，暴露分析引擎达到了85.71%的F1分数，而流验证引擎将精度从22.61%提升到了87.23%。此外，SIExVulTS在主要Apache项目中成功发现了六个此前未知的CVE。 **结论：** 结果表明，SIExVulTS对于改进软件安全以防止敏感数据泄露非常有效且实用，解决了现有工具在检测和验证CWE-200漏洞方面的局限性。
URL
https://arxiv.org/abs/2508.19472
PDF
-
Two-Factor Authentication Smart Entryway Using Modified LBPH Algorithm
2025-08-19 08:28:40Zakiah Ayop, Wan Mohamad Hariz Bin Wan Mohamad Rosdi, Looi Wei Hua, Syarulnaziah Anawar, Nur Fadzilah OthmanAbstract
Face mask detection has become increasingly important recently, particularly during the COVID-19 pandemic. Many face detection models have been developed in smart entryways using IoT. However, there is a lack of IoT development on face mask detection. This paper proposes a two-factor authentication system for smart entryway access control using facial recognition and passcode verification and an automation process to alert the owner and activate the surveillance system when a stranger is detected and controls the system remotely via Telegram on a Raspberry Pi platform. The system employs the Local Binary Patterns Histograms for the full face recognition algorithm and modified LBPH algorithm for occluded face detection. On average, the system achieved an Accuracy of approximately 70%, a Precision of approximately 80%, and a Recall of approximately 83.26% across all tested users. The results indicate that the system is capable of conducting face recognition and mask detection, automating the operation of the remote control to register users, locking or unlocking the door, and notifying the owner. The sample participants highly accept it for future use in the user acceptance test.
Abstract (translated)
最近，面部口罩检测变得越来越重要，特别是在COVID-19大流行期间。许多智能入口处的面部识别模型已经使用物联网技术开发出来，然而，在面部口罩检测方面的物联网开发却相对不足。本文提出了一种用于智能入口门禁控制的双因素认证系统，该系统结合了面部识别和密码验证，并在检测到陌生人时通过自动化过程向所有者发出警报并激活监控系统，同时还可以通过Telegram平台远程控制树莓派上的整个系统运行。 该系统采用局部二值模式直方图（LBPH）算法进行全脸识别，并且使用修改后的局部二值模式算法来检测被遮挡的面部。经过测试，在所有用户中，平均准确率约为70%，精确度约为80%，召回率为约83.26%。 结果显示，该系统能够执行面部识别和口罩检测、远程控制操作自动化（包括注册用户、锁定或解锁门以及通知所有者）。在用户体验接受性测试中，参与者对该系统的未来使用表示高度认可。
URL
https://arxiv.org/abs/2508.13617
PDF
-
Backdoor Attacks on Deep Learning Face Detection
2025-08-01 13:29:26Quentin Le Roux, Yannick Teglia, Teddy Furon, Philippe Loubet-MoundiAbstract
Face Recognition Systems that operate in unconstrained environments capture images under varying conditions,such as inconsistent lighting, or diverse face poses. These challenges require including a Face Detection module that regresses bounding boxes and landmark coordinates for proper Face Alignment. This paper shows the effectiveness of Object Generation Attacks on Face Detection, dubbed Face Generation Attacks, and demonstrates for the first time a Landmark Shift Attack that backdoors the coordinate regression task performed by face detectors. We then offer mitigations against these vulnerabilities.
Abstract (translated)
在非约束环境中运行的面部识别系统会在不同条件下捕捉图像，例如光照不一致或不同的面部姿态。这些挑战需要包括一个面部检测模块来回归边界框和地标坐标以实现正确的面部对齐。本文展示了针对面部检测系统的物体生成攻击（称为面部生成攻击）的有效性，并首次演示了一种地标偏移攻击，这种攻击会篡改面部探测器执行的坐标回归任务。最后，我们提供了针对这些漏洞的缓解措施。
URL
https://arxiv.org/abs/2508.00620
PDF
-
The Impact of Image Resolution on Face Detection: A Comparative Analysis of MTCNN, YOLOv XI and YOLOv XII models
2025-07-31 08:41:33Ahmet Can \"Omerciko\u{g}lu (D\"uzce University, Department of Computer Engineering, D\"uzce, T\"urkiye), Mustafa Mansur Y\"on\"ug\"ul (D\"uzce University, Department of Computer Engineering, D\"uzce, T\"urkiye), Pakize Erdo\u{g}mu\c{s} (D\"uzce University, Department of Computer Engineering, D\"uzce, T\"urkiye)Abstract
Face detection is a crucial component in many AI-driven applications such as surveillance, biometric authentication, and human-computer interaction. However, real-world conditions like low-resolution imagery present significant challenges that degrade detection performance. In this study, we systematically investigate the impact of input resolution on the accuracy and robustness of three prominent deep learning-based face detectors: YOLOv11, YOLOv12, and MTCNN. Using the WIDER FACE dataset, we conduct extensive evaluations across multiple image resolutions (160x160, 320x320, and 640x640) and assess each model's performance using metrics such as precision, recall, mAP50, mAP50-95, and inference time. Results indicate that YOLOv11 outperforms YOLOv12 and MTCNN in terms of detection accuracy, especially at higher resolutions, while YOLOv12 exhibits slightly better recall. MTCNN, although competitive in landmark localization, lags in real-time inference speed. Our findings provide actionable insights for selecting resolution-aware face detection models suitable for varying operational constraints.
Abstract (translated)
面部检测是许多人工智能驱动应用（如监控、生物识别认证和人机交互）的关键组成部分。然而，现实条件下的低分辨率图像对检测性能构成了重大挑战。在这项研究中，我们系统地调查了输入分辨率对三种基于深度学习的面部检测器——YOLOv11、YOLOv12 和 MTCNN 的准确性和鲁棒性的影响。使用 WIDER FACE 数据集，在多个图像分辨率（160x160、320x320 和 640x640）下进行了广泛的评估，并采用包括精度、召回率、mAP50、mAP50-95 及推断时间等指标来评估每个模型的性能。结果表明，YOLOv11 在检测准确性方面优于 YOLOv12 和 MTCNN，特别是在较高分辨率下，而 YOLOv12 的召回率略高。MTCNN 尽管在地标定位上具有竞争力，但在实时推断速度方面落后。我们的发现为选择适合各种操作约束的分辨率感知面部检测模型提供了可操作的见解。
URL
https://arxiv.org/abs/2507.23341
PDF
-
Bi-Level Optimization for Self-Supervised AI-Generated Face Detection
2025-07-30 16:38:29Mian Zou, Nan Zhong, Baosheng Yu, Yibing Zhan, Kede MaAbstract
AI-generated face detectors trained via supervised learning typically rely on synthesized images from specific generators, limiting their generalization to emerging generative techniques. To overcome this limitation, we introduce a self-supervised method based on bi-level optimization. In the inner loop, we pretrain a vision encoder only on photographic face images using a set of linearly weighted pretext tasks: classification of categorical exchangeable image file format (EXIF) tags, ranking of ordinal EXIF tags, and detection of artificial face manipulations. The outer loop then optimizes the relative weights of these pretext tasks to enhance the coarse-grained detection of manipulated faces, serving as a proxy task for identifying AI-generated faces. In doing so, it aligns self-supervised learning more closely with the ultimate goal of AI-generated face detection. Once pretrained, the encoder remains fixed, and AI-generated faces are detected either as anomalies under a Gaussian mixture model fitted to photographic face features or by a lightweight two-layer perceptron serving as a binary classifier. Extensive experiments demonstrate that our detectors significantly outperform existing approaches in both one-class and binary classification settings, exhibiting strong generalization to unseen generators.
Abstract (translated)
AI生成的面部检测器通过监督学习训练时，通常依赖于特定生成器合成的图像，这限制了它们对新兴生成技术的泛化能力。为克服这一局限性，我们提出了一种基于双层优化的自监督方法。在内部循环中，我们仅使用一组线性加权预设任务（如EXIF标签分类、顺序EXIF标签排序以及人工面部操作检测）对摄影人脸图像进行视觉编码器的预训练。外部循环则通过优化这些预设任务的相对权重来增强粗粒度的人脸篡改检测能力，作为识别AI生成面孔的代理任务，从而更紧密地将自监督学习与AI生成面部检测的最终目标相结合。完成预训练后，编码器保持固定状态不变，然后可以通过高斯混合模型适应摄影人脸特征下的异常检测或使用轻量级两层感知机进行二元分类来识别AI生成的人脸。 广泛的实验表明，在单类和二元分类设置下，我们的探测器在未见过的生成器上表现出显著优于现有方法的泛化能力。
URL
https://arxiv.org/abs/2507.22824
PDF
-
Bias Analysis for Synthetic Face Detection: A Case Study of the Impact of Facial Attributes
2025-07-29 13:45:39Asmae Lamsaf, Lucia Cascone, Hugo Proen\c{c}a, Jo\~ao NevesAbstract
Bias analysis for synthetic face detection is bound to become a critical topic in the coming years. Although many detection models have been developed and several datasets have been released to reliably identify synthetic content, one crucial aspect has been largely overlooked: these models and training datasets can be biased, leading to failures in detection for certain demographic groups and raising significant social, legal, and ethical issues. In this work, we introduce an evaluation framework to contribute to the analysis of bias of synthetic face detectors with respect to several facial attributes. This framework exploits synthetic data generation, with evenly distributed attribute labels, for mitigating any skew in the data that could otherwise influence the outcomes of bias analysis. We build on the proposed framework to provide an extensive case study of the bias level of five state-of-the-art detectors in synthetic datasets with 25 controlled facial attributes. While the results confirm that, in general, synthetic face detectors are biased towards the presence/absence of specific facial attributes, our study also sheds light on the origins of the observed bias through the analysis of the correlations with the balancing of facial attributes in the training sets of the detectors, and the analysis of detectors activation maps in image pairs with controlled attribute modifications.
Abstract (translated)
合成面部检测中的偏见分析必将成为未来几年的一个关键议题。尽管已经开发了许多检测模型，并且发布了几个可靠地识别合成内容的数据集，但一个重要的方面被广泛忽视了：这些模型和训练数据集可能会存在偏差，导致某些人口群体的检测失败，并引发重大的社会、法律和伦理问题。在本项工作中，我们介绍了一个评估框架，旨在对五种最先进的合成面部探测器相对于多种面部属性进行偏见水平分析。该框架利用具有均匀分布属性标签的合成数据生成技术来缓解可能影响偏见分析结果的数据偏差。 基于提出的框架，我们在包含25个受控面部属性的合成数据集中进行了详尽案例研究，以评估五种最先进的检测模型的偏见程度。虽然结果显示，总的来说，合成面部探测器对特定面部特征的存在或缺失存在偏向性，但我们的研究还通过分析训练集中的面部属性平衡以及在具有控制属性修改的图像对中探测器激活图的变化，揭示了观察到的偏见的起源。 这项工作强调了需要更加关注和理解合成检测模型中存在的潜在偏差，并为未来的研究提供了重要的见解，以开发更公平、透明且可靠的技术。
URL
https://arxiv.org/abs/2507.19705
PDF
-
Evaluation of facial landmark localization performance in a surgical setting
2025-07-24 09:40:47Ines Frajtag, Marko \v{S}vaco, Filip \v{S}uligojAbstract
The use of robotics, computer vision, and their applications is becoming increasingly widespread in various fields, including medicine. Many face detection algorithms have found applications in neurosurgery, ophthalmology, and plastic surgery. A common challenge in using these algorithms is variable lighting conditions and the flexibility of detection positions to identify and precisely localize patients. The proposed experiment tests the MediaPipe algorithm for detecting facial landmarks in a controlled setting, using a robotic arm that automatically adjusts positions while the surgical light and the phantom remain in a fixed position. The results of this study demonstrate that the improved accuracy of facial landmark detection under surgical lighting significantly enhances the detection performance at larger yaw and pitch angles. The increase in standard deviation/dispersion occurs due to imprecise detection of selected facial landmarks. This analysis allows for a discussion on the potential integration of the MediaPipe algorithm into medical procedures.
Abstract (translated)
机器人技术、计算机视觉及其应用在包括医学在内的各个领域中越来越广泛。许多面部检测算法已经在神经外科、眼科和整形外科等领域找到了应用。使用这些算法的一个常见挑战是变化的照明条件以及为准确识别和定位患者而灵活调整检测位置的需求。本次实验在一个受控环境中测试了MediaPipe算法，用于在手术灯保持固定且模型头部静止的情况下，由机械臂自动调节不同角度来检测面部特征点。 该研究结果表明，在手术照明条件下改进后的面部特征点检测精度显著提高了偏航角（左右旋转）和俯仰角（上下倾斜）较大时的检测性能。标准偏差/离散度的增加是由于对选定面部特征点识别的不精确性所致。这项分析为MediaPipe算法在医疗程序中的潜在整合提供了讨论基础。
URL
https://arxiv.org/abs/2507.18248
PDF
-
Divisive Decisions: Improving Salience-Based Training for Generalization in Binary Classification Tasks
2025-07-22 20:17:08Jacob Piland, Chris Sweet, Adam CzajkaAbstract
Existing saliency-guided training approaches improve model generalization by incorporating a loss term that compares the model's class activation map (CAM) for a sample's true-class ({\it i.e.}, correct-label class) against a human reference saliency map. However, prior work has ignored the false-class CAM(s), that is the model's saliency obtained for incorrect-label class. We hypothesize that in binary tasks the true and false CAMs should diverge on the important classification features identified by humans (and reflected in human saliency maps). We use this hypothesis to motivate three new saliency-guided training methods incorporating both true- and false-class model's CAM into the training strategy and a novel post-hoc tool for identifying important features. We evaluate all introduced methods on several diverse binary close-set and open-set classification tasks, including synthetic face detection, biometric presentation attack detection, and classification of anomalies in chest X-ray scans, and find that the proposed methods improve generalization capabilities of deep learning models over traditional (true-class CAM only) saliency-guided training approaches. We offer source codes and model weights\footnote{GitHub repository link removed to preserve anonymity} to support reproducible research.
Abstract (translated)
现有的基于注意力引导的训练方法通过引入一个损失项来改进模型泛化能力，该损失项将模型对样本真实类别的类别激活图（CAM）与人类参考注意力图进行比较。然而，先前的工作忽略了错误类别（即错误标签类别）的CAM。我们假设在二元任务中，真类别和假类别的CAM应该在人类认为重要的分类特征上有所区分（这些特征体现在人类的注意力图中）。基于这一假设，我们提出了三种新的基于注意力引导训练方法，将模型的真类别和假类别的CAM同时纳入训练策略，并提出了一种新颖的事后工具来识别重要特征。我们在多个不同的二元闭集和开集分类任务上评估了所有引入的方法，包括合成人脸检测、生物计量攻击呈现检测以及胸部X光片异常分类，发现所提出的这些方法相比传统的仅基于真类别CAM的注意力引导训练方法能够提高深度学习模型的泛化能力。为了支持可重复研究，我们提供了源代码和模型权重\footnote{GitHub仓库链接已移除以保证匿名性}。
URL
https://arxiv.org/abs/2507.17000
PDF
-
QRetinex-Net: Quaternion-Valued Retinex Decomposition for Low-Level Computer Vision Applications
2025-07-22 15:17:24Sos Agaian, Vladimir FrantsAbstract
Images taken in low light often show color shift, low contrast, noise, and other artifacts that hurt computer-vision accuracy. Retinex theory addresses this by viewing an image S as the pixel-wise product of reflectance R and illumination I, mirroring the way people perceive stable object colors under changing light. The decomposition is ill-posed, and classic Retinex models have four key flaws: (i) they treat the red, green, and blue channels independently; (ii) they lack a neuroscientific model of color vision; (iii) they cannot perfectly rebuild the input image; and (iv) they do not explain human color constancy. We introduce the first Quaternion Retinex formulation, in which the scene is written as the Hamilton product of quaternion-valued reflectance and illumination. To gauge how well reflectance stays invariant, we propose the Reflectance Consistency Index. Tests on low-light crack inspection, face detection under varied lighting, and infrared-visible fusion show gains of 2-11 percent over leading methods, with better color fidelity, lower noise, and higher reflectance stability.
Abstract (translated)
在低光条件下拍摄的照片通常会出现色偏、对比度低、噪点和其他影响计算机视觉准确性的伪像。Retinex理论通过将图像S视为反射率R和照明I的逐像素乘积来解决这些问题，这种模型反映了人们在不同光照条件下感知稳定物体颜色的方式。然而，这种分解是不适定问题（ill-posed problem），经典的Retinex模型存在四个主要缺陷：(i) 它们独立处理红色、绿色和蓝色通道；(ii) 缺乏神经科学色彩视觉模型；(iii) 无法完美重建输入图像；以及(iv) 不能解释人类的色恒常性。 我们引入了首个四元数Retinex公式，在该公式中，场景被描述为四元数值反射率与照明的汉密尔顿积。为了评估反射率保持不变的程度，我们提出了反射一致性指标（Reflectance Consistency Index）。在低光裂纹检测、不同光照条件下的面部识别以及红外和可见光图像融合方面的测试显示，相较于领先的方法，我们的方法提高了2-11%的性能，在色彩保真度、噪声降低和反射率稳定性方面表现更优。
URL
https://arxiv.org/abs/2507.16683
PDF
-
Seeing Through Deepfakes: A Human-Inspired Framework for Multi-Face Detection
2025-07-20 03:53:52Juan Hu, Shaojing Fan, Terence SimAbstract
Multi-face deepfake videos are becoming increasingly prevalent, often appearing in natural social settings that challenge existing detection methods. Most current approaches excel at single-face detection but struggle in multi-face scenarios, due to a lack of awareness of crucial contextual cues. In this work, we develop a novel approach that leverages human cognition to analyze and defend against multi-face deepfake videos. Through a series of human studies, we systematically examine how people detect deepfake faces in social settings. Our quantitative analysis reveals four key cues humans rely on: scene-motion coherence, inter-face appearance compatibility, interpersonal gaze alignment, and face-body consistency. Guided by these insights, we introduce \textsf{HICOM}, a novel framework designed to detect every fake face in multi-face scenarios. Extensive experiments on benchmark datasets show that \textsf{HICOM} improves average accuracy by 3.3\% in in-dataset detection and 2.8\% under real-world perturbations. Moreover, it outperforms existing methods by 5.8\% on unseen datasets, demonstrating the generalization of human-inspired cues. \textsf{HICOM} further enhances interpretability by incorporating an LLM to provide human-readable explanations, making detection results more transparent and convincing. Our work sheds light on involving human factors to enhance defense against deepfakes.
Abstract (translated)
多面深度伪造视频正变得越来越普遍，通常出现在自然社交环境中，这使得现有的检测方法难以应对。大多数当前的方法在单人脸检测方面表现出色，但在处理多个人脸的场景时却遇到困难，因为它们缺乏对关键上下文线索的理解。在这项工作中，我们开发了一种新的方法，该方法利用人类认知来分析和防御多面深度伪造视频。通过一系列的人类研究，我们系统地检查了人们在社交环境中如何检测深度伪造人脸的关键线索。定量分析揭示了四个关键因素：场景运动一致性、人脸间外观兼容性、人际注视对齐以及面部与身体的一致性。 基于这些见解，我们引入了一个新的框架\textsf{HICOM}，旨在识别多面场景中每一个假脸。在基准数据集上的广泛实验表明，\textsf{HICOM}使内部分类检测的平均准确率提高了3.3%，在现实世界扰动下的准确性提高了2.8%。此外，在未见过的数据集中，它比现有方法高出5.8%，展示了人类启发式线索的一般化能力。通过整合LLM（大型语言模型）来提供可读性解释，\textsf{HICOM}进一步增强了检测结果的透明度和说服力。 我们的工作为利用人类因素增强对深度伪造防御的研究提供了新的思路。
URL
https://arxiv.org/abs/2507.14807
PDF
-
Secure and Efficient UAV-Based Face Detection via Homomorphic Encryption and Edge Computing
2025-07-14 02:07:08Nguyen Van Duc, Bui Duc Manh, Quang-Trung Luu, Dinh Thai Hoang, Van-Linh Nguyen, Diep N. NguyenAbstract
This paper aims to propose a novel machine learning (ML) approach incorporating Homomorphic Encryption (HE) to address privacy limitations in Unmanned Aerial Vehicles (UAV)-based face detection. Due to challenges related to distance, altitude, and face orientation, high-resolution imagery and sophisticated neural networks enable accurate face recognition in dynamic environments. However, privacy concerns arise from the extensive surveillance capabilities of UAVs. To resolve this issue, we propose a novel framework that integrates HE with advanced neural networks to secure facial data throughout the inference phase. This method ensures that facial data remains secure with minimal impact on detection accuracy. Specifically, the proposed system leverages the Cheon-Kim-Kim-Song (CKKS) scheme to perform computations directly on encrypted data, optimizing computational efficiency and security. Furthermore, we develop an effective data encoding method specifically designed to preprocess the raw facial data into CKKS form in a Single-Instruction-Multiple-Data (SIMD) manner. Building on this, we design a secure inference algorithm to compute on ciphertext without needing decryption. This approach not only protects data privacy during the processing of facial data but also enhances the efficiency of UAV-based face detection systems. Experimental results demonstrate that our method effectively balances privacy protection and detection performance, making it a viable solution for UAV-based secure face detection. Significantly, our approach (while maintaining data confidentially with HE encryption) can still achieve an accuracy of less than 1% compared to the benchmark without using encryption.
Abstract (translated)
本文旨在提出一种结合同态加密（HE）的新型机器学习（ML）方法，以解决无人机（UAV）面部检测中的隐私限制问题。由于距离、高度和面部方向的变化带来的挑战，高分辨率图像和复杂的神经网络能够在动态环境中实现准确的人脸识别。然而，UAV的强大监控能力引发了隐私担忧。为了解决这一问题，我们提出了一种新颖的框架，该框架将HE与高级神经网络相结合，在推理阶段保护面部数据的安全性。这种方法确保了面部数据在安全的前提下不会显著影响检测准确性。 具体而言，所提出的系统利用Cheon-Kim-Kim-Song（CKKS）方案直接对加密数据进行计算，从而优化了计算效率和安全性。此外，我们开发了一种有效的数据编码方法，该方法专门用于将原始面部数据预处理为适合CKKS的格式，并采用单指令多数据流（SIMD）方式进行处理。在此基础上，设计了一个安全推理算法，在不解密的情况下对加密后的数据进行计算。这种方法不仅在处理人脸数据时保护了数据隐私，还提高了基于UAV的人脸检测系统的效率。 实验结果表明，我们提出的方法能够有效平衡隐私保护与检测性能之间的关系，使其成为基于UAV的面部识别系统的一种可行解决方案。特别值得注意的是，在保持数据机密性的同时（通过HE加密），我们的方法相比不使用加密的标准基准仍能达到不到1%误差率的准确性。
URL
https://arxiv.org/abs/2507.09860
PDF
-
Survivability of Backdoor Attacks on Unconstrained Face Recognition Systems
2025-07-02 11:21:27Quentin Le Roux, Yannick Teglia, Teddy Furon, Philippe Loubet-Moundi, Eric BourbaoAbstract
The widespread use of deep learning face recognition raises several security concerns. Although prior works point at existing vulnerabilities, DNN backdoor attacks against real-life, unconstrained systems dealing with images captured in the wild remain a blind spot of the literature. This paper conducts the first system-level study of backdoors in deep learning-based face recognition systems. This paper yields four contributions by exploring the feasibility of DNN backdoors on these pipelines in a holistic fashion. We demonstrate for the first time two backdoor attacks on the face detection task: face generation and face landmark shift attacks. We then show that face feature extractors trained with large margin losses also fall victim to backdoor attacks. Combining our models, we then show using 20 possible pipeline configurations and 15 attack cases that a single backdoor enables an attacker to bypass the entire function of a system. Finally, we provide stakeholders with several best practices and countermeasures.
Abstract (translated)
深度学习面部识别的广泛应用引发了几项安全问题。尽管先前的研究指出了现有的漏洞，但针对实际环境中捕获图像的真实无约束系统进行DNN后门攻击仍然是文献中的盲点。本文首次对基于深度学习的面部识别系统的后门进行了系统级研究。本文通过全面探索这些管道中DNN后门的可能性，在四个方面做出了贡献。我们首次展示了在面部检测任务上的两个后门攻击：面部生成和面部关键点偏移攻击。接着，我们还表明使用大边距损失训练的面部特征提取器同样容易受到后门攻击的影响。结合我们的模型，我们通过20种可能的管道配置和15个攻击案例展示了一个单一的后门就足以使攻击者绕过整个系统的功能。最后，本文为利益相关者提供了几种最佳实践和对策。
URL
https://arxiv.org/abs/2507.01607
PDF
-
Speaking images. A novel framework for the automated self-description of artworks
2025-05-28 09:13:41Valentine Bernasconi, Gustavo MarfiaAbstract
Recent breakthroughs in generative AI have opened the door to new research perspectives in the domain of art and cultural heritage, where a large number of artifacts have been digitized. There is a need for innovation to ease the access and highlight the content of digital collections. Such innovations develop into creative explorations of the digital image in relation to its malleability and contemporary interpretation, in confrontation to the original historical object. Based on the concept of the autonomous image, we propose a new framework towards the production of self-explaining cultural artifacts using open-source large-language, face detection, text-to-speech and audio-to-animation models. The goal is to start from a digitized artwork and to automatically assemble a short video of the latter where the main character animates to explain its content. The whole process questions cultural biases encapsulated in large-language models, the potential of digital images and deepfakes of artworks for educational purposes, along with concerns of the field of art history regarding such creative diversions.
Abstract (translated)
近期在生成式人工智能领域的突破为艺术和文化遗产领域的新研究视角打开了大门，尤其是在大量文物已经数字化的情况下。为了简化数字收藏的访问并突出其内容，需要创新性的解决方案。这些创新激发了对可塑性和当代诠释下数字图像关系的创造性探索，并与原始历史对象形成了对比。 基于自主图像的概念，我们提出了一种新的框架，用于利用开源大型语言模型、面部检测技术、文字转语音和音频到动画转换模型自动生成具有自我解释性质的文化艺术品。我们的目标是从数字化的艺术作品出发，自动组装一个短片，在该短片中主要角色会动起来并解释其内容。 整个过程质疑了大型语言模型中的文化偏见，探讨了数字图像以及艺术作品的深度伪造技术在教育领域的潜在应用，并引发了艺术史领域对这类创意延伸的关注和担忧。
URL
https://arxiv.org/abs/2506.05368
PDF
