Mitigating Spurious Correlations from Image Datasets Using Advanced Neural Networks

نوع: Type: Thesis

مقطع: Segment: masters

عنوان: Title: Mitigating Spurious Correlations from Image Datasets Using Advanced Neural Networks

ارائه دهنده: Provider: Ali Nafisi

اساتید راهنما: Supervisors: Prof. Hassan Khotanlou

اساتید مشاور: Advisory Professors:

اساتید ممتحن یا داور: Examining professors or referees: Dr. Muharram Mansoorizadeh, Dr. Reza Mohammadi

زمان و تاریخ ارائه: Time and date of presentation: 2026

مکان ارائه: Place of presentation: 283

چکیده: Abstract: Machine learning models often suffer from performance degradation and reduced generalizability when faced with out-of-distribution data and spurious correlations, as they tend to rely on biased shortcut features rather than learning the core characteristics. Although various solutions in the form of data manipulation, representation learning, and learning strategy have been proposed to address this challenge, a major limitation of many successful methods is their heavy reliance on group annotations for data balancing, a requirement that is costly and sometimes infeasible in real-world applications. To overcome this limitation, the present study proposes a novel method, termed CIA, which improves upon the DFR framework. By leveraging the capabilities of pre-trained image editing models, the proposed method generates counterfactual image data to implicitly neutralize spurious features at the dataset level. The key advantage of this approach is a significant reduction in the need for group annotations; reliance on these labels is eliminated during the balancing and training phases, and they are required solely for hyperparameter tuning. The performance of the proposed method was evaluated on standard datasets, including Waterbirds, CelebA, and UrbanCars. Experimental results demonstrate that, despite a severe reduction in access to group annotations, this method achieves highly competitive performance. It exhibits substantial superiority in improving Worst-Group Accuracy compared to baseline models like ERM (for instance, increasing this accuracy from 21.4% to 79.5% in the UrbanCars dataset and from 46.6% to 85.6% in CelebA). Ultimately, this research demonstrates that employing generative models for semantic data balancing is an efficient and cost-effective strategy to mitigate model bias, effectively enhancing the robustness of computer vision systems against spurious correlations. The implementation for the proposed method is available online (https://github.com/safinal/ms-thesis).

فایل: ّFile: Download فایل