Supervised Contrastive Learning of Environment-Invariant Representations
Nov 1, 2026·,,,,,,,,,,,,,·
0 min read
Taro Makino
Ji Won Park
Natasa Tagasovska
Takamasa Kudo
Paula Coelho
Heming Yao
Jan-Christian Huetter
Ana Carolina Leote
Burkhard Hoeckendorf
Stephen Ra
David Richmond
Kyunghyun Cho
Aviv Regev
Romain Lopez
Abstract
Real-world datasets often combine data collected under different experimental conditions. This yields larger datasets, but can also introduce environment-associated correlations that make it difficult to model the phenomena of interest. We address this with Supervised Contrastive Invariance Learning (SCIL), which learns a task-supervised representation while regularizing its environment-related structure, together with an environment-supervised auxiliary representation. SCIL is based purely on Supervised Contrastive Learning and uses a single hyperparameter to control the strength of environment regularization. We study this regularization under distribution shift with Colored MNIST and Camelyon17-WILDS, where increasing α decreases in-distribution validation accuracy and increases held-out test accuracy. We also evaluate SCIL on batch correction for 26 million Optical Pooled Screening images, where zc preserves biological signal while becoming less predictive of inter-well variation.
Type
Publication
Transactions on Machine Learning Research