Supervised Contrastive Learning of Environment-Invariant Representations

Nov 1, 2026·
Taro Makino
,
Ji Won Park
,
Natasa Tagasovska
,
Takamasa Kudo
,
Paula Coelho
,
Heming Yao
,
Jan-Christian Huetter
,
Ana Carolina Leote
,
Burkhard Hoeckendorf
,
Stephen Ra
,
David Richmond
,
Kyunghyun Cho
,
Aviv Regev
,
Romain Lopez
· 0 min read
Abstract
Real-world datasets often combine data collected under different experimental conditions. This yields larger datasets, but can also introduce environment-associated correlations that make it difficult to model the phenomena of interest. We address this with Supervised Contrastive Invariance Learning (SCIL), which learns a task-supervised representation while regularizing its environment-related structure, together with an environment-supervised auxiliary representation. SCIL is based purely on Supervised Contrastive Learning and uses a single hyperparameter to control the strength of environment regularization. We study this regularization under distribution shift with Colored MNIST and Camelyon17-WILDS, where increasing α decreases in-distribution validation accuracy and increases held-out test accuracy. We also evaluate SCIL on batch correction for 26 million Optical Pooled Screening images, where zc preserves biological signal while becoming less predictive of inter-well variation.
Type
Publication
Transactions on Machine Learning Research