MIT/Tuebingen Saliency Benchmark

Leaderboard COCO Freeview

Click on a metric name to sort that metric. Models evaluated as probabilistic models are shown with green background. The performance under the metric which a model has been trained on is shown in italic. The code for evaluation models can be found here.

Name	Published	Code	IG	AUC	sAUC	NSS	CC	KLDiv	SIM	Date tested
Saliency Detection by Self-Resemblance (SSR)	Hae Jong Seo, Peyman Milanfar Nonparametric Bottom-Up Saliency Detection by Self-Resemblance [CVPR 2009].			0.7078	0.6184	0.7851	0.3007	1.6363	0.3832	First tested 2023-07-13 Last tested 2023-07-13 saliency maps computed with SMILER
http://cseweb.ucsd.edu/~l6zhang/	Lingyun Zhang, Matthew H. Tong, Tim K. Marks, Honghao Shan, Garrison W. Cottrell. SUN: A Bayesian framework for saliency using natural statistics [JoV 2008]	matlab		0.7270	0.6358	0.8709	0.3129	1.3096	0.3713	First tested 2023-07-15 Last tested 2023-07-15 maps from code via SMILER. Params: rescale=0.5
Quaternion-Based Spectral Saliency (QSS)	B. Schauerte, R. Stiefelhagen. Quaternion-based Spectral Saliency Detection for Eye Fixation Prediction [ECCV 2012]	matlab		0.7369	0.6343	0.9272	0.3504	1.2178	0.3933	First tested 2023-07-13 Last tested 2023-07-13 saliency maps computed with SMILER
Image Signature	Xiaodi Hou, Jonathan Harel, Christof Koch. Image Signature: Highlighting Sparse Salient Regions [PAMI 2011]	matlab		0.7543	0.6271	0.9822	0.3778	1.1881	0.3935	First tested 2023-07-13 Last tested 2023-07-13 saliency maps computed with SMILER
Context-Aware Saliency	Stas Goferman, Lihi Zelnik-Manor, Ayellet Tal. Context-Aware Saliency Detection [CVPR 2010] [PAMI 2012]	matlab		0.7616	0.6179	0.9871	0.3783	1.1575	0.3987	First tested 2023-07-13 Last tested 2023-07-13 saliency maps computed with SMILER
Dynamic Visual Attention (DVA)	Hou, Xiaodi, and Liqing Zhang. Dynamic visual attention: Searching for coding length increments NIPS 2008	matlab		0.7686	0.6448	1.0503	0.3916	1.1569	0.4309	First tested 2023-07-13 Last tested 2023-07-13 saliency maps computed with SMILER
AIM	Neil Bruce, John Tsotsos. Attention based on information maximization [JoV 2007]	matlab		0.7729	0.6631	1.0010	0.3747	1.2724	0.3749	First tested 2023-07-14 Last tested 2023-07-14 maps from code via SMILER. Params: resize=0.5, thebasis='31infomax975'
RARE2012	Nicolas Riche, Matei Mancas, Matthieu Duvinage, Makiese Mibulumukini, Bernard Gosselin, Thierry Dutoit. RARE2012: A multi-scale rarity-based saliency detection with its comparative statistical analysis [Signal Processing: Image Communication, 2013]	matlab		0.7789	0.6482	1.1460	0.4284	1.0822	0.4275	First tested 2023-07-13 Last tested 2023-07-13 saliency maps computed with SMILER
IttiKoch2	Implementation by Jonathan Harel (part of GBVS toolbox)	matlab		0.7810	0.6077	1.0939	0.4248	1.0838	0.4232	First tested 2023-07-13 Last tested 2023-07-13 saliency maps computed with SMILER
Boolean Map based Saliency (BMS)	Jianming Zhang, Stan Sclaroff. Saliency detection: a boolean map approach [ICCV 2013]	matlab, executable		0.7978	0.6773	1.2373	0.4558	1.0304	0.4325	First tested 2023-07-13 Last tested 2023-07-13 saliency maps computed with SMILER
Fast and Efficient Saliency (FES)	Hamed Rezazadegan Tavakoli, Esa Rahtu, Janne Heikkila. Fast and efficient saliency detection using sparse sampling and kernel density estimation [SCIA 2011]	matlab		0.7978	0.5805	1.2224	0.4622	1.9651	0.4734	First tested 2023-07-13 Last tested 2023-07-13 saliency maps computed with SMILER
Graph-Based Visual Saliency (GBVS)	Jonathan Harel, Christof Koch, Pietro Perona. Graph-Based Visual Saliency [NIPS 2006]	matlab		0.8003	0.6021	1.2053	0.4609	1.0158	0.4404	First tested 2023-07-13 Last tested 2023-07-13 saliency maps computed with SMILER
Centerbias	kernel density from training fixations with uniform mixture component, hyperparameters estimated on training dataset with leave-one-image-out crossvalidation		0.0000	0.8008	0.4999	1.1979	0.4768	0.9724	0.4838	First tested 2023-07-04 Last tested 2023-07-04
Ensembles of Deep Networks (eDN)	Eleonora Vig, Michael Dorr, David Cox. Large-Scale Optimization of Hierarchical Features for Saliency Prediction in Natural Images [CVPR 2014]	python		0.8191	0.5953	1.1260	0.4438	1.2378	0.3713	First tested 2023-07-13 Last tested 2023-07-13 saliency maps computed with SMILER
CovSal	Erkut Erdem, Aykut Erdem. Visual saliency estimation by nonlinearly integrating features using region covariances [JoV 2013]	matlab		0.8197	0.5705	1.3061	0.4956	1.5682	0.4938	First tested 2023-07-13 Last tested 2023-07-13 saliency maps computed with SMILER
LDS	Shu Fang, Jia Li, Yonghong Tian, Tiejun Huang, Xiaowu Chen. Learning Discriminative Subspaces on Random Contrasts for Image Saliency Analysis [TNNLS 2016]	matlab		0.8213	0.5931	1.3996	0.5310	0.9695	0.5098	First tested 2023-07-13 Last tested 2023-07-13 saliency maps computed with SMILER
ML-Net	Marcella Cornia, Lorenzo Baraldi, Giuseppe Serra, Rita Cucchiara. A Deep Multi-Level Network for Saliency Prediction [ICPR 2016]	python		0.8335	0.7122	1.6734	0.5895	0.8382	0.5128	First tested 2023-07-13 Last tested 2023-07-13 saliency maps computed with SMILER
Gold Standard (leave-one-subject-out)	Gaussian kernel density estimate using all fixations of an image with uniform mixture component. Crossvalidated over subjects. Leave-one-subject-out performance			0.8339	0.7667	1.3976				First tested 2024-02-07 Last tested 2024-02-07
Saliency Attentive Model (SAM-VGG)	Marcella Cornia, Lorenzo Baraldi, Giuseppe Serra, Rita Cucchiara. Predicting Human Eye Fixations via an LSTM-based Saliency Attentive Model [IEEE TIP 2018]	python (theano)		0.8430	0.6947	1.6829	0.5974	1.0533	0.5453	First tested 2023-07-13 Last tested 2023-07-13 saliency maps computed with SMILER
DeepGaze I	Matthias Kümmerer, Lucas Theis, Matthias Bethge. Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNet [arxiv 2014, ICLR 2015 workshop]		0.2252	0.8436	0.6891	1.5509	0.5674	0.8331	0.5278	First tested 2023-07-21 Last tested 2023-07-21 predictions from authors
EML-NET	Sen Jia & Neil D.B. Bruce EML-NET: An Expandable Multi-Layer NETwork for Saliency Prediction [arXiv 2018]	python (pytorch)		0.8447	0.7069	1.7354	0.6067	1.3571	0.5512	First tested 2024-05-14 Last tested 2024-05-14 maps computed from model source code using a downscaling which was found to be optimal on the training set
Saliency Attentive Model (SAM-ResNet)	Marcella Cornia, Lorenzo Baraldi, Giuseppe Serra, Rita Cucchiara. Predicting Human Eye Fixations via an LSTM-based Saliency Attentive Model [IEEE TIP 2018]	python (theano)		0.8478	0.6985	1.7695	0.6261	1.0553	0.5612	First tested 2023-07-13 Last tested 2023-07-13 saliency maps computed with SMILER
SalGAN	Junting Pan, Cristian Canton, Kevin McGuinness, Noel E. O'Connor, Jordi Torres, Elisa Sayrol and Xavier Giro-i-Nieto. SalGAN: Visual Saliency Prediction with Generative Adversarial Networks [arXiv 2017]	python		0.8500	0.7019	1.6702	0.6227	0.8117	0.5495	First tested 2023-07-13 Last tested 2023-07-13 saliency maps computed with SMILER
SALICON	X. Hun C. Shen, X. Boix, Q. Zhao: SALICON: Reducing the Semantic Gap in Saliency Prediction by Adapting Deep Neural Networks. ICCV 2015			0.8640	0.7188	1.8301	0.6690	0.6626	0.5845	First tested 2023-07-24 Last tested 2023-07-24 maps from SALICON web demo
DeepGaze II	Matthias Kümmerer, Thomas S. A. Wallis, Leon A. Gatys, Matthias Bethge. Understanding Low- and High-Level Contributions to Fixation Prediction [ICCV 2017]	python (tensorflow)	0.6636	0.8699	0.7399	2.0028	0.6909	0.5858	0.6043	First tested 2023-07-25 Last tested 2023-07-25
SalFBNet	G. Ding, N. Imamoglu, A. Caglayan, M. Murakawa, R. Nakamura: SalFBNet: Learning Pseudo-Saliency Distribution via Feedback Convolutional Networks. Image and Vision Computing 2022.	python (pytorch)		0.8722	0.7099	2.0275	0.7088	0.8623	0.6178	First tested 2024-05-14 Last tested 2024-05-14 saliency maps computed with public code
UNISAL	R. Droste, J. Jiao, J.A. Noble: Unified Image and Video Saliency Modeling. ECCV 2020 (arXiv)	python (pytorch)	0.7494	0.8774	0.7585	2.0954	0.7155	0.5515	0.6203	First tested 2024-02-16 Last tested 2024-02-16 predictions from public model implementation. Images have been downscaled by a factor, which was found to optimal on the COCO Freeview training set
DeepGaze IIE	A. Linardos, M. Kümmerer, O. Press, M. Bethge: DeepGaze IIE: Calibrated prediction in and out-of-domain for state-of-the-art saliency modeling [ICCV 2021]	python (pytorch)	0.8596	0.8825	0.7669	2.2558	0.7563	0.4863	0.6447	First tested 2023-07-21 Last tested 2023-07-21 densities from authors
Gold Standard (leave-one-subject-out)	Gaussian kernel density estimate using all fixations of an image with uniform mixture component. Crossvalidated over subjects. Leave-one-subject-out performance		0.8673	0.8829		2.2837				First tested 2024-02-06 Last tested 2024-02-06

Baseline models

Gold Standard: Our gold standard model is a Gaussian Kernel Density Estimate. There are two versions of it. The crossvalidated performance is the leave-one-subject-out performance where for each subject and image the fixations of all other subject on the same image are used to construct a kernel density estimate that is then evaluated on the remaining subject. The kernel size and the mixture weight of a uniform regularization component are fitted by maximizing the cross-validated log-likelihood of the model. In addition to this crossvalidated version of the model, we also report the performance of a KDE model that uses all fixations on each image with the same parameters as the cross-validated model. One can interpret the cross-validated performance as a lower bound on the explainable performance and the joint performance as an upper bound.
Centerbias: The center bias model is again a Gaussian Kernel Density Estimate. However, unlike the gold standard, it uses the fixations of all other images to predict the fixations on any given image. Kernel size and the mixture weight oa a uniform regularization component are again fitted by maximizing the model log-likelihood.