MIT/Tuebingen Saliency Benchmark

Leaderboard CAT2000

Click on a metric name to sort that metric. Models evaluated as probabilistic models are shown with green background. The performance under the metric which a model has been trained on is shown in italic. The code for evaluation models can be found here.

Name Published Code IG AUC sAUC NSS CC KLDiv SIM Date tested

http://cseweb.ucsd.edu/~l6zhang/

Lingyun Zhang, Matthew H. Tong, Tim K. Marks, Honghao Shan, Garrison W. Cottrell. SUN: A Bayesian framework for saliency using natural statistics [JoV 2008]

matlab

0.7128 0.5815 0.8127 0.3164 2.2040 0.4391 First tested 2021-04-07
Last tested 2023-07-07

maps from code via SMILER. Params: rescale=0.5

Saliency Detection by Self-Resemblance (SSR)

Hae Jong Seo, Peyman Milanfar Nonparametric Bottom-Up Saliency Detection by Self-Resemblance [CVPR 2009].

0.7209 0.5943 0.8543 0.3370 1.5695 0.4540 First tested 2021-04-01
Last tested 2023-07-07

maps from SMILER

Quaternion-Based Spectral Saliency (QSS)

B. Schauerte, R. Stiefelhagen. Quaternion-based Spectral Saliency Detection for Eye Fixation Prediction [ECCV 2012]

matlab

0.7510 0.6132 0.8368 0.3318 1.0189 0.4564 First tested 2021-04-01
Last tested 2023-07-06

maps from SMILER

AIM

Neil Bruce, John Tsotsos. Attention based on information maximization [JoV 2007]

matlab

0.7547 0.6092 0.9680 0.3819 1.1924 0.4564 First tested 2021-04-07
Last tested 2023-07-05

maps from code via SMILER. Params: resize=0.5, thebasis='31infomax975'

OpenSALICON

Christopher Lee Thomas. OpenSalicon: An Open Source Implementation of the Salicon Saliency Model [arXiv 2016]

python (caffe)

0.7627 0.6341 1.1985 0.4400 1.0007 0.5032 First tested 2021-04-01
Last tested 2023-07-07

maps from SMILER

Dynamic Visual Attention (DVA)

Hou, Xiaodi, and Liqing Zhang. Dynamic visual attention: Searching for coding length increments NIPS 2008

matlab

0.7664 0.6089 1.0686 0.4120 1.1251 0.4889 First tested 2021-03-31
Last tested 2023-07-05

maps from SMILER

RARE2012

Nicolas Riche, Matei Mancas, Matthieu Duvinage, Makiese Mibulumukini, Bernard Gosselin, Thierry Dutoit. RARE2012: A multi-scale rarity-based saliency detection with its comparative statistical analysis [Signal Processing: Image Communication, 2013]

matlab

0.7689 0.6108 1.0960 0.4302 0.9584 0.4880 First tested 2021-04-01
Last tested 2023-07-06

maps from SMILER

Image Signature

Xiaodi Hou, Jonathan Harel, Christof Koch. Image Signature: Highlighting Sparse Salient Regions [PAMI 2011]

matlab

0.7704 0.6082 1.0864 0.4320 0.9095 0.4812 First tested 2021-04-01
Last tested 2023-07-06

maps from SMILER

Context-Aware Saliency

Stas Goferman, Lihi Zelnik-Manor, Ayellet Tal. Context-Aware Saliency Detection [CVPR 2010] [PAMI 2012]

matlab

0.7775 0.5939 1.0901 0.4340 0.9240 0.4889 First tested 2021-03-31
Last tested 2023-07-05

maps from SMILER

Boolean Map based Saliency (BMS)

Jianming Zhang, Stan Sclaroff. Saliency detection: a boolean map approach [ICCV 2013]

matlab, executable

0.7826 0.6267 1.1879 0.4562 1.0294 0.5106 First tested 2021-03-31
Last tested 2023-07-05

maps from SMILER

ML-Net

Marcella Cornia, Lorenzo Baraldi, Giuseppe Serra, Rita Cucchiara. A Deep Multi-Level Network for Saliency Prediction [ICPR 2016]

python

0.7852 0.6318 1.3280 0.4964 0.9369 0.5258 First tested 2021-04-01
Last tested 2023-07-06

maps from SMILER

IttiKoch2

Implementation by Jonathan Harel (part of GBVS toolbox)

matlab

0.7916 0.5954 1.1945 0.4772 0.8342 0.5030 First tested 2021-04-01
Last tested 2023-07-06

maps from SMILER

Graph-Based Visual Saliency (GBVS)

Jonathan Harel, Christof Koch, Pietro Perona. Graph-Based Visual Saliency [NIPS 2006]

matlab

0.7979 0.5863 1.2342 0.4919 0.8262 0.5115 First tested 2021-04-01
Last tested 2023-07-06

maps from SMILER

Deep Visual Attention (DVA)

W. Wang, and J. Shen. Deep Visual Attention Prediction [IEEE TIP 2018]

matlab (caffe)

0.8000 0.6253 1.4287 0.5394 0.8283 0.5398 First tested 2021-03-31
Last tested 2023-07-05

maps from SMILER

SalGAN

Junting Pan, Cristian Canton, Kevin McGuinness, Noel E. O'Connor, Jordi Torres, Elisa Sayrol and Xavier Giro-i-Nieto. SalGAN: Visual Saliency Prediction with Generative Adversarial Networks [arXiv 2017]

python

0.8085 0.6354 1.4624 0.5668 0.9392 0.5441 First tested 2021-04-01
Last tested 2023-07-07

maps from SMILER

Fast and Efficient Saliency (FES)

Hamed Rezazadegan Tavakoli, Esa Rahtu, Janne Heikkila. Fast and efficient saliency detection using sparse sampling and kernel density estimation [SCIA 2011]

matlab

0.8212 0.5450 1.6103 0.6325 2.5243 0.5643 First tested 2021-03-31
Last tested 2023-07-06

maps from SMILER

LDS

Shu Fang, Jia Li, Yonghong Tian, Tiejun Huang, Xiaowu Chen. Learning Discriminative Subspaces on Random Contrasts for Image Saliency Analysis [TNNLS 2016]

matlab

0.8281 0.5669 1.5356 0.6131 0.8719 0.5785 First tested 2021-04-01
Last tested 2023-07-06

maps from SMILER

EML-NET

Sen Jia & Neil D.B. Bruce EML-NET: An Expandable Multi-Layer NETwork for Saliency Prediction [arXiv 2018]

python (pytorch)

0.8310 0.5853 1.5649 0.6209 1.6914 0.5840 First tested 2024-05-14
Last tested 2024-05-14

maps computed from model source code using a downscaling which was found to be optimal on the training set

CovSal

Erkut Erdem, Aykut Erdem. Visual saliency estimation by nonlinearly integrating features using region covariances [JoV 2013]

matlab

0.8402 0.5570 1.7411 0.6814 1.4373 0.6061 First tested 2021-03-31
Last tested 2023-07-05

maps from SMILER

SALICON

X. Hun C. Shen, X. Boix, Q. Zhao: SALICON: Reducing the Semantic Gap in Saliency Prediction by Adapting Deep Neural Networks. ICCV 2015

0.8406 0.6466 1.6754 0.6402 0.8011 0.6158 First tested 2023-07-11
Last tested 2023-07-11

maps from SALICON web demo

Judd Model

Tilke Judd, Krista Ehinger, Fredo Durand, Antonio Torralba. Learning to predict where humans look [ICCV 2009]

matlab

0.8430 0.5654 1.2978 0.5353 0.9321 0.4639 First tested 2023-07-04
Last tested 2023-07-06

maps from original matlab code via pysaliency

Centerbias

leave-one-image out kernel density estimate with uniform mixture component

0.0000 0.8445 0.1166 2.0870 0.7833 0.4875 0.6559 First tested 2021-04-09
Last tested 2023-07-04

Ensembles of Deep Networks (eDN)

Eleonora Vig, Michael Dorr, David Cox. Large-Scale Optimization of Hierarchical Features for Saliency Prediction in Natural Images [CVPR 2014]

python

0.8470 0.5782 1.2092 0.5003 0.9832 0.4508 First tested 2021-04-01
Last tested 2023-07-07

maps from SMILER

DeepGaze I

Matthias Kümmerer, Lucas Theis, Matthias Bethge. Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNet [arxiv 2014, ICLR 2015 workshop]

-0.1546 0.8524 0.6184 1.8430 0.7403 0.5094 0.6391 First tested 2023-07-04
Last tested 2023-07-04

predictions from authors

SalFBNet

G. Ding, N. Imamoglu, A. Caglayan, M. Murakawa, R. Nakamura: SalFBNet: Learning Pseudo-Saliency Distribution via Feedback Convolutional Networks. Image and Vision Computing 2022.

python (pytorch)

0.8549 0.6330 1.8789 0.7027 1.1983 0.6425 First tested 2024-02-29
Last tested 2024-02-29

saliency maps computed with public code

ICF

Matthias Kümmerer, Thomas S. A. Wallis, Leon A. Gatys, Matthias Bethge. Understanding Low- and High-Level Contributions to Fixation Prediction [ICCV 2017]

python (tensorflow)

-0.0229 0.8561 0.6187 1.9588 0.7791 0.4448 0.6697 First tested 2023-07-24
Last tested 2023-07-24

UNISAL

R. Droste, J. Jiao, J.A. Noble: Unified Image and Video Saliency Modeling. ECCV 2020 (arXiv)

python (pytorch)

0.0321 0.8604 0.6684 1.9359 0.7399 0.4703 0.6633 First tested 2024-02-28
Last tested 2024-02-28

predictions from public model implementation. Images have beend downscaled by a factor of 4.6, which was found to optimal on the CAT2000 training set

DeepGaze II

Matthias Kümmerer, Thomas S. A. Wallis, Leon A. Gatys, Matthias Bethge. Understanding Low- and High-Level Contributions to Fixation Prediction [ICCV 2017]

python (tensorflow)

0.0839 0.8640 0.6498 1.9619 0.7950 0.3815 0.6865 First tested 2021-04-09
Last tested 2023-07-04

Ani

(work in progress)

0.8687 0.6046 2.0657 0.7963 0.4818 0.7064 First tested 2022-05-23
Last tested 2023-07-11

maps from authors

DeepGaze IIE

A. Linardos, M. Kümmerer, O. Press, M. Bethge: DeepGaze IIE: Calibrated prediction in and out-of-domain for state-of-the-art saliency modeling [ICCV 2021]

python (pytorch)

0.1893 0.8692 0.6677 2.1122 0.8189 0.3448 0.7060 First tested 2023-07-05
Last tested 2023-07-05

densities from authors

Gold Standard (leave-one-subject-out)

Gaussian kernel density estimate using all fixations of an image with uniform mixture component. Crossvalidated over subjects. Leave-one-subject-out performance

0.4730 0.8840 0.6930 2.4878 First tested 2023-07-05
Last tested 2023-07-05

Gold Standard

Gaussian kernel density estimate using all fixations of an image with uniform mixture component. Crossvalidated over subjects. Joint performance.

0.8026 0.9159 0.7865 2.7429 0.9685 0.0893 0.8657 First tested 2023-07-06
Last tested 2023-07-06

Baseline models