Performance Visualizations for the bbob-noisy test suite

Below, we provide postprocessed data showing the performance of all 40+ officially supported algorithm data sets for the bbob-noisy test suite.

Due to the large amount of algorithms (and the limited space in the figures), we currently group algorithm data sets by year of publication.

Performance Comparisons per Year

• 2009: ALPS, AMALGAM, BAYEDA, BFGS, BIPOP-CMA-ES, CMA-ESPLUSSEL, DASA, DE-PSO, EDA-PSO, FULLNEWUOA, GLOBAL, IPOP-SEP-CMA-ES, MA-LS-CHAIN, MCS, ONEFIFTH, PSO, PSO, RANDOMSEARCH, SNOBFIT, VNS, iAMALGAM

• 2010: 1komma2, 1komma2mir, 1komma2mirser, 1komma2ser, 1komma4, 1komma4mir, 1komma4mirser, 1komma4ser, AVGNEWUOA, CMAEGS, IPOP-ACTCMA-ES, IPOP-CMA-ES, MOS, NEWUOA, RCGA, SPSA

• 2012: IPOPsaACM, SNES, xNES, xNESas

• 2016: PSAaLmC-CMA-ES, PSAaLmD-CMA-ES, PSAaSmC-CMA-ES, PSAaSmD-CMA-ES

Example code to produce the figures

The Python code to locally generate the second entry, 2010, above (other entries work respectively) reads

import cocopp               # see https://pypi.org/project/cocopp
cocopp.genericsettings.background = {None: cocopp.archives.bbob_noisy.get_all('2009')}
cocopp.main('bbob-noisy/2010/*')  # will take several minutes to process