IEDB Analysis Resource

MHC I Automated Server Benchmarks


Accumulated overall ranking scores

Ranking scores based on data sets submitted to the IEDB for the last at least 5 references.

Server 2018-05-11 2018-01-26 2017-09-01 2017-03-23 2016-12-09 2016-05-03 2016-02-19 2015-08-07 2015-07-31 2015-07-17 2015-06-26 2015-06-19 2015-05-15 2015-02-06 2015-01-16 2014-10-31 2014-06-20 2014-05-23 2014-03-28 2014-03-21 2013-01-01
NetMHCpan 2.8 70 72 64 61 57 59 59 72 56 57 54 55 51 52 53 45 65 63 59 59 58
NetMHCpan 3.0 65 60 63 62 64 65 79 83 69 64 62 59 56 56 59 55 58 57 60 59 64
NetMHCpan 4.0 66 56 56 56 49 44 59 70 64 68 66 62 59 56 64 62 55 57 62 61 68
SMM 38 43 48 49 49 38 31 51 54 58 52 51 45 43 42 53 61 48 39 39 52
ANN 3.4 73 77 76 77 81 74 67 77 65 67 64 67 63 63 58 58 58 65 47 50 69
ANN 4.0 57 51 59 72 69 68 72 76 57 54 58 58 55 58 55 54 78 73 68 67 66
ARB 13 14 13 25 20 20 25 55 32 34 29 31 28 29 30 18 52 45 21 18 25
SMMPMBEC 56 62 62 71 87 75 73 81 52 46 45 43 37 36 33 51 59 43 39 40 52
IEDB Consensus 44 44 58 66 62 47 41 55 42 44 45 44 36 39 38 48 52 48 57 57 53
NetMHCcons 71 76 74 73 69 66 58 68 66 68 69 71 67 65 64 71 56 61 63 63 71
PickPocket 51 53 45 52 48 43 43 53 34 38 34 35 28 24 32 28 64 66 51 46 26
mhcflurry 1.2.0 56 55 59 74 73 83 90 88 90 92 92 92 93 93 82 55 90 85 85 85 77

Weekly ranking scores

Ranking scores based on data sets submitted to the IEDB within a single week.

Server 2018-05-11 2018-01-26 2017-09-01 2017-03-23 2016-12-09 2016-05-03 2016-02-19 2015-08-07 2015-07-31 2015-07-17 2015-06-26 2015-06-19 2015-05-15 2015-02-06 2015-01-16 2014-10-31 2014-06-20 2014-05-23 2014-03-28 2014-03-21 2013-01-01
NetMHCpan 2.8 45 81 81 70 60 56 9 95 60 77 50 89 49 54 52 45 64 86 82 59 58
NetMHCpan 3.0 77 47 74 65 65 36 64 95 72 77 100 89 54 58 56 55 91 9 82 59 64
NetMHCpan 4.0 80 50 65 70 70 16 27 95 49 55 88 89 64 52 58 62 64 9 100 61 68
SMM 25 19 34 35 85 68 46 0 9 68 - 89 50 41 42 53 64 55 18 39 52
ANN 3.4 61 79 78 59 85 100 73 95 46 55 - 95 62 66 62 58 32 36 36 50 69
ANN 4.0 93 33 41 89 40 55 82 95 100 9 - 89 51 59 57 54 100 73 36 67 66
ARB 2 - 13 26 0 0 0 100 0 0 - 89 26 20 37 18 9 36 82 18 25
SMMPMBEC 30 64 52 49 90 82 64 95 100 32 - 89 39 36 35 51 91 55 18 40 52
IEDB Consensus 61 9 30 61 85 73 91 9 27 36 - 100 32 44 33 48 36 36 0 57 53
NetMHCcons 57 67 82 74 65 88 36 95 60 41 50 89 68 70 63 71 18 82 82 63 71
PickPocket 7 75 40 47 85 24 18 95 34 82 0 70 31 31 24 28 0 77 82 46 26
mhcflurry 1.2.0 68 - 38 89 10 59 100 95 87 86 - 89 93 92 95 55 91 95 91 85 77

Benchmark results get recalculated as new servers are added to the benchmarks. Numbers in light grey represent scores on datasets before the server was incorporated into the benchmarks. Previous versions of the benchmark results can be downloaded below