IEDB Analysis Resource

MHC I Automated Server Benchmarks


This is a live ranking of MHC I servers based on performance, which continues to be reevaluated over time. The weekly IEDB releases are automatically checked for datasets large enough to add to the benchmarks. The benchmark metrics in the table below will only be updated on releases where such new data is becoming available.

Accumulated overall ranking scores

Ranking scores based on data sets submitted to the IEDB for the last at least 5 references.

Server 2022-11-24 2022-10-28 2022-10-06 2022-07-22 2022-05-20 2022-04-29 2021-12-24 2021-12-10 2021-11-05 2021-10-08 2021-10-01 2021-09-03 2021-08-20 2021-06-11 2021-05-21 2021-04-16 2021-03-05 2021-02-19 2021-01-29 2021-01-15 2021-01-01 2020-12-25 2020-12-18 2020-12-11 2020-11-13 2020-10-02 2020-09-11 2020-07-10 2020-04-24 2020-03-13 2019-11-01 2019-10-04 2019-08-30 2019-06-21 2019-06-07 2019-05-24 2019-03-15 2019-02-01 2018-05-11 2018-01-26 2017-09-01 2017-03-23 2016-12-09 2016-05-03 2016-02-19 2015-08-07 2015-07-31 2015-07-17 2015-06-26 2015-06-19 2015-05-15 2015-02-06 2015-01-16 2014-10-31 2014-06-20 2014-05-23 2014-03-28 2014-03-21
NetMHCpan 3.0 71 72 69 73 82 78 77 66 63 56 55 57 47 60 59 59 57 65 68 69 71 70 68 59 58 55 56 56 66 68 57 59 58 54 64 70 63 64 65 58 62 62 64 66 81 83 70 65 63 59 56 56 60 56 58 53 59 59
ANN 4.0 69 72 73 64 67 64 58 65 53 51 53 47 52 61 57 60 58 62 67 68 65 70 67 64 51 47 41 41 37 53 53 75 78 82 86 91 77 66 60 54 56 71 66 65 68 76 58 55 58 59 56 58 56 55 78 72 68 69
IEDB Consensus 69 70 62 55 53 49 52 59 58 63 59 53 52 40 49 66 74 66 68 54 43 57 62 68 59 53 43 44 36 45 45 65 48 49 48 51 44 45 51 51 59 62 56 36 24 55 46 48 48 47 37 39 37 48 52 48 59 58
PickPocket 67 68 51 43 32 31 34 40 54 54 52 53 36 36 25 26 20 28 36 26 43 53 51 53 55 51 66 67 60 59 53 20 25 28 21 23 35 35 47 50 44 57 52 47 47 53 33 36 32 33 27 24 29 25 64 62 46 42
NetMHCpan 4.1 65 71 76 76 87 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
NetMHCpan 2.8 62 66 41 51 53 72 71 68 62 47 52 57 58 75 73 51 51 55 49 52 51 52 48 55 65 63 65 69 72 56 63 61 60 66 47 42 46 53 66 68 66 67 63 66 67 72 55 56 53 54 50 52 51 43 65 67 61 60
SMMPMBEC 61 61 72 69 47 41 44 47 53 64 62 67 73 66 55 71 79 65 71 56 41 55 73 78 67 64 61 57 50 73 73 38 29 19 32 35 47 52 58 65 65 72 91 77 76 81 54 48 46 44 37 36 35 53 59 46 40 41
ANN 3.4 59 46 39 42 43 47 44 53 50 51 54 53 61 68 65 78 76 80 84 70 68 76 58 61 62 58 59 67 66 78 78 88 78 74 62 54 58 68 75 80 81 78 82 74 65 77 67 69 65 68 64 63 59 59 58 65 46 48
NetMHCpan 4.0 59 66 72 75 84 83 83 69 70 62 60 71 53 58 63 53 49 62 62 69 76 74 76 68 72 70 71 71 74 69 74 70 74 66 69 78 72 65 65 54 55 60 52 47 64 70 62 66 64 60 57 56 65 62 55 54 62 61
NetMHCcons 58 53 44 56 60 73 71 66 57 48 53 56 62 72 67 73 68 72 73 62 66 69 57 60 70 68 70 73 73 67 73 81 84 79 68 59 60 67 73 78 80 78 73 70 61 68 66 68 69 71 67 65 64 71 56 63 64 63
mhcflurry 1.2.0 56 58 49 35 48 55 58 60 55 56 64 90 85 84 84 53 59 52 35 79 29 58 58 39 15 0 0 0 0 - - - 71 71 85 80 80 70 63 61 61 70 69 80 88 88 90 92 92 92 93 93 83 55 90 88 86 85
SMM 54 60 57 59 51 37 40 44 50 64 60 60 61 48 46 54 63 56 63 51 35 47 63 74 66 63 52 47 41 43 43 42 32 26 38 39 37 39 40 45 51 49 49 36 26 51 54 58 51 50 44 43 42 53 61 50 40 40
ARB 51 37 28 24 15 16 13 22 28 34 34 29 23 24 42 73 74 60 62 36 38 67 51 38 51 46 53 53 56 - - - 0 0 38 26 26 19 9 10 10 30 25 25 33 55 31 34 29 30 28 29 30 18 52 52 24 21
DeepSeqPa - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Weekly ranking scores

Ranking scores based on data sets submitted to the IEDB within a single week.

Server 2022-11-24 2022-10-28 2022-10-06 2022-07-22 2022-05-20 2022-04-29 2021-12-24 2021-12-10 2021-11-05 2021-10-08 2021-10-01 2021-09-03 2021-08-20 2021-06-11 2021-05-21 2021-04-16 2021-03-05 2021-02-19 2021-01-29 2021-01-15 2021-01-01 2020-12-25 2020-12-18 2020-12-11 2020-11-13 2020-10-02 2020-09-11 2020-07-10 2020-04-24 2020-03-13 2019-11-01 2019-10-04 2019-08-30 2019-06-21 2019-06-07 2019-05-24 2019-03-15 2019-02-01 2018-05-11 2018-01-26 2017-09-01 2017-03-23 2016-12-09 2016-05-03 2016-02-19 2015-08-07 2015-07-31 2015-07-17 2015-06-26 2015-06-19 2015-05-15 2015-02-06 2015-01-16 2014-10-31 2014-06-20 2014-05-23 2014-03-28 2014-03-21
NetMHCpan 3.0 68 77 17 82 73 85 84 88 63 73 55 66 30 55 73 96 17 54 65 79 91 54 65 92 88 36 59 18 65 - 53 80 75 0 58 67 88 60 77 42 73 65 65 36 - 95 72 77 100 89 54 58 56 56 91 9 82 59
ANN 4.0 52 74 75 55 73 80 61 81 52 9 75 47 40 46 73 83 42 42 71 63 64 66 80 50 88 73 45 64 38 - 30 - 75 - 75 100 67 94 93 42 35 89 40 55 - 95 100 9 - 89 52 59 57 55 100 73 36 69
IEDB Consensus 78 69 88 27 100 46 47 59 46 91 73 53 64 27 - 33 58 54 83 79 46 33 60 33 100 82 36 91 44 - 0 - 90 - 53 28 17 63 61 14 36 61 85 73 - 9 27 36 - 100 36 44 33 48 36 36 0 58
PickPocket 61 68 50 100 62 15 28 29 41 55 55 62 35 23 73 42 8 2 42 0 36 44 5 88 88 18 55 18 64 - 100 10 75 25 0 44 8 32 7 81 27 47 85 24 - 95 34 82 0 70 27 31 24 25 0 77 82 42
NetMHCpan 4.1 53 69 75 55 87 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
NetMHCpan 2.8 21 79 8 82 28 51 62 75 89 27 49 55 35 91 100 58 50 78 31 54 82 56 10 50 88 59 32 46 73 - 70 70 0 100 64 72 63 22 45 75 74 70 60 56 - 95 60 77 50 89 46 54 52 43 64 86 82 60
SMMPMBEC 69 51 92 100 73 47 38 45 39 100 51 57 84 73 73 29 88 42 98 88 18 49 75 17 88 100 86 100 46 - 70 - 75 - 19 0 33 51 30 64 61 49 90 82 - 95 100 32 - 89 40 36 35 53 91 55 18 41
ANN 3.4 88 55 50 18 16 68 44 42 46 36 60 47 52 73 73 100 25 64 98 83 55 78 55 75 96 64 0 73 66 - 70 - 85 - 89 67 63 43 61 75 92 59 85 100 - 95 46 55 - 95 64 66 62 59 32 36 36 48
NetMHCpan 4.0 45 65 50 55 87 78 86 86 82 64 36 82 46 41 100 92 0 62 52 67 100 51 75 100 92 46 68 36 82 - 60 70 85 100 47 67 96 75 80 50 56 70 70 16 - 95 49 55 88 89 60 52 58 62 64 9 100 61
NetMHCcons 59 60 8 82 28 59 72 73 79 18 49 51 48 91 73 92 33 69 83 54 73 68 50 71 88 32 45 64 80 - 60 70 75 100 86 78 79 49 57 72 89 74 65 88 - 95 60 41 50 89 69 70 63 71 18 82 82 63
mhcflurry 1.2.0 52 58 100 0 69 9 47 83 58 46 48 73 86 100 100 50 88 100 14 79 - - - 29 88 - - 0 0 - - - - - - - 71 90 68 - 50 89 10 59 - 95 87 86 - 89 93 92 95 55 91 95 91 85
SMM 57 56 75 9 96 32 43 45 26 82 64 55 78 23 73 29 67 32 73 92 27 52 30 13 88 100 86 82 47 - 10 - 75 - 25 11 25 61 25 19 39 35 85 68 - 0 9 68 - 89 49 41 42 53 64 55 18 40
ARB 85 40 50 - 14 32 11 26 10 0 36 45 21 0 0 0 96 57 100 8 0 - 100 13 88 0 55 36 56 - - - - - - - 0 50 2 - 0 26 0 0 - 100 0 0 - 89 25 20 37 18 9 36 82 21
DeepSeqPa - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Benchmark results get recalculated as new servers are added to the benchmarks. Numbers in light grey represent scores on datasets before the server was incorporated into the benchmarks. Previous versions of the benchmark results can be downloaded below