15th Community Wide Experiment on the
Critical Assessment of Techniques for Protein Structure Prediction
Menu
Multimer Predictions Analysis : Group performance based on combined z-scores
Results Home Table Browser
  • Analysis on the models designated as "1"
  • Analysis on the models with the best scores
Targets :
  • TBM (easy)
  • TBM/FM (medium)
  • FM (hard)
  • X-ray
  • NMR
  • CryoEM
Groups:
  • CASP groups on all targets
  • CASP+CAPRI groups on CAPRI targets only

Assessors' formula: Z-score(ICS) + Z-score(IPS) + Z-score(LDDTo) + Z-score(TM)

*ICS: Interface Contact Score (a.k.a. F1 score)
*IPS: Interface Patch Score (a.k.a. Jaccard coefficient)

** The performance graph and table were updated on May 3, 2023 to exlude two targets with unreliable experimental stoichiometry data. Originally, results for 43 targets were assessed. This change had marginal effect on the cumulative Z-scores and did not affect the group ranking reported at the CASP15 meeting.

    #     GR
    name		     GR
    code		      Targets
     Count		     SUM Zscore
    (>0.0)		     Rank SUM
    Zscore (>0.0)		     AVG Zscore
    (>0.0)		     Rank AVG 
    Zscore  (>0.0)		     SUM Zscore
    (>-2.0)		     Rank SUM 
    Zscore  (>-2.0)		     AVG Zscore
    (>-2.0)		     Rank AVG 
    Zscore  (>-2.0)
1 Zheng 374 43 35.4255 1 0.8238 1 32.9891 1 0.7672 1
2 Venclovas 494 43 29.9505 2 0.6965 4 25.2607 2 0.5875 4
3 Wallner 037 37 28.4162 3 0.7680 2 8.8626 15 0.5639 6
4 Yang-Multimer 239 41 25.0147 4 0.6101 7 13.2474 7 0.4207 10
5 Yang 439 41 24.4500 5 0.5963 8 13.2530 6 0.4208 9
6 Kiharalab 119 43 22.0481 6 0.5127 12 12.8254 9 0.2983 19
7 MULTICOM_human 003 43 22.0362 7 0.5125 13 18.8965 3 0.4395 8
8 Manifold 248 43 21.2929 8 0.4952 18 11.4332 11 0.2659 22
9 MULTICOM 367 43 20.9716 9 0.4877 19 16.9190 4 0.3935 12
10 PEZYFoldings 278 41 20.9068 10 0.5099 15 6.1361 17 0.2472 25
11 McGuffin 180 43 20.2875 11 0.4718 21 13.0558 8 0.3036 17
12 Manifold-E 035 43 19.8640 12 0.4620 22 10.2428 14 0.2382 26
13 MULTICOM_qa 086 43 19.6209 13 0.4563 23 15.5974 5 0.3627 13
14 MULTICOM_deep 158 43 17.5549 14 0.4083 28 12.6211 10 0.2935 21
15 DFolding-server 288 35 17.4518 15 0.4986 17 -8.3708 31 0.2180 28
16 CoDock 444 43 17.2736 16 0.4017 29 2.2477 24 0.0523 43
17 Takeda-Shitaka_Lab 348 43 16.6656 17 0.3876 32 11.0721 12 0.2575 23
18 Kiharalab_Server 131 42 16.5675 18 0.3945 31 -5.6682 30 -0.0873 51
19 BAKER 185 43 16.4136 19 0.3817 34 4.8994 21 0.1139 39
20 UltraFold 054 43 15.8641 20 0.3689 36 5.5397 19 0.1288 37
21 BeijingAIProtein 399 42 15.8006 21 0.3762 35 3.5931 23 0.1332 35
22 UltraFold_Server 125 43 15.7925 22 0.3673 37 5.4206 20 0.1261 38
23 Elofsson 320 43 15.7143 23 0.3654 39 10.8371 13 0.2520 24
24 MUFold_H 360 43 15.4133 24 0.3584 40 8.0208 16 0.1865 30
25 MultiFOLD 462 42 15.3530 25 0.3655 38 -1.8143 27 0.0044 46
26 colabfold_human 461 42 14.7284 26 0.3507 42 -1.0919 26 0.0216 45
27 Pierce 314 28 14.2946 27 0.5105 14 -25.6419 41 0.1556 31
28 MUFold 298 43 14.0905 28 0.3277 46 5.6243 18 0.1308 36
29 ColabFold 446 41 13.1542 29 0.3208 48 -5.3746 29 -0.0335 47
30 Grudinin 150 39 12.9245 30 0.3314 45 -0.3457 25 0.1963 29
31 NBIS-AF2-multimer 390 43 12.2984 31 0.2860 50 4.4196 22 0.1028 40
32 RaptorX-Multimer 071 42 12.0760 32 0.2875 49 -3.9771 28 -0.0471 48
33 DMP 477 33 11.7082 33 0.3548 41 -19.0556 36 0.0286 44
34 GuijunLab-Human 169 41 10.7417 34 0.2620 57 -10.4431 32 -0.1571 55
35 GuijunLab-Assembly 098 43 10.4977 35 0.2441 59 -12.2899 33 -0.2858 61
36 Yang-Server 229 21 10.4950 36 0.4998 16 -36.8774 49 0.3392 14
37 SHT 147 42 10.2048 37 0.2430 62 -19.9782 38 -0.4281 68
38 DFolding-refine 073 37 9.7678 38 0.2640 56 -26.8375 42 -0.4010 65
39 ShanghaiTech 225 22 9.2790 39 0.4218 25 -38.8589 51 0.1428 33
40 ClusPro 350 38 9.2762 40 0.2441 60 -13.2527 34 -0.0856 50
41 Coqualia 434 21 8.6567 41 0.4122 26 -42.0125 52 0.0946 41
42 FTBiot0119 165 43 8.1694 42 0.1900 67 -21.2768 39 -0.4948 71
43 Shen-CAPRI 493 36 7.8376 43 0.2177 63 -28.6649 45 -0.4074 66
44 trComplex 423 40 7.7061 44 0.1927 66 -22.6600 40 -0.4165 67
45 DELCLAB 447 37 7.1612 45 0.1935 65 -47.2793 55 -0.9535 77
46 Zou 205 38 6.7856 46 0.1786 68 -19.3724 37 -0.2466 60
47 GinobiFold-SER 011 17 6.7751 47 0.3985 30 -53.0828 58 -0.0637 49
48 GuijunLab-DeepDA 188 42 6.5885 48 0.1569 70 -16.0973 35 -0.3357 62
49 TRFold 187 39 6.3898 49 0.1638 69 -28.1857 44 -0.5176 72
50 UNRES 091 39 5.8870 50 0.1509 71 -38.7577 50 -0.7887 76
51 GinobiFold 227 21 5.5483 51 0.2642 55 -48.6555 56 -0.2217 57
52 Manifold-X 304 10 5.4396 52 0.5440 11 -62.6086 62 0.3391 15
53 WL_team 257 21 5.3112 53 0.2529 58 -51.5712 57 -0.3605 63
54 Kozakov-Vajda 291 27 5.2588 54 0.1948 64 -35.8413 47 -0.1423 54
55 Agemo_mix 092 21 5.1190 55 0.2438 61 -46.4132 54 -0.1149 52
56 ChaePred 398 36 4.9363 56 0.1371 73 -34.3635 46 -0.5657 73
57 FoldEver-Hybrid 385 38 4.8792 57 0.1284 74 -27.6328 43 -0.4640 69
58 DFolding 074 10 4.8492 58 0.4849 20 -64.6654 63 0.1335 34
59 ShanghaiTech-TS-SER 133 17 4.7210 59 0.2777 51 -54.3994 59 -0.1411 53
60 FoldEver 245 31 4.5593 60 0.1471 72 -36.0000 48 -0.3871 64
61 Manifold-LC-E 046 9 4.0502 61 0.4500 24 -66.6889 64 0.1457 32
62 bio3d 397 6 3.8369 62 0.6395 6 -72.2028 69 0.2995 18
63 Fernandez-Recio 312 32 3.8177 63 0.1193 78 -44.7973 53 -0.7124 75
64 OpenFold-SingleSeq 433 27 3.2768 64 0.1214 75 -59.3247 60 -1.0120 78
65 OpenFold 441 27 3.2768 64 0.1214 75 -59.3247 60 -1.0120 78
66 AIchemy_LIG3 347 10 2.7365 66 0.2737 52 -67.6011 65 -0.1601 56
67 AIchemy_LIG2 456 10 2.7082 67 0.2708 53 -68.2274 67 -0.2227 58
68 AIchemy_LIG 325 10 2.7082 67 0.2708 53 -68.2274 67 -0.2227 58
69 ddquest 472 3 2.2704 69 0.7568 3 -77.8743 77 0.7086 2
70 KORP-PL 352 4 2.1949 70 0.5487 10 -76.1479 74 0.4630 7
71 Convex-PL-R 460 3 2.0276 71 0.6759 5 -77.9724 78 0.6759 3
72 zax 122 6 2.0003 72 0.3334 44 -72.5982 71 0.2336 27
73 UTMB 201 5 1.9189 73 0.3838 33 -74.5262 73 0.2948 20
74 Convex-PL 338 3 1.7158 74 0.5719 9 -78.3057 79 0.5648 5
75 TB_model_prediction 199 4 1.3073 75 0.3268 47 -77.6242 75 0.0939 42
76 XRC_VU 215 14 1.1453 76 0.0818 80 -72.9593 72 -1.0685 80
77 Graphen_Medical 097 10 0.9378 77 0.0938 79 -72.3170 70 -0.6317 74
78 ESM-single-sequence 067 13 0.8883 78 0.0683 82 -77.6771 76 -1.3598 83
79 Gonglab-THU 052 7 0.5344 79 0.0763 81 -79.6460 80 -1.0923 81
80 Cerebra 315 8 0.5344 79 0.0668 83 -79.6460 80 -1.2058 82
81 noxelis 236 1 0.4100 81 0.4100 27 -83.5900 83 0.4100 11
82 TensorLab 132 3 0.3590 82 0.1197 77 -81.4791 82 -0.4930 70
83 Panlab 234 29 0.3532 83 0.0122 84 -67.8731 66 -1.3749 84
84 GuijunLab-Meta 481 1 0.3431 84 0.3431 43 -83.6894 84 0.3106 16
85 FALCON2 368 20 0.0000 85 0.0000 85 -84.2142 85 -1.9107 85
86 FALCON0 333 20 0.0000 85 0.0000 85 -84.2142 85 -1.9107 86
87 wuqi 370 1 0.0000 85 0.0000 85 -86.0000 87 -2.0000 87
The cummulative z-scores in this table are calculated according to the following procedure (example for the "first" models):
1. Calculate zscores from the raw scores for all "first" models (corresponding values from the main result table);
2. Remove outliers - models with zscores below the tolerance threshold (set to -2.0);
3. Recalculate zscores on the reduced dataset;
4. Assign z-scores below the penalty threshold (either -2.0 or 0.0) to the value of this threshold.
Protein Structure Prediction Center
Sponsored by the US National Institute of General Medical Sciences (NIH/NIGMS)
Please address any questions or queries to:
© 2007-2022, University of California, Davis