File tree 1 file changed +7
-4
lines changed
1 file changed +7
-4
lines changed Original file line number Diff line number Diff line change 6
6
7
7
| Models | Main Problem Resolve Rate | <span style =" color :grey " >Subproblem</span > |
8
8
| --------------------------| -------------------------------------| -------------------------------------|
9
- | 🥇 OpenAI o1-preview | <div align =" center " >** 7.7** </div > | <div align =" center " style =" color :grey " >28.5</div > |
10
- | 🥈 Claude3.5-Sonnet | <div align =" center " >** 4.6** </div > | <div align =" center " style =" color :grey " >26.0</div > |
11
- | 🥉 Claude3.5-Sonnet (new) | <div align =" center " >** 4.6** </div > | <div align =" center " style =" color :grey " >25.3</div > |
9
+ | 🥇 OpenAI o3-mini | <div align =" center " >** 9.2** </div > | <div align =" center " style =" color :grey " >33.0</div > |
10
+ | 🥈 OpenAI o1-preview | <div align =" center " >** 7.7** </div > | <div align =" center " style =" color :grey " >28.5</div > |
11
+ | 🥉 Deepseek-R1 | <div align =" center " >** 4.6** </div > | <div align =" center " style =" color :grey " >28.5</div > |
12
+ | Claude3.5-Sonnet | <div align =" center " >** 4.6** </div > | <div align =" center " style =" color :grey " >26.0</div > |
13
+ | Claude3.5-Sonnet (new) | <div align =" center " >** 4.6** </div > | <div align =" center " style =" color :grey " >25.3</div > |
14
+ | Deepseek-v3 | <div align =" center " >** 3.1** </div > | <div align =" center " style =" color :grey " >23.7</div > |
12
15
| Deepseek-Coder-v2 | <div align =" center " >** 3.1** </div > | <div align =" center " style =" color :grey " >21.2</div > |
13
16
| GPT-4o | <div align =" center " >** 1.5** </div > | <div align =" center " style =" color :grey " >25.0</div > |
14
17
| GPT-4-Turbo | <div align =" center " >** 1.5** </div > | <div align =" center " style =" color :grey " >22.9</div > |
31
34
</center >
32
35
33
36
!!! tip "How to submit"
34
- Want to submit your own model? Head over to the [ documentation ] ( docs/index.md ) .
37
+ Want to submit your own model? Submit a request via a [ Github issue ] ( https://github.com/scicode-bench/SciCode/issues ) .
You can’t perform that action at this time.
0 commit comments