Skip to content

Commit def5e1e

Browse files
authoredJul 15, 2024
Update index.md
1 parent 133b2a7 commit def5e1e

File tree

1 file changed

+7
-2
lines changed

1 file changed

+7
-2
lines changed
 

‎docs/index.md

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -68,7 +68,7 @@ SciCode sources challenging and realistic research-level coding problems across
6868

6969
Among various coding necessities, Scicode mainly focuses on 1. Numerical methods 2.Simulation of systems 3. Scientific calculation. These are the tasks we believe require intense scientific knowledge and reasoning to optimally test LM’s science capability. The below figure is an example of the combination of 1 and 3.
7070

71-
In designing test cases for evaluation, we incorporate domain-specific test cases in addition to numerical cases. These tests are extracted from real scientific workflows: scientists must design domain-specific test cases to verify code accuracy by reproducing results published in papers or matching analytical solutions derived from theoretical models. Each problem goes through 3 rounds of validation (i.e. by in-domain scientists, out-of-domain scientists, GPT4) for quality control.
71+
In designing test cases for evaluation, we incorporate domain-specific test cases in addition to numerical cases. These tests are extracted from real scientific workflows: scientists must design domain-specific test cases to verify code accuracy by reproducing results published in papers or matching analytical solutions derived from theoretical models. Each problem goes through **3** rounds of validation (i.e. by in-domain scientists, out-of-domain scientists, GPT4) for quality control.
7272
![Image Title](figures/SciCode_example_problem.png)
7373
## Benchmark Statistics
7474

@@ -80,7 +80,12 @@ In designing test cases for evaluation, we incorporate domain-specific test case
8080
| **Biology** | [Ecology](#ecology) (6), [Biochemistry](#biochemistry) (1), [Genetics](#genetics) (1) |
8181
| **Material Science** | [Semiconductor Materials](#semiconductor-materials) (7), [Molecular Modeling](#molecular-modeling) (6) |
8282

83-
Nobel prized related problems:
83+
<figure markdown="span">
84+
![Image Title](figures/SciCode_chart.png)
85+
<figcaption>Left: Distribution of Main Problems Right: Distribution of Subproblems </figcaption>
86+
</figure>
87+
88+
8489

8590
### Numerical Linear Algebra
8691
1_Conjugate_Gradient

0 commit comments

Comments
 (0)