Skip to content

Commit 6fa8471

Browse files
committed
Update Teuken with more thorough guidance.
1 parent 10e0b04 commit 6fa8471

File tree

1 file changed

+4
-4
lines changed

1 file changed

+4
-4
lines changed

Teuken.yaml

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ org:
3333
datasources_basemodel:
3434
class: partial
3535
link: https://arxiv.org/pdf/2410.08800
36-
notes: Dataset described as deriving from the CommonCrawl, but no filtered dataset provided. Either a filtered dataset or a fully reproducible and persistent data pipeline would be warranted.
36+
notes: Dataset described as deriving from the CommonCrawl, but no filtered dataset provided. Either a filtered dataset or a fully reproducible and persistent data pipeline would be preferred here.
3737

3838
datasources_endmodel:
3939
class: open
@@ -53,13 +53,13 @@ weights_endmodel:
5353
trainingcode:
5454
class: partial
5555
link: https://github.com/OpenGPTX/Megatron-LM/blob/main/examples/7B_EU24_juwels_part_3_fw_after3T.sbatch
56-
notes: SBATCH script with training code available at fork of Megatron-LM. However, no easily visible and easily navigable repository containing the code used to train the model is available.
56+
notes: SBATCH script with training code available at fork of Megatron-LM. However, no easily visible and easily navigable repository containing the code used to train the model is available. Making the repository more easily visible would alleviate this.
5757

5858
# documentation:
5959
code:
6060
class: closed
6161
link:
62-
notes: README of Megatron-LM repo containing training code is unchanged from base repo. More elaborate documentation would be warranted.
62+
notes: "README of containing training code is unchanged from base repo. More elaborate documentation would be warranted. A good example for a good documentation style would be the repository for the OLMo model: https://github.com/allenai/OLMo"
6363

6464
hardware_architecture:
6565
class: open
@@ -84,7 +84,7 @@ modelcard:
8484
datasheet:
8585
class: closed
8686
link:
87-
notes: No datasheet containing a detailed description of data collection and curation is found attached to a persistent version of the model data, as would be preferred here.
87+
notes: No datasheet containing a detailed description of data collection and curation is found attached to a persistent version of the model data, as would be preferred here. A persistent version of the filtered data with attached the information in the data preprint at https://arxiv.org/abs/2410.08800 would be sufficient here.
8888

8989
# access:
9090
package:

0 commit comments

Comments
 (0)