[draft] equivalent mixtral mlperf data pipeline

[ZhiyuLi-goog]

Description

Add mlperf 5.0 moe data pipeline.

We reuse the same mlperf dataset in gpt3-175b submission:

• stay the same in train dataset
• add a text eval dataset pipeline: In gpt3, eval dataset is pretokenized, see this PR, while we need to tokenize text eval dataset on the fly for moe data pipeline given a different tokenizer

Tests

• Matched the total_weights in the eval dataset, i.e. the total number of next token prediction. see notebook, and please remove this notebook before merge the codes.

Checklist

Before submitting this PR, please make sure (put X in square brackets):

• I have performed a self-review of my code.
• I have necessary comments in my code, particularly in hard-to-understand areas.
• I have run end-to-end tests tests and provided workload links above if applicable.
• I have made or will make corresponding changes to the doc if needed.

Others

If I am not be able to merge it, @suexu1025 @RissyRan could you help me address the comments?