Missing values when replicating train.json file

[tmgli]

Hi,

Thanks for your work.

I'm working on replicating the train.json file, so I followed the scripts on .run folder (prefix'#'1-'#'4). However, the generated train.json file has some differences, including some missing values compared to the original train.json file you released. Could you please clarify why I'm not able to replicate the same exact train.json file used during Valuenet training? Maybe I'm missing a step in this process (see differences below). Thanks for your time.

Differences between original train.json (for Valuenet training) and generated train.json file (created with scripts with prefix'#'1-'#'4 in .run folder):

1. Missing values: diff between original and generated files show some missing values in generated train.json: e.g., for the "question": "List the name, born state and age of the heads of departments ordered by age.", the values include "State" (Line 622 in original train.json), but this value is not found in generated train.json ("values":[]).
2. The order of values found is different for many queries so rule in some of these cases is different too (maybe not so relevant if rule is valid).
3. Generated train.json has a new field in all question entries in the file, "all_values_found", this entry is not found in original train.json.

@brunnurs again thanks for any clarification on this matter.