CSV extension strictly uses local culture, not InvariantCulture

[stefanedwards]

The CSV Extension appears to be strictly using the local culture, when writing CSV files. The CSV writer appears to be using the invariant culture, but alas, the result is very much not invariant.

data-migration-desktop-tool/Extensions/Csv/Cosmos.DataTransfer.CsvExtension/CsvFormatWriter.cs

Line 25 in 574cf77

await using var writer = new CsvWriter(textWriter, new CsvConfiguration(CultureInfo.InvariantCulture)

This produces a issue, where numbers are formatted according to the local settings -- in Denmark with a comma as decimal separator.

Reproducible example:

A simple json with 2 numeric values. When running on my Danish (da-DK) computer, the CSV writer outputs the numbers with Danish formatting, i.e., comma as decimal separator.

input.json

[{"id": "pi", "x": 3.1415 }, {"id": "euler", "x": 2.71828 }]

migrationsettings.json

{
  "Source": "JSON",
  "Sink": "CSV",
  "SourceSettings": {
	"FilePath": "input.json"
  },
  "SinkSettings": {
	"FilePath": "output.csv",
        "Delimiter": ";"
  }
}

Result (with dmt-2.1.6-win-x64):

id;x
pi;3,1415
euler;2,71828

Incidentially, this is the Nordic "standard" for CSV files (looking at you Excel!) and is in R known as csv2.

The output is much worse (or uglier) if the Delimiter is omitted or set to a comma:

id,x
pi,"3,1415"
euler,"2,71828"

Solution:

I will make a pull request with a proposed solution.