[BUG] Data mapping with Elastic Common Schema fail

[waldo2188]

ECS integration/library project(s) (e.g. Elastic.CommonSchema.Serilog): Elastic.Serilog.Sinks
ECS schema version (e.g. 1.4.0): The last one (v8.11.0)
ECS .NET assembly version (e.g. 1.4.2): v8.11.0
Elasticsearch version (if applicable): v8.14.2
.NET framework / OS: Docker Linux with .net 8
Description of the problem, including expected versus actual behavior:

Hello,

I’m trying to replace the previous Serilog Sink Serilog.Sinks.Elasticsearch with the new one Elastic.Serilog.Sinks.

During our process with add elements in LogContext with LogContext.PushProperty().

When I use LogContext.PushProperty("client.nat.ip", “0.0.0.1”); the value is well set in the Elasticsearch CommonSchema in the client object, but this one no: LogContext.PushProperty("client.ip", “0.0.0.1”);. client.ip is added in the metadata object.

Another one, if I set LogContext.PushProperty("client.user.id", "regis"); no log are sent to elasticsearch. So, it has to fail somewhere.

Steps to reproduce:

1. Add LogContext.PushProperty("client.ip", “0.0.0.1”);` in a controller or a service
2. Add LogContext.PushProperty("client.user.id", "regis"); in a controller or a service

Is there a way to me to write tests in Elastic.CommonSchema.Serilog.Tests project ?

[waldo2188]

I found why "client.user.id" key wasn't processed (but not why it silently crash...)
I start by adding in class Elastic.CommonSchema.LogTemplateProperties :

// [...]
public static string ClientUserId = nameof(ClientUserId);
// [...]
public static readonly HashSet<string> All = new()
{
// [...]
	"client.user.id", ClientUserId,
// [...]
}

And next in class Elastic.CommonSchema. EcsDocument, I add case "client.user.id" in the right part of the switch case and manage it in the method called TrySetClient, like bellow :

public static bool TrySetClient(EcsDocument document, string path, object value)
{
	Func<Client, object, bool> assign = path switch
	{
// [...]
		"client.user.id" => static (e, v) => TrySetString(e, v, static (ee, p) => (ee.User ?? (ee.User = new User())).Id = p),
		_ => null
	};
	if (assign == null) return false;

	var entity = document.Client ?? new Client();
	var assigned = assign(entity, value);
	if (assigned) document.Client = entity;
	return assigned;
}

Is that the right way ?
By the way I add a test method in namespace Elastic.CommonSchema.Serilog.Tests.Repro

public class GithubIssue402 : LogTestsBase
{
	public GithubIssue402(ITestOutputHelper output) : base(output) { }

	[Fact]
	public void Reproduce() => TestLogger((logger, getLogEvents) =>
	{
		using (LogContext.PushProperty("client.nat.ip", "10.1.2.3"))
		using (LogContext.PushProperty("client.ip", "11.2.3.4"))
		using (LogContext.PushProperty("client.user.id", "regis"))
			logger.Information("Logging something with log context");

		var logEvents = getLogEvents();
		logEvents.Should().HaveCount(1);

		var ecsEvents = ToEcsEvents(logEvents);

		var (_, info) = ecsEvents.First();
		info.Message.Should().Be("Logging something with log context");

		info.Client.NatIp.Should().Be("10.1.2.3");
		info.Client.Ip.Should().Be("11.2.3.4");
		info.Labels.Should().NotContainKey("client.user.id");
		info.Client.User.Id.Should().Be("regis");


	});
}

For the part of the LogContext with "client.ip" key, it’s work in the main git branch...

[waldo2188]

Thanks !