Mask one single field and select/unselect the other fields

[cleverton-heusner]

I was using the encoder LoggingEventCompositeJsonEncoder for selecting the fields which would be showed up in console logs:

logback.xml:

<springProfile name="prod">
    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
            <providers>
                <version/>
                <logLevel/>
                <loggerName/>
                <pattern>
                    <pattern>
                        {
                            "timestamp": "%d{yyyy-MM-dd'T'HH:mm:ss.SSSZZ}",
                            "appName": "${appName}"
                        }
                    </pattern>
                </pattern>
                <threadName/>
                <message/>
                <logstashMarkers/>
                <arguments/>
                <stackTrace/>
            </providers>
        </encoder>
    </appender>
</springProfile>

output JSON:

{
  "@version":"1",
  "level":"INFO",
  "logger_name":"uol.dna.profile.filter.TokenFilter",
  "timestamp":"2021-09-01T20:31:45.349-0300",
  "appName":"dna-profile",
  "thread_name":"http-nio-8080-exec-3",
  "message":"[email protected]"
}

Last days, I've been at a spike with the purpose of masking emails in these logs. After reading the Logstash Logback Encoder doc, I realized that my only choice was the encoder LogstashEncoder wrapping the decorator MaskingJsonGeneratorDecorator. The masking itself works, but some original fields was lost e new ones was added:

logback.xml:

<springProfile name="prod">
    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder class="net.logstash.logback.encoder.LogstashEncoder">
            <jsonGeneratorDecorator class="net.logstash.logback.mask.MaskingJsonGeneratorDecorator">
                <valueMasker class="uol.dna.profile.logging.masking.SensitiveDataMasker" />
            </jsonGeneratorDecorator>
        </encoder>
    </appender>
</springProfile>

SensitiveDataMasker.java:

import com.fasterxml.jackson.core.JsonStreamContext;
import java.util.regex.Pattern;
import net.logstash.logback.mask.ValueMasker;

public class SensitiveDataMasker implements ValueMasker
{
    private Pattern emailPattern = Pattern.compile(
            "(?:[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:\\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*|\""
                    + "(?:[\\x01-\\x08\\x0b\\x0c\\x0e-\\x1f\\x21\\x23-\\x5b\\x5d-\\x7f]|"
                    + "\\\\[\\x01-\\x09\\x0b\\x0c\\x0e-\\x7f])*\")@(?:(?:[a-z0-9](?:"
                    + "[a-z0-9-]*[a-z0-9])?\\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?|\\[(?:(?:"
                    + "25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.){3}(?:25[0-5]|2[0-4][0-9]"
                    + "|[01]?[0-9][0-9]?|[a-z0-9-]*[a-z0-9]:(?:[\\x01-\\x08\\x0b\\x0c\\"
                    + "x0e-\\x1f\\x21-\\x5a\\x53-\\x7f]|\\\\[\\x01-\\x09\\x0b\\x0c\\x0e-\\x7f])+)\\])");

    @Override
    public Object mask(JsonStreamContext jsonStreamContext, Object o)
    {
        if (o instanceof CharSequence && emailPattern.matcher((CharSequence) o).matches())
        {
            String[] emailParts = o.toString().split("@");
            String userName = emailParts[0].substring(0, 2).concat(emailParts[0].substring(2).replaceAll(".", "*"));
            String[] serverDomain = emailParts[1].split("((?=\\.))");
            String server = serverDomain[0].substring(0, 2).concat(serverDomain[0].substring(2).replaceAll(".", "*"));

            return userName.concat("@").concat(server).concat(serverDomain[1]);
        }

        return o;
    }
}

output JSON with mask:

{
  "@timestamp":"2021-09-01T20:35:29.974-03:00",
  "@version":"1",
  "message":"cl*******@gm***.com",
  "logger_name":"uol.dna.profile.filter.TokenFilter",
  "thread_name":"http-nio-8080-exec-1",
  "level":"INFO",
  "level_value":20000
}

After the masking, I would like to choose what fields would compose the JSON . Since the masking has a certain complexity
(for example: [email protected] -> na**********@gm***.com), the word conversion "%replace(%msg){'\s', ''}" is not flexible enough to me.

Is my objective possible? What paths could I take?

[philsttr]

You can use MaskingJsonGeneratorDecorator with LoggingEventCompositeJsonEncoder...

<springProfile name="prod">
    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
            <providers>
                <version/>
                <logLevel/>
                <loggerName/>
                <pattern>
                    <pattern>
                        {
                            "timestamp": "%d{yyyy-MM-dd'T'HH:mm:ss.SSSZZ}",
                            "appName": "${appName}"
                        }
                    </pattern>
                </pattern>
                <threadName/>
                <message/>
                <logstashMarkers/>
                <arguments/>
                <stackTrace/>
            </providers>
            <jsonGeneratorDecorator class="net.logstash.logback.mask.MaskingJsonGeneratorDecorator">
                <valueMasker class="uol.dna.profile.logging.masking.SensitiveDataMasker" />
            </jsonGeneratorDecorator>
        </encoder>
    </appender>
</springProfile>

[cleverton-heusner]

Thanks a lot, man.
You saved a spike.