Feature request: Change powertools lambda context capture log key order

[WtfJoke]

Use case

We were in the process of enabling the capturing of the lambda context in our logger.

Although we like the additional context, we didnt enable it because the actual logged message is only visible anymore when expanding the log message which makes it almost impossible to skim through log-message and spot the one which looks interesting because they all look the same in the x-ray-tracing-view and I would be forced to scroll sideways.

See this screenshot for reference:

Solution/User Experience

Is it possible for you to change the order of the logging keys exposed from capturing the lambda context and put it to the end by default (or after the message)?

Alternative solutions

Since v2.8.0 its also possible to define log key order on their own, see here: https://docs.powertools.aws.dev/lambda/typescript/latest/core/logger/#reordering-log-keys-position

Acknowledgment

• This feature request meets Powertools for AWS Lambda (TypeScript) Tenets
• Should this be considered in other Powertools for AWS Lambda languages? i.e. Python, Java, and .NET

Future readers

Please react with 👍 and your use case to help us understand customer demand.

[dreamorosi]

Hi @WtfJoke, thanks for opening the issue.

As you pointed out in the "Alternative Solutions" section of the issue, in the last release we added the ability to specify the order of the keys when initializing the Logger.

I think however that we could have done a better job in the documentation for the feature. After re-reading it, if I understand correctly your feature request, I think it's not accurate.

The docs say this:

You can change the order of standard Logger keys or any keys that will be appended later at runtime via the logRecordOrder parameter

Technically speaking, the context keys are included in the or any keys that will be appended later at runtime, but I can see how this could be confusing and make you think they are not.

In practice, you can specify any key that Powertools manages - which includes the standard keys and the context keys - but also any other key that your code adds.

For example, if you want message, timestamp, and level to be first you could do it like this:

import { Logger } from '@aws-lambda-powertools/logger';
import type { Handler } from 'aws-lambda';

const logger = new Logger({
  logRecordOrder: ['message', 'timestamp', 'level'],
});

export const handler: Handler = async (_, context) => {
  logger.addContext(context);
  logger.info('Hello, World!');
  logger.info('Hello, World!');
  logger.info('Hello, World!');
  logger.info('Hello, World!');

  return {
    statusCode: 200,
    body: JSON.stringify('Hello, World!'),
  };
};

Which would result in these logs on CloudWatch:

The types should also help you with choosing the keys in the IDE:

If instead you want full control of how the log is structured and ordered, you can also bring your own custom log formatter and pass it to the Logger when initializing it, below an example of this that I have used in another application:

import type { Handler } from 'aws-lambda';
import { Logger, LogFormatter, LogItem } from '@aws-lambda-powertools/logger';
import type {
  LogAttributes,
  UnformattedAttributes,
} from '@aws-lambda-powertools/logger/types';

/**
 * Custom log formatter to reorganize the log attributes.
 */
class CustomLogFormatter extends LogFormatter {
  public formatAttributes(
    attributes: UnformattedAttributes,
    additionalLogAttributes: LogAttributes
  ): LogItem {
    const {
      feature,
      area,
      appVersion,
      eventId,
      eventType,
      installationId,
      ...otherAttributes
    } = additionalLogAttributes;

    return new LogItem({
      attributes: {
        message: attributes.message,
        timestamp: this.formatTimestamp(attributes.timestamp),
        level: attributes.logLevel,
        logMetadata: {
          feature,
          area,
        },
        service: {
          environment: attributes.environment,
          name: attributes.serviceName,
          version: appVersion,
        },
        correlationIds: {
          awsRequestId: attributes.lambdaContext?.awsRequestId,
          xrayTraceId: attributes.xRayTraceId,
          eventId,
          eventType,
          installationId,
        },
        functionMetadata: {
          memorySize: attributes.lambdaContext?.memoryLimitInMB,
          name: attributes.lambdaContext?.functionName,
          version: attributes.lambdaContext?.functionVersion,
          arn: attributes.lambdaContext?.invokedFunctionArn,
        },
      },
    }).addAttributes(otherAttributes);
  }
}

const logger = new Logger({
  logFormatter: new CustomLogFormatter(),
});

Please let me know if this helps, and also if I understood your feature request correctly.

If yes, please leave the issue open so we can clarify the docs.

[WtfJoke]

I think my description was not clear enough 😅
Thanks for the detailed explanation of the functionality, though :)

The feature request was targeted into the direction, if it would make sense to change the default logRecordOrder of Powertools Logger, so that the same result is achieved without specifying a custom logRecordOrder (which fixes my problem as well).

E.g that we go from this:

to this:

When you add the lambda context to the logger without specifying this custom logRecordOrder the context log keys take so much space in the log message, that you cannot see the interesting bits (such as the log message itself) anymore.

Which brings me to the point, at least for me, that it looks like it makes no sense to add the lambda context to the logger without specifying a custom logRecordOrder. Instead it would be cool if the default configuration (without specififying any order) when logging the lambda context would already add value (by not inserting them to the beginning and instead at the end for example).

I hope the description was a bit clearer this time, I had a bit a hard time to describe it 🙈 . If something is unclear, please feel free to ask and I try my best to make it more detailed :)