[TT-1952] try to split logs by level in in-memory tests [WIP]

[Tofel]

DO NOT REVIEW

Trying out some ideas.

[github-actions]

AER Report: CI Core

aer_workflow , commit , Detect Changes , Clean Go Tidy & Generate , Scheduled Run Frequency , GolangCI Lint (.) , Core Tests (go_core_tests) , GolangCI Lint (integration-tests) , Core Tests (go_core_tests_integration) , GolangCI Lint (deployment) , test-scripts , Core Tests (go_core_ccip_deployment_tests) , Core Tests (go_core_fuzz) , Core Tests (go_core_race_tests) , lint , SonarQube Scan

1. Comment formatting issue: Golang Lint (deployment)

Source of Error:

Golang Lint (deployment)	2025-01-22T18:52:04.0218040Z ##[error]deployment/ccip/changeset/cs_active_candidate_test.go:211:9: commentFormatting: put a space between `//` and comment text (gocritic)
Golang Lint (deployment)	2025-01-22T18:52:04.0220548Z 								//changeset.WithDefaultCommitOffChainConfig(tenv.FeedChainSel, tokenConfig.GetTokenInfo(logger.TestLogger(t), state.Chains[dest].LinkToken, state.Chains[dest].Weth9)),
Golang Lint (deployment)	2025-01-22T18:52:04.0221678Z 								^

Why: The linter detected a comment formatting issue where there is no space between // and the comment text.

Suggested fix: Add a space between // and the comment text to adhere to the formatting standards.

2. Test failure due to missing directory: Core Tests (go_core_ccip_deployment_tests)

Source of Error:

Run tests	2025-01-22T18:54:53.6907711Z time=2025-01-22T18:54:53.620Z level=INFO msg="Failed to create debug.log: open logs/Test_PromoteCandidate/MCMS_enabled_debug.log: no such file or directory"
Run tests	2025-01-22T18:54:53.6908680Z FAIL	github.com/smartcontractkit/chainlink/deployment/ccip/changeset	225.007s

Why: The test failed because it attempted to create a debug log file in a directory that does not exist.

Suggested fix: Ensure that the directory logs/Test_PromoteCandidate/ exists before attempting to create the log file.

3. Test failure due to missing directory: Core Tests (go_core_ccip_deployment_tests)

Source of Error:

Run tests	2025-01-22T18:54:53.8895226Z time=2025-01-22T18:51:13.590Z level=INFO msg="Failed to create debug.log: open logs/TestSetConfigMCMSVariants/MCMS_disabled_debug.log: no such file or directory"
Run tests	2025-01-22T18:54:53.8896428Z FAIL	github.com/smartcontractkit/chainlink/deployment/common/changeset	1.588s

Why: The test failed because it attempted to create a debug log file in a directory that does not exist.

Suggested fix: Ensure that the directory logs/TestSetConfigMCMSVariants/ exists before attempting to create the log file.

4. Test failure due to missing directory: Core Tests (go_core_ccip_deployment_tests)

Source of Error:

Run tests	2025-01-22T18:54:53.8921225Z time=2025-01-22T18:51:13.023Z level=INFO msg="Failed to create debug.log: open logs/TestConfigureOCR3/no_mcms_debug.log: no such file or directory"
Run tests	2025-01-22T18:54:53.8922027Z FAIL	github.com/smartcontractkit/chainlink/deployment/keystone/changeset	1.020s

Why: The test failed because it attempted to create a debug log file in a directory that does not exist.

Suggested fix: Ensure that the directory logs/TestConfigureOCR3/ exists before attempting to create the log file.

5. Test failure due to timeout: Core Tests (go_core_tests)

Source of Error:

Run tests	2025-01-22T19:13:30.0258858Z FAIL	github.com/smartcontractkit/chainlink/v2/core/services/relay/evm	1200.329s

Why: The test failed due to a timeout, indicating that it took too long to complete.

Suggested fix: Investigate the test to identify any performance bottlenecks or infinite loops. Optimize the code or increase the timeout duration if necessary.

AER Report: Operator UI CI ran successfully ✅

aer_workflow , commit

[github-actions]

Flakeguard Summary

Ran new or updated tests between develop and a9a605c (tt-1952-split-in-memory-logs).

View Flaky Detector Details | Compare Changes

Found Flaky Tests ❌

Name	Pass Ratio	Panicked?	Timed Out?	Race?	Runs	Successes	Failures	Skips	Package	Package Panicked?	Avg Duration	Code Owners
TestInvalidOCR3Params	0.00%	false	false	false	3	0	3	0	github.com/smartcontractkit/chainlink/deployment/ccip/changeset	false	4.47s	@smartcontractkit/ccip, @smartcontractkit/core, @smartcontractkit/deployment-automation
Test_ActiveCandidate	0.00%	false	false	false	3	0	3	0	github.com/smartcontractkit/chainlink/deployment/ccip/changeset	false	3m51.926666666s	@smartcontractkit/ccip, @smartcontractkit/core, @smartcontractkit/deployment-automation

Artifacts

For detailed logs of the failed tests, please refer to the artifact failed-test-results-with-logs.json.

[github-actions]

Flakeguard Summary

Ran new or updated tests between develop and b10043d (tt-1952-split-in-memory-logs).

View Flaky Detector Details | Compare Changes

Found Flaky Tests ❌

Name	Pass Ratio	Panicked?	Timed Out?	Race?	Runs	Successes	Failures	Skips	Package	Package Panicked?	Avg Duration	Code Owners
TestInvalidOCR3Params	0.00%	false	false	false	3	0	3	0	github.com/smartcontractkit/chainlink/deployment/ccip/changeset	false	4.446666666s	@smartcontractkit/ccip, @smartcontractkit/core, @smartcontractkit/deployment-automation
Test_ActiveCandidate	0.00%	false	false	false	3	0	3	0	github.com/smartcontractkit/chainlink/deployment/ccip/changeset	false	3m50.103333333s	@smartcontractkit/ccip, @smartcontractkit/core, @smartcontractkit/deployment-automation

Artifacts

For detailed logs of the failed tests, please refer to the artifact failed-test-results-with-logs.json.

[github-actions]

Flakeguard Summary

Ran new or updated tests between develop and db92c9e (tt-1952-split-in-memory-logs).

View Flaky Detector Details | Compare Changes

Found Flaky Tests ❌

Name	Pass Ratio	Panicked?	Timed Out?	Race?	Runs	Successes	Failures	Skips	Package	Package Panicked?	Avg Duration	Code Owners
TestInvalidOCR3Params	0.00%	false	false	false	3	0	3	0	github.com/smartcontractkit/chainlink/deployment/ccip/changeset	false	4.583333333s	@smartcontractkit/ccip, @smartcontractkit/core, @smartcontractkit/deployment-automation
Test_ActiveCandidate	0.00%	false	false	false	3	0	3	0	github.com/smartcontractkit/chainlink/deployment/ccip/changeset	false	3m49.37s	@smartcontractkit/ccip, @smartcontractkit/core, @smartcontractkit/deployment-automation

Artifacts

For detailed logs of the failed tests, please refer to the artifact failed-test-results-with-logs.json.

[cl-sonarqube-production]

Quality Gate failed

Failed conditions
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube

Catch issues before they fail your Quality Gate with our IDE extension SonarLint