coverage: Allow specifying coverage flags via a `yaml` file #1954

Dhruv-Maradiya · 2024-12-31T15:53:37Z

PR Description

Overview

Fixes Allow specifying coverage flags via a yaml file #532
Implemented functionality to specify coverage flags via a YAML file.
Deprecated flags are ignored and not used from the YAML file.
Functionality only allows specifying common flags.

Concerns

parseArgs Handling:
The parseArgs function is defined as private, preventing it from being imported into test files. As a workaround, I created a minimal version for testing purposes. I am wondering if there is a more efficient way to handle this.
Test Coverage:
I have added test cases to validate the new functionality. I'm unsure if I have organized the tests properly and would appreciate feedback on their coverage and structure.

I’ve reviewed the contributor guide and applied the relevant portions to this PR.

…lags in yaml file

…via YAML file

…g cli_config library

…ling

github-actions · 2025-01-08T22:57:22Z

Package publishing

Package	Version	Status	Publish tag (post-merge)
package:coverage	1.12.0	ready to publish	`coverage-v1.12.0`
package:graphs	2.3.3-wip	WIP (no publish necessary)
package:boolean_selector	2.1.2	already published at pub.dev
package:browser_launcher	1.1.3	already published at pub.dev
package:cli_config	0.2.1-wip	WIP (no publish necessary)
package:cli_util	0.4.2	already published at pub.dev
package:clock	1.1.2	already published at pub.dev
package:code_builder	4.10.1	already published at pub.dev
package:benchmark_harness	2.3.1	already published at pub.dev
package:csslib	1.0.2	already published at pub.dev
package:extension_discovery	2.1.0	already published at pub.dev
package:file	7.0.2-wip	WIP (no publish necessary)
package:file_testing	3.1.0-wip	WIP (no publish necessary)
package:bazel_worker	1.1.3-wip	WIP (no publish necessary)
package:html	0.15.5	already published at pub.dev
package:io	1.0.5	already published at pub.dev
package:json_rpc_2	3.0.3	already published at pub.dev
package:markdown	7.3.1-wip	WIP (no publish necessary)
package:mime	2.0.0	already published at pub.dev
package:oauth2	2.0.4-wip	WIP (no publish necessary)
package:package_config	2.1.1	already published at pub.dev
package:pool	1.5.2-wip	WIP (no publish necessary)
package:pub_semver	2.1.5	already published at pub.dev
package:pubspec_parse	1.5.0	already published at pub.dev
package:source_map_stack_trace	2.1.3-wip	WIP (no publish necessary)
package:yaml_edit	2.2.2	already published at pub.dev
package:source_span	1.10.1	already published at pub.dev
package:sse	4.1.7	already published at pub.dev
package:stack_trace	1.12.1	already published at pub.dev
package:stream_channel	2.1.4	already published at pub.dev
package:stream_transform	2.1.2-wip	WIP (no publish necessary)
package:string_scanner	1.4.1	already published at pub.dev
package:term_glyph	1.2.3-wip	WIP (no publish necessary)
package:test_reflective_loader	0.2.3	already published at pub.dev
package:timing	1.0.2	already published at pub.dev
package:unified_analytics	7.0.1	already published at pub.dev
package:watcher	1.1.1	already published at pub.dev
package:yaml	3.1.3	already published at pub.dev
package:source_maps	0.10.14-wip	WIP (no publish necessary)

Documentation at https://github.com/dart-lang/ecosystem/wiki/Publishing-automation.

github-actions · 2025-01-08T23:04:43Z

PR Health

Breaking changes ✔️

Package	Change	Current Version	New Version	Needed Version	Looking good?
coverage	Non-Breaking	1.11.1	1.12.0-wip	1.12.0	✔️

Changelog Entry ✔️

Package	Changed Files

Changes to files need to be accounted for in their respective changelogs.

Coverage

⚠️

File	Coverage
pkgs/coverage/bin/collect_coverage.dart	💔 Not covered
pkgs/coverage/bin/format_coverage.dart	💔 5 % ⬇️ 5 %
pkgs/coverage/bin/test_with_coverage.dart	💔 Not covered
pkgs/coverage/lib/src/coverage_options.dart	💚 95 %

This check for test coverage is informational (issues shown here will not fail the PR).

This check can be disabled by tagging the PR with skip-coverage-check.

API leaks ✔️

The following packages contain symbols visible in the public API, but not exported by the library. Export these symbols or remove them from your publicly visible API.

Package	Leaked API symbols

License Headers

⚠️

// Copyright (c) 2025, the Dart project authors. Please see the AUTHORS file
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.

Files
pkgs/coverage/lib/src/coverage_options.dart
pkgs/coverage/test/collect_coverage_config_test.dart

All source files should start with a license header.

Unrelated files missing license headers

Files
pkgs/bazel_worker/benchmark/benchmark.dart
pkgs/bazel_worker/example/client.dart
pkgs/bazel_worker/example/worker.dart
pkgs/benchmark_harness/integration_test/perf_benchmark_test.dart
pkgs/boolean_selector/example/example.dart
pkgs/clock/lib/clock.dart
pkgs/clock/lib/src/clock.dart
pkgs/clock/lib/src/default.dart
pkgs/clock/lib/src/stopwatch.dart
pkgs/clock/lib/src/utils.dart
pkgs/clock/test/clock_test.dart
pkgs/clock/test/default_test.dart
pkgs/clock/test/stopwatch_test.dart
pkgs/clock/test/utils.dart
pkgs/html/example/main.dart
pkgs/html/lib/dom.dart
pkgs/html/lib/dom_parsing.dart
pkgs/html/lib/html_escape.dart
pkgs/html/lib/parser.dart
pkgs/html/lib/src/constants.dart
pkgs/html/lib/src/encoding_parser.dart
pkgs/html/lib/src/html_input_stream.dart
pkgs/html/lib/src/list_proxy.dart
pkgs/html/lib/src/query_selector.dart
pkgs/html/lib/src/token.dart
pkgs/html/lib/src/tokenizer.dart
pkgs/html/lib/src/treebuilder.dart
pkgs/html/lib/src/utils.dart
pkgs/html/test/dom_test.dart
pkgs/html/test/parser_feature_test.dart
pkgs/html/test/parser_test.dart
pkgs/html/test/query_selector_test.dart
pkgs/html/test/selectors/level1_baseline_test.dart
pkgs/html/test/selectors/level1_lib.dart
pkgs/html/test/selectors/selectors.dart
pkgs/html/test/support.dart
pkgs/html/test/tokenizer_test.dart
pkgs/pubspec_parse/test/git_uri_test.dart
pkgs/stack_trace/example/example.dart
pkgs/watcher/test/custom_watcher_factory_test.dart
pkgs/yaml_edit/example/example.dart

This check can be disabled by tagging the PR with skip-license-check.

liamappelbe

Looks like there's some test failures. See if you can reproduce them locally.

https://github.com/dart-lang/tools/actions/runs/12686464325/job/35499098457?pr=1954

pkgs/coverage/CHANGELOG.md

pkgs/coverage/lib/src/coverage_options.dart

Dhruv-Maradiya · 2025-01-16T17:20:26Z

Hi @liamappelbe

I've implemented the mentioned changes, and all test cases passed successfully locally. I look forward to your review. Thank you!

pkgs/coverage/CHANGELOG.md

pkgs/coverage/bin/test_with_coverage.dart

pkgs/coverage/lib/src/coverage_options.dart

…d search for options file

coveralls · 2025-01-19T22:04:51Z

Pull Request Test Coverage Report for Build 12868001598

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

40 of 42 (95.24%) changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-6.8%) to 93.166%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
pkgs/coverage/lib/src/coverage_options.dart	40	42	95.24%

Totals
Change from base Build 12728848832:	-6.8%
Covered Lines:	668
Relevant Lines:	717

💛 - Coveralls

liamappelbe

Almost there. Just a few more comments.

pkgs/coverage/bin/collect_coverage.dart

pkgs/coverage/bin/format_coverage.dart

pkgs/coverage/lib/src/coverage_options.dart

pkgs/coverage/test/test_coverage_options/partial_fields2.yaml

… handling

liamappelbe

Implementation looks good. Reviewing the tests now.

There's one more interesting bit of logic that deserves to be better tested: _findOptionsFilePath. Make _findOptionsFilePath public for testing, then write a unit test (in a separate test file) that tests all its edge cases. You'll also need to make some more directory structures and yaml files (maybe in a new directory under the test/ directory to keep things simple):

Typical case: walk up a few directories to find the pubspec.yaml, and find the coverage_options.yaml file next to it
No config case: walk up a few directories to find the pubspec.yaml, but there's no coverage_options.yaml file
Edge case: when there's no pubspec.yaml file at all (this is tricky to test because any walk in the test directory is going to find package:coverage's pubspec, so maybe start in Directory.systemTemp)

The other issue with testing this function is that it starts in Directory.current, which is not safe to alter in a test. So you'll need to change the currentDir variable into an argument, and pass Directory.current when invoking it from _getOptionsFile. Then you can pass whatever you need to in the unit tests.

pkgs/coverage/lib/src/coverage_options.dart

liamappelbe · 2025-01-20T22:35:33Z

pkgs/coverage/test/test_util.dart

@@ -89,3 +92,123 @@ extension ListTestExtension on List {
        },
      );
 }
+
+CoverageOptions parseArgsCollectCoverage(


I think it'd be better to just make the _parseArgs functions in each tool public, and import them in your test. If you copy/paste those functions here, you're not really testing the tools at all, and the tool could go out of sync with the tests when more flags are added/changed.

You could use @visibleForTesting to document that they're only public for testing purposes.

liamappelbe · 2025-01-20T23:27:35Z

pkgs/coverage/lib/src/coverage_options.dart

+
+    final isFileEmpty = fileContents?.isEmpty ?? false;
+
+    // Pass null to fileContents if the file is empty


Just a question: Why do you need to pass null to fileContents if the file is empty?

If we pass empty file content to fromConfigFileContents, it throws a FormatException. Should we also handle cases where the file is empty but contains only blank spaces?

liamappelbe · 2025-01-20T23:40:44Z

pkgs/coverage/test/collect_coverage_config_test.dart

+    });
+  });
+
+  test('override yaml values with command line args', () async {


Test coverage is generally pretty good in this file. The only other case I can see that needs to be tested is the way collect_coverage and format_coverage will output to stdout if you pass a literal 'stdout' to the --out command line arg. Add a test for that case.

pkgs/coverage/test/test_util.dart

Dhruv-Maradiya added 3 commits December 31, 2024 20:13

Implemented functionality to allows specifying coverage options and f…

99afba1

…lags in yaml file

Added test cases to cover YAML configuration options.

ff390a9

Bump version to 1.11.2 and add support for specifying coverage flags …

9cd8dce

…via YAML file

github-actions bot added the package:coverage label Dec 31, 2024

Dhruv-Maradiya added 2 commits January 7, 2025 19:17

feat: update dependencies and refactor coverage options handling usin…

949917c

…g cli_config library

test: refactor test cases to align with updated coverage options hand…

eebc68f

…ling

Dhruv-Maradiya force-pushed the coverage-flags-yaml branch from a9044f0 to eebc68f Compare January 7, 2025 13:49

Dhruv-Maradiya added 2 commits January 7, 2025 20:41

test: optimized test cases

bb0eb60

fix: set default VM service port to 8181 in test_with_coverage.dart

781b2d7

refactor: streamline argument parsing in coverage scripts

c3cbd55

Dhruv-Maradiya force-pushed the coverage-flags-yaml branch from 4580e6b to c3cbd55 Compare January 9, 2025 08:46

liamappelbe self-requested a review January 12, 2025 22:40

liamappelbe reviewed Jan 12, 2025

View reviewed changes

pkgs/coverage/CHANGELOG.md Outdated Show resolved Hide resolved

pkgs/coverage/lib/src/coverage_options.dart Show resolved Hide resolved

pkgs/coverage/lib/src/coverage_options.dart Outdated Show resolved Hide resolved

Dhruv-Maradiya added 2 commits January 16, 2025 22:23

removed support for all uncommon flags

4a1d57f

refactor: changed test cases to align with updated coverage options

c610504

Dhruv-Maradiya requested a review from liamappelbe January 16, 2025 17:18

liamappelbe reviewed Jan 17, 2025

View reviewed changes

change: Resolve output and package paths to absolute paths; add upwar…

b1efad0

…d search for options file

Dhruv-Maradiya requested a review from liamappelbe January 17, 2025 16:11

liamappelbe reviewed Jan 19, 2025

View reviewed changes

refactor: use absolute and normalize instead of canonicalize for path…

beac649

… handling

Dhruv-Maradiya requested a review from liamappelbe January 20, 2025 12:28

liamappelbe reviewed Jan 20, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

coverage: Allow specifying coverage flags via a `yaml` file #1954

coverage: Allow specifying coverage flags via a `yaml` file #1954

Dhruv-Maradiya commented Dec 31, 2024 •

edited

Loading

github-actions bot commented Jan 8, 2025 •

edited

Loading

github-actions bot commented Jan 8, 2025 •

edited

Loading

liamappelbe left a comment

Dhruv-Maradiya commented Jan 16, 2025

coveralls commented Jan 19, 2025 •

edited

Loading

liamappelbe left a comment

liamappelbe left a comment

liamappelbe Jan 20, 2025

liamappelbe Jan 20, 2025

Dhruv-Maradiya Jan 21, 2025

liamappelbe Jan 20, 2025


		final isFileEmpty = fileContents?.isEmpty ?? false;

		// Pass null to fileContents if the file is empty

coverage: Allow specifying coverage flags via a yaml file #1954

Are you sure you want to change the base?

coverage: Allow specifying coverage flags via a yaml file #1954

Conversation

Dhruv-Maradiya commented Dec 31, 2024 • edited Loading

PR Description

Overview

Concerns

github-actions bot commented Jan 8, 2025 • edited Loading

Package publishing

github-actions bot commented Jan 8, 2025 • edited Loading

PR Health

liamappelbe left a comment

Choose a reason for hiding this comment

Dhruv-Maradiya commented Jan 16, 2025

coveralls commented Jan 19, 2025 • edited Loading

Pull Request Test Coverage Report for Build 12868001598

Warning: This coverage report may be inaccurate.

Details

💛 - Coveralls

liamappelbe left a comment

Choose a reason for hiding this comment

liamappelbe left a comment

Choose a reason for hiding this comment

liamappelbe Jan 20, 2025

Choose a reason for hiding this comment

liamappelbe Jan 20, 2025

Choose a reason for hiding this comment

Dhruv-Maradiya Jan 21, 2025

Choose a reason for hiding this comment

liamappelbe Jan 20, 2025

Choose a reason for hiding this comment

coverage: Allow specifying coverage flags via a `yaml` file #1954

coverage: Allow specifying coverage flags via a `yaml` file #1954

Dhruv-Maradiya commented Dec 31, 2024 •

edited

Loading

github-actions bot commented Jan 8, 2025 •

edited

Loading

github-actions bot commented Jan 8, 2025 •

edited

Loading

coveralls commented Jan 19, 2025 •

edited

Loading