feat: testing-isolation split into prod and dev

[davidcavazos]

Description

Splitting into two different configs, one for prod and one for dev. The prod config contains all the tests that are consistently passing. The dev config is to run everything else that we're still working on.

The prod config excludes packages that are still not supported. The dev config excludes all the packages passing in the prod config, so it will only run "failing" tests, but that way we can keep iterating on them without blocking PRs.

The workflow got copied for the dev config, the only difference is the config file used and that linting doesn't need to happen on the dev workflow.

Checklist

• I have followed guidelines from CONTRIBUTING.MD and Samples Style Guide
• Tests pass: npm test (see Testing)
• Lint pass: npm run lint (see Style)
• These samples need a new API enabled in testing projects to pass (let us know which ones)
• These samples need a new/updated env vars in testing projects set to pass (let us know which ones)
• This pull request is from a branch created directly off of GoogleCloudPlatform/nodejs-docs-samples. Not a fork.
• This sample adds a new sample directory, and I updated the CODEOWNERS file with the codeowners for this sample
• This sample adds a new sample directory, and I created GitHub Actions workflow for this sample
• This sample adds a new Product API, and I updated the Blunderbuss issue/PR auto-assigner with the codeowners for this sample
• Please merge this PR for me once it is approved

[code-review-assist]

Hi @davidcavazos and reviewers,

I'm currently reviewing this pull request and will post my detailed review in a few minutes. In the meantime, here's a quick summary to help everyone get up to speed:

This PR refactors the testing infrastructure to improve the CI/CD process. The main change is splitting the testing configuration into two separate files: nodejs-prod.jsonc and nodejs-dev.jsonc. nodejs-prod.jsonc includes only consistently passing tests, excluding unsupported packages. nodejs-dev.jsonc runs all other tests, allowing for iterative development without blocking PRs. The CI workflows (ci-prod.yaml and ci-dev.yaml) have been updated to use these respective config files. The ci-dev.yaml workflow also removes linting, as it's not needed for the experimental tests.

Specifically, the changes include:

• Creation of a new configuration file: .github/config/nodejs-dev.jsonc, which lists packages to exclude from the development testing workflow.
• Modification of the production configuration file: .github/config/nodejs-prod.jsonc, which now only includes packages that consistently pass tests.
• Modification of the CI workflows: .github/workflows/ci-dev.yaml and .github/workflows/ci-prod.yaml, to use the new configuration files and adjust the testing process accordingly.

Let me know if you have any questions.

---

And now, a little haiku to lighten the mood:

Tests split, now we see,
Dev and prod, a clearer path,
Code flows smoothly on.

[code-review-assist]

This pull request introduces a good approach to separate production and development tests by creating two distinct config files, nodejs-prod.jsonc and nodejs-dev.jsonc, and corresponding workflows. This allows for isolating consistently passing tests from those still under development, which helps streamline the PR process by preventing failing tests from blocking PRs. The exclusion logic, where the development configuration excludes passing production tests, effectively focuses development efforts on the areas needing attention. However, consider adding more details to the documentation, specifically the README and CONTRIBUTING files, to explain the new setup, how to run tests with each configuration, and how to update the configuration files.

[davidcavazos]

Failing tests are all under "experimental" which are expected.

[davidcavazos]

With this, it's safe to disable any workflow that is already covered in prod.

[davidcavazos]

The following 13 packages were previously not tested on GitHub Actions, but now they are and tests passing:

• appengine/hello-world/flexible_nodejs16_and_earlier
• appengine/metadata/flexible_nodejs16_and_earlier
• appengine/storage/flexible_nodejs16_and_earlier
• functions/http/parseXML
• routeoptimization/snippets
• run/hello-broken
• run/helloworld
• run/image-processing
• run/jobs
• run/logging-manual
• run/markdown-preview/renderer
• run/pubsub
• run/websockets

There are currently 122 packages in prod where tests are passing consistently.

There are 52 packages that are currently failing, some of them are parent directories that should actually just be skipped.

[code-review-assist]

Hello @eddie-santos! I'm Gemini and I'm here to help. Here are some things I can help with via the following comment based commands:

• @code-review-assist review: Start a code review at the latest commit.
• @code-review-assist summarize: Summarize the current state of the pull request.
• @code-review-assist help: Show this message again.
• @code-review-assist walkthrough: Provide a walkthrough of the repository with respect to the current pull request.

Otherwise you can tag me in a comment via @code-review-assist and I'll do my best to help!

[code-review-assist]

Gemini encountered an error creating the walkthrough. You can try again by commenting @code-review-assist walkthrough.

[iennae]

@code-review-assist walkthrough

[code-review-assist]

Gemini encountered an error creating the walkthrough. You can try again by commenting @code-review-assist walkthrough.

[umairidris]

@code-review-assist walkthrough

[code-review-assist]

Google Cloud Platform Node.js Samples Walkthrough

This repository contains Node.js code samples for various Google Cloud Platform (GCP) products. The samples are designed to be used with the official GCP documentation and tutorials, providing practical examples of how to interact with different GCP services.

Repository Purpose: The primary goal of this repository is to provide readily usable, well-documented code examples that developers can integrate into their projects. These samples cover a wide range of GCP services, allowing developers to quickly learn and implement GCP functionalities in their Node.js applications.

Repository Structure: The repository is organized into subdirectories, each representing a specific GCP product (e.g., appengine, cloud-sql, functions, compute, etc.). Within each product directory, you'll find further subdirectories containing individual samples, often with separate folders for the sample code itself, unit tests, and integration tests. Many samples also include a README.md file with specific instructions for setup, execution, and deployment.

Code Flow and Execution:

1. Dependencies: Each sample directory usually has a package.json file listing the necessary Node.js packages. These dependencies are installed using npm install.
2. Build (if applicable): Some samples, particularly those using TypeScript, require a build step (npm run build) to compile the code into JavaScript before execution.
3. Execution: Samples are typically run using npm start (or a similar command specified in the package.json file). This often involves setting environment variables to configure the sample with your GCP project credentials and other necessary parameters.
4. Testing: A significant portion of the repository is dedicated to testing. Unit tests verify the correctness of individual functions, while integration tests ensure that different parts of the sample code work together correctly. Tests are typically run using npm test.
5. Deployment: Many samples include instructions for deploying them to various GCP environments, such as App Engine, Cloud Run, or Cloud Functions. Deployment commands are usually provided in the sample's README.md file and often involve using the gcloud command-line tool.

Pull Request Impact:

This pull request, "feat: testing-isolation split into prod and dev," introduces a significant change to the repository's testing infrastructure. It aims to improve the efficiency of the CI/CD pipeline by separating the test suite into two configurations: one for production (nodejs-prod.jsonc) and one for development (nodejs-dev.jsonc).

• Production Configuration: This configuration includes only consistently passing tests and excludes packages that are not yet fully supported.
• Development Configuration: This configuration runs all other tests, including those that are still under development or are known to be flaky. This allows for continuous iteration on these tests without blocking pull requests.

The pull request also duplicates the CI/CD workflow, creating separate workflows for production and development testing. The development workflow omits the linting step, further streamlining the development process. This approach is generally effective in separating stable and unstable tests, but it's crucial to ensure that the production configuration remains comprehensive enough to catch regressions in the main codebase. Regular review and updates to both configurations will be necessary to maintain their effectiveness.

Additional Notes: The repository utilizes various tools and configurations for code style enforcement (ESLint, Prettier), automated testing (Mocha), and CI/CD (GitHub Actions). Understanding these tools and their configurations is essential for contributing to the repository effectively. The CONTRIBUTING.md file provides detailed guidelines for contributing to the project.