Added a CI job for WPCloud testing of wpcomsh. #39258

zinigor · 2024-09-05T17:58:20Z

Adds a new CI flow specifically for starting WPCloud test runs in case wpcomsh is modified.

@a8caustin we would need to make some changes in the WPCloud environment to properly check out and run wpcomsh using the Monorepo, can you help with that please?

Proposed changes:

Adds a conditional WPCloud CI run for wpcomsh.

Other information:

Have you written new tests for your changes, if applicable?
Have you checked the E2E test CI results, and verified that your changes do not break them?
Have you tested your changes on WordPress.com, if applicable (if so, you'll see a generated comment below with a script to run)?

Jetpack product discussion

Does this pull request change what data or activity we track or use?

Testing instructions:

Go to '..'

github-actions · 2024-09-05T17:59:17Z

Thank you for your PR!

When contributing to Jetpack, we have a few suggestions that can help us test and review your patch:

✅ Include a description of your PR changes.
🔴 Add a "[Status]" label (In Progress, Needs Team Review, ...).
🔴 Add a "[Type]" label (Bug, Enhancement, Janitorial, Task).
✅ Add testing instructions.
✅ Specify whether this PR includes any changes to data or privacy.
✅ Add changelog entries to affected projects

This comment will be updated as you work on your PR and make changes. If you think that some of those checks are not needed for your PR, please explain why you think so. Thanks for cooperation 🤖

The e2e test report can be found here. Please note that it can take a few minutes after the e2e tests checks are complete for the report to be available.

Follow this PR Review Process:

Ensure all required checks appearing at the bottom of this PR are passing.
Choose a review path based on your changes:
- A. Team Review: add the "[Status] Needs Team Review" label
  - For most changes, including minor cross-team impacts.
  - Example: Updating a team-specific component or a small change to a shared library.
- B. Crew Review: add the "[Status] Needs Review" label
  - For significant changes to core functionality.
  - Example: Major updates to a shared library or complex features.
- C. Both: Start with Team, then request Crew
  - For complex changes or when you need extra confidence.
  - Example: Refactor affecting multiple systems.
Get at least one approval before merging.

Still unsure? Reach out in #jetpack-developers for guidance!

Wpcomsh plugin:

Next scheduled release: Atomic deploys happen twice daily on weekdays (p9o2xV-2EN-p2).

If you have any questions about the release process, please ask in the #jetpack-releases channel on Slack.

anomiex · 2024-09-05T20:15:02Z

.github/workflows/wpcloud.yml

+          touch ~/.ssh/askpass
+          chmod -v 700 ~/.ssh/askpass
+          cat >>~/.ssh/askpass <<END
+          #!/bin/bash
+          echo "\$WPCLOUD_SSH_KEY_PASSPHRASE"
+          END
+          echo "wrote ~/.ssh/askpass"


I wonder if we could switch to a key with no password, so we could skip this. Is there an attack scenario where someone could steal the key but not also steal this file with the password?

anomiex · 2024-09-05T20:16:34Z

.github/workflows/wpcloud.yml

+          export SSH_ASKPASS="$HOME/.ssh/askpass"
+          echo "exported SSH_ASKPASS"
+          export SSH_ASKPASS_REQUIRE="force"
+          echo "exported SSH_ASKPASS_REQUIRE"
+          export DISPLAY=":"
+          echo "exported DISPLAY"


(this along with the comment on lines 61–67 above)

anomiex · 2024-09-05T20:19:01Z

.github/workflows/wpcloud.yml

+          echo "::endgroup::"
+
+          echo "::group::execution"
+          cat ~/.ssh/stdin | setsid ssh -i ~/.ssh/id_site -l "$WPCLOUD_SSH_USER" ssh.atomicsites.net "$WPCLOUD_SSH_COMMAND" || CODE=$?


I'm left wondering what's in WPCLOUD_SSH_COMMAND that we can't just put it in here.

Is whatever this runs going to work if multiple PRs run this at the same time?

anomiex · 2024-10-24T18:08:06Z

.github/workflows/wpcloud.yml

-          END
-          echo "wrote ~/.ssh/stdin"
+          echo "::group::Installing and building wpcomsh"
+          pnpm jetpack build plugins/wpcomsh


You probably need to include --deps, and adding -v would probably be helpful in case anything fails.

Personally, I'd do the build as a separate step, if not as a separate job to make re-runs not have to re-build (cf. #32686).

anomiex · 2024-10-25T17:04:35Z

.github/workflows/wpcloud.yml

+
+      - name: Restore wpcomsh build cache
+        id: wpcomsh-build-cache
+        if: needs.build.result == 'success'


I think we can leave out this line, it's there in the E2E workflow because the build might be skipped when it's going to run against Atomic or something.

Suggested change

if: needs.build.result == 'success'

anomiex · 2024-10-25T17:07:33Z

.github/workflows/wpcloud.yml

+    name: Run PHPUnit on the WPCloud test site
+    runs-on: ubuntu-latest
+    needs: build
+    if: needs.build.changed.outputs.wpcomsh == 'true'


Oops, this is why it skipped this time.

Suggested change

if: needs.build.changed.outputs.wpcomsh == 'true'

if: needs.build.outputs.wpcomsh == 'true'

.github/workflows/wpcloud.yml

+    name: Install the Monorepo and build wpcomsh
+    runs-on: ubuntu-latest
+    outputs:
+      wpcomsh: ${{ steps.changed.outputs.wpcomsh }}
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          # codecov.io requires a fetch depth > 1.
+          fetch-depth: 2
+
+      # For pull requests, list-changed-projects.sh needs the merge base.
+      # But it doesn't have to be checked out.
+      - name: Deepen to merge base
+        if: github.event_name == 'pull_request'
+        uses: ./.github/actions/deepen-to-merge-base
+        with:
+          checkout: false
+
+      - name: Setup tools
+        uses: ./.github/actions/tool-setup
+        with:
+          php: ${{ matrix.php }}
+          node: ${{ matrix.node }}
+      - name: Monorepo install
+        run: |
+          echo "::group::Pnpm"
+          pnpm install
+          echo "::endgroup::"
+      - name: Detect if wpcomsh has changed
+        id: changed
+        run: |
+          CHANGED="$(EXTRA=test .github/files/list-changed-projects.sh)"
+
+          # WPCOMSH_CHANGED="$(jq --argjson changed "$CHANGED" -n '$changed | has( "plugins/wpcomsh" ) ')"
+          WPCOMSH_CHANGED="true"
+          echo "wpcomsh=${WPCOMSH_CHANGED}" >> "$GITHUB_OUTPUT"
+      - name: Build wpcomsh
+        if: steps.changed.outputs.wpcomsh == 'true'
+        run: |
+          find . -path ./.github -prune -o -type f -print | sort > /tmp/before.txt
+          echo "::group::Installing and building wpcomsh"
+          pnpm jetpack build -v --deps plugins/wpcomsh
+          echo "::endgroup::"
+
+          # We only want to save the files that were actually created or changed.
+          # But we can't just list them for actions/cache/save, "Argument list too long".
+          # So instead we delete all the unchanged files so we can tell actions/cache/save
+          # to save everything that's left.
+          git -c core.quotepath=off diff --name-only | sort > /tmp/changed.txt
+          if [[ -s /tmp/changed.txt ]]; then
+            grep -F -x -v -f /tmp/changed.txt /tmp/before.txt > /tmp/remove.txt
+          else
+            cp /tmp/before.txt /tmp/remove.txt
+          fi
+          xargs -d '\n' rm < /tmp/remove.txt
+          find . -type d -empty -delete
+
+      - name: Save wpcomsh build cache
+        if: steps.changed.outputs.wpcomsh == 'true'
+        id: wpcomsh-build-cache-save
+        uses: actions/cache/save@v4
+        with:
+          path: |
+            .
+            !./.github/
+          key: ${{ github.sha }}
+  deploy:


.github/workflows/wpcloud.yml

+    name: Run PHPUnit on the WPCloud test site
+    runs-on: ubuntu-latest
+    needs: build
+    if: needs.build.outputs.wpcomsh == 'true'
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Restore wpcomsh build cache
+        id: wpcomsh-build-cache
+        uses: actions/cache/restore@v4
+        with:
+          path: |
+            .
+            !./.github/
+          key: ${{ github.sha }}
+          fail-on-cache-miss: true
+
+      - name: Setup tools
+        uses: ./.github/actions/tool-setup
+
+      - name: Install monorepo
+        run: |
+          pnpm install
+
+      - name: Configure Github to be able to SSH to the Atomic site
+        run: |
+          echo "::group::Intializing"
+
+          mkdir -vp ~/.ssh/
+          chmod -v 700 ~/.ssh
+
+          touch ~/.ssh/id_site
+          touch ~/.ssh/known_hosts
+          chmod 600 ~/.ssh/id_site
+          chmod 600 ~/.ssh/known_hosts
+          echo "$SSH_KEY" > ~/.ssh/id_site
+          echo "wrote ~/.ssh/id_site"
+          echo "$SSH_KNOWN_HOSTS" > ~/.ssh/known_hosts
+          echo "wrote ~/.ssh/known_hosts"
+
+          echo "::endgroup::"
+
+          echo "::group::Transferring wpcomsh to the testing server"
+          pnpm jetpack rsync wpcomsh [email protected]:~/htdocs/wp-content/mu-plugins
+          scp -r projects/plugins/wpcomsh/bin [email protected]:/srv/htdocs/wp-content/mu-plugins/wpcomsh
+          scp -r projects/plugins/wpcomsh/tests [email protected]:/srv/htdocs/wp-content/mu-plugins/wpcomsh/
+          scp projects/plugins/wpcomsh/phpunit.xml.dist [email protected]:/srv/htdocs/wp-content/mu-plugins/wpcomsh/
+
+          echo "::engroup::"
+
+          echo "::group::execution"
+          ssh -i ~/.ssh/id_site [email protected] "~/htdocs/github-action-handler.sh" || CODE=$?
+          echo "::endgroup::"
+
+          echo "::group::teardown"
+          rm -rvf ~/.ssh/
+          echo "::endgroup::"
+          echo "Exiting with exit code $CODE"
+          exit $CODE
+        env:
+          SSH_KEY: ${{ secrets.UPDATEJETPACKSTAGING_SSH_KEY }}
+          SSH_KNOWN_HOSTS: ${{ secrets.UPDATEJETPACKSTAGING_SSH_KNOWN_HOSTS }}


Added a CI job for WPCloud testing of wpcomsh.

3a57a51

zinigor requested review from a8caustin and a team September 5, 2024 17:58

github-actions bot added the Actions GitHub actions used to automate some of the work around releases and repository management label Sep 5, 2024

github-actions bot added the [Status] Needs Author Reply We would need you to make some changes or provide some more details about your PR. Thank you! label Sep 5, 2024

Add pnpm and install.

2ab2c4a

zinigor added the [Status] In Progress label Sep 5, 2024

anomiex reviewed Sep 5, 2024

View reviewed changes

zinigor added 7 commits October 7, 2024 20:24

Testing WPCloud on a new test site for jetpackisbestpack.

0d40db9

Merge branch 'trunk' into add/wpcomsh-wpcloud-testing

a8da4b4

Temporarily marking wpcomsh as always changed.

a3a5533

Merge branch 'trunk' into add/wpcomsh-wpcloud-testing

8b74cdf

Added transferring of tests.

4df8ed7

Added DB password slash escaping.

b102738

changelog

27bbd07

github-actions bot added the [Plugin] Wpcomsh label Oct 23, 2024

Added a mock for get_option_and_ensure_autoload.

aad27e3

github-actions bot added the [Tests] Includes Tests label Oct 23, 2024

zinigor added 4 commits October 24, 2024 19:14

Added a missing cache expulsion function.

9e892e4

Removed set -x from test installer.

dfb7439

Using pnpm to not bother with linking binaries.

bf302d5

Added a build step.

8f1f194

anomiex reviewed Oct 24, 2024

View reviewed changes

zinigor added 4 commits October 25, 2024 18:36

Merge branch 'trunk' into add/wpcomsh-wpcloud-testing

df0ed81

Created a separate job to avoid re-building on a re-run, h/t @anomiex.

3618211

Added cache saving and restoring, moved if to parent job.

510482f

Added the before file.

747555d

anomiex reviewed Oct 25, 2024

View reviewed changes

zinigor added 4 commits October 25, 2024 20:09

Trying with instead of .

cdff320

Fixing var schema.

a60df09

Merge branch 'trunk' into add/wpcomsh-wpcloud-testing

4f4f451

Merge branch 'trunk' into add/wpcomsh-wpcloud-testing

f2ff12f

robertsreberski assigned zinigor Jan 14, 2025

Merge branch 'trunk' into add/wpcomsh-wpcloud-testing

333469a

github-advanced-security bot found potential problems Jan 14, 2025

View reviewed changes

zinigor added 11 commits January 15, 2025 18:47

Merge branch 'trunk' into add/wpcomsh-wpcloud-testing

7de6b72

Fixed the user string in SSH connection.

3ff1bea

Trying to create an SSH config file.

131ac21

Fixed the SSH key file name.

6b04ffa

Making sure to use the preset configured host.

738ee0b

Moved the polyfill load to after WordPress loading.

e1dbc3e

Removed polyfills.

88dc9a6

Merge branch 'trunk' into add/wpcomsh-wpcloud-testing

6189efa

Removed null coalescing and added a short ternary.

b9de362

Removed the change detector prop.

a6b8715

Added concurrency settings to make sure there can be only one.

805d152

zinigor requested a review from a team January 17, 2025 16:32

zinigor added [Status] Needs Review To request a review from fellow Jetpack developers. Label will be renamed soon. and removed [Status] Needs Author Reply We would need you to make some changes or provide some more details about your PR. Thank you! [Status] In Progress labels Jan 17, 2025

github-actions bot added [Status] Needs Author Reply We would need you to make some changes or provide some more details about your PR. Thank you! and removed [Status] Needs Review To request a review from fellow Jetpack developers. Label will be renamed soon. labels Jan 17, 2025

Added updating of source files and cleanup after that updating.

e482ee5

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added a CI job for WPCloud testing of wpcomsh. #39258

Added a CI job for WPCloud testing of wpcomsh. #39258

zinigor commented Sep 5, 2024

github-actions bot commented Sep 5, 2024 •

edited

Loading

anomiex Sep 5, 2024

anomiex Sep 5, 2024

anomiex Sep 5, 2024

anomiex Oct 24, 2024

anomiex Oct 25, 2024

anomiex Oct 25, 2024

	if: needs.build.changed.outputs.wpcomsh == 'true'
	if: needs.build.outputs.wpcomsh == 'true'

Added a CI job for WPCloud testing of wpcomsh. #39258

Are you sure you want to change the base?

Added a CI job for WPCloud testing of wpcomsh. #39258

Conversation

zinigor commented Sep 5, 2024

Proposed changes:

Other information:

Jetpack product discussion

Does this pull request change what data or activity we track or use?

Testing instructions:

github-actions bot commented Sep 5, 2024 • edited Loading

anomiex Sep 5, 2024

Choose a reason for hiding this comment

anomiex Sep 5, 2024

Choose a reason for hiding this comment

anomiex Sep 5, 2024

Choose a reason for hiding this comment

anomiex Oct 24, 2024

Choose a reason for hiding this comment

anomiex Oct 25, 2024

Choose a reason for hiding this comment

anomiex Oct 25, 2024

Choose a reason for hiding this comment

github-actions bot commented Sep 5, 2024 •

edited

Loading