Enable Docs DAG in CI leveraging existing CI connections

[pankajkoti]

It seems that the dbt docs DAG in our example DAGs suite was silently failing to run because the specified connections in the DAG were not present in our CI environment. I am removing the connection checks, allowing the previously skipped tasks to execute even when the remote Airflow connections are unavailable. Additionally, I’ve updated the DAG to use our existing CI credentials and Airflow connections, as well as updated the bucket name to the ones I created in our object stores. This change will help catch issues like #1420 earlier. While fix #1422 has already addressed #1420, this PR will now validate that the fix is functioning as expected.

closes: #1420

[netlify]

✅ Deploy Preview for sunny-pastelito-5ecb04 ready!

Name	Link
🔨 Latest commit	bdde29f
🔍 Latest deploy log	https://app.netlify.com/sites/sunny-pastelito-5ecb04/deploys/676eb522415fe600087d6dda
😎 Deploy Preview	https://deploy-preview-1428--sunny-pastelito-5ecb04.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[cloudflare-workers-and-pages]

Deploying astronomer-cosmos with    Cloudflare Pages

Latest commit:	bdde29f
Status:	✅  Deploy successful!
Preview URL:	https://d780da1a.astronomer-cosmos.pages.dev
Branch Preview URL:	https://enable-docs-dag.astronomer-cosmos.pages.dev

View logs

[pankajkoti]

Action run https://github.com/astronomer/astronomer-cosmos/actions/runs/12500718837/job/34877247783?pr=1428 before merging and rebasing PR #1422 here shows that we now are running the example DAG for docs upload test in our CI with this PR.

[tatiana]

Thanks for addressing this, @pankajkoti ! It seems the CI is failing for other reasons. Happy for this to be merged once we address the other issues in a separate PR - so the CI is green.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.92%. Comparing base (0f89e01) to head (bdde29f).
Report is 2 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1428      +/-   ##
==========================================
+ Coverage   96.52%   96.92%   +0.39%     
==========================================
  Files          73       73              
  Lines        4320     4320              
==========================================
+ Hits         4170     4187      +17     
+ Misses        150      133      -17     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[pankajkoti]

Thanks for addressing this, @pankajkoti ! It seems the CI is failing for other reasons. Happy for this to be merged once we address the other issues in a separate PR - so the CI is green.

thanks @tatiana . There were a couple of issues. One of them was resolved in PR #1432 . The other being that the DAG operators were leading to compilation errors as dbt deps were not installed, which led to failing command execution and hence no target directory was created from where we were trying to upload the files. We now are installing dbt deps in the DAG operators.

[tatiana]

Thanks for fixing all the issues, @pankajkoti !