Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ci: carbonix_build zip build artifacts #282

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 105 additions & 3 deletions .github/workflows/carbonix_build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -265,7 +265,10 @@ jobs:
- name: Collect XML files
id: collect
run: |
xml_files=$(find libraries/AP_HAL_ChibiOS/hwdef/CarbonixCommon/aircraft_configuration -name "*.xml" -print0 | xargs -0 echo | tr -d '\n' | jq -R -s -c 'split(" ")')
xml_files=$(find libraries/AP_HAL_ChibiOS/hwdef/CarbonixCommon/aircraft_configuration -name "*.xml" -print0 | \
xargs -0 -n 1 basename | \
sed 's/.xml$//' | \
jq -R -s -c 'split("\n")[:-1]')
echo "aircraft-config-files=$xml_files" >> $GITHUB_OUTPUT

process-ac:
Expand Down Expand Up @@ -316,11 +319,19 @@ jobs:
shell: sh -e {0}

- name: Run aircraft_config.py
id: aircraft_config
run: |
python Tools/Carbonix_scripts/aircraft_config.py ${{ matrix.xml_file }} ${{ env.commit_id }}
ls -la final-output/*/ || echo "No files found"
python Tools/Carbonix_scripts/aircraft_config.py "libraries/AP_HAL_ChibiOS/hwdef/CarbonixCommon/aircraft_configuration/${{ matrix.xml_file }}.xml" "${{ env.commit_id }}"
if [ -d "final-output" ]; then
echo "exists=true" >> $GITHUB_ENV
echo "Found final-output directory"
else
echo "exists=false" >> $GITHUB_ENV
echo "No final-output directory found"
fi

- name: Install AWS CLI
if: ${{ env.exists == 'true' }}
run: |
apt-get update -y
DEBIAN_FRONTEND=noninteractive apt-get install -y curl unzip
Expand All @@ -329,18 +340,28 @@ jobs:
./aws/install --update

- name: Configure AWS credentials
if: ${{ env.exists == 'true' }}
uses: aws-actions/configure-aws-credentials@v4
with:
aws-access-key-id: ${{ secrets.AWS_S3_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_S3_SECRET_ACCESS_KEY }}
aws-region: us-east-1

- name: Upload to S3
if: ${{ env.exists == 'true' }}
run: |
PATH_TO_S3=${{ needs.setup-s3-path.outputs.s3_path }}
echo "Uploading Artifacts to: $PATH_TO_S3"
aws s3 cp final-output/ $PATH_TO_S3 --recursive

- name: Upload build artifacts
if: ${{ env.exists == 'true' }}
uses: actions/upload-artifact@v4
with:
name: ${{ matrix.xml_file }}
path: final-output/${{ matrix.xml_file }}/


build-sitl:
runs-on: 'windows-latest'
if: ${{ !contains(github.event.pull_request.labels.*.name, 'SKIP_BUILD') }}
Expand Down Expand Up @@ -446,3 +467,84 @@ jobs:
$env:PATH_TO_S3 = '${{ needs.setup-s3-path.outputs.s3_path }}'
echo "Uploading to: $env:PATH_TO_S3"
aws s3 cp temp/ $env:PATH_TO_S3 --recursive

build-upload-zip:
runs-on: ubuntu-22.04
if: ${{ !contains(github.event.pull_request.labels.*.name, 'SKIP_BUILD') }}
needs: [process-ac, build-sitl, setup-s3-path]
steps:
- name: Extract folder name from s3_path
id: extract-folder-name
run: |
s3_path="${{ needs.setup-s3-path.outputs.s3_path }}"
folder_name=$(basename "$s3_path")
echo "folder_name=$folder_name" >> $GITHUB_ENV

- name: Create folder from s3_path
run: mkdir -p aircraft_zip

- name: List available artifacts
id: list-artifacts
uses: actions/github-script@v6
with:
script: |
const artifacts = await github.rest.actions.listArtifactsForRepo({
owner: context.repo.owner,
repo: context.repo.repo,
});
return artifacts.data.artifacts.map(artifact => artifact.name);

- name: Filter artifacts
id: filter-artifacts
run: |
echo "Filtered artifacts:" > filtered_artifacts.txt
for artifact in ${{ steps.list-artifacts.outputs.result }}; do
if [[ $artifact =~ .*_AC_.*[0-9]+(\.[0-9]+)? ]]; then
echo $artifact >> filtered_artifacts.txt
fi
done
echo "sitl" >> filtered_artifacts.txt
cat filtered_artifacts.txt

- name: Download filtered artifacts
uses: actions/download-artifact@v4
with:
name: ${{ steps.filter-artifacts.outputs.result }}
path: aircraft_zip/

- name: Delete unnecessary folders
run: |
rm -rf aircraft_zip/build-periph-*

- name: List downloaded files
run: ls -R aircraft_zip/

- name: Install p7zip
run: sudo apt-get install -y p7zip-full

- name: 7zip the folder
run: |
cd aircraft_zip
7z a ../${{ env.folder_name }}.7z .
loki077 marked this conversation as resolved.
Show resolved Hide resolved

- name: Upload 7z folder to artifacts
uses: actions/upload-artifact@v4
with:
name: ${{ env.folder_name }}
path: ${{ env.folder_name }}.7z
retention-days: 15

- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
aws-access-key-id: ${{ secrets.AWS_S3_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_S3_SECRET_ACCESS_KEY }}
aws-region: us-east-1

- name: Upload 7z folder to S3
run: |
PATH_TO_S3=${{ needs.setup-s3-path.outputs.s3_path }}
# Remove trailing slash if it exists
PATH_TO_S3=${PATH_TO_S3%/}
echo "Uploading to: $PATH_TO_S3"
aws s3 cp ${{ env.folder_name }}.7z $PATH_TO_S3/${{ env.folder_name }}.7z
26 changes: 26 additions & 0 deletions Tools/Carbonix_scripts/aircraft_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -231,6 +231,28 @@ def build_flight_controller_firmware(board_name : str) -> None:
raise RuntimeError(f"Error building firmware for {board_name}")


def check_config_status(xml_file : str) -> bool:
"""Check the status of the aircraft configuration.

Args:
xml_file (str): Path to the XML file.
Returns:
bool: False if the configuration is deprecated, True otherwise.
"""
tree = ET.parse(xml_file)
root = tree.getroot()

aircraft = root.find('aircraft')
if aircraft is None:
raise AssertionError(f"'aircraft' element not found in {xml_file}")

status = aircraft.find('status')
if status is None:
raise AssertionError(f"'status' element not found in the 'aircraft' element of {xml_file}")

return status.text == 'active'


if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('xml_file', help='Path to the XML file')
Expand All @@ -241,6 +263,10 @@ def build_flight_controller_firmware(board_name : str) -> None:
print('XML file:', args.xml_file)
print('Commit ID:', args.commit_id)

if not check_config_status(args.xml_file):
print('Aircraft configuration is deprecated. No further action needed.')
exit(0)

xml_file = copy_configuration_file(args.xml_file, args.commit_id)
copy_lua_scripts(xml_file)
fc_board_name = get_flight_controller_board_name(xml_file)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
CAN Airspeed,
CAN Battery Monitor
</model_description>
<status>deprecated</status>
<configuration_version>1</configuration_version>
</aircraft>
<flight_controller>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
CAN Airspeed,
CAN Battery Monitor
</model_description>
<status>active</status>
<configuration_version>1</configuration_version>
</aircraft>
<flight_controller>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
Non Crystal CPNs,
CAN Airspeed
</model_description>
<status>active</status>
<configuration_version>1</configuration_version>
</aircraft>
<flight_controller>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
CAN GPS,
CAN Airspeed
</model_description>
<status>active</status>
<configuration_version>1</configuration_version>
</aircraft>
<flight_controller>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
CAN GPS,
CAN Airspeed
</model_description>
<status>active</status>
<configuration_version>1</configuration_version>
</aircraft>
<flight_controller>
Expand Down
Loading