Migration of meta-qcom (and merge with meta-qcom-hwe)

[ndechesne]

Related to: Linaro/meta-qcom#680

This is probably impossible to properly review in Github. The key commits to review are the most recent commits in the PR branch.

First I am doing a 'non trivial' merge of meta-qcom (master) into meta-qcom-hwe (main) using "git merge --allow-unrelated-histories". Most of the merge is trivial with no conflicts, e.g. merging recipes, conf files, ... from both layers, just fine. The main conflicts were with README.md and conf/layer.conf. For both of these files, I have done a manual merge/edit so that they look right to me. These files need to be reviewed carefully.

The next 2 commits are changes in the Github workflows. What I am trying to do with the workflows is:

• For all stable up to styhead, they predate the migration, so Linaro will continue to maintain them, and the CI should be based on existing Linaro CI
• For master and all future stable branches, we are moving to Qualcomm CI

Note: the Linaro CI is not expected to work for now, since we will need to setup the secrets/token.

The result of the merge represents a very decent history of meta-qcom with ~1650 commits from ~50 developers over the last 10 years.

In terms of process, we eventually want to migrate meta-qcom into github.com/quic-yocto org. We can decide to merge this PR (when it's tested/ready) and continue the development of meta-qcom-hwe (main branch) here after the initial merge. When the repo is migrated, we will just need to do a final merge of the meta-qcom-hwe main branch into meta-qcom repo.

[lumag]

I completely forgot about that. I will take a look at the CI for other branches

[koenkooi]

FWIW, a kas based build for qcs6490-rb3gen2-core-kit using this PR builds without issues.

[lumag]

@ndechesne I think I've fixed all outstanding CI issues.

[igoropaniuk]

Taking into consideration that there are currently 113 commits in meta-qcom-hwe and it's not currently "used in production" (in the terms of maturity), is there any reasons why the decision was made to merge it this way and not the other way around: fork meta-qcom first and then merge meta-qcom-hwe into it? This would definitely have required less effort in merging and follow-up testing, and as a result we would end up with 1:1 replication of meta-qcom git history

Another concern is that as for meta-qcom-hwe "rebase and merge" strategy was chosen from the very beginning, and pulling all merge commits from meta-qcom IMO might "spoil" existing git history in meta-qcom-hwe (I know it's just a matter of preference, but anyway there will be some chaos). Have we considered getting rid of merge commits (I know that it's a huge amount work, but just curious if it's feasible)?

[ndechesne]

Taking into consideration that there are currently 113 commits in meta-qcom-hwe and it's not currently used in production, is there any reasons why the decision was made to merge it this way and not the other way around: fork meta-qcom first and then merge meta-qcom-hwe into it? This would definitely have required less effort in merging and follow-up testing, and as a result we would end up with 1:1 replication of meta-qcom git history

git merge A from B, or git merge B from A. I don't see how it makes the merge any different. The same merge conflicts would exist. And in fact, there really was no difficult merge at all. It was fairly trivial to do the merge, the way I did it.

Since we are merging the 2 branches, we still keep the history of both branches, so the history of meta-qcom is replicated. Can you please elaborate? I am open to doing things in a different way if that makes anything simpler.

Another concern is that as for meta-qcom-hwe "rebase and merge" strategy was chosen from the very beginning, and pulling all merge commits from meta-qcom IMO might "spoil" existing git history in meta-qcom-hwe (I know it's just a matter of preference, but anyway there will be some chaos). Have we considered getting rid of merge commits (I know that it's a huge amount work, but just curious if it's feasible)?

That is correct. the history is going to be a bit of mess, but moving forward we will adopt rebase and merge again. As long as we kept the 2 branches history in the merge, I think it's fine.

[igoropaniuk]

@ndechesne

Since we are merging the 2 branches, we still keep the history of both branches, so the history of meta-qcom is replicated. Can you please elaborate? I am open to doing things in a different way if that makes anything simpler.

I got confused as I was thinking about rebase for some reason when I wrote "1:1 replication of meta-qcom git history", however you explicitly mentioned "merge" in the PR description, so sorry for the nohistoryise.
My point was that having PR with 112 commits would look more lightweight and less scary :) , but I do agree that in the end we would get the same git tree and it's just a matter of preference

[koenkooi]

When using 'kas build' for the rbd3, this doesn't seem to generate a qcomflash tarball anymore.

[igoropaniuk]

@koenkooi I think that's because it doesn't include this commit:

commit d6358019cce14f21e95a6a31cec28b134e2fa0d7
Author: Ricardo Salveti <[email protected]>
Date:   Thu Nov 14 19:22:20 2024 -0300

    image_types_qcom: add support for qcomflash IMAGE_FSTYPE
    
    Add support for a new IMAGE_FSTYPE which creates a single tarball
    containing all the relevant files to perform a full clean flash,
    including partition metadata, boot firmware, ESP partition and the
    rootfs.
    
    Signed-off-by: Ricardo Salveti <[email protected]>

[ndechesne]

I updated the PR. What's changed:

1. I did the merge again with the most recent commits from meta-qcom and meta-qcom-hwe
2. completely disabled (removed the workflow) the Linaro CI in main
3. added meta-qcom generic machines in CI

[lumag]

Nice! Ack from my side.

[ndechesne]

@ricardosalveti when we decide to merge this PR, we will need to do a merge commit, it's an exception to our process, but i believe that a rebase/squash will break everything..

[lumag]

My quick test succeeds. I'd say, let's merge this, unify repos and continue the development as agreed.

[ndechesne]

I pushed a fix in meta-qcom that should address the yocto-check-layer error. It was a problem in the original Linaro/meta-qcom which we found during the merge.

[sbanerjee-quic]

LGTM

[ricardosalveti]

LGTM

[quaresmajose]

LGTM