Exaggerating the importance of pinning in CIs

[bagder]

I work on a project (@curl) where we use a lot of CI - almost two hundred individual jobs right now. All of them done for testing. They fire up things, test them and nothing other than logs survive from them. (I don't think this is unique for our project, I believe this is a common style of doing this.)

These are the jobs that the scorecard will check and complain that we don't do pinning (enough).

It seems that the scorecard assumes that pinning are done for important product things. It seems to assume that a missed pin actually can infect the product or that a malicious dependency there can inject something into the product - but here they can't.

A missed pin can make us load a malicious dependency in a temporary virtual machine and it can at worst trick us that a PR works when in reality it doesn't. Such a test would then still fail when run locally and in other CI services.

The intended use of the dependency that the scorecard insists should be pinned is therefore extremely important as to how important or not pinning is.

Since the scorecard has no idea what the jobs do, it seems to exaggerate the importance of pinning for this use case.

[johnbillion]

There's some discussion about this on #2018