Optimizing CR processing after Kubernetes CRD controller restart

[lingdie]

I'm working with a Kubernetes Custom Resource Definition (CRD) controller. I've noticed that when the controller restarts, it attempts to reconcile all existing Custom Resources (CRs). This process causes a delay in processing newly created CRs.

My questions are:

1. What are the best practices to mitigate this delay in processing new CRs after a controller restart?

2. Does the controller-runtime library provide any built-in mechanisms to implement a priority queue or similar functionality to prioritize the processing of new CRs?

3. If controller-runtime doesn't offer such functionality, what are some effective ways to implement a priority system within the reconciliation loop?

4. Are there any performance considerations or potential drawbacks to implementing a priority system in a CRD controller?

I'm particularly interested in solutions that can be implemented using Go and the controller-runtime library. Any insights, code examples, or best practices would be greatly appreciated.

[sbueringer]

Great timing 😃. I would recommend trying #2374 once v0.20 is out