fix: properly garbage collecting orphaned network interfaces #642
Conversation
| nodeClaim.Labels = labels | ||
| nodeClaim.Annotations = annotations | ||
| nodeClaim.CreationTimestamp = metav1.Time{Time: *vm.Properties.TimeCreated} | ||
| nodeClaim.CreationTimestamp = metav1.Time{Time: lo.FromPtr(vm.Properties.TimeCreated)} |
There was a problem hiding this comment.
this path was executed when my List query returned an empty result.
In the case where ARG has an outage, garbage collection will break, but there is no need to panic here in that scenario. So switched to lo.FromPtr() instead of normal dereference.
|
Testing this locally. Initially after our first scaleup, we were at 810 network interfaces. After garbage collection ran we can see that we deleted 435 network interfaces, and were left with 375 for our 375 nodes. ~/dev/focus/karpenter-provider-azure (bsoghigian/network-interface-gc*) » cat logs.txt| grep "garbage collected NIC" | wc -l sillygoose@Bryces-MacBook-Pro
|
Pull Request Test Coverage Report for Build 12738183194Warning: This coverage report may be inaccurate.This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.
Details
💛 - Coveralls |
tallaxes
added
area/networking
| "karpenter.sh_cluster": lo.ToPtr("test-cluster"), | ||
| "karpenter.sh_nodepool": lo.ToPtr(nodepoolName), |
There was a problem hiding this comment.
Should use constants from instance provider? But then cyclic dependency?
…tor of arg related functions to their own file
…ctoring gc functions slightly
…ate between them to prevent conflicts in nic deletion calls
…ontroller to avoid attempts to delete nics that are managed by a vm
…cleaner state list
…ased on existing karp-core pattern
Bryce-Soghigian
force-pushed
the
bsoghigian/network-interface-gc
branch
from
9eb0362 to
20c6625
Compare
| GetNic(context.Context, string, string) (*armnetwork.Interface, error) | ||
| DeleteNic(context.Context, string) error | ||
| ListNics(context.Context) ([]*armnetwork.Interface, error) |
There was a problem hiding this comment.
Future refactor here would be to factor out these calls into a network interface provider.
Fixes #92
Description
This introduces a garbage collection controller that will remove network interfaces that are orphaned.
How was this change tested?
Does this change impact docs?
Release Note