Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: rework options flow to have a more consistent pattern #390

Draft
wants to merge 9 commits into
base: main
Choose a base branch
from
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,6 @@ require (
github.com/stretchr/testify v1.9.0
go.uber.org/multierr v1.11.0
go.uber.org/zap v1.27.0
golang.org/x/crypto v0.23.0
k8s.io/api v0.29.3
k8s.io/apiextensions-apiserver v0.29.3
k8s.io/apimachinery v0.29.3
Expand Down Expand Up @@ -126,6 +125,7 @@ require (
go.opentelemetry.io/otel/trace v1.21.0 // indirect
go.uber.org/automaxprocs v1.5.3 // indirect
go.uber.org/mock v0.4.0 // indirect
golang.org/x/crypto v0.23.0 // indirect
golang.org/x/exp v0.0.0-20231006140011-7918f672742d // indirect
golang.org/x/net v0.25.0 // indirect
golang.org/x/oauth2 v0.16.0 // indirect
Expand Down
9 changes: 3 additions & 6 deletions karpenter-values-template.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -34,15 +34,12 @@ controller:
value: ${AZURE_SUBSCRIPTION_ID}
- name: LOCATION
value: ${AZURE_LOCATION}
# settings for managed workload ideneity
- name: ARM_USE_CREDENTIAL_FROM_ENVIRONMENT
value: "true"
- name: ARM_USE_MANAGED_IDENTITY_EXTENSION
value: "false"
- name: ARM_USER_ASSIGNED_IDENTITY_ID
- name: ARM_KUBELET_IDENTITY_CLIENT_ID
value: ""
- name: AZURE_NODE_RESOURCE_GROUP
value: ${AZURE_RESOURCE_GROUP_MC}
- name: ARM_AUTH_METHOD
value: "workload-identity"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: the reason its called "from environment" is because its actually using a more generic/general auth option that finds the creds in the environment. This is the flow that works for workload-identity. However, I think the naming should probably keep the more generic term since that is what its actually doing. Unless you have more tied to this.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, by theory, there could be "from environment" auth option that is not workload identity, right?
If that's the case then I think keeping "from-environment" would be a better choice, given we doesn't seem to have ties to workload identity (right?).

serviceAccount:
name: ${KARPENTER_SERVICE_ACCOUNT_NAME}
annotations:
Expand Down
111 changes: 111 additions & 0 deletions pkg/auth/authmanager.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
/*
Portions Copyright (c) Microsoft Corporation.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package auth

import (
"fmt"

"github.com/Azure/go-autorest/autorest"
"github.com/Azure/go-autorest/autorest/adal"
"github.com/Azure/go-autorest/autorest/azure"
"k8s.io/klog/v2"

"github.com/Azure/azure-sdk-for-go/sdk/azcore"
"github.com/Azure/azure-sdk-for-go/sdk/azidentity"
"github.com/jongio/azidext/go/azidext"
)

const (
// auth methods
AuthMethodSysMSI = "system-assigned-msi"
AuthMethodWorkloadIdentity = "workload-identity"
)

// AuthManager manages the authentication logic for Azure clients used by Karpenter to make requests
type AuthManager struct {
authMethod string
location string
}

func NewAuthManagerWorkloadIdentity(location string) *AuthManager {
return &AuthManager{
authMethod: AuthMethodWorkloadIdentity,
location: location,
}

}

func NewAuthManagerSystemAssignedMSI(location string) *AuthManager {
return &AuthManager{
authMethod: AuthMethodSysMSI,
location: location,
}
}

// NewCredential provides a token credential
func (am AuthManager) NewCredential() (azcore.TokenCredential, error) {
if am.authMethod == AuthMethodWorkloadIdentity {
klog.V(2).Infoln("cred: using workload identity for new credential")
return azidentity.NewDefaultAzureCredential(nil)
}

if am.authMethod == AuthMethodSysMSI {
klog.V(2).Infoln("cred: using system assigned MSI for new credential")
msiCred, err := azidentity.NewManagedIdentityCredential(nil)
if err != nil {
return nil, err
}
return msiCred, nil
}

return nil, fmt.Errorf("cred: unsupported auth method: %s", am.authMethod)
}
Comment on lines +59 to +76
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this belong in the shared SDK? For my token fix i imagine other ppl may want the same fix. Uing azure sdk for go extensions may be a good idea for our auth so we can share that logic too. Should we move this there?


func (am AuthManager) NewAutorestAuthorizer() (autorest.Authorizer, error) {
// TODO (charliedmcb): need to get track 2 support for the skewer API, and align all auth under workload identity in the same way within cred.go
if am.authMethod == AuthMethodWorkloadIdentity {
klog.V(2).Infoln("auth: using workload identity for new authorizer")
cred, err := azidentity.NewDefaultAzureCredential(nil)
if err != nil {
return nil, fmt.Errorf("default cred: %w", err)
}
return azidext.NewTokenCredentialAdapter(cred, []string{azidext.DefaultManagementScope}), nil
}

if am.authMethod == AuthMethodWorkloadIdentity {
comtalyst marked this conversation as resolved.
Show resolved Hide resolved
klog.V(2).Infoln("auth: using system assigned MSI to retrieve access token")
msiEndpoint, err := adal.GetMSIVMEndpoint()
if err != nil {
return nil, fmt.Errorf("getting the managed service identity endpoint: %w", err)
}

azureEnvironment, err := azure.EnvironmentFromName(am.location)
if err != nil {
return nil, fmt.Errorf("failed to get AzureEnvironment: %w", err)
}

token, err := adal.NewServicePrincipalTokenFromMSI(
msiEndpoint,
azureEnvironment.ServiceManagementEndpoint)
if err != nil {
return nil, fmt.Errorf("retrieve service principal token: %w", err)
}
return autorest.NewBearerAuthorizer(token), nil
}

return nil, fmt.Errorf("auth: unsupported auth method %s", am.authMethod)
}
105 changes: 0 additions & 105 deletions pkg/auth/autorest_auth.go

This file was deleted.

Loading
Loading