Log analytics oci-onm-discovery Pods failing in Kubernetes

[mads-garrido]

Getting following error when discovery pods try to upload payload to OCI Log Analytics. Any ideas how to resolve or troubleshoot further?

2025-01-14 13:55:18 +0000 [error] Util::LogAnalytics Error while uploading payload to 'Log Analytics': { 'message': 'Authorization failed or requested resou rce not found.', 'status': 404, 'code': 'NotAuthorizedOrNotFound', 'opc-request-id': 'BED30022EDDD4BB8AD8BD51048541DC8/D0DD461FC7DF9A3E47CB581247DBB220/899E 56E333D7975462853648B2B12FF4' }

We have the following policies enabled for the compartment/tenancy:

Allow dynamic-group hiestage1-oci-kubernetes-monitoring to {LOG_ANALYTICS_LOG_GROUP_UPLOAD_LOGS} in compartment id ocid1.compartment.oc1..aaaaaaaa25vnentz2xpam3b2sk4slidqsntc3gekadudsj477uhregi5dedq Allow dynamic-group hiestage1-oci-kubernetes-monitoring to use METRICS in compartment id ocid1.compartment.oc1..aaaaaaaa25vnentz2xpam3b2sk4slidqsntc3gekadudsj477uhregi5dedq WHERE target.metrics.namespace = 'mgmtagent_kubernetes_metrics' Allow dynamic-group hiestage1-oci-kubernetes-monitoring to {LOG_ANALYTICS_DISCOVERY_UPLOAD} in tenancy

Many thanks,
Madhava

[palukuri]

Madhava, can you confirm the OCI region where you have configured your cluster so we can take a look at the logs and confirm the missing policy.

[palukuri]

I was able to find that your cluster is in Ashburn, where we see that it is failing for "LOG_ANALYTICS_DISCOVERY_UPLOAD" permission. Can you validate the instances (in the cluster) are part of the dynamic group "hiestage1-oci-kubernetes-monitoring".

[mads-garrido]

Thank you Kiran for support and advice - the dynamic group was not linked correctly to the policy as suggested. Now the logs are uploading to Log Analytics.