Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add OpenCTI Connector Charms #3

Open
wants to merge 77 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 60 commits
Commits
Show all changes
77 commits
Select commit Hold shift + click to select a range
07170cf
add opencti rock
weiiwang01 Dec 2, 2024
11dbfc8
Add basic charm
weiiwang01 Dec 9, 2024
4f210cf
Fix linting issues
weiiwang01 Dec 9, 2024
1dbcebf
Update config description
weiiwang01 Dec 9, 2024
a948dcb
Fix linting issues
weiiwang01 Dec 9, 2024
e990cd3
Fix linting issues
weiiwang01 Dec 9, 2024
3632067
Fix linting issues
weiiwang01 Dec 9, 2024
22077d2
Fix linting issues
weiiwang01 Dec 9, 2024
676b9a0
Fix tests
weiiwang01 Dec 10, 2024
dc9ce01
Add license headers
weiiwang01 Dec 10, 2024
d1c0c05
Update runner
weiiwang01 Dec 10, 2024
38db050
Improved start-up callback
weiiwang01 Dec 10, 2024
8a1264b
Use github hosted runners
weiiwang01 Dec 10, 2024
0caebdc
Update rockcraft.yaml
weiiwang01 Dec 10, 2024
020665a
Debug
weiiwang01 Dec 10, 2024
336b84c
Remove unnecessary files
weiiwang01 Dec 10, 2024
702add8
Remove unnecessary files
weiiwang01 Dec 10, 2024
1f2b5c5
Fix charm.py
weiiwang01 Dec 11, 2024
403aa75
Remove debug info
weiiwang01 Dec 11, 2024
4dbc03a
Stop callback in callback itself
weiiwang01 Dec 11, 2024
e494868
clean up requirements.txt
weiiwang01 Dec 11, 2024
875e116
Update documents
weiiwang01 Dec 11, 2024
9387460
Update unit tests
weiiwang01 Dec 11, 2024
61e7daa
Apply suggestions from review comments
weiiwang01 Dec 12, 2024
89e1c49
work in progress
weiiwang01 Dec 16, 2024
c13a476
work in progress
weiiwang01 Dec 16, 2024
e5c0518
work in progress
weiiwang01 Dec 18, 2024
ab0eb50
Apply suggestions from review comments
weiiwang01 Dec 18, 2024
88ae9c1
Merge platform/graphql and platform/graphql-deps in rockcraft.yaml
weiiwang01 Dec 18, 2024
a781a43
Add missing stage-snaps and stage-packages
weiiwang01 Dec 18, 2024
92d83ef
Merge branch 'init-charm' into connectors
weiiwang01 Dec 19, 2024
83203ef
Fixing merging
weiiwang01 Dec 19, 2024
8063053
Add all 18 connectors
weiiwang01 Dec 19, 2024
baa8794
Add more tests
weiiwang01 Dec 19, 2024
09ddd12
Remove connectors with name too long
weiiwang01 Dec 19, 2024
54ad996
Fix linting issues
weiiwang01 Dec 19, 2024
c5bb243
Fix linting issues
weiiwang01 Dec 19, 2024
fc961a0
Fix linting issues
weiiwang01 Dec 19, 2024
0c77534
Fix rockcraft.yaml
weiiwang01 Dec 19, 2024
a8ab93c
Fix rockcraft.yaml
weiiwang01 Dec 19, 2024
ecbc738
Fix rockcraft.yaml
weiiwang01 Dec 19, 2024
8e15609
Fix rockcraft.yaml
weiiwang01 Dec 19, 2024
e8be78f
Fix rockcraft.yaml
weiiwang01 Dec 19, 2024
5676858
Update .trivyignore
weiiwang01 Dec 19, 2024
758231a
Apply suggestions from review comments
weiiwang01 Dec 20, 2024
06bce03
Cleanup secrets
weiiwang01 Dec 20, 2024
6d95cc3
Merge branch 'init-charm' into connectors
weiiwang01 Dec 20, 2024
f99ae13
Fix linting issues
weiiwang01 Dec 20, 2024
3fd81c1
Merge branch 'init-charm' into connectors
weiiwang01 Dec 20, 2024
f5e2459
Update .trivyignore
weiiwang01 Dec 20, 2024
d0fba99
Update connector generator
weiiwang01 Jan 6, 2025
d726821
Update .woke.yaml
weiiwang01 Jan 6, 2025
5a6e508
Update linting
weiiwang01 Jan 7, 2025
3de4d69
Update .trivyignore
weiiwang01 Jan 7, 2025
5eb4d6c
Revert "Update .trivyignore"
weiiwang01 Jan 7, 2025
b4ab9cd
Update copyright years
weiiwang01 Jan 7, 2025
6a3efdb
Merge branch 'init-charm' into connectors
weiiwang01 Jan 8, 2025
fc7a38d
Update connectors
weiiwang01 Jan 8, 2025
aeb5724
Fix linting
weiiwang01 Jan 8, 2025
41153f6
Fix unit tests
weiiwang01 Jan 8, 2025
980ba73
Update copyright
weiiwang01 Jan 8, 2025
d97b412
Update connectors
weiiwang01 Jan 8, 2025
397d070
Update connectors
weiiwang01 Jan 8, 2025
02b05d7
Test license
weiiwang01 Jan 8, 2025
de0619d
Test license
weiiwang01 Jan 8, 2025
7b19242
Revert "Test license"
weiiwang01 Jan 8, 2025
635b74e
Revert "Test license"
weiiwang01 Jan 8, 2025
2011fac
Update opencti_connector.py
weiiwang01 Jan 14, 2025
2baabee
Update publish_charm.yaml
weiiwang01 Jan 15, 2025
32c6c1d
Merge branch 'main' into connectors
weiiwang01 Jan 15, 2025
63ea921
Update publish_charm.yaml
weiiwang01 Jan 15, 2025
568084a
Update publish_charm.yaml
weiiwang01 Jan 15, 2025
b9d58ab
Update publish_charm.yaml
weiiwang01 Jan 15, 2025
630894c
Update publish_charm.yaml
weiiwang01 Jan 15, 2025
4cd5b01
Update publish_charm.yaml
weiiwang01 Jan 15, 2025
31bae7c
Update publish_charm.yaml
weiiwang01 Jan 15, 2025
dc90606
Update publish_charm.yaml
weiiwang01 Jan 15, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 0 additions & 22 deletions .github/workflows/integration_test.yaml

This file was deleted.

13 changes: 11 additions & 2 deletions .github/workflows/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,14 @@ jobs:
uses: canonical/operator-workflows/.github/workflows/test.yaml@main
secrets: inherit
with:
self-hosted-runner: true
self-hosted-runner-label: "edge"
self-hosted-runner: false
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think our team should be using the edge self hosted runners.

integration-tests:
uses: canonical/operator-workflows/.github/workflows/integration_test.yaml@main
secrets: inherit
with:
channel: 1.29-strict/stable
charmcraft-channel: latest/edge
juju-channel: 3.6/stable
microk8s-addons: "dns ingress rbac storage"
pre-run-script: tests/integration/prepare.sh
self-hosted-runner: false
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,4 @@ __pycache__/
.vscode
.mypy_cache
*.egg-info/
*/*.rock
*.rock
16 changes: 15 additions & 1 deletion .licenserc.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,29 @@ header:
- '**'
paths-ignore:
- '.github/**'
- '**/.gitkeep'
- '**/*.cfg'
- '**/*.conf'
- '**/*.j2'
- '**/*.json'
- '**/*.md'
- '**/*.rule'
- '**/*.tmpl'
- '**/*.txt'
- '.codespellignore'
- '.dockerignore'
- '.flake8'
- '.jujuignore'
- '.gitignore'
- '.licenserc.yaml'
- '.trivyignore'
- '.woke.yaml'
- '.woke.yml'
- 'CODEOWNERS'
- 'icon.svg'
- 'LICENSE'
- 'trivy.yaml'
- 'pyproject.toml'
- 'trivy.yaml'
- 'zap_rules.tsv'
- 'lib/**'
comment: on-failure
8 changes: 8 additions & 0 deletions .trivyignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# cross-spawn (package.json)
CVE-2024-21538
# esbuild
CVE-2024-24790
CVE-2023-45288
CVE-2024-34156
# pebble
CVE-2024-45338
5 changes: 5 additions & 0 deletions .woke.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
ignore_files:
- lib/charms/redis_k8s/v0/redis.py
- connectors/**
- scripts/**
- tests/unit/test_connectors.py
107 changes: 97 additions & 10 deletions charmcraft.yaml
Original file line number Diff line number Diff line change
@@ -1,13 +1,100 @@
# Copyright 2024 Canonical Ltd.
# Copyright 2025 Canonical Ltd.
# See LICENSE file for licensing details.
# This file configures Charmcraft.
# See https://juju.is/docs/sdk/charmcraft-config for guidance.

name: opencti
title: OpenCTI Charm
summary: OpenCTI charm.
links:
documentation: https://github.com/canonical/opencti-operator/blob/main/README.md
issues: https://github.com/canonical/opencti-operator/issues
source: https://github.com/canonical/opencti-operator
contact: https://launchpad.net/~canonical-is-devops

description: |
A [Juju](https://juju.is/) [charm](https://juju.is/docs/olm/charmed-operators)
for deploying and managing the [OpenCTI](https://filigran.io/solutions/open-cti/)
open-source threat intelligence platform in your systems.

This charm simplifies the configuration and maintenance of OpenCTI across a
range of environments, organize your cyber threat intelligence to enhance
and disseminate actionable insights.

config:
options:
admin-user:
type: string
description: |
OpenCTI admin user email and password.
The content of this configuration should be a Juju user secret ID.
The Juju user secret should contain two fields, `email` and `password`,
where `email` is the admin user email, and `password` is the admin user password.
Use the following commands to create a Juju user secret for this configuration:
`juju add-secret opencti-admin-user [email protected] password#file=/path/to/password.txt`
`juju grant-secret opencti-admin-user opencti`

requires:
opensearch-client:
interface: opensearch_client
optional: false
limit: 1
redis:
interface: redis
optional: false
limit: 1
amqp:
interface: rabbitmq
optional: false
limit: 1
s3:
interface: s3
optional: false
limit: 1
ingress:
interface: ingress
optional: false
limit: 1
opencti-connector:
interface: opencti_connector
logging:
interface: loki_push_api
optional: true

provides:
metrics-endpoint:
interface: prometheus_scrape
optional: true
grafana-dashboard:
interface: grafana_dashboard
optional: true

peers:
opencti-peer:
interface: opencti_peer

type: charm
bases:
- build-on:
- name: ubuntu
channel: "22.04"
run-on:
- name: ubuntu
channel: "22.04"
base: [email protected]
build-base: [email protected]
platforms:
amd64:
parts:
charm:
build-snaps:
- rustup
override-build: |
rustup default stable
craftctl default
build-packages:
- libffi-dev
- libssl-dev
- pkg-config

containers:
opencti:
resource: opencti-image
resources:
opencti-image:
type: oci-image
description: OCI image for the OpenCTI platform/worker.

assumes:
- juju >= 3.4
16 changes: 0 additions & 16 deletions config.yaml

This file was deleted.

46 changes: 46 additions & 0 deletions connector-template/charmcraft.yaml.j2
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Copyright 2024 Canonical Ltd.
# See LICENSE file for licensing details.

name: opencti-{{ name }}-connector
title: OpenCTI {{ display_name }} Charm
summary: OpenCTI {{ display_name }} charm.
links:
documentation: https://discourse.charmhub.io
issues: https://github.com/canonical/opencti-operator/issues
source: https://github.com/canonical/opencti-operator
contact: https://launchpad.net/~canonical-is-devops

description: |
A [Juju](https://juju.is/) [charm](https://juju.is/docs/olm/charmed-operators)
for deploying and managing the [OpenCTI Connectors](https://docs.opencti.io/latest/deployment/connectors/)
for the OpenCTI charm.

This charm simplifies the configuration and maintenance of OpenCTI Connectors
across a range of environments, organize your cyber threat intelligence to
enhance and disseminate actionable insights.

{{ config | safe }}

provides:
opencti-connector:
interface: opencti_connector
limit: 1

type: charm
base: [email protected]
build-base: [email protected]
platforms:
amd64:
parts:
charm: {}

containers:
opencti-{{ name }}-connector:
resource: opencti-{{ name }}-connector-image
resources:
opencti-{{ name }}-connector-image:
type: oci-image
description: OCI image for the OpenCTI {{ display_name }} connector.

assumes:
- juju >= 3.4
1 change: 1 addition & 0 deletions connector-template/requirements.txt
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing newline at the end of the file.

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
ops
39 changes: 39 additions & 0 deletions connector-template/rock/rockcraft.yaml.j2
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Copyright 2024 Canonical Ltd.
# See LICENSE file for licensing details.

name: opencti-{{ name }}-connector
base: [email protected]
version: &version '{{ version }}'
summary: OpenCTI {{ display_name }} Connector
description: >-
OpenCTI connectors are the cornerstone of the OpenCTI platform and
allow organizations to easily ingest, enrich or export data.
platforms:
amd64:

parts:
{{ name }}-connector:
source: https://github.com/OpenCTI-Platform/connectors.git
source-type: git
source-tag: *version
source-depth: 1
plugin: nil
build-packages:
- python3-pip
stage-packages:
- python3-dev
- libmagic1
- libffi-dev
override-build: |
craftctl default
ls -lah
mkdir -p $CRAFT_PART_INSTALL/opt
cd {{ constant_to_kebab(connector_type) }}/{{ connector_name }}
cp -rp src $CRAFT_PART_INSTALL/opt/{{ install_location }}
{{ generate_entrypoint }}
cat entrypoint.sh | grep {{ install_location }}
mkdir -p $CRAFT_PART_INSTALL/usr/local/lib/python3.12/dist-packages
pip install \
--target $CRAFT_PART_INSTALL/usr/local/lib/python3.12/dist-packages \
-r $(find -name requirements.txt)
cp entrypoint.sh $CRAFT_PART_INSTALL/
25 changes: 25 additions & 0 deletions connector-template/src/charm.py.j2
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#!/usr/bin/env python3

# Copyright 2024 Canonical Ltd.
# See LICENSE file for licensing details.

"""OpenCTI {{ display_name }} connector charm the service."""

import pathlib

import ops

from charms.opencti.v0.opencti_connector import OpenctiConnectorCharm


class Opencti{{ kebab_to_pascal(name) }}ConnectorCharm(OpenctiConnectorCharm):
connector_type = "{{ connector_type }}"

@property
def charm_dir(self) -> pathlib.Path:
return pathlib.Path(__file__).parent.parent.absolute()

{{ charm_override | safe | indent(4) }}

if __name__ == "__main__":
ops.main(Opencti{{ kebab_to_pascal(name) }}ConnectorCharm)
Loading
Loading