Skip to content

Commit

Permalink
refactor charm to use ROCK image
Browse files Browse the repository at this point in the history
  • Loading branch information
kelkawi-a committed Jun 21, 2024
1 parent 16bd79a commit fe237b7
Show file tree
Hide file tree
Showing 36 changed files with 299 additions and 845 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,4 @@ __pycache__/
.vscode/
*.whl
*.tar.gz
*.rock
15 changes: 6 additions & 9 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ deployment, follow the following steps:
newgrp microk8s

# Enable the necessary Microk8s addons:
microk8s enable hostpath-storage dns
microk8s enable hostpath-storage dns registry

# Install the Juju CLI client, juju:
sudo snap install juju --classic
Expand All @@ -60,20 +60,17 @@ deployment, follow the following steps:
juju model-config logging-config="<root>=INFO;unit=DEBUG"

# Pack the charm:
charmcraft pack [--destructive-mode]
charmcraft pack

# Build wheel file:
cd resource_sample && poetry build -f wheel
# Build ROCK file and push it to local registry:
cd resource_sample_py && make build_rock

# Deploy the charm:
juju deploy ./temporal-worker-k8s_ubuntu-22.04-amd64.charm --resource temporal-worker-image=python:3.8.2-slim-buster
juju deploy ./temporal-worker-k8s_ubuntu-22.04-amd64.charm --resource temporal-worker-image=localhost:32000/temporal-worker-rock
juju config temporal-worker-k8s --file=path/to/config.yaml

# Attach wheel file resource:
juju attach-resource temporal-worker-k8s workflows-file=./resource_sample/dist/python_samples-1.1.0-py3-none-any.whl

# Check progress:
juju status --relations --watch 1s
juju status --relations --watch 2s
juju debug-log

# Clean-up before retrying:
Expand Down
57 changes: 15 additions & 42 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,22 @@ connect to a deployed Temporal server.

### Deploying

The Temporal worker operator can be deployed and connected to a deployed
To deploy the Temporal worker operator, you can start by creating a Temporal
workflow, or use the one provided in
[`resource_sample_py`](./resource_sample_py/). Once done, the project can be
built as a [ROCK](https://documentation.ubuntu.com/rockcraft/en/stable/) and
pushed to the [local registry](https://microk8s.io/docs/registry-built-in) by
running the following command inside the `resource_sample_py` directory:

```bash
make build_rock
```

The Temporal worker operator can then be deployed and connected to a deployed
Temporal server using the Juju command line as follows:

```bash
juju deploy temporal-worker-k8s
juju deploy temporal-worker-k8s --resource temporal-worker-image=localhost:32000/temporal-worker-rock
juju config temporal-worker-k8s --file=path/to/config.yaml
```

Expand All @@ -35,38 +46,6 @@ temporal-worker-k8s:
host: "localhost:7233" # Replace with Temporal server hostname
queue: "test-queue"
namespace: "test"
workflows-file-name: "python_samples-1.1.0-py3-none-any.whl"
# To support all defined workflows and activities, use the 'all' keyword
supported-workflows: "all"
supported-activities: "all"
```
### Attaching "workflows-file" resource
The Temporal worker operator expects a "workflows-file" resource to be attached
after deployment, which contains a set of defined Temporal workflows and
activities as defined in the [resource_sample](./resource_sample/) directory.
The structure of the built wheel file must follow the same structure:
```
- workflows/
- workflow-a.py
- workflow-b.py
- activities/
- activity-a.py
- activity-b.py
- some_other_directory/
- some_helper_file.py
```
The sample wheel file can be built by running `poetry build -f wheel` in the
[resource_sample](./resource_sample/) directory.

Once ready, the resource can be attached as follows:

```bash
make -C resource_sample/ build
juju attach-resource temporal-worker-k8s workflows-file=./resource_sample/dist/python_samples-1.1.0-py3-none-any.whl
```
Once done, the charm should enter an active state, indicating that the worker is
Expand All @@ -77,15 +56,9 @@ pod to ensure there are no errors with the workload container:
kubectl -n <juju_model_name> logs temporal-worker-k8s-0 -c temporal-worker -f
```

Note: Files defined under the "workflows" directory must only contain classes
decorated using the `@workflow.defn` decorator. Files defined under the
"activities" directory must only contain methods decorated using the
`@activity.defn` decorator. Any additional methods or classes needed should be
defined in other files.

## Verifying

To verify that the setup is running correctly, run `juju status --watch 1s` and
To verify that the setup is running correctly, run `juju status --watch 2s` and
ensure the pod is active.

To run a basic workflow, you may use a simple client (e.g.
Expand Down Expand Up @@ -165,7 +138,7 @@ instructions found [here](https://charmhub.io/vault-k8s/docs/h-getting-started).

For a reference on how to access credentials from Vault through the workflow
code,
[`activity2.py`](./resource_sample/resource_sample/activities/activity2.py)
[`activity2.py`](./resource_sample_py/resource_sample/activities/activity2.py)
under the `resource_sample` directory shows a sample for writing and reading
secrets in Vault.

Expand Down
30 changes: 0 additions & 30 deletions config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -31,16 +31,6 @@ options:
default: ""
type: string

supported-workflows:
description: Comma-separated list of workflow names to extract from attached wheel file.
default: ""
type: string

supported-activities:
description: Comma-separated list of workflow activities to extract from attached wheel file.
default: ""
type: string

sentry-dsn:
description: Sentry Data Source Name to send events to.
default: ""
Expand Down Expand Up @@ -69,11 +59,6 @@ options:
default: 1.0
type: float

workflows-file-name:
description: Name of the wheel file resource attached to the charm.
default: ""
type: string

encryption-key:
description: Base64-encoded key used for data encryption.
default: ""
Expand Down Expand Up @@ -161,18 +146,3 @@ options:
description: Client certificate URL for OIDC authentication.
default: ""
type: string

http-proxy:
description: Used to set HTTP_PROXY environment variable.
default: ""
type: string

https-proxy:
description: Used to set HTTPS_PROXY environment variable.
default: ""
type: string

no-proxy:
description: Used to set NO_PROXY environment variable.
default: ""
type: string
6 changes: 0 additions & 6 deletions metadata.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -35,17 +35,11 @@ peers:
containers:
temporal-worker:
resource: temporal-worker-image
# Included for simplicity in integration tests.
upstream-source: python:3.8.2-slim-buster

resources:
temporal-worker-image:
type: oci-image
description: OCI image containing Python package.
workflows-file:
type: file
description: Wheel file containing Temporal workflows and activities.
filename: 'workflows-file.whl'
env-file:
type: file
description: .env file containing environment variables to be sourced to the workload container.
Expand Down
29 changes: 0 additions & 29 deletions resource_sample/resource_sample/workflows/workflow2.py

This file was deleted.

18 changes: 15 additions & 3 deletions resource_sample/Makefile → resource_sample_py/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,17 @@

# Makefile to help automate tasks

# The name of the python package/project
PY_PACKAGE := resource_sample

# ROCK build parameters
ROCK_NAME := temporal-worker_1.0_amd64.rock
IMAGE_NAME := temporal-worker-rock:latest

# build and dist folders
BUILD := build
DIST := dist

# The name of the python package/project
PY_PACKAGE := resource_sample

# Paths to venv executables
POETRY := poetry
PY := python3
Expand Down Expand Up @@ -71,6 +75,14 @@ changelog: ## Add a new entry to the Changelog and bump the package version
build: ## Create a Python source distribution and a wheel in dist
$(POETRY) build

.PHONY: build_rock
build_rock:
rockcraft pack
rockcraft.skopeo --insecure-policy copy oci-archive:$(ROCK_NAME) docker-daemon:$(IMAGE_NAME)
IMAGE_ID=$$(docker inspect --format='{{.Id}}' $(IMAGE_NAME)); \
docker tag $$IMAGE_ID localhost:32000/$(IMAGE_NAME)
docker push localhost:32000/$(IMAGE_NAME)

.PHONY: publish
publish: ## Publish the package to PYPI
$(POETRY) publish
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ packages = [
]

[tool.poetry.dependencies]
python = "^3.8"
protobuf = "^3.2.0"
PyYAML = "^6.0"
temporal-lib-py = "^1.3.1"
Expand Down
Empty file.
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

from temporalio import activity
from dataclasses import dataclass
from resource_sample.common.messages import ComposeGreetingInput
from common.messages import ComposeGreetingInput

# Basic activity that logs and does string concatenation
@activity.defn(name="compose_greeting")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

from temporalio import activity
from dataclasses import dataclass
from resource_sample.common.messages import ComposeGreetingInput
from common.messages import ComposeGreetingInput
import os
import hvac

Expand Down
Empty file.
Loading

0 comments on commit fe237b7

Please sign in to comment.