diff --git a/.github/github-secret.png b/.github/github-secret.png new file mode 100644 index 0000000..deb5d27 Binary files /dev/null and b/.github/github-secret.png differ diff --git a/.github/service-status.png b/.github/service-status.png new file mode 100644 index 0000000..0faee55 Binary files /dev/null and b/.github/service-status.png differ diff --git a/.gitignore b/.gitignore index 3b6f56c..86f2795 100644 --- a/.gitignore +++ b/.gitignore @@ -1 +1,2 @@ -.datex-cache \ No newline at end of file +.datex-cache +res/uix-app-docker/new/repo/ \ No newline at end of file diff --git a/README.md b/README.md index 772a3d9..b1b594a 100644 --- a/README.md +++ b/README.md @@ -1,36 +1,149 @@ # Docker Host -Creates and manages docker containers (e.g. UIX Apps). -Containers can be created via a DATEX interface. +The **Docker Host** is a service to create and manage [UIX](https://github.com/unyt-org/uix) App containers as portable Docker containers used for UIX app deployment. +Remote containers are created via [DATEX interface](https://docs.unyt.org/manual/datex/public-endpoint-interfaces) on the Docker Host. + +## Prerequisites +> [!WARNING] +> **Docker Hosts** are only support on Linux systems. If you experience some issues with your Linux distribution please let us know. + +Make sure to have [`git`](https://git-scm.com/book/en/v2/Getting-Started-The-Command-Line), [`Docker`](https://docs.docker.com/get-started/get-docker/) and [`unzip`](https://linux.die.net/man/1/unzip) installed on your machine and available on the system path. ## Setup -A docker host instance can be created by running the `setup.sh` script: -```shell -curl -s https://raw.githubusercontent.com/unyt-org/docker-host/master/setup.sh | bash -s @+YOUR_DOCKER_HOST + +You can setup your personal **Docker Host** on the target machine using the following command: +```bash +curl -s https://raw.githubusercontent.com/unyt-org/docker-host/main/setup.sh | bash -s @+YOUR_DOCKER_HOST +``` + +Make sure to pass a unique [endpoint id](https://docs.unyt.org/manual/datex/endpoints) to the install script. The setup script will create a Docker Host instance by installing [Deno](https://github.com/denoland/deno) and creating a persistent service inside of `etc/systemd/system`. + +Make sure that the service is up and running: +```bash +systemctl status unyt_YOUR_DOCKER_HOST ``` +![Status ](.github/service-status.png) + +## Configuration +The `config.dx` file is used to apply custom configuration to the Docker Host: +```ts +{ + token: "ACCESS_TOKEN", + enableTraefik: false, + hostPort: 80, + allowArbitraryDomains: true, + setDNSEntries: false +} +``` +* **token** - If an access token is configured, the Docker Host will deploy apps only if they have the correct token configured. It is highly recommended to use a strong access token if the Docker Host is used for personal deployment only. +* **enableTraefik** - If enabled, a [Traefik Proxy](https://traefik.io/traefik/) is installed automatically to act as reverse proxy on your system to handle different domains and automatic SSL. If this option is disabled, you have to make sure to handle the routing of HTTP traffic to your personal container by yourself. +* **hostPort** - Configure the Docker port to expose traefik on. +--- +* **allowArbitraryDomains** (*internal*) - Allow arbitrary domains to be configured. If set to false, only [*.unyt.app](https://unyt.app)-domains can be used for deployment. +* **setDNSEntries** (*internal*) - If you have access to the [unyt.org DNS service](https://github.com/unyt-org/dns), you can enable this option to allow for [*.unyt.app](https://unyt.app)-domains. + +To reload the configuration, the service must be restarted using the following command: +```bash +systemctl restart unyt_YOUR_DOCKER_HOST +``` -## Deploying a UIX app +## Deploy your UIX app -A UIX app is automatically deployed to this host if the `location` option in the `.dx` -file is set to the host endpoint. -This can also be configured for a specific stage only, e.g.: +Your UIX app is automatically deployed to the Docker Host if the `location` option in the `backend/.dx` file is set to the Docker Host endpoint. Please refer to the [Deployment Documentation](https://docs.unyt.org/manual/uix/deployment#example) for more details. -```datex +The location can be customized for specific stages: + +```ts use stage from #public.uix; location: stage { - staging: @+YOUR_DOCKER_HOST_1, - prod: @+YOUR_DOCKER_HOST_2 + staging: @+YOUR_STAG_HOST, + prod: @+YOUR_PROD_HOST +} +``` + +You can configure custom (sub)-domains to be used by your app for different stages: + +```ts +domain: stage { + staging: "staging.example.com", + prod: "example.com" } ``` -### Manual deployment via the DATEX interface +You can configure custom endpoints to be used as your app backend endpoints within different stages: + +```ts +endpoint: stage { + staging: @+example-stage, + prod: @+example +} +``` + +> [!WARNING] +> If the Docker Host you plan to deploy to has a access token configured, you need to pass this access token to UIX to make sure your app can authenticate.
+> You can set the access token as `HOST_TOKEN` environment variable on your local UIX projects console. +> ```bash +> export HOST_TOKEN=YOUR_TOKEN +> ``` + +To deploy your UIX app, please make sure you have the latest changes in sync with your remote git repository. This is required by the Docker Host, since it will clone your sources via [GitHub](https://github.com) or [GitLab](https://gitlab.com) API on deployment. + +To deploy your app, start `uix` via CLI. If you want to select a custom stage pass the `--stage ` argument. + +```bash +uix --stage prod +``` + +Above command will select the location configured for the `prod`-stage and deploy your app: +* **Host**: `@+YOUR_PROD_HOST` +* **Domain**: `example.com` +* **Endpoint**: `@+example` + +## Automated Git Deployment +> [!WARNING] +> **Git Deployment** is only support for [GitHub](https://github.com) and [GitLab](https://gitlab.com) including [self-hosted option](https://docs.gitlab.com/ee/topics/offline/quick_start_guide.html). + + +The [git_deploy plugin](https://docs.unyt.org/manual/uix/deployment#github-deployment) can be configured in your `app.dx` file to automate deployment: + +```ts +plugin git_deploy ( + // Deploy this app in the 'prod' stage + // when a push to the main branch occurs + // The 'MY_SECRET_TOKEN' GitHub secret is passed as an + // environment variables to the UIX app + prod: { + branch: 'main', + on: 'push', + secrets: ['MY_SECRET_TOKEN'] + } +) +``` + +The `git_deploy` plugin takes an object where the keys are the stage names and the values are an object with the following options: + +* `branch: (text or text[])` - One or more branches on which the deployment is triggered +* `on: (text or text[])` - GitHub event name that triggers the deployment +* `secrets (text or text[])` - List of GitHub secrets that are exposed to the app as environment variables +* `tests: (boolean)` - If this option is enabled, tests for all `*.test.ts`, `*.test.js` and `*.test.dx` files in the repository are automatically executed before deployment *(enabled per default)* -```datex +Make sure to run the `uix` command locally before commiting the changes to the remote repository, since when the `git_deploy` plugin is defined, all GitHub workflow files are generated automatically when the app is run. + +> [!WARNING] +> If the Docker Host you plan to deploy to has a access token configured, you need to configure the `HOST_TOKEN` as GitHub/GitLab actions secrets for your repository.
+> ![](.github/github-secret.png) + +### Manual deployment via DATEX interface + +You can use [DATEX](https://github.com/unyt-org/datex-core-js-legacy) to interact with the `ContainerManager` interface for your Docker Host. + +```ts use ContainerManager from @+YOUR_DOCKER_HOST; ref container = ContainerManager.createUIXAppContainer( + "HOST_TOKEN", // Optional access token if configured so "git@github.com:benStre/xam.git", // git origin for the UIX app "main", // branch name @+my_app_deployment, // endpoint for the deployment stage @@ -42,12 +155,16 @@ ref container = ContainerManager.createUIXAppContainer( print container.status // current container status ``` - - -## Create a new Workbench (Development) Container -```datex +## Create Workbench Container +Workbench container might be helpful for development. +```ts use ContainerManager from @+YOUR_DOCKER_HOST; -container = ContainerManager.createWorkbenchContainer(); +container = ContainerManager.createWorkbenchContainer("HOST_TOKEN"); container.start() -``` \ No newline at end of file +``` + + +--- + +© unyt 2024 • [unyt.org](https://unyt.org) \ No newline at end of file diff --git a/config.dx b/config.dx index af071bd..7c612be 100644 --- a/config.dx +++ b/config.dx @@ -1,7 +1,8 @@ // Configuration for the docker host { - enableTraefik: true, // Set up a traefik container to route traffic to other containers + enableTraefik: false, // Set up a traefik container to route traffic to other containers hostPort: 80, // Port to expose traefik on - setDNSEntries: true, // Set up DNS entries for unyt.app domains (requires access to unyt.app DNS) - allowArbitraryDomains: false // Allow arbitrary domains to be used (otherwise, only unyt.app domains can be used) + setDNSEntries: false, // Set up DNS entries for unyt.app domains (requires access to unyt.app DNS) + allowArbitraryDomains: true, // Allow arbitrary domains to be used (otherwise, only unyt.app domains can be used) + token: "admin" // Optional access token for the docker host } \ No newline at end of file diff --git a/deno.json b/deno.json index e478bf2..8664df9 100644 --- a/deno.json +++ b/deno.json @@ -1,6 +1,5 @@ { "importMap": "importmap.dev.json", - "compilerOptions": {}, + "compilerOptions": {}, "lib": ["deno.window", "dom"] -} - \ No newline at end of file +} \ No newline at end of file diff --git a/deno.lock b/deno.lock index a3a5fc7..22459c0 100644 --- a/deno.lock +++ b/deno.lock @@ -1,9 +1,8 @@ { - "version": "3", + "version": "4", "redirects": { - "https://deno.land/std/async/mod.ts": "https://deno.land/std@0.218.2/async/mod.ts", - "https://deno.land/std/uuid/mod.ts": "https://deno.land/std@0.218.2/uuid/mod.ts", - "https://deno.land/x/get_ip/mod.ts": "https://deno.land/x/get_ip@v2.0.0/mod.ts" + "https://deno.land/std/async/mod.ts": "https://deno.land/std@0.224.0/async/mod.ts", + "https://deno.land/std/uuid/mod.ts": "https://deno.land/std@0.224.0/uuid/mod.ts" }, "remote": { "https://deno.land/std@0.168.0/_util/asserts.ts": "d0844e9b62510f89ce1f9878b046f6a57bf88f208a10304aab50efcb48365272", @@ -19,39 +18,65 @@ "https://deno.land/std@0.172.0/path/posix.ts": "0874b341c2c6968ca38d323338b8b295ea1dae10fa872a768d812e2e7d634789", "https://deno.land/std@0.172.0/path/separator.ts": "0fb679739d0d1d7bf45b68dacfb4ec7563597a902edbaf3c59b50d5bcadd93b1", "https://deno.land/std@0.172.0/path/win32.ts": "672942919dd66ce1b8c224e77d3447e2ad8254eaff13fe6946420a9f78fa141e", - "https://deno.land/std@0.218.2/assert/assert.ts": "bec068b2fccdd434c138a555b19a2c2393b71dfaada02b7d568a01541e67cdc5", - "https://deno.land/std@0.218.2/assert/assertion_error.ts": "9f689a101ee586c4ce92f52fa7ddd362e86434ffdf1f848e45987dc7689976b8", - "https://deno.land/std@0.218.2/async/_util.ts": "64f8d0da48f908c651736317a36e97890ea9ea22a5e281c8acf9ee6cafbb6de9", - "https://deno.land/std@0.218.2/async/abortable.ts": "87bbc086dfdbb3556fe47d12038dc3919b4f717b3660abb21f800e020c159664", - "https://deno.land/std@0.218.2/async/deadline.ts": "65cf43eb30948f5122fa66a0d6c0fc166af23621632b6e440303efe164e8a0db", - "https://deno.land/std@0.218.2/async/debounce.ts": "025a8e1a7c73e477f0c88a48f826fcdb022b04812ad7fc2f54de756ab4ed7624", - "https://deno.land/std@0.218.2/async/delay.ts": "8e1d18fe8b28ff95885e2bc54eccec1713f57f756053576d8228e6ca110793ad", - "https://deno.land/std@0.218.2/async/mod.ts": "02ba6a7d065481b6318d0783ad7243426e4f1bc4cdee0d1b6460bf73e9eab3e3", - "https://deno.land/std@0.218.2/async/mux_async_iterator.ts": "33aa184f27fd2be1467dab3785dfdf525c8b8acd150a049331f0fdf927a17dda", - "https://deno.land/std@0.218.2/async/pool.ts": "2b972e3643444b73f6a8bcdd19799a2d0821b28a45fbe47fd333223eb84327f0", - "https://deno.land/std@0.218.2/async/retry.ts": "718335f89d1fbd2156109b05d2399a2b47c848a5a8f7237640538902526b38f7", - "https://deno.land/std@0.218.2/async/tee.ts": "34373c58950b7ac5950632dc8c9908076abeefcc9032d6299fff92194c284fbd", - "https://deno.land/std@0.218.2/bytes/concat.ts": "9cac3b4376afbef98ff03588eb3cf948e0d1eb6c27cfe81a7651ab6dd3adc54a", - "https://deno.land/std@0.218.2/crypto/_fnv/fnv32.ts": "ba2c5ef976b9f047d7ce2d33dfe18671afc75154bcf20ef89d932b2fe8820535", - "https://deno.land/std@0.218.2/crypto/_fnv/fnv64.ts": "580cadfe2ff333fe253d15df450f927c8ac7e408b704547be26aab41b5772558", - "https://deno.land/std@0.218.2/crypto/_fnv/mod.ts": "8dbb60f062a6e77b82f7a62ac11fabfba52c3cd408c21916b130d8f57a880f96", - "https://deno.land/std@0.218.2/crypto/_fnv/util.ts": "27b36ce3440d0a180af6bf1cfc2c326f68823288540a354dc1d636b781b9b75f", - "https://deno.land/std@0.218.2/crypto/_wasm/lib/deno_std_wasm_crypto.generated.mjs": "76c727912539737def4549bb62a96897f37eb334b979f49c57b8af7a1617635e", - "https://deno.land/std@0.218.2/crypto/_wasm/mod.ts": "c55f91473846827f077dfd7e5fc6e2726dee5003b6a5747610707cdc638a22ba", - "https://deno.land/std@0.218.2/crypto/crypto.ts": "b7bacddd990ad4193556697b88c7291ed61edc145fafbc060559984f3e8e4977", - "https://deno.land/std@0.218.2/uuid/_common.ts": "05c787c5735776c4e48e30294878332c39cb7738f50b209df4eb9f2b0facce4d", - "https://deno.land/std@0.218.2/uuid/constants.ts": "eb6c96871e968adf3355507d7ae79adce71525fd6c1ca55c51d32ace0196d64e", - "https://deno.land/std@0.218.2/uuid/mod.ts": "b9cb1cf73c3d87e15817486df7e885a63b74a7384768a927c708ccd045fcbf78", - "https://deno.land/std@0.218.2/uuid/v1.ts": "a089755e9ba5a172f3d568b617164d4692bd71e548b018e039d1bcb17d4f1bb6", - "https://deno.land/std@0.218.2/uuid/v3.ts": "aff081baee55498ed5804d006735a77b252ac1645e3b418058807218371de577", - "https://deno.land/std@0.218.2/uuid/v4.ts": "1319a2eeff7259adda416ec5f7997ded80d3165ef0787012793fc8621c18c493", - "https://deno.land/std@0.218.2/uuid/v5.ts": "f6771dc89e89f26e74a9b51d25d6b711c27d2ddf3a3650312dd46e7edfe2491e", + "https://deno.land/std@0.224.0/assert/assert.ts": "09d30564c09de846855b7b071e62b5974b001bb72a4b797958fe0660e7849834", + "https://deno.land/std@0.224.0/assert/assertion_error.ts": "ba8752bd27ebc51f723702fac2f54d3e94447598f54264a6653d6413738a8917", + "https://deno.land/std@0.224.0/async/_util.ts": "3e94e674c974c5c9277f6b3ba2d4e8403c320ba5cebb891f50afa3af4b8e0ac9", + "https://deno.land/std@0.224.0/async/abortable.ts": "ea6ddb98c1c6f066d5b26c8fd030e2d5afa54571182674aa07929c39dfa8c5b2", + "https://deno.land/std@0.224.0/async/deadline.ts": "008929d69b1efdd11b1fa55784bb4882add6adf8722868b26e87f68e35efc573", + "https://deno.land/std@0.224.0/async/debounce.ts": "e7bcccb17b6fe34646ac5d37c2a6d3830ca049e7e3534d2aace4001596f42eff", + "https://deno.land/std@0.224.0/async/delay.ts": "f90dd685b97c2f142b8069082993e437b1602b8e2561134827eeb7c12b95c499", + "https://deno.land/std@0.224.0/async/mod.ts": "ae2b6869ad7563f825e037e02c263507da4a55563b4a0bcd9079d2c3eb2670a2", + "https://deno.land/std@0.224.0/async/mux_async_iterator.ts": "8d960e951c7bf6cb682522c2ebf5bd3e738b27c6a7ac9500ab64d27054930972", + "https://deno.land/std@0.224.0/async/pool.ts": "2b972e3643444b73f6a8bcdd19799a2d0821b28a45fbe47fd333223eb84327f0", + "https://deno.land/std@0.224.0/async/retry.ts": "29025b09259c22123c599b8c957aeff2755854272954776dc9a5846c72ea4cfe", + "https://deno.land/std@0.224.0/async/tee.ts": "34373c58950b7ac5950632dc8c9908076abeefcc9032d6299fff92194c284fbd", + "https://deno.land/std@0.224.0/bytes/concat.ts": "86161274b5546a02bdb3154652418efe7af8c9310e8d54107a68aaa148e0f5ed", + "https://deno.land/std@0.224.0/crypto/_wasm/lib/deno_std_wasm_crypto.generated.mjs": "7cd490ae1553c97459bd02de4c3f0a552768a85621949b2366003f3cf84b99d7", + "https://deno.land/std@0.224.0/crypto/_wasm/mod.ts": "e89fbbc3c4722602ff975dd85f18273c7741ec766a9b68f6de4fd1d9876409f8", + "https://deno.land/std@0.224.0/crypto/crypto.ts": "e58d78f3db111a499261dbab037ec78cc89da0516a50e1f0205665980a3417e3", + "https://deno.land/std@0.224.0/fs/_get_file_info_type.ts": "da7bec18a7661dba360a1db475b826b18977582ce6fc9b25f3d4ee0403fe8cbd", + "https://deno.land/std@0.224.0/fs/_is_subdir.ts": "c68b309d46cc8568ed83c000f608a61bbdba0943b7524e7a30f9e450cf67eecd", + "https://deno.land/std@0.224.0/fs/_to_path_string.ts": "29bfc9c6c112254961d75cbf6ba814d6de5349767818eb93090cecfa9665591e", + "https://deno.land/std@0.224.0/fs/copy.ts": "7ab12a16adb65d155d4943c88081ca16ce3b0b5acada64c1ce93800653678039", + "https://deno.land/std@0.224.0/fs/ensure_dir.ts": "51a6279016c65d2985f8803c848e2888e206d1b510686a509fa7cc34ce59d29f", + "https://deno.land/std@0.224.0/path/_common/assert_path.ts": "dbdd757a465b690b2cc72fc5fb7698c51507dec6bfafce4ca500c46b76ff7bd8", + "https://deno.land/std@0.224.0/path/_common/basename.ts": "569744855bc8445f3a56087fd2aed56bdad39da971a8d92b138c9913aecc5fa2", + "https://deno.land/std@0.224.0/path/_common/constants.ts": "dc5f8057159f4b48cd304eb3027e42f1148cf4df1fb4240774d3492b5d12ac0c", + "https://deno.land/std@0.224.0/path/_common/from_file_url.ts": "d672bdeebc11bf80e99bf266f886c70963107bdd31134c4e249eef51133ceccf", + "https://deno.land/std@0.224.0/path/_common/normalize.ts": "684df4aa71a04bbcc346c692c8485594fc8a90b9408dfbc26ff32cf3e0c98cc8", + "https://deno.land/std@0.224.0/path/_common/normalize_string.ts": "33edef773c2a8e242761f731adeb2bd6d683e9c69e4e3d0092985bede74f4ac3", + "https://deno.land/std@0.224.0/path/_common/strip_trailing_separators.ts": "7024a93447efcdcfeaa9339a98fa63ef9d53de363f1fbe9858970f1bba02655a", + "https://deno.land/std@0.224.0/path/_os.ts": "8fb9b90fb6b753bd8c77cfd8a33c2ff6c5f5bc185f50de8ca4ac6a05710b2c15", + "https://deno.land/std@0.224.0/path/basename.ts": "7ee495c2d1ee516ffff48fb9a93267ba928b5a3486b550be73071bc14f8cc63e", + "https://deno.land/std@0.224.0/path/constants.ts": "0c206169ca104938ede9da48ac952de288f23343304a1c3cb6ec7625e7325f36", + "https://deno.land/std@0.224.0/path/from_file_url.ts": "911833ae4fd10a1c84f6271f36151ab785955849117dc48c6e43b929504ee069", + "https://deno.land/std@0.224.0/path/join.ts": "ae2ec5ca44c7e84a235fd532e4a0116bfb1f2368b394db1c4fb75e3c0f26a33a", + "https://deno.land/std@0.224.0/path/posix/_util.ts": "1e3937da30f080bfc99fe45d7ed23c47dd8585c5e473b2d771380d3a6937cf9d", + "https://deno.land/std@0.224.0/path/posix/basename.ts": "d2fa5fbbb1c5a3ab8b9326458a8d4ceac77580961b3739cd5bfd1d3541a3e5f0", + "https://deno.land/std@0.224.0/path/posix/from_file_url.ts": "951aee3a2c46fd0ed488899d024c6352b59154c70552e90885ed0c2ab699bc40", + "https://deno.land/std@0.224.0/path/posix/join.ts": "7fc2cb3716aa1b863e990baf30b101d768db479e70b7313b4866a088db016f63", + "https://deno.land/std@0.224.0/path/posix/normalize.ts": "baeb49816a8299f90a0237d214cef46f00ba3e95c0d2ceb74205a6a584b58a91", + "https://deno.land/std@0.224.0/path/posix/resolve.ts": "08b699cfeee10cb6857ccab38fa4b2ec703b0ea33e8e69964f29d02a2d5257cf", + "https://deno.land/std@0.224.0/path/resolve.ts": "a6f977bdb4272e79d8d0ed4333e3d71367cc3926acf15ac271f1d059c8494d8d", + "https://deno.land/std@0.224.0/path/windows/_util.ts": "d5f47363e5293fced22c984550d5e70e98e266cc3f31769e1710511803d04808", + "https://deno.land/std@0.224.0/path/windows/basename.ts": "6bbc57bac9df2cec43288c8c5334919418d784243a00bc10de67d392ab36d660", + "https://deno.land/std@0.224.0/path/windows/from_file_url.ts": "ced2d587b6dff18f963f269d745c4a599cf82b0c4007356bd957cb4cb52efc01", + "https://deno.land/std@0.224.0/path/windows/join.ts": "8d03530ab89195185103b7da9dfc6327af13eabdcd44c7c63e42e27808f50ecf", + "https://deno.land/std@0.224.0/path/windows/normalize.ts": "78126170ab917f0ca355a9af9e65ad6bfa5be14d574c5fb09bb1920f52577780", + "https://deno.land/std@0.224.0/path/windows/resolve.ts": "8dae1dadfed9d46ff46cc337c9525c0c7d959fb400a6308f34595c45bdca1972", + "https://deno.land/std@0.224.0/uuid/_common.ts": "05c787c5735776c4e48e30294878332c39cb7738f50b209df4eb9f2b0facce4d", + "https://deno.land/std@0.224.0/uuid/constants.ts": "eb6c96871e968adf3355507d7ae79adce71525fd6c1ca55c51d32ace0196d64e", + "https://deno.land/std@0.224.0/uuid/mod.ts": "cefc8e2f77d9e493739c8dc4ec141b12b855414bf757e778bf9b00f783506b76", + "https://deno.land/std@0.224.0/uuid/v1.ts": "cc45e7eb1d463d7d38b21a3c6e4de55ff98598ca442309321575fe841b323a54", + "https://deno.land/std@0.224.0/uuid/v3.ts": "689f2d64a9460a75877a2eed94662d9cb31bedb890d72fce0d161ef47d66cc26", + "https://deno.land/std@0.224.0/uuid/v4.ts": "1319a2eeff7259adda416ec5f7997ded80d3165ef0787012793fc8621c18c493", + "https://deno.land/std@0.224.0/uuid/v5.ts": "75f76d9e53583572fe3d4893168530986222d439b1545b56d4493c6d5d1cd81d", "https://deno.land/std@0.91.0/encoding/base64.ts": "eecae390f1f1d1cae6f6c6d732ede5276bf4b9cd29b1d281678c054dc5cc009e", "https://deno.land/std@0.91.0/encoding/hex.ts": "f952e0727bddb3b2fd2e6889d104eacbd62e92091f540ebd6459317a61932d9b", "https://deno.land/std@0.91.0/hash/_wasm/hash.ts": "cb6ad1ab429f8ac9d6eae48f3286e08236d662e1a2e5cfd681ba1c0f17375895", "https://deno.land/std@0.91.0/hash/_wasm/wasm.js": "94b1b997ae6fb4e6d2156bcea8f79cfcd1e512a91252b08800a92071e5e84e1a", - "https://deno.land/std@0.91.0/hash/hasher.ts": "57a9ec05dd48a9eceed319ac53463d9873490feea3832d58679df6eec51c176b", "https://deno.land/std@0.91.0/hash/mod.ts": "5d032bd34186cda2f8d17fc122d621430953a6030d4b3f11172004715e3e2441", + "https://deno.land/x/caller_metadata@v0.0.3/src/main.ts": "2258e7eb41a54ac9a99058f40a487c6313232feafcb5f02f22d51fe6b526dd73", "https://deno.land/x/exec@0.0.5/mod.ts": "2a71f7e23e25be883275b22d872bbd2c3dfa3058934f1f156c8663fb81f5894f", "https://deno.land/x/get_ip@v2.0.0/mod.ts": "8a967dece886ddf7f7bf6c8d6000c98a4ff374834a90a4e0fde868ce4301873d", "https://deno.land/x/input@2.0.3/history.ts": "3cad3fee1e2f86d4202ca6ab49f1a5609aff2323ff762cc637e1da3edbef9608", @@ -64,17 +89,17 @@ "https://deno.land/x/reflect_metadata@v0.1.12/mod.ts": "8b5e3b20f1e604c118df433e84a409b1d5116e885001047134cc42ecf84fa2cd", "https://dev.cdn.unyt.org/unyt_core/VERSION.ts": "ca02332766074ec3a0dd1f6814c1f913e627076bc91d885025cf4681fbf12dcb", "https://dev.cdn.unyt.org/unyt_core/compiler/binary_codes.ts": "e88c1ac8145141eda3f14568efc1b984c5e769349c7153105419b342c3fe1f40", - "https://dev.cdn.unyt.org/unyt_core/compiler/compiler.ts": "85dbbf5258a77bcc6911647d6ca8ca4d12a5b052d26fa782b8a8da2034893e42", + "https://dev.cdn.unyt.org/unyt_core/compiler/compiler.ts": "c3ad1bacd75fd7a50b7636188158bd8c2395eafd1379328396187b3ec9805b55", "https://dev.cdn.unyt.org/unyt_core/compiler/protocol_types.ts": "e4d4dc21ed2d4a67cb251d92f1aff319f0193952f5b963e7d9e625a3983843cc", "https://dev.cdn.unyt.org/unyt_core/compiler/tokens_regex.ts": "632e87ca07cc0d56cfa8a2a9b77248a296c775f04c6b235060407adbdcc343ee", "https://dev.cdn.unyt.org/unyt_core/compiler/unit_codes.ts": "bc9b15229cab9913f9b01af749433e681dc2b5e0df552c109d9c4f17e66eb6e9", "https://dev.cdn.unyt.org/unyt_core/datex.ts": "71bb6d0b4d50df4678bc156480f58cf0b65389199ab24d8c4d6aabfb1e2e2b2f", "https://dev.cdn.unyt.org/unyt_core/datex_all.ts": "88c70810290283c8aa6d898d30629eb864acebb9be72e7f92ce9317ae845817c", - "https://dev.cdn.unyt.org/unyt_core/datex_short.ts": "a218074299715530ef9dcdacc809d01971499fd029f913a3080358e3ca640ea3", - "https://dev.cdn.unyt.org/unyt_core/functions.ts": "b380199a40811143d880b9330f5aa841015105e77f74c4ced1f98d8e1249c536", - "https://dev.cdn.unyt.org/unyt_core/init.ts": "761ab9deb3e3a6f2c60dde2cb37946df0379b957ff1323922e614d9fabcabf1a", - "https://dev.cdn.unyt.org/unyt_core/js_adapter/decorators.ts": "12fa32c0be8a55078e178750f82b4046c214b8a63c75f871361e396196597b1a", - "https://dev.cdn.unyt.org/unyt_core/js_adapter/js_class_adapter.ts": "bc58c32b650c81cda788846bdb2a6e0e62c1d9559168b7d3f554afd66d6d6550", + "https://dev.cdn.unyt.org/unyt_core/datex_short.ts": "c8c7ded24fc37656a0e70f29bc258cc96c0ca0c2617f713b7a8ae9e366017de8", + "https://dev.cdn.unyt.org/unyt_core/functions.ts": "280a1258437f0cb65710a17268e95f3cac9ee83980d16b8cc0dd383bf73e62c9", + "https://dev.cdn.unyt.org/unyt_core/init.ts": "428ffa715c1464f7dfbcfc0b086720161e1930ade9f90ee5ad66beb8e435d24e", + "https://dev.cdn.unyt.org/unyt_core/js_adapter/decorators.ts": "a5a8778a82d8f3b17ed27d697c8092d971a98806bbe811e0fbcd3a5914e995e3", + "https://dev.cdn.unyt.org/unyt_core/js_adapter/js_class_adapter.ts": "28ff29e2753d8b434931a920edfccee0a61c2f4e9416bfe828ec77d2f2ec5e0a", "https://dev.cdn.unyt.org/unyt_core/lib/localforage/drivers/indexeddb.js": "94a6cfe0f3bfbe677bae59127c991405d68c4c54ae4d3dbbdb56072a471ece02", "https://dev.cdn.unyt.org/unyt_core/lib/localforage/drivers/localstorage.js": "62f07f7063c3ff385f307b0cf84f8daf10e3896b623e720c39ee5daab68863d1", "https://dev.cdn.unyt.org/unyt_core/lib/localforage/drivers/websql.js": "5c69bdff0af1cbc3103feb76964490817e221091dc3ae573c8f9a94a78437609", @@ -94,89 +119,101 @@ "https://dev.cdn.unyt.org/unyt_core/lib/localforage/utils/serializer.js": "5960c40deb26d81ca4841c9fe829a5305ea09bfd3d9deb6c1d74875934923b03", "https://dev.cdn.unyt.org/unyt_core/lib/marked.js": "0a1fa6c8bbe5f691c8d5bed2041fdfb9e22494bd3caca2cbc076cb142c19b014", "https://dev.cdn.unyt.org/unyt_core/mod.ts": "6750bfd667dc516d8d14a9e27504f39e6da192f8ba7a938590db8922bcb918a8", - "https://dev.cdn.unyt.org/unyt_core/network/blockchain_adapter.ts": "91b5a6a5dff1c26783bbcb9a628c6e3577c877b64007fdf0166f5c081f5cdaf2", - "https://dev.cdn.unyt.org/unyt_core/network/communication-hub.ts": "874fb910c706944991b13e5047e3afc19ccc5536d13d38082c5e848347a7c578", - "https://dev.cdn.unyt.org/unyt_core/network/communication-interface.ts": "328a715e6710144911626c30a29ee56f62985760138ee8678e02816df77e0db9", + "https://dev.cdn.unyt.org/unyt_core/network/blockchain_adapter.ts": "2222c17db4ba8710718473ad61bb0b24981799e575eba13b7e74a15a5b711e89", + "https://dev.cdn.unyt.org/unyt_core/network/communication-hub.ts": "4c80d051dbabb69fa3ed519a466d9d4f216f6b4a8b05bbd5248d799db6615ac0", + "https://dev.cdn.unyt.org/unyt_core/network/communication-interface.ts": "9ce7f9d906e688ee31c247dba8496030e363b78326268200a6732efac7c0058c", "https://dev.cdn.unyt.org/unyt_core/network/communication-interfaces/local-loopback-interface.ts": "1f82bc942c0e2266a64827b8d8316562b2f7704de591c3f55153718967b4c45e", + "https://dev.cdn.unyt.org/unyt_core/network/communication-interfaces/webrtc-interface.ts": "ed3ac73c2fab5f96cbaf23deddd5629d90bd2e139e854e579c5c7f0e824dd3cb", "https://dev.cdn.unyt.org/unyt_core/network/communication-interfaces/websocket-client-interface.ts": "0552dbc5eaeaf042aeda9b0363dd9750925570bca91fdf0d49ab332ba724ba14", - "https://dev.cdn.unyt.org/unyt_core/network/communication-interfaces/websocket-interface.ts": "b0f9008f48c2805c293feb5ffbb991482d997539a96bac108d36bb98369bbc18", + "https://dev.cdn.unyt.org/unyt_core/network/communication-interfaces/websocket-interface.ts": "9cf5b4bc67bbd1640dfda5ad03e8f1a1d4d283afafd929ed906623a4130d927b", + "https://dev.cdn.unyt.org/unyt_core/network/communication-interfaces/window-interface.ts": "98af9526c035cbee15d4d516d68ef6d367c74f425bf9201f24f0f33d9e208192", "https://dev.cdn.unyt.org/unyt_core/network/datex-http-channel.ts": "9fdd46c55ae56b4e17300a3cf8139961687ba3fff4ad723e982cebd1cd6cc3a8", + "https://dev.cdn.unyt.org/unyt_core/network/network-interface.ts": "cda194ac74e8cb7778df6b4032df91da318da6aff929ecbe07a2ade8c1f97853", "https://dev.cdn.unyt.org/unyt_core/network/network_utils.ts": "ecf26e1ed96db973aa3c6fcb42db00645df9e5ebba9d39fa08113c9d834dd743", - "https://dev.cdn.unyt.org/unyt_core/network/supranet.ts": "24bf12529fd770e1af84a417dcf2a50b51e60b707fe5d263938a20f119c4228d", - "https://dev.cdn.unyt.org/unyt_core/network/unyt.ts": "b8b5798293cd129dc8d20383918b6fd5253ae06a82b1e16260c64766f78a34af", - "https://dev.cdn.unyt.org/unyt_core/runtime/cache_path.ts": "9ddd39ac5fe6b45e71189a4aad31b928a29552fc01134c4dca618001fe8d8d6e", + "https://dev.cdn.unyt.org/unyt_core/network/online-state.ts": "133186a036aa5820fef6e576563b79060c44e3e49ed6e25e4a7d09a1ddcaeb24", + "https://dev.cdn.unyt.org/unyt_core/network/supranet.ts": "8e4cef78a50ebc910fe0f802cb249f8e221e95894ee04a3c6a6c3c5bb140b9de", + "https://dev.cdn.unyt.org/unyt_core/network/unyt.ts": "2e74e4f6e97a57babadca21727556589614eb2eca6b06271c529c1a3c2ee4cb4", + "https://dev.cdn.unyt.org/unyt_core/runtime/cache_path.ts": "bb7de31ccefad9689ca78f2cd4cfe1829186617baa32d61e413680d9324b57b3", "https://dev.cdn.unyt.org/unyt_core/runtime/cli.ts": "a4811ce51a337057420b590d80c943d38c42a5a086198029ae40bfc850b2fbdc", - "https://dev.cdn.unyt.org/unyt_core/runtime/constants.ts": "3e19c703ead7de018bba03edf1b0f4a2d30ad274e7bc22630a3d7b2f743a755e", - "https://dev.cdn.unyt.org/unyt_core/runtime/crypto.ts": "4031650c358bcd0fc448e95ea248afa84442362a46048ffecbb20ce97aea3bb0", + "https://dev.cdn.unyt.org/unyt_core/runtime/constants.ts": "0c8c4ddde80eac9a39b7e10d9d9d18ad6861f0e8b1d75cb0b7a96fd1315b17dd", + "https://dev.cdn.unyt.org/unyt_core/runtime/crypto.ts": "429cdfe676a442c77f9dc9f1a7c08801ff03dee4667cfd5a3cd5246c9a715f18", "https://dev.cdn.unyt.org/unyt_core/runtime/debugger.ts": "ad1b12229b5d8749107e12a5b6c157b4d65e675e53c3182dc4c01cee75efe9b7", - "https://dev.cdn.unyt.org/unyt_core/runtime/display.ts": "40d5a5c19e32db7d16e11fc19acf786e3855d4823c506e1ac8567b9852a81753", - "https://dev.cdn.unyt.org/unyt_core/runtime/endpoint_config.ts": "28111e1919110712601f0d1ff8c1a1f230b532c84658b3688a1847ddfcfc7751", - "https://dev.cdn.unyt.org/unyt_core/runtime/io_handler.ts": "bc4b8d4613e6786535b4115f3113b7c45dbc5a0a37a65a0351590b825970169e", - "https://dev.cdn.unyt.org/unyt_core/runtime/js_interface.ts": "409a417f4da6670d36946189f0595c5c35b02f365c98c5437bedaba69a2e41e7", + "https://dev.cdn.unyt.org/unyt_core/runtime/display.ts": "3a9c46b047345db380920d01c642550a7aff029956e2c2d9d6f67d8a674f17c6", + "https://dev.cdn.unyt.org/unyt_core/runtime/endpoint_config.ts": "63d3073ffa68566cdeccc6d8bf7786b5ffc47c8ee30b4888cbf8c79f4d3e5665", + "https://dev.cdn.unyt.org/unyt_core/runtime/io_handler.ts": "21e66a38e704a38bf24bec1e483bbc8c23473292b130a036b6e6dae688735b1b", + "https://dev.cdn.unyt.org/unyt_core/runtime/js_interface.ts": "44d1b3882b802fbb13f287366dd3d6b1615665ac9f9301b495b61a47a2b7c7b7", "https://dev.cdn.unyt.org/unyt_core/runtime/lazy-pointer.ts": "8649bf4d88eb56949b3db1d1ed33e36e22946e2d0d8ed8a8dfe1ba4ea56a0365", "https://dev.cdn.unyt.org/unyt_core/runtime/performance_measure.ts": "ec6a72315c9e0924abbb9c9667edd1dfc28245d8b2b743332a8f0997f2fc6b4d", - "https://dev.cdn.unyt.org/unyt_core/runtime/pointers.ts": "a9da042e153ab5c20285c4c44c8bc99d6d973f517dda905d8593f2e1afa84800", - "https://dev.cdn.unyt.org/unyt_core/runtime/runtime.ts": "1c2cee40c363b5739b7f9b23014c20d42804a94178af6e642ecc5bd1431b756b", + "https://dev.cdn.unyt.org/unyt_core/runtime/pointers.ts": "097f2c7680b525079ad39292da091c7992d32d7b8df1057cffae42f1f1e6ebfc", + "https://dev.cdn.unyt.org/unyt_core/runtime/reset.ts": "68006e941ec1778933f29b1cc3a0b0dc79d5bbc503f1b40162f12f040a7d7612", + "https://dev.cdn.unyt.org/unyt_core/runtime/runtime.ts": "3b72b472e4709b2a8be8e6c0484b9217846dc5a36ba92e79ed4bad688e1b7882", "https://dev.cdn.unyt.org/unyt_core/storage/storage-locations/deno-kv.ts": "c457114a9fda19acb73e4e7f9f6e92abec9ecc30b255e39ca487233c6555acf7", "https://dev.cdn.unyt.org/unyt_core/storage/storage-locations/indexed-db.ts": "e54277691028cf3fbd1682900d98f4b583184fe62a6d3bddf338422f0f42515c", "https://dev.cdn.unyt.org/unyt_core/storage/storage-locations/local-storage-compat.ts": "d49a9ef4a95bf875c213964681362d896545116a3859141859222f342475eb79", "https://dev.cdn.unyt.org/unyt_core/storage/storage-locations/local-storage.ts": "8abad8288c38090e2b615bb83b06300a397aba86d3481f8ef7f5468e0f87eb0c", - "https://dev.cdn.unyt.org/unyt_core/storage/storage.ts": "ef969269a0a832cbe77c664837669350d5647526db044f7d4cd0c673cecf1d51", - "https://dev.cdn.unyt.org/unyt_core/types/abstract_types.ts": "04ad3192b3b0973fdf28f730dd6f16feb63b6d9ea0963e280d035889d8abb0c8", - "https://dev.cdn.unyt.org/unyt_core/types/addressing.ts": "522d4ae05b67f9a2da7601eb6cf22401268bea013a5471ce804ccbedccaf714f", + "https://dev.cdn.unyt.org/unyt_core/storage/storage.ts": "29abe2fc660eb5ce4d1040ef829bca4f20033b00c86ebd603f2be842dc913313", + "https://dev.cdn.unyt.org/unyt_core/types/addressing.ts": "62c8a48b84237409780faf7601daf9abbf581036005218f3b9b60cc66e89cbf4", "https://dev.cdn.unyt.org/unyt_core/types/assertion.ts": "ffa9614cf4ad904df1007d0a6441788eceb3347d26791771372c17b9249ea2ce", "https://dev.cdn.unyt.org/unyt_core/types/deferred.ts": "eb308e59468c7ca5bd3179e4aa86b5c73d7c4de60d1d656a6c244e3414e61e0d", "https://dev.cdn.unyt.org/unyt_core/types/error_codes.ts": "e4c8c23a07a5e71c70a6c2769de73f45547a4086152155b84ac080699bfb02ea", "https://dev.cdn.unyt.org/unyt_core/types/errors.ts": "2a847093e64705a8837be14f93f2810f4438953edc80a653a1b56f66aea8757d", - "https://dev.cdn.unyt.org/unyt_core/types/function-utils.ts": "c8040eb9ed67fd3af9bda5966693d815243845948c07100f553a4192c71ec36e", - "https://dev.cdn.unyt.org/unyt_core/types/function.ts": "facbfd050df3c73076e9f280f0a00c48f63eb631c665e6f52738e0dfb286b81a", + "https://dev.cdn.unyt.org/unyt_core/types/function-utils.ts": "1be5dc21dd460fa03f0a28fad84eee212becb220073b0f0c1e338490cec5defa", + "https://dev.cdn.unyt.org/unyt_core/types/function.ts": "2f4443d71ed36493f3709fc60c7d3aa6350cce2f7896e51eae4c285529bee217", "https://dev.cdn.unyt.org/unyt_core/types/iterator.ts": "f59386c4730f877f7ebd5a9ea77c6a89c6b8e6523a530f27d8a31175015da020", - "https://dev.cdn.unyt.org/unyt_core/types/js-function.ts": "b3300910934f617671c972dacb37f80a38590ce91505ba70e7b2cb8dc994f480", + "https://dev.cdn.unyt.org/unyt_core/types/js-function.ts": "7aca0693c88bd11702184af2533eb52d8a5d3fa93be8f3293b7f146576603a3c", "https://dev.cdn.unyt.org/unyt_core/types/logic.ts": "b60c9935897986cb4a7c39f53cd2e9c1f7daa8093c344cc69ab09ffd5527bd1a", "https://dev.cdn.unyt.org/unyt_core/types/markdown.ts": "7c762ad0c64eea3495f13371551424d971cc3d98eba09464f07f5ac3c1417fb6", - "https://dev.cdn.unyt.org/unyt_core/types/native_types.ts": "755e789956828afa344b4ed7846a36ff383cc8e1549fa79b822cec42b440569d", + "https://dev.cdn.unyt.org/unyt_core/types/native_types.ts": "179390b28499356e0b59243bc44e52e8bdd3892af4784b31aee3b14852999e39", "https://dev.cdn.unyt.org/unyt_core/types/object.ts": "9711f523ae51ebf53f992837e921dbe7b572f4150a1379a6ac6d0b240ed978dd", - "https://dev.cdn.unyt.org/unyt_core/types/quantity.ts": "27746882b141fb5f007219788ce839429c99c52d35bda7b44af9131e4955358d", + "https://dev.cdn.unyt.org/unyt_core/types/quantity.ts": "669aab9de6caf6179ba7d6e1fec8bfec16e0194638a3014bcd88f1d2a59ca53a", "https://dev.cdn.unyt.org/unyt_core/types/reactive-methods/array.ts": "e8877a11e173e5cf73789d76e517106d2ce4573daac315020053790f3ebb21ce", "https://dev.cdn.unyt.org/unyt_core/types/reactive-methods/map.ts": "8081929ad6d509615dda1978af6625c1e74aa4d7bee089981ce8a296c0a65eff", "https://dev.cdn.unyt.org/unyt_core/types/scope.ts": "4bf20285be521b142c735cfedf779a68a4315940f4b4deeb4d1f2ca3d322d7eb", - "https://dev.cdn.unyt.org/unyt_core/types/storage-map.ts": "4bb69f620093ce37433f8f1eb18035cffa4241b51e0ea0665ba08cd474d4f2dc", - "https://dev.cdn.unyt.org/unyt_core/types/storage-set.ts": "71512eb797916ef3a98c5e101bf7befdcbd3fda91c4d189d6044bd69d49b4d07", + "https://dev.cdn.unyt.org/unyt_core/types/storage-map.ts": "fdfb0b97d64865b2a223efb891093a882fedd46abae34cc30622c91966937ccd", + "https://dev.cdn.unyt.org/unyt_core/types/storage-set.ts": "05f3fcdb0df4569c9e17a59342cc69a70a597622aaacc0a25f2fa26edd5fa8ba", "https://dev.cdn.unyt.org/unyt_core/types/storage_set.ts": "0c2ff5e667891461079c2fd50533873ace42d8351fa60a640e39735ea76b7b02", - "https://dev.cdn.unyt.org/unyt_core/types/stream.ts": "fe705222631bf8abad019fe7c4f2edf5030dd7459e340038e671d021c5ba27ff", - "https://dev.cdn.unyt.org/unyt_core/types/struct.ts": "813a096287e42f16489bf1bb243984bfa9659157d51301f1a626a74be61912ed", + "https://dev.cdn.unyt.org/unyt_core/types/stream.ts": "c9c9413776f111556a7d6d2be27f64c26f027d3f7d09bee8d95b516d3ca0adff", + "https://dev.cdn.unyt.org/unyt_core/types/struct.ts": "eb3e6d572f252c32aa27acab902df095a3d1427794efbe627e4778b82b2da86f", "https://dev.cdn.unyt.org/unyt_core/types/task.ts": "b5073e9682ca9cb93b44646e0fbe7896f96cd369d151a94d2d11728e5d7e2da9", - "https://dev.cdn.unyt.org/unyt_core/types/time.ts": "c5d768dcf1f22c5e99929654a47ee800e79a92bca5d60d046b23d646e0ff3f8b", + "https://dev.cdn.unyt.org/unyt_core/types/time.ts": "13ccb01766381406c007ace5b26598fdb4ecdeb3323a588dec32bcd847fe2dc9", "https://dev.cdn.unyt.org/unyt_core/types/tuple.ts": "798c635102d46f4c29903f82fb50cd1f6c0449eb0090654ae08e59e9ae98a80e", - "https://dev.cdn.unyt.org/unyt_core/types/type.ts": "02864b2b417e5e49d0bad1aecec6ca2a8024bf7040b12845c304e70161208afb", - "https://dev.cdn.unyt.org/unyt_core/utils/_command_line_options.ts": "b6ac395803c34fc69c59a853769d44770bda4da6bad6904ae65fbdca6ccbe308", + "https://dev.cdn.unyt.org/unyt_core/types/type.ts": "208352fdac9a5ad57a238475ce009bfd8aa4e9dfe87f7757303637bcaf4713a0", "https://dev.cdn.unyt.org/unyt_core/utils/ansi_compat.ts": "d220d4387c98d6bb1fb2422b7e40f5bbdcfcf7f198f087460ab27712020baed8", - "https://dev.cdn.unyt.org/unyt_core/utils/args.ts": "35d5e9750935fd60519180deb3c6e99f7f4fe639258b12a68ce2999e1aee9fbb", - "https://dev.cdn.unyt.org/unyt_core/utils/auto_map.ts": "030496a476b889b43fad9b12d3b04d7f9e87b901766f5615066726bdcb1dd93f", + "https://dev.cdn.unyt.org/unyt_core/utils/args.ts": "3e11e2208bb3d1505140fe81ae535f5485d1c03f72c711de0c8903addc2120bd", + "https://dev.cdn.unyt.org/unyt_core/utils/auto_map.ts": "f417f528db567ac25a6d8614eaa27e8597131f262db0d6c9baa8e549453ddbe6", "https://dev.cdn.unyt.org/unyt_core/utils/caller_metadata.ts": "57f79bff64cfb24749cdebec41230a416a32e8d0e80c319096e653778e7a5fde", + "https://dev.cdn.unyt.org/unyt_core/utils/command-line-args/ansi.ts": "945dfe165cba067b9cc0035c454dc4991c8af826404573938848f84f79a97ba5", + "https://dev.cdn.unyt.org/unyt_core/utils/command-line-args/generators/cli-generator.ts": "39a570056a82da1ba983057b317d9baae7a0a4b715bd6738c977ea6f1991512b", + "https://dev.cdn.unyt.org/unyt_core/utils/command-line-args/generators/markdown-generator.ts": "47a39eede55a20f1431634525786b58ae93f4df0e751de301de5d0c35bfdc72f", + "https://dev.cdn.unyt.org/unyt_core/utils/command-line-args/main.ts": "7a0303af851b5acd29114d00877aeb58769c5a37d529800fd28132d823e474d2", + "https://dev.cdn.unyt.org/unyt_core/utils/command-line-args/types.ts": "f1bd9b69321731bcb5e53d26e895a9310b8c82ca821364545a0f9336463d6bc8", "https://dev.cdn.unyt.org/unyt_core/utils/constants.ts": "a7992c1dbb4d28a22c52d02281564f0236508217cc1b7a035ff0e95988fa83cb", - "https://dev.cdn.unyt.org/unyt_core/utils/cookies.ts": "4875c1a2739c07ce9b6f4a72806fa67d2c414d1cb3b0cd8b3d71ce6df18b14ba", + "https://dev.cdn.unyt.org/unyt_core/utils/cookies.ts": "108ed3d74c0a7766e07e8d956069723b960d5bae33630e15e38b89acb23baf9f", "https://dev.cdn.unyt.org/unyt_core/utils/debug-cookie.ts": "81c2b9ec6a0503a967998273022f66f61a8eb160478c349843f4c80aaa201609", + "https://dev.cdn.unyt.org/unyt_core/utils/disposable-callback-handler.ts": "28ff7277ecbdf90dfa18a18bef6c8930a52f32bd3976243b1fef2361f093536b", + "https://dev.cdn.unyt.org/unyt_core/utils/error-handling.ts": "3b3c89db93c693acc47f52019fc47c6772220becce0e667844d7d82c285f3c9d", "https://dev.cdn.unyt.org/unyt_core/utils/error-reporting.ts": "6caaa2dd93dd445034e7aafc0b6a461010555225e21f0bd51b6a5cd64ec95599", "https://dev.cdn.unyt.org/unyt_core/utils/eternals.ts": "e765fdab5e147ae6d3508eb56069b227ca39a20e30e8eeeab82a88b568ef16c6", "https://dev.cdn.unyt.org/unyt_core/utils/format-endpoint-url.ts": "d581d10140d928f691abb59ffdb151277fd1adbaf1cef1a1af85876f6e2d464b", - "https://dev.cdn.unyt.org/unyt_core/utils/global_types.ts": "aede78063eb8d9897ecd0f214d95b8144ea45891eb9611ec1f2e533bce7f30c6", - "https://dev.cdn.unyt.org/unyt_core/utils/global_values.ts": "d73efa6efa6b1d583e502c7a429c7ba313116a544082a0143d2caf98392ffaba", + "https://dev.cdn.unyt.org/unyt_core/utils/global_types.ts": "9bb80346c32fc0c6be741c78d0944bdf8c488f9aa4edba61122a12b343646196", + "https://dev.cdn.unyt.org/unyt_core/utils/global_values.ts": "56018cdee1ca67d01fdce014b7873024c70ba5f8dd7419a0293db0c150610200", + "https://dev.cdn.unyt.org/unyt_core/utils/indent.ts": "100a412fcc9625c625f85a672cd95d73dc2e10a23afd999cd1ce61e0e88cfcb0", + "https://dev.cdn.unyt.org/unyt_core/utils/interface-generator.ts": "e4aee848d8be6e0d802795df8ea77955e66cd4845e9ed06040551d3f75e3378e", "https://dev.cdn.unyt.org/unyt_core/utils/isolated-scope.ts": "814e356590f2dd8b9aae3d602da64f05fd0b905a5fb4dd49d93d22252015c1d5", "https://dev.cdn.unyt.org/unyt_core/utils/iterable-handler.ts": "2f442e8a4646db3dc8b96b256e8518ae57a88f0f98cd091d59559254b0c0a035", - "https://dev.cdn.unyt.org/unyt_core/utils/iterable-weak-map.ts": "0d01d06d3d6a5205cea478c468b0fce205aadf91b28f0194424f4a1e314552ff", - "https://dev.cdn.unyt.org/unyt_core/utils/iterable-weak-set.ts": "f5e27d4031e5836e0ce46371cb48b138d8c83ca0e1cebb5f8ec1f67f8648e7b0", + "https://dev.cdn.unyt.org/unyt_core/utils/iterable-weak-map.ts": "febd797ab2a16a13fe51e60afb9ae223d902dd5b8ec189fc2a086851570306e0", + "https://dev.cdn.unyt.org/unyt_core/utils/iterable-weak-set.ts": "d516716ff72b5605c5aaf4a057f27acedbed4be0e591c1ad9bfa52afcc268213", "https://dev.cdn.unyt.org/unyt_core/utils/local_files.ts": "ff90c8d3480bfdb0e6021bdb01aff189867017a3514595bd2303308274ebfcfc", - "https://dev.cdn.unyt.org/unyt_core/utils/logger.ts": "aa85166b56f2c8f521db05439e950c3debdb608b16a314936add9adceb8ecd59", - "https://dev.cdn.unyt.org/unyt_core/utils/match.ts": "ddeed31e9a591fc12bba7ced3d7cc2b749aa7e0ad4b81bf3b286e1484d423f9a", - "https://dev.cdn.unyt.org/unyt_core/utils/message_logger.ts": "ddeb69833dead38d5504117149dafa8d33f2b7828df9f3415c958737a3fdb71e", + "https://dev.cdn.unyt.org/unyt_core/utils/logger.ts": "9a4ba3372f24407341c2975d802c37dfdbcd84239b996288075e32cf764cec0d", + "https://dev.cdn.unyt.org/unyt_core/utils/match.ts": "ed083ebca2ee90bc265750a02197be55c8715af1269105fbb329b367aa26c394", + "https://dev.cdn.unyt.org/unyt_core/utils/message_logger.ts": "fa0eb1777c5b7d50384ed3c34516027a8c49c03035edaa93303c9c904e1a4c80", "https://dev.cdn.unyt.org/unyt_core/utils/normalize-path.ts": "1f3fea7a349600fec02ea434fd18ef34476e59f977c09a00146702e71e92d342", "https://dev.cdn.unyt.org/unyt_core/utils/observers.ts": "df82f2dfbc0189d7f3f6d1f21461ac4a329a6dbbd7ec1bd72d3cd2f5afada9a6", "https://dev.cdn.unyt.org/unyt_core/utils/path.ts": "1a6a91c4d36b2e8249bfc6f2341ca07e397cad256d96f2f8a285871dd3d47147", "https://dev.cdn.unyt.org/unyt_core/utils/persistent-listeners.ts": "8ab5b2750c8104caae00e275a33655868afbc76eee3807523d09f93315509898", - "https://dev.cdn.unyt.org/unyt_core/utils/polyfills.ts": "93b7f96ca03cf99df0081368e101b90dc36755352de798687668c83d4dd90002", + "https://dev.cdn.unyt.org/unyt_core/utils/polyfills.ts": "6cc15f4d5f3e5e2ec5a6911beae99732c9ee2886e9e1438212934f8f9674d637", "https://dev.cdn.unyt.org/unyt_core/utils/promises.ts": "b43d38724a2fd42d1d0a3cc479bc21f7a75ccee872f680c46f50e99839e17803", - "https://dev.cdn.unyt.org/unyt_core/utils/sha256.ts": "52bfee6d3276da5fad865886a323198827c2c86e6491711f4875c5fa50be2248", + "https://dev.cdn.unyt.org/unyt_core/utils/sha256.ts": "3bec80dff6e4663e9a1d9153288873c5dbe1aa40807f5a0412ff2057e628676b", "https://dev.cdn.unyt.org/unyt_core/utils/utils.ts": "4700b8f6dddbfb80d6465f437bf327c761f0b460fb59cbc92c129d2e35ef8714", "https://dev.cdn.unyt.org/unyt_core/utils/volatile-map.ts": "c60551a48b4900b8e29f0601c2d5cc9f9f0063b86e1d8a70c5a4369faa2b11cc", "https://dev.cdn.unyt.org/unyt_core/utils/weak-action.ts": "1b792c3992e54a72b0d3b89eccf5fbe8fa3b11f0b545f0551eaef1f0ae684345", diff --git a/importmap.dev.json b/importmap.dev.json index 5a5d005..c0b79bc 100644 --- a/importmap.dev.json +++ b/importmap.dev.json @@ -1,24 +1,11 @@ { - "imports": { - "unyt/": "https://dev.cdn.unyt.org/", - - "unyt_core": "https://dev.cdn.unyt.org/unyt_core/datex.ts", - "uix": "https://dev.cdn.unyt.org/uix/uix.ts", - - "unyt_core/": "https://dev.cdn.unyt.org/unyt_core/", - "uix/": "https://dev.cdn.unyt.org/uix/", - "uix_std/": "https://dev.cdn.unyt.org/uix/uix_std/", - "unyt_tests/": "https://dev.cdn.unyt.org/unyt_tests/", - "unyt_web/": "https://dev.cdn.unyt.org/unyt_web/", - "unyt_node/": "https://dev.cdn.unyt.org/unyt_node/", - "unyt_cli/": "https://dev.cdn.unyt.org/unyt_cli/", - - "supranet/": "https://portal.unyt.org/ts_module_resolver/", - - "uix/jsx-runtime": "https://dev.cdn.unyt.org/uix/jsx-runtime/jsx.ts", - - "backend/": "./backend/", - "common/": "./common/", - "frontend/": "./frontend/" - } + "imports": { + "datex-core-legacy": "https://dev.cdn.unyt.org/unyt_core/datex.ts", + "datex-core-legacy/": "https://dev.cdn.unyt.org/unyt_core/", + "unyt_core": "https://dev.cdn.unyt.org/unyt_core/datex.ts", + "unyt_core/": "https://dev.cdn.unyt.org/unyt_core/", + "uix": "https://dev.cdn.unyt.org/uix1/uix.ts", + "uix/": "https://dev.cdn.unyt.org/uix1/src/", + "uix/jsx-runtime": "https://dev.cdn.unyt.org/uix1/src/jsx-runtime/jsx.ts" + } } \ No newline at end of file diff --git a/main.ts b/main.ts index 88a3230..454f75b 100644 --- a/main.ts +++ b/main.ts @@ -1,898 +1,39 @@ -import { OutputMode, exec } from "https://deno.land/x/exec@0.0.5/mod.ts"; - -import { EndpointConfig } from "./endpoint-config.ts"; -import { Datex, property, sync } from "unyt_core"; +// deno-lint-ignore-file require-await +import { EndpointConfig } from "./src/endpoint-config.ts"; +import { Datex, property } from "unyt_core/datex.ts"; import { Class } from "unyt_core/utils/global_types.ts"; - -import { createHash } from "https://deno.land/std@0.91.0/hash/mod.ts"; -import { ESCAPE_SEQUENCES } from "unyt_core/utils/logger.ts"; -import { config } from "./config.ts"; - -import { getIP } from "https://deno.land/x/get_ip@v2.0.0/mod.ts"; -import { Path } from "unyt_core/utils/path.ts"; -import { formatEndpointURL } from "unyt_core/utils/format-endpoint-url.ts"; -const publicServerIP = await getIP({ipv6: false}); - - -const defaulTraefikToml = ` -[entryPoints] - [entryPoints.web] - address = ":80" - - [entryPoints.web-secure] - address = ":443" - -[api] - dashboard = true - -[providers.docker] - endpoint = "unix:///var/run/docker.sock" - exposedByDefault = false - network = "main" - -[certificatesresolvers.myhttpchallenge.acme] - caserver = "https://acme-v02.api.letsencrypt.org/directory" - email = "postmaster@unyt.org" - [certificatesresolvers.myhttpchallenge.acme.httpchallenge] - entrypoint = "web" -` - -const logger = new Datex.Logger("docker host"); - -logger.info("Config: ", config); - +import { config } from "./src/config.ts"; +import { Container} from "./src/container/Container.ts"; +import { RemoteImageContainer } from "./src/container/RemoteImageContainer.ts"; +import { UIXAppContainer, AdvancedUIXContainerOptions } from "./src/container/UIXAppContainer.ts"; +import { WorkbenchContainer } from "./src/container/WorkbenchContainer.ts"; +import { Endpoint, StorageMap } from "unyt_core/datex_all.ts"; +import { ContainerStatus } from "./src/container/Types.ts"; + +const logger = new Datex.Logger("Docker Host"); +logger.info("Starting up Docker Host with config:", config); await Datex.Supranet.connect(); -enum ContainerStatus { - STOPPED = 0, - STARTING = 1, - RUNNING = 2, - STOPPING = 3, - FAILED = 4, - INITIALIZING = 5, - ONLINE = 6 -} - -// parent class for all types of containers -@sync class Container { - - protected logger!:Datex.Logger; - #initialized = false; - - // docker container image + id - @property image!: string - @property container_name = "Container" - @property name = "Container" - @property id = '0'; - - network = "main" - - @property owner!: Datex.Endpoint - @property status: ContainerStatus = ContainerStatus.INITIALIZING; - @property errorMessage?: string - - #labels: string[] = [] - #ports: [number, number][] = [] - #env: Record = {} - #volumes: Record = {} - - debugPort: string|null = null - - get volumes() {return this.#volumes} - - addLabel(label: string) { - this.#labels.push(label) - } - - formatVolumeName(name: string) { - return name.replace(/[^a-zA-Z0-9_.-]/g, '-') - } - - async addVolume(name: string, path: string) { - await execCommand(`docker volume create ${name}`) - this.#volumes[name] = path; - } - - addVolumePath(hostPath: string, path: string) { - this.#volumes[hostPath] = path; - } - - addEnvironmentVariable(name: string, value: string) { - this.#env[name] = value; - } - - getFormattedLabels() { - return this.#labels.map(label => `--label ${label - .replaceAll('`', '\\`') - .replaceAll('(', '\\(') - .replaceAll(')', '\\)') - }`).join(" ") - } - getFormattedPorts() { - return this.#ports.map(ports => `-p ${ports[1]}:${ports[0]}`).join(" ") - } - getFormattedEnvVariables() { - return Object.entries(this.#env).map(([name, value]) => `--env ${name}=${value}`).join(" ") - } - getFormattedVolumes() { - return Object.entries(this.#volumes).map(([name, path]) => `-v ${name}:${path}`).join(" ") - } - - uniqueID(size = 4) { - return new Array(size).fill('xxxx').join('-').replace(/[xy]/g, function(c) { - const r = Math.random()*16|0, v = c == 'x' ? r : (r&0x3|0x8); - return v.toString(16); - }); - } - - construct(owner: Datex.Endpoint) { - this.owner = owner; - this.container_name = this.uniqueID(); - this.logger = new Datex.Logger(this); - } - replicate(){ - this.logger = new Datex.Logger(this); - this.#initialized = true; - this.updateAfterReplicate(); - } - - @property async start(){ - - // start => RUNNING or FAILED - const running = await this.handleStart(); - if (running) { - this.status = ContainerStatus.RUNNING; - - // get online state - const online = await this.handleOnline(); - if (online) this.status = ContainerStatus.ONLINE; - else this.status = ContainerStatus.FAILED; - } - else this.status = ContainerStatus.FAILED; - - - return running; - } - - // create docker for the first time - protected async init(){ - if (this.#initialized) return true; - - // INITIALIZING ... - this.status = ContainerStatus.INITIALIZING; - - // STOPPED (default state) or FAILED - const initialized = await this.handleInit(); - if (initialized) this.status = ContainerStatus.STOPPED; - else this.status = ContainerStatus.FAILED; - - this.#initialized = initialized; - - return initialized; - } - - - protected async handleInit(){ - try { - const restartPolicy = "always" - await execCommand(`docker run --network=${this.network}${this.debugPort ? ` -p ${this.debugPort}:9229`:''} --log-opt max-size=10m -d --restart ${restartPolicy} --name ${this.container_name} ${this.getFormattedPorts()} ${this.getFormattedVolumes()} ${this.getFormattedEnvVariables()} ${this.getFormattedLabels()} ${this.image}`) - } catch (e) { - console.log(e); - this.logger.error("error while creating container"); - return false; - } - return true; - } - - protected updateAfterReplicate(){ - // continue start/stop if in inbetween state - if (this.status == ContainerStatus.STARTING) this.start(); - else if (this.status == ContainerStatus.STOPPING) this.stop(); - } - - protected async handleStart(){ - // first init docker container (if not yet initialized) - if (!await this.init()) return false; - - this.logger.info("Starting Container " + this.container_name); - - await this.onBeforeStart(); - - if (this.status == ContainerStatus.FAILED) return false; - - // STARTING ... - this.status = ContainerStatus.STARTING; - - // start the container - try { - await execCommand(`docker container start ${this.container_name}`) - } catch (e) { - this.logger.error("error while starting container") - return false; - } - - await sleep(2000); - - // check if container is running - const running = await this.isRunning(); - if (running) { - this.logger.success("Container is running") - } - else { - this.logger.error("Container is not running") - return false; - } - - return true; +const ensureToken = (token?: string) => { + if (config.token && config.token.length && config.token !== token) { + logger.error(`Got request with invalid or missing access token "${token ?? "none"}"`); + throw new Error(`Invalid access token: The Docker Host ${Datex.Runtime.endpoint} requires authentication. Please make sure to set the HOST_TOKEN environment variable.`); } - - protected onBeforeStart() {} - protected onBeforeStop() {} - - protected async handleOnline() { - return false; - } - - @property async stop(force = true){ - this.logger.info("Stopping Container " + this.container_name); - - this.onBeforeStop(); - - this.status = ContainerStatus.STOPPING; - try { - await execCommand(`docker container ${force?'kill':'stop'} ${this.container_name}`) - } catch (e) { - this.logger.error("error while stopping container",e); - // TODO FAILED or RUNNING? - this.status = ContainerStatus.FAILED; - return false; - } - this.status = ContainerStatus.STOPPED; - return true; - } - - /** - * Get a stream of the container logs - * @param timeout timeout in minutes after which the stream will be closed - * @returns - */ - @property public getLogs(timeout = 60) { - const p = Deno.run({ - cmd: ['docker', 'logs', '--follow', this.container_name], - stdout: 'piped', - stderr: 'piped' - }) - const stream = $$(new Datex.Stream()); - stream.pipe(p.stdout.readable); - stream.pipe(p.stderr.readable); - - // close stream after timeout - setTimeout(() => { - // \u0004 is the EOT character - stream.write(new TextEncoder().encode("\n[Stream was closed after " + timeout + " minutes]\n\u0004").buffer); - stream.close(); - p.close(); - }, timeout * 60 * 1000); - - return stream; - } - - public async remove(){ - // remove from containers list - containers.getAuto(this.owner).delete(this); - - await Container.removeContainer(this.container_name); - await Container.removeImage(this.image); - - return true; - } - - private async isRunning(){ - try { - const ps = await execCommand(`docker ps | grep ${this.container_name}`); - return !!ps; - } - catch (e) { - console.log("err:",e) - return false - } - } - - exposePort(port:number, hostPort:number) { - this.#ports.push([port, hostPort]) - } - - protected enableTraefik(host: string, port?: number) { - const name = this.image + "-" + createHash("md5").update(host).toString() - const hasWildcard = host.startsWith('*.'); - - const hostRule = hasWildcard ? - `HostRegexp(\`{subhost:[a-z0-9-_]+}.${host.slice(2)}\`)` : - `Host(\`${host}\`)`; - - this.addLabel(`traefik.enable=true`); - this.addLabel(`traefik.http.routers.${name}.rule=${hostRule}`); - this.addLabel(`traefik.http.routers.${name}.entrypoints=web`); - // TODO: only workaoound for *.unyt.app domains: prio 1 - this.addLabel(`traefik.http.routers.${name}.priority=${hasWildcard ? 1 : 10}`); - this.addLabel(`traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https`); - this.addLabel(`traefik.http.routers.${name}.middlewares=redirect-to-https@docker`); - this.addLabel(`traefik.http.routers.${name}-secured.rule=${hostRule}`); - this.addLabel(`traefik.http.routers.${name}-secured.tls=true`); - // TODO: only workaoound for *.unyt.app domains: prio 1 - this.addLabel(`traefik.http.routers.${name}-secured.priority=${hasWildcard ? 1 : 10}`); - - if (hasWildcard) { - const rawHost = host.slice(2); - this.addLabel(`traefik.http.routers.${name}-secured.tls.domains[0].main=${rawHost}`); - this.addLabel(`traefik.http.routers.${name}-secured.tls.domains[0].sans=${host}`); - this.addLabel(`traefik.http.routers.${name}.tls.domains[0].main=${rawHost}`); - this.addLabel(`traefik.http.routers.${name}.tls.domains[0].sans=${host}`); - this.addLabel(`traefik.http.routers.${name}-secured.tls.certresolver=mydnschallenge`); - } else { - this.addLabel(`traefik.http.routers.${name}-secured.tls.certresolver=myhttpchallenge`); - } - - if (port) { - this.addLabel(`traefik.http.routers.${name}.service=${name}`); - this.addLabel(`traefik.http.routers.${name}-secured.service=${name}`); - this.addLabel(`traefik.http.services.${name}.loadbalancer.server.port=${port}`); - } - } - - public static async removeContainer(name: string) { - try { - await execCommand(`docker rm -f ${name}`) - } catch (e) { - logger.error("error while removing container",e); - return false; - } - } - - public static async removeImage(name: string) { - try { - await execCommand(`docker image rm -f ${name}`) - } catch (e) { - logger.error("error while removing container image",e); - return false; - } - } - -} - -@sync class WorkbenchContainer extends Container { - - @property config!: EndpointConfig - - override construct(owner: Datex.Endpoint, config: EndpointConfig) { - super.construct(owner) - this.config = config; - this.name = "unyt Workbench" - } - - // custom workbench container init - override async handleInit(){ - try { - const username = "user"; // TODO other usernames? - const config_exported = Datex.Runtime.valueToDatexString(this.config, true, true, true); - - this.image = 'unyt-workbench-' + this.config.endpoint.toString().replace(/[^A-Za-z0-9_-]/g,'').toLowerCase(); - - logger.info("image: " + this.image); - logger.info("config: " + config_exported); - logger.info("username: " + username); - - // create new config directory to copy to docker - const tmp_dir = `res/config-files-${new Date().getTime()}` - await execCommand(`cp -r ./res/config-files ${tmp_dir}`); - await Deno.writeTextFile(`${tmp_dir}/endpoint.dx`, config_exported) - - // create docker container - await execCommand(`docker build --build-arg username=${username} --build-arg configpath=${tmp_dir} -f ./res/Dockerfile -t ${this.image} .`) - - // remove tmp directory - await execCommand(`rm -r ${tmp_dir}`); - } - - catch (e) { - console.log(e) - this.logger.error("Error initializing workbench container"); - return false; - } - - return super.handleInit(); - } - - override async handleOnline() { - await sleep(6000); - try { - await Datex.Supranet.pingEndpoint(this.config.endpoint); - } - catch (e) { - this.logger.error("Workbench Endpoint not reachable") - await this.stop(); // stop container again for consistant container state - return false; - } - this.logger.success("Workbench Endpoint is reachable"); - return true; - } - } - -@sync class RemoteImageContainer extends Container { - - @property version?:string - @property url!:string - - construct(owner: Datex.Endpoint, url: string, version?: string) { - super.construct(owner) - this.version = version; - this.url = url; - this.name = url; - } - - // update docker image - @property async update(){ - try { - this.image = `${this.url}${this.version?':'+this.version:''}`; - await execCommand(`docker pull ${this.image}`); - } - - catch (e) { - console.log(e) - this.logger.error("Error initializing remote image container"); - return false; - } - return true; - } - - // custom workbench container init - override async handleInit(){ - if (!await this.update()) return false; - return super.handleInit(); - } - - // custom start - override async handleStart(){ - // always pull image first - if (!await this.update()) return false; - - return super.handleStart(); - } -} - - -@sync class UIXAppContainer extends Container { - - @property branch?:string - @property gitSSH!:string - @property gitHTTPS!:URL - @property stage!:string - @property domains!:Record // domain name -> internal port - @property endpoint!:Datex.Endpoint - @property advancedOptions?: AdvancedUIXContainerOptions - - @property args?: string[] - - // use v.0.1 - isVersion1 = false; - - static VALID_DOMAIN = /^(([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\-]*[a-zA-Z0-9])\.)*([A-Za-z0-9]|[A-Za-z0-9][A-Za-z0-9\-]*[A-Za-z0-9])$/ - - async construct(owner: Datex.Endpoint, endpoint: Datex.Endpoint, gitURL: string, branch?: string, stage = 'prod', domains?: Record, env?:string[], args?:string[], persistentVolumePaths?: string[], gitOAuthToken?: string, advancedOptions?: AdvancedUIXContainerOptions) { - super.construct(owner) - - // validate domains - for (const domain of Object.keys(domains ?? {})) { - if (!( - UIXAppContainer.VALID_DOMAIN.test(domain) || - (config.allowArbitraryDomains && domain.startsWith('*.')) - )) { - this.errorMessage = `Invalid domain name "${domain}". Only alphanumeric characters and dashes are allowed.`; - this.status = ContainerStatus.FAILED; - throw new Error(this.errorMessage); - } - } - - // convert from https url - if (gitURL.startsWith("https://")) { - this.gitHTTPS = new URL(gitURL); - this.gitSSH = `git@${this.gitHTTPS.host}:${this.gitHTTPS.pathname.slice(1)}`; - } - // convert from ssh url - else if (gitURL.startsWith("git@")) { - const [host, pathname] = gitURL.match(/git@([^:]*)+?:(.*)/i)?.slice(1) ?? []; - if (!host || !pathname) - throw new Error(`Invalid git URL '${gitURL}'!`); - this.gitSSH = gitURL; - this.gitHTTPS = new URL(`https://${host}/${pathname}`); - } - - // add gh token to URL - if (gitOAuthToken) { - this.gitHTTPS.username = "oauth2"; - this.gitHTTPS.password = gitOAuthToken; - } - - this.container_name = endpoint.name.toLowerCase() + (endpoint.name.endsWith(stage) ? '' : (stage ? '-' + stage : '')) - this.advancedOptions = advancedOptions; - - this.endpoint = endpoint; // TODO: what if @@local is passed - this.args = args; - - this.branch = branch; - this.stage = stage; - this.domains = domains ?? {}; - - // check if only unyt.app domains are used - if (!config.allowArbitraryDomains) { - for (const domain of Object.keys(this.domains)) { - if (!domain.endsWith('.unyt.app')) { - this.errorMessage = `Invalid domain "${domain}". Only unyt.app domains are allowed`; - this.status = ContainerStatus.FAILED; - throw new Error(this.errorMessage); - } - } - } - - this.isVersion1 = !! env?.includes("UIX_VERSION=0.1") - - if (this.isVersion1) console.log("using UIX v0.1") - - // inject environment variables - for (const envVar of env??[]) { - const [key, val] = envVar.split("="); - this.addEnvironmentVariable(key, val) - } - - // add persistent volumes - for (const path of persistentVolumePaths??[]) { - const mappedPath = path.startsWith("./") ? `/app${path.slice(1)}` : path; - const volumeName = this.formatVolumeName(this.container_name + '-persistent-' + (Object.keys(this.volumes).length)) - await this.addVolume(volumeName, mappedPath); - } - - } - - protected async handleNetwork() { - // make sure main network exists - await execCommand(`docker network inspect ${this.network} &>/dev/null || docker network create ${this.network}`) - - if (config.enableTraefik) { - // has traefik? - try { - await execCommand(`docker container ls | grep traefik`) - console.log("has traefik container"); - } - catch { - console.log("no traefik container detected, creating a new traefik container"); - const traefikDir = new Path("/etc/traefik/"); - - // init and start traefik container - if (!traefikDir.fs_exists) - await Deno.mkdir("/etc/traefik/", { recursive: true }); - - const traefikTomlPath = traefikDir.asDir().getChildPath("traefik.toml"); - const acmeJsonPath = traefikDir.asDir().getChildPath("acme.json"); - - await Deno.create(acmeJsonPath.normal_pathname) - await execCommand(`chmod 600 ${acmeJsonPath.normal_pathname}`) - await Deno.writeTextFile(traefikTomlPath.normal_pathname, defaulTraefikToml) - - const traefikContainer = new RemoteImageContainer(Datex.LOCAL_ENDPOINT, "traefik", "v2.5"); - traefikContainer.exposePort(80, config.hostPort) - traefikContainer.exposePort(443, 443) - traefikContainer.addVolumePath("/var/run/docker.sock", "/var/run/docker.sock") - traefikContainer.addVolumePath(traefikTomlPath.normal_pathname, "/etc/traefik/traefik.toml") - traefikContainer.addVolumePath(acmeJsonPath.normal_pathname, "/acme.json") - - traefikContainer.start(); - logger.error(traefikContainer) - // this.exposePort(80, 80); - } - } - } - - protected override async onBeforeStart() { - if (config.setDNSEntries) { - for (const domain of Object.keys(this.domains)) { - // currently only for unyt.app - if (domain.endsWith('.unyt.app')) { - this.logger.info("Setting DNS entry for " + domain + " to " + publicServerIP); - try { - await datex `@+unyt-dns-1.DNSManager.addARecord(${domain}, ${publicServerIP})` - this.logger.success("Successfully set DNS entry for " + domain); - } - catch (e) { - this.logger.error("Error setting DNS entry for " + domain); - this.errorMessage = `Could not set DNS entry for ${domain} (Internal error)`; - this.status = ContainerStatus.FAILED; - } - } - } - - } - } - - protected override async onBeforeStop() { - if (config.setDNSEntries) { - for (const domain of Object.keys(this.domains)) { - // currently only for unyt.app - if (domain.endsWith('.unyt.app')) { - this.logger.info("Removing DNS entry for " + domain); - try { - await datex `@+unyt-dns-1.DNSManager.removeARecord(${domain})` - this.logger.success("Successfully removed DNS entry for " + domain); - } - catch (e) { - this.logger.error("Error removing DNS entry for " + domain); - console.error(e) - } - } - } - } - } - - get orgName() { - return this.gitHTTPS.pathname.split("/").at(1)!; +@endpoint @entrypointProperty export class ContainerManager { + @property static async getContainers(token?: string): Promise> { + ensureToken(token); + return await containers.get(datex.meta!.caller) ?? new Set(); } - get repoName() { - return this.gitHTTPS.pathname.split('/').slice(2).join("/")!.replace('.git', ''); + @property static async getAllContainers(token?: string): Promise> { + ensureToken(token); + return this.getContainerList(); } - get gitOrigin() { - return ({ - "github.com": "GitHub", - "gitlab.com": "GitLab" - } as const)[this.gitHTTPS.hostname] ?? "GitLab"; - } - - get gitOriginURL() { - return new URL(`/${this.orgName}/${this.repoName}/`, this.gitHTTPS); - } - - // custom workbench container init - override async handleInit(){ - // setup network - await this.handleNetwork() - - // remove any existing previous container - const existingContainers = ContainerManager.findContainer({type: UIXAppContainer, properties: { - gitHTTPS: this.gitHTTPS, - stage: this.stage - }}) - for (const existingContainer of existingContainers) { - this.logger.error("removing existing container", existingContainer) - await existingContainer.remove() - } - - this.image = this.container_name - - try { - const domains = this.domains ?? [formatEndpointURL(this.endpoint)]; - - this.logger.info("image: " + this.image); - this.logger.info("repo: " + this.gitHTTPS + " / " + this.gitSSH); - this.logger.info("branch: " + this.branch); - this.logger.info("endpoint: " + this.endpoint); - this.logger.info("domains: " + Object.entries(domains).map(([d,p])=>`${d} (port ${p})`).join(", ")); - this.logger.info("advancedOptions: " + this.advancedOptions ? JSON.stringify(this.advancedOptions) : "-"); - - // clone repo - const dir = await Deno.makeTempDir({prefix:'uix-app-'}); - const dockerfilePath = `${dir}/Dockerfile`; - const repoPath = `${dir}/repo`; - let repoIsPublic = false; - - try { - // TODO add for GitLab - repoIsPublic = (await (await fetch(`https://api.github.com/repos/${this.orgName}/${this.repoName}`)).json()).visibility == "public" - } - catch {} - - // try clone with https first - try { - await execCommand(`git clone --depth 1 --single-branch --branch ${this.branch} --recurse-submodules ${this.gitHTTPS} ${repoPath}`, true) - } - catch (e) { - - Object.freeze(this.gitHTTPS); - // was probably a github token error, don't try ssh - if (this.gitHTTPS.username === "oauth2") { - this.errorMessage = `Could not clone git repository ${this.gitHTTPS}: Authentication failed.\nPlease make sure the ${this.gitOrigin} access token is valid and enables read access to the repository.`; - throw e; - } - - let sshKey: string|undefined; - try { - sshKey = await this.tryGetSSHKey(); - console.log("ssh public key: " + sshKey) - } - catch (e) { - console.log("Failed to generate ssh key: ", e) - } - - // try clone with ssh - try { - await execCommand(`git clone --depth 1 --recurse-submodules ${sshKey ? this.gitSSH.replace(this.gitHTTPS.hostname, this.uniqueGitHostName) : this.gitSSH} ${repoPath}`, true) - } - catch (e) { - console.log(e) - - let errorMessage = `Could not clone git repository ${this.gitSSH}. Please make sure the repository is accessible by ${Datex.Runtime.endpoint.main}. You can achieve this by doing one of the following:\n\n` - - let opt = 1; - const appendOption = (option: string) => { - errorMessage += `${opt++}. ${option}\n` - } - - if (!repoIsPublic) appendOption(`Make the repository publicly accessible (${this.gitOrigin === "GitHub" ? new URL(`./settings`, this.gitOriginURL).toString() : new URL("./edit", this.gitOriginURL).toString()})`); - appendOption(`Pass a ${this.gitOrigin} access token with --git-token= (Generate at ${this.gitOrigin === "GitHub" ? `https://github.com/settings/personal-access-tokens/new` : new URL(`./-/settings/access_tokens`, this.gitOriginURL)})`) - if (sshKey) appendOption(`Add the following SSH key to your repository (${this.gitOrigin === "GitHub" ? new URL(`./settings/keys/new`, this.gitOriginURL) : new URL(`./-/settings/repository`, this.gitOriginURL)}): \n\n${ESCAPE_SEQUENCES.GREY}${sshKey}${ESCAPE_SEQUENCES.RESET}\n`); - this.errorMessage = errorMessage; - throw e; - } - } - - // set debug port - let i=0; - for (const arg of this.args??[]) { - if (arg.startsWith("--inspect")) { - this.debugPort = arg.match(/\:(\d+)$/)?.[1] ?? "9229"; - this.args![i] = `--inspect=0.0.0.0:${this.debugPort}` - } - i++; - } - - // copy dockerfile - const dockerfile = await this.getDockerFileContent(); - await Deno.writeTextFile(dockerfilePath, dockerfile); - - // also remove docker container + docker image with same name remove to make sure - await Container.removeContainer(this.container_name); - await Container.removeImage(this.image); - - // create docker container - // TODO: --build-arg uix_args="${this.args?.join(" ")??""}" - await execCommand(`docker build -f ${dockerfilePath} --build-arg stage=${this.stage} --build-arg host_endpoint=${Datex.Runtime.endpoint} -t ${this.image} ${dir}`) - - // remove tmp dir - await Deno.remove(dir, {recursive: true}); - - // enable traefik routing - if (config.enableTraefik) { - for (const [domain, port] of Object.entries(domains)) { - this.enableTraefik(domain, port); - } - this.addEnvironmentVariable("UIX_HOST_DOMAINS", Object.keys(domains).join(",")); - } - // expose port 80 - else { - this.exposePort(80, config.hostPort) - } - - // add persistent volume for datex cache - await this.addVolume(this.formatVolumeName(this.container_name+'-'+'datex-cache'), '/datex-cache') - - // add persistent volume for deno localStoragae - await this.addVolume(this.formatVolumeName(this.container_name+'-'+'localstorage'), '/root/.cache/deno/location_data') - - // // add volume for host data, available in /app/hostdata - // this.addVolumePath('/root/data', '/app/hostdata') - } - - catch (e) { - console.log("error", e); - // this.logger.error("Error initializing UIX container"); - return false; - } - - return super.handleInit(); - } - - private get sshKeyName() { - return Datex.Runtime.endpoint.main.name.replaceAll('-','_') + - '_' + this.orgName?.replaceAll('-','_').replaceAll('/','_') + - '_' + this.repoName?.replaceAll('-','_').replaceAll('/','_'); - } - - private get sshKeyPath() { - const homeDir = Deno.env.get("HOME"); - if (!homeDir) throw new Error("Could not get home directory"); - return `${homeDir}/.ssh/id_rsa_${this.sshKeyName}`; - } - - private get uniqueGitHostName() { - return `git_${this.sshKeyName}`; - } - - private async tryGetSSHKey() { - const homeDir = Deno.env.get("HOME"); - const keyPath = this.sshKeyPath; - // return public key if already exists - try { - return await Deno.readTextFile(keyPath+".pub"); - } - // generate new key - catch { - - // ssh keyscan - await execCommand(`ssh-keyscan -H ${this.gitHTTPS.hostname} >> ${homeDir}/.ssh/known_hosts`) - - await execCommand(`ssh-keygen -t rsa -b 4096 -N '' -C '${Datex.Runtime.endpoint.main}' -f ${keyPath}`) - // add to ssh/config - let existingConfig = ""; - try { - existingConfig = await Deno.readTextFile(`${homeDir}/.ssh/config`); - } - catch {} - await Deno.writeTextFile(`${homeDir}/.ssh/config`, `${existingConfig} - -Host ${this.uniqueGitHostName} - User git - Hostname ${this.gitHTTPS.hostname} - IdentityFile ${keyPath} -`) - // return public key - return await Deno.readTextFile(keyPath+".pub"); - } - - - } - - private async getDockerFileContent() { - let dockerfile = await Deno.readTextFile(this.isVersion1 ? './res/uix-app-docker/Dockerfile_v0.1' : './res/uix-app-docker/Dockerfile'); - - // add uix run args + custom importmap/run path - dockerfile = dockerfile - .replace("{{UIX_ARGS}}", this.args?.join(" ")??"") - .replace("{{IMPORTMAP_PATH}}", this.advancedOptions?.importMapPath ?? 'https://dev.cdn.unyt.org/importmap_compat.json') - .replace("{{UIX_RUN_PATH}}", this.advancedOptions?.uixRunPath ?? 'https://cdn.unyt.org/uix@0.1.x/run.ts') - - // expose port - if (this.debugPort) { - dockerfile = dockerfile.replace("{{EXPOSE_DEBUG}}", "EXPOSE 9229") - } - else { - dockerfile = dockerfile.replace("{{EXPOSE_DEBUG}}", "") - } - return dockerfile; - } - - - override async handleOnline(){ - // wait until endpoint inside container is reachable - this.logger.info("Waiting for "+this.endpoint+" to come online"); - let iterations = 0; - while (true) { - await sleep(2000); - if (await this.endpoint.isOnline()) { - this.logger.success("Endpoint "+this.endpoint+" is online"); - return true; - } - if (iterations++ > 20) { - this.logger.error("Endpoint "+this.endpoint+" not reachable") - return false; - } - } - } - -} - -type AdvancedUIXContainerOptions = { - importMapPath?:string, - uixRunPath?:string -} - -@endpoint @entrypointProperty class ContainerManager { - - @property static async getContainers():Promise>{ - return containers.getAuto(datex.meta!.sender); - } - - @property static async createWorkbenchContainer():Promise{ - const sender = datex.meta!.sender; + @property static async createWorkbenchContainer(token?: string): Promise { + ensureToken(token); + const sender = datex.meta!.caller; logger.info("Creating new Workbench Container for " + sender); // create config @@ -900,57 +41,96 @@ type AdvancedUIXContainerOptions = { config.endpoint = Datex.Endpoint.getNewEndpoint(); // init and start WorkbenchContainer + // @ts-ignore $ const container = new WorkbenchContainer(sender, config); container.start(); // link container to requesting endpoint - this.addContainer(sender, container); - + await this.addContainer(sender, container); return container; } - @property static async createRemoteImageContainer(url:string):Promise{ - const sender = datex.meta!.sender; - - logger.info("Creating new Remote Image Container for " + sender, url); + @property static async createRemoteImageContainer(token: string, name: string): Promise { + ensureToken(token); + const sender = datex.meta!.caller; + if (!name || typeof name !== "string" || name.length < 2 || name.length > 80 || !/^[a-z\.\-\/#%?=0-9:&]+$/gi.test(name)) + throw new Error(`Can not create remote image container with name '${name}'`); + logger.info(`Creating new Remote Image Container '${name}' for`, sender); // init and start RemoteImageContainer - const container = new RemoteImageContainer(sender, url); - container.start(); + // @ts-ignore $ + const container = new RemoteImageContainer(sender, name); + container.start().then(async ()=>{ + await this.addContainer(sender, container); + }).catch(); // link container to requesting endpoint - this.addContainer(sender, container); - return container; } - @property static async createUIXAppContainer(gitURL:string, branch: string, endpoint: Datex.Endpoint, stage?: string, domains?: Record, env?: string[], args?: string[], persistentVolumePaths?: string[], gitAccessToken?: string, advancedOptions?: AdvancedUIXContainerOptions):Promise{ - const sender = datex.meta!.sender; - - console.log("Creating new UIX App Container for " + sender, gitURL, branch, env); + @property static async createUIXAppContainer( + gitURL: string, + branch: string, + endpoint: Datex.Endpoint, + stage?: string, + domains?: Record, + env?: string[], + args?: string[], + persistentVolumePaths?: string[], + gitAccessToken?: string, + advancedOptions?: AdvancedUIXContainerOptions, + token?: string + ): Promise { + ensureToken(token); + const sender = datex.meta!.caller; + logger.info(`Creating new UIX App Container for ${sender}`, gitURL, branch, env); + + if (!branch || typeof branch !== "string" || branch.length < 2 || branch.length > 50 || !/^[a-z\.\-\/#0-9:&]+$/gi.test(branch)) + throw new Error(`Can not create UIX App container with branch '${branch}'`); + if (!gitURL || typeof gitURL !== "string" || gitURL.length < 2) + throw new Error(`Can not create UIX App container with url '${gitURL}'`); + if (!endpoint || !(endpoint instanceof Endpoint)) + throw new Error(`Can not create UIX App container with endpoint '${endpoint}'`); // init and start RemoteImageContainer + // @ts-ignore $ const container = new UIXAppContainer(sender, endpoint, gitURL, branch, stage, domains, env, args, persistentVolumePaths, gitAccessToken, advancedOptions); - container.start(); + container.start().then(async ()=>{ + if (container.status === ContainerStatus.FAILED) { + container.stop(true); + logger.error(`Could not start app container for '${gitURL}'`); + } else { + // link container to requesting endpoint + await this.addContainer(sender, container); + } + }).catch(); await sleep(2000); // wait for immediate status updates - - // link container to requesting endpoint - this.addContainer(sender, container); - return container; } - private static addContainer(endpoint:Datex.Endpoint, container:Container) { - containers.getAuto(endpoint).add(container); + public static async getContainerList() { + const list = new Set(); + for await (const containerList of containers.values()) + containerList.forEach(e => list.add(e)); + return list; } + private static async addContainer(endpoint: Datex.Endpoint, container: Container) { + if (await containers.has(endpoint)) + (await containers.get(endpoint))!.add(container); + else await containers.set(endpoint, new Set([container])); + console.log("Added", container.name, endpoint, await containers.getSize(), containers.constructor.name) + } - public static findContainer({type, properties, endpoint}: {type?: Class, endpoint?: Datex.Endpoint, properties?: Record}) { - const matches = [] - iterate: for (const [containerEndpoint, containerSet] of containers) { + public static async findContainer({type, properties, endpoint}: { + type?: Class, + endpoint?: Datex.Endpoint, + properties?: Record + }): Promise { + const matches: T[] = []; + iterate: for await (const [containerEndpoint, containerSet] of containers.entries()) { // match endpoint if (endpoint && !containerEndpoint.equals(endpoint)) continue; - for (const container of containerSet) { // match container type if (!type || container instanceof type) { @@ -960,33 +140,16 @@ type AdvancedUIXContainerOptions = { if ((container as any)[key] !== value) continue iterate; } } - matches.push(container); + matches.push(container as T); } } } return matches; } - } -const containers = (await lazyEternalVar("containers") ?? $$(new Map>)).setAutoDefault(Set); -logger.info(containers.size + " containers in cache") - +export const containers = eternalVar("containers") ?? $$(new StorageMap>()); -async function execCommand(command:string, denoRun?:DenoRun): Promise { - console.log("exec: " + command) - - if (denoRun) { - const status = await Deno.run({ - cmd: command.split(" "), - }).status(); - - if (!status.success) throw status.code; - else return status as any; - } - else { - const {status, output} = (await exec(`bash -c "${command.replaceAll('"', '\\"')}"`, {output: OutputMode.Capture})); - if (!status.success) throw output; - else return output as any; - } -} +logger.info(`Found ${await containers.getSize()} unique endpoint(s) in cache.`); +logger.info("Container List:", [...(await ContainerManager.getContainerList())].map(e => `${e.constructor.name} ${e.container_name}: ${e.owner} ${e.image} (${ContainerStatus[e.status]})`)); +logger.success("Docker Host is up and running"); \ No newline at end of file diff --git a/res/Dockerfile b/res/Dockerfile index 823b361..7925604 100644 --- a/res/Dockerfile +++ b/res/Dockerfile @@ -1,14 +1,13 @@ FROM ubuntu - RUN apt-get update && apt-get install -y \ - software-properties-common \ + software-properties-common \ curl\ lsof\ - npm + npm RUN npm install npm -g && \ - npm install n -g && \ - n 18.7.0 + npm install n -g && \ + n 18.7.0 RUN apt-get install -y \ wget \ git \ @@ -34,8 +33,6 @@ ARG username=user ARG configpath=res/config-files ENV username=${username} - - # add user RUN useradd -m -d /home/${username} ${username} -s /usr/bin/zsh @@ -57,10 +54,6 @@ USER $username EXPOSE 22 CMD ["/usr/sbin/sshd", "-D"] - - - - WORKDIR /home/${username} @@ -73,11 +66,5 @@ WORKDIR /home/${username} #LABEL traefik.http.routers.workbench-secured.tls="true" #LABEL traefik.http.routers.workbench-secured.tls.certresolver="myhttpchallenge" - - - - - - # start endpoint -ENTRYPOINT node /unyt-workbench/run.js /home/${username}/.unyt/endpoint.dx +ENTRYPOINT node /unyt-workbench/run.js /home/${username}/.unyt/endpoint.dx \ No newline at end of file diff --git a/res/config-files/endpoint.dx b/res/config-files/endpoint.dx index 6663698..ae5ac59 100644 --- a/res/config-files/endpoint.dx +++ b/res/config-files/endpoint.dx @@ -1,7 +1,5 @@ - - -{ - version: 1, - endpoint: @@010E52616CFD208F2D837611, - 'expose-database': true -} + { + version: void, + endpoint: @@C9BCF134050000000070F566AF2C5C59, + exposeDatabase: void +} \ No newline at end of file diff --git a/res/uix-app-docker/Dockerfile b/res/uix-app-docker/Dockerfile_v0.0 similarity index 100% rename from res/uix-app-docker/Dockerfile rename to res/uix-app-docker/Dockerfile_v0.0 diff --git a/res/uix-app-docker/Dockerfile_v0.1 b/res/uix-app-docker/Dockerfile_v0.1 index 0228ee7..aaa0f12 100644 --- a/res/uix-app-docker/Dockerfile_v0.1 +++ b/res/uix-app-docker/Dockerfile_v0.1 @@ -16,4 +16,4 @@ COPY ./repo . EXPOSE 80 {{EXPOSE_DEBUG}} -CMD sh -c "rm -f deno.lock; rm -f ./src/deno.lock; deno upgrade --version 1.45.5; deno run --config deno.json --import-map {{IMPORTMAP_PATH}} -Aqr {{UIX_RUN_PATH}} --port 80 --stage $APP_STAGE --cache-path /datex-cache {{UIX_ARGS}}" \ No newline at end of file +CMD sh -c "rm -f deno.lock; rm -f ./src/deno.lock; deno run --config deno.json --import-map {{IMPORTMAP_PATH}} -Aqr {{UIX_RUN_PATH}} --port 80 --stage $APP_STAGE --cache-path /datex-cache {{UIX_ARGS}}" \ No newline at end of file diff --git a/res/uix-app-docker/Dockerfile_v0.3 b/res/uix-app-docker/Dockerfile_v0.3 new file mode 100644 index 0000000..b1c1bdd --- /dev/null +++ b/res/uix-app-docker/Dockerfile_v0.3 @@ -0,0 +1,23 @@ +FROM ubuntu:latest +RUN apt-get update && apt-get install -y \ + bash \ + unzip \ + curl + +ARG stage +ARG host_endpoint +# ARG uix_args + +ENV APP_STAGE=$stage +ENV UIX_HOST_ENDPOINT=$host_endpoint + +# Copy repo +WORKDIR /app +COPY ./repo . + +EXPOSE 80 +{{EXPOSE_DEBUG}} + +RUN curl -fsSL https://unyt.land/install.sh | bash + +CMD sh -c "rm -f deno.lock; rm -f ./src/deno.lock; $HOME/.uix/bin/deno run --config deno.json --import-map {{IMPORTMAP_PATH}} -Aqr {{UIX_RUN_PATH}} --port 80 --stage $APP_STAGE --cache-path /datex-cache {{UIX_ARGS}}" \ No newline at end of file diff --git a/res/uix-app-docker/new/Dockerfile b/res/uix-app-docker/new/Dockerfile new file mode 100644 index 0000000..78bdc59 --- /dev/null +++ b/res/uix-app-docker/new/Dockerfile @@ -0,0 +1,33 @@ +FROM --platform=linux/x86-64 ubuntu:latest + +# Install Deno for UIX dependencies +RUN apt-get update && apt-get install -y \ + bash \ + unzip \ + curl + +# List of packages to be installed by apt-get +ARG INSTALL_PACKAGES +RUN if [ ! -z "$INSTALL_PACKAGES" ] ; then apt-get install -y $INSTALL_PACKAGES ; fi + +# Expose port 80 +EXPOSE 80 + +# Copy startup.sh script +WORKDIR /uix +VOLUME /repo +COPY ./uix-startup.sh . +RUN chmod +x uix-startup.sh + +# Install Deno +ARG DENO +RUN if [ "$DENO" = "legacy" ] ; then curl -fsSL https://deno.land/install.sh | bash; fi +RUN if [ "$DENO" != "legacy" ] ; then curl -fsSL https://unyt.land/install.sh | bash ; fi + +# Set version for Deno +ARG DENO_VERSION +RUN if [ "$DENO_VERSION" ] && [ "$DENO" = "legacy" ] ; then $HOME/.deno/bin/deno upgrade --version "$DENO_VERSION"; fi +RUN if [ "$DENO_VERSION" ] && [ "$DENO" != "legacy" ] ; then $HOME/.uix/bin/deno upgrade --version "$DENO_VERSION"; fi + +# Startup UIX app +ENTRYPOINT ["./uix-startup.sh"] \ No newline at end of file diff --git a/res/uix-app-docker/new/README.md b/res/uix-app-docker/new/README.md new file mode 100644 index 0000000..c201808 --- /dev/null +++ b/res/uix-app-docker/new/README.md @@ -0,0 +1,5 @@ +# README + +* `docker build -f ./Dockerfile --tag 'uix' --build-arg DENO="legacy" --build-arg DENO_VERSION="v1.46.2" .` +* `docker build -f ./Dockerfile --tag 'uix' --build-arg DENO="uix" --build-arg INSTALL_PACKAGES="git" .` +* `docker run --publish 9999:80 --platform linux/x86-64 -v $(pwd)/repo:/repo -it uix ./importmap.json https://dev.cdn.unyt.org/uix1/run.ts prod` \ No newline at end of file diff --git a/res/uix-app-docker/new/uix-startup.sh b/res/uix-app-docker/new/uix-startup.sh new file mode 100755 index 0000000..60fec2e --- /dev/null +++ b/res/uix-app-docker/new/uix-startup.sh @@ -0,0 +1,30 @@ +#!/bin/bash +IMPORT_MAP_PATH="$1" +RUN_TS_URL="$2" +STAGE="$3" +shift 3 +UIX_ARGS="$@" + +cd /repo/ + +if [ -x "$HOME/.uix/bin/deno" ]; then + DENO_EXEC="$HOME/.uix/bin/deno" +elif [ -x "$HOME/.deno/bin/deno" ]; then + DENO_EXEC="$HOME/.deno/bin/deno" +else + echo "Error: Deno executable not found in either $HOME/.uix/bin/deno or $HOME/.deno/bin/deno" + exit 1 +fi + +echo "Using Deno from $DENO_EXEC..." +rm -f deno.lock + +"$DENO_EXEC" run \ + --config deno.json \ + --import-map "$IMPORT_MAP_PATH" \ + -Aqr "$RUN_TS_URL" \ + --port 80\ + --stage "$STAGE" \ + --cache-path /datex-cache \ + -y \ + "$UIX_ARGS" \ No newline at end of file diff --git a/setup.sh b/setup.sh index 6001f4e..9227544 100755 --- a/setup.sh +++ b/setup.sh @@ -6,7 +6,7 @@ if ! [ -x "$(command -v git)" ]; then fi if ! [ -x "$(command -v docker)" ]; then - echo "docker must be installed" + echo "Docker must be installed" exit 1 fi @@ -15,6 +15,7 @@ if ! [ -x "$(command -v unzip)" ]; then exit 1 fi + # Install deno if ! [ -x "$(command -v deno)" ]; then echo 'Installing deno...' @@ -37,25 +38,36 @@ if ! [ -x "$(command -v deno)" ]; then fi -# echo "Please enter the endpoint id for this docker host:" -# read ENDPOINT ENDPOINT="$1" +if [ ${#ENDPOINT} -gt 20 ]; then + echo "Error: Endpoint id/name must be <=18 bytes" + exit 1 +fi mkdir -p $HOME/.unyt-docker-host/ DIR=$HOME/.unyt-docker-host/$ENDPOINT GIT_ORIGIN=https://github.com/unyt-org/docker-host.git -SERVICE_NAME=$(systemd-escape "unyt_docker_host_$ENDPOINT") +SERVICE_NAME=$(systemd-escape "unyt_$(echo "$ENDPOINT" | sed 's/^[^a-z0-9]*//' | sed 's/[^a-z0-9_]/_/g')") DENO_DIR=$(which deno) +if [ -d "$DIR" ]; then + echo "$DIR does already exist. Please pick another endpoint or remove the existing Docker Host" + exit +fi + # clone git repo echo "Cloning git repo to $DIR ..." -git clone $GIT_ORIGIN $DIR +git clone -b v2 $GIT_ORIGIN $DIR + +# set access token +RANDOM_STRING=$(head /dev/urandom | LC_ALL=C tr -dc A-Za-z0-9 | head -c 16) +NEW_TOKEN="\"$(echo $RANDOM_STRING | sed 's/.\{4\}/&-/g;s/-$//')\"" +sed -i.bak 's/token: "[^"]*"/token: '"$NEW_TOKEN"'/g' "$DIR/config.dx" # rename endpoint echo "endpoint: $ENDPOINT" > "$DIR/.dx" - # Create the service unit file cat > /etc/systemd/system/$SERVICE_NAME.service <("./config.dx") \ No newline at end of file +export const config = await datex.get("../config.dx"); \ No newline at end of file diff --git a/src/container/Container.ts b/src/container/Container.ts new file mode 100644 index 0000000..146b06d --- /dev/null +++ b/src/container/Container.ts @@ -0,0 +1,331 @@ +// deno-lint-ignore-file require-await +import { Datex } from "unyt_core/mod.ts"; +import { ContainerStatus } from "./Types.ts"; +import { createHash } from "https://deno.land/std@0.91.0/hash/mod.ts"; +import { executeDocker, executeShell } from "../CMD.ts"; +import { containers } from "../../main.ts"; + +const logger = new Datex.Logger("Container"); + +// parent class for all types of containers +@sync export class Container { + protected logger!: Datex.Logger; + #initialized = false; + + // docker container image + id + @property image!: string; + @property container_name = "Container"; + @property name = "Container"; + @property id = '0'; + @property owner!: Datex.Endpoint; + @property status: ContainerStatus = ContainerStatus.INITIALIZING; + @property errorMessage?: string; + + #labels: string[] = [] + #ports: [number, number][] = [] + #env: Record = {} + #volumes: Record = {} + + network = "main" + debugPort: string|null = null + + get volumes() {return this.#volumes;} + + addLabel(label: string) { + this.#labels.push(label); + } + + formatVolumeName(name: string) { + return name.replace(/[^a-zA-Z0-9_.-]/g, '-'); + } + + async addVolume(name: string, path: string) { + await executeDocker(["volume", "create", name], false); + this.#volumes[name] = path; + } + + addVolumePath(hostPath: string, path: string) { + this.#volumes[hostPath] = path; + } + + addEnvironmentVariable(name: string, value: string) { + this.#env[name] = value; + } + + getFormattedLabels() { + return this.#labels + .map(label => ["--label", `${label}`]) + .flat(); + } + getFormattedPorts() { + return this.#ports.map(ports => ["-p", `${ports[1]}:${ports[0]}`]).flat(); + } + getFormattedEnvVariables() { + return Object.entries(this.#env).map(([name, value]) => ["--env", `${name}=${value}`]).flat(); + } + getFormattedVolumes() { + return Object.entries(this.#volumes).map(([name, path]) => ["-v", `${name}:${path}`]).flat(); + } + + uniqueID(size = 16) { + return crypto.randomUUID().replaceAll("-", "").slice(0, size); + } + + construct(owner: Datex.Endpoint, ..._: unknown[]) { + this.owner = owner; + this.container_name = this.uniqueID(); + this.logger = new Datex.Logger(this); + } + replicate() { + this.logger = new Datex.Logger(this); + this.#initialized = true; + this.updateAfterReplicate(); + } + + @property async start() { + // start => RUNNING or FAILED + const running = await this.handleStart(); + if (running) { + this.status = ContainerStatus.RUNNING; + + // get online state + const online = await this.handleOnline(); + if (online) this.status = ContainerStatus.ONLINE; + else this.status = ContainerStatus.FAILED; + } else this.status = ContainerStatus.FAILED; + return running; + } + + // create docker for the first time + protected async init() { + if (this.#initialized) + return true; + + // INITIALIZING ... + this.status = ContainerStatus.INITIALIZING; + + // STOPPED (default state) or FAILED + const initialized = await this.handleInit(); + if (initialized) + this.status = ContainerStatus.STOPPED; + else this.status = ContainerStatus.FAILED; + + this.#initialized = initialized; + return initialized; + } + + + protected async handleInit() { + try { + const restartPolicy = "always"; + await executeDocker([ + "run", + "--network", this.network, + ...(this.debugPort ? [`-p`, `${this.debugPort}:9229`] : []), + "--log-opt", + "max-size=10m", + "-d", + "--restart", restartPolicy, + "--name", this.container_name, + ...this.getFormattedPorts(), + ...this.getFormattedVolumes(), + ...this.getFormattedEnvVariables(), + ...this.getFormattedLabels(), + this.image + ], false); + this.logger.success(`Running docker ${this.container_name}...`); + } catch(error) { + this.logger.error("Could not create container", error); + return false; + } + return true; + } + + protected updateAfterReplicate() { + // continue start/stop if in inbetween state + if (this.status == ContainerStatus.STARTING) this.start(); + else if (this.status == ContainerStatus.STOPPING) this.stop(); + } + + protected async handleStart() { + // first init docker container (if not yet initialized) + if (!await this.init()) return false; + this.logger.info("Starting Container", this.container_name); + await this.onBeforeStart(); + if (this.status == ContainerStatus.FAILED) + return false; + + // STARTING ... + this.status = ContainerStatus.STARTING; + // start the container + try { + await executeDocker([ + "container", + "start", + this.container_name + ], false); + } catch (error) { + this.logger.error("error while starting container", error); + return false; + } + await sleep(2000); + + // check if container is running + const running = await this.isRunning(); + if (running) + this.logger.success("Container is running") + else this.logger.error("Container is not running") + return running; + } + + protected onBeforeStart() {} + protected onBeforeStop() {} + + protected async handleOnline() { + return false; + } + + @property async stop(force = true){ + this.logger.info("Stopping Container " + this.container_name); + this.onBeforeStop(); + this.status = ContainerStatus.STOPPING; + try { + await executeDocker([ + "container", + (force ? "kill" : "stop"), + this.container_name + ], false); + } catch (e) { + this.logger.error("error while stopping container",e); + // TODO FAILED or RUNNING? + this.status = ContainerStatus.FAILED; + return false; + } + this.status = ContainerStatus.STOPPED; + return true; + } + + /** + * Get a stream of the container logs + * @param timeout timeout in minutes after which the stream will be closed + * @returns + */ + @property public getLogs(timeout = 60) { + const p = new Deno.Command("docker", { + args: [ + "logs", + "--follow", + this.container_name + ], + stdout: "piped", + stderr: "piped" + }).spawn() + + const stream = $$(new Datex.Stream()); + stream.pipe(p.stdout); + stream.pipe(p.stderr); + + // close stream after timeout + setTimeout(() => { + // \u0004 is the EOT character + stream.write(new TextEncoder().encode("\n[Stream was closed after " + timeout + " minutes]\n\u0004").buffer); + stream.close(); + p.kill(); + }, timeout * 60 * 1000); + return stream; + } + + public async remove() { + // remove from containers list + if (await containers.has(this.owner)) + (await containers.get(this.owner))?.delete(this); + await Container.removeContainer(this.container_name); + await Container.removeImage(this.image); + return true; + } + + private async isRunning() { + try { + await executeShell([ + "docker", + "ps", + "|", + "grep", + `"${this.container_name}"` + ], false); + return true; + } catch (e) { + this.logger.error(e); + return false + } + } + + exposePort(port: number, hostPort: number) { + this.#ports.push([port, hostPort]) + } + + protected enableTraefik(host: string, port?: number) { + const name = this.image + "-" + createHash("md5").update(host).toString() + const hasWildcard = host.startsWith('*.'); + + const hostRule = hasWildcard ? + `HostRegexp(\`{subhost:[a-z0-9-_]+}.${host.slice(2)}\`)` : + `Host(\`${host}\`)`; + + this.addLabel(`traefik.enable=true`); + this.addLabel(`traefik.http.routers.${name}.rule=${hostRule}`); + this.addLabel(`traefik.http.routers.${name}.entrypoints=web`); + // TODO: only workaoound for *.unyt.app domains: prio 1 + this.addLabel(`traefik.http.routers.${name}.priority=${hasWildcard ? 1 : 10}`); + this.addLabel(`traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https`); + this.addLabel(`traefik.http.routers.${name}.middlewares=redirect-to-https@docker`); + this.addLabel(`traefik.http.routers.${name}-secured.rule=${hostRule}`); + this.addLabel(`traefik.http.routers.${name}-secured.tls=true`); + // TODO: only workaoound for *.unyt.app domains: prio 1 + this.addLabel(`traefik.http.routers.${name}-secured.priority=${hasWildcard ? 1 : 10}`); + + if (hasWildcard) { + const rawHost = host.slice(2); + this.addLabel(`traefik.http.routers.${name}-secured.tls.domains[0].main=${rawHost}`); + this.addLabel(`traefik.http.routers.${name}-secured.tls.domains[0].sans=${host}`); + this.addLabel(`traefik.http.routers.${name}.tls.domains[0].main=${rawHost}`); + this.addLabel(`traefik.http.routers.${name}.tls.domains[0].sans=${host}`); + this.addLabel(`traefik.http.routers.${name}-secured.tls.certresolver=mydnschallenge`); + } else { + this.addLabel(`traefik.http.routers.${name}-secured.tls.certresolver=myhttpchallenge`); + } + + if (port) { + this.addLabel(`traefik.http.routers.${name}.service=${name}`); + this.addLabel(`traefik.http.routers.${name}-secured.service=${name}`); + this.addLabel(`traefik.http.services.${name}.loadbalancer.server.port=${port}`); + } + } + + public static async removeContainer(name: string) { + try { + await executeDocker([ + "rm", + "-f", + name + ], false); + } catch (e) { + logger.error("Could not remove container", e); + return false; + } + } + + public static async removeImage(name: string) { + try { + await executeDocker([ + "image", + "rm", + "-f", + name + ], false); + } catch (e) { + logger.error("Could not remove image", e); + return false; + } + } +} \ No newline at end of file diff --git a/src/container/RemoteImageContainer.ts b/src/container/RemoteImageContainer.ts new file mode 100644 index 0000000..67492b8 --- /dev/null +++ b/src/container/RemoteImageContainer.ts @@ -0,0 +1,50 @@ +import { Datex } from "unyt_core/mod.ts"; +import { Container } from "./Container.ts"; +import { executeDocker } from "../CMD.ts"; + +@sync export class RemoteImageContainer extends Container { + @property version?: string + @property url!: string + + construct(owner: Datex.Endpoint, url: string, version?: string) { + super.construct(owner); + this.version = version; + this.url = url; + this.name = url; + } + + // update docker image + @property async update() { + try { + const image = `${this.url}${this.version ? `:${this.version}` : ''}`; + if (!/^[a-z\.\-\/#%?=0-9:&]+$/gi.test(image)) + throw new Error(`Could not pull image with name ${image}`); + this.image = image; + await executeDocker([ + "pull", + this.image + ], false); + this.logger.success(`Successfully pulled remote image ${this.image}`); + } catch (e) { + this.logger.error(e); + this.logger.error("Error pulling remote image"); + return false; + } + return true; + } + + // custom workbench container init + override async handleInit() { + if (!await this.update()) + return false; + return super.handleInit(); + } + + // custom start + override async handleStart() { + // always pull image first + if (!await this.update()) + return false; + return super.handleStart(); + } +} diff --git a/src/container/Types.ts b/src/container/Types.ts new file mode 100644 index 0000000..c23995d --- /dev/null +++ b/src/container/Types.ts @@ -0,0 +1,9 @@ +export enum ContainerStatus { + STOPPED = 0, + STARTING = 1, + RUNNING = 2, + STOPPING = 3, + FAILED = 4, + INITIALIZING = 5, + ONLINE = 6 +}; \ No newline at end of file diff --git a/src/container/UIXAppContainer.ts b/src/container/UIXAppContainer.ts new file mode 100644 index 0000000..0e3ab68 --- /dev/null +++ b/src/container/UIXAppContainer.ts @@ -0,0 +1,516 @@ +import { logger, ESCAPE_SEQUENCES } from "unyt_core/datex_all.ts"; +import { Datex } from "unyt_core/mod.ts"; +import { formatEndpointURL } from "unyt_core/utils/format-endpoint-url.ts"; +import { Path } from "unyt_core/utils/path.ts"; +import { config } from "../config.ts"; +import { Container } from "./Container.ts"; +import { RemoteImageContainer } from "./RemoteImageContainer.ts"; +import { ContainerStatus } from "./Types.ts"; +import { getIP } from "https://deno.land/x/get_ip@v2.0.0/mod.ts"; +import { ContainerManager } from "../../main.ts"; +import { executeDocker, executeGit, executeShell } from "../CMD.ts"; +import { containers } from "../../main.ts"; + +const publicServerIP = await getIP({ ipv6: false }); +const defaulTraefikToml = ` +[entryPoints] + [entryPoints.web] + address = ":80" + + [entryPoints.web-secure] + address = ":443" + +[api] + dashboard = true + +[providers.docker] + endpoint = "unix:///var/run/docker.sock" + exposedByDefault = false + network = "main" + +[certificatesresolvers.myhttpchallenge.acme] + caserver = "https://acme-v02.api.letsencrypt.org/directory" + email = "postmaster@unyt.org" + [certificatesresolvers.myhttpchallenge.acme.httpchallenge] + entrypoint = "web" +`; + +export type AdvancedUIXContainerOptions = { + importMapPath?: string, + uixRunPath?: string +}; + +@sync export class UIXAppContainer extends Container { + @property branch!: string; + @property gitSSH!: string; + @property gitHTTPS!: URL; + @property stage!: string; + @property domains!: Record; // domain name -> internal port + @property endpoint!: Datex.Endpoint; + @property advancedOptions?: AdvancedUIXContainerOptions; + @property args?: string[] + + // use v.0.1 + version = 0; + static VALID_DOMAIN = /^(([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\-]*[a-zA-Z0-9])\.)*([A-Za-z0-9]|[A-Za-z0-9][A-Za-z0-9\-]*[A-Za-z0-9])$/ + + async construct( + owner: Datex.Endpoint, + endpoint: Datex.Endpoint, + gitURL: string, + branch: string, + stage = 'prod', + domains?: Record, + env?:string[], + args?:string[], + persistentVolumePaths?: string[], + gitOAuthToken?: string, + advancedOptions?: AdvancedUIXContainerOptions) { + super.construct(owner); + + // validate domains + for (const domain of Object.keys(domains ?? {})) { + if (!( + UIXAppContainer.VALID_DOMAIN.test(domain) || + (config.allowArbitraryDomains && domain.startsWith('*.')) + )) { + this.errorMessage = `Invalid domain name "${domain}". Only alphanumeric characters and dashes are allowed.`; + this.status = ContainerStatus.FAILED; + throw new Error(this.errorMessage); + } + } + + // convert from https url + if (gitURL.startsWith("https://")) { + this.gitHTTPS = new URL(gitURL); + this.gitSSH = `git@${this.gitHTTPS.host}:${this.gitHTTPS.pathname.slice(1)}`; + } + // convert from ssh url + else if (gitURL.startsWith("git@")) { + const [host, pathname] = gitURL.match(/git@([^:]*)+?:(.*)/i)?.slice(1) ?? []; + if (!host || !pathname) + throw new Error(`Invalid git URL '${gitURL}'!`); + this.gitSSH = gitURL; + this.gitHTTPS = new URL(`https://${host}/${pathname}`); + } + + // add gh token to URL + if (gitOAuthToken) { + this.gitHTTPS.username = "oauth2"; + this.gitHTTPS.password = gitOAuthToken; + } + + this.container_name = (endpoint.name + (endpoint.name.toLowerCase().endsWith(stage.toLowerCase()) ? '' : (stage ? '-' + stage : ''))) + .replace(/[^a-z0-9@-_#]/gmi, '-') + .toLowerCase(); + this.advancedOptions = advancedOptions; + + this.endpoint = endpoint; // TODO: what if @@local is passed + this.args = args; + + this.branch = branch ?? "main"; + this.stage = stage; + this.domains = domains ?? {}; + + // check if only unyt.app domains are used + if (!config.allowArbitraryDomains) { + for (const domain of Object.keys(this.domains)) { + if (!domain.endsWith('.unyt.app')) { + this.errorMessage = `Invalid domain "${domain}". Only unyt.app domains are allowed`; + this.status = ContainerStatus.FAILED; + throw new Error(this.errorMessage); + } + } + } + // inject environment variables + for (const envVar of env??[]) { + const [key, val] = envVar.split("="); + this.addEnvironmentVariable(key, val); + + if (key === "UIX_VERSION") { + this.version = +val.replace("0.", ""); + this.logger.info(`Using UIX v${this.version}`); + } + } + + // add persistent volumes + for (const path of persistentVolumePaths ?? []) { + const mappedPath = path.startsWith("./") ? `/app${path.slice(1)}` : path; + const volumeName = this.formatVolumeName(`${this.container_name}-persistent-${(Object.keys(this.volumes).length)}`); + await this.addVolume(volumeName, mappedPath); + } + } + + protected async handleNetwork() { + // make sure main network exists + try { + await executeDocker([ + "network", + "inspect", + this.network, + ], false); + } catch { + await executeDocker([ + "network", + "create", + this.network, + ], false); + } + + if (config.enableTraefik) { + // has traefik? + try { + await executeShell(["docker", "ps", "|", "grep", "traefik"], false); + this.logger.success("Found existing traefik container"); + } catch { + this.logger.info("Could not detect existing traefik container. Creating new traefik container..."); + const traefikDir = new Path("/etc/traefik/"); + + // init and start traefik container + if (!traefikDir.fs_exists) + await Deno.mkdir("/etc/traefik/", { recursive: true }); + + const traefikTomlPath = traefikDir.asDir().getChildPath("traefik.toml"); + const acmeJsonPath = traefikDir.asDir().getChildPath("acme.json"); + + await Deno.create(acmeJsonPath.normal_pathname) + await executeShell(["chmod", "600", acmeJsonPath.normal_pathname], false); + await Deno.writeTextFile(traefikTomlPath.normal_pathname, defaulTraefikToml) + + // @ts-ignore $ + const traefikContainer = new RemoteImageContainer(Datex.LOCAL_ENDPOINT, "traefik", "v2.5"); + traefikContainer.exposePort(80, config.hostPort) + traefikContainer.exposePort(443, 443) + traefikContainer.addVolumePath("/var/run/docker.sock", "/var/run/docker.sock") + traefikContainer.addVolumePath(traefikTomlPath.normal_pathname, "/etc/traefik/traefik.toml") + traefikContainer.addVolumePath(acmeJsonPath.normal_pathname, "/acme.json") + traefikContainer.start(); + logger.success("Created traefik container", traefikContainer); + } + } + } + + protected override async onBeforeStart() { + if (config.setDNSEntries) { + for (const domain of Object.keys(this.domains)) { + // currently only for unyt.app + if (domain.endsWith('.unyt.app')) { + this.logger.info("Setting DNS entry for " + domain + " to " + publicServerIP); + try { + await datex `@+unyt-dns-1.DNSManager.addARecord(${domain}, ${publicServerIP})` + this.logger.success("Successfully set DNS entry for " + domain); + } + catch { + this.logger.error("Error setting DNS entry for " + domain); + this.errorMessage = `Could not set DNS entry for ${domain} (Internal error)`; + this.status = ContainerStatus.FAILED; + } + } + } + } + } + + protected override async onBeforeStop() { + if (config.setDNSEntries) { + for (const domain of Object.keys(this.domains)) { + // currently only for unyt.app + if (domain.endsWith('.unyt.app')) { + this.logger.info("Removing DNS entry for " + domain); + try { + await datex `@+unyt-dns-1.DNSManager.removeARecord(${domain})` + this.logger.success("Successfully removed DNS entry for " + domain); + } catch (e) { + this.logger.error("Error removing DNS entry for " + domain); + this.logger.error(e); + } + } + } + } + } + + get orgName() { + return this.gitHTTPS.pathname.split("/").at(1)!; + } + get repoName() { + return this.gitHTTPS.pathname.split('/').slice(2).join("/")!.replace('.git', ''); + } + + get gitOrigin() { + return ({ + "github.com": "GitHub", + "gitlab.com": "GitLab" + } as const)[this.gitHTTPS.hostname] ?? "GitLab"; + } + + get gitOriginURL() { + return new URL(`/${this.orgName}/${this.repoName}/`, this.gitHTTPS); + } + + // custom workbench container init + override async handleInit() { + // setup network + await this.handleNetwork(); + + // remove any existing previous container + const existingContainers = await ContainerManager.findContainer({type: UIXAppContainer, properties: { + // gitHTTPS: this.gitHTTPS, + // stage: this.stage + container_name: this.container_name + }}); + if (existingContainers.length === 0) { + this.logger.info(`Found no existing containers ${this.gitHTTPS} (${this.stage}). Creating new...`); + // FIXME + // for await (const [endpoint, cs] of containers.entries()) { + // for (const c of cs) { + // console.log(c instanceof UIXAppContainer) + // console.log(endpoint, c.container_name, (c as unknown as UIXAppContainer).stage, c.gitHTTPS) + // } + // } + } + for (const existingContainer of existingContainers) { + this.logger.warn("Removing existing container", existingContainer) + await existingContainer.remove(); + } + this.image = this.container_name; + try { + const domains = this.domains ?? [formatEndpointURL(this.endpoint)]; + this.logger.info("Image: " + this.image); + this.logger.info("Repo: " + this.gitHTTPS + " / " + this.gitSSH); + this.logger.info("Branch: " + this.branch); + this.logger.info("Endpoint: " + this.endpoint); + this.logger.info("Domains: " + Object.entries(domains).map(([d,p])=>`${d} (port ${p})`).join(", ")); + this.logger.info("Options: " + (this.advancedOptions ? JSON.stringify(this.advancedOptions) : "{}")); + + // clone repo + const dir = await Deno.makeTempDir({ prefix:'uix-app-' }); + const dockerfilePath = `${dir}/Dockerfile`; + const repoPath = `${dir}/repo`; + let repoIsPublic = false; + + try { + if (this.gitOrigin === "GitHub") + repoIsPublic = (await (await fetch(`https://api.github.com/repos/${this.orgName}/${this.repoName}`)).json()).visibility == "public" + else if (this.gitOrigin === "GitLab") + repoIsPublic = (await (await fetch(`${this.gitOriginURL}info/refs?service=git-upload-pack`)).json()); + } catch { /* */} + + // try clone with https first + try { + await executeGit([ + "clone", + "--depth", "1", + "--single-branch", + "--branch", this.branch, + "--recurse-submodules", + this.gitHTTPS.toString(), + repoPath + ], true); + } catch (e) { + Object.freeze(this.gitHTTPS); + // was probably a github token error, don't try ssh + if (this.gitHTTPS.username === "oauth2") { + this.errorMessage = `Could not clone git repository ${this.gitHTTPS}: Authentication failed.\nPlease make sure the ${this.gitOrigin} access token is valid and enables read access to the repository.`; + throw e; + } + + let sshKey: string|undefined; + try { + sshKey = await this.tryGetSSHKey(); + this.logger.info("SSH public key: " + sshKey) + } catch (e) { + this.logger.info("Failed to generate SSH key: ", e) + } + + // try clone with ssh + try { + const key = sshKey ? this.gitSSH.replace(this.gitHTTPS.hostname, this.uniqueGitHostName) : this.gitSSH; + await executeGit([ + "clone", + "--depth", "1", + "--recurse-submodules", + key, + repoPath + ], true); + } catch (e) { + this.logger.error(e); + let errorMessage = `Could not clone git repository ${this.gitSSH}. Please make sure the repository is accessible by ${Datex.Runtime.endpoint.main}. You can achieve this by doing one of the following:\n\n` + + let opt = 1; + const appendOption = (option: string) => { + errorMessage += `${opt++}. ${option}\n`; + } + + if (!repoIsPublic) + appendOption(`Make the repository publicly accessible (${this.gitOrigin === "GitHub" ? new URL(`./settings`, this.gitOriginURL).toString() : new URL("./edit", this.gitOriginURL).toString()})`); + appendOption(`Pass a ${this.gitOrigin} access token with --git-token= (Generate at ${this.gitOrigin === "GitHub" ? `https://github.com/settings/personal-access-tokens/new` : new URL(`./-/settings/access_tokens`, this.gitOriginURL)})`) + if (sshKey) + appendOption(`Add the following SSH key to your repository (${this.gitOrigin === "GitHub" ? new URL(`./settings/keys/new`, this.gitOriginURL) : new URL(`./-/settings/repository`, this.gitOriginURL)}): \n\n${ESCAPE_SEQUENCES.GREY}${sshKey}${ESCAPE_SEQUENCES.RESET}\n`); + this.errorMessage = errorMessage; + throw e; + } + } + + // set debug port + let i=0; + if (!this.args) + this.args = []; + for (const arg of this.args??[]) { + if (arg.startsWith("--inspect")) { + this.debugPort = arg.match(/\:(\d+)$/)?.[1] ?? "9229"; + this.args[i] = `--inspect=0.0.0.0:${this.debugPort}` + } + i++; + } + + // copy dockerfile + const dockerfile = await this.getDockerFileContent(); + await Deno.writeTextFile(dockerfilePath, dockerfile); + + // also remove docker container + docker image with same name remove to make sure + await Container.removeContainer(this.container_name); + await Container.removeImage(this.image); + + // create docker container + // TODO: --build-arg uix_args="${this.args?.join(" ")??""}" + await executeDocker([ + "build", + "-f", dockerfilePath, + "--build-arg", `stage=${this.stage}`, + "--build-arg", `host_endpoint=${Datex.Runtime.endpoint}`, + "-t", + this.image, + dir + ], true) + + // remove tmp dir + await Deno.remove(dir, { recursive: true }); + + // enable traefik routing + if (config.enableTraefik) { + for (const [domain, port] of Object.entries(domains)) { + this.enableTraefik(domain, port); + } + this.addEnvironmentVariable("UIX_HOST_DOMAINS", Object.keys(domains).join(",")); + } + // expose port 80 + else { + this.exposePort(80, config.hostPort) + } + + // add persistent volume for datex cache + await this.addVolume(this.formatVolumeName(`${this.container_name}-datex-cache`), '/datex-cache'); + + // add persistent volume for deno localStoragae + await this.addVolume(this.formatVolumeName(`${this.container_name}-localstorage`), '/root/.cache/deno/location_data'); + + // // add volume for host data, available in /app/hostdata + // this.addVolumePath('/root/data', '/app/hostdata') + } catch (e) { + this.logger.error(e); + return false; + } + return super.handleInit(); + } + + private get sshKeyName() { + return Datex.Runtime.endpoint.main.name.replaceAll('-','_') + + '_' + this.orgName?.replaceAll('-','_').replaceAll('/','_') + + '_' + this.repoName?.replaceAll('-','_').replaceAll('/','_'); + } + + private get sshKeyPath() { + const homeDir = Deno.env.get("HOME"); + if (!homeDir) + throw new Error("Could not get home directory"); + return `${homeDir}/.ssh/id_rsa_${this.sshKeyName}`; + } + + private get uniqueGitHostName() { + return `git_${this.sshKeyName}`; + } + + private async tryGetSSHKey() { + const homeDir = Deno.env.get("HOME"); + const keyPath = this.sshKeyPath; + // return public key if already exists + try { + return await Deno.readTextFile(keyPath+".pub"); + } + // generate new key + catch { + + // ssh keyscan + await executeShell([ + "ssh-keyscan", + "-H", this.gitHTTPS.hostname, + ">>", `${homeDir}/.ssh/known_hosts` + ], false); + await executeShell([ + "ssh-keygen", + "-t", "rsa", + "-b", "4096", + "-N", "''", + "-C", `'${Datex.Runtime.endpoint.main.toString()}'`, + "-f", keyPath + ]); + // add to ssh/config + let existingConfig = ""; + try { + existingConfig = await Deno.readTextFile(`${homeDir}/.ssh/config`); + } catch {/* pass */} + await Deno.writeTextFile(`${homeDir}/.ssh/config`, `${existingConfig} + +Host ${this.uniqueGitHostName} + User git + Hostname ${this.gitHTTPS.hostname} + IdentityFile ${keyPath} +`); + // return public key + return await Deno.readTextFile(`${keyPath}.pub`); + } + } + + private async getDockerFileContent() { + let dockerfile = await Deno.readTextFile( + `./res/uix-app-docker/Dockerfile_v0.${this.version}` + ); + + // add uix run args + custom importmap/run path + dockerfile = dockerfile + .replace("{{UIX_ARGS}}", this.args?.join(" ")??"") + .replace("{{IMPORTMAP_PATH}}", ( + this.advancedOptions?.importMapPath ? + this.advancedOptions.importMapPath.toString() : + 'https://dev.cdn.unyt.org/importmap_compat.json' + )) + .replace("{{UIX_RUN_PATH}}", ( + this.advancedOptions?.uixRunPath ? + this.advancedOptions.uixRunPath.toString() : + 'https://cdn.unyt.org/uix@0.1.x/run.ts' + )); + + // expose port + if (this.debugPort) + dockerfile = dockerfile.replace("{{EXPOSE_DEBUG}}", "EXPOSE 9229") + else + dockerfile = dockerfile.replace("{{EXPOSE_DEBUG}}", "") + return dockerfile; + } + + override async handleOnline() { + // wait until endpoint inside container is reachable + this.logger.info(`Waiting for ${this.endpoint} to come online`); + let iterations = 0; + while (true) { + await sleep(2000); + if (await this.endpoint.isOnline()) { + this.logger.success(`Endpoint ${this.endpoint} is online`); + return true; + } + if (iterations++ > 20) { + this.logger.error(`Endpoint ${this.endpoint} not reachable"`) + return false; + } + } + } +} \ No newline at end of file diff --git a/src/container/WorkbenchContainer.ts b/src/container/WorkbenchContainer.ts new file mode 100644 index 0000000..65a809c --- /dev/null +++ b/src/container/WorkbenchContainer.ts @@ -0,0 +1,67 @@ +import { logger } from "unyt_core/datex_all.ts"; +import { Datex } from "unyt_core/mod.ts"; +import { EndpointConfig } from "../endpoint-config.ts"; +import { Container } from "./Container.ts"; +import { copy } from "https://deno.land/std@0.224.0/fs/copy.ts"; +import { executeDocker } from "../CMD.ts"; + +@sync export class WorkbenchContainer extends Container { + @property config!: EndpointConfig + + override construct(owner: Datex.Endpoint, config: EndpointConfig) { + super.construct(owner) + this.config = config; + this.name = "unyt Workbench" + } + + // custom workbench container init + override async handleInit() { + try { + const username = "user"; // TODO other usernames? + const config_exported = Datex.Runtime.valueToDatexString(this.config, true, true, true); + + this.image = 'unyt-workbench-' + this.config.endpoint.toString().replace(/[^A-Za-z0-9_-]/g,'').toLowerCase(); + + logger.info("image: " + this.image); + logger.info("config: " + config_exported); + logger.info("username: " + username); + + // create new config directory to copy to docker + const tmp_dir = `./res/config-files-${crypto.randomUUID()}`; + await copy("./res/config-files", tmp_dir, { overwrite: true }); + await Deno.writeTextFile(`${tmp_dir}/endpoint.dx`, config_exported) + + // create docker container + await executeDocker([ + "build", + "--build-arg", `username=${username}`, + "--build-arg", `configpath=${tmp_dir}`, + "-f", "./res/Dockerfile", + "-t", this.image, + "." + ]); + // remove tmp directory + await Deno.remove(tmp_dir, { recursive: true }); + } catch (e) { + this.logger.error(e); + this.logger.error("Error initializing workbench container"); + return false; + } + return super.handleInit(); + } + + override async handleOnline() { + await sleep(6000); + try { + if (!await this.config.endpoint.isOnline()) + throw new Error("Endpoint not reachable"); + } catch (e) { + this.logger.error("Workbench Endpoint not reachable"); + this.logger.error(e); + await this.stop(); // stop container again for consistant container state + return false; + } + this.logger.success("Workbench Endpoint is reachable"); + return true; + } +} \ No newline at end of file diff --git a/endpoint-config.ts b/src/endpoint-config.ts similarity index 100% rename from endpoint-config.ts rename to src/endpoint-config.ts diff --git a/tsconfig.json b/tsconfig.json deleted file mode 100644 index 92c9be1..0000000 --- a/tsconfig.json +++ /dev/null @@ -1,18 +0,0 @@ -{ - "compilerOptions": { - "module": "ES2022", - "target": "ES2022", - "noImplicitOverride":true, - "removeComments": true, - "preserveConstEnums": true, - "esModuleInterop": true, - "experimentalDecorators": true, - "emitDecoratorMetadata": true, - "useDefineForClassFields": true, - "lib": [ - "esnext", - "dom" - ] - } - } - \ No newline at end of file