Skip to content

Commit

Permalink
Merge pull request #461 from JohannesARENA2036/patch-1
Browse files Browse the repository at this point in the history 
doc(tutorials): Adaptation new structure for tutorial
  • Loading branch information
@stephanbcbauer
stephanbcbauer authored Nov 9, 2023
2 parents 9d0a907 + 0591274 commit e20c6f3
Show file tree
Hide file tree
Showing 3 changed files with 237 additions and 69 deletions.
4 changes: 2 additions & 2 deletions docs/tutorials/e2e/boost/provideData.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ curl --location 'http://localhost/bob/management/v2/policydefinitions' \
}'
```

## Request Bob's catalog
## Request catalog - second try

Now that Bob created an access policy, Alice can once again try to access Bob's asset.

Expand Down Expand Up @@ -148,7 +148,7 @@ curl --location 'http://localhost/bob/management/v2/contractdefinitions' \
}'
```

## Request catalog
## Request catalog - third try

Let´s see if Alice can finally see the Asset.
Action (Alice): Execute the request again using the following `curl` command:
Expand Down
27 changes: 25 additions & 2 deletions docs/tutorials/e2e/connect/deployComponents.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,17 @@ once and are accessible by all participants.
For the most bare-bones installation of the dataspace, execute the following commands in a shell:

```shell
# get the tutorial including the config file for the cluster by cloning the repository locally
git clone https://github.com/eclipse-tractusx/tutorial-resources.git
# will install the mxd directory under <current working directory>/tutorial-resources/mxd
# the directory should contain the config file for kind: kind.config.yaml
cd <path/of/mxd>
kind create cluster -n mxd --config kind.config.yaml
#
# Now we activate ingress for the later port forwarding ?
# the next step is specific to KinD and will be different for other Kubernetes runtimes!
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/kind/deploy.yaml
kubectl apply -f \
https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/kind/deploy.yaml
# wait until the ingress controller is ready
kubectl wait --namespace ingress-nginx \
--for=condition=ready pod \
Expand Down Expand Up @@ -80,7 +87,23 @@ Naturally there are several ways to enable access to those services (Load balanc
of simplicity we will use a plain Kubernetes port-forwarding:

```shell
kubectl port-forward postgres-5b788f6bdd-bvt9b 5432:5423
# find out about the names of your pods
kubectl get pods
# you should see a list like:
NAME READY STATUS RESTARTS AGE
alice-tractusx-connector-controlplane-6f7555bcb7-mhxh6 1/1 Running 0 4m
alice-tractusx-connector-dataplane-6db78d69c5-sw87d 1/1 Running 0 4m
alice-vault-0 1/1 Running 0 4m
bob-tractusx-connector-controlplane-76ccd79946-84zjw 1/1 Running 0 4m
bob-tractusx-connector-dataplane-85554c7dcd-6pzzs 1/1 Running 0 4m
bob-vault-0 1/1 Running 0 4m
keycloak-6bdf4d7689-8mwfd 1/1 Running 0 4m58s
miw-574bf87bc-c4n6c 1/1 Running 0 4m58s
postgres-66677b8665-lxxp2 1/1 Running 0 5m14s
#
# We will use the last entry for ingress for our port-forwarding, the name of the postgres pod will be slightly differnt in your local cluster.
# now we activtae the kuberntes port-forwarding:
kubectl port-forward postgres-66677b8665-lxxp2 5432:5423
```

> Note that the actual pod name will be slightly different in your local cluster.
Expand Down
275 changes: 210 additions & 65 deletions docs/tutorials/e2e/prerequisites/prerequisites.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,9 @@ title: Prerequisites
sidebar_position: 1
---


## Preface

The components and tools that are described here are to be understood as a proposal from Catena-X and not as standards that must be used. Using the proposed software stack of this tutorial will make it easier to complete it. Nevertheless, if your company policy requires e.g. the use of open stack, or you can't use docker containers or kubernetes, the Catena-X components will also work, you might just need a little extra effort.
The components and tools that are described here are to be understood as a proposal from Tractus-X and not as standards that must be used. Using the proposed software stack of this tutorial will make it easier to complete it. Nevertheless, if your company policy requires e.g. the use of open stack, or you can't use docker containers or kubernetes, the Catena-X components will also work, you might just need a little extra effort.

:::info

Expand All @@ -16,89 +15,235 @@ You can either complete the tutorial in a cloud space (e.g. AWS or Azure) or loc

## Required Skills and Technologies

AAs mentioned in the introduction, no preliminary knowledge about Catena-X is required. However, to complete the tutorial you will have to work with the following tech stack. A basic understanding of those technologies is advised.
As mentioned in the introduction, no preliminary knowledge about Catena-X is required. However, to complete the tutorial you will have to work with the following technical software stack. A basic understanding of those technologies is advised.

- Cloud Environment (AWS/Azure) or Local Machine with at least 16GB RAM, 20GB free storage
- Cloud Environment (AWS/Azure) or a local machine with at least 16GB RAM, 20GB free storage
- Docker
- Kubernetes
- Minikube (for local)
- Helm
- Kind (for local systems)
- Helm (will be installed by Terraform)
- Terraform

## Using cloud based environments
## Chosing your environment

We recommend to use cloudbased environmengts like (AWS, Google or Azure) for most cases.
The tutorial is designed to be used in cloud environments, such as AWS, Google or Azure. If you intendend to build your own local environment independly from Cloud based offers, you may use ths tutorial as well, but you need to ensure, you have the rigth technical software stack installed, see below.

## Setting up your own environment on local systens
## Setting up your own environment on local systems

IIn case you want to install Catena-X components or Kits directly on your local system you need the following:
In case you want to install Catena-X components or Kits directly on your local system you need the following:

- Access to the internet (see next section)
- One local server instance, either a physical server or a virtual machine with at least 2 CPUs 16 GB menory and 20 GB storage
- Your local system should run a Linux Version (Debian or Ubuntu are recommended)
- You need super user privileges (either root access or the right to use sudo)
- Access to the internet (see next section)
- The above tools should be installed (Docker, Kubernetes, Kind, Helm, Terraform)

### Access to the Internet

Most companies run are more complex network, usually direct open access to the Internet is not allowed. Firewalls and Proxy server isolate the local network and most communication links are blocked (by blocking ports and external URLs).
Most companies run are more complex network, usually direct open access to the Internet is not allowed. Firewalls and proxy server isolate the local network and most communication links are blocked (by blocking ports and external URLs).

Before you start configuring your environment ensure that you know your

- network addresses
- routes (and router) as well as gateways
- proxy server

You may need the support of your local IT department. Once you are confident to understand the network layout you can start to configure (or ask to configure) the different ports and URLS which are required for the installation process. For the implemented installation process as configured you will need access to a couple of URLs for different repositories. Those URLs will be passed only if your proxy server is configured to forward the following whitelist of URLs
You may need the support of your local IT department. Once you are confident to understand the network layout you can start to configure (or ask to configure) the access to required resources (repositories) for the installation process. For the implemented installation process as configured you will need access to a couple of URLs for different repositories:

### URL Whitelist

#### Specific URLs​

- .download.docker.com​
- .k8s.io​
- .io/v2​
- .charts.bitnami.com​
- .download.docker.com/linux/ubuntu/gpg​
- .storage.googleapis.com​
- .hub.docker.com # Docker Hub​
- .kubernetes.io # Kubernetes​
- .helm.sh # Helm Charts​
- .hashicorp.com​
- .github.io​

#### General URLs​

- .access.redhat.com/solutions/15840 # yum​
- .apps.fedoraproject.org/packages # dnf​
- .software.opensuse.org # zypper​
- .snapcraft.io # snap​
- .pypi.org # pip​
- .packagist.org # composer​
- .deb.debian.org # Debian Package Updates​
- .mirrors.centos.org # CentOS Package #Updates​
- .mirrors.fedoraproject.org # Fedora Package Updates​
- .download.opensuse.org # openSUSE Package Updates​
- .packages.debian.org # Debian Package Repositories​
- .archlinux.org # Arch Linux Package Repositories​
- .packages.gentoo.org # Gentoo Package Repositories​
- .pkgs.org # RHEL/CentOS Package Repositories​
- .kernel.org # Linux Kernel Updates​
- .git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git # Linux Firmware Updates​
- .sourceforge.net # SourceForge​
- .npmjs.com # Node Package Manager (NPM)​
- .rubygems.org # RubyGems​
- .ubuntu.com # Ubuntu Package Updates​
- .github.com​
- .raw.githubusercontent.com​

TODO: Add list s:

- Ports to be open

## Notice

This work is licensed under the [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/legalcode).

- SPDX-License-Identifier: CC-BY-4.0
- SPDX-FileCopyrightText: 2023 sovity GmbH
- SPDX-FileCopyrightText: 2023 SAP SE
- SPDX-FileCopyrightText: 2023 msg systems AG
- Source URL: [https://github.com/eclipse-tractusx/eclipse-tractusx.github.io](https://github.com/eclipse-tractusx/eclipse-tractusx.github.io)
#### Specific URLs​, required for MxD

```bash
.download.docker.com​
.k8s.io​
.io/v2​
.charts.bitnami.com​
.download.docker.com/linux/ubuntu/gpg​
.storage.googleapis.com​
.hub.docker.com # Docker Hub​
.kind.sigs.k8s.io # Kind
.kubernetes.io # Kubernetes​
.helm.sh # Helm Charts​
.hashicorp.com​
.github.io​
apt.releases.hashicorp.com
```

#### General URLs​, may be used for further components of the EDC

```bash
.access.redhat.com/solutions/15840 # yum​
.apps.fedoraproject.org/packages # dnf​
.software.opensuse.org # zypper​
.snapcraft.io # snap​
.pypi.org # pip​
.packagist.org # composer​
.deb.debian.org # Debian Package Updates​
.mirrors.centos.org # CentOS Package #Updates​
.mirrors.fedoraproject.org # Fedora Package Updates​
.download.opensuse.org # openSUSE Package Updates​
.packages.debian.org # Debian Package Repositories​
.archlinux.org # Arch Linux Package Repositories​
.packages.gentoo.org # Gentoo Package Repositories​
.pkgs.org # RHEL/CentOS Package Repositories​
.kernel.org # Linux Kernel Updates​
.git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git. # Linux Firmware Updates​
.sourceforge.net # SourceForge​
.npmjs.com # Node Package Manager (NPM)​
.rubygems.org # RubyGems​
.ubuntu.com # Ubuntu Package Updates​
.github.com​
.raw.githubusercontent.com​
```

:::note

The above list is currently a candidate for changes, especially as long as the used installations scripts and tools are still in development.

:::

### Ports

#### https (443)

You will need https (port 443) as open port for getting access to the above repositories. If you do not have direct access from your system, you most likely work in an environment which is using proxy forwarding for https. An easy way to configure your system to use the proxy server is by setting the envionment variabale "https_proxy". For example with the command below (bash), if the port 8080 is used for the forwarding:

```bash
export https_proxy=http://arena2036-proxy.rus.uni-stuttgart.de:8080
```

The complete format is:

```bash
export https_proxy=http://[username]:[password]@ [proxy-web-or-IP-address]:[port-number]
```

:::tip

The above URLs then will be passed only if your proxy server is configured to forward the above whitelist of URLs. To ensure your setting is permant, you may want to add the above command in your .bashrc or /etc/environment. Futher you can configure apt to use the proxy by entering the following into the configuration file /etc/apt/apt.conf:

```bash
Acquire::https::Proxy "http://[username]:[password]@ [proxy-web-or-IP-address]:[port-number]";
```

To ensure, that the local access is not forwarded, you should set NO_PROXY as environment variable or in your .bashrc or in /etc/environment.

```bash
export NO_PROXY="localhost,127.0.0.1,::1"
```

:::

#### http (80)

The port http (80) should not be used, but in case it will, you can apply the above hints just by replacing https by http.

#### ssh (22)

For the MxD, which is running locally, you only need secure shell access, which means port 22 should be open.

### Install the basic tools (on Ubuntu 20.x and higher)

Within this section we briefly describe how to install the required tools on an Ubuntu system. We have tested this on 22.04.3 LTS (GNU/Linux 5.15.0-86-generic x86_64). Please check the online availabe documentation for further details.

#### Install docker

```bash
sudo apt install docker.io
```

#### Install kubernetes

```bash
sudo apt install kubernetes
```

#### Install kubectl

```bash
sudo snap install --classic kubectl
```

#### Install Kind

:::note

It might be the case, not all of these steps are needed on your machine.

:::

##### Install go

```bash
sudo apt install golang-go
```

##### Install kind

```bash
sudo go install sigs.k8s.io/[email protected]
```

##### Check go version

```bash
go version
``````

:::note

```bash
Response should be: "go version go1.18.1 linux/amd64"
```

:::

#### Terraform

Prepare the installation of Terraform including helm:

```bash
sudo apt-get update && sudo apt-get install -y gnupg software-properties-common
sudo wget -O- https://apt.releases.hashicorp.com/gpg | gpg --dearmor | sudo tee /usr/share/keyrings/hashicorp-archive-keyring.gpg
```

Verify the generated key is working:

```bash
gpg --no-default-keyring \
--keyring /usr/share/keyrings/hashicorp-archive-keyring.gpg \
--fingerprint
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] \
https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
```

Now install Terraform, which automatically also will install helm.

```bash
sudo apt-get install terraform
```

#### Webbrowser

:::note

If not already installed, install a webbroser like firefox or goggle-chrome, you also may need to install an x-environment (xterm).

:::

Check if you can access your system by using

```bash
ssh -X <your system>
```

Then you should be able to run the webrowser locally to open the links given later in the tutorial.

:::info

Your enviroment for the tutorial starting with chapter "deploy" should be ready.

:::

0 comments on commit e20c6f3

Please sign in to comment.