diff --git a/docs/pangeo/dask_spider.ipynb b/docs/pangeo/dask_spider.ipynb index dc9d2ad..92a8cc2 100644 --- a/docs/pangeo/dask_spider.ipynb +++ b/docs/pangeo/dask_spider.ipynb @@ -2785,16 +2785,16 @@ "\n", "For this workshop, according to the Pangeo EOSC deployment, you learned how to use Dask Gateway to manage Dask clusters over Kubernetes, allowing to run our data analysis in parallel e.g. distribute tasks across several workers.\n", "\n", - "Lets now try set up your Dask cluster using HPC infrastructure with Dask jobqueue. \n", - "As Dask jobqueue is configured by default on this ifnrastructure thanks to https://github.com/RS-DAT/JupyterDaskOnSLURM/blob/main/user-guide.md#container-wrapper-for-spider-system we just installed in the last section, you just need copy the SLURMCluster configuration cell below and execute it to connect the Dask jobqueue SLURMCluster. " + "Lets now try set up your Dask cluster using HPC infrastructure with [Dask jobqueue](https://jobqueue.dask.org). \n", + "As Dask jobqueue is configured by default on this ifnrastructure thanks to [JupyterDaskOnSLURM](https://github.com/RS-DAT/JupyterDaskOnSLURM/blob/main/user-guide.md#container-wrapper-for-spider-system) we just installed in the last section, you just need drag & drop the SLURMCluster configuration cell, and execute it to connect the Dask jobqueue SLURMCluster. " ] }, { "cell_type": "markdown", - "id": "830b67ad-3a82-4dd7-8ab6-3015d12c3240", + "id": "63ef6918-baa8-42db-8cc7-a0b3eb4d71d1", "metadata": {}, "source": [ - "Make sure you use the right port (taken from the left panel), and click scale, to have several workers. " + "![Slurmcluster](slurmcluster.png)" ] }, { @@ -2908,11 +2908,38 @@ "client" ] }, + { + "cell_type": "markdown", + "id": "830b67ad-3a82-4dd7-8ab6-3015d12c3240", + "metadata": {}, + "source": [ + "Make sure you use the right port (taken from the left panel), and click scale, to have several workers. " + ] + }, + { + "cell_type": "markdown", + "id": "dad4af05-a14c-4fb8-9a46-06dc2f2bc351", + "metadata": {}, + "source": [ + "![scale_daskjobqueue](scale_daskjobqueue.png)" + ] + }, + { + "cell_type": "markdown", + "id": "aa88e469-a632-46ee-bbbc-997d8f4d9d3d", + "metadata": {}, + "source": [ + "Adding a worker corresoinds to submiting a job using Slurm to start a node running a worker. " + ] + }, { "cell_type": "code", "execution_count": 24, "id": "8631b6bc-adf9-446c-b115-4c090aacbe62", "metadata": { + "jupyter": { + "source_hidden": true + }, "tags": [] }, "outputs": [ @@ -2933,6 +2960,20 @@ "!squeue -u $USER" ] }, + { + "cell_type": "markdown", + "id": "95355523-23fe-4715-b250-5a420d801128", + "metadata": {}, + "source": [ + "