Merge branch 'main' into dem-development

.condarc

@@ -0,0 +1,4 @@
+channels:
+  - conda-forge
+  - nodefaults
+channel_priority: strict

.github/workflows/run-pytest.yml

@@ -14,14 +14,15 @@ jobs:
       with:
         generate-run-shell: true
         environment-file: environment.yml
-        condarc: |
-          channels:
-            - conda-forge
+        condarc-file: .condarc
     - name: Check packages
       shell: micromamba-shell {0}
       run: |
-        python --version
         pytest --version
+    - name: Install sar-antarctica package
+      shell: micromamba-shell {0}
+      run: |
+        pip install .
     - name: Run tests
       shell: micromamba-shell {0}
       run: |

README.md

@@ -1,65 +1,11 @@
-# sar-pyrosar-nci
+# sar-antarctica
 
-This repository is designed to be used inside a specific project directory on the NCI, located at `/g/data/yp75/projects/pyrosar_processing/`
+This repository contains code for running SAR processing pipelines on the NCI.
 
-The `pyrosar_processing` folder already contains:
+## Environment set up
 
-- a micromamba install with an environment called `pyrosar_rtc`
-- folders for storing processed data
-    - `data/gamma` for GAMMA processing
-    - `data/snap` for SNAP processing
-- folder for codebases (to review as reference)
-    - `codebases/pyroSAR`
-- a clone of the [`sar-pyrosar-nci` repository](https://github.com/GeoscienceAustralia/sar-pyrosar-nci)
-- a clone of the [`s1-rtc-pyrosar-notebook` repository](https://github.com/abradley60/s1-rtc-pyrosar-notebook)
-- symbolic links to shared objects required by GAMMA
-    - `libfftw3f_GNU.so.3 -> /apps/gdal/3.6.4/lib64/libgdal.so.32`
-    - `libgdal.so.20 -> /apps/fftw3/3.3.10/lib/libfftw3f_GNU.so.3`
+For more information see [Environment set up](docs/setup/README.md)
 
-## First time set up
+## Workflows
 
-### Add the project's micromamba to your environment variables
-1. Add the following lines to your `$HOME/.bashrc` file:
-```
-# >>> mamba initialize >>>
-# !! Contents within this block are managed by 'micromamba shell init' !!
-export MAMBA_EXE='/g/data/yp75/projects/pyrosar_processing/.local/bin/micromamba';
-export MAMBA_ROOT_PREFIX='/g/data/yp75/projects/pyrosar_processing/micromamba';
-__mamba_setup="$("$MAMBA_EXE" shell hook --shell bash --root-prefix "$MAMBA_ROOT_PREFIX" 2> /dev/null)"
-if [ $? -eq 0 ]; then
-    eval "$__mamba_setup"
-else
-    alias micromamba="$MAMBA_EXE"  # Fallback on help from micromamba activate
-fi
-unset __mamba_setup
-# <<< mamba initialize <<<
-```
-2. Source the file to apply the changes to the current session.
-```
-source .bashrc
-```
-
-This is only required once. After the first time, your `.bashrc` file will be run anytime you start a new session on NCI.
-
-### Initialise pyroSAR's python API for GAMMA
-If this is the first time you are using this repository, you must build `pyroSAR`'s Python API for GAMMA. This will be installed in your home directory on the NCI.
-
-1. Activate the micromamba environment
-```
-micromamba activate pyrosar_rtc
-```
-2. Run the `initialise_gamma.py` script
-```
-python initialise_gamma.py
-```
-3. Check that you have the following in your home directory:
-```
-.pyrosar/
-└── gammaparse
-    ├── diff.py
-    ├── disp.py 
-    ├── __init__.py
-    ├── isp.py
-    ├── lat.py
-    └── __pycache__
-```
+For more information see [Workflows](docs/workflows/README.md)

docs/setup/README.md

@@ -0,0 +1,4 @@
+## Environment set up
+
+- If you are a developer, see [Developer set up](developer.md)
+- If you want to use the repository in NCI jobs, see [NCI Module set up](nci.md)

docs/setup/developer.md

@@ -0,0 +1,48 @@
+# Developer set up
+On the NCI, complete these steps on a login node, as the installs require internet access.
+
+## Clone the repository
+On the NCI, navigate to the place you want to keep your development. Suggestions are:
+- home directory: `~/`
+- user directory in project: `/g/data/yp75/<nci-username>/`
+
+Clone the repository into the chosen directory using
+```
+git clone https://github.com/GeoscienceAustralia/sar-antarctica.git
+```
+
+## Install micromamba in the current directory:
+Run `"${SHELL}" <(curl -L https://micro.mamba.pm/install.sh)` and provide the following answers to the prompts
+
+- *Micromamba binary folder? [~/.local/bin]* `./micromamba/bin`
+- *Init shell (bash)? [Y/n]* `Y`
+- *Configure conda-forge? [Y/n]* Y
+- *Prefix location? [~/micromamba]* ./micromamba
+
+This will add this version of micromamba to your `~/.bashrc` file, so that it is initialised when you log in to NCI.
+
+Run `source ~/.bashrc` to initialise the micromamba environment. This will be done automatically on future log ins.
+
+Run `micromamba env list` to check if micromamba has been set up correctly. You should see:
+```
+  Name            Active  Path                                                                  
+────────────────────────────────────────────────────
+  base            *       /<install-path>/micromamba             
+```
+
+## Install the required packages using environment.yml
+Change directories to the cloned repository directory, then run
+```
+micromamba create -f environment.yml 
+```
+
+Activate the environment using
+```
+micromamba activate sar-antarctica
+```
+
+## Pip install the repository code
+Once the environment is activated, install the source code in editable mode using
+```
+pip install -e .
+```

docs/setup/nci.md

@@ -0,0 +1,4 @@
+# NCI Module set up
+When looking to run code from this package in NCI jobs, it is possible to load a stable version of the code that has been installed into the NCI as a module. 
+
+> **_NOTE:_**  This is a work in progress and has not been finalised.

docs/workflows/README.MD

@@ -0,0 +1,4 @@
+## Workflows
+The repository will handle multiple workflows and backends, some of which have specific requirements.
+
+- [pyroSAR + GAMMA](pyrosar_gamma.md)

docs/workflows/pyrosar_gamma.md

@@ -0,0 +1,41 @@
+# pyroSAR + GAMMA
+> **_NOTE:_**  This is a work in progress and has not been finalised.
+
+Ensure there are symbolic links in the source code directory:
+
+Load required shared objects for GAMMA binaries by running
+```
+module load gdal
+module load fftw3
+```
+
+Move to project directory
+`cd <path/to/source/code>`
+
+Create symlinks in project directory by running
+```
+ln -s /apps/gdal/3.6.4/lib64/libgdal.so.32 libgdal.so.20
+ln -s /apps/fftw3/3.3.10/lib/libfftw3f_GNU.so.3
+```
+
+Activate the python env by running 
+```
+micromamba activate sar-antarctica
+```
+
+Run the one-time script to create pyroSAR's python api using 
+```
+python initialise_gamma.py
+```
+
+Check that you have the following in your home directory:
+```
+.pyrosar/
+└── gammaparse
+    ├── diff.py
+    ├── disp.py 
+    ├── __init__.py
+    ├── isp.py
+    ├── lat.py
+    └── __pycache__
+```

environment.yml

@@ -104,7 +104,6 @@ dependencies:
       - pytest==8.3.4
       - python-dateutil==2.9.0.post0
       - pytz==2024.2
-      - sar-antarctica==0.1
       - six==1.17.0
       - tomli==2.2.1
       - tzdata==2024.2