Implement web app dashboard to present the results of executing LST-B…

…ench automatically (#234)

Fixes #239

.github/workflows/webapp-deploy.yaml

@@ -0,0 +1,108 @@
+# Copyright (c) Microsoft Corporation.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Docs for the Azure Web Apps Deploy action: https://github.com/Azure/webapps-deploy
+# More GitHub Actions for Azure: https://github.com/Azure/actions
+# More info on Python, GitHub Actions, and Azure App Service: https://aka.ms/python-webapps-actions
+
+name: Build and deploy Web App - lst-bench
+
+on:
+  push:
+    paths:
+      - metrics/**
+      - run/**
+    branches:
+      - main
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+env:
+  AZURE_WEBAPP_NAME: lst-bench
+  WORKING_DIRECTORY: './metrics/app'
+  STARTUP_COMMAND: 'python -m streamlit run main.py --server.port 8000 --server.address 0.0.0.0 --client.toolbarMode minimal'
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: 'Set up Python version'
+        uses: actions/setup-python@v1
+        with:
+          python-version: '3.11'
+
+      - name: 'Create and start virtual environment'
+        working-directory: ${{ env.WORKING_DIRECTORY }}
+        run: |
+          python -m venv venv
+          source venv/bin/activate
+
+      - name: 'Install dependencies'
+        working-directory: ${{ env.WORKING_DIRECTORY }}
+        run: |
+          pip install setuptools
+          pip install -r requirements.txt
+
+      - name: 'Copy .duckdb files from ./run/'
+        run: |
+          find ./run -type f -name "*.duckdb" -exec cp {} ${{ env.WORKING_DIRECTORY }} \;
+
+      - name: Zip artifact for deployment
+        working-directory: ${{ env.WORKING_DIRECTORY }}
+        run: zip release.zip ./* -r
+
+      - name: Upload artifact for deployment jobs
+        uses: actions/upload-artifact@v3
+        with:
+          name: python-app
+          path: |
+            ${{ env.WORKING_DIRECTORY }}/release.zip
+
+  deploy:
+    runs-on: ubuntu-latest
+    needs: build
+    environment:
+      name: 'webapp-deploy'
+      url: ${{ steps.deploy-to-webapp.outputs.webapp-url }}
+    permissions:
+      id-token: write #This is required for requesting the JWT
+
+    steps:
+      - name: Download artifact from build job
+        uses: actions/download-artifact@v3
+        with:
+          name: python-app
+          path: .
+
+      - name: Unzip artifact for deployment
+        run: unzip release.zip
+
+      - name: Login to Azure
+        uses: azure/login@v1
+        with:
+          client-id: ${{ secrets.AZURE_CLIENT_ID }}
+          tenant-id: ${{ secrets.AZURE_TENANT_ID }}
+          subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
+
+      - name: 'Deploy to Azure Web App'
+        uses: azure/webapps-deploy@v3
+        id: deploy-to-webapp
+        with:
+          app-name: ${{ env.AZURE_WEBAPP_NAME }}
+          startup-command: ${{ env.STARTUP_COMMAND }}

metrics/app/README.md

@@ -0,0 +1,72 @@
+<!--
+{% comment %}
+Copyright (c) Microsoft Corporation.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+{% endcomment %}
+-->
+
+# LST-Bench: Dashboard
+
+**Dashboard:** [https://lst-bench.azurewebsites.net/](https://lst-bench.azurewebsites.net/)
+
+The LST-Bench dashboard is powered by [Streamlit](https://github.com/streamlit/streamlit) and deployed to Azure App Service through GitHub actions. 
+You can find the deployment workflow [here](/.github/workflows/webapp-deploy.yaml). 
+The dashboard provides insights derived from metrics collected from LST-Bench, including execution time and degradation rate. 
+
+## Evaluation
+The results displayed in the dashboard are specific to the versions and configurations we tested. 
+Their performance is subject to change and improvement through further tuning and future developments.
+Thus, the primary aim of sharing them is not to assert that one LST or engine is superior (in terms of speed, cost, etc.) to another. 
+Instead, it is to showcase LST-Bench's capability in quantifying significant trade-offs across various combinations of engines and LSTs. 
+Further details about the runs and setups are available [here](/run).
+
+## Adding a New Result
+To include data from a new system, duplicate one of the directories in the [run folder](/run) and modify the necessary files within. 
+For a deeper understanding of the directory structure, consult the [README file](/run/README.md). 
+The LST-Bench dashboard web app automatically retrieves results from the .duckdb files within those folders and displays them on the dashboard.
+
+## Dashboard Development
+To run the LST-Bench dashboard locally and test your changes, follow these steps:
+
+### 1. Set up Python version
+Ensure you have Python version 3.11 installed on your system. If not, you can download and install it from the official Python website.
+
+### 2. Create and Start Virtual Environment
+To isolate the dependencies of the LST-Bench dashboard, it's recommended to use a virtual environment. You can create one by running the following command in your terminal:
+
+```bash
+python -m venv venv
+```
+
+Once the virtual environment is created, activate it by executing:
+
+```bash
+source venv/bin/activate
+```
+
+### 3. Install Dependencies
+Install the the necessary packages specified in the requirements.txt using pip:
+
+```bash
+pip install -r requirements.txt
+```
+
+### 4. Execute Streamlit App
+With the dependencies installed, you can now start the Streamlit app by running the following command:
+
+```bash
+python -m streamlit run main.py
+```
+
+This command will launch the LST-Bench dashboard locally in your browser.