Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scripts for managing and processing logs parameterized #21

Open
wants to merge 74 commits into
base: melodic-devel
Choose a base branch
from

Conversation

rayvburn
Copy link
Owner

@rayvburn rayvburn commented Aug 10, 2023

@rayvburn rayvburn force-pushed the eval-scripts-parameterization branch from 8106421 to 9ea6909 Compare September 25, 2023 13:01
- Aim is to avoid copying each script to log directories. Instead, all logs now take parameters that define where are the logs etc.
- Deleted `evaluate_newest` as its functionality would be the same as `evaluate_from_dir`'s after parameterization
@rayvburn rayvburn force-pushed the eval-scripts-parameterization branch from 9ea6909 to 72cc224 Compare September 25, 2023 13:07
…able_from_results` script

- added note about a possible cause of None value appearing in a non-empty cell
…ics map, putting a newline between metric and its unit dynamically decided
@rayvburn rayvburn force-pushed the eval-scripts-parameterization branch 2 times, most recently from b651bfb to e7b88c6 Compare October 8, 2023 11:50
@rayvburn rayvburn force-pushed the eval-scripts-parameterization branch from e7b88c6 to 7cb7af5 Compare November 21, 2023 19:49
…ncrease robustness when a corrupted or incomplete data is present, support for bigger datasets (column numbering)
…ded into parts but related to the same benchmark session
…r the robot and people data

- with a longer computation times there might be a slight timing offset between, e.g. robot and people data
…atures, updated automation scripts [#25]

- added `GlobalPlannerLogger` and `GlobalPlannerData` classes
- added unit tests evaluating the conversion to/from `GlobalPlannerData` from/to `string`
- updated `evaluation` scripts
…le [#25]

- adjusted `evaluate_from_dir` script invoking the executable
@rayvburn rayvburn force-pushed the eval-scripts-parameterization branch from 7cb7af5 to ac5ea9d Compare December 6, 2023 11:21
- leaves the already renamed directories untouched
- assigns the ID regarding the list of existing, already renamed directories
…metric-specific filtering function from `SrpbMetrics`
…lso raw values from the sheet (not only filtered); updated `latex_table_from_results`
…oords_to_sheet_cell_id` fun. uses `utils` now)

- this also seems to fix omitting some entries while creating a spreadsheet from the big amount of data
…from_excel` extracted as separate functions
…ics related to unfinished trials (relies on `create_excel_from_results`)
…rocessing of the spreadsheet; adjusted related script `remove_failed_trials_from_excel`
…ed keys; extended to handle multiple input spreadsheets (data merged internally)
… input spreadsheets (data merged internally)
…calculated column arrangement and "_" in planner names replaced with "_\"
…ssociating multiple spreadsheets to a single scenario
@rayvburn rayvburn force-pushed the eval-scripts-parameterization branch from b39a6cb to 96e3cbe Compare June 3, 2024 18:43
@rayvburn
Copy link
Owner Author

rayvburn commented Jun 29, 2024

FIXME: create_bar_plot_from_results.py script does not handle multiple input sheets well. Traceback:

Traceback (most recent call last):
  File "srpb_evaluation/scripts/create_bar_plot_from_results.py", line 260, in <module>
    metric_name, planner_values = prepare_data_for_plotting(data_selected)
  File "srpb_evaluation/scripts/create_bar_plot_from_results.py", line 58, in prepare_data_for_plotting
    ] for planner in planner_names
  File "srpb_evaluation/scripts/create_bar_plot_from_results.py", line 58, in <dictcomp>
    ] for planner in planner_names
  File "srpb_evaluation/scripts/create_bar_plot_from_results.py", line 57, in <listcomp>
    for metric in metric_names
TypeError: 'NoneType' object is not subscriptable

EDIT: it might not be the case of loading multiple sheets but rather loading sheets with invalid data (e.g., all trials failed)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant