You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Having used the advanced csv import feature multiple times for various different departments now I have a few things I think we can improve on the tool.
Automatically create job titles that do not exist yet, instead of failing. They can just be created in a random order, or maybe based on how many values we have per job title. By default all should be marked "sworn". An admin or area coordinator can then sort them in the correct order with the web ui (the order also doesn't seem to have any impact expect for the sort order in the selection menu).
Instead of providing unit-id just provide unit names and create all the units that are provided in the data set. Currently we have no way of creating units via commands and relying on unit-ids means generating different csv files for the different environments (like local dev, staging and production).
Add another flag for local development that will truncate the existing tables for the given department
Either relax the condition on the department name column when having --force-create set or add a department name column to the csv download. Also make sure that all other fields of the csv download match fields expected by the csv import command with the goal that downloading the csv files and then importing locally gives you a basically exact copy of the production data for that department
Dive deeper into different fields and decide if they can actually be optional or not. We have a lot of fields in our tables that are allowed to be NULL but if you actually set them NULL some script breaks because it can't handle None.
For example
May 04 04:07:33 staging gunicorn[1350]: {% for type, list in obj.links|groupby('link_type') %}
May 04 04:07:33 staging gunicorn[1350]: File "[..]/lib/python3.5/site-packages/jinja2/filters.py", line 852, in do_groupby
May 04 04:07:33 staging gunicorn[1350]: in groupby(sorted(value, key=expr), expr)]
May 04 04:07:33 staging gunicorn[1350]: TypeError: unorderable types: NoneType() < NoneType()
A result of not setting link_type.
The text was updated successfully, but these errors were encountered:
Having used the advanced csv import feature multiple times for various different departments now I have a few things I think we can improve on the tool.
Automatically create job titles that do not exist yet, instead of failing. They can just be created in a random order, or maybe based on how many values we have per job title. By default all should be marked "sworn". An admin or area coordinator can then sort them in the correct order with the web ui (the order also doesn't seem to have any impact expect for the sort order in the selection menu).
Instead of providing unit-id just provide unit names and create all the units that are provided in the data set. Currently we have no way of creating units via commands and relying on unit-ids means generating different csv files for the different environments (like local dev, staging and production).
Add another flag for local development that will truncate the existing tables for the given department
Either relax the condition on the department name column when having
--force-create
set or add a department name column to the csv download. Also make sure that all other fields of the csv download match fields expected by the csv import command with the goal that downloading the csv files and then importing locally gives you a basically exact copy of the production data for that departmentDive deeper into different fields and decide if they can actually be optional or not. We have a lot of fields in our tables that are allowed to be
NULL
but if you actually set themNULL
some script breaks because it can't handleNone
.For example
A result of not setting
link_type
.The text was updated successfully, but these errors were encountered: