Completed the Third Phase Targets for GSOC

[im-prakher]

Targets achieved in this PR -

• Visualized the polygons as they should have been, also modified the tests for that in #0dcd13f.

• Implemented Data Validation and Improved Error Logging & Handling in #629583f and #1c39008

• Improved SheetToCSV functionality in #ca9adac

• API Key Validation done in #6842d16

• Changed the Docker container ports for visualization #ed56e5c

• Completed the documentation of YABA App #8b3c15c

Also before merging this PR please merge #63.
Kindly review @dlebauer @KristinaRiemer @Chris-Schnaufer

[KristinaRiemer]

@im-prakher whether I put in my real API key from the TERRA REF Bety or some random characters, I get this error.

And this is the pop up window after I put a key in and hit the start button. What is supposed to go in as [object Object]?

[im-prakher]

@KristinaRiemer Bety database image currently doesn't contain apikey for any user so it couldn't find it for you. Meanwhile, I will make a commit so that it displays an appropriate message for this.

Here's the issue for this
#46

[im-prakher]

@KristinaRiemer Also, you can insert the API key for any user if you want by using these comands,

1. docker exec -ti <container_id_for_postgres> bin/bash
2. su postgres
3. psql
   To get potgres container id you can use docker ps,
   You will now be accessing the postgres database
4. Switch user \c bety
5. Run the query to set the apikey for a user in users table
   update users set apikey = 'oxDrvOozw+TVV0GKYOcTPooBuHIQymBaVxV4BbRo' where name = 'user 1';
   You can set the apikey= oxDrvOozw+TVV0GKYOcTPooBuHIQymBaVxV4BbRo for user1
   as it the apikey column has some constraints of its own

[im-prakher]

@KristinaRiemer This error is now solved in #41db3ae

[im-prakher]

@dlebauer You can merge this PR and #63 as well

[KristinaRiemer]

@im-prakher I just went through this PR to see how everything worked. I'm documenting all the steps I took here, in case I did something wrong and also because there were a few issues.

I removed all of the existing containers, including the database, to prevent duplicate data errors with docker system prune -a. Then, while on the containerize branch I followed all the README instructions to get the interface and visualization containers running. I then switched over to this validate branch and ran docker-compose up to get these containers.

I opened the app and put in the API key after following these instructions, and that went great.

The three visualization pages also looked good except for one thing. When I clicked on the plots in the experiment viz, they said they were locations at range 39, but the input data in experiments_sites.csv said range 20. (See screenshot below) Why are those different?

Then after the visualization pages, I got this not iterable error on the following page:

Lastly, I checked the data validation. I redid all the steps from above to remove and recreate the containers to make sure there was no example data in the database already. I then removed the first column (name) from the cultivars.csv to see if the validation would catch that. I uploaded all the data files, but the validate page said all tables were successfully validated, though I expected an error about the cultivars table. Should there have been an error?

There was an error on the next page for the cultivars visualization:

[im-prakher]

@KristinaRiemer The difference between the ranges of plots in viz part is due to the different sitename in the experiments table and shapefiles, e.g.
In the experiments table, MAC Field Season 8 Range 20 Column 1 W
But in shapefile, it is Range 39 Column 1
This is happening as the table contains East and West for 2 polygons with the same range and column but in shapefile, there is no east and west, only ranges and columns so 2 polygons with the same range and column in experiments will have different range and column in shapefile.
And the viz part is using the shapefile for displaying the plots name.

We have discussed this type of situation during the GSoC meeting and we planned to get the sitename for plots from shapefile

[im-prakher]

The iterable error that occurred for three tables is because the insert_sites route isn't working properly and all these tables are dependent on the sites table which must be uploaded before uploading them. Once the sites route is fixed, it will work.

[im-prakher]

I'll look into the validation error, it must be something about the yaba_api code in python

[Chris-Schnaufer]

Added some comments for improving documentation, and optional suggestions for code improvements

[KristinaRiemer]

@im-prakher great, thank you for the explanations about the plot names and iterable error!

With respect to the validation error, the error shown in the screenshot is correct, I think. The problem is that the validation page said that all of the tables were valid, even though I had purposely changed the cultivars table so that it wasn't valid. The validation page should return that there is something wrong with the cultivars table, I believe?