Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validation error for Sample pid field when ICAT value is None #324

Merged
merged 24 commits into from
Feb 11, 2022

Conversation

VKTB
Copy link
Contributor

@VKTB VKTB commented Feb 4, 2022

This PR will close #314

Description

The changes in this PR add an alternative ICAT mapping for the PaNOSC Sample pid field. For ISIS, the pid values of some ICAT sample are None but because the PaNOSC pid field is defined as mandatory, pydantic throws a validation error. It was decided that when pid of an ICAT sample is None, its id to instead be taken and used in setting the value of the PaNOSC pid field in the pid:<id> format. As this is being set at the Search API level, a logic was needed to ensure that the relevant ICAT field is queried with the value in the right format when WHERE filter is applied on the PaNOSC pid field.

Testing Instructions

  • Review code
  • Check GitHub Actions build
  • If icatdb Generator Script Consistency Test CI job fails, is this because of a deliberate change made to the script to change generated data (which isn't actually a problem) or is here an underlying issue with the changes made?
  • Review changes to test coverage
  • Does this change mean a new patch, minor or major version should be made? If so, does one of the commit messages feature fix:, feat: or BREAKING CHANGE: so a release is automatically made via GitHub Actions upon merge?
  • No validation error should be raised
  • http://{{datagateway-api}}/search-api/datasets?filter={"include": [{"relation": "samples"}]} should return all datasets with their samples

@codecov
Copy link

codecov bot commented Feb 4, 2022

Codecov Report

Merging #324 (35569fb) into master (80222ee) will increase coverage by 0.00%.
The diff coverage is 83.33%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #324   +/-   ##
=======================================
  Coverage   92.62%   92.62%           
=======================================
  Files          43       43           
  Lines        3199     3215   +16     
  Branches      318      321    +3     
=======================================
+ Hits         2963     2978   +15     
  Misses        198      198           
- Partials       38       39    +1     
Impacted Files Coverage Δ
datagateway_api/src/search_api/filters.py 89.15% <71.42%> (-0.46%) ⬇️
datagateway_api/src/search_api/models.py 96.69% <100.00%> (+0.16%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 80222ee...35569fb. Read the comment docs.

@VKTB VKTB marked this pull request as ready for review February 4, 2022 17:01
@VKTB VKTB linked an issue Feb 4, 2022 that may be closed by this pull request
1 task
@VKTB VKTB requested a review from MRichards99 February 4, 2022 17:01
Base automatically changed from feature/search-endpoints-#268 to master February 7, 2022 12:44
Copy link
Collaborator

@MRichards99 MRichards99 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Could this issue appear on the other pid fields on other entities? I've looked at the mappings, and like Sample.pid, all the ICAT fields are optional so the ICAT value could be None.

datagateway_api/src/search_api/filters.py Outdated Show resolved Hide resolved
@VKTB
Copy link
Contributor Author

VKTB commented Feb 8, 2022

Looks good. Could this issue appear on the other pid fields on other entities? I've looked at the mappings, and like Sample.pid, all the ICAT fields are optional so the ICAT value could be None.

Considering that all pid fields in the PaNOSC entities are mandatory, a None value returned from ICAT for it will result in the same issue occurring. An alternative ICAT field will need to be provided in the mappings file and a set_pid validator will need to be added to all the PaNOSC pydantic models that have a pid field. I am happy to add these if need be, just let me know.

@VKTB
Copy link
Contributor Author

VKTB commented Feb 8, 2022

The failing test jobs will be fixed by #325

@VKTB VKTB requested a review from MRichards99 February 10, 2022 17:11
Copy link
Collaborator

@MRichards99 MRichards99 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making the changes on the other entities. This was actually a problem on ISIS - they have PIDs/DOIs for documents, but not for datasets or instruments. So this change means we can actually query them without having it 500 :)

I just have a query on the returns on set_pid() as per the comment

datagateway_api/src/search_api/models.py Show resolved Hide resolved
@VKTB VKTB requested a review from MRichards99 February 11, 2022 08:19
@VKTB VKTB merged commit 48c1e29 into master Feb 11, 2022
@VKTB VKTB deleted the bugfix/validation-error-sample-pid-panosc-field-#314 branch February 11, 2022 10:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Validation error for Sample pid field when ICAT value is None
2 participants