Accessing details about (selected) Adaptation options

[patrickkaleta]

As mentioned in #15 and #8, Emikat needs to access the details of the Adapatation options stored in Drupal. For this reason I expanded the currently available REST end-points relevant for Emikat in the following way:

• the initial Study information endpoint now has a path_selected_adaptation_options attribute, which leads to a new endpoint showing all AOs, which have been selected in that Study (technically this attribute here is not even necessary, because the URL of that endpoint only has an extra '.../selected_adaptation_options' in comparison to the initial Study info endpoint. I just added it for the sake of completeness)
• that new endpoint (example link for Study 1) shows the paths to each selected adaptation option. For now, it' shows the JSON:API endpoint (meaning everything of the AO is shown, relevant or not), because that didn't require any additional configuration from my side.

@humerh please have a look at this and let me know if this is enough for Emikat or if you need additional data in those endpoints. It's not a big problem to adapt the new endpoint to only show the data you really need, but I need to know which fields are required by Emikat.

Another question for Heinrich: Could it be helpful for Emikat to know in advance all the available Adaptation options of the selected Data package, before the user even gets to the point of selecting them or is there no need for this?

[patrickkaleta]

@humerh please have a look at this. Does Emikat need any additional information regarding the selected adaptation options?

[humerh]

Now I have finished the implementation of Adaptation options, including the cost model in version 1.0.

See:
https://github.com/clarity-h2020/csis/wiki/Services-endpoints-(used-by-CSIS)

[p-a-s-c-a-l]

I still have problems retrieving data for adaptation scenarios. Especially aggregated data needed for Scenario Analysis doesn't seem to be available in "view.2994" which is btw not listed under Services Endpoints.

Examples:

• https://service.emikat.at/EmiKatTst/api/scenarios/3189/feature/view.2994/table/data
• https://service.emikat.at/EmiKatTst/api/scenarios/3209/feature/view.2994/table/data

AFAIR I've "found" once a study that contained ADAPTATION-01 in "view.2994", but I can't remember which one. Recalculating a study doesn't seem to work either. Any advise @patrickkaleta @DenoBeno ?

[p-a-s-c-a-l]

I've tried to recalculate study 35, but it's again stuck at 14/17 processes. Encountered the same problem yesterday and over night it resolved it self, either because the processes actually finished or EMIKAT was restarted and the process counter was reset.

[p-a-s-c-a-l]

I don't know what happened, but suddenly view.2994 of study 163/3207 now contains ADAPTATION-01 scenario data:
https://service.emikat.at/EmiKatTst/api/scenarios/3207/feature/view.2994/table/data

It really seems that adaptation scenarios appear and disappear at random. ❓

[p-a-s-c-a-l]

Strange: Study 35 still shows 14/17 processes but adaptation data is now available for this study.

[p-a-s-c-a-l]

And suddenly, the data disappeared:

{"name":"TAB Impact on Heat wave (Mortality) ","description":"TAB Impact on Heat wave (Mortality) \nSum of impacts (calculated statistically expected mortality count as consequence of this Event) on Heatwave in Study Area\nEach cell expresses the calculated statistically expected mortality count as consequence of this Event\n","columnnames":["STUDY_VARIANT","TIME_PERIOD","EMISSIONS_SCENARIO","EVENT_FREQUENCY","EXPOSEDQUANTITY","DAMAGEQUANTITY","DISCOMFORT_LEVEL","HEAT_WAVE_IMPACT","COST_DEVELOPMENT","COST_MAINTENANCE","COST_RETROFITTING","SZM_SZENARIO_REF"],"rows":[{"rownum":0,"values":["BASELINE","20110101-20401231","rcp26","Frequent",1340309.0,500.36590553409417,0.0,0.0,0.0,0.0,0.0,3287.0]},{"rownum":1,"values":["BASELINE","20110101-20401231","rcp26","Occasional",1340309.0,761.0178972003282,0.0,0.0,0.0,0.0,0.0,3287.0]},{"rownum":2,"values":["BASELINE","20110101-20401231","rcp26","Rare",1340309.0,904.3099295819127,0.0,0.0,0.0,0.0,0.0,3287.0]},{"rownum":3,"values":["BASELINE","20110101-20401231","rcp45","Frequent",1340309.0,500.36590553409417,0.0,0.0,0.0,0.0,0.0,3287.0]},{"rownum":4,"values":["BASELINE","20110101-20401231","rcp45","Occasional",1340309.0,854.202422997841,0.0,0.0,0.0,0.0,0.0,3287.0]},{"rownum":5,"values":["BASELINE","20110101-20401231","rcp45","Rare",1340309.0,956.8663869339189,0.0,0.0,0.0,0.0,0.0,3287.0]},{"rownum":6,"values":["BASELINE","20110101-20401231","rcp85","Frequent",1340309.0,470.41144359818765,0.0,0.0,0.0,0.0,0.0,3287.0]},{"rownum":7,"values":["BASELINE","20110101-20401231","rcp85","Occasional",1340309.0,717.781563201184,0.0,0.0,0.0,0.0,0.0,3287.0]},{"rownum":8,"values":["BASELINE","20110101-20401231","rcp85","Rare",1340309.0,956.8663869339189,0.0,0.0,0.0,0.0,0.0,3287.0]},{"rownum":9,"values":["BASELINE","20410101-20701231","rcp26","Frequent",1340309.0,500.36590553409417,0.0,0.0,0.0,0.0,0.0,3287.0]},{"rownum":10,"values":["BASELINE","20410101-20701231","rcp26","Occasional",1340309.0,761.0178972003282,0.0,0.0,0.0,0.0,0.0,3287.0]},{"rownum":11,"values":["BASELINE","20410101-20701231","rcp26","Rare",1340309.0,904.3099295819127,0.0,0.0,0.0,0.0,0.0,3287.0]},{"rownum":12,"values":["BASELINE","20410101-20701231","rcp45","Frequent",1340309.0,531.9896152960491,0.0,0.0,0.0,0.0,0.0,3287.0]},{"rownum":13,"values":["BASELINE","20410101-20701231","rcp45","Occasional",1340309.0,854.202422997841,0.0,0.0,0.0,0.0,0.0,3287.0]},{"rownum":14,"values":["BASELINE","20410101-20701231","rcp45","Rare",1340309.0,1011.9497651150506,0.0,0.0,0.0,0.0,0.0,3287.0]},{"rownum":15,"values":["BASELINE","20410101-20701231","rcp85","Frequent",1340309.0,600.5419169707824,0.0,0.0,0.0,0.0,0.0,3287.0]},{"rownum":16,"values":["BASELINE","20410101-20701231","rcp85","Occasional",1340309.0,904.3099295819127,0.0,0.0,0.0,0.0,0.0,3287.0]},{"rownum":17,"values":["BASELINE","20410101-20701231","rcp85","Rare",1340309.0,1069.6366554824883,0.0,0.0,0.0,0.0,0.0,3287.0]},{"rownum":18,"values":["BASELINE","20710101-21001231","rcp26","Frequent",1340309.0,470.41144359818765,0.0,0.0,0.0,0.0,0.0,3287.0]},{"rownum":19,"values":["BASELINE","20710101-21001231","rcp26","Occasional",1340309.0,761.0178972003282,0.0,0.0,0.0,0.0,0.0,3287.0]},{"rownum":20,"values":["BASELINE","20710101-21001231","rcp26","Rare",1340309.0,1552.2775793522562,0.0,0.0,0.0,0.0,0.0,3287.0]},{"rownum":21,"values":["BASELINE","20710101-21001231","rcp45","Frequent",1340309.0,565.3564919260386,0.0,0.0,0.0,0.0,0.0,3287.0]},{"rownum":22,"values":["BASELINE","20710101-21001231","rcp45","Occasional",1340309.0,904.3099295819127,0.0,0.0,0.0,0.0,0.0,3287.0]},{"rownum":23,"values":["BASELINE","20710101-21001231","rcp45","Rare",1340309.0,1011.9497651150506,0.0,0.0,0.0,0.0,0.0,3287.0]},{"rownum":24,"values":["BASELINE","20710101-21001231","rcp85","Frequent",1340309.0,717.781563201184,0.0,0.0,0.0,0.0,0.0,3287.0]},{"rownum":25,"values":["BASELINE","20710101-21001231","rcp85","Occasional",1340309.0,1130.001839078864,0.0,0.0,0.0,0.0,0.0,3287.0]},{"rownum":26,"values":["BASELINE","20710101-21001231","rcp85","Rare",1340309.0,1633.2319420749068,0.0,0.0,0.0,0.0,0.0,3287.0]},{"rownum":27,"values":["BASELINE","Baseline","Baseline","Frequent",1340309.0,442.0539170266677,0.0,0.0,0.0,0.0,0.0,3287.0]},{"rownum":28,"values":["BASELINE","Baseline","Baseline","Occasional",1340309.0,600.5419169707824,0.0,0.0,0.0,0.0,0.0,3287.0]},{"rownum":29,"values":["BASELINE","Baseline","Baseline","Rare",1340309.0,904.3099295819127,0.0,0.0,0.0,0.0,0.0,3287.0]}]}

ADAPTATION-01 is gone again. 😭

[p-a-s-c-a-l]

Here we go again:

''
{"name":"TAB Impact on Heat wave (Mortality) ","description":"TAB Impact on Heat wave (Mortality) \nSum of impacts (calculated statistically expected mortality count as consequence of this Event) on Heatwave in Study Area\nEach cell expresses the calculated statistically expected mortality count as consequence of this Event\n","columnnames":["STUDY_VARIANT","TIME_PERIOD","EMISSIONS_SCENARIO","EVENT_FREQUENCY","EXPOSEDQUANTITY","DAMAGEQUANTITY","DISCOMFORT_LEVEL","HEAT_WAVE_IMPACT","COST_DEVELOPMENT","COST_MAINTENANCE","COST_RETROFITTING","SZM_SZENARIO_REF"],"rows":[{"rownum":0,"values":["ADAPTATION-01","20110101-20401231","rcp26","Frequent",681537.0,293.6861876177559,4.0,16.0,1.016472729E9,0.0,1.222132059E9,3189.0]},{"rownum":1,"values":["ADAPTATION-01","20110101-20401231","rcp26","Occasional",681537.0,437.6637124382251,5.0,18.0,1.016472729E9,0.0,1.222132059E9,3189.0]},{"rownum":2,"values":["ADAPTATION-01","20110101-20401231","rcp26","Rare",681537.0,467.18944837939927,5.0,18.0,1.016472729E9,0.0,1.222132059E9,3189.0]},{"rownum":3,"values":["ADAPTATION-01","20110101-20401231","rcp45","Occasional",681537.0,498.5212857275805,5.0,18.0,1.016472729E9,0.0,1.222132059E9,3189.0]},{"rownum":4,"values":["ADAPTATION-01","20110101-20401231","rcp85","Frequent",681537.0,314.13311278189536,5.0,17.0,1.016472729E9,0.0,1.222132059E9,3189.0]},{"rownum":5,"values":["ADAPTATION-01","20110101-20401231","rcp85","Occasional",681537.0,409.8559252389223,5.0,18.0,1.016472729E9,0.0,1.222132059E9,3189.0]},{"rownum":6,"values":["ADAPTATION-01","20110101-20401231","rcp85","Rare",681537.0,437.6637124382251,5.0,18.0,1.016472729E9,0.0,1.222132059E9,3189.0]},{"rownum":7,"values":["ADAPTATION-01","20410101-20701231","rcp26","Frequent",681537.0,335.8977631942467,5.0,17.0,1.016472729E9,0.0,1.222132059E9,3189.0]},{"rownum":8,"values":["ADAPTATION-01","20410101-20701231","rcp26","Occasional",681537.0,498.5212857275805,5.0,18.0,1.016472729E9,0.0,1.222132059E9,3189.0]},{"rownum":9,"values":["ADAPTATION-01","20410101-20701231","rcp45","Frequent",681537.0,359.05446951377735,5.0,17.0,1.016472729E9,0.0,1.222132059E9,3189.0]},{"rownum":10,"values":["ADAPTATION-01","20410101-20701231","rcp45","Occasional",681537.0,531.7497507815855,5.0,18.0,1.016472729E9,0.0,1.222132059E9,3189.0]},{"rownum":11,"values":["ADAPTATION-01","20410101-20701231","rcp45","Rare",681537.0,729.6453743981631,5.0,18.0,1.016472729E9,0.0,1.222132059E9,3189.0]},{"rownum":12,"values":["ADAPTATION-01","20410101-20701231","rcp85","Frequent",681537.0,409.8559252389223,5.0,18.0,1.016472729E9,0.0,1.222132059E9,3189.0]},{"rownum":13,"values":["ADAPTATION-01","20410101-20701231","rcp85","Occasional",681537.0,566.967529025143,5.0,18.0,1.016472729E9,0.0,1.222132059E9,3189.0]},{"rownum":14,"values":["ADAPTATION-01","20410101-20701231","rcp85","Rare",681537.0,643.7509761801463,5.0,18.0,1.016472729E9,0.0,1.222132059E9,3189.0]},{"rownum":15,"values":["ADAPTATION-01","20710101-21001231","rcp26","Frequent",681537.0,314.13311278189536,5.0,17.0,1.016472729E9,0.0,1.222132059E9,3189.0]},{"rownum":16,"values":["ADAPTATION-01","20710101-21001231","rcp26","Occasional",681537.0,531.7497507815855,5.0,18.0,1.016472729E9,0.0,1.222132059E9,3189.0]},{"rownum":17,"values":["ADAPTATION-01","20710101-21001231","rcp26","Rare",681537.0,604.2692082438551,5.0,18.0,1.016472729E9,0.0,1.222132059E9,3189.0]},{"rownum":18,"values":["ADAPTATION-01","20710101-21001231","rcp45","Frequent",681537.0,359.05446951377735,5.0,17.0,1.016472729E9,0.0,1.222132059E9,3189.0]},{"rownum":19,"values":["ADAPTATION-01","20710101-21001231","rcp45","Occasional",681537.0,604.2692082438551,5.0,18.0,1.016472729E9,0.0,1.222132059E9,3189.0]},{"rownum":20,"values":["ADAPTATION-01","20710101-21001231","rcp45","Rare",681537.0,643.7509761801463,5.0,18.0,1.016472729E9,0.0,1.222132059E9,3189.0]},{"rownum":21,"values":["ADAPTATION-01","20710101-21001231","rcp85","Frequent",681537.0,531.7497507815855,5.0,18.0,1.016472729E9,0.0,1.222132059E9,3189.0]},{"rownum":22,"values":["ADAPTATION-01","20710101-21001231","rcp85","Occasional",681537.0,776.2549681924796,5.0,18.0,1.016472729E9,0.0,1.222132059E9,3189.0]},{"rownum":23,"values":["ADAPTATION-01","20710101-21001231","rcp85","Rare",681537.0,931.9122999640626,5.0,18.0,1.016472729E9,0.0,1.222132059E9,3189.0]},{"rownum":24,"values":["ADAPTATION-01","Baseline","Baseline","Frequent",681537.0,256.46292894877126,4.0,14.0,1.016472729E9,0.0,1.222132059E9,3189.0]},{"rownum":25,"values":["ADAPTATION-01","Baseline","Baseline","Occasional",681537.0,359.05446951377735,5.0,17.0,1.016472729E9,0.0,1.222132059E9,3189.0]},{"rownum":26,"values":["ADAPTATION-01","Baseline","Baseline","Rare",681537.0,437.6637124382251,5.0,18.0,1.016472729E9,0.0,1.222132059E9,3189.0]},{"rownum":27,"values":["BASELINE","20110101-20401231","rcp26","Frequent",681537.0,292.0173732368457,4.0,15.0,0.0,0.0,0.0,3189.0]},{"rownum":28,"values":["BASELINE","20110101-20401231","rcp26","Occasional",681537.0,435.34889236188724,5.0,18.0,0.0,0.0,0.0,3189.0]},{"rownum":29,"values":["BASELINE","20110101-20401231","rcp26","Rare",681537.0,464.7501258202455,5.0,18.0,0.0,0.0,0.0,3189.0]},{"rownum":30,"values":["BASELINE","20110101-20401231","rcp45","Occasional",681537.0,495.95250213145323,5.0,18.0,0.0,0.0,0.0,3189.0]},{"rownum":31,"values":["BASELINE","20110101-20401231","rcp85","Frequent",681537.0,312.36824097095416,5.0,17.0,0.0,0.0,0.0,3189.0]},{"rownum":32,"values":["BASELINE","20110101-20401231","rcp85","Occasional",681537.0,407.6606830984516,5.0,18.0,0.0,0.0,0.0,3189.0]},{"rownum":33,"values":["BASELINE","20110101-20401231","rcp85","Rare",681537.0,435.34889236188724,5.0,18.0,0.0,0.0,0.0,3189.0]},{"rownum":34,"values":["BASELINE","20410101-20701231","rcp26","Frequent",681537.0,334.0323220057781,5.0,17.0,0.0,0.0,0.0,3189.0]},{"rownum":35,"values":["BASELINE","20410101-20701231","rcp26","Occasional",681537.0,495.95250213145323,5.0,18.0,0.0,0.0,0.0,3189.0]},{"rownum":36,"values":["BASELINE","20410101-20701231","rcp45","Frequent",681537.0,357.08383945519796,5.0,17.0,0.0,0.0,0.0,3189.0]},{"rownum":37,"values":["BASELINE","20410101-20701231","rcp45","Occasional",681537.0,529.0465353910879,5.0,18.0,0.0,0.0,0.0,3189.0]},{"rownum":38,"values":["BASELINE","20410101-20701231","rcp45","Rare",681537.0,726.1968827164152,5.0,18.0,0.0,0.0,0.0,3189.0]},{"rownum":39,"values":["BASELINE","20410101-20701231","rcp85","Frequent",681537.0,407.6606830984516,5.0,18.0,0.0,0.0,0.0,3189.0]},{"rownum":40,"values":["BASELINE","20410101-20701231","rcp85","Occasional",681537.0,564.1249233605449,5.0,18.0,0.0,0.0,0.0,3189.0]},{"rownum":41,"values":["BASELINE","20410101-20701231","rcp85","Rare",681537.0,640.6149027916475,5.0,18.0,0.0,0.0,0.0,3189.0]},{"rownum":42,"values":["BASELINE","20710101-21001231","rcp26","Frequent",681537.0,312.36824097095416,5.0,17.0,0.0,0.0,0.0,3189.0]},{"rownum":43,"values":["BASELINE","20710101-21001231","rcp26","Occasional",681537.0,529.0465353910879,5.0,18.0,0.0,0.0,0.0,3189.0]},{"rownum":44,"values":["BASELINE","20710101-21001231","rcp26","Rare",681537.0,601.28229336262,5.0,18.0,0.0,0.0,0.0,3189.0]},{"rownum":45,"values":["BASELINE","20710101-21001231","rcp45","Frequent",681537.0,357.08383945519796,5.0,17.0,0.0,0.0,0.0,3189.0]},{"rownum":46,"values":["BASELINE","20710101-21001231","rcp45","Occasional",681537.0,601.28229336262,5.0,18.0,0.0,0.0,0.0,3189.0]},{"rownum":47,"values":["BASELINE","20710101-21001231","rcp45","Rare",681537.0,640.6149027916475,5.0,18.0,0.0,0.0,0.0,3189.0]},{"rownum":48,"values":["BASELINE","20710101-21001231","rcp85","Frequent",681537.0,529.0465353910879,5.0,18.0,0.0,0.0,0.0,3189.0]},{"rownum":49,"values":["BASELINE","20710101-21001231","rcp85","Occasional",681537.0,772.6435327037456,5.0,18.0,0.0,0.0,0.0,3189.0]},{"rownum":50,"values":["BASELINE","20710101-21001231","rcp85","Rare",681537.0,927.7878624142631,5.0,18.0,0.0,0.0,0.0,3189.0]},{"rownum":51,"values":["BASELINE","Baseline","Baseline","Frequent",681537.0,254.97316542256627,4.0,14.0,0.0,0.0,0.0,3189.0]},{"rownum":52,"values":["BASELINE","Baseline","Baseline","Occasional",681537.0,357.08383945519796,5.0,17.0,0.0,0.0,0.0,3189.0]},{"rownum":53,"values":["BASELINE","Baseline","Baseline","Rare",681537.0,435.34889236188724,5.0,18.0,0.0,0.0,0.0,3189.0]}]}
''

[patrickkaleta]

Strange: Study 35 still shows 14/17 processes but adaptation data is now available for this study.

Batch job "Rebuild Table CLY_HW_T_MRT#1856" has been stuck since 12:08, so there's definitely a problem there. @humerh please investigate this.

The batch jobs for the Adaptation options are executed before that, so this might explain why results are available. But since the whole process is not completed, it's difficult to say which results are actually available and which might just have been cached from previous calculations and not yet renewed, since the process got stuck...

[humerh]

You have to add a filter criteria: &filter=SZ_ID%3D3183

like this: https://service.emikat.at/EmiKatTst/api/scenarios/3183/feature/view.2994/table/data?rownum=1000&filter=SZ_ID%3D3183

Otherwise you get an arbitrary study back.