Data model for "adaptation project"

[DenoBeno]

Originally posted by @DenoBeno in #154 (comment)

We need to define a new node or taxonomy type that will be used to define "adaptation strategy". Something like this:

1. one land use category (e.g. residential middle density houses)
2. one or or more adaptation options
3. percentage of the land use category to apply this to (one or one for each adaptation option?)
4. "strategy" (need another name for this) of choosing where to apply this - e.g. random or "cells with high heat hazard" or "cells with high population exposure" or "cells with high heat mortality"
5. some text & illustrations
6. Other?

Each adaptation strategy defines one or more adaptation options applied to one land use category. E.g. "apply cool roofs and green walls on 50% of the residential middle density houses".

It is important to think how users will be able to choose such strategies later and how EMIKAT can process this in the end. Since we aren't sending the explicit geometry, EMIKAT will have to choose where to apply the the "strategy". E.g., if the strategy is to apply something to 50% of the houses, preferably in the "cells with high mortality", then EMIKAT would needs to find how many houses are in the project, find the cell with highest mortality, apply this to all houses in this cell, find the next highest mortality cell, apply there and so on - until 50% mark has been reached.

This sounds too complex to me, we need to re-design this somehow. @ALL, any ideas?

[humerh]

My concept for adaptation option calculation looks like that:

1. Within a CSIS study one (or more) ADAPTATION_PROJECTs can be defined. An Adaptation Projects is given by a NAME, DESCRIPTION (purpose) and a bounding box ('POLYGON((x1 y1), (x2 y2) ...(x1 y1))' for the area, which is affected by this project. If the bounding box is missing, the adaption projects is defined for the whole Study Area.

2. What we have defined currently as "Adaptation Options" in CSIS should be transformed to ADAPTATION_MEASURE_TEMPLATES, because they only describe different "patterns" or "templates", but no concrete implementation of a sratagy.

3. For a specific ADAPTATION PROJECT one or more ADAPTATION MEASURES can be be applied, which are concrete implementations of TEMPLATES for a given PROJECT. At this level severalt aditional paramters, like fix costs, variable costs, cost per length, etc. should be added.

If all these things are defined, CSIS should forward the NAME and ID of the ADAPTATION PROJECT to emikat, and emikat will fetch for all necessary details from CSIS via REST interface.

[patrickkaleta]

To think about

Maybe we can remove the Study variant field in the Study scenarios (ATM you can only select Study Variant: no Adaptation Options anyway). Otherwise users will have to create the exact same Study scenario twice (once without AOs, once with them), since I don't see the value of comparing a baseline scenario with an adapted one that has completely different parameters (RCP and Em. scenario). How would the user know what difference the AOs did?

ATM in the Table tab of the IAO step the user can only see AOs, which are included in the selected Data package. @DenoBeno why is that? Why not allow all AOs, since they already contain all necessary information so that Emikat can work with them and they are otherwise in no way coupled to data packages. If we keep this restriction, we will have to update all data packages everytime a new AO is added in the CSIS.

Regarding bounding boxes for Adaptation Projects

I saw that for the Drupal 8 Leaflet module there has been quite some new development for the past 12 months (when we decided to used the Geofield Map module for displaying maps in Drupal). Don't get your hopes up too much, but I'll look into it again to see if this module (or some submodules) could allow us the create polygons directly on a map out-of-the-box. Or if at least this module could now replace the Google maps for our Twins, which could help us with this issue.

[DenoBeno]

@patrickkaleta : I wanted to add "adaptation project" node type to csis, but I realised that your idea of defining the parameters and possibly even the areas in the group node collides with @mattia-leone's idea of pre-defining the adaptation projects so that users don't have to do this.

Maybe we even need both?

1. "Adaptation strategies" to simplify the life of the users and assure that the adaptation chosen adaptation options are complementary.
2. "Adaptation project" to let the users assign 0-1 of the possible adaptation strategies to each of the layers.

The way I understand this, an adaptation strategy applied to a layer could result in part of this layer completely changing nature. However, we aren't really going to change any layers (I suppose) and are instead of this going to change the parameters of the existing layers. Which is what we are already doing with the "adaptation options". So do we really need the AO -> AS -> AP chain? Unfortunately, this may be the case, but let me try to construct something just with AOs and AP. I'll do this in next comment.

Something completely different:

• The reason why data package explicitly lists the adaptation options is the following: Lena wanted to define some adaptation options that are only used in expert mode and don't have the parameters necessary for EMIKAT defined.
• I agree that we should remove the Study variant field in the Study scenarios, since it should be possible to compare the with/without AOs for all study scenarios and defining everything twice (or even N times?) is a PITA.

[DenoBeno]

So, back to proposal by Patrick. What we want to achieve is that user explicitly selects one or more combinations of:

LAYER + Adaptation Option + percentage of the layer + strategy

Bounding box would then always be the whole study area, and the "strategy" would assure that the AOs are applied in the best possible place. Example:

1. Dense urban fabric + replace by trees + 5% + strategy: highest T_A
4. Streets + cool streets + 10% + strategy: highest T_A
2. Dense urban fabric + Cool roofs + 20% + strategy: highest T_MRT
3. Medium urban fabric + replace by dense urban fabric + 10% + strategy: lowest population density

In principle, we could indeed implement this without "Adaptation strategy" node type. Instead of this, we could define "adaptation strategy" paragraph and let the user add 1-* such paragraphs to the "Adaptation project" node type.

Alternatively, we could replace the "strategy" by "Area" - then it's up to user to choose some sub-area within the project where "LAYER + Adaptation Option + percentage of the layer" will be applied to. From the usability point of view, it's IMO better to use the "strategy". The question is how difficult would it be for EMIKAT to apply some strategies?

Layer replacement AOs

As illustrated above, we would need to define some adaptation options that effectively replace one layer type by a different one. There is nothing really special in these AOs, and we don't really need to change the layers - we just need to apply a "mask" over the existing layers to change their parameters. .

How to define a strategy in EMIKAT?

I have listed a couple of possible strategies as illustration of what the strategies could look like. Highest T_MRT, Lowest T_a, lowest population density etc etc.

To implement this in EMIKAT, we would need to sort the cells in the project area by the required parameter and then apply the AO to the part of the layer that's in the cell in each cell one by one, until we have run out of the AO area.

E.g., we start with the hottest cell, replace all roofs on dense residential buildings by cool roofs there, then go to the next cell etc etc.

possible complications

Main problem that I see is overlapping of the AOs.

Imagine that we want to apply green roofs on 10% of the houses and cool roofs on 10% too. In the worst case, we will end up with 10% of the houses where the cool roofs are applied on top of the green roofs => not what the user expects, I suppose.

One way to handle this would be to ignore the areas that have already received the adaptation option when applying the next AO. Which also means that ordering of the adaptation options in the adaptation project would matter.

@stefanon , @patrickkaleta , @humerh, WDYT?

[patrickkaleta]

Something completely different:

• The reason why data package explicitly lists the adaptation options is the following: Lena wanted to define some adaptation options that are only used in expert mode and don't have the parameters necessary for EMIKAT defined.

Ok, good. So let's stick to the current concept, but in the future we might consider addind an "expert only" checkbox to Adaptation options, so that "normal" Studies could show all regular AOs, while Expert Studies would get a new "View as application" node, which could either list "expert" AOs only or the once pre-selected explictly in the data package.
That way we would eliminate the need to keep tracking normal data packages and normal AOs, as well as make filtering them easier in Views, EntityBrowser or the csis helpers module.

• I agree that we should remove the Study variant field in the Study scenarios, since it should be possible to compare the with/without AOs for all study scenarios and defining everything twice (or even N times?) is a PITA.

Good, I'll mark the Study variant field as "probably obsolete" and we'll take care of it once this Adaptation project issue is solved.

[patrickkaleta]

So, back to proposal by Patrick. What we want to achieve is that user explicitly selects one or more combinations of:

LAYER + Adaptation Option + percentage of the layer + strategy

No, I don't think the user should select the layer. AOs already have an Applicable to field, which determines to what kind of land usage category this AO can be applied. Users should not have to possibility to meddle with that.

Layer + Adaptation Option + percentage of the layer + STRATEGY

Also I don't believe it will be feasible for Heinrich to implement strategies as you have them in mind. Heinrich's proposal doesn't mention anything like that. Yes, it would be a cool feature to say "hey use this AO where the T_MRT is highest or use it where the population density is highest", but IMO this is not feasible for this project and we should forget about it. @humerh WDYT?

Layer + Adaptation Option + PERCENTAGE OF THE LAYER + strategy

"percentage of the layer" concept could be used, but then I would leave it entirely up to Emikat to make the decision where to apply it. Problem: I bet the user would like to know where Emikat applied his AOs, but this way he just wouldn't.

Alternative:

That's what the bounding box in an Adaptation project could solve. Later, in a more advanced prototype the user would draw a rectangle inside the Study area to explicitly say where it shall be applied. The initial prototype won't have this feature, so either the Adaptation project will be automatically applied to the whole Study area (yes, it's unrealistic) or the user has to enter the bounding box as a string, which he gets some other way (yes, not very user-friendly).

[patrickkaleta]

Status update after several discussions with @mattia-leone and @humerh:

Specifications provided by Mattia:

• example-calculations.docx
• general-description.pptx

Currently, I would favor the following approach:

1. Since Mattia already defined Adaptation strategies (=AS), we will use them. Users won't be able to select individual Adapt. options (=AO), but rather our pre-defined Adapt. Strategies.
2. Currently Mattia defined 4 different types of AS, based on the land-use category which they affect (those are mutually exclusive, no land-use category can be in two different AS types):
   1. Build open spaces
   2. Buildings
   3. Roads
   4. Vegetated areas
3. Since AS change the distribution of the land usage in a cell, I suggest that users can only select one AS for each AS type, since otherwise the AS would interfere with each other
4. Therefore, users can select up to 4 AS (one for each AS type, in the future more types could be added) for the Study, choose an "application coefficient" (this is called "editable" in the word document) and potentially later even explicitly select the bounding box inside the Study area, where the AS should be applied. This would all be stored in an Adaptation Project, which would be modelled as a Group node.

Based on that I unfortunately don't see a way around the AO -> AS -> AP structure feared by @DenoBeno.

In detail, I suggest the following data model:

1) New Node: Adaptation Strategy with the following fields

1. Title
2. Description
3. Type (Roads, Buildings, Build-open spaces, vegetated areas. Probably reference to a new taxonomy?)
4. targeted land-use categories (1..n references to existing land-use category taxonomy defining, which land-use categories will be transformed)
5. included Adaptation options (needs to be a reference to a new taxonomy, since we don't only need the AO, but also the "partition factor" saying how much of the original land-use category will be transformed into the new one. See page 2 of word document for more details)

2) New Group Node: Adaptation Project with fields:

1. Title
2. Description
3. Area (initially hidden and empty, so Emikat will use the Study area bounding box)
4. application coefficient (called "editable" in word document) says how much of the targeted land-use category will be transformed by the AOs
5. AS for Buildings: 0..1 references to AS of the type Buildings (Entity Browsers could be used to to filter the correct AS types)
6. same as (5) for all remaining AS types defined in the CSIS

3) New Taxonomy Adaptation options with fields:

1. Title of the new layer name (as seen on last page of word document, usually equal to AO name)
2. reference to the corresponding AO
3. "partition factor" in percentage as shown on page 2 of word document
4. some kind of boolean telling whether or not it's going to be a "real" new layer (see page 3 of word document with example for Building AS, where "selective glasses" and "ventilated facades" don't show up un the resulting land-use composition, but are still required for cost calculations

[DenoBeno]

How much of this has already been implemented? Can/should I help with something?

[patrickkaleta]

Those 3 new Node/Taxonomy types have yet to be implemented.

I didn't want to introduce such major changes in the CSIS data model, since it would affect the public Beta release (#145) because these data model changes need to be reflected in the content/config synchronization between Drupal instances (#135).

However, setting up the new GeoServer instance is stalling the Beta release anyway, I've started to implement those new Nodes/Taxonomies in my local instance. And if everything goes smooth, I can add this to the live CSIS before the release (end of this week / start of next week). @humerh is waiting to access the Adaptation data from CSIS in order to proceed with the AO calculation implementation in Emikat, so this issue can't be stalled much longer.

@DenoBeno no, ATM there's no need for you to intervene, communication with Heinrich and Mattia is working well so far and things are moving forward. I'll let you know if I run into unexpected issues.

[patrickkaleta]

I've now implemented the above mentioned Data model for Adaptation Projects (=AP) and created the necessary Views so that users can add a Adaptation Project to their Study.
See example here: https://csis.myclimateservice.eu/study/25/step/3686/view/table

I've also adapted the REST Views, so that @humerh can access all the parameters of the AP. During the triggering of the Study calculations the respective REST endpoints for the chosen Adaptation Strategies are sent along as well now. @humerh I'll send you a more detailed description of how to get to all the needed parameters soon and/or we can discuss it during a telco.

ToDo:

• check/adjust permissions so that ordinary users can work with APs (as long as they are members of the Study)
• fine-tune the UI to make it more user-friendly out of scope for this issue
• add all the Adaptation Strategies, that were defined by PLENIVS. (@mattia-leone can you or someone else take care of that? It will be just like adding/editing Adaptation options, which you've done before. We can have a separate telco and discuss the details.)
• Update the Content migration files with the 2 new Taxonomies (Strategy types, AO implementations) and the new Node type Adaptation Strategy (Adaptation Project is part of the Study and will not be synchronized between Drupal instances)

[patrickkaleta]

IMO the data model now stands and Emikat is able to pull data from the CSIS and perform the calculation.

Closing this issue and resuming discussion here #154.