IFRC Geo Application Proposal

[udaynwa]

Problem Statement

• Using a single model for multiple entities is complicating development efforts.
• There is confusion among end-users and developers regarding the definition and purpose of each entity.
• Creating bespoke solutions for each form is increasing maintenance overhead.
• Current limitations prevent associating multiple national societies with a country and vice versa.
• Previous issues have highlighted problems stemming from this dependency.
• Addressing these issues will remove a significant technical debt.
• Lacks re-usability
• Lacks separation of concern
  • Example: Fields like Tab 1 name belongs to the Country model
• Boundaries for colonial countries don't work as expected

Proposed Solution

Introduction

We're proposing to divide the Country table into smaller, independent tables. This change will help us better organize different uses of the data and make our APIs much easier to work with. It will also improve how well developers, stakeholders, and end users understand our system, while also reducing our technical debt. Additionally, we aim to link GeoJSON data with each geographical entity. This will make it easier to use this data across different parts of the IFRC network, enhancing our overall efficiency and integration capabilities.

Our solution is structured into two distinct stages, with Stage 1 serving as a prerequisite for Stage 2. Subtasks within Stage 1 can proceed concurrently, ensuring efficient progress towards our project goals.

Stage 1.1. Create Geo App and New Models

Tasks

1. Create Geo app in go-api

   • Develop a new Geo app

2. Create New Models

   • Develop initial models for
     • Region
     • Country
     • Cluster
     • Country Groups
     • Admin level 1
     • Admin level 2
     • Admin level 3
     • Admin level 4
   • Each model should include a geoId field referencing the corresponding models in the present go-api

3. Implement APIs to fetch GeoJSON files

   • Develop APIs to fetch GeoJSON files for
     • Country
     • Region
     • Admin level 1
     • Admin level 2
     • Admin level 3
     • Admin level 4
   • GeoJSON files should include bounds, labels(in 4 IFRC languages), centroid, geoID and other relevant metadata

Stage 1.2. Document new fields on existing models

Tasks

1. Review the excel sheet shared by David
2. Document the new fields required for each existing model

Stage 1.3. Make Mapbox style generic

Tasks

1. Review and modify the existing Mapbox style to make it generic
   • Ensure data for admin levels is available across all zoom levels
   • Remove unused label layers
   • Remove unnecessary layer styling
2. Add geoID to each layers in Mapbox style
3. Regenerate the Mapbox layers and styles

Stage 2. Migration and Cleanup

Tasks

1. Create additional models like National Society, Country data, Region data, Organizations (ICRC, IFRC, etc) etc

2. Update serializers

   • Remove usage of old fields from serializers in the go-api
   • Add new fields to serializers

3. Update UI

   • Modify the UI to reflect the changes in data structure and functionality
   • Run an extensive QA testing

4. Add Migration script to migrate old data and use the new Geo app

   • Develop a migration script to copy information to new fields
   • Implement a QA process to verify the migration
   • Use SQL based approach to verify the migration

5. Remove old fields from models

   • Identify and remove old fields from models to clean up the data structure

Open Questions

• Do we keep the UI as is, or do we reflect the changes in the UI?
• Should we update labels for the combined fields to reflect their actual behavior?
  • Example: Reporting National Society has options of organizations as well

cc @davidmuchatiza @batpad @geohacker @tovari @szabozoltan69 @LukeCaley @tnagorra @frozenhelium @samshara @thenav56

[fionabrc]

See this Slack thread for a related question and problem regarding a user friendly interface being needed for non-technical users to access the GO Data https://sims-ifrc.slack.com/archives/CEFDXURA9/p1720685659311549

[udaynwa]

Hey @fionabrc , seems like we do not have access to Slack. Would it be possible to share the content here directly?

[fionabrc]

Sorry didn't realise you wouldn't be on the SIMS Slack - copy/pasted below. Thanks

[tnagorra]

Move from another thread by @fionabrc

[@david Muchatiza], [@daniel Tovari] (or anyone else who knows), 2 years ago the below thread pointed me to the Go API to get the list of IFRC countries, territories, regions etc. The link to retrieve the data is https://goadmin.ifrc.org/api/v2/country/?limit=500.
Is there a more user friendly link somewhere on the Go Platform, that people would be able to search for/navigate to to get this data, ideally with a description? I'm trying to embed a link to this data in a product, but to users the above could appear complicated/without explanation.

David Muchatiza
9:23 AM
@Fiona Harvey
Yes, the GO Wiki provides details on how to get this data through the API and also how to access it on other platforms such as PowerBI.

Fiona Harvey
9:30 AM
Thanks

Moved from another thread by

@David Muchatiza
. I'm assuming the best sub-page is this one https://go-wiki.ifrc.org/en/go-api/Data_dictionary_country.
The GO Wiki is probably still too complicated for the average user (non-IM people), and I'm not sure how someone could navigate there themselves from the main GO Platform.
The equivalent link I've embedded for how people can view the GO Map layers is this one https://go-user-library.ifrc.org/maps. Is there anything similar (with descriptions, aimed at general users being able to download the data without any technical knowledge) for the country/territory/region data?

David Muchatiza
9:44 AM
Okay, I understand. For now, those are the two sites we have that provide instructions on the use and access of GO geospatial data. However, there is room for improvement, and we are working on enhancements, which we are tracking using this GitHub ticket. Please, if you have time, add comments to it highlighting some of the issues you have raised here and any related products or files that make it easier for non-technical users.

Fiona Harvey
9:54 AM
Thanks David. I'll add comments and for now in this product, explain the links as best I can in the FAQs.

[nanometrenat]

Hi there, agree lots to consider if changing around the way countries/territories/regions/National Societies etc. are modelled on GO.

Create New Models

• Develop initial models for

  • Region
  • Country
  • Cluster
  • Country Groups
  • Admin level 1
  • Admin level 2
  • Admin level 3
  • Admin level 4

• Each model should include a geoId field referencing the corresponding models in the present go-api

Can I just check, what is intended by "Country Groups" in this context?

From my end, I'm more knowledgeable about "gazetteer" style geodata (i.e. lists/tables/sets rather than spatial/boundary/point/polygon data per se), but superkeen to help the cause - and currently personally I'm not clear why Regions and Clusters wouldn't just be groupings of countries rather than having their own separate geospatial attributes? Grateful for any insights on the rationale for this bit.

Noting that I think the lists/gazetteer info covers the bulk of @fionabrc 's mentioned use cases as well. Although boundaries/polygons also useful for other cases.

Question: how will changes over time be modelled? When boundaries change, or names of admin areas change, or if new countries come into being/countries merge? Not sure if that is already considered in current design, but need to take an active decision about what geodata is used when looking at historic emergencies/events/data.

[samshara]

@nanometrenat ,

1. Country Groups are collections of countries grouped based on specific criteria, like Benelux.

2. Regions and Clusters are indeed groups of countries within the IFRC Go application.

   • Regions have dedicated pages and associated geo-information necessary for visualization. The geospatial attributes are a necessity.
   • Clusters are stored in the database but are not used in the application and could potentially be removed. Examples of Clusters include: "Bangkok Country Cluster," "Beijing Country Cluster," "Caribbean Country Cluster," "Central Africa Country Cluster."

3. Additionally, there are countries without ISO3 codes, such as "Aksai Chin," "Ashmore and Cartier Islands," "Bassas da India," "Bird Island." How should we handle these entities?

Modeling changes over time, such as boundary changes, will require a separate discussion. This is a complex topic, especially if we also track changes for administrative levels beyond admin level 0.

[nanometrenat]

• Clusters are stored in the database but are not used in the application and could potentially be removed. Examples of Clusters include: "Bangkok Country Cluster," "Beijing Country Cluster," "Caribbean Country Cluster," "Central Africa Country Cluster."

Hi, I believe Clusters are used in the application (or at least they used to be!) - for when data-mapping Appeals that are wider than one country etc. Also the country pages' reference to "IFRC Presence" should in theory have the linkage to which [cluster] delegation a country is part of. There's work ongoing to better support IFRC Delegation (i.e. country/country cluster/regional) data.

[samshara]

@nanometrenat, ah okay. Then we will need to keep these clusters and outline their usage on the platform as well.

[nanometrenat]

xref IFRCGo/go-frontend#1639 and other previous tickets - have these considerations already been taken into account in the above proposal? Or do we need to do a validation exercise against the prior tickets? THANKS!

[davidmuchatiza]

@udaynwa Here is the link to a document listing relationships that exist between National Societies and Countries (Admin0 Areas)