Skip to content

Commit

Permalink
Merge branch 'main' into develop
Browse files Browse the repository at this point in the history
  • Loading branch information
PawelPawelec-RDX committed Jan 21, 2025
2 parents 2ed871a + efe0b83 commit 5de9fd8
Show file tree
Hide file tree
Showing 3 changed files with 49 additions and 22 deletions.
30 changes: 30 additions & 0 deletions .github/workflows/build-typescript-sdk.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
name: Build TypeScript Gateway SDK

on:
push:
branches:
- '*'
paths:
- '/sdk/typescript/**'

jobs:
build-gateway-sdk:
runs-on: ubuntu-22.04
defaults:
run:
working-directory: ./sdk/typescript
permissions:
id-token: write
contents: read
steps:
- uses: RDXWorks-actions/checkout@main
- name: Use Node.js
uses: RDXWorks-actions/setup-node@main
with:
node-version: "18.x"
- run: cat $NPM_CONFIG_USERCONFIG
- name: Build gateway-sdk
run: |
npm ci
npm run build
npm run test
16 changes: 3 additions & 13 deletions docs/api-specifications.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,12 @@
# API Specifications

There are (at time of writing) three main Radix APIs:

* [Gateway API](https://redocly.github.io/redoc/?url=https://raw.githubusercontent.com/radixdlt/radixdlt-network-gateway/main/gateway-api-schema.yaml) - The main public facing API, exposed by the Network Gateway
* [Core API](https://redocly.github.io/redoc/?url=https://raw.githubusercontent.com/radixdlt/radixdlt/main/radixdlt-core/radixdlt/src/main/java/com/radixdlt/api/core/api.yaml) - An API exposed by radixdlt full nodes, intended to be exposed on private networks
* [System API](https://redocly.github.io/redoc/?url=https://raw.githubusercontent.com/radixdlt/radixdlt/main/radixdlt-core/radixdlt/src/main/java/com/radixdlt/api/system/api.yaml) - An API exposed privately by radixdlt nodes to get information about the node health/status.

The links above link to the ReDocly docs, reading the schemas on the `main` branches of each repo. As such, they may include features that are not yet on the latest release.
The documentation site covers details about the various [Network APIs](https://docs.radixdlt.com/docs/network-apis) exposed in the Radix stack.

## Gateway API

As the Gateway API is exposed by the Gateway Service, the Gateway API specification source-of-truth lives in the [Open API Schema in the Network Gateway repository](../src/RadixDlt.NetworkGateway.GatewayApi/gateway-api-schema.yaml).

The Open API specification should be kept up to date with the interface that the Gateway API service exposes.

## Core and System APIs
The current [Gateway API schema](https://radix-babylon-gateway-api.redoc.ly/) of the foundation Gateway is available on Redocly.

The source of truth for the Core and System APIs lives on the full node - and the specs on the `main` branch are here: [Core Open API Spec](https://github.com/radixdlt/radixdlt/blob/main/radixdlt-core/radixdlt/src/main/java/com/radixdlt/api/core/api.yaml) | [System Open API Spec](https://github.com/radixdlt/radixdlt/blob/main/radixdlt-core/radixdlt/src/main/java/com/radixdlt/api/system/api.yaml).
The Gateway API schema source-of-truth lives in this repository as an Open API definition in [gateway-api-schema.yaml](../src/RadixDlt.NetworkGateway.GatewayApi/gateway-api-schema.yaml).

## Model and Client Generation

Expand Down
25 changes: 16 additions & 9 deletions docs/database-migrations.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,31 +4,38 @@ For most of the releases database schema differs between versions. To easily mig
For further reading, visit the official Microsoft documentation on Entity Framework migration https://learn.microsoft.com/en-us/ef/core/managing-schemas/migrations/?tabs=dotnet-core-cli

## Executing migrations
There are two supported methods of applying migrations. Executing Idempotent SQL script or using programmatic Entity Framework migrations (for simplicity they can be applied using docker image). You can choose whichever you prefer.
There are two supported methods of applying migrations, discussed below:

* Using programmatic Entity Framework migrations (for simplicity they can be applied using docker image)
* Executing an Idempotent SQL script

### Docker image
For each release, we publish a docker image which is responsible for migrating the database. Similarly to the idempotent SQL script, it takes care of applying each migration only once and applying only missing migrations.

Image can be found here:
https://hub.docker.com/r/radixdlt/babylon-ng-database-migrations
The image can be found here: https://hub.docker.com/r/radixdlt/babylon-ng-database-migrations

To migrate the database you just need to run that image providing a connection string to the database. To do that you need to set an environment variable for the docker container `ConnectionStrings__NetworkGatewayMigrations`.

#### Known issues and limitations
In the current version of Ngpsql, if using a schema other than `public`, the migrations may fail to execute with an error message such as "__EFMigrationsHistory already exists on dbContext.Database.Migrate();". This is caused by [this underlying issue](https://github.com/npgsql/efcore.pg/issues/2878).
In the current version of NpgSQL, if using a schema other than `public`, the migrations may fail to execute with an error message such as "__EFMigrationsHistory already exists on dbContext.Database.Migrate();". This is caused by [this underlying issue](https://github.com/npgsql/efcore.pg/issues/2878).

We have implemented a workaround for this, which requires specifying the `SearchPath=schema_name` parameter to the connection string as [described here in the Ngpsql docs](https://www.npgsql.org/doc/api/Npgsql.NpgsqlConnectionStringBuilder.html#Npgsql_NpgsqlConnectionStringBuilder_SearchPath).

Providing the schema **ONLY** at the user level i.e `ALTER user someuser in database mainnet SET search_path TO schema_name, public;` is not enough to activate the work-around and will result in errors.

### Idempotent SQL script
It is a Raw SQL script that has to be executed on the database. It takes care of applying each migration only once and applying only missing migrations.

It can be found in the [migrations directory](../src/RadixDlt.NetworkGateway.PostgresIntegration/Migrations/IdempotentApplyMigrations.sql)
This is a Raw SQL script that has to be executed on the database. It takes care of applying each migration only once and applying only missing migrations. It can be found as [IdempotentApplyMigrations.sql](../src/RadixDlt.NetworkGateway.PostgresIntegration/Migrations/IdempotentApplyMigrations.sql) in the migrations directory.

Before running the script, may need to ensure that the user running the script has SUPERUSER privileges, for example with:

```sql
ALTER USER db_dev_superuser WITH SUPERUSER;
```

For more information check:
https://learn.microsoft.com/en-us/ef/core/managing-schemas/migrations/applying?tabs=dotnet-core-cli#idempotent-sql-scripts
For more information, see the [Microsoft documentation](https://learn.microsoft.com/en-us/ef/core/managing-schemas/migrations/applying?tabs=dotnet-core-cli#idempotent-sql-scripts) on the idempotent SQL scripts.

Because of a bug in CLI responsible for generating these SQL scripts https://github.com/dotnet/efcore/issues/24512, it is possible that it will generate invalid SQL scripts (missing semicolon at the end of each command). We are trying to make sure it does not happen but since we are internally using docker images to migrate the database there is a chance that it might slip through. To fix that, simply add a semicolon at the end of the invalid command.
Because of [a bug in CLI responsible for generating these SQL scripts](https://github.com/dotnet/efcore/issues/24512), it is possible that it will generate invalid SQL scripts (missing semicolon at the end of each command). We are trying to make sure it does not happen but since we are internally using docker images to migrate the database there is a chance that it might slip through. To fix that, simply add a semicolon at the end of the invalid command.

## Schema upgrade scenarios
There are two scenarios that might happen between gateway versions.
Expand Down

0 comments on commit 5de9fd8

Please sign in to comment.