Skip to content

Commit

Permalink
Update documentation with Pekko (#9)
Browse files Browse the repository at this point in the history
  • Loading branch information
DybekK authored Feb 9, 2024
1 parent 3465c63 commit 94e9b4a
Show file tree
Hide file tree
Showing 11 changed files with 141 additions and 180 deletions.
8 changes: 4 additions & 4 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# Contributing to Akka Persistence Postgres
# Contributing to Pekko Persistence Postgres

## General Workflow

This is the process for committing code into master.

1. Before starting to work on a feature or a fix, make sure that there is a ticket for your work in the [issue tracker](https://github.com/SwissBorg/akka-persistence-postgres/issues). If not, create it first.
1. Before starting to work on a feature or a fix, make sure that there is a ticket for your work in the [issue tracker](https://github.com/SwissBorg/pekko-persistence-postgres/issues). If not, create it first.
2. Perform your work according to the [pull request requirements](#pull-request-requirements).
3. When the feature or fix is completed you should open a [Pull Request](https://help.github.com/articles/using-pull-requests) on [GitHub](https://github.com/SwissBorg/akka-persistence-postgres/pulls).
3. When the feature or fix is completed you should open a [Pull Request](https://help.github.com/articles/using-pull-requests) on [GitHub](https://github.com/SwissBorg/pekko-persistence-postgres/pulls).
4. The Pull Request should be reviewed by other maintainers (as many as feasible/practical). Outside contributors are encouraged to participate in the review process, it is not a closed process.
5. After the review you should fix the issues (review comments, CI failures) by pushing a new commit for new review, iterating until the reviewers give their thumbs up and CI tests pass.
6. If the branch merge conflicts with its target, rebase your branch onto the target branch.
Expand Down Expand Up @@ -73,5 +73,5 @@ Example:

## How To Enforce These Guidelines?

1. [GitHub Actions](https://github.com/SwissBorg/akka-persistence-postgres/actions) automatically builds the code, runs the tests and sets Pull Request status accordingly of results in GitHub.
1. [GitHub Actions](https://github.com/SwissBorg/pekko-persistence-postgres/actions) automatically builds the code, runs the tests and sets Pull Request status accordingly of results in GitHub.
2. [Scalafmt](https://scalameta.org/scalafmt/) enforces some of the code style rules.
46 changes: 24 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
![Akka Persistence Postgres](./docs/assets/project-logo.png)
![Pekko Persistence Postgres](./docs/assets/project-logo.png)

[![License](https://img.shields.io/:license-Apache%202-red.svg)](https://www.apache.org/licenses/LICENSE-2.0.txt)
[![Actions Status](https://github.com/SwissBorg/akka-persistence-postgres/workflows/Scala%20CI/badge.svg)](https://github.com/SwissBorg/akka-persistence-postgres/actions)
[![Actions Status](https://github.com/SwissBorg/pekko-persistence-postgres/workflows/Scala%20CI/badge.svg)](https://github.com/SwissBorg/pekko-persistence-postgres/actions)

> [!WARNING]
> This Pekko fork is a work in progress.
The Akka Persistence Postgres plugin allows for using [PostgreSQL 11](https://www.postgresql.org/) and [Amazon Aurora](https://aws.amazon.com/rds/aurora/) databases as backend for Akka Persistence and Akka Persistence Query.
The Pekko Persistence Postgres plugin allows for using [PostgreSQL 11](https://www.postgresql.org/) and [Amazon Aurora](https://aws.amazon.com/rds/aurora/) databases as backend for Pekko Persistence and Pekko Persistence Query.

It’s been originally created as a fork of [Akka Persistence JDBC plugin](https://github.com/akka/akka-persistence-jdbc), focused on PostgreSQL features such as partitions, arrays, BRIN indexes and others.

Expand All @@ -17,30 +17,30 @@ This plugin supports different schema variants for different use-cases: from sma
through the ones with a small, finite number of persistent actors but each with huge and still growing journals,
to the services with an increasing number of unique persistent actors.

You can read more about DAOs and schema variants in [the official documentation](https://swissborg.github.io/akka-persistence-postgres/features#support-for-partitioned-tables).
You can read more about DAOs and schema variants in [the official documentation](https://swissborg.github.io/pekko-persistence-postgres/features#support-for-partitioned-tables).

## Adding Akka Persistence Postgres to your project
## Adding Pekko Persistence Postgres to your project

To use `akka-persistence-postgres` in your SBT project, add the following to your `build.sbt`:
To use `pekko-persistence-postgres` in your SBT project, add the following to your `build.sbt`:

```scala
libraryDependencies += "com.swissborg" %% "akka-persistence-postgres" % "0.6.0"
libraryDependencies += "com.swissborg" %% "pekko-persistence-postgres" % "0.1.0"
```

For a maven project add:
```xml
<dependency>
<groupId>com.swissborg</groupId>
<artifactId>akka-persistence-postgres_2.13</artifactId>
<version>0.6.0</version>
<artifactId>pekko-persistence-postgres_2.13</artifactId>
<version>0.1.0</version>
</dependency>
```
to your `pom.xml`.

## Enabling Akka Persistence Postgres in your project
## Enabling Pekko Persistence Postgres in your project
To use this plugin instead of the default one, add the following to application.conf:
```hocon
akka.persistence {
pekko.persistence {
journal.plugin = "postgres-journal"
snapshot-store.plugin = "postgres-snapshot-store"
}
Expand All @@ -51,20 +51,22 @@ PersistenceQuery(system).readJournalFor[PostgresReadJournal](PostgresReadJournal
```

## Documentation
* [Akka Persistence Postgres documentation](https://swissborg.github.io/akka-persistence-postgres/)
* [Pekko Persistence Postgres documentation](https://swissborg.github.io/pekko-persistence-postgres/)
* [demo-akka-persistence-postgres](https://github.com/mkubala/demo-akka-persistence-postgres)
> :warning: Please note that this library is based on Pekko, but demo uses an older version of `akka-persistence-postgres` and there might be inconsistencies between the documentation and the provided code
## Key features when compared to the original Akka Persistence JDBC plugin

## Key features when compared to the original Pekko Persistence JDBC plugin

### BRIN index on the ordering column
This plugin has been re-designed in terms of handling very large journals.
The original plugin (akka-persistence-jdbc) uses B-Tree indexes on three columns: `ordering`, `persistence_id` and `sequence_number`. They are great in terms of the query performance and guarding column(s) data uniqueness, but they require relatively a lot of memory.
The original plugin (pekko-persistence-jdbc) uses B-Tree indexes on three columns: `ordering`, `persistence_id` and `sequence_number`. They are great in terms of the query performance and guarding column(s) data uniqueness, but they require relatively a lot of memory.


Wherever it makes sense, we decided to use more lightweight [BRIN indexes](https://www.postgresql.org/docs/11/brin-intro.html).

### Tags as an array of int
Akka-persistence-jdbc stores all tags in a single column as String separated by an arbitrary separator (by default it’s a comma character).
Pekko-persistence-jdbc stores all tags in a single column as String separated by an arbitrary separator (by default it’s a comma character).

This solution is quite portable, but not perfect. Queries rely on the `LIKE ‘%tag_name%`’ condition and some additional work needs to be done in order to filter out tags that don't fully match the input `tag_name` (imagine a case when you have the following tags: _healthy_, _unhealthy_ and _neutral_ and want to find all events tagged with _healthy_. The query will return events tagged with both, _healthy_ and _unhealthy_ tags).

Expand All @@ -77,7 +79,7 @@ When you have big volumes of data and they keep growing, appending events to the
Postgres allows you to split your data between smaller tables (logical partitions) and attach new partitions on demand. Partitioning also applies to indexes, so instead of a one huge B-Tree you can have a number of capped tables with smaller indexes.


You can read more on how Akka Persistence Postgres leverages partitioning in the _Supported journal schema variants_ section below.
You can read more on how Pekko Persistence Postgres leverages partitioning in the _Supported journal schema variants_ section below.

### Minor PostgreSQL optimizations
Beside the aforementioned major changes we did some minor optimizations, like changing the column ordering for [more efficient space utilization](https://www.2ndquadrant.com/en/blog/on-rocks-and-sand/).
Expand All @@ -99,7 +101,7 @@ This variant is aimed for services that have a finite and/or small number of uni

In order to start using partitioned journal, you have to create either a partitioned table (here is [the schema](core/src/test/resources/schema/postgres/nested-partitions-schema.sql)) and set the Journal DAO FQCN:
```
postgres-journal.dao = "akka.persistence.postgres.journal.dao.NestedPartitionsJournalDao"
postgres-journal.dao = "org.apache.pekko.persistence.postgres.journal.dao.NestedPartitionsJournalDao"
```

The size of the nested partitions (`sequence_number`’s range) can be changed by setting `postgres-journal.tables.journal.partitions.size`. By default partition size is set to `10000000` (10M).
Expand All @@ -119,19 +121,19 @@ Keep in mind that the default maximum length for a table name in Postgres is 63
## Migration

Please see the documentation regarding migrations [here](https://swissborg.github.io/akka-persistence-postgres/migration).
Please see the documentation regarding migrations [here](https://swissborg.github.io/pekko-persistence-postgres/migration).

## Contributing
We are also always looking for contributions and new ideas, so if you’d like to join the project, check out the [open issues](https://github.com/SwissBorg/akka-persistence-postgres/issues), or post your own suggestions!
We are also always looking for contributions and new ideas, so if you’d like to join the project, check out the [open issues](https://github.com/SwissBorg/pekko-persistence-postgres/issues), or post your own suggestions!

## Sponsors

Development and maintenance of akka-persistence-postgres is sponsored by:
Development and maintenance of pekko-persistence-postgres is sponsored by:

![SoftwareMill](https://raw.githubusercontent.com/SwissBorg/akka-persistence-postgres/master/docs/assets/softwaremill-logo.png)
![SoftwareMill](https://raw.githubusercontent.com/SwissBorg/pekko-persistence-postgres/master/docs/assets/softwaremill-logo.png)

[SoftwareMill](https://softwaremill.com) is a software development and consulting company. We help clients scale their business through software. Our areas of expertise include backends, distributed systems, blockchain, machine learning and data analytics.

![SwissBorg](https://raw.githubusercontent.com/SwissBorg/akka-persistence-postgres/master/docs/assets/swissborg-logo.png)
![SwissBorg](https://raw.githubusercontent.com/SwissBorg/pekko-persistence-postgres/master/docs/assets/swissborg-logo.png)

[SwissBorg](https://swissborg.com) makes managing your crypto investment easy and helps control your wealth.
8 changes: 4 additions & 4 deletions docs/_config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,13 @@
# you will see them accessed via {{ site.title }}, {{ site.email }}, and so on.
# You can create any custom variable you would like, and they will be accessible
# in the templates via {{ site.myvariable }}.
title: Akka Persistence Postgres
title: Pekko Persistence Postgres
description: >- # this means to ignore newlines until "baseurl:"
The Akka Persistence Postgres plugin allows for using PostgreSQL 11 and Amazon Aurora databases as backend for Akka Persistence and Akka Persistence Query.
The Pekko Persistence Postgres plugin allows for using PostgreSQL 11 and Amazon Aurora databases as backend for Pekko Persistence and Pekko Persistence Query.
The main goal is to keep index size and memory consumption on a moderate level while being able to cope with an increasing data volume.
baseurl: "" # the subpath of your site, e.g. /blog
url: "https://swissborg.github.io/akka-persistence-postgres"
repo: "https://github.com/SwissBorg/akka-persistence-postgres/tree/master"
url: "https://swissborg.github.io/pekko-persistence-postgres"
repo: "https://github.com/SwissBorg/pekko-persistence-postgres/tree/master"

# Build settings
markdown: kramdown
Expand Down
26 changes: 13 additions & 13 deletions docs/basic-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,14 +17,14 @@ has_toc: true

---

## Enabling Akka Persistence Postgres plugin
## Enabling Pekko Persistence Postgres plugin

The plugin relies on [Slick-pg](https://github.com/tminglei/slick-pg) to do create the SQL dialect for the database in use, therefore the following must be configured in `application.conf`

Configure `akka-persistence`:
Configure `pekko-persistence`:

- instruct akka persistence to use the `postgres-journal` plugin,
- instruct akka persistence to use the `postgres-snapshot-store` plugin,
- instruct pekko persistence to use the `postgres-journal` plugin,
- instruct pekko persistence to use the `postgres-snapshot-store` plugin,

Configure `slick.db`:

Expand All @@ -40,7 +40,7 @@ Depending on the journal variant, choose the appropriate schema:

## Reference Configuration

akka-persistence-postgres provides the defaults as part of the [reference.conf]({{ site.repo }}/core/src/main/resources/reference.conf). This file documents all the values which can be configured.
pekko-persistence-postgres provides the defaults as part of the [reference.conf]({{ site.repo }}/core/src/main/resources/reference.conf). This file documents all the values which can be configured.

There are several possible ways to configure loading your database connections. Options will be explained below.

Expand All @@ -57,15 +57,15 @@ In order to create only one connection pool which is shared between all journals
### Customized loading of the db connection

It is also possible to load a custom database connection.
In order to do so a custom implementation of [SlickDatabaseProvider]({{ site.repo }}/core/src/main/scala/akka/persistence/postgres/db/SlickExtension.scala#L46-L54)
In order to do so a custom implementation of [SlickDatabaseProvider]({{ site.repo }}/core/src/main/scala/org/apache/pekko/persistence/postgres/db/SlickExtension.scala#L46-L54)
needs to be created. The method that need to be implemented supply the Slick `Database` to the journals.

To enable your custom `SlickDatabaseProvider`, the fully qualified class name of the `SlickDatabaseProvider`
needs to be configured in the application.conf. In addition, you might want to consider whether you want
the database to be closed automatically:

```hocon
akka-persistence-postgres {
pekko-persistence-postgres {
database-provider-fqcn = "com.mypackage.CustomSlickDatabaseProvider"
}
postgres-journal {
Expand Down Expand Up @@ -93,7 +93,7 @@ postgres-journal {
When using the `use-shared-db = slick` setting, the follow configuration can serve as an example:

```hocon
akka-persistence-postgres {
pekko-persistence-postgres {
shared-databases {
slick {
jndiName = "java:/jboss/datasources/bla"
Expand All @@ -119,19 +119,19 @@ You can find the schema [here]({{ site.repo }}/core/src/test/resources/schema/po

### Using flat journal

This is the default variant, a [schema without any partitions]({{ site.repo }}/core/src/test/resources/schema/postgres/plain-schema.sql) similar to what's used by Akka Persistence JDBC.
This is the default variant, a [schema without any partitions]({{ site.repo }}/core/src/test/resources/schema/postgres/plain-schema.sql) similar to what's used by Pekko Persistence JDBC.

You do not have to override anything in order to start using it, although if you'd like to set it up explicitly, here's the necessary config:

```hocon
postgres-journal.dao = "akka.persistence.postgres.journal.dao.FlatJournalDao"
postgres-journal.dao = "org.apache.pekko.persistence.postgres.journal.dao.FlatJournalDao"
```

### Using journal partitioned by persistence id and sequence number

In order to start using journal with nested partitions, you have to create a table with nested partitions (here is [the schema]({{ site.repo }}/core/src/test/resources/schema/postgres/nested-partitions-schema.sql)) and set the Journal DAO FQCN:
```hocon
postgres-journal.dao = "akka.persistence.postgres.journal.dao.NestedPartitionsJournalDao"
postgres-journal.dao = "org.apache.pekko.persistence.postgres.journal.dao.NestedPartitionsJournalDao"
```

#### Partition size
Expand All @@ -156,7 +156,7 @@ Keep in mind that the default maximum length for a table name in Postgres is 63

In order to start using partitioned journal, you have to apply [this schema]({{ site.repo }}/core/src/test/resources/schema/postgres/partitioned-schema.sql) and set the Journal DAO FQCN:
```hocon
postgres-journal.dao = "akka.persistence.postgres.journal.dao.PartitionedJournalDao"
postgres-journal.dao = "org.apache.pekko.persistence.postgres.journal.dao.PartitionedJournalDao"
```

#### Partition size
Expand Down Expand Up @@ -184,4 +184,4 @@ Default value is **1 hour**.
## Explicitly shutting down the database connections

The plugin automatically shuts down the HikariCP connection pool when the ActorSystem is terminated.
This is done using [ActorSystem.registerOnTermination](https://doc.akka.io/api/akka/current/akka/actor/ActorSystem.html).
This is done using [ActorSystem.registerOnTermination](https://pekko.apache.org/api/pekko/current/org/apache/pekko/actor/ActorSystem.html#registerOnTermination(code:Runnable):Unit).
Loading

0 comments on commit 94e9b4a

Please sign in to comment.