Skip to content

Commit

Permalink
Adjust Lake Loader options (#920)
Browse files Browse the repository at this point in the history
  • Loading branch information
stanch authored Jun 11, 2024
1 parent 451d231 commit 2362848
Show file tree
Hide file tree
Showing 3 changed files with 2 additions and 25 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -52,31 +52,9 @@ import Admonition from '@theme/Admonition';
</table>
</TabItem>

<TabItem value="iceberg-biglake" label="Iceberg / BigLake">
<Admonition type="note" title="Alternative Docker image">
To use the Lake Loader with BigLake support, pull the <code>snowplow/lake-loader-gcp:{`${versions.lakeLoader}`}-biglake</code> image from Docker Hub.
</Admonition>
<table>
<thead>
<tr>
<th>Parameter</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<IcebergBigLakeConfig/>
</tbody>
</table>
</TabItem>

<TabItem value="hudi" label="Hudi">
<Admonition type="note" title="Alternative Docker image">
To use the Lake Loader with Hudi support, pull the appropriate alternative image from Docker Hub:
<ul>
<li><code>snowplow/lake-loader-aws:{`${versions.lakeLoader}`}-hudi</code></li>
<li><code>snowplow/lake-loader-gcp:{`${versions.lakeLoader}`}-hudi</code></li>
<li><code>snowplow/lake-loader-azure:{`${versions.lakeLoader}`}-hudi</code></li>
</ul>
To use the Lake Loader with Hudi support, pull the appropriate alternative image from Docker Hub, e.g. <code>snowplow/lake-loader-aws:{`${versions.lakeLoader}`}-hudi</code>.
</Admonition>
<table>
<thead>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ The Lake Loader is an application that loads Snowplow events to a cloud storage

The Lake Loader supports the three major Open Table Formats: [Delta](https://delta.io/), [Iceberg](https://iceberg.apache.org/) and [Hudi](https://hudi.apache.org/).

For Iceberg tables, the loader supports [AWS Glue](https://docs.aws.amazon.com/glue/) and [GCP BigLake](https://cloud.google.com/bigquery/docs/iceberg-tables) as Iceberg catalogs.
For Iceberg tables, the loader supports [AWS Glue](https://docs.aws.amazon.com/glue/) as catalog.

:::

Expand Down
1 change: 0 additions & 1 deletion docs/storing-querying/storage-options/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,6 @@ All lake loaders are micro-batching.
| Lake | Format | Compatibility | Loader application | Status |
| --- | --- | --- | --- | --- |
| GCS | Delta | Databricks | [Lake Loader](/docs/pipeline-components-and-applications/loaders-storage-targets/lake-loader/index.md) | Early release |
| GCS | Iceberg | BigLake / BigQuery | [Lake Loader](/docs/pipeline-components-and-applications/loaders-storage-targets/lake-loader/index.md) | Early release |
| GCS | JSON | BigQuery | [GCS Loader](/docs/pipeline-components-and-applications/loaders-storage-targets/google-cloud-storage-loader/index.md) | Only recommended for [failed events](/docs/understanding-your-pipeline/failed-events/index.md) |

</TabItem>
Expand Down

0 comments on commit 2362848

Please sign in to comment.