Support unpartitioned tables in Kudu #24674
Open
+41
−4
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
chenjian2664
force-pushed
the
kudu_unpartitioned
branch
2 times, most recently
from
2c165e3 to
68aee9d
Compare
ebyhr
reviewed
Jan 10, 2025
plugin/trino-kudu/src/main/java/io/trino/plugin/kudu/properties/KuduTableProperties.java
Outdated
Show resolved
Hide resolved
plugin/trino-kudu/src/main/java/io/trino/plugin/kudu/properties/KuduTableProperties.java
Outdated
Show resolved
Hide resolved
plugin/trino-kudu/src/main/java/io/trino/plugin/kudu/properties/KuduTableProperties.java
Outdated
Show resolved
Hide resolved
chenjian2664
force-pushed
the
kudu_unpartitioned
branch
2 times, most recently
from
643fdcd to
cd0ac2f
Compare
Praveen2112
reviewed
Jan 10, 2025
|
@Praveen2112 |
Praveen2112
reviewed
Jan 10, 2025
plugin/trino-kudu/src/test/java/io/trino/plugin/kudu/TestKuduConnectorTest.java
Outdated
Show resolved
Hide resolved
plugin/trino-kudu/src/main/java/io/trino/plugin/kudu/properties/KuduTableProperties.java
Show resolved
Hide resolved
plugin/trino-kudu/src/main/java/io/trino/plugin/kudu/properties/KuduTableProperties.java
Outdated
Show resolved
Hide resolved
Comment on lines
111
to
115
| String sqlStatement = (String) computeScalar("SHOW CREATE TABLE " + tableName); | ||
| String tableProperties = sqlStatement.split("\\)\\s*WITH\\s*\\(")[1]; | ||
| // empty partition keys | ||
| assertTableProperty(tableProperties, "partition_by_hash_columns", Pattern.quote("ARRAY[]")); | ||
| assertTableProperty(tableProperties, "partition_by_range_columns", Pattern.quote("ARRAY[]")); |
There was a problem hiding this comment.
Can we check the output of SHOW CREATE .. and assert that String ?
There was a problem hiding this comment.
As a follow-up we can also introduce a SystemTable for Kudu and simplify these further instead of parsing the String.
chenjian2664
force-pushed
the
kudu_unpartitioned
branch
6 times, most recently
from
44b966c to
ab9b3ae
Compare
chenjian2664
force-pushed
the
kudu_unpartitioned
branch
from
ab9b3ae to
c6b18d2
Compare
Praveen2112
approved these changes
Jan 10, 2025
| @@ -153,11 +156,16 @@ else if (!hashColumns2.isEmpty()) { | |||
| } | |||
|
|
|||
| @SuppressWarnings("unchecked") | |||
| List<String> rangeColumns = (List<String>) tableProperties.get(PARTITION_BY_RANGE_COLUMNS); | |||
| List<String> rangeColumns = (List<String>) tableProperties.getOrDefault(PARTITION_BY_RANGE_COLUMNS, ImmutableList.of()); | |||
| requireNonNull(rangeColumns, "rangeColumns is null"); | |||
There was a problem hiding this comment.
nit: rnn check would be redundant in this case.
| @@ -133,7 +134,9 @@ public static PartitionDesign getPartitionDesign(Map<String, Object> tableProper | |||
| requireNonNull(tableProperties); | |||
|
|
|||
| @SuppressWarnings("unchecked") | |||
| List<String> hashColumns = (List<String>) tableProperties.get(PARTITION_BY_HASH_COLUMNS); | |||
| List<String> hashColumns = (List<String>) tableProperties.getOrDefault(PARTITION_BY_HASH_COLUMNS, ImmutableList.of()); | |||
| requireNonNull(hashColumns, "hashColumns is null"); | |||
There was a problem hiding this comment.
nit: rnn check would be redundant in this case.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Fix #24661
The current Kudu connector does not support unpartitioned tables, as it assumes the existence of partition keys. However, Kudu itself allows the creation of unpartitioned tables. Interacting with such tables may result in issues like the error partitioning information is missing, as mentioned in #24661.
Refer IMPALA-5546, unpartitioned tables can be handled by using range partitions without specifying any columns.
Additional context and related issues
Release notes
( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(x) Release notes are required, with the following suggested text: