Skip to content

Commit

Permalink
feat: Support string and int arrays (#330)
Browse files Browse the repository at this point in the history
Like the title said, this adds support for
- `ARRAY(BIGINT())`
- `ARRAY(TEXT())`

Technically this is a breaking change. Where previously all array types
were `ARRAY(JSONB())`, now there's more nuance. Unlikely that many
people rely on this behaviour, but we had to write a migration. Should
we increment the major version (or at least the minor) to signal this
change? Or introduce a feature switch?

---------

Co-authored-by: Ruben Vereecken <[email protected]>
Co-authored-by: Edgar Ramírez Mondragón <[email protected]>
  • Loading branch information
3 people authored Jul 17, 2024
1 parent 20142a1 commit a93a12b
Showing 1 changed file with 16 additions and 1 deletion.
17 changes: 16 additions & 1 deletion target_postgres/connector.py
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,7 @@ def prepare_table( # type: ignore[override]
column_object = None
if property_name in columns:
column_object = columns[property_name]

self.prepare_column(
full_table_name=table.fullname,
column_name=property_name,
Expand Down Expand Up @@ -246,6 +247,14 @@ def to_sql_type(self, jsonschema_type: dict) -> sa.types.TypeEngine: # type: ig
json_type_dict["format"] = jsonschema_type["format"]
if encoding := jsonschema_type.get("contentEncoding", False):
json_type_dict["contentEncoding"] = encoding
# Figure out array type, but only if there's a single type
# (no array union types)
if (
"items" in jsonschema_type
and "type" in jsonschema_type["items"]
and isinstance(jsonschema_type["items"]["type"], str)
):
json_type_dict["items"] = jsonschema_type["items"]["type"]
json_type_array.append(json_type_dict)
else:
msg = "Invalid format for jsonschema type: not str or list."
Expand Down Expand Up @@ -282,7 +291,13 @@ def pick_individual_type(self, jsonschema_type: dict):
if "object" in jsonschema_type["type"]:
return JSONB()
if "array" in jsonschema_type["type"]:
return ARRAY(JSONB())
items_type = jsonschema_type.get("items")
if "string" == items_type:
return ARRAY(TEXT())
if "integer" == items_type:
return ARRAY(BIGINT())
else:
return ARRAY(JSONB())

# string formats
if jsonschema_type.get("format") == "date-time":
Expand Down

0 comments on commit a93a12b

Please sign in to comment.