3.2.0

3.2.0 (2022-09-15)

Features

• add api to get backend entry points (0152f5e)
• api: add and_ and or_ helpers (94bd4df)
• api: add argmax and argmin column methods (b52216a)
• api: add distinct to Intersection and Difference operations (cd9a34c)
• api: add ibis.memtable API for constructing in-memory table expressions (0cc6948)
• api: add ibis.sql to easily get a formatted SQL string (d971cc3)
• api: add Table.unpack() and StructValue.lift() APIs for projecting struct fields (ced5f53)
• api: allow transmute-style select method (d5fc364)
• api: implement all bitwise operators (7fc5073)
• api: promote psql to a show_sql public API (877a05d)
• clickhouse: add dataframe external table support for memtables (bc86aa7)
• clickhouse: add enum, ipaddr, json, lowcardinality to type parser (8f0287f)
• clickhouse: enable support for working window functions (310a5a8)
• clickhouse: implement argmin and argmax (ee7c878)
• clickhouse: implement bitwise operations (348cd08)
• clickhouse: implement struct scalars (1f3efe9)
• dask: implement StringReplace execution (1389f4b)
• dask: implement ungrouped argmin and argmax (854aea7)
• deps: support duckdb 0.5.0 (47165b2)
• duckdb: handle query parameters in ibis.connect (fbde95d)
• duckdb: implement argmin and argmax (abf03f1)
• duckdb: implement bitwise xor (ca3abed)
• duckdb: register tables from pandas/pyarrow objects (36e48cc)
• duckdb: support unsigned integer types (2e67918)
• impala: implement bitwise operations (c5302ab)
• implement dropna for SQL backends (8a747fb)
• log: make BaseSQLBackend._log print by default (12de5bb)
• mysql: register BLOB types (1e4fb92)
• pandas: implement argmin and argmax (bf9b948)
• pandas: implement NotContains on grouped data (976dce7)
• pandas: implement StringReplace execution (578795f)
• pandas: implement Contains with a group by (c534848)
• postgres: implement bitwise xor (9b1ebf5)
• pyspark: add option to treat nan as null in aggregations (bf47250)
• pyspark: implement ibis.connect for pyspark (a191744)
• pyspark: implement Intersection and Difference (9845a3c)
• pyspark: implement bitwise operators (33cadb1)
• sqlalchemy: implement bitwise operator translation (bd9f64c)
• sqlalchemy: make ibis.connect with sqlalchemy backends (b6cefb9)
• sqlalchemy: properly implement Intersection and Difference (2bc0b69)
• sql: implement StringReplace translation (29daa32)
• sqlite: implement bitwise xor and bitwise not (58c42f9)
• support table.sort_by(ibis.random()) (693005d)
• type-system: infer pandas' string dtype (5f0eb5d)
• ux: add duckdb as the default backend (8ccb81d)
• ux: use rich to format Table.info() output (67234c3)
• ux: use sqlglot for pretty printing SQL (a3c81c5)
• variadic union, intersect, & difference functions (05aca5a)

Bug Fixes

• api: make sure column names that are already inferred are not overwritten (6f1cb16)
• api: support deferred objects in existing API functions (241ce6a)
• backend: ensure that chained limits respect prior limits (02a04f5)
• backends: ensure select after filter works (e58ca73)
• backends: only recommend installing ibis-foo when foo is a known backend (ac6974a)
• base-sql: fix String-generating backend string concat implementation (3cf78c1)
• clickhouse: add IPv4/IPv6 literal inference (0a2f315)
• clickhouse: cast repeat times argument to UInt64 (b643544)
• clickhouse: fix listing tables from databases with no tables (08900c3)
• compilers: make sure memtable rows have names in the SQL string compilers (18e7f95)
• compiler: use repr for SQL string VALUES data (75af658)
• dask: ensure predicates are computed before projections (5cd70e1)
• dask: implement timestamp-date binary comparisons (48d5058)
• dask: set dask upper bound due to large scale test breakage (796c645), closes #9221
• decimal: add decimal type inference (3fe3fd8)
• deps: update dependency duckdb-engine to >=0.1.8,<0.4.0 (113dc8f)
• deps: update dependency duckdb-engine to >=0.1.8,<0.5.0 (ef97c9d)
• deps: update dependency parsy to v2 (9a06131)
• deps: update dependency shapely to >=1.6,<1.8.4 (0c787d2)
• deps: update dependency shapely to >=1.6,<1.8.5 (d08c737)
• deps: update dependency sqlglot to v5 (f210bb8)
• deps: update dependency sqlglot to v6 (5ca4533)
• duckdb: add missing types (59bad07)
• duckdb: ensure that in-memory connections remain in their creating thread (39bc537)
• duckdb: use fetch_arrow_table() to be able to handle big timestamps (85a76eb)
• fix bug in pandas & dask difference implementation (88a78fa)
• fix dask where implementation (49f8845)
• impala: add date column dtype to impala to ibis type dict (c59e94e), closes #4449
• pandas where supports scalar for left (48f6c1e)
• pandas: fix anti-joins (10a659d)
• pandas: implement timestamp-date binary comparisons (4fc666d)
• pandas: properly handle empty groups when aggregating with GroupConcat (6545f4d)
• pyspark: fix broken StringReplace implementation (22cb297)
• pyspark: make sure ibis.connect works with pyspark (a7ab107)
• pyspark: translate predicates before projections (b3d1c80)
• sqlalchemy: fix float64 type mapping (8782773)
• sqlalchemy: handle reductions with multiple arguments (5b2039b)
• sqlalchemy: implement SQLQueryResult translation (786a50f)
• sql: fix sql compilation after making InMemoryTable a subclass of PhysicalTable (aac9524)
• squash several bugs in sort_by asc/desc handling (222b2ba)
• support chained set operations in SQL backends (227aed3)
• support filters on InMemoryTable exprs (abfaf1f)
• typo: in BaseSQLBackend.compile docstring (0561b13)

Deprecations

• right kwarg in union/intersect/difference (719a5a1)
• duckdb: deprecate path argument in favor of database (fcacc20)
• sqlite: deprecate path argument in favor of database (0f85919)

Performance

• pandas: remove reexecution of alias children (64efa53)
• pyspark: ensure that pyspark DDL doesn't use VALUES (422c98d)
• sqlalchemy: register DataFrames cheaply where possible (ee9f1be)

Documentation

• add to_sql (e2821a5)
• add back constraints for transitive doc dependencies and fix docs (350fd43)
• add coc reporting information (c2355ba)
• add community guidelines documentation (fd0893f)
• add HeavyAI to the readme (4c5ca80)
• add how-to bfill and ffill (ff84027)
• add how-to for ibis+duckdb register (73a726e)
• add how-to section to docs (33c4b93)
• duckdb: add installation note for duckdb >= 0.5.0 (608b1fb)
• fix memtable docstrings (72bc0f5)
• fix flake8 line length issues (fb7af75)
• fix markdown (4ab6b95)
• fix relative links in tutorial (2bd075f), closes #4064 #4201
• make attribution style uniform across the blog (05561e0)
• move the blog out to the top level sidebar for visibility (417ba64)
• remove underspecified UDF doc page (0eb0ac0)