Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for partial unique indexes #237

Merged
merged 3 commits into from
Mar 26, 2024
Merged

Add support for partial unique indexes #237

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e2c92b8
Add support for partial unique indexes
thgreasi Mar 21, 2024
cbd1177
Add support for unique indexes with NOT DISTINCT NULLS
thgreasi Mar 21, 2024
6d00b7d
Export the generateRuleHashAcronym helper
thgreasi Mar 21, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 43 additions & 7 deletions src/AbstractSQLCompiler.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ export { Binding, SqlResult } from './AbstractSQLRules2SQL';
import type { SbvrType } from '@balena/sbvr-types';
import sbvrTypes from '@balena/sbvr-types';
import * as _ from 'lodash';
import { optimizeSchema } from './AbstractSQLSchemaOptimiser';
import { optimizeSchema, generateRuleSlug } from './AbstractSQLSchemaOptimiser';
import {
getReferencedFields,
getRuleReferencedFields,
Expand Down Expand Up @@ -449,6 +449,15 @@ export interface Trigger {
level: 'ROW' | 'STATEMENT';
when: 'BEFORE' | 'AFTER' | 'INSTEAD OF';
}
export interface Index {
type: string;
fields: string[];
name?: string;
/** For rules converted to partial unique indexes this holds the actual rule expression */
description?: string;
distinctNulls?: boolean;
predicate?: BooleanTypeNodes;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I got the name from the PG CREATE INDEX synopsis.
See: https://www.postgresql.org/docs/16/sql-createindex.html

}
export interface Check {
description?: string;
name?: string;
Expand All @@ -466,10 +475,7 @@ export interface AbstractSqlTable {
resourceName: string;
idField: string;
fields: AbstractSqlField[];
indexes: Array<{
type: string;
fields: string[];
}>;
indexes: Index[];
primitive: false | string;
triggers?: Trigger[];
checks?: Check[];
Expand Down Expand Up @@ -815,6 +821,7 @@ ${compileRule(definitionAbstractSql as AbstractSqlQuery, engine, true).replace(
const foreignKeys: string[] = [];
const depends: string[] = [];
const createSqlElements: string[] = [];
const createIndexes: string[] = [];

for (const field of table.fields) {
const { fieldName, references, dataType, computed } = field;
Expand Down Expand Up @@ -873,9 +880,36 @@ $$;`);

createSqlElements.push(...foreignKeys);
for (const index of table.indexes) {
createSqlElements.push(
index.type + '("' + index.fields.join('", "') + '")',
let nullsSql = '';
if (index.distinctNulls != null) {
nullsSql =
index.distinctNulls === false
? ` NULLS NOT DISTINCT`
: ` NULLS DISTINCT`;
}
// Non-partial indexes are added directly to the CREATE TABLE statement
if (index.predicate == null) {
createSqlElements.push(
index.type + nullsSql + '("' + index.fields.join('", "') + '")',
);
continue;
}
if (index.name == null) {
throw new Error('No name provided for partial index');
}
const comment = index.description
? `-- ${index.description.replaceAll(/\r?\n/g, '\n-- ')}\n`
: '';
const whereSql = compileRule(
index.predicate as AbstractSqlQuery,
engine,
true,
);
createIndexes.push(`\
${comment}\
CREATE ${index.type} INDEX IF NOT EXISTS "${index.name}"
ON "${table.name}" ("${index.fields.join('", "')}")${nullsSql}
WHERE (${whereSql});`);
}

if (table.checks) {
Expand Down Expand Up @@ -932,6 +966,7 @@ $$`);
CREATE TABLE ${ifNotExistsStr}"${table.name}" (
${createSqlElements.join('\n,\t')}
);`,
...createIndexes,
...createTriggers,
],
dropSQL: [...dropTriggers, `DROP TABLE "${table.name}";`],
Expand Down Expand Up @@ -1051,6 +1086,7 @@ CREATE TABLE ${ifNotExistsStr}"${table.name}" (
const generateExport = (engine: Engines, ifNotExists: boolean) => {
return {
optimizeSchema,
generateRuleSlug,
compileSchema: (abstractSqlModel: AbstractSqlModel) =>
compileSchema(abstractSqlModel, engine, ifNotExists),
compileRule: (abstractSQL: AbstractSqlQuery) =>
Expand Down
18 changes: 12 additions & 6 deletions src/AbstractSQLSchemaOptimiser.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,17 @@ const countFroms = (n: AbstractSqlType[]) => {
return count;
};

export const generateRuleSlug = (
tableName: string,
ruleBody: AbstractSqlType,
) => {
const sha = sbvrTypes.SHA.validateSync(
`${tableName}$${JSON.stringify(ruleBody)}`,
).replace(/^\$sha256\$/, '');
// Trim the trigger to a max of 63 characters, reserving at least 32 characters for the hash
return `${tableName.slice(0, 30)}$${sha}`.slice(0, 63);
Comment on lines +39 to +43
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it'd make sense for this to take inspiration from the roughly equivalent code in https://github.com/balena-io-modules/odata-to-abstract-sql/blob/v6.2.4/src/odata-to-abstract-sql.ts#L393-L396 - the things of note to me are:

  1. using base36 allows it to still be a string but also fit as much of the hash in as possible in the same number of characters (vs hex/base 16)
  2. adjusting the truncation dynamically based upon the actual strings
  3. using a named constant for MAX_ALIAS_LENGTH rather than a magic number for which the reasoning is not as clear

};

export const optimizeSchema = (
abstractSqlModel: AbstractSqlModel,
createCheckConstraints: boolean = true,
Expand Down Expand Up @@ -110,10 +121,6 @@ export const optimizeSchema = (
convertReferencedFieldsToFields(whereNode);

const tableName = fromNode[1];
const sha = sbvrTypes.SHA.validateSync(
`${tableName}$${JSON.stringify(ruleBody)}`,
).replace(/^\$sha256\$/, '');

const table = _.find(
abstractSqlModel.tables,
(t) => t.name === tableName,
Expand All @@ -122,8 +129,7 @@ export const optimizeSchema = (
table.checks ??= [];
table.checks!.push({
description: ruleSE,
// Trim the trigger to a max of 63 characters, reserving at least 32 characters for the hash
name: `${tableName.slice(0, 30)}$${sha}`.slice(0, 63),
name: generateRuleSlug(tableName, ruleBody),
abstractSql: whereNode,
});
return;
Expand Down
Loading