Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: PagoPA-2483 V3 DB debtor fields 2️⃣ ㏈ #278

Open
wants to merge 19 commits into
base: main
Choose a base branch
from
Open
Changes from 4 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
de7d274
PAGOPA-2483 DB debtor fields: create new fields and enhance them
pagopa-github-bot Dec 16, 2024
73e25e5
PAGOPA-2483 DB debtor fields: fix
pagopa-github-bot Dec 16, 2024
78ed383
Merge branch 'PAGOPA-2483-v3-DB-debtor-fields' of
Dec 17, 2024
80d9750
Merge branch 'main' into PAGOPA-2483-v3-DB-debtor-fields
alessio-acitelli Dec 17, 2024
41b5649
[PAGOPA-2483] v3 DB debtor fields: procedure for standalone transaction
Dec 17, 2024
777f950
Merge branch 'PAGOPA-2483-v3-DB-debtor-fields' of
Dec 18, 2024
88e3942
Merge branch 'PAGOPA-2483-v3-DB-debtor-fields' of
Dec 18, 2024
57bc1f1
Merge branch 'PAGOPA-2483-v3-DB-debtor-fields' of
Dec 18, 2024
32e9916
[PAGOPA-2483] v3 DB debtor fields: update to version 23
Dec 18, 2024
132d892
Merge branch 'PAGOPA-2483-v3-DB-debtor-fields' of
Dec 18, 2024
c1457b8
[PAGOPA-2483] v3 DB debtor fields: added default value
Dec 18, 2024
942916e
Merge branch 'PAGOPA-2483-v3-DB-debtor-fields' of https://github.com/…
Dec 19, 2024
dd66962
Merge branch 'main' into PAGOPA-2483-v3-DB-debtor-fields
alessio-acitelli Dec 19, 2024
dac825f
[PAGOPA-2483] v3 DB debtor fields: only new fields without past data
Dec 19, 2024
4d0f116
[PAGOPA-2483] v3 DB debtor fields: added missing script
Dec 19, 2024
a53c9eb
Merge branch 'PAGOPA-2483-v3-DB-debtor-fields' of https://github.com/…
Dec 19, 2024
e7f8914
Merge branch 'main' into PAGOPA-2483-v3-DB-debtor-fields
alessio-acitelli Dec 19, 2024
83965b6
Merge branch 'main' into PAGOPA-2483-v3-DB-debtor-fields
cap-ang Jan 9, 2025
c868ccb
Merge branch 'main' into PAGOPA-2483-v3-DB-debtor-fields
cap-ang Jan 13, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 60 additions & 0 deletions src/main/resources/db/migration/V022__ALTER_PO_DEBTOR_FIELDS.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
-- Step 1: Added columns without default to avoid FULL TABLE SCAN
ALTER TABLE payment_option
ADD COLUMN IF NOT EXISTS fiscal_code varchar(255),
ADD COLUMN IF NOT EXISTS full_name varchar(255),
ADD COLUMN IF NOT EXISTS "type" varchar(255),
ADD COLUMN IF NOT EXISTS street_name varchar(255),
ADD COLUMN IF NOT EXISTS civic_number varchar(255),
ADD COLUMN IF NOT EXISTS postal_code varchar(255),
ADD COLUMN IF NOT EXISTS city varchar(255),
ADD COLUMN IF NOT EXISTS province varchar(255),
ADD COLUMN IF NOT EXISTS region varchar(255),
ADD COLUMN IF NOT EXISTS country varchar(255),
ADD COLUMN IF NOT EXISTS email varchar(255),
ADD COLUMN IF NOT EXISTS phone varchar(255);

-- Step 2: Batch update to minimize lock
DO $$
DECLARE
batch_size INT := 10000; -- Batch size
rows_updated INT;
BEGIN
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The cycle is all in one transaction, what is the acquired lock?

Copy link
Collaborator Author

@alessio-acitelli alessio-acitelli Dec 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, it's a single transaction so the lock on the table is held until the end (even if I do updates by block size). I correct.

LOOP
WITH rows_to_update AS (
SELECT po.payment_position_id, pp.fiscal_code, pp.full_name, pp."type",
cap-ang marked this conversation as resolved.
Show resolved Hide resolved
pp.street_name, pp.civic_number, pp.postal_code, pp.city,
pp.province, pp.region, pp.country, pp.email, pp.phone
FROM payment_option AS po
JOIN payment_position AS pp
ON po.payment_position_id = pp.id
WHERE po.fiscal_code IS NULL -- To update only the rows not yet processed
LIMIT batch_size
)
UPDATE payment_option AS po
SET fiscal_code = rows_to_update.fiscal_code,
full_name = rows_to_update.full_name,
"type" = rows_to_update."type",
street_name = rows_to_update.street_name,
civic_number = rows_to_update.civic_number,
postal_code = rows_to_update.postal_code,
city = rows_to_update.city,
province = rows_to_update.province,
region = rows_to_update.region,
country = rows_to_update.country,
email = rows_to_update.email,
phone = rows_to_update.phone
FROM rows_to_update
WHERE po.payment_position_id = rows_to_update.payment_position_id;

GET DIAGNOSTICS rows_updated = ROW_COUNT;

EXIT WHEN rows_updated = 0;
END LOOP;
END $$;

-- Step 3: Set the columns that must always be filled to NOT NULL
ALTER TABLE payment_option
ALTER COLUMN fiscal_code SET NOT NULL,
ALTER COLUMN full_name SET NOT NULL,
ALTER COLUMN "type" SET DEFAULT 'F',
ALTER COLUMN "type" SET NOT NULL;
Loading