fix: store v3 validate cursor & remove messages

[SionoiS]

Fixes #2635 and #2634

[github-actions]

You can find the image built from this PR at

quay.io/wakuorg/nwaku-pr:2636-rln-v2

Built from 6f4da99

[github-actions]

You can find the image built from this PR at

quay.io/wakuorg/nwaku-pr:2636-rln-v1

Built from 6f4da99

[richard-ramos]

For a message that exists in the DB, the response is incorrect. It should have included a WakuMessageKeyValue with just the message hash.

REQUEST
{"request_id":"d4b573abb709d067da8afca3ec7341a8fbebee09923b4ade414e6b510bb6e24f","include_data": false, "message_hashes":["PmqobzCu78df5nprhzOUUu0e6pCFcOACjQpJistzVKM="],"pagination_forward":true,"pagination_limit":20}

RESPONSE
{"request_id":"d4b573abb709d067da8afca3ec7341a8fbebee09923b4ade414e6b510bb6e24f","status_code":200,"status_desc":"OK"}

If I change the include_data attribute to true, the response returns as expected (hash + message)

[richard-ramos]

What should happen if I send a request with a valid cursor that does not exist? i.e. 0x0000000000000000000000000000000000000000000000000000000000000000:

{"request_id":"ec560c592484291ca6dfa1491507a45dba55b238c2b271cf0cb81ce1e0590fce","include_data":true,"pubsub_topic":"/waku/2/default-waku/proto","content_topics":["test"],"time_start":1714155079182460704,"time_end":1714155079285053447,"pagination_cursor":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=","pagination_forward":true,"pagination_limit":20}

Currently it is returning all the records but it seems to me that this behavior is not valid, and perhaps it should return either no results, or an invalid_cursor error. I think we should do the later, and probably do a SELECT EXISTS(SELECT 1 FROM messages WHERE message_hash=?) to determine if the cursor is valid. Since the message_hash is the primary key of the table I imagine this should execute fast with no major impact on the total store query response time

[github-actions]

This PR may contain changes to database schema of one of the drivers.

If you are introducing any changes to the schema, make sure the upgrade from the latest release to this change passes without any errors/issues.

Please make sure the label release-notes is added to make sure upgrade instructions properly highlight this change.

[SionoiS]

Currently it is returning all the records but it seems to me that this behavior is not valid, and perhaps it should return either no results, or an invalid_cursor error. I think we should do the later, and probably do a SELECT EXISTS(SELECT 1 FROM messages WHERE message_hash=?) to determine if the cursor is valid. Since the message_hash is the primary key of the table I imagine this should execute fast with no major impact on the total store query response time

Seams like the old code never validated the cursor, it's only used to order the results. I agree that we should do it though.

[richard-ramos]

Seams like the old code never validated the cursor

I see! It can be done later in a separate PR then. Will open an issue so we don't lose track of this :)
I just tested this PR with my set of tests in go-waku and works fine!

--
EDIT:
Tracking issue in here #2653

[Ivansete-status]

LGTM! Thanks!
Just added some comments that I hope you find useful

[Ivansete-status]

I'm not saying that is incorrect to label those procs as deprecated but how we should handle those? Do they still work/compile?

[SionoiS]

I assumed it would only be a visual thing but I can't find any documentation so 🤷 They do compile/work though.