Improve the filtering query, and indexes needed #1504
Conversation
joshk
force-pushed
the
better-indexes-and-a-better-filter-query
branch
from
6468871 to
ba3e7b7
Compare
|
Looks very nice! |
|
Thats a fair question. I've tested it locally, and the filters work as expected, but the test suite should be expanded. |
|
Leaving a note post close because I'm a little concerned with adding in 6 indexes on a table that gets hammered with heartbeat updates. Something to look out for once this gets deployed at smartrent @jjcarstens |
|
Yeah, that's a valid concern. I would imagine it won't have to touch indexes unless the indexed value actually changes? |
| @@ -163,7 +176,7 @@ defmodule NervesHub.Devices do | |||
| where( | |||
| query, | |||
| [d], | |||
| fragment("array_to_string(?, ',') ILIKE ?", d.tags, ^"%#{tag}%") | |||
| fragment("array_to_string(?, ' ', ' ') ILIKE ?", d.tags, ^"%#{tag}%") | |||
There was a problem hiding this comment.
What does adding a trailing space do here?
There was a problem hiding this comment.
If this isn't using the string_array_to_string function that we're creating in the migration it's probably not going to use that index.
There was a problem hiding this comment.
Ah, I get it. A third value added 🤦🏼
There was a problem hiding this comment.
Thanks for catching this @esvinson
|
|
||
| def change do | ||
| create index("devices", [:product_id], concurrently: true) | ||
| create index("devices", [:updates_enabled], concurrently: true) |
There was a problem hiding this comment.
Indexes on boolean fields probably won't be used.
There was a problem hiding this comment.
My general thought was "index more stuff, see what is used, remove indexes as we see fit"
This was some general advice from some Postgres contributors at Heroku that I was friendly with.
| @disable_ddl_transaction true | ||
|
|
||
| def change do | ||
| create index("devices", [:product_id], concurrently: true) |
There was a problem hiding this comment.
Additional note, if we create the indexes using create_if_not_exists we can create some of the indexes in advance to help with performance.
As indexes are separate, this should be fine. It would only be a concern if the indexed data is updated often. |
A few things came up while looking into #1501
This PR simplifies the filter query, adds indexes, including some
pg_trgmgoodness.The
pg_trgmbits have been inspired by: