Repository navigation
Conversation
Message had only its primary key and instanceId, so every lookup by chat or by message id read all messages of the instance. Prisma JSON path filters compile to "key" #> '{field}' and the raw queries use "key"->>'field', so the migration adds an (instanceId, expression) index for both forms: id and remoteJid as text, id, remoteJid and remoteJidAlt as jsonb path.
Prisma cannot declare expression indexes, so they live in a raw SQL migration, with a comment on the Message model in both PostgreSQL schemas. The indexes are built CONCURRENTLY, which prisma migrate deploy allows because it runs each statement outside a transaction, and IF NOT EXISTS keeps the migration a no-op where they already exist.
Refs evolution-foundation#2753
Contributor
Reviewer's GuideThis performance-focused change adds five concurrently built PostgreSQL expression indexes on Entity relationship diagram for Message expression indexeserDiagram
MESSAGE {
string instanceId
json key
index key_id_expression
index key_remoteJid_expression
index key_id_path_expression
index key_remoteJid_path_expression
index key_remoteJidAlt_path_expression
}
File-Level Changes
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
📋 Description
Messagehas only its primary key andMessage_instanceId_idx, so every lookup by chat or by message id reads all messages of the instance. This adds a migration with five(instanceId, expression)indexes onkey.The key is filtered in two forms in the code, and PostgreSQL uses an expression index only for the exact same expression, so both are covered:
"key" ->> 'id',"key" ->> 'remoteJid'getMessage,messages.update,updateMessagesReadedByTimestamp,updateChatUnreadMessages(whatsapp.baileys.service.ts),fetchChats(channel.service.ts),updateChatwootMessageId/getMessageByKeyId(chatwoot.service.ts)Message_instanceId_keyId_idx,Message_instanceId_keyRemoteJid_idx"key" #> '{id}',#> '{remoteJid}',#> '{remoteJidAlt}'key: { path: ['remoteJid'], equals }), e.g.findMessagesMessage_instanceId_keyIdPath_idx,Message_instanceId_keyRemoteJidPath_idx,Message_instanceId_keyRemoteJidAltPath_idxPrisma cannot declare expression indexes, so they live in a raw SQL migration (Prisma's documented way for unsupported features). A comment on the
Messagemodel in both PostgreSQL schemas points to it.🔗 Related Issue
Refs #2753 (the
Messagepart of it; theIsOnWhatsapplookup is not changed here).🧪 Type of Change
🧪 Testing
On a production install (v2.3.7, PostgreSQL, one busy instance with 69,210 messages and several clients polling
findMessages):evolution_postgreswas at 196% CPU on 4 cores and reads timed out under load. After creating the same indexes (builtCONCURRENTLY, 1 to 2 s each on a 194 MB table):key #> '{remoteJid}'key #> '{remoteJidAlt}'key ->> 'id'lookupDatabase CPU went to 55% (one reading before, one after).
On this branch, against an empty PostgreSQL 16 with Prisma 7.8.0:
prisma migrate deployapplies all migrations, this one included (CONCURRENTLY, no error), all sevenMessageindexes are valid (indisvalid), andprisma migrate statusreports the schema up to date.EXPLAIN ANALYZEon 50,000 seeded messages: each of the five query forms uses its index.Drift:
prisma migrate diff --from-config-datasource --to-schema prisma/postgresql-schema.prismareturns an empty migration, so a latermigrate devdoes not try to drop these indexes. Control: a plain index that is not in the schema does show up there asDROP INDEX.prisma validatepasses forpostgresql-schema.prismaandpsql_bouncer-schema.prisma.Manual testing completed
Functionality verified in development environment
No breaking changes introduced
Tested with different connection types (if applicable)
✅ Checklist
📝 Additional Notes
CONCURRENTLY.prisma migrate deploy(7.8.0) runs the statements of a migration file one by one, outside a transaction, so this works inside the migration; I tested it with one and with twoCONCURRENTLYstatements in a file. If a build fails part-way, PostgreSQL leaves an INVALID index under that name: drop it, mark the migration rolled back and deploy again. Installs that already built these indexes by hand under the same names get a no-op (IF NOT EXISTS).Messageadd work to each insert and some disk space. I did not measure that overhead; on the install above the reads were the bottleneck.Summary by Sourcery
Improve PostgreSQL message and chat lookup performance by indexing the JSON message key expressions used throughout the application.
Enhancements: