Redacted event remains in the timeline

[wrjlewis]

Description

CSAM was posted in the 'Office of the Matrix.org Foundation' room and I experienced the following after it was redacted..

In ElementX:

• Still there, even after cache clear.

In Element Web:

• Removed from the timeline
• Returned to the timeline after a cache clear (which I did to test if it had truly gone)

When the event was posted in 'Office of the Matrix.org Foundation' room my server produced the following logs:

2024-09-14 05:01:17,796 - synapse.federation.federation_server - 1259 - INFO - _process_incoming_pdus_in_room_inner-417 - handling received PDU in room !dSMpkVKGgQHlgBDSpo:matrix.org: <FrozenEventV3 event_id=$DmdfHQ7_YxEKsY1ACQpMx9PP27g0RI91rcCAsCMclrs, type=m.room.message, state_key=None, outlier=False>

and

2024-09-14 05:01:17,824 - synapse.handlers.federation_event - 1907 - INFO - _process_incoming_pdus_in_room_inner-417-$DmdfHQ7_YxEKsY1ACQpMx9PP27g0RI91rcCAsCMclrs - event's auth_events are different to our calculated auth_events. Claimed but not calculated: [<FrozenEventV3 event_id=$1y-Q06UMYvnmFQtHiPeEhe-bpNVO3ZnwxWf38OwaQQQ, type=m.room.power_levels, state_key=, outlier=True>]. Calculated but not claimed: []

The event was redacted sometime after this. I see the following logs when I grep for the redacted eventid (this is a snippet, there are more identical logs produced afterwards too):

2024-09-14 05:05:55,704 - synapse.storage.databases.main.events_worker - 687 - INFO - notify_interested_services-785 - Withholding redaction $K2TJIOmbH6LRF95-NsiW-yCWuYWK9fCow_o6EItVZs0 whose sender domain matrix.org doesn't match that of redacted event $DmdfHQ7_YxEKsY1ACQpMx9PP27g0RI91rcCAsCMclrs mtrx.nz
2024-09-14 05:12:33,073 - synapse.storage.databases.main.events_worker - 687 - INFO - _censor_redactions-145 - Withholding redaction $K2TJIOmbH6LRF95-NsiW-yCWuYWK9fCow_o6EItVZs0 whose sender domain matrix.org doesn't match that of redacted event $DmdfHQ7_YxEKsY1ACQpMx9PP27g0RI91rcCAsCMclrs mtrx.nz
2024-09-14 06:53:15,266 - synapse.storage.databases.main.events_worker - 687 - INFO - notify_interested_services-798 - Withholding redaction $0rqSRB5qj1PI3rMwJ5uk_XGlw7cOrC3U4AJWE7Y_kRQ whose sender domain matrix.org doesn't match that of redacted event $DmdfHQ7_YxEKsY1ACQpMx9PP27g0RI91rcCAsCMclrs mtrx.nz
2024-09-14 07:02:33,079 - synapse.storage.databases.main.events_worker - 687 - INFO - _censor_redactions-167 - Withholding redaction $0rqSRB5qj1PI3rMwJ5uk_XGlw7cOrC3U4AJWE7Y_kRQ whose sender domain matrix.org doesn't match that of redacted event $DmdfHQ7_YxEKsY1ACQpMx9PP27g0RI91rcCAsCMclrs mtrx.nz
2024-09-14 07:54:53,327 - synapse.storage.databases.main.events_worker - 687 - INFO - POST-110950 - Withholding redaction $0rqSRB5qj1PI3rMwJ5uk_XGlw7cOrC3U4AJWE7Y_kRQ whose sender domain matrix.org doesn't match that of redacted event $DmdfHQ7_YxEKsY1ACQpMx9PP27g0RI91rcCAsCMclrs mtrx.nz
2024-09-14 07:54:53,327 - synapse.storage.databases.main.events_worker - 687 - INFO - POST-110950 - Withholding redaction $K2TJIOmbH6LRF95-NsiW-yCWuYWK9fCow_o6EItVZs0 whose sender domain matrix.org doesn't match that of redacted event $DmdfHQ7_YxEKsY1ACQpMx9PP27g0RI91rcCAsCMclrs mtrx.nz
2024-09-14 08:35:45,805 - synapse.storage.databases.main.events_worker - 687 - INFO - GET-115951 - Withholding redaction $0rqSRB5qj1PI3rMwJ5uk_XGlw7cOrC3U4AJWE7Y_kRQ whose sender domain matrix.org doesn't match that of redacted event $DmdfHQ7_YxEKsY1ACQpMx9PP27g0RI91rcCAsCMclrs mtrx.nz
2024-09-14 08:35:46,441 - synapse.storage.databases.main.events_worker - 687 - INFO - POST-115955 - Withholding redaction $0rqSRB5qj1PI3rMwJ5uk_XGlw7cOrC3U4AJWE7Y_kRQ whose sender domain matrix.org doesn't match that of redacted event $DmdfHQ7_YxEKsY1ACQpMx9PP27g0RI91rcCAsCMclrs mtrx.nz
2024-09-14 08:35:46,441 - synapse.storage.databases.main.events_worker - 687 - INFO - POST-115955 - Withholding redaction $K2TJIOmbH6LRF95-NsiW-yCWuYWK9fCow_o6EItVZs0 whose sender domain matrix.org doesn't match that of redacted event $DmdfHQ7_YxEKsY1ACQpMx9PP27g0RI91rcCAsCMclrs mtrx.nz
2024-09-14 08:51:58,648 - synapse.storage.databases.main.events_worker - 687 - INFO - POST-118061 - Withholding redaction $0rqSRB5qj1PI3rMwJ5uk_XGlw7cOrC3U4AJWE7Y_kRQ whose sender domain matrix.org doesn't match that of redacted event $DmdfHQ7_YxEKsY1ACQpMx9PP27g0RI91rcCAsCMclrs mtrx.nz
2024-09-14 08:51:58,649 - synapse.storage.databases.main.events_worker - 687 - INFO - POST-118061 - Withholding redaction $K2TJIOmbH6LRF95-NsiW-yCWuYWK9fCow_o6EItVZs0 whose sender domain matrix.org doesn't match that of redacted event $DmdfHQ7_YxEKsY1ACQpMx9PP27g0RI91rcCAsCMclrs mtrx.nz
2024-09-14 08:51:58,934 - synapse.storage.databases.main.events_worker - 687 - INFO - GET-118064 - Withholding redaction $0rqSRB5qj1PI3rMwJ5uk_XGlw7cOrC3U4AJWE7Y_kRQ whose sender domain matrix.org doesn't match that of redacted event $DmdfHQ7_YxEKsY1ACQpMx9PP27g0RI91rcCAsCMclrs mtrx.nz

Steps to reproduce

I haven't attempted to reproduce this somehow

Homeserver

wi11.co.uk

Synapse Version

version 1.115.0rc2 (b=HEAD,t=v1.115.0rc2,4c66a7cbed)

Installation Method

pip (from PyPI)

Database

Postgres

Workers

Single process

Platform

Ubuntu

Configuration

I have authenticated_media set to false: which I presume means my server both the deprecated and the updated endpoints for media, and shouldn't be an issue? But mentioning just in case.

Relevant log output

In the description. Happy to provide me,

Anything else that would be useful to know?

Thanks

[wrjlewis]

Weirdly, only now when I clear the cache and reload on ElementX, the 'Office of the Matrix.org Foundation' room is totally empty and I get the 'This is the beginning of ...' message.

On Element Web, previous messages (including the non-redacted one) is still there after a cache clear.

No idea if that is related at all and if it suggests the room state isn't great.

[wrjlewis]

Some people in the room report the same and say the room broke.

It's also the same on element.io.

[wrjlewis]

This was part of the general issue experienced with the room in question, and is being tracked/covered elsewhere.