Duplicate events when adding Appender Listener?

[will-sargent-eero]

I've added a custom appender listener that will write events to Sentry -- nothing very complicated -- and now I'm seeing duplicate log entries in some cases. I know they're duplicates because I have a unique id added to each LoggingEvent, and I'm seeing the same @id entry in there twice -- and this wasn't happening before I added the listener. This is happening on the local filesystem, so it's not filebeat or some log aggregation bug.

I'm still tracking down this issue so I don't have enough data to determine whether it's a bug in the async appender, but I don't see how I could get the same event written out twice otherwise.

Any thoughts?

[brenuart]

Hi @will-sargent-eero,

What you are reporting looks very strange indeed :-(
Are you talking about registering an AppenderListener to a LoggingEventAsyncDisruptorAppender?
Do you have any sample application I could use to reproduce the issue? I made several attempts but without success so far...

The only reason I see so far that may cause an event to be forwarded twice to the wrapped appender is if the Disruptor EventHandler does not update the sequence correctly (this is handled by EventClearingEventHandler). However, I don't see how this part could be affected by the registration of an new AppenderListener... I'm puzzled.

Some more information would definitely be useful to troubleshoot this issue.
I'm waiting for your feedback.

[will-sargent-eero]

I'll see what I can do to reproduce the issue in a sample project.

[will-sargent-eero]

I think the basic issue is that the Sentry.captureEvent may block internally, and that could cause issues in event clearing:

class SentryFailureSummaryListener[E <: DeferredProcessingAware] extends ContextAwareBase with AppenderListener[E] {
   override def eventAppendFailed(appender: Appender[E], event: E, reason: Throwable): Unit = {
          val event = new SentryEvent
          val message = new Message

          val ctx = getContext
          message.setMessage(s"eventAppendFailed: append failed from appender ${appender.getName}")
          event.setThrowable(reason)
          event.setMessage(message)

          event.setLevel(SentryLevel.ERROR)
          event.setLogger(getClass.getName)

         Sentry.captureEvent(event)
  }
}