Skip to content

Commit

Permalink
fix: remove InMemoryCollector from liveness check on shutdown (#1349)
Browse files Browse the repository at this point in the history
## Which problem is this PR solving?

related to #1348 

Currently, the collector's timeout for healthcheck is set to 3 seconds.
During a shutdown, there could be a large number of traces or spans that
need to be forwarded to peers. If this is taking longer than 3 seconds,
it will definitely timeout the liveness check.

Since we are shutting down anyway, just let the ShutdownDelay to do the
work and not worry about the liveness.
## Short description of the changes

- Unregister `collector` from Refinery's liveness check
  • Loading branch information
VinozzZ authored and TylerHelmuth committed Oct 16, 2024
1 parent 637a3f7 commit 8c3b91a
Showing 1 changed file with 3 additions and 2 deletions.
5 changes: 3 additions & 2 deletions collect/collect.go
Original file line number Diff line number Diff line change
Expand Up @@ -850,9 +850,10 @@ func (i *InMemCollector) send(trace *types.Trace, sendReason string) {
func (i *InMemCollector) Stop() error {
i.redistributeTimer.Stop()
close(i.done)
// signal the health system to not be ready
// signal the health system to not be ready and
// stop liveness check
// so that no new traces are accepted
i.Health.Ready(CollectorHealthKey, false)
i.Health.Unregister(CollectorHealthKey)

i.mutex.Lock()

Expand Down

0 comments on commit 8c3b91a

Please sign in to comment.