v2.0.0

Full Changelog: v1.0.0...v2.0.0

This release introduces significant changes to the resque-throttler gem, including a complete overhaul of the rate-limiting logic. The new implementation provides better performance, improved reliability, and a cleaner integration with Resque. The primary changes involve refactoring the throttling mechanism from modifying Resque::Job to extending Resque::Worker, eliminating complex tracking of individual job UUIDs, and introducing Redis locks for concurrency control.

Major Refactor of Throttling Logic

• Old Implementation:
  • Extended Resque::Job and used method aliasing to inject throttling behavior.
  • Tracked individual job UUIDs to monitor job execution and rate limits.
  • Used Redis sets and hash maps to store job states (started_at, ended_at).
  • Required garbage collection of rate limit data for each queue to clean up old job records.
• New Implementation:
  • Extends Resque::Worker to manage rate-limited queue jobs.
  • Introduces Redis locks to prevent race conditions when accessing queues.
  • Uses atomic counters to track the number of jobs processed within the rate limit window.
  • Simplifies the rate-limiting logic by removing the need to track individual job states.
  • Adds methods to reset throttling data for specific queues or all queues.

Key Features

Rate Limit Configuration

Set rate limits per queue using Resque.rate_limit
Resque.rate_limit(:my_queue, at: 5, per: 60) # Allows 5 jobs per 60 seconds

Lock Management

Acquires a Redis lock before processing jobs from a rate-limited queue to ensure only one worker accesses the queue at a time. Locks have a 30-second expiration to prevent deadlocks in case a worker crashes.

Job Counter

Uses Redis INCR and EXPIRE commands to maintain a job counter with an expiration time matching the rate limit period. The counter resets automatically after the expiration period.

Reset Throttling Data

Provides methods to reset locks and job counters

# Reset throttling for a specific queue
Resque.reset_throttling(:my_queue)

# Reset throttling for all rate-limited queues
Resque.reset_throttling

Changes

API Changes:

• Methods and internal logic have changed significantly.
• Any code relying on the old implementation’s internal methods or job tracking will need to be updated.

Removal of UUID Tracking:

• The new implementation does not track individual job UUIDs.