class JobIteration::ThrottleEnumerator
ThrottleEnumerator
allows you to throttle iterations based on external signal (e.g. database health). @example
def build_enumerator(_params, cursor:) enumerator_builder.build_throttle_enumerator( enumerator_builder.active_record_on_batches( Account.inactive, cursor: cursor ), throttle_on: -> { DatabaseStatus.unhealthy? }, backoff: 30.seconds ) end
The enumerator from above will mimic active_record_on_batches
, except when DatabaseStatus.unhealthy?
starts to return true. In that case, it will re-enqueue the job with a specified backoff.
Public Class Methods
new(enum, job, throttle_on:, backoff:)
click to toggle source
# File lib/job-iteration/throttle_enumerator.rb, line 21 def initialize(enum, job, throttle_on:, backoff:) @enum = enum @job = job @throttle_on = throttle_on @backoff = backoff end
Public Instance Methods
should_throttle?()
click to toggle source
# File lib/job-iteration/throttle_enumerator.rb, line 42 def should_throttle? @throttle_on.call end
to_enum()
click to toggle source
# File lib/job-iteration/throttle_enumerator.rb, line 28 def to_enum Enumerator.new(-> { @enum.size }) do |yielder| @enum.each do |*val| if should_throttle? ActiveSupport::Notifications.instrument("throttled.iteration", job_class: @job.class.name) @job.retry_job(wait: @backoff) throw(:abort, :skip_complete_callbacks) end yielder.yield(*val) end end end