class JobIteration::EnumeratorBuilder

Public Class Methods

new(job, wrapper: Wrapper) click to toggle source
# File lib/job-iteration/enumerator_builder.rb, line 30
def initialize(job, wrapper: Wrapper)
  @job = job
  @wrapper = wrapper
end

Public Instance Methods

active_record_on_batch_relations(scope, cursor:, **args)
active_record_on_batches(scope, cursor:, **args)
active_record_on_records(scope, cursor:, **args)
array(enumerable, cursor:)
build_active_record_enumerator_on_batch_relations(scope, cursor:, **args) click to toggle source

Builds Enumerator from Active Record Relation and enumerates on batches, yielding Active Record Relations. See documentation for build_active_record_enumerator_on_batches.

# File lib/job-iteration/enumerator_builder.rb, line 121
def build_active_record_enumerator_on_batch_relations(scope, cursor:, **args)
  JobIteration::ActiveRecordBatchEnumerator.new(
    scope,
    cursor: cursor,
    **args
  ).each
end
build_active_record_enumerator_on_batches(scope, cursor:, **args) click to toggle source

Builds Enumerator from Active Record Relation and enumerates on batches of records. Each Enumerator tick moves the cursor batch_size rows forward.

batch_size: sets how many records will be fetched in one batch. Defaults to 100.

For the rest of arguments, see documentation for build_active_record_enumerator_on_records

# File lib/job-iteration/enumerator_builder.rb, line 110
def build_active_record_enumerator_on_batches(scope, cursor:, **args)
  enum = build_active_record_enumerator(
    scope,
    cursor: cursor,
    **args
  ).batches
  wrap(self, enum)
end
Also aliased as: active_record_on_batches
build_active_record_enumerator_on_records(scope, cursor:, **args) click to toggle source

Builds Enumerator from Active Record Relation. Each Enumerator tick moves the cursor one row forward.

columns: argument is used to build the actual query for iteration. columns: defaults to primary key:

1) SELECT * FROM users ORDER BY id LIMIT 100

When iteration is resumed, cursor: and columns: values will be used to continue from the point where iteration stopped:

2) SELECT * FROM users WHERE id > $CURSOR ORDER BY id LIMIT 100

columns: can also take more than one column. In that case, cursor will contain serialized values of all columns at the point where iteration stopped.

Consider this example with +columns: [:created_at, :id]+. Here's the query will use on the first iteration:

1) SELECT * FROM `products` ORDER BY created_at, id LIMIT 100

And the query on the next iteration:

2) SELECT * FROM `products`
     WHERE (created_at > '$LAST_CREATED_AT_CURSOR'
       OR (created_at = '$LAST_CREATED_AT_CURSOR' AND (id > '$LAST_ID_CURSOR')))
     ORDER BY created_at, id LIMIT 100

As a result of this query pattern, if the values in these columns change for the records in scope during iteration, they may be skipped or yielded multiple times depending on the nature of the update and the cursor's value. If the value gets updated to a greater value than the cursor's value, it will get yielded again. Similarly, if the value gets updated to a lesser value than the curor's value, it will get skipped.

# File lib/job-iteration/enumerator_builder.rb, line 95
def build_active_record_enumerator_on_records(scope, cursor:, **args)
  enum = build_active_record_enumerator(
    scope,
    cursor: cursor,
    **args
  ).records
  wrap(self, enum)
end
Also aliased as: active_record_on_records
build_array_enumerator(enumerable, cursor:) click to toggle source

Builds Enumerator object from a given array, using cursor as an offset.

# File lib/job-iteration/enumerator_builder.rb, line 49
def build_array_enumerator(enumerable, cursor:)
  unless enumerable.is_a?(Array)
    raise ArgumentError, "enumerable must be an Array"
  end
  if enumerable.any? { |i| defined?(ActiveRecord) && i.is_a?(ActiveRecord::Base) }
    raise ArgumentError, "array cannot contain ActiveRecord objects"
  end
  drop =
    if cursor.nil?
      0
    else
      cursor + 1
    end

  wrap(self, enumerable.each_with_index.drop(drop).to_enum { enumerable.size })
end
Also aliased as: array
build_once_enumerator(cursor:) click to toggle source

Builds Enumerator objects that iterates once.

# File lib/job-iteration/enumerator_builder.rb, line 38
def build_once_enumerator(cursor:)
  wrap(self, build_times_enumerator(1, cursor: cursor))
end
Also aliased as: once
build_throttle_enumerator(enum, throttle_on:, backoff:) click to toggle source
# File lib/job-iteration/enumerator_builder.rb, line 129
def build_throttle_enumerator(enum, throttle_on:, backoff:)
  JobIteration::ThrottleEnumerator.new(
    enum,
    @job,
    throttle_on: throttle_on,
    backoff: backoff
  ).to_enum
end
Also aliased as: throttle
build_times_enumerator(number, cursor:) click to toggle source

Builds Enumerator objects that iterates N times and yields number starting from zero.

# File lib/job-iteration/enumerator_builder.rb, line 43
def build_times_enumerator(number, cursor:)
  raise ArgumentError, "First argument must be an Integer" unless number.is_a?(Integer)
  wrap(self, build_array_enumerator(number.times.to_a, cursor: cursor))
end
Also aliased as: times
once(cursor:)
throttle(enum, throttle_on:, backoff:)
times(number, cursor:)

Private Instance Methods

build_active_record_enumerator(scope, cursor:, **args) click to toggle source
# File lib/job-iteration/enumerator_builder.rb, line 148
def build_active_record_enumerator(scope, cursor:, **args)
  unless scope.is_a?(ActiveRecord::Relation)
    raise ArgumentError, "scope must be an ActiveRecord::Relation"
  end

  JobIteration::ActiveRecordEnumerator.new(
    scope,
    cursor: cursor,
    **args
  )
end