class Datadog::Transport::Traces::Chunker

Traces chunker

Constants

DEFAULT_MAX_PAYLOAD_SIZE

Trace agent limit payload size of 10 MiB (since agent v5.11.0): github.com/DataDog/datadog-agent/blob/6.14.1/pkg/trace/api/api.go#L46

We set the value to a conservative 5 MiB, in case network speed is slow.

Attributes

encoder[R]
max_size[R]

Public Class Methods

new(encoder, max_size: DEFAULT_MAX_PAYLOAD_SIZE) click to toggle source

Single traces larger than max_size will be discarded.

@param encoder [Datadog::Encoding::Encoder] @param max_size [String] maximum acceptable payload size

# File lib/ddtrace/transport/traces.rb, line 49
def initialize(encoder, max_size: DEFAULT_MAX_PAYLOAD_SIZE)
  @encoder = encoder
  @max_size = max_size
end

Public Instance Methods

encode_in_chunks(traces) click to toggle source

Encodes a list of traces in chunks. Before serializing, all traces are normalized. Trace nesting is not changed.

@param traces [Enumerable<Trace>] list of traces @return [Enumerable[Array]] list of encoded chunks: each containing a byte array and

number of traces
# File lib/ddtrace/transport/traces.rb, line 60
def encode_in_chunks(traces)
  encoded_traces = if traces.respond_to?(:filter_map)
                     # DEV Supported since Ruby 2.7, saves an intermediate object creation
                     traces.filter_map { |t| encode_one(t) }
                   else
                     traces.map { |t| encode_one(t) }.reject(&:nil?)
                   end

  Datadog::Chunker.chunk_by_size(encoded_traces, max_size).map do |chunk|
    [encoder.join(chunk), chunk.size]
  end
end

Private Instance Methods

encode_one(trace) click to toggle source
# File lib/ddtrace/transport/traces.rb, line 75
def encode_one(trace)
  encoded = Encoder.encode_trace(encoder, trace)

  if encoded.size > max_size
    # This single trace is too large, we can't flush it
    Datadog.logger.debug { "Dropping trace. Payload too large: '#{trace.map(&:to_hash)}'" }
    Datadog.health_metrics.transport_trace_too_large(1)

    return nil
  end

  encoded
end