Add API to prestart threads in threadpools

[catlee]

For my use case I would like to ensure that when I create a thread pool with min_threads > 0, that the minimum number of workers are created immediately.

The Java interface for ThreadPoolExecutor calls this "prestart". For example: prestartCoreThread and prestartAllCoreThreads: https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ThreadPoolExecutor.html#prestartCoreThread--

I have a draft PR (here) that implements a similar API for the CRuby implementation (I left the JRuby implementation for later). It adds both methods, as well as a prestart option to the initializer.

Is this an API change you would consider accepting?

[eregon]

What is the advantage of doing so?
The disadvantage is it likely causes extra resource consumption (CPU & memory).

[catlee]

The advantage is that you get slightly improved latency on handling the first few items that are posted to the pool. On my system, I see about a 0.5ms improvement to handling the first few items when using prestart.

[eregon]

I see. Could you share a repro for that? I'd like to try it locally.

[catlee]

Here's how I'm trying to measure the impact:

require "concurrent"

def gettime
  Process.clock_gettime(Process::CLOCK_MONOTONIC)
end

def measure_latency(prestart)
  pool = Concurrent::FixedThreadPool.new(1, prestart: prestart)
  times = []
  start = gettime
  pool.post { times << (gettime - start) }
  pool.shutdown
  pool.wait_for_termination
  times.first
end

def percentiles(times, p)
  times.sort!
  times[(times.size * p).ceil - 1]
end

n = 1000
no_prestart_times = n.times.map { measure_latency(false) }
prestart_times = n.times.map { measure_latency(true) }

puts "No prestart:"
puts "  50th percentile: #{percentiles(no_prestart_times, 0.5)}"
puts "  90th percentile: #{percentiles(no_prestart_times, 0.9)}"
puts "  99th percentile: #{percentiles(no_prestart_times, 0.99)}"

puts "Prestart:"
puts "  50th percentile: #{percentiles(prestart_times, 0.5)}"
puts "  90th percentile: #{percentiles(prestart_times, 0.9)}"
puts "  99th percentile: #{percentiles(prestart_times, 0.99)}"

puts "Delta:"
puts "  50th percentile: #{percentiles(no_prestart_times, 0.5) - percentiles(prestart_times, 0.5)}"
puts "  90th percentile: #{percentiles(no_prestart_times, 0.9) - percentiles(prestart_times, 0.9)}"
puts "  99th percentile: #{percentiles(no_prestart_times, 0.99) - percentiles(prestart_times, 0.99)}"

[bensheldon]

I think a small place to start would be to expose a Ruby equivalent of #prestartCoreThread: which creates a single thread unless max threads is reached.

Then one could manage the lifecycle decision themselves of whether they want to do something like:

executor = Concurrent::ThreadPoolExecutor.new(max_threads: 5)

# immediately, or maybe later...
executor.max_length.times { executor.add_worker }
# or...
executor.min_length.times { executor.add_worker }
# or...
arbitrary_number.times { executor.add_worker }