Ferry

Coalesces individual calls into batched calls to an external system, and limits the load put on it.

Each input submitted through apply is queued. Queued inputs are dispatched together in a single batch call when maxBatchSize inputs have accumulated, or at the latest maxBatchDelay after the oldest of them was submitted. At most maxInFlightBatches batch calls run at the same time, and at most maxQueueing inputs wait for a batch: beyond that, calls are rejected immediately rather than piling up.

Every caller receives its own share of the batch result, or the batch call's failure.