Quickstart
Twenty minutes, start to finish, on a machine you already have set up. This page is written to be followed in order rather than skimmed. If you only read one section, read the last one, because choosing a cap badly is the only way to make this library worse than the buffered channel you already had.
Get it running
go get github.com/tomasleitner/sluice
Go 1.21 or newer. Generics are used in the public API and are not optional
Import it and call sluice.New with a cap and a worker function
Feed it from a loop and check the error returned by Submit
Call Close and then Wait. In that order, always
Step 2, And The Only Real Gotcha
Close stops new submissions. Wait blocks until in-flight work drains. Calling Wait without Close deadlocks, because the workers are waiting for a queue that nobody has told them will end. This is the single most common mistake and it accounts for roughly a third of the issues ever opened. It is now a documented panic with a message that names the mistake rather than a hang.
Wire it into something real
Replace one buffered channel. Only one. Keep the old path behind a flag
Set the cap to your current buffer size so behaviour is comparable
Run your normal load and watch producer latency rather than throughput
Lower the cap until producer latency starts to move
Stop one step above that
Why Producer Latency
Throughput will look flat across a wide range of caps, which makes it useless for tuning. Producer latency is the number that tells you the gate is doing something. When it starts to rise, the ceiling is finally binding.
The Error From Submit
Submit returns an error only after Close. It never returns an error because the pool is full, because being full is not an error, it is the mechanism. Treating a full pool as a failure is how people accidentally rebuild dropping.
Time to first working pool
about 20 min
Lines in the smallest useful example
14
Recommended starting cap
your current buffer size
Allocations per submit after warmup
0
Choosing A Cap
The cap is how many items may be in flight, not how many workers exist. Those are separate numbers and conflating them is the second most common issue. A cap below the worker count starves workers. A cap far above it recreates the unbounded buffer inside the library, which is a slower version of the problem you came here to fix. Start at worker count times two and adjust with the latency method above.
"If the quickstart does not work on a clean machine it is a bug in the quickstart. Open an issue against the docs and we will treat it like any other broken build.
"from the contributing guide
