Go - Data Races and Channels
Concept: Data races occur when multiple goroutines concurrently access and modify shared memory, leading to unpredictable and erroneous program behavior.
Why it matters: A data race is undefined behavior in Go — the result is not just wrong, it can change between runs, making bugs invisible in testing and catastrophic in production.
package main
import (
"fmt"
"sync"
)
// demonstrates three categories of data race
func main() {
var wg sync.WaitGroup
// race 1: concurrent counter increment — read-modify-write not atomic
counter := 0
for i := 0; i < 100; i++ {
wg.Add(1)
go func() {
defer wg.Done()
counter++ // three instructions: LOAD, ADD, STORE — preemptible between any
}()
}
wg.Wait()
fmt.Printf("racy counter: %d (expected 100)\n", counter)
// race 2: concurrent map write — map is not concurrency safe
// m := map[string]int{}
// go func() { m["a"] = 1 }() // concurrent write — likely panic
// go func() { m["b"] = 2 }() // concurrent write
// race 3: read after write without synchronization
data := 0
ready := false
go func() { data = 42; ready = true }() // write
for !ready {} // read — no happens-before guarantee
fmt.Println(data) // may print 0 — CPU reordering
}
// go run -race main.go — shows all three races
Gotcha: Thinking sequential-looking code is race-free when goroutines are involved — any shared variable accessed from multiple goroutines without sync is a race.
Concept: The Go race detector is a valuable tool for identifying potential data races during testing but does not guarantee their complete absence.
Why it matters: The race detector only catches races that actually execute — a race that requires specific timing may only surface under production load, not in tests.
package main
import (
"fmt"
"sync"
"testing"
)
// race detector: add -race flag to go test or go run
// go test -race ./...
// go run -race main.go
type SharedState struct {
mu sync.Mutex
value int
}
func (s *SharedState) Increment() {
s.mu.Lock()
defer s.mu.Unlock()
s.value++
}
func (s *SharedState) Get() int {
s.mu.Lock()
defer s.mu.Unlock()
return s.value
}
// race detector catches this during test execution IF the race actually triggers
func TestRaceDetection(t *testing.T) {
s := &SharedState{}
var wg sync.WaitGroup
for i := 0; i < 1000; i++ {
wg.Add(1)
go func() {
defer wg.Done()
s.Increment() // protected: no race
}()
}
wg.Wait()
if got := s.Get(); got != 1000 {
t.Errorf("got %d want 1000", got)
}
fmt.Println("race-free result:", s.Get())
}
func main() { fmt.Println("run: go test -race to enable race detection") }
Gotcha: Running tests without -race in CI — the race detector adds ~5–10× overhead but is mandatory for concurrent code; always include it in your test pipeline.
Concept: Atomic instructions provide a low-level, hardware-supported mechanism for synchronizing access to shared memory.
Why it matters: Atomic operations complete in a single CPU instruction — no lock overhead, no goroutine context switch, maximum throughput for simple numeric operations.
package main
import (
"fmt"
"sync"
"sync/atomic"
)
func main() {
var wg sync.WaitGroup
// atomic: single hardware instruction — indivisible by definition
var atomicCounter int64
for i := 0; i < 10000; i++ {
wg.Add(1)
go func() {
defer wg.Done()
atomic.AddInt64(&atomicCounter, 1) // atomic: fetch-and-add instruction
}()
}
wg.Wait()
fmt.Printf("atomic counter: %d\n", atomic.LoadInt64(&atomicCounter)) // always 10000
// atomic for flags: compare-and-swap (CAS) — conditional atomic update
var flag int32 = 0
swapped := atomic.CompareAndSwapInt32(&flag, 0, 1) // set to 1 only if currently 0
fmt.Printf("CAS swapped: %v, flag: %d\n", swapped, flag) // true, 1
swapped = atomic.CompareAndSwapInt32(&flag, 0, 1) // already 1 — fails
fmt.Printf("CAS swapped: %v, flag: %d\n", swapped, flag) // false, 1
}
Gotcha: Using atomic for a counter but a non-atomic read for the result — counter++ is not atomic even if increments are; always use atomic.LoadInt64 to read.
Concept: Atomic instructions are particularly efficient for simple operations like incrementing counters but have limitations in terms of data size they can handle.
Why it matters: Atomics only work on primitive types (int32, int64, uint64, pointer) — for structs, slices, or maps you must use a mutex.
package main
import (
"fmt"
"sync"
"sync/atomic"
"unsafe"
)
// atomic pointer swap — advanced: swap an entire config struct atomically
type Config struct {
Timeout int
MaxConns int
}
type AtomicConfig struct {
ptr unsafe.Pointer // stores *Config
}
func (ac *AtomicConfig) Load() *Config {
return (*Config)(atomic.LoadPointer(&ac.ptr))
}
func (ac *AtomicConfig) Store(cfg *Config) {
atomic.StorePointer(&ac.ptr, unsafe.Pointer(cfg))
}
func main() {
ac := &AtomicConfig{}
ac.Store(&Config{Timeout: 30, MaxConns: 100})
var wg sync.WaitGroup
for i := 0; i < 5; i++ {
wg.Add(1)
go func(n int) {
defer wg.Done()
cfg := ac.Load() // atomic load — always sees a complete Config
fmt.Printf("reader %d: timeout=%d\n", n, cfg.Timeout)
}(i)
}
// writer atomically replaces config — readers never see partial update
ac.Store(&Config{Timeout: 60, MaxConns: 200})
wg.Wait()
}
Gotcha: Storing a struct by value in atomic.Value then mutating its fields — atomic.Value stores and loads the entire value atomically; mutate the fields on a copy, then Store the new copy.
Concept: Mutexes — a mutex acts like a lock, allowing only one goroutine to access a critical section at a time.
Why it matters: A mutex serializes access to any shared data structure — it's the universal solution for protecting state that atomics can't handle.
package main
import (
"fmt"
"sync"
)
// mutex pattern: lock before access, unlock after — defer guarantees unlock on panic
type RateLimiter struct {
mu sync.Mutex
counters map[string]int
limit int
}
func NewRateLimiter(limit int) *RateLimiter {
return &RateLimiter{
counters: make(map[string]int),
limit: limit,
}
}
func (r *RateLimiter) Allow(key string) bool {
r.mu.Lock()
defer r.mu.Unlock() // always unlocked — even if Allow panics mid-execution
r.counters[key]++
return r.counters[key] <= r.limit
}
func (r *RateLimiter) Reset(key string) {
r.mu.Lock()
defer r.mu.Unlock()
delete(r.counters, key)
}
func main() {
rl := NewRateLimiter(3)
var wg sync.WaitGroup
for i := 0; i < 6; i++ {
wg.Add(1)
go func(n int) {
defer wg.Done()
allowed := rl.Allow("user-1")
fmt.Printf("request %d: allowed=%v\n", n, allowed)
}(i)
}
wg.Wait()
}
Gotcha: Locking a mutex and then calling a function that also tries to lock the same mutex — Go's sync.Mutex is not reentrant; this deadlocks.
Concept: Mutexes provide broader applicability than atomic instructions but can introduce latency and performance overheads if not used carefully.
Why it matters: Every lock acquisition is a potential goroutine park — holding a mutex too long or too often creates contention that serializes what should be parallel work.
package main
import (
"fmt"
"sync"
"time"
)
// bad: mutex held during slow operation — other goroutines starve
type SlowCache struct {
mu sync.Mutex
data map[string]string
}
func (c *SlowCache) GetBad(key string) string {
c.mu.Lock()
defer c.mu.Unlock()
time.Sleep(10 * time.Millisecond) // simulates DB lookup INSIDE the lock — bad
return c.data[key]
}
// good: do expensive work outside the lock — lock only for the map operation
func (c *SlowCache) GetGood(key string) string {
c.mu.Lock()
v, ok := c.data[key]
c.mu.Unlock() // release lock before expensive work
if !ok {
time.Sleep(10 * time.Millisecond) // DB lookup OUTSIDE the lock
c.mu.Lock()
c.data[key] = "fetched"
v = c.data[key]
c.mu.Unlock()
}
return v
}
func main() {
cache := &SlowCache{data: make(map[string]string)}
var wg sync.WaitGroup
start := time.Now()
for i := 0; i < 5; i++ {
wg.Add(1)
go func() {
defer wg.Done()
cache.GetGood("key") // parallel DB fetches — contention reduced
}()
}
wg.Wait()
fmt.Printf("elapsed: %v\n", time.Since(start))
}
Gotcha: Using defer mu.Unlock() when the critical section is a small part of a long function — the lock is held for the entire function duration; unlock explicitly after the critical section.
Concept: Channels are for signaling events, not storing data.
Why it matters: Treating a channel as a queue (high capacity buffer) hides the true nature of the communication — channels are about synchronization and signal, not storage.
package main
import "fmt"
func main() {
// signal with data: one producer, one consumer — rendezvous
result := make(chan int) // unbuffered: signal WITH guaranteed delivery
go func() {
work := 2 + 2
result <- work // signal: "work is done, here is the result"
}()
fmt.Println("waiting for result...")
fmt.Println("got:", <-result) // receive the signal
// signal without data: notify an event occurred
done := make(chan struct{}) // struct{}: zero bytes — pure signal, no data
go func() {
fmt.Println("goroutine working")
close(done) // broadcast: signal termination to ALL receivers
}()
<-done
fmt.Println("goroutine finished")
}
Gotcha: Using a large buffered channel to "decouple" producer and consumer — a large buffer hides backpressure and can cause OOM under sustained load.
Concept: Senders determine guarantees — the sending goroutine decides whether to wait for confirmation that the receiver has received the signal.
Why it matters: The choice between buffered and unbuffered channels is a choice about delivery guarantee — unbuffered guarantees receipt, buffered guarantees only deposit.
package main
import (
"fmt"
"time"
)
func main() {
// unbuffered: guaranteed delivery — sender blocks until receiver is ready
unbuf := make(chan string)
go func() {
time.Sleep(50 * time.Millisecond) // receiver arrives late
msg := <-unbuf
fmt.Println("received:", msg)
}()
unbuf <- "guaranteed" // blocks here until receiver is ready
fmt.Println("sender: delivery guaranteed — receiver has the message")
time.Sleep(100 * time.Millisecond)
// buffered: deposit guarantee — sender proceeds if buffer has space
buf := make(chan string, 1)
buf <- "deposited" // does NOT block — buffer has space
fmt.Println("sender: message deposited — receiver may not have read it yet")
// receiver may arrive later
time.Sleep(10 * time.Millisecond)
fmt.Println("received:", <-buf)
}
Gotcha: Using a buffered channel of 1 and assuming the receiver has processed the message after a send — "deposited" is not "received".
Concept: Signaling can be done with or without data — closing a channel broadcasts to multiple listeners.
Why it matters: close(ch) is a one-to-many signal — all goroutines blocked on <-ch are unblocked simultaneously, which ch <- struct{}{} cannot achieve for multiple receivers.
package main
import (
"fmt"
"sync"
)
func worker(id int, stop <-chan struct{}, wg *sync.WaitGroup) {
defer wg.Done()
for {
select {
case <-stop:
fmt.Printf("worker %d: stopping\n", id)
return // ALL workers unblocked simultaneously by close
default:
// do work
}
}
}
func main() {
stop := make(chan struct{})
var wg sync.WaitGroup
for i := 1; i <= 3; i++ {
wg.Add(1)
go worker(i, stop, &wg)
}
fmt.Println("workers running — closing stop channel to broadcast shutdown")
close(stop) // broadcasts to all 3 workers simultaneously
wg.Wait()
fmt.Println("all workers stopped")
}
Gotcha: Sending N individual stop signals for N goroutines — you must know N; close works for any number of receivers without knowing the count.
Concept: Channel states — open (active communication), nil (blocked communication), closed (signaling termination).
Why it matters: Each channel state has precise semantics — nil channels can be used to dynamically disable select cases, which is a powerful but underused pattern.
package main
import "fmt"
func main() {
// state 1: open — normal send/receive
ch := make(chan int, 1)
ch <- 42
fmt.Println("open, receive:", <-ch)
// state 2: closed — receive returns zero value + false; send panics
close(ch)
v, ok := <-ch
fmt.Printf("closed, receive: val=%d ok=%v\n", v, ok) // 0, false
// ch <- 1 // panic: send on closed channel
// state 3: nil — all operations block forever; disables select cases
var nilCh chan int
select {
case v := <-nilCh: // nil channel: this case is never selected
fmt.Println("never:", v)
default:
fmt.Println("nil channel: case disabled, fell through to default")
}
// nil channel trick: disable a case dynamically
a := make(chan int, 1)
b := make(chan int, 1)
a <- 1
for i := 0; i < 2; i++ {
select {
case v := <-a:
fmt.Println("from a:", v)
a = nil // disable this case — next iteration only selects b
case v := <-b:
fmt.Println("from b:", v)
}
}
}
Gotcha: Ranging over a channel that is never closed — for v := range ch blocks forever waiting for more values; always close the channel when done sending.
Concept: Guarantees and their costs — guaranteeing signal receipt introduces a synchronization point; not guaranteeing minimizes waiting time but risks signal loss.
Why it matters: Every synchronization point is a potential goroutine park — choose the minimum guarantee your correctness requirements demand.
package main
import (
"fmt"
"time"
)
// high guarantee: unbuffered — sender pays the cost of waiting for receiver
func highGuarantee(payload string) time.Duration {
ch := make(chan string) // unbuffered — receiver must be ready
start := time.Now()
go func() {
time.Sleep(20 * time.Millisecond) // simulate receiver delay
<-ch
}()
ch <- payload // blocks until receiver is ready — guaranteed delivery
return time.Since(start)
}
// low guarantee: buffered of 1 — sender never waits if buffer free; signal may be lost if receiver gone
func lowGuarantee(payload string) time.Duration {
ch := make(chan string, 1) // buffered — sender deposits and moves on
start := time.Now()
go func() {
time.Sleep(20 * time.Millisecond)
<-ch // receiver arrives later — message was waiting
}()
ch <- payload // does not block — deposits into buffer immediately
return time.Since(start)
}
func main() {
t1 := highGuarantee("event")
fmt.Printf("high guarantee latency: %v\n", t1) // ~20ms — waited for receiver
t2 := lowGuarantee("event")
fmt.Printf("low guarantee latency: %v\n", t2) // ~0ms — returned immediately
}
Gotcha: Using a buffered channel for audit events thinking "it's fine if one is dropped" — dropped audit events may violate compliance requirements; use high guarantee for critical signals.
Concept: Channels are created using make — they cannot be directly initialized with values.
Why it matters: make initializes the internal channel data structure (ring buffer, semaphores, lock) — a nil channel is unusable; make is not optional.
package main
import "fmt"
func main() {
// nil channel: zero value of channel type — unusable
var nilCh chan int
fmt.Printf("nil channel: %v\n", nilCh) // <nil>
// make: initializes the channel — ready to use
unbuffered := make(chan int) // capacity 0 — synchronous
buffered := make(chan int, 5) // capacity 5 — async up to 5
signalOnly := make(chan struct{}) // zero-size type — pure signal
fmt.Printf("unbuffered cap: %d\n", cap(unbuffered)) // 0
fmt.Printf("buffered cap: %d\n", cap(buffered)) // 5
fmt.Printf("signal cap: %d\n", cap(signalOnly)) // 0
// demonstrate capacity
for i := 0; i < 5; i++ {
buffered <- i // does not block — buffer absorbs sends
}
fmt.Printf("buffered len: %d\n", len(buffered)) // 5
}
Gotcha: Using cap(ch) to determine if a channel is unbuffered — cap == 0 means unbuffered, but a buffered channel with all slots full also won't accept sends.
Concept: Unbuffered channels ensure synchronous communication — senders are blocked until a receiver is ready.
Why it matters: Unbuffered channels are rendezvous points — both goroutines must be present simultaneously, making the handoff a synchronization guarantee.
package main
import (
"fmt"
"time"
)
// unbuffered: both goroutines must meet at the channel — like a baton handoff
func relay(value int) int {
ch := make(chan int) // unbuffered — synchronous handoff
go func() {
time.Sleep(10 * time.Millisecond) // simulate work
ch <- value * 2 // blocks until receiver is ready
fmt.Println("sender: value delivered")
}()
// receiver arrives — handoff happens here
result := <-ch
fmt.Println("receiver: value received")
return result
}
func main() {
result := relay(21)
fmt.Println("result:", result) // 42
// both prints appear together — the handoff is simultaneous
}
Gotcha: Using an unbuffered channel when the sender and receiver run on the same goroutine — both operations block each other, causing a deadlock.
Concept: Order of output from concurrent goroutines is not guaranteed.
Why it matters: Non-deterministic output order is not a bug — it's the correct behavior of concurrent systems; designing for a specific order requires explicit synchronization.
package main
import (
"fmt"
"sync"
)
func main() {
var wg sync.WaitGroup
results := make([]int, 5)
for i := 0; i < 5; i++ {
wg.Add(1)
go func(idx int) {
defer wg.Done()
results[idx] = idx * idx // write to FIXED index — no race, deterministic result
fmt.Printf("goroutine %d computed %d\n", idx, results[idx]) // ORDER not guaranteed
}(i)
}
wg.Wait()
// results slice is always correct regardless of print order
fmt.Println("results:", results) // [0 1 4 9 16] — always correct
}
Gotcha: Using append instead of index assignment in concurrent goroutines — append has undefined behavior under concurrent access; pre-allocate and write by index.
Concept: Fan Out — breaking down a larger task into smaller units for concurrent processing by multiple goroutines.
Why it matters: Fan out turns a sequential O(n×t) operation into O(t) where t is task time — all tasks run in parallel instead of one after another.
package main
import (
"fmt"
"sync"
"time"
)
type Task struct {
ID int
Data string
}
type Result struct {
TaskID int
Output string
}
// fan out: N tasks distributed to N goroutines — all run concurrently
func fanOut(tasks []Task) []Result {
results := make(chan Result, len(tasks))
var wg sync.WaitGroup
for _, task := range tasks {
wg.Add(1)
go func(t Task) {
defer wg.Done()
time.Sleep(10 * time.Millisecond) // simulate work
results <- Result{
TaskID: t.ID,
Output: "processed:" + t.Data,
}
}(task) // pass task by value — avoid loop variable capture
}
go func() { wg.Wait(); close(results) }()
var out []Result
for r := range results {
out = append(out, r)
}
return out
}
func main() {
tasks := []Task{{1, "a"}, {2, "b"}, {3, "c"}, {4, "d"}, {5, "e"}}
start := time.Now()
results := fanOut(tasks)
fmt.Printf("processed %d tasks in %v\n", len(results), time.Since(start))
// ~10ms not 50ms — all tasks ran in parallel
}
Gotcha: Fanning out without a result collection goroutine — the channel send blocks when the buffer fills; always size the buffer or drain concurrently.
Concept: Buffered channels minimize blocking — senders deposit data without waiting for an immediate receiver.
Why it matters: A buffered channel decouples the producer's pace from the consumer's pace within the buffer capacity — essential when rates differ temporarily.
package main
import (
"fmt"
"time"
)
// buffered channel absorbs bursts — producer doesn't wait for each consumer cycle
func producerConsumer(bufSize int) {
ch := make(chan int, bufSize) // buffer absorbs up to bufSize sends
// producer: fast — sends without waiting for consumer
go func() {
for i := 0; i < 10; i++ {
ch <- i
fmt.Printf("produced: %d (buffered: %d/%d)\n", i, len(ch), cap(ch))
}
close(ch)
}()
// consumer: slow — processes one at a time with delay
for v := range ch {
time.Sleep(5 * time.Millisecond)
fmt.Printf("consumed: %d\n", v)
}
}
func main() {
producerConsumer(5) // buffer of 5 lets producer run ahead
}
Gotcha: Setting buffer size to the number of tasks (unbounded fan-out pattern) — if tasks error and aren't received, the goroutines leak because they can never send.
Concept: Fan out can overload systems if too many goroutines are launched simultaneously.
Why it matters: Launching one goroutine per request with no bound creates unbounded resource consumption — memory, file descriptors, and downstream connections all exhaust.
package main
import (
"fmt"
"sync"
"time"
)
// unbounded fan out — dangerous under load
func unboundedFanOut(requests []int) {
var wg sync.WaitGroup
for _, req := range requests {
wg.Add(1)
go func(r int) { // N goroutines for N requests — no limit
defer wg.Done()
time.Sleep(10 * time.Millisecond) // each holds a DB connection
fmt.Printf("processed %d\n", r)
}(req)
}
wg.Wait()
}
// bounded fan out — safe under load
func boundedFanOut(requests []int, limit int) {
sem := make(chan struct{}, limit) // semaphore limits concurrent goroutines
var wg sync.WaitGroup
for _, req := range requests {
wg.Add(1)
sem <- struct{}{} // acquire slot — blocks when limit reached
go func(r int) {
defer wg.Done()
defer func() { <-sem }() // release slot
time.Sleep(10 * time.Millisecond)
fmt.Printf("processed %d\n", r)
}(req)
}
wg.Wait()
}
func main() {
requests := make([]int, 20)
for i := range requests { requests[i] = i }
fmt.Println("bounded fan out (limit=5):")
boundedFanOut(requests, 5) // at most 5 goroutines at a time
}
Gotcha: Using a semaphore with a channel but forgetting to release it in the deferred function — exhausted semaphore deadlocks all future goroutines permanently.
Concept: Wait for Task — goroutines initialized in a waiting state; unbuffered channels guarantee each task is delivered to a waiting goroutine.
Why it matters: Pre-positioned workers eliminate the goroutine creation latency from the hot path — workers are ready before the first task arrives.
package main
import (
"fmt"
"sync"
)
type Job struct {
ID int
Data string
}
func startWorkers(count int, jobs <-chan Job, results chan<- string, wg *sync.WaitGroup) {
for i := 0; i < count; i++ {
wg.Add(1)
go func(workerID int) {
defer wg.Done()
// worker waits here — ready before any job arrives
for job := range jobs {
result := fmt.Sprintf("worker-%d processed job-%d:%s",
workerID, job.ID, job.Data)
results <- result
}
}(i)
}
}
func main() {
jobs := make(chan Job) // unbuffered: each job handed to a waiting worker
results := make(chan string, 10)
var wg sync.WaitGroup
startWorkers(3, jobs, results, &wg) // workers started BEFORE any jobs
// send jobs — each goes directly to a waiting worker
for i := 0; i < 6; i++ {
jobs <- Job{ID: i, Data: fmt.Sprintf("data-%d", i)}
}
close(jobs)
go func() { wg.Wait(); close(results) }()
for result := range results {
fmt.Println(result)
}
}
Gotcha: Making the jobs channel buffered when using wait-for-task — buffering allows sends without a waiting worker, breaking the guaranteed handoff.
Concept: Pooling — a fixed set of goroutines efficiently handle a stream of tasks; channel closing for graceful shutdown.
Why it matters: A fixed pool bounds memory, goroutine count, and downstream connection count — essential for production stability under variable load.
package main
import (
"fmt"
"sync"
"time"
)
type Pool struct {
tasks chan func()
wg sync.WaitGroup
}
func NewPool(workers int) *Pool {
p := &Pool{tasks: make(chan func())}
for i := 0; i < workers; i++ {
p.wg.Add(1)
go func() {
defer p.wg.Done()
for task := range p.tasks { // blocks waiting for tasks
task() // execute the task
}
}()
}
return p
}
func (p *Pool) Submit(task func()) {
p.tasks <- task // send to any available worker
}
func (p *Pool) Shutdown() {
close(p.tasks) // signal all workers: no more tasks
p.wg.Wait() // wait for all in-flight tasks to complete
}
func main() {
pool := NewPool(3) // exactly 3 workers — no more goroutines ever created
for i := 0; i < 9; i++ {
taskID := i
pool.Submit(func() {
time.Sleep(10 * time.Millisecond)
fmt.Printf("task %d done\n", taskID)
})
}
pool.Shutdown()
fmt.Println("all tasks complete, pool shut down cleanly")
}
Gotcha: Submitting to a pool after calling Shutdown — sending to a closed channel panics; use a sync.Once or atomic flag to guard against post-shutdown submissions.
Concept: Fan Out Semaphore — buffered channel with capacity set to the concurrency limit acts as a semaphore.
Why it matters: A semaphore gives you fan-out's parallelism with a hard ceiling on concurrent goroutines — protecting databases and external APIs from being overwhelmed.
package main
import (
"fmt"
"sync"
"time"
)
// semaphore: buffered channel with capacity = max concurrent goroutines
func processWithSemaphore(items []string, maxConcurrent int) []string {
sem := make(chan struct{}, maxConcurrent)
results := make([]string, len(items))
var wg sync.WaitGroup
for i, item := range items {
wg.Add(1)
go func(idx int, data string) {
defer wg.Done()
sem <- struct{}{} // acquire — blocks when maxConcurrent goroutines active
defer func() { <-sem }() // release — always released, even on panic
// only maxConcurrent goroutines reach here at any time
time.Sleep(20 * time.Millisecond) // simulate DB/API call
results[idx] = "done:" + data
}(i, item)
}
wg.Wait()
return results
}
func main() {
items := []string{"a", "b", "c", "d", "e", "f", "g", "h"}
start := time.Now()
results := processWithSemaphore(items, 3) // max 3 concurrent at any moment
fmt.Printf("processed %d in %v\n", len(results), time.Since(start))
fmt.Println(results[:3])
}
Gotcha: Placing sem <- struct{}{} inside the goroutine body after go func() — if the semaphore is full, the goroutine is created (spending memory) then blocks; acquire BEFORE creating the goroutine.
Concept: Fan Out Bounded — combines fan-out with pooling to process a set number of tasks using a controlled number of goroutines.
Why it matters: When you know the total task count upfront, bounded fan-out pre-sizes the result collection and terminates cleanly — no goroutine leak possible.
package main
import (
"fmt"
"sync"
"time"
)
func fanOutBounded(items []int, maxWorkers int) []int {
n := len(items)
tasks := make(chan int, n) // pre-load all tasks
results := make([]int, n) // pre-allocated result slice indexed by task position
var wg sync.WaitGroup
// launch exactly maxWorkers goroutines — bounded
for w := 0; w < maxWorkers; w++ {
wg.Add(1)
go func() {
defer wg.Done()
for idx := range tasks {
time.Sleep(5 * time.Millisecond) // work
results[idx] = items[idx] * items[idx]
}
}()
}
// load all tasks by index — workers write results[idx] safely (different indices)
for i := range items {
tasks <- i
}
close(tasks)
wg.Wait()
return results
}
func main() {
items := []int{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}
start := time.Now()
results := fanOutBounded(items, 4) // 4 workers, 10 tasks
fmt.Printf("done in %v\n", time.Since(start))
fmt.Println(results) // [1 4 9 16 25 36 49 64 81 100]
}
Gotcha: Using results[idx] from multiple goroutines without index isolation — works only if each goroutine writes to a unique index; if indices overlap, you need a mutex.
Concept: Drop Pattern — gracefully manages system overload by discarding requests when a predetermined capacity is reached.
Why it matters: Dropping excess load cleanly is better than crashing under it — the drop pattern is the foundation of load shedding in production services.
package main
import (
"fmt"
"time"
)
type Request struct {
ID int
Body string
}
func processRequest(req Request) {
time.Sleep(20 * time.Millisecond)
fmt.Printf("processed request %d\n", req.ID)
}
// drop pattern: bounded buffer + non-blocking send
func startServer(capacity int) (submit func(Request) bool, shutdown func()) {
queue := make(chan Request, capacity)
go func() {
for req := range queue {
processRequest(req)
}
}()
submit = func(req Request) bool {
select {
case queue <- req:
return true // accepted
default:
fmt.Printf("DROPPED request %d — server at capacity\n", req.ID)
return false // dropped — non-blocking
}
}
shutdown = func() { close(queue) }
return
}
func main() {
submit, shutdown := startServer(3) // capacity of 3
for i := 0; i < 8; i++ {
accepted := submit(Request{ID: i, Body: fmt.Sprintf("data-%d", i)})
fmt.Printf("request %d: accepted=%v\n", i, accepted)
}
time.Sleep(300 * time.Millisecond)
shutdown()
}
Gotcha: Dropping requests silently without metrics — always count drops; a sudden spike in drop rate is the first signal of a capacity problem.
Concept: Select statement provides a mechanism for monitoring multiple channels and responding to events from any channel that is ready.
Why it matters: select is the multiplexer for concurrent channels — it blocks until one case is ready, enabling timeout, cancellation, and fan-in patterns.
package main
import (
"fmt"
"time"
)
func main() {
ch1 := make(chan string, 1)
ch2 := make(chan string, 1)
timeout := time.After(100 * time.Millisecond)
go func() {
time.Sleep(30 * time.Millisecond)
ch1 <- "from ch1"
}()
go func() {
time.Sleep(60 * time.Millisecond)
ch2 <- "from ch2"
}()
// select: blocks until any case is ready — first ready case executes
for i := 0; i < 3; i++ {
select {
case msg := <-ch1:
fmt.Println("received:", msg)
case msg := <-ch2:
fmt.Println("received:", msg)
case <-timeout:
fmt.Println("timed out — no more waiting")
return
}
}
}
Gotcha: Putting a default case in a select loop without a sleep — the loop burns 100% CPU polling channels with no work to do.
Concept: Cancellation Pattern — using the context package to manage timeouts and gracefully stop long-running operations.
Why it matters: Without cancellation, goroutines blocked on I/O or computation run forever — context propagates the cancellation signal through the entire call tree.
package main
import (
"context"
"fmt"
"time"
)
func longOperation(ctx context.Context, id int) (string, error) {
result := make(chan string, 1) // buffered: goroutine can send even if ctx cancelled
go func() {
time.Sleep(100 * time.Millisecond) // simulate work
result <- fmt.Sprintf("result-%d", id)
}()
select {
case r := <-result:
return r, nil // completed in time
case <-ctx.Done():
return "", fmt.Errorf("operation %d: %w", id, ctx.Err())
}
}
func main() {
// case 1: context with sufficient deadline — operation completes
ctx1, cancel1 := context.WithTimeout(context.Background(), 200*time.Millisecond)
defer cancel1()
r1, err1 := longOperation(ctx1, 1)
fmt.Printf("op1: result=%q err=%v\n", r1, err1)
// case 2: context with tight deadline — operation cancelled
ctx2, cancel2 := context.WithTimeout(context.Background(), 50*time.Millisecond)
defer cancel2()
r2, err2 := longOperation(ctx2, 2)
fmt.Printf("op2: result=%q err=%v\n", r2, err2)
}
Gotcha: Not passing context as the first parameter — Go convention requires ctx context.Context as the first parameter of every function that might block or call I/O.
Concept: Buffered channel with capacity of 1 ensures a result can be sent even if the receiver has exited due to timeout.
Why it matters: A goroutine sending to an unbuffered channel when the receiver has already cancelled will leak forever — capacity of 1 prevents this goroutine leak.
package main
import (
"context"
"fmt"
"time"
)
// unbuffered version: goroutine leaks if context is cancelled before send
func leakyVersion(ctx context.Context) (string, error) {
result := make(chan string) // unbuffered — leak risk
go func() {
time.Sleep(200 * time.Millisecond)
result <- "done" // blocks forever if receiver left due to timeout
}()
select {
case r := <-result:
return r, nil
case <-ctx.Done():
return "", ctx.Err()
// goroutine is now stuck trying to send to result with nobody receiving
}
}
// buffered of 1: goroutine sends and exits even if receiver cancelled
func safeVersion(ctx context.Context) (string, error) {
result := make(chan string, 1) // capacity 1 — goroutine always succeeds
go func() {
time.Sleep(200 * time.Millisecond)
result <- "done" // deposits into buffer and exits cleanly
}()
select {
case r := <-result:
return r, nil
case <-ctx.Done():
return "", ctx.Err()
// goroutine will send to buffer when ready — then GC'd
}
}
func main() {
ctx, cancel := context.WithTimeout(context.Background(), 50*time.Millisecond)
defer cancel()
// leakyVersion(ctx) — goroutine leaks
r, err := safeVersion(ctx)
fmt.Printf("result=%q err=%v\n", r, err)
time.Sleep(300 * time.Millisecond) // wait for goroutine to finish
fmt.Println("goroutine has exited cleanly — no leak")
}
Gotcha: Using capacity > 1 for the result channel — only 1 result will ever be sent; capacity > 1 wastes memory and signals misunderstanding of the pattern.