Circuit Breaker — Dev Helper Library

Intent

The Circuit Breaker detects repeated failures in an external service and, once a threshold is reached, "trips" — rejecting calls immediately without trying to reach the failing service. This prevents the wait time for a degraded service from propagating to the calling service, turning a localized failure into a cascade of errors across the whole system.

The analogy with an electrical circuit breaker is precise: when current exceeds the safe limit, the breaker "trips" and interrupts the circuit to protect the wiring. Likewise, the software Circuit Breaker "trips" and stops calls to protect the calling system. After a period, it allows testing whether the service has recovered before returning to normal operation.

Problem

In distributed systems, calls to external services fail. Network timeouts, restarts, load spikes, buggy deploys — all of these events are predictable and inevitable. The problem isn't the failure itself, but what happens to the calling system when the failure persists:

Threads stuck in timeout: every call that waits for a response from a degraded service ties up a thread or connection. If the service takes 30 seconds to time out and 100 requests per second arrive, the thread pool is exhausted within moments and the calling system stops responding — even for functionality that doesn't depend on the failing service.
Error cascade: Service A calls Service B, which calls Service C. Service C is slow. Service B starts accumulating timeouts. Service A starts accumulating timeouts waiting for Service B. The failure of one leaf service brings down the whole call chain.
Retry making the problem worse: systems with automatic retry bombard the already overloaded service with more attempts, preventing recovery and turning temporary instability into prolonged unavailability.
Invisible degradation: without explicit monitoring of dependency state, the application keeps trying calls that fail with a full timeout, responding slowly to every user instead of failing fast for the affected ones and continuing to work for the rest.

How it works

State machine: CLOSED, OPEN, and HALF-OPEN

The Circuit Breaker is essentially a three-state machine that wraps calls to an external service. The current state determines whether the call passes through, is rejected immediately, or is used as a recovery probe.

  ┌────────────┐      failures > threshold      ┌────────────┐
  │            │                                │            │
  │   CLOSED   │───────────────────────────────►│    OPEN    │
  │  (normal)  │◄───────────────────────────────│   (open)   │
  │            │ enough successes in HALF-OPEN  │            │
  └────────────┘                                └──────┬─────┘
                                                       │
                                                       │ timer expired
                                                       ▼
                                                ┌────────────┐
                                                │ HALF-OPEN  │
                                                │ (probing)  │
                                                └────────────┘

                                                on failure → back to OPEN

  CLOSED   : calls pass through normally. Failures are counted.
  OPEN     : calls are rejected immediately (fail-fast). Timer active.
  HALF-OPEN: a limited number of test calls pass through. The result
             determines the transition to CLOSED (recovered) or OPEN (still failing).

CLOSED state — normal operation

All calls pass through to the downstream service. The Circuit Breaker monitors the results: errors (exceptions, timeouts, 5xx responses) increment a failure counter. When the failure percentage within a time window exceeds the configured failure threshold, the circuit transitions to OPEN.

OPEN state — breaker tripped

No calls reach the downstream service. The Circuit Breaker rejects the request immediately — fail-fast — without waiting for a timeout. A timer starts. The response to the caller can be an explicit error or, preferably, a fallback (cached data, default value, degraded response). The timer defines how long the circuit stays open before attempting recovery.

HALF-OPEN state — recovery probing

After the timer expires, the circuit enters HALF-OPEN. A limited number of test calls are allowed through. If those calls succeed and reach the success threshold, the circuit closes (CLOSED) and normal operation resumes. If any test call fails, the circuit reopens (OPEN) and the timer restarts.

Critical parameters

Failure threshold: percentage or absolute number of failures within a time window to trigger opening. E.g., 50% errors in a sliding 60-second window with a minimum of 10 calls.
Timeout (sleep duration): how long the circuit stays OPEN before going to HALF-OPEN. Should be enough for the downstream service to recover — typically between 10 seconds and a few minutes.
Success threshold in HALF-OPEN: how many consecutive successful calls (or what percentage) are needed to close the circuit. Avoids closing prematurely based on a single success.

Fallback: graceful degradation

Fail-fast without a fallback just replaces a slow error with a fast one — the user experience improves in latency, but not in functionality. The real value of the Circuit Breaker lies in combining it with a fallback strategy:

Stale cache: return the last cached value, even if outdated.
Default value: return a generic result when personalization isn't critical.
Degraded response: return a simplified version of the functionality.
Explicit, fast error: when there's no acceptable fallback, at least fail in milliseconds instead of seconds.

Circuit Breaker vs Retry

Retry and Circuit Breaker are complementary, not substitutes. Retry attempts the operation again immediately — useful for transient network failures, but harmful when the service is overloaded (more calls make things worse). Circuit Breaker stops trying for an entire period, giving room for recovery.

The common combination is: Retry with exponential backoff (tries a few times with increasing wait between attempts) within the context of a Circuit Breaker (if the retries keep failing beyond the threshold, the circuit opens and stops trying entirely for a period).

When to use

Synchronous calls to external services: third-party APIs, downstream microservices, payment services — any network call that can fail or become slow. The Circuit Breaker protects the caller from getting stuck waiting on a degraded service.
Microservices architectures: when there are dependencies between services, a failure in one can cascade. Circuit Breaker at every integration point is a standard resilience practice.
Services with a stable failure pattern: when the service has a history of periodic failures with predictable recovery, the threshold can be calibrated to open at the right moment and close when the service recovers.

When to avoid

Local or intra-process calls: a database on the same private network with sub-1ms latency, calls between modules of the same process. The Circuit Breaker's overhead isn't justified when there's no real risk of prolonged timeout.
Services without a stable failure pattern: if the downstream service rarely fails consistently enough to reach the threshold, the Circuit Breaker never opens and adds no value — but adds complexity.
Critical operations without an acceptable fallback: if no degraded response is possible and the operation blocks the user flow, the Circuit Breaker still helps with latency, but the UX benefit is limited. In those cases, the priority should be improving the service's reliability.

Pros and cons

Pros

Prevents cascading failures: isolates the failure at the integration point, preventing it from spreading to the rest of the system.
Fail-fast: the caller gets an immediate response (error or fallback) instead of waiting for the full timeout, improving perceived latency.
Protects the downstream service: by stopping requests to an overloaded service, it gives room for recovery without being bombarded.
Graceful degradation: combined with a fallback, the system keeps working in a limited way instead of failing completely.
Visibility: circuit state (OPEN/CLOSED) is a valuable operational metric — it signals dependency problems in real time.

Cons

Added complexity: implementing and calibrating the parameters (threshold, timeout, success threshold) requires knowledge of the dependencies' behavior.
False positives: a poorly calibrated threshold opens the circuit on normal latency spikes, causing unnecessary degradation.
Distributed state: in systems with multiple instances, each instance has its own Circuit Breaker. State isn't shared — one instance may have the circuit open while another is still trying.
Fallback requires maintenance: fallback logic needs to be implemented, tested, and maintained throughout the system's life.

Common pitfalls

1. Threshold too low

A failure threshold of 10% may seem conservative, but during normal peak moments, the transient error rate can briefly exceed that value. The circuit opens, rejects legitimate calls, and the downstream service wasn't really in trouble — the problem was the calibration. Start with a more permissive threshold (50%) and adjust with real production data.

2. Timeout too short in the OPEN state

If the downstream service needs 2 minutes to recover after a spike, a 10-second sleep duration makes the circuit go to HALF-OPEN too soon. The probe call fails, the circuit reopens for another 10 seconds, and the cycle repeats — the circuit flaps — without giving enough time for recovery to happen. The timeout should be estimated based on the service's recovery history.

3. No fallback defined

A Circuit Breaker in the OPEN state without a fallback returns an instant 5xx error instead of the original slow error. Latency improved, but the user experience is the same — an error. The real gain of the pattern only materializes when there's a fallback response that lets the system keep working in a degraded but useful way.

4. Circuit Breaker without observability

If the circuit opens silently in production and there's no alert, the team finds out about the problem when users complain — and maybe long after. Circuit state (OPEN/HALF-OPEN/CLOSED), failure rate metrics, and the number of rejected calls should be exposed as monitored metrics with configured alerts.

Rule of thumb: if the Circuit Breaker isn't generating metrics and alerts, it isn't complete. The pattern serves both resilience and observability — an open circuit is a signal that something needs immediate attention.

Related architectures and patterns

Event-Driven Architecture and Circuit Breaker solve the same problem in opposite ways: the Circuit Breaker protects synchronous (request/response) calls, while event-driven architecture eliminates synchronous coupling by making communication asynchronous via messages. In systems that mix both styles, the Circuit Breaker protects the points where a synchronous call is still necessary.

Message Queues are a structural alternative: instead of calling the service directly (and needing a Circuit Breaker for protection), the message goes to the queue and the service processes it when available. The Circuit Breaker treats the symptom; the queue can eliminate the cause by decoupling producer and consumer.