Skip to content

Asynchronism (Message Queues, Task Queues, Back Pressure)

Why async?

Asynchronous workflows reduce request times for expensive operations that would otherwise run in-line. They also let you do time-consuming work in advance (e.g., periodic aggregation).

Message Queues

Message queues receive, hold, and deliver messages. Workflow:

  1. Application publishes a job to the queue and notifies the user of job status.
  2. A worker picks up the job, processes it, and signals completion.

The user isn't blocked; the job runs in the background. The client may do a small amount of work to make it look complete (e.g., a tweet appears on your timeline immediately, but takes time to reach all followers).

Queue tech options:

  • Redis — simple message broker, but messages can be lost.
  • RabbitMQ — popular, but requires the AMQP protocol and managing your own nodes.
  • Amazon SQS — hosted, but can have high latency and at-least-once delivery (possible duplicate messages).

Task Queues

Task queues receive tasks + data, run them, and deliver results. Support scheduling and background computationally-intensive jobs.

  • Celery — supports scheduling; primarily Python.

Message queue = move messages between services. Task queue = execute work and return results. A task queue often uses a message broker underneath.

Back Pressure

If queues grow too large, they can exceed memory → cache misses, disk reads, and slower performance.

Back pressure limits queue size to keep high throughput and good response times for jobs already queued:

  • When the queue fills, clients get "server busy" or HTTP 503, to retry later.
  • Clients retry with exponential backoff.

Disadvantages of asynchronism

  • Inexpensive calculations and real-time workflows may be better synchronous — queues add delay and complexity.

Key takeaways

  • Async trades immediate consistency for responsiveness and scalability.
  • Always pair unbounded queues with back pressure to avoid cascading failures.
  • Choose a queue for delivery semantics: Redis (fast, lossy), RabbitMQ (robust, self-managed), SQS (managed, at-least-once).

Common Interview Questions

Design a web crawler

Why it's asked here: crawlers are built on queues and async workers.

Key points to discuss:

  • URL frontier (a queue) with priorities; dedup via a Bloom filter.
  • Workers fetch asynchronously; respect robots.txt and per-domain rate limits.
  • A message queue (Kafka) for the URL stream; a task queue for parsing jobs.
  • Back pressure when the frontier grows faster than workers drain it.
flowchart LR
    F[(URL frontier)] --> W[Fetch workers]
    W --> P[Parser]
    P -->|new URLs| F
    P -->|dedup| B[Bloom filter]
    P --> S[(Index store)]
Design a notification system

Why it's asked here: notifications are a classic async fan-out.

Key points to discuss:

  • Producers enqueue notifications; workers deliver via push/email/SMS.
  • A message queue decouples senders from delivery (reliability, retries).
  • Back pressure and rate limiting to avoid overwhelming downstream providers.
  • At-least-once vs exactly-once delivery semantics.
flowchart LR
    P[Producer] --> Q[(Message queue)]
    Q --> W1[Push worker]
    Q --> W2[Email worker]
    Q --> W3[SMS worker]
    W1 & W2 & W3 --> U[User]
Design Mint.com (financial aggregation)

Why it's asked here: Mint aggregates account data asynchronously.

Key points to discuss:

  • Periodically pull account data via scheduled background jobs.
  • Task queues (Celery) for scraping/aggregating transactions.
  • Categorize asynchronously and cache results.
  • Notify the user when async jobs complete.
flowchart LR
    S[Scheduler] --> Q[(Task queue)]
    Q --> W[Scrape / aggregate worker]
    W --> A[(Accounts DB)]
    A --> C[Async categorizer]
    C --> N[Notify user]

Further reading