On this page

What asyncio.gather does when one task fails

Loose tasks and lost tasks

What an asyncio TaskGroup does

Stopping a TaskGroup early with cancel()

ExceptionGroup and except

Timeouts: one deadline instead of a limit per call

A TaskGroup sets no concurrency limit

Migrating from gather to TaskGroup: a checklist

TaskGroup vs gather compared

Frequently asked questions

Related reading

Asyncio TaskGroup vs gather: Structured Concurrency in Python

Image
Arslan Ahmad
Asyncio TaskGroup vs gather, measured: what gather leaves running after a failure, how TaskGroup cancels and waits, except*, deadlines, and a migration checklist.
Image

What asyncio.gather does when one task fails

Loose tasks and lost tasks

What an asyncio TaskGroup does

Stopping a TaskGroup early with cancel()

ExceptionGroup and except

Timeouts: one deadline instead of a limit per call

A TaskGroup sets no concurrency limit

Migrating from gather to TaskGroup: a checklist

TaskGroup vs gather compared

Frequently asked questions

Related reading

An asyncio TaskGroup is an async with block that owns every task started through it. When one task fails, the group cancels the others, waits until they have stopped, and then raises all the errors together. The older call, asyncio.gather, does none of that. When one of its tasks fails, it passes the first error to the caller at once and leaves the other tasks running, with nobody waiting for them.

The practical rule is short. Use asyncio.TaskGroup (Python 3.11 and later) for any group of tasks that should succeed or fail as a unit. Keep gather with return_exceptions=True for the case where you want every result and every error as a list.

This article shows the difference with timestamped runs rather than assertions. It then covers three things that surprise engineers after they switch. Errors arrive as an ExceptionGroup, timeouts work best as one deadline, and a task group sets no limit on how many tasks run at once.

Unless a section says otherwise, every number comes from Python 3.15.2 on a Linux cloud runner with 2 virtual CPUs (one physical core). The examples wait on a small local test site, so the CPU count does not change the timings.

What asyncio.gather does when one task fails

asyncio.gather(*aws) runs several awaitables at the same time and returns their results as a list, in the order they were passed. An awaitable is anything that can follow await: a coroutine, a task or a future. Each coroutine is wrapped in a task, so all of them start at once.

The test crawl fetches five pages from a local site. Page 2 answers with status 500 after 100 ms, page 4 fails after 500 ms, and pages 0, 1 and 3 succeed after 200, 300 and 700 ms. Each successful page is written into an index.

async def crawl(): await asyncio.gather(*(fetch_and_store(n) for n in PATHS))

Output, where each line starts with the seconds since the crawl started:

0.10 s crawl failed: /broken/2 answered 500 0.10 s pages in the index: [] 0.20 s stored page 0 0.30 s stored page 1 0.70 s stored page 3 1.10 s one second later, pages in the index: [0, 1, 3]

The caller received the error at 0.10 seconds and treated the crawl as failed. The other tasks kept running, and three of them wrote into the index after their caller had given up. A task that keeps running after the code that started it has stopped waiting for it is called an orphaned task. In a service, those late writes change the state of a request that has already returned an error. A retry can even run while the orphans of the first attempt are still writing.

Page 4 also failed, at 0.50 seconds, and that error appears nowhere. The call had already finished with the first exception, so it discarded the second one without a warning.

The option return_exceptions=True makes gather wait for everything and put each error into the result list in place of a value. On the same crawl, the returned list held three str results and two FetchError objects. That stops the orphans, but now every caller must check the type of each item. Forgetting one check treats an error object as a page of text.

Loose tasks and lost tasks

The other common pattern is asyncio.create_task() with no owner. A program fetches four pages in a loop and starts a background task to store each one, without awaiting those tasks:

0.10 s fetched page 0, store started 0.20 s fetched page 1, store started 0.31 s fetched page 2, store started 0.41 s fetched page 3, store started 0.41 s main returns, pages in the index: [0, 1] 0.41 s store of page 2 cancelled 0.41 s store of page 3 cancelled after asyncio.run, pages in the index: [0, 1]

When main() returned, asyncio.run() canceled every task that was still running. Two pages were never stored, and no error was raised, because the cancellation is part of normal shutdown.

Two warnings in the log point to a related mistake. The event loop keeps only a weak reference to a task. A task that no code holds can therefore be removed by the garbage collector while it waits. In a second test, an indexer task that nobody stored was destroyed by a gc.collect() call, and the log printed Task was destroyed but it is pending!. A task that fails and is never awaited produces a different line, Task exception was never retrieved. In that test the task failed at 0.05 seconds, yet the line appeared only at 1.00 seconds, when the last reference to the task was dropped.

Both warnings mean the same thing: some task had no owner that waited for it and read its result.

Timeline of the same five-page crawl with asyncio.gather and with asyncio.TaskGroup

What an asyncio TaskGroup does

A task group is an asynchronous context manager (an object used with async with that can await while the block is entered and left). It holds a reference to every task created with tg.create_task(), and its block does not end until all of those tasks have finished. Because the group holds the tasks, none of them can be lost to the garbage collector.

async def crawl(paths): async with asyncio.TaskGroup() as tg: for n, path in paths.items(): tg.create_task(fetch_and_store(n, path)) log("all tasks created, the body ends here") log("the async with block has ended")

With three pages that all succeed, the body ends at 0.00 seconds, and the line after the block runs at 0.30 seconds, together with the slowest page. That wait is the guarantee of structured concurrency. When the block ends, nothing it started is still running, so code after it can trust that the work is complete.

On the five-page crawl with two failures, the same block gives this output:

0.00 s all tasks created, the body ends here 0.10 s page 4 cancelled 0.10 s page 3 cancelled 0.10 s page 0 cancelled 0.10 s page 1 cancelled 0.10 s crawl failed: ExceptionGroup('unhandled errors in a TaskGroup', [FetchError('/broken/2 answered 500')]) 1.10 s one second later, pages in the index: []

At 0.10 seconds, page 2 raised, and the group responded in three steps:

  1. It canceled every other task that had not finished, and the body too if it was still running.
  2. It waited until those tasks had actually stopped.
  3. It raised the errors of the failed tasks together, as one ExceptionGroup.

Nothing was orphaned, and the index stayed empty. The caller learned about the failure at 0.10 seconds instead of waiting 0.70 seconds for a crawl that had already failed. Page 4 is missing from the group because it was canceled before its own error arrived.

You can see the same behavior with no test site. This script uses only the standard library and runs on Python 3.11 or later:

import asyncio import time INDEX = [] START = time.perf_counter() def log(message): print(f"{time.perf_counter() - START:4.2f} s {message}") async def fetch_and_store(n, delay, fails=False): await asyncio.sleep(delay) # stands in for a network call if fails: raise ConnectionError(f"page {n} answered 500") INDEX.append(n) log(f"stored page {n}") PLAN = [(0, 0.2), (1, 0.3), (2, 0.1, True), (3, 0.7)] async def with_gather(): await asyncio.gather(*(fetch_and_store(*p) for p in PLAN)) async def with_taskgroup(): async with asyncio.TaskGroup() as tg: for p in PLAN: tg.create_task(fetch_and_store(*p)) async def main(): global START for crawl in (with_gather, with_taskgroup): INDEX.clear() START = time.perf_counter() print(f"--- {crawl.__name__}") try: await crawl() except (ConnectionError, ExceptionGroup) as error: log(f"crawl failed: {error!r}") await asyncio.sleep(1) log(f"one second later, index: {INDEX}") asyncio.run(main())

Output on Python 3.15.2, Windows, Ryzen 7 5800X (the Windows timer adds about 10 ms to each line):

--- with_gather 0.11 s crawl failed: ConnectionError('page 2 answered 500') 0.21 s stored page 0 0.30 s stored page 1 0.71 s stored page 3 1.12 s one second later, index: [0, 1, 3] --- with_taskgroup 0.11 s crawl failed: ExceptionGroup('unhandled errors in a TaskGroup', [ConnectionError('page 2 answered 500')]) 1.11 s one second later, index: []

Want to see every asyncio failure mode measured like this? Grokking Concurrency in Python builds one crawler through threads, processes, asyncio, free-threaded Python and subinterpreters, with timestamped output for each design.

Stopping a TaskGroup early with cancel()

A crawler often needs to stop once it has enough results. In Python 3.15 and later, tg.cancel() cancels every unfinished task in the group, and the block then ends without raising an error. The test starts eight fetches with different delays and calls tg.cancel() from inside a task after the third page is stored:

0.10 s stored page 3 0.15 s stored page 1 0.25 s stored page 5 0.25 s 3 pages stored, cancelling the group 0.25 s the block ended without an error, 5 fetches cancelled 0.25 s pages in the index: [1, 3, 5]

The group stopped at 0.25 seconds, five fetches were canceled, and the code after the block ran normally. On Python 3.11 to 3.14, the documented approach is different. A task raises a private exception, and the caller catches the ExceptionGroup outside the block and ignores that one exception.

ExceptionGroup and except*

An ExceptionGroup is an exception that carries a list of other exceptions, so that several failures can be raised and caught as one. A task group always raises this form, even when only one task failed. That is the first change a migration must handle: an except FetchError around the block no longer matches, and neither does code that expected a single exception type.

The test crawl checks four paths, and two fail for different reasons: one with ValueError and one with PermissionError. A plain except ValueError misses the group entirely, and the program exits with ExceptionGroup: unhandled errors in a TaskGroup (2 sub-exceptions) and both tracebacks.

The except* syntax (Python 3.11 and later) matches the exceptions inside a group by type, and each matching clause runs once with a smaller group:

try: await crawl(frontier) except* ValueError as group: for error in group.exceptions: print(f"bad path skipped: {error}") except* PermissionError as group: for error in group.exceptions: print(f"blocked path skipped: {error}")
bad path skipped: 'page/2' is not a path blocked path skipped: /private/3 is closed to crawlers

If only the ValueError clause is present, it handles its part, and the rest of the group is raised again. The caller then receives ExceptionGroup('unhandled errors in a TaskGroup', [PermissionError('/private/3 is closed to crawlers')]), so errors that no clause handles still reach the caller. A bare except Exception does catch the group, because ExceptionGroup is a subclass of Exception. It then treats a bad URL and a refused connection as the same failure.

Timeouts: one deadline instead of a limit per call

asyncio.timeout(seconds) (Python 3.11 and later) cancels the code inside its async with block when the time runs out and turns that cancellation into a TimeoutError when the block exits. It combines directly with a task group: the timeout cancels the block, and the group cancels its tasks.

The common mistake is to give every call its own limit and assume the total is bounded. The test crawl has a budget of 2.0 seconds for four pages, three of which take 0.9 seconds each:

2.96 s limit of 1.0 s per call: all 4 pages, budget was 2.0 s page 0: 2.00 s left page 1: 1.10 s left page 2: 0.20 s left 2.00 s one deadline for the crawl: TimeoutError

With a limit of 1.0 second per call, no call ever exceeded its limit, and the crawl took 2.96 seconds against a budget of 2.0. With one deadline computed once and passed down through asyncio.timeout_at(deadline), each call received the time that was actually left, and the crawl stopped at exactly 2.00 seconds.

Nested limits are reported separately. An outer limit of 1 second around an inner limit of 3 seconds raised TimeoutError at 1.00 seconds. The two context managers then reported outer expired: True, inner expired: False, so the code can tell which limit fired.

A TaskGroup sets no concurrency limit

A task group decides when tasks stop, but it does not decide how many run at once. Starting one task per URL sends every request immediately. The test site serves at most 32 requests at a time and refuses the rest:

no limit 32 pages, 168 refused, 0.26 s, site: peak 32 served 32 rejected 168 semaphore 16 200 pages, 0 refused, 3.27 s, site: peak 16 served 200 rejected 0

A semaphore is a counter that lets a fixed number of tasks enter a block and makes the others wait. With asyncio.Semaphore(16) created once and shared by every task, all 200 pages arrived in 3.27 seconds, close to the minimum of 13 rounds of 250 ms. Create the semaphore outside the task function, since a semaphore created inside it gives each task a private limit and no limit at all in total.

limit = asyncio.Semaphore(16) async def fetch_limited(n): async with limit: return await fetch_or_none(n) async def crawl_all(): async with asyncio.TaskGroup() as tg: tasks = [tg.create_task(fetch_limited(n)) for n in range(200)] return [task.result() for task in tasks]

Migrating from gather to TaskGroup: a checklist

Most migrations are mechanical, but each item below changes behavior that existing code may depend on.

  • Replace the call. await asyncio.gather(*coros) becomes an async with asyncio.TaskGroup() as tg: block with one tg.create_task() per coroutine. Read results with task.result() after the block ends.
  • Rewrite the error handling. except SomeError around the old call must become except* SomeError, or except ExceptionGroup with a check of group.exceptions.
  • Decide about partial results. Some old code used return_exceptions=True to keep the pages that succeeded. For that code, catch the error inside each task and return a marker value, or keep gather for that call.
  • Do not swallow cancellation. A task that catches asyncio.CancelledError must raise it again. Otherwise the task keeps running after the group asked it to stop, and the block waits until it finishes.
  • Add a limit. A semaphore or a bounded queue is still needed, since the group starts every task at once.
  • Give the whole operation one deadline. Put asyncio.timeout() around the block, or pass a deadline down, rather than a limit on each call.
  • Name the tasks. tg.create_task(coro, name="fetch page 7") makes the burst of cancellations after a failure much easier to read in logs.

TaskGroup vs gather compared

QuestiongatherTaskGroup
One task failsRaises at once, the rest keep runningCancels the rest, waits, then raises
Several tasks failOnly the first error reaches the callerEvery error raised before the stop is in the group
Caller is canceledCancels the childrenCancels the children
Add tasks while runningNot possibletg.create_task() from the body or any task
Stop early on purposeCancel each task by handtg.cancel() (Python 3.15 and later)
Concurrency limitNoneNone
Minimum versionEvery version with asyncioPython 3.11

Frequently asked questions

Is asyncio.gather deprecated? No. gather is still part of asyncio and is the right call when you want every result and every error as one list, with return_exceptions=True. For tasks that should succeed or fail together, the asyncio documentation recommends TaskGroup because it gives stronger safety guarantees.

Why does except ValueError not catch an error from a TaskGroup? A task group always wraps errors in an ExceptionGroup, even when only one task failed. A plain except ValueError does not match the group. Use except* ValueError, which matches the ValueError inside the group.

What does "Task was destroyed but it is pending!" mean? It means a task was removed by the garbage collector before it finished, because no code held a reference to it. The event loop keeps only weak references to tasks. Keep each task in a collection, or create it through a task group, which holds it for you.

Does a TaskGroup limit how many tasks run at the same time? No. It starts every task as soon as you create it. To limit concurrency, share one asyncio.Semaphore among the tasks, or feed them from a bounded asyncio.Queue.

How do I stop a TaskGroup early? On Python 3.15 and later, call tg.cancel(), and the block ends without an error. On Python 3.11 to 3.14, raise a private exception from a task, then catch the ExceptionGroup outside the block and ignore that exception.

Ready to write asyncio code that fails cleanly? Grokking Concurrency in Python covers task groups, cancellation, deadlines, backpressure and testing, with every claim backed by a measured run.

Python
Coding Interview

What our users say

Brandon Lyons

The famous "grokking the system design interview course" on http://designgurus.io is amazing. I used this for my MSFT interviews and I was told I nailed it.

Nathan Thomas

My newest course recommendation for all of you is to check out Grokking the System Design Interview on designgurus.io. I'm working through it this month, and I'd highly recommend it.

Arijeet

Just completed the “Grokking the system design interview”. It's amazing and super informative. Have come across very few courses that are as good as this!

More From Designgurus
Annual Subscription
Get instant access to all current and upcoming courses for one year.

Access to 50+ courses

New content added monthly

Certificate of completion

$31.08

/month

Billed Annually

Recommended Course
Grokking Dynamic Programming Patterns for Coding Interviews

Grokking Dynamic Programming Patterns for Coding Interviews

13,417+ students

4.4

Grokking Dynamic Programming Patterns for Coding Interviews in Python, Java, JavaScript, and C++. A complete guide to grokking dynamic programming.

View Course
Join our Newsletter

Get the latest system design articles and interview tips delivered to your inbox.

Read More

Landing a Tech Job in 2025: A Roadmap from Application to Offer

Arslan Ahmad

Arslan Ahmad

Wipro Job Levels Explained: Bands A to E and What Each One Pays

Arslan Ahmad

Arslan Ahmad

Future-Proof Your Career With 5 Must-Learn Programming Languages for 2025

Arslan Ahmad

Arslan Ahmad

OpenAI Interview Guide: Process, Questions, and How to Prepare

Arslan Ahmad

Arslan Ahmad

Design Gurus logo
One-Stop Portal For Tech Interviews.
About Us
Our Team
Careers
Partners
Contact Us
Become Affiliate
Copyright © 2026 Design Gurus, LLC. All rights reserved.