Once - Decorator for background tasks¶

Description¶

The @once decorator is a convenience wrapper around the same distributed locking mechanism used by lock, but tailored for “only one run at a time” semantics on a given key.

once can come in handy when you have a lot of background tasks, which usually are powered by celery, darq, taskiq, or dramatiq.

Theoretical example¶

Let's say we have some sort of a spawner task, which spawns a lot of small ones. Like, for example, the spawner gets all the orders in progress and submits a task for each one to check the status on it.

It could look like this:

from celery import shared_task

# This is scheduled to run every 5 minutes
@shared_task()
def check_in_progress_orders() -> None:
    orders = ...  # hit the database and get all orders
    [check_order.s(order_id=order.id).delay() for order in orders]


# This is being spawned from the previous one
@shared_task()
def check_order(order_id: UUID) -> None
    # check the order progress, update state, save

So in this scenario, we don't care about the results of each task, but we DO care that we are not running the second task for the same order_id twice since it could break things.

This is where @once could come in handy: it will make sure that only one task is being run at the same time for a given order_id, and all subsequent tasks on the same order_id will exit early while at least one task is running.

The full code will look like this:

from py_cachify import once
from celery import shared_task

# This is scheduled to run every 5 minutes
@shared_task()
def check_in_progress_orders() -> None:
    orders = ...  # hit the database and get all orders
    [check_order.s(order_id=order.id).delay() for order in orders]


# This is being spawn from the previous one
@shared_task()
@once(key='check_order-{order_id}', raise_on_locked=False, return_on_locked=None)  # raise_on_locked and return_on_locked can be omitted (those values are defaults)
def check_order(order_id: UUID) -> None
    # check the order progress, update state, save
    pass

This will make sure you won't run into multiple update tasks running at the same time for one order.

What's next¶

You can always check the full reference for once here.

Conslusion¶

This concludes the tutorial for py-cachify.

We have covered the basics of the package and glanced over common cases, the topics of caching and locking are pretty common yet they are always unique to the specifics of the app and tasks that the programmer wants to solve.

Py-Cachify tries to help you cover your specific cases by giving you lock- and once-based tools built on the same underlying mechanism, so you can adapt them to your needs without bloating your codebase.

For full API reference here.