Skip to content

Tutorial: Basic Workflow

Tip

The code for this page is available on GitHub.

Labtasker supports 2 sets of client APIs:

  • Python Demo: Modify your Python Demo code with a few lines of changes to support Labtasker.
  • Bash Demo: No modification to your Python Demo code is required. Simply wrap your command with labtasker loop ....

demo step by step

This is a step-by-step demo of the basic workflow described in this page.

Note: During the port configuration step, you can specify your own port.

Prerequisites

Make sure you have a deployed server.

You can follow the Deployment guide to easily deploy a server.

Make sure you have installed client tools.

Following Installation.

Make sure you have configured client.

labtasker init

It will guide you step-by-step:

See more details about creating a queue in Queue Manual#create-queue.

Step 1. Submit job arguments via Python Demo or CLI tool

demo/basic/bash_demo/submit_job.sh
#!/bin/bash

# This script submits jobs with different combinations of arg1 and arg2.

# Loop through arg1 and arg2 values
for arg1 in {0..2}; do
    for arg2 in {3..5}; do
        echo "Submitting with arg1=$arg1, arg2=$arg2"
        labtasker task submit --args '{"arg1": '$arg1', "arg2": '$arg2'}'
        # Also a simpler equivalent fashion using -- as delimiter
        # labtasker task submit -- --arg1 $arg1 --arg2 $arg2
    done
done
demo/basic/python_demo/submit_job.py
import labtasker

if __name__ == "__main__":
    for arg1 in range(3):
        for arg2 in range(3, 6):
            print(f"Submitting with arg1={arg1}, arg2={arg2}")
            labtasker.submit_task(
                args={"arg1": arg1, "arg2": arg2},
            )

See more details in Task Manual#submit-tasks.

Step 2. Run job

demo/basic/bash_demo/run_job.sh
#!/bin/bash

# This script run jobs in loop by calling python job_main.py via labtasker loop
# The argument can be automatically injected into the command line via %(...) syntax
labtasker loop -c 'python demo/basic/bash_demo/job_main.py --arg1 %(arg1) --arg2 %(arg2)'

# Also a simpler fashion:
# labtasker loop -- python demo/basic/bash_demo/job_main.py --arg1 '%(arg1)' --arg2 '%(arg2)'

where

demo/basic/bash_demo/job_main.py
"""
Suppose this is the given script to run certain job.
You would normally run this with ` python demo/bash_demo/job_main.py --arg1 1 --arg2 2`
"""

import argparse
import time


def job(arg1: int, arg2: int):
    """Simulate a long-running job"""
    time.sleep(3)  # simulate a long-running job
    return arg1 + arg2


if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("--arg1", type=int)
    parser.add_argument("--arg2", type=int)

    args = parser.parse_args()
    result = job(args.arg1, args.arg2)
    print(f"The result is {result}")
demo/basic/python_demo/run_job.py
import time

import labtasker
from labtasker import Required


def job(arg1: int, arg2: int):
    """Simulate a long-running job"""
    time.sleep(3)  # simulate a long-running job
    return arg1 + arg2


@labtasker.loop()
def main(arg1: int = Required(), arg2: int = Required()):
    # The labtasker autofills the parameter specified by Required()
    # or Annotated[Any, Required()]
    # Alternatively, you can fill in the required fields
    # in loop(required_fields=["arg1", "arg2"]
    # and access it via labtasker.task_info().args
    result = job(arg1, arg2)
    print(f"The result is {result}")


if __name__ == "__main__":
    main()

Check pending/running jobs

labtasker task ls -s pending

labtasker task ls -s running

See more details in Loop Manual#create.