Prompt Template

This is a simple prompt to tell AI how to turn your serial task script into Labtasker submit and run script.

    # Labtasker Task Decomposition Guide

    ## Overview
    Labtasker is a CLI tool that distributes tasks across computing resources. It uses two primary commands:
    - `labtasker task submit` - Queues task arguments for later execution
    - `labtasker loop` - Continuously fetches and executes queued tasks

    This guide will help you decompose a serial task script into Labtasker submit and run scripts.

    ## Task Decomposition Process
    For any serial task script, you need to create:
    1. A **submit script** that queues all task variations
    2. A **run script** that executes those tasks when resources are available

    ### Pattern Recognition
    - Identify loops and parameter variations in the original script
    - Move parameter generation to the submit script
    - Create a template for task execution in the run script

    ## Example 1: Simple Parameter Grid

    ### Original Script
    ```bash
    #!/bin/bash
    # Simple parameter grid search
    # tags: experimental

    for arg1 in {0..2}; do
        for arg2 in {3..5}; do
            python main.py --arg1 $arg1 --arg2 $arg2
        done
    done
    ```

    ### Submit Script
    ```bash
    #!/bin/bash
    submit_date=$(date +%s)

    for arg1 in {0..2}; do
        for arg2 in {3..5}; do
            labtasker task submit --name simple_grid_search --metadata "{'tags': ['experimental', '$submit_date']}" -- --arg1=$arg1 --arg2=$arg2
        done
    done
    ```

    ### Run Script
    ```bash
    #!/bin/bash
    labtasker loop -- python main.py --arg1 '%(arg1)' --arg2 '%(arg2)'
    ```

    ## Example 2: Complex Environment and Parameters

    ### Original Script
    ```bash
    #!/bin/bash

    export CUDA_HOME=/usr/local/cuda-12.1

    BASE_LOG_DIR=/path/to/logs

    DATASETS=("imagenet" "cifar10" "mnist" "custom_dataset")
    DATASET_DESCRIPTIONS=("A person's \"image\" net" "A person's cifar 10" "A person's mnist" "A person's custom dataset")

    MODELS=("resnet50" "vit" "transformer" "alexnet")

    for idx in "${!DATASETS[@]}"; do
      for model_idx in "${!MODELS[@]}"; do

        DATASET_DESCRIPTION=${DATASET_DESCRIPTIONS[$idx]}
        DATASET=${DATASETS[$idx]}
        MODEL=${MODELS[$model_idx]}
        LOG_DIR="$BASE_LOG_DIR/$(echo "$DATASET" | tr '[:upper:]' '[:lower:]')/$MODEL"

        echo "Processing Dataset: $(echo "$DATASET" | tr '[:lower:]' '[:upper:]')"
        echo "Dataset Description: ${DATASET_DESCRIPTION}"
        echo "Model (short): ${MODEL:0:3}"
        echo "Log Directory: ${LOG_DIR}"

        # Execute training command
        python train.py --dataset "$DATASET" \
                        --dataset-description "$DATASET_DESCRIPTION" \
                        --model "$MODEL" \
                        --cuda-home "$CUDA_HOME" \
                        --log-dir "$LOG_DIR"
      done
    done

    echo "All tasks completed successfully."
    ```

    ### Submit Script
    ```bash
    #!/bin/bash

    export CUDA_HOME=/usr/local/cuda-12.1
    BASE_LOG_DIR=/path/to/logs

    DATASETS=("imagenet" "cifar10" "mnist" "custom_dataset")
    DATASET_DESCRIPTIONS=("A person's \"image\" net" "A person's cifar 10" "A person's mnist" "A person's custom dataset")
    MODELS=("resnet50" "vit" "transformer" "alexnet")

    for idx in "${!DATASETS[@]}"; do
      for model_idx in "${!MODELS[@]}"; do
        DATASET_DESCRIPTION=${DATASET_DESCRIPTIONS[$idx]}
        DATASET=${DATASETS[$idx]}
        MODEL=${MODELS[$model_idx]}
        LOG_DIR="$BASE_LOG_DIR/$(echo "$DATASET" | tr '[:upper:]' '[:lower:]')/$MODEL"

        echo "Submitting task for dataset: $DATASET, model: $MODEL"

        labtasker task submit -- \
                        --dataset="$DATASET" \
                        --dataset-description="$DATASET_DESCRIPTION" \
                        --model="$MODEL" \
                        --cuda-home="$CUDA_HOME" \
                        --log-dir="$LOG_DIR"
      done
    done

    echo "All tasks submitted successfully."
    ```

    ### Run Script
    ```bash
    #!/bin/bash
    export CUDA_HOME=/usr/local/cuda-12.1

    labtasker loop -- \
                python train.py --dataset '%(dataset)' \
                --dataset-description '%(dataset_description)' \
                --model '%(model)' \
                --cuda-home '%(cuda_home)' \
                --log-dir '%(log_dir)'
    ```

    ## Example 3: Complex Scripts (Using `labtasker loop --script-path` option)

    ### Original Script
    ```bash
    #!/bin/bash

    export CUDA_HOME=/usr/local/cuda-12.1

    for dataset in imagenet cifar10 mnist; do
      for model in resnet50 vit transformer; do
        LOG_DIR=/path/to/logs/$dataset/$model
        python train.py --dataset $dataset \
          --model $model \
          --cuda-home $CUDA_HOME \
          --log-dir $LOG_DIR
      done
    done

    echo "done"
    ```

    ### Submit Script

    ```bash
    #!/bin/bash
    export CUDA_HOME=/usr/local/cuda-12.1

    for dataset in imagenet cifar10 mnist; do
      for model in resnet50 vit transformer; do
        LOG_DIR=/path/to/logs/$dataset/$model

        labtasker task submit -- --CUDA_HOME="$CUDA_HOME" --LOG_DIR="$LOG_DIR" --dataset="$dataset" --model="$model"

      done
    done
    ```

    ### Run Script
    ```bash
    #!/bin/bash

    export CUDA_HOME=/usr/local/cuda-12.1

    LABTASKER_TASK_SCRIPT=$(mktemp)

    cat <<'LABTASKER_LOOP_EOF' > "$LABTASKER_TASK_SCRIPT"
    CUDA_HOME=%(CUDA_HOME)
    LOG_DIR=%(LOG_DIR)
    dataset=%(dataset)
    model=%(model)
        python train.py --dataset $dataset \
          --model $model \
          --cuda-home $CUDA_HOME \
          --log-dir $LOG_DIR
    LABTASKER_LOOP_EOF

    labtasker loop --executable /bin/bash --script-path $LABTASKER_TASK_SCRIPT

    echo "done"
    ```

    ## Key Points to Remember

    - All variables passed to `labtasker task submit` become available as `%(variable_name)` in the run script. All submitted variables MUST be used in the run script. All variables used in the run script MUST be submitted in advance.
    - DO NOT APPLY NORMALIZATION ON YOUR OWN. E.g. this is prohibited: original: `--value_a=1`; run: `--value_a 1`. Your script should respect the original format.
    - The submitted arguments WILL BE normalized. E.g. `--value-a=1` will be normalized to `--value-a=%(value_a)`
    - Complex values with spaces or special characters should be properly quoted
    - Environment variables and preprocessing can be included in the submit script
    - Environment variables should also be preserved in the run script in case they're needed
    - The run script acts as a template that's filled with task-specific values at runtime
    - For special characters in submit script, e.g. negative numbers `--value=-1` or empty strings `--value=""` or `--value=" "`, you should use `--value=<value>` instead of just `--value <value>`
    - Argument interpolation using %(variable_name) syntax is only valid under `labtasker loop` command. If bash script needs some %(variable_name) syntax, use the `--script-path` option to specify a path to a script that contains the interpolation syntax.
    - Properly handle env variables: wrong: `labtasker loop -- CUDA_VISIBLE_DEVICES=0 python main.py ...`; right: `CUDA_VISIBLE_DEVICES=0 labtasker loop -- python main.py ...`.
    - If user provided with task relevant info such as name or tags or description, consider using `--name` or `--metadata` to record more information. Remember that `--metadata` must be a JSON string that can be converted to a Python dict using literal_eval.

    Now, I will provide an original script that needs to be decomposed for Labtasker. You need to decompose it as per the above steps.