Tasks and Task Workers

Output of a TaskRun in LittleHorse Dashboard

A WfSpec defines a series of steps in a process to be orchestrated by LittleHorse (in technical terms, each step is a Node). The most common type of step to execute is a TaskNode.

Concepts

A task, or unit of work executed by a computer in LittleHorse, is represented by the dual objects TaskDef and TaskRun:

• A TaskDef is a LittleHorse API Object which defines a certain type of task that can be executed by a computer. You can think of a TaskDef like a method signature, as it defines the inputs and outputs of a task.
• A TaskRun is a LittleHorse API Object representing an instance of such a task being executed by a computer as part of a WfRun.

Tasks are executed by Task Workers, which are software processes written by the users of LittleHorse and represent bite-sized chunks of business logic.

Task Workers

A Task Worker is a program written by a LittleHorse user that connects to the LittleHorse Kernel, polls for ScheduledTasks to execute, executes the Tasks, and returns the result to LittleHorse. Task Workers can be easily developed using the Task Worker SDK's in Java, GoLang, and Python according to our documentation here.

When a ThreadRun arrives at a TaskNode, the LittleHorse Kernel puts a ScheduledTask on a virtual Task Queue. The ThreadRun blocks at that NodeRun until a Task Worker pulls the ScheduledTask off the Task Queue, executes it, and reports the result (or if the configured timeout expires).

Once the TaskRun is completed, the output from the Task Worker's method invocation can be used by the workflow to mutate variables and control the flow of execution.

Task Workers open a web connection to the LittleHorse Kernel and do not need to receive connections on any ports. This has several benefits:

• Added security, since your systems do not need to accept any incoming connections.
• Built-in throttling, since the LittleHorse Kernel only dispatches a ScheduledTask to a Task Worker once the worker notifies the LittleHorse Kernel that it has capacity to perform a task.
• Performance, since the protocol is based on grpc bi-directional streaming.

tip

Task Workers can be embedded into existing systems (in Java, for example, all you need to do is add the @LHTaskMethod annotation), or they can be lightweight standalone wrappers which invoke your existing services via network calls.

Task Logic

The work performed by Task Workers can be incredibly diverse, ranging from charging a customer's credit card to fetching data from an API to deploying infrastructure (as in a devops pipeline). Our Java, Go, and Python SDK's provide utilities that allow you to easily convert a function or method into a Task Worker in five lines of code or less.

In short, a Task Worker can perform any arbitrary function in code. Moreover, a TaskRun can wait on the order of seconds, minutes, or even hours for a Task Worker process to complete, making it adaptable for any type of process that can be executed by a Task Worker (so long as the timeout that you configure on the TaskRun is long enough).

Deploying a Task Worker

A Task Worker is any program that uses the Task Worker SDK's to execute TaskRun's. LittleHorse is not opinionated about where or how the Task Worker is deployed: it can be a JVM process running on a bare metal server under a desk; it can be a docker container on ECS, or a Pod in Kubernetes.

Additionally, a single process can run multiple Task Workers for different TaskDef's at once. This is often useful if you want to take advantage of workflow-driven processes but you have no need for microservices and as such want to avoid managing multiple deployable artifacts.

info

In real-world scenarios, Task Workers can be deployed in a variety of ways. We have seen successful implementations where a single process manages dozens of task workers for different TaskDefs, and we have seen other cases wherein a single Task Worker is managed by a dedicated team and deployed on its own process (eg. Kubernetes pod or ECS container).

In Practice

tip

If you want to follow along, all of the code snippets in this section can be copy-n-pasted and executed as standalone files. However, to do this, you'll need access to a LittleHorse Kernel and lhctl. You can do that via the instructions in our installation docs or by running the following commands:

• brew install littlehorse-enterprises/lh/lhctl
• docker run --rm -d --name littlehorse --pull always -p 2023:2023 -p 8080:8080 ghcr.io/littlehorse-enterprises/littlehorse/lh-standalone:latest

To deploy a Task Worker, you have to do two things:

1. Register the TaskDef.
2. Create and start the Task Worker, which is the long-lived process that polls the LittleHorse Kernel for tasks, executes them, and reports the result.

Our SDK's in Java, Go, Python, and DotNet all provide an LHTaskWorker class/struct/object/utility which makes it easier to do this.

tip

If you change the method signature of your TaskDef, the rpc PutTaskDef (called by LHTaskWorker#registerTaskDef()) will fail with ALREADY_EXISTS. This is because changing the API of a TaskDef will break existing WfSpecs, which is dangerous in production.

If you want to iterate in local development, you can delete the TaskDef and start again:

lhctl delete taskDef <taskDefName>

Define your Task Method

A Task in LittleHorse is just any other normal function or method. In order to create an LHTaskWorker, you must pass a pointer to that function or method into the LittleHorse SDK. In this example, we will create a TaskDef named greet that takes in a String (representing a "name") and prints it out to stdout.

In real life, your task method would do something more useful such as query a database, make an API call to an external system, or perform some calculations.

Java

In Java, you can specify a Task Method using the @LHTaskMethod annotation. That is the only line of LittleHorse-specific code in this file; everything else is Plain Old Java!

package io.littlehorse.quickstart;

import io.littlehorse.sdk.worker.LHTaskMethod;

public class Greeter {

    @LHTaskMethod("greet")
    public String greeting(String name) {
        String result = "Hello there, " + name + "!";
        System.out.println(result);
        return result;
    }
}

Go

TODO

Python

TODO

DotNet

TODO

Register the TaskDef and Start Polling

Now that we've written our Task Method, we need to create a Task Worker that connects to the LittleHorse Kernel and executes our Task Method when appropriate. We can do that with the LHTaskWorker utility.

Java

package io.littlehorse.quickstart;

import io.littlehorse.sdk.common.config.LHConfig;
import io.littlehorse.sdk.worker.LHTaskMethod;
import io.littlehorse.sdk.worker.LHTaskWorker;

// The `Greeter` class is implicitly imported because it's in the same package.

public class Main {

    public static void main(String[] args) {
        LHConfig config = new LHConfig();

        Greeter greeter = new Greeter();

        // Create a Task Worker
        LHTaskWorker worker = new LHTaskWorker(greeter, "greet", config);
        Runtime.getRuntime().addShutdownHook(new Thread(worker::close));

        // Register the TaskDef
        worker.registerTaskDef();

        // Start the Worker
        worker.start();
    }
}

Go

TODO

Python

TODO

DotNet

TODO