Skip to main content

Variables

A WfSpec in LittleHorse gives developers a distributed version of all of the primitives you use when writing code that runs on a single machine. This includes variables.

Concepts​

In LittleHorse, a Variable is defined by a WfSpec and is created when a WfRun is run. A Variable can be used inside a WfRun and it can also be fetched as a LittleHorse API Object.

Variables allow you to:

  • Pass different inputs into different WfRun instances (via the rpc RunWf request).
  • Pass data between different nodes in the WfSpec (eg. pass data between tasks).
  • Search for specific WfRun instances based on variable values.

Variables are defined in the WfSpec. Each variable definition has a name and a type. When you run your WfSpec and create a WfRun, LittleHorse creates a Variable object for each variable definition in the WfSpec. These Variables are then used by the WfRun and can also be inspected and fetched from the LittleHorse API.

Declaring Variables​

Just like in programming languages, LittleHorse supports different types of variables that allow you to store various kinds of data and handle a wide range of tasks. Before you can use a Variable, you need to declare it. The following shows how to define a variable of each type in your WfSpec.

WfRunVariable itemId = wf.declareStr("item-id");
WfRunVariable quantity = wf.declareInt("quantity");
WfRunVariable price = wf.declareDouble("price");
WfRunVariable isPaid = wf.declareBool("is-paid");
WfRunVariable order = wf.declareJsonObj("order");
WfRunVariable items = wf.declareJsonArr("items");

Modifying Variables​

When you execute a Task in LittleHorse, the SDK returns a NodeOutput object. This object contains the output of the task and can be used to modify the value of a variable in the WfSpec. You can use this handle to modify the value of a variable in the WfSpec by using the WfRunVariable's assign() method.

// The .required() makes it a required input variable
WfRunVariable itemId = wf.declareStr("item-id").required();

WfRunVariable price = wf.declareDouble("price");

NodeOutput priceOutput = wf.execute("get-price", itemId);
price.assign(itemId);
Note

You can form complex expressions with LittleHorse Variables. Check out the .add(), .jsonPath(), and other methods on the WfRunVariable 😃.

JSON Variables​

LittleHorse's SDK has advanced support for JSON. The .jsonPath() method in the WfRunVariable allows you to access and modify sub-fields of a JSON_ARR or JSON_OBJ variable. You can also use .jsonPath() on a NodeOutput (i.e. the output of a Task that returns a JSON Object) to access fields within the output of a Task.

WfRunVariable itemId = wf.declareStr("item-id").required();
WfRunVariable item = wf.declareJsonObj("item");

NodeOutput itemOutput = wf.execute("fetch-item", itemId);
item.assign(itemOutput);

// Accessing a sub-field of a JSON Object
wf.execute("charge-credit-card", item.jsonPath("$.price"));

// Mutate a subfield of a JSON Object
item.jsonPath("$.isSold").assign(true);
warning

JSON variables in LittleHorse do not have type information or schemas. This can make debugging with them quite difficult. In order to address this limitation, we are working on adding a new type of variable to LittleHorse, called a Struct. For more information, follow this GitHub Issue.

Variable Modifiers​

Just like Java (private, public, final, static, etc), variables in LittleHorse can have modifiers. Modifiers in LittleHorse include:

  • .required(), which makes a Variable a required input to the workflow (specifically, the ThreadSpec).
  • .searchable(), which allows you to search for instances of the variable by their value.
  • .public(), which makes the Variable considered as part of the WfSpec's public API. This is important for upcoming (very advanced) LittleHorse features such as the Output Topic.

In Practice​

In order to use Variables in a workflow, you need to:

  1. Register a WfSpec that defines some Variables.
  2. Run a WfRun, passing in those variables (if they are required as input variables).

After this, you can observe the result on the LittleHorse Dashboard.

Registering the WfSpec​

In this example, we will build a WfSpec that models a simple user notification workflow. The WfSpec requires as input a user-id and a message. It then loads information about the user (this is a TaskRun that most likely hits a user microservice) and uses that information to formulate a message to send via the email TaskDef.

Background: the Tasks​

We'll create two TaskDefs for this example. The first one will take in a String userId and return a User object (which will be serialized as a JSON_OBJ variable value in LittleHorse). The second takes in an email address and a message and sends an email.

package io.littlehorse.quickstart;

import io.littlehorse.sdk.common.config.LHConfig;
import io.littlehorse.sdk.common.exception.LHTaskException;
import io.littlehorse.sdk.worker.LHTaskMethod;
import io.littlehorse.sdk.worker.LHTaskWorker;

class User {
    public String email;
    public String title;
    public int age;

    public User() {}

    public User(String email, String title, int age) {
        this.email = email;
        this.title = title;
        this.age = age;
    }
}

class MyTasks {

    // This Task Method returns a POJO, which is automatically serialized to a JSON_OBJ variable
    // value in LittleHorse.
    @LHTaskMethod("fetch-user")
    public User fetchUser(String userId) {
        if (userId.equals("obiwan")) {
            return new User("obiwan@jedi.temple", "Master Kenobi", 37);
        } else if (userId.equals("anakin")) {
            return new User("anakin@jedi.temple", "Padawan Skywalker (not Master)", 22);
        } else {
            throw new LHTaskException("user-not-found", "Could not find specified user");
        }
    }

    @LHTaskMethod("send-email")
    public String sendEmail(String toAddress, String message) {
        String result = "sent email " + message + " to address " + toAddress;
        System.out.println(result);
        return result;
    }
}

public class Main {

    public static void main(String[] args) throws Exception {
        LHConfig config = new LHConfig();

        MyTasks taskFuncs = new MyTasks();
        LHTaskWorker emailer = new LHTaskWorker(taskFuncs, "send-email", config);
        LHTaskWorker userService = new LHTaskWorker(taskFuncs, "fetch-user", config);
        emailer.registerTaskDef();
        userService.registerTaskDef();

        Runtime.getRuntime().addShutdownHook(new Thread(emailer::close));
        Runtime.getRuntime().addShutdownHook(new Thread(userService::close));

        emailer.start();
        userService.start();
    }
}

At this point, you should have two new TaskDefs available in your LittleHorse cluster.

The WfSpec​

We'll create the following variables in our WfSpec:

  • user-id, of type STR, which is required as input.
  • email, of type STR, which is searchable.
  • age, of type INT, which is just used internally.
package io.littlehorse.quickstart;

import io.littlehorse.sdk.common.config.LHConfig;
import io.littlehorse.sdk.wfsdk.LHFormatString;
import io.littlehorse.sdk.wfsdk.NodeOutput;
import io.littlehorse.sdk.wfsdk.WfRunVariable;
import io.littlehorse.sdk.wfsdk.Workflow;
import io.littlehorse.sdk.wfsdk.WorkflowThread;
import io.littlehorse.sdk.common.proto.VariableType;

public class Main {

    public static String WF_NAME = "variables-example";

    public static void wfLogic(WorkflowThread wf) {
        WfRunVariable userId = wf.declareStr("user-id").required().searchable();
        WfRunVariable userObject = wf.declareJsonObj("user-obj");
        WfRunVariable age = wf.declareInt("age");

        // This task returns a json blob (the `User` class)
        NodeOutput userOutput = wf.execute("fetch-user", userId);

        // You can take a Json output from a task and just copy it into a JSON_OBJ variable
        userObject.assign(userOutput);

        // You can also use specific fields of the task output to edit a non-json variable
        age.assign(userOutput.jsonPath("$.age"));

        // This is somewhat advanced; we create an expression which combines a few
        // strings together.
        LHFormatString message = wf.format(
                "Hello there, {0}! You are {1} years old",
                userObject.jsonPath("$.title"),
                age);
        wf.execute("send-email", userObject.jsonPath("$.email"), message);
    }

    public static void main(String[] args) throws Exception {
        LHConfig config = new LHConfig();
        Workflow wfGenerator = Workflow.newWorkflow(WF_NAME, Main::wfLogic);
        wfGenerator.registerWfSpec(config.getBlockingStub());
    }
}
tip

In the above example,

At this point, you should see the WfSpec in the dashboard. It will look something like this:

A WfSpec diagram with two tasks in sequence: fetch-user and send-email
The Variables Example WfSpec

Running the WfRun​

Now let's run the workflow!

->lhctl run variables-example
2025/03/08 11:32:31 rpc error: code = InvalidArgument desc = Must provide required input variable user-id of type STR

Oops, that didn't go too well. The LittleHorse Server got grumpy with us because our WfSpec has a required variable user-id and we didn't pass it in. Let's fix that:

lhctl run variables-example user-id anakin

Now, if we go to the Dashboard, we can see the WfRun is COMPLETED. When we click on the WfRun, we can easily inspect the ending state of the variables after all of the mutations happened:

Three variables on the LittleHorse dashboard: user-id, email, and user-object
The Variables from your WfRun

If you inspect the user-object variable, you'll notice it exactly matches the output of the fetch-user task, which is just the User object serialized to json.

{
  "email": "anakin@jedi.temple",
  "title": "Padawan Skywalker (not Master)",
  "age": 22
}

Have some fun and explore! Look at what was passed into each of your TaskRuns and compare that to our WfSpec code. This will help you understand the parallels between LittleHorse Variables + WfSpecs and Java variables + programs.

Searching by Variables​

Before we start searching, let's run one more WfRun:

lhctl run variables-example user-id obiwan

You can search for a Variable using rpc SearchVariable, or via lhctl as follows:

lhctl search variable --wfSpecName variables-example --name user-id --varType STR --value anakin

The output looks like:

{
  "results": [
    {
      "wfRunId": {
        "id": "52357bdec63241ed9b3dfef2b5f4a1d4"
      },
      "threadRunNumber": 0,
      "name": "user-id"
    }
  ]
}
note

The command above searches for Variables with a specific type. The VariableId includes a WfRunId, so you can easily find the associated WfRun.

Let's look at the actual Variable:

lhctl get variable 52357bdec63241ed9b3dfef2b5f4a1d4 0 user-id

The output shows that the value is indeed anakin!

{
  "id": {
    "wfRunId": {
      "id": "52357bdec63241ed9b3dfef2b5f4a1d4"
    },
    "threadRunNumber": 0,
    "name": "user-id"
  },
  "value": {
    "str": "anakin"
  },
  "createdAt": "2025-03-08T19:33:45.431Z",
  "wfSpecId": {
    "name": "variables-example",
    "majorVersion": 0,
    "revision": 0
  },
  "masked": false
}

Lastly, you can search for variables on the dashboard, by going to the WfSpec page and clicking Search By Variable.

note

Please note that, while Obi-Wan Kenobi is indeed a Jedi Master, Anakin Skywalker is not.

Gotchas​

LittleHorse Variables are designed to be as close to programming language variables as possible. However, there are some key differences to be aware of.

Why WfRunVariable is Special​

It's important to note that your Workflow Function (using the WorkflowThread) is not executed when you run a WfRun. Rather, it is executed once to compile a WfSpec using the LittleHorse DSL, and gets translated into a WfSpec protobuf object which is a directed graph of nodes and edges.

The WorkflowThread#declareStr() method tells LittleHorse to create a LittleHorse Variable inside the WfSpec. At runtime, when an instance of the workflow (WfRun) is started, LittleHorse schedules and executes the tasks defined in the WfSpec, and the LittleHorse variables in the WfSpec will also be created and stored in the LittleHorse server.

Variable Payload Size​

Variables are great for passing small pieces of data around (and even works for JSON files up to a few dozen KB). However, if you need to pass around large files or data, you should consider using a different mechanism, you should store the data payload in an external system and use a LittleHorse Variable to pass around a reference to the data (eg. an S3 URL). There is a hard limit to LittleHorse Variables of size 250KB.

Further Resources​

Congrats on learning how to use Variables in your LittleHorse workflows! There's a lot more you can do with Variables, though, so keep on reading: