Build a human-approved AI triage workflow in Kestra

Outcome

What you will build

By the end, you will have a Kestra flow that:

1. Reads one GitHub issue using the GitHub REST API.
2. Extracts deterministic facts: issue number, title, author, current labels and body.
3. Sends those facts to an OpenAI chat completion task for a structured triage suggestion.
4. Pauses the execution and asks a human to approve, edit or reject the suggestion.
5. If approved and not in dry-run mode, calls GitHub to add a label and create a comment.

Kestra is a workflow orchestration platform with YAML flows, a UI and plugin-based tasks. Its quickstart shows a local Docker launch and its Docker Compose guide covers a PostgreSQL-backed local setup. The core Pause task stops downstream scheduling and can collect inputs when a person resumes the flow.

Fit

When this pattern is useful

Use this pattern when the work has a repeatable shape but still needs judgement:

• customer support issue triage;
• internal IT request routing;
• operations tickets that need human sign-off before an external system changes;
• lightweight compliance checks where auditability matters more than speed.

Do not use it for fully autonomous production changes, high-volume real-time queues, or decisions where a mistaken label or comment could create legal, financial or reputational harm.

Before you start

Prerequisites

• Docker and Docker Compose installed.
• A GitHub sandbox repository where you can create a test issue.
• A GitHub token with the narrowest practical access to that repository.
• An OpenAI API key if you use the OpenAI plugin shown below.
• Basic comfort editing YAML.

Step 01

Start Kestra locally with Docker Compose

Aim

Run a local PostgreSQL-backed Kestra instance that you can inspect and discard safely.

Action

mkdir kestra-ai-triage
cd kestra-ai-triage

Download Kestra’s Docker Compose file and start it:

curl -o docker-compose.yml \
  https://raw.githubusercontent.com/kestra-io/kestra/develop/docker-compose.yml

docker compose up -d

Open http://localhost:8080.

Check

The Kestra UI should load and let you create a flow.

Common mistake

Kestra’s single-container H2 quickstart is useful for a brief product tour, but use Compose for this tutorial because its PostgreSQL-backed setup is closer to a real operator environment.

Step 02

Add secrets without putting tokens in YAML

Aim

Keep GitHub and OpenAI credentials out of the flow definition.

Action

The flow uses two secrets: GITHUB_TOKEN and OPENAI_TOKEN.

Kestra exposes secrets at runtime with {{ secret('NAME') }}. In Open Source, secret() reads base64-encoded environment variables prefixed with SECRET_. Add these to the Kestra service in docker-compose.yml:

environment:
  SECRET_GITHUB_TOKEN: "<base64-encoded-token>"
  SECRET_OPENAI_TOKEN: "<base64-encoded-key>"

Restart Kestra after changing the Compose file:

docker compose up -d

Check

The flow should resolve both secrets at runtime without exposing them in its YAML.

Least-privilege checklist

• Use a sandbox repository first.
• Grant issue read access for fetching.
• Grant issue write access only for the mutation step.
• Keep dry_run true until the approval path is tested.
• Avoid broad organisation tokens.
• Rotate test tokens after the exercise.

Step 03

Create a test issue

Aim

Give the workflow one controlled, non-sensitive record to process.

Action

In your sandbox repository, create an issue with a clear title and body:

Title: Checkout fails when postcode contains a space
Body: Customers report that SW1A 1AA is rejected during checkout. Expected: valid UK postcodes with spaces should be accepted.

Check

Note the owner, repository name and issue number. You will pass them into the flow as inputs.

Common mistake

Do not copy a real support ticket containing customer details into the sandbox.

Step 04

Add the Kestra flow

Aim

Build an inspectable route where the human decision remains between AI output and GitHub changes.

Action

In the Kestra UI, create a new flow and paste the YAML below. Keep dry_run set to true for the first execution.

id: human_approved_github_issue_triage
namespace: company.ops

description: >
  Fetch one GitHub issue, ask an LLM for a constrained triage suggestion,
  pause for a human decision, then add a label and comment only if approved.

inputs:
  - id: owner
    type: STRING
    displayName: GitHub owner or organisation

  - id: repo
    type: STRING
    displayName: GitHub repository

  - id: issue_number
    type: INT
    displayName: GitHub issue number

  - id: dry_run
    type: BOOLEAN
    defaults: true
    displayName: Dry run only

tasks:
  - id: fetch_issue
    type: io.kestra.plugin.core.http.Request
    uri: "https://api.github.com/repos/{{ inputs.owner }}/{{ inputs.repo }}/issues/{{ inputs.issue_number }}"
    method: GET
    headers:
      Accept: application/vnd.github+json
      Authorization: "Bearer {{ secret('GITHUB_TOKEN') }}"
      X-GitHub-Api-Version: "2022-11-28"

  - id: deterministic_facts
    type: io.kestra.plugin.core.log.Log
    message: |
      Issue: #{{ outputs.fetch_issue.body.number }}
      Title: {{ outputs.fetch_issue.body.title }}
      Author: {{ outputs.fetch_issue.body.user.login }}
      State: {{ outputs.fetch_issue.body.state }}
      Existing labels: {{ outputs.fetch_issue.body.labels | jq('[.[].name]') | first }}
      URL: {{ outputs.fetch_issue.body.html_url }}

  - id: ai_triage_suggestion
    type: io.kestra.plugin.openai.ChatCompletion
    apiKey: "{{ secret('OPENAI_TOKEN') }}"
    model: gpt-4o
    prompt: |
      You are helping a UK SME operations team triage a GitHub issue.
      Use only the issue facts below. Do not claim you inspected code, logs or customers.

      Return only compact JSON with these keys:
      - label: one of bug, enhancement, question, needs-info
      - confidence: low, medium or high
      - rationale: one sentence
      - draft_comment: a concise, polite GitHub comment

      Issue facts:
      number: {{ outputs.fetch_issue.body.number }}
      title: {{ outputs.fetch_issue.body.title }}
      author: {{ outputs.fetch_issue.body.user.login }}
      current_labels: {{ outputs.fetch_issue.body.labels | jq('[.[].name]') | first }}
      body: {{ outputs.fetch_issue.body.body }}

  - id: approval
    type: io.kestra.plugin.core.flow.Pause
    onResume:
      - id: approve
        type: BOOLEAN
        required: true
        displayName: Approve GitHub update?
        defaults: false

      - id: final_label
        type: SELECT
        required: true
        displayName: Final label
        values:
          - bug
          - enhancement
          - question
          - needs-info

      - id: final_comment
        type: STRING
        required: true
        displayName: Final comment to post
        defaults: "Thanks for raising this. We have reviewed it and added a triage label."

      - id: decision_notes
        type: STRING
        required: false
        displayName: Internal decision notes

  - id: apply_github_updates
    type: io.kestra.plugin.core.flow.If
    condition: "{{ outputs.approval.onResume.approve == true and inputs.dry_run == false }}"
    then:
      - id: add_label
        type: io.kestra.plugin.core.http.Request
        uri: "https://api.github.com/repos/{{ inputs.owner }}/{{ inputs.repo }}/issues/{{ inputs.issue_number }}/labels"
        method: POST
        contentType: application/json
        headers:
          Accept: application/vnd.github+json
          Authorization: "Bearer {{ secret('GITHUB_TOKEN') }}"
          X-GitHub-Api-Version: "2022-11-28"
        body: |
          {{ {"labels": [outputs.approval.onResume.final_label]} }}

      - id: post_comment
        type: io.kestra.plugin.core.http.Request
        uri: "https://api.github.com/repos/{{ inputs.owner }}/{{ inputs.repo }}/issues/{{ inputs.issue_number }}/comments"
        method: POST
        contentType: application/json
        headers:
          Accept: application/vnd.github+json
          Authorization: "Bearer {{ secret('GITHUB_TOKEN') }}"
          X-GitHub-Api-Version: "2022-11-28"
        body: |
          {{ {
            "body": outputs.approval.onResume.final_comment ~ "\n\n---\nTriage approved by " ~ outputs.approval.resumed.by ~ " on " ~ outputs.approval.resumed.on ~ "."
          } }}

  - id: audit_log
    type: io.kestra.plugin.core.log.Log
    message: |
      Dry run: {{ inputs.dry_run }}
      Approved: {{ outputs.approval.onResume.approve }}
      Final label: {{ outputs.approval.onResume.final_label }}
      Decision notes: {{ outputs.approval.onResume.decision_notes }}
      AI raw suggestion: {{ outputs.ai_triage_suggestion.choices[0].message.content }}

The example uses Kestra’s core HTTP Request task so the GitHub URI, method, headers and request body remain explicit. The OpenAI task receives only issue facts and returns a constrained suggestion. The Pause task then collects the final label and comment from the person who resumes the flow.

Check

Kestra should accept the flow definition and show each task in the route.

Common mistake

Do not remove either condition from the mutation guard: approval must be true and dry-run must be false.

Step 05

Run in dry-run mode

Aim

Prove fetching, suggestion and approval without changing GitHub.

Action

owner: your-org
repo: your-sandbox-repo
issue_number: 1
dry_run: true

Check

• fetch_issue returns HTTP 200.
• deterministic_facts shows the title, author, state and labels.
• ai_triage_suggestion returns compact JSON.
• approval puts the execution into a paused state.

A Pause task moves the execution to PAUSED. With no duration or timeout configured, it waits until someone resumes it through the UI or API.

Step 06

Resume with a human decision

Aim

Review and, if necessary, rewrite the prepared work before it can become an external action.

Action

Open the paused execution in Kestra, select Resume and fill in:

approve: true
final_label: bug
final_comment: Thanks for raising this. We can reproduce the postcode validation problem and have labelled it for investigation.
decision_notes: Approved after checking the issue body; no production change made by AI.

Because dry_run is still true, the update branch should not call GitHub. Confirm the audit task records the decision. Then run the flow again with dry_run: false against the same sandbox issue and resume it only when you are happy with the final label and comment.

Check

The first run records the decision but makes no external change. The second run proceeds only after explicit approval.

Step 07

Review the result in GitHub

In the sandbox repository, confirm:

• the selected label was added;
• the approved comment was posted;
• the comment does not overstate what the automation did;
• the Kestra execution log shows who resumed the approval step.

The Pause output includes who resumed the execution and when, which supports an audit log.

Safe testing

How to test safely

1. Run with a public or throwaway issue and dry_run: true.
2. Approve the pause and inspect the audit log only.
3. Run with dry_run: false in a sandbox repository.
4. Confirm the exact GitHub label and comment.
5. Try a rejection and confirm no external update runs.
6. Try a weak issue body and check the suggestion says needs-info rather than inventing details.
7. Only then consider a private repository with a narrow token.

Exceptions

Troubleshooting

Before wider use

Production hardening checklist

• Replace personal tokens with GitHub App installation tokens where possible.
• Separate read-only fetch credentials from write-capable mutation credentials.
• Add an explicit allow-list of labels the flow may apply.
• Add a maximum comment length.
• Log the original suggestion, approved text and approving user.
• Monitor executions stuck in PAUSED.
• Store the flow in Git and review changes through pull requests.
• Pin plugin and image versions rather than relying on latest.
• Consider Enterprise HumanTask if approval must be assigned to named users or RBAC groups.

Tool fit

When Kestra is a fit

Kestra is a good fit when:

• the work has clear steps and audit requirements;
• operators need a UI to inspect executions, logs and paused approvals;
• YAML and Git review are acceptable for workflow definitions;
• you need deterministic API calls, scripts, notifications and human decisions in one route.

When Kestra is not a fit

Kestra may not be the right first choice when:

• the process is a simple one-off script;
• the team wants only a visual canvas and no YAML ownership;
• every decision must happen in real time without human waiting time;
• the organisation is not ready to manage credentials, logs and approval ownership.

Common questions

Frequently asked questions

References

Sources

1. Kestra quickstart
2. Kestra Docker Compose installation
3. Kestra approval processes
4. Kestra Git integration
5. Kestra Replay
6. Kestra Pause task
7. Kestra GitHub Issues plugin
8. Kestra OpenAI ChatCompletion task
9. Kestra secrets
10. Kestra HTTP Request task
11. GitHub Issues REST API
12. GitHub issue labels REST API
13. GitHub issue comments REST API