Skip to main content
The Simforge TypeScript SDK captures your AI function calls to automatically generate evaluations. Re-run your prompts with different models, parameters, and inputs to iterate faster.

​
Installation

# npm
npm install @goharvest/simforge

# pnpm
pnpm add @goharvest/simforge

# yarn
yarn add @goharvest/simforge

​
Quick Start

import { Simforge } from "@goharvest/simforge"

const simforge = new Simforge({ apiKey: process.env.SIMFORGE_API_KEY })
Need an API key? Get one from the Simforge dashboard or see the API Keys guide for detailed setup instructions.
Copy this prompt into your coding agent (tested with Cursor and Claude Code using Sonnet 4.5):
Modify existing TypeScript code to add Simforge tracing.
Do NOT browse or web search. Use ONLY the API described below.

Simforge TypeScript SDK (authoritative excerpt):
- Install: `npm install @goharvest/simforge` or `pnpm add @goharvest/simforge`
- Init:
  import { Simforge } from "@goharvest/simforge"
  const simforge = new Simforge({ apiKey: process.env.SIMFORGE_API_KEY })
- Instrumentation (ONLY allowed form - use getFunction):
  // Declare trace function key once
  const myService = simforge.getFunction("<trace_function_key>")

  // Wrap functions with withSpan
  const tracedFn = myService.withSpan(originalFunction)

  // Or with options:
  const tracedFn = myService.withSpan({ name: "DisplayName", type: "function" }, originalFunction)

  // Span types: "llm", "agent", "function", "guardrail", "handoff", "custom"
- DO NOT modify the original function.
- DO NOT extract helper methods.

Task:
1) Ensure @goharvest/simforge is installed and initialization exists.
2) Read the codebase and identify ALL AI workflows (LLM calls, agent runs, AI-driven decisions).
3) Present me with a numbered list of workflows you found. For each, describe:
   - What it does
   - Why it's worth instrumenting — what visibility tracing gives you into each step
4) After I choose which workflow(s) to instrument:
   - Create a function wrapper with `simforge.getFunction("<trace_function_key>")`
   - Wrap the functions with `myService.withSpan(originalFunction)`
   - Instrument intermediate steps (not just the final output) so each trace has enough context to diagnose issues
   - Replace usages of the original functions with the traced versions
5) Do not change function signature, behavior, or return value. Minimal diff.

Output:
- First: your numbered list of workflows with why each is worth instrumenting
- After my selection: minimal diffs for dependencies, initialization, and the function wrapping

​
Basic Configuration

new Simforge({ apiKey: string })

// Disable tracing (functions still execute, but no spans are sent)
new Simforge({ apiKey: string, enabled: false })
Missing API key doesn’t crash. If the API key is missing, empty, or whitespace-only, the SDK automatically disables tracing and logs a warning. All wrapped functions still execute normally — no spans are sent, no errors are thrown. You don’t need any conditional logic around the API key.

​
Tracing

Declare the trace function key once and wrap multiple functions: 
const orderService = simforge.getFunction("order-processing")

async function processOrder(orderId: string) {
  return { orderId }
}

async function validateOrder(orderId: string) {
  return { valid: true }
}

// Wrap functions - all share the same trace function key
const tracedProcessOrder = orderService.withSpan(processOrder)
const tracedValidateOrder = orderService.withSpan(validateOrder)

​
Multi-File Projects

For projects with instrumented functions spread across multiple files, create a dedicated file that initializes Simforge and exports the function. Import it wherever you need to instrument. 
// lib/simforge.ts — single source of truth
import { Simforge } from "@goharvest/simforge"
const simforge = new Simforge({ apiKey: process.env.SIMFORGE_API_KEY })
export const orderService = simforge.getFunction("order-processing")

// services/processOrder.ts
import { orderService } from "../lib/simforge"

async function processOrder(orderId: string) {
  return { orderId }
}

export const tracedProcessOrder = orderService.withSpan(processOrder)

// services/validateOrder.ts
import { orderService } from "../lib/simforge"

async function validateOrder(orderId: string) {
  return { valid: true }
}

export const tracedValidateOrder = orderService.withSpan(validateOrder)
Spans from different files are automatically linked as parent-child when one wrapped function calls another.

​
Wrapping Existing Functions Inline

When wrapping a function you didn’t define (e.g. an SDK or library call), pass it directly to withSpan and call the result immediately. This ensures the arguments are captured as span input. 
// ✅ GOOD — pass function directly, arguments are captured as span input
const result = await orderService.withSpan(
  { name: "ProcessOrder", type: "function" },
  processOrder,
)(orderId)

// ❌ BAD — anonymous wrapper loses all input capture (span has no input)
const result = await orderService.withSpan(
  { name: "ProcessOrder", type: "function" },
  async () => processOrder(orderId),
)()
Never wrap functions in an anonymous function like async () => fn(args). The SDK captures the wrapper function’s arguments as span input — an anonymous wrapper has no arguments, so the span records nothing.

​
Using withSpan() Directly

For a single span without linking to a function group: 
const standaloneTask = simforge.withSpan("one-off-operation", () => {
  return "done"
})

​
Automatic Nesting

Spans nest automatically based on call stack: 
const outer = simforge.withSpan("outer", { type: "agent" }, async () => {
  await inner()  // Becomes a child of "outer"
})

const inner = simforge.withSpan("inner", { type: "function" }, async () => {
  // ...
})

​
Span Options

Parameters:
  • traceFunctionKey (required): String identifier for grouping spans
  • name (optional): Display name. Defaults to function name, then trace function key
  • type (optional): Span type. Defaults to "custom"
Span Types: 
type SpanType =
  | "llm"        // LLM calls
  | "agent"      // Agent workflows
  | "function"   // Function calls
  | "guardrail"  // Safety checks
  | "handoff"    // Human handoffs
  | "custom"     // Default
Examples: 
// Function name is automatically captured as span name
async function processOrder(orderId: string) {
  return { orderId }
}
const traced = simforge.withSpan("order-processing", processOrder)
// Span name: "processOrder"

// Override with name option
const traced = simforge.withSpan(
  "order-processing",
  { name: "OrderProcessor" },
  processOrder
)
// Span name: "OrderProcessor"

// Set span type
const checkSafety = simforge.withSpan(
  "safety-check",
  { type: "guardrail" },
  async (content: string) => ({ safe: true })
)

// With getFunction()
const service = simforge.getFunction("order-processing")
const traced = service.withSpan({ name: "CustomName", type: "function" }, processOrder)

​
Span Context

Use getCurrentSpan() to get a handle to the active span, then call .addContext() to attach contextual key-value pairs from inside a traced function — useful for runtime values like request IDs, computed scores, or dynamic context: 
import { getCurrentSpan } from "@goharvest/simforge"

async function processOrder(orderId: string) {
  const userId = await getCurrentUser()
  getCurrentSpan()?.addContext({ user_id: userId, order_id: orderId })
  return { orderId, status: "completed" }
}

const traced = simforge.withSpan("order-processing", { type: "function" }, processOrder)
Each addContext call pushes the entire object as one entry. Multiple calls accumulate entries: 
getCurrentSpan()?.addContext({ user_id: "u-123" })
getCurrentSpan()?.addContext({ request_id: "req-789" })
// Result: contexts: [{ user_id: "u-123" }, { request_id: "req-789" }]

​
Span Trace ID

Access the current trace ID from within a span using getCurrentSpan().traceId. This is useful for capturing trace IDs to use with replay or for logging: 
import { getCurrentSpan } from "@goharvest/simforge"

const traced = simforge.withSpan("my-function", async () => {
  const traceId = getCurrentSpan().traceId  // UUID string
  console.log("Current trace:", traceId)
  return { traceId }
})
Outside a span context, getCurrentSpan().traceId returns an empty string.

​
Span Prompt

Use getCurrentSpan() to set the prompt string on the current span. This is stored in span_data.prompt and is useful for capturing the exact prompt text sent to an LLM: 
import { getCurrentSpan } from "@goharvest/simforge"

async function classifyText(text: string) {
  const prompt = `Classify the following text: ${text}`
  getCurrentSpan()?.setPrompt(prompt)
  const result = await llm.complete(prompt)
  return result
}

const traced = simforge.withSpan("classification", { type: "llm" }, classifyText)
The last setPrompt call wins — it overwrites any previously set prompt on the span. Calling setPrompt outside a span context is a no-op (it never crashes).

​
BAML Auto-Instrumentation

If you use BAML for your LLM calls, wrapBAML automatically captures the rendered prompt and LLM metadata (model, provider, token counts, duration) on the current span — no manual setPrompt or addContext calls needed. 
npm install @boundaryml/baml

import { b } from "./baml_client"

// Pass your BAML client to the constructor
const simforge = new Simforge({
  apiKey: process.env.SIMFORGE_API_KEY,
  bamlClient: b,
})

// Wrap a BAML method — prompt and metadata are captured automatically
const tracedClassify = simforge.withSpan(
  "classify",
  { type: "llm" },
  simforge.wrapBAML(b.ClassifyText),
)

const result = await tracedClassify("Hello world")
wrapBAML works by creating a BAML Collector, running the method through a tracked client, then extracting:
  • Prompt → setPrompt() with the rendered messages (system + user)
  • Context → addContext() with { model, provider, inputTokens, outputTokens, durationMs }
If @boundaryml/baml is not installed, the method is called directly without instrumentation.

​
Trace Context

Use getCurrentTrace() to set context that applies to the entire trace (all spans within a single execution). This is useful for grouping traces by session or attaching trace-level metadata: 
import { getCurrentTrace } from "@goharvest/simforge"

const traced = simforge.withSpan("order-processing", { type: "function" }, async () => {
  const trace = getCurrentTrace()

  // Set session ID (stored as database column, filterable in dashboard)
  trace?.setSessionId("session-123")

  // Set trace metadata (stored in raw trace data)
  trace?.setMetadata({ region: "us-west-2", environment: "production" })

  // Add context entries (stored as key-value pairs, accumulates across calls)
  trace?.addContext({ workflow: "checkout-flow", batch_id: "batch-2024-01" })

  return { status: "completed" }
})
  • setSessionId(id) — Groups traces by user session. Stored as a database column for efficient filtering.
  • setMetadata(obj) — Arbitrary key-value metadata on the trace. Merges with existing metadata.
  • addContext(obj) — Key-value context entries. Accumulates across multiple calls.

​
Error Handling

Errors are captured in the span and re-raised: 
const risky = simforge.withSpan("risky-service", () => {
  throw new Error("error")
})

try {
  risky()
} catch (e) {
  // Span records error and timing
}

​
OpenAI Agents SDK

Attach a trace processor to capture agent runs: 
npm install @openai/agents

import { setTraceProcessors } from "@openai/agents"

const processor = simforge.getOpenAiTracingProcessor()
setTraceProcessors([processor])

​
Native Functions

Simforge’s native functions improve prompt tuning efficacy. The auto-tuning engine has full access to prompts—unlike other Agent SDKs that nest user instructions inside system prompts, making prompts inaccessible to tracing. 
const result = await simforge.call("ExtractName", { text: "My name is John Doe" })
// Returns typed result

​
Advanced Configuration

new Simforge({
  apiKey: string,                    // Required
  serviceUrl?: string,               // Default: https://simforge.goharvest.ai
  timeout?: number,                  // Request timeout in ms (default: 120000)
  envVars?: { OPENAI_API_KEY: string },  // For native function execution
  enabled?: boolean,                 // Default: true
  bamlClient?: unknown               // Generated BAML client (for wrapBAML)
})
  • timeout: Request timeout in milliseconds for API calls. Defaults to 120000 (2 minutes).
  • envVars: Pass LLM provider API keys for native function execution via call().
  • enabled: When false, all tracing is disabled. Wrapped functions still execute normally but no spans are sent.
  • bamlClient: The generated BAML client instance (e.g., b from @baml). Used by wrapBAML() when no explicit client is passed at the call site.

​
Replay

Replay historical traces through an updated function version to compare outputs: 
const pipeline = simforge.getFunction("my-function")

const updatedFn = pipeline.withSpan(
  { name: "UpdatedPipeline", type: "function" },
  async (input: string) => {
    // New implementation to test
    return { result: input.toUpperCase() }
  },
)

// Replay all traces (up to limit)
const result = await simforge.replay("my-function", updatedFn, {
  limit: 10,
  maxConcurrency: 5,  // Default: 10
})

// Replay specific traces by ID
const result2 = await simforge.replay("my-function", updatedFn, {
  traceIds: ["trace-id-1", "trace-id-2"],
})

// Result structure
console.log(result.testRunId)   // Test run identifier
console.log(result.testRunUrl)  // Dashboard URL
for (const item of result.items) {
  console.log(item.input)           // Original input
  console.log(item.result)          // New output
  console.log(item.originalOutput)  // Original output
  console.log(item.error)           // Error if any
}
Options:
  • limit — Maximum number of traces to replay (default: all)
  • traceIds — Specific trace IDs to replay
  • maxConcurrency — Number of traces to replay in parallel (default: 10)

​
Sim Runners

Coming soon.