Available Integrations
Braintrust
Enterprise-grade LLM evaluation and monitoring
Langfuse
Open-source LLM observability and tracing
Braintrust
Braintrust is an enterprise-grade platform for LLM evaluation, monitoring, and experimentation.Setup
- Go to Settings > Integrations in the Simforge portal
- Click Connect on the Braintrust card
- Enter your API Key (starts with
sk-) - Click Test Connection to verify
- Click Save
What Gets Traced
When Braintrust is connected, each SDK call creates a span with:- Input: The function inputs you provided
- Output: The parsed result from the LLM
- Model and provider: Which LLM was used
- Token usage: Input, output, and cached tokens
- Metadata: Function ID, version ID, and Simforge trace ID
- Tags:
simforgeandsdk-call
Viewing Traces
After making SDK calls, you can view spans in your Braintrust dashboard. The Simforge trace ID is included in the metadata for cross-referencing.Langfuse
Langfuse is an open-source LLM observability platform that provides tracing, analytics, and prompt management.Setup
- Go to Settings > Integrations in the Simforge portal
- Click Connect on the Langfuse card
- Enter your credentials:
- Host: Your Langfuse host (e.g.,
https://cloud.langfuse.comor your self-hosted URL) - Public Key: Your Langfuse public key (starts with
pk-lf-) - Secret Key: Your Langfuse secret key (starts with
sk-lf-)
- Host: Your Langfuse host (e.g.,
- Click Test Connection to verify
- Click Save
What Gets Traced
When Langfuse is connected, each SDK call creates a trace with:- Input: The function inputs you provided
- Output: The parsed result from the LLM
- Generation details: Model, token usage, and timing
- Metadata: Function ID, version ID, and Simforge trace ID
Viewing Traces
After making SDK calls, you can view traces in your Langfuse dashboard. Each trace includes a direct link back to the corresponding trace in Simforge.Managing Integrations
Updating Credentials
To update an integration’s credentials:- Go to Settings > Integrations
- Click Update on the integration card
- Enter new credentials
- Click Save
Disconnecting
To disconnect an integration:- Go to Settings > Integrations
- Click Disconnect on the integration card
- Confirm the disconnection