TypeScript SDK v4 (beta)
This TypeScript SDK is currently in beta. If you find any issues or have feedback, please open an issue on GitHub.
This SDK is built on top of OpenTelemetry, the industry standard for observability. This allows for a better developer experience, more robust context management, and seamless integration with a wide range of third-party libraries. 🚀
If you are self-hosting Langfuse, the TypeScript SDK v4 requires Langfuse platform version ≥ 3.95.0 for all features to work correctly.
Setup
Installation
The SDK is modular, so you can install only the packages you need.
This SDK requires Node.js version 20 or higher.
Package | Description | Environment |
---|---|---|
@langfuse/core | Core utilities, types, and logger shared across packages. | Universal JS |
@langfuse/client | Client for features like prompts, datasets, and scores. | Universal JS |
@langfuse/tracing | Core OpenTelemetry-based tracing functions (startSpan , etc.). | Node.js ≥ 20 |
@langfuse/otel | The LangfuseSpanProcessor to export traces to Langfuse. | Node.js ≥ 20 |
@langfuse/openai | Automatic tracing integration for the OpenAI SDK. | Universal JS |
@langfuse/langchain | CallbackHandler for tracing LangChain applications. | Universal JS |
Full Tracing SDK (Recommended)
For tracing LLM applications in a Node.js environment, you need the core tracing functions, the OpenTelemetry span processor, and the client for additional features like scoring.
npm install @langfuse/core@beta @langfuse/client@beta @langfuse/tracing@beta @langfuse/otel@beta
Client-only
If you only need to interact with the Langfuse API for features like prompt management, you can install just the client. This lightweight setup is compatible with any JavaScript runtime (Node.js, browsers, edge functions).
npm install @langfuse/core@beta @langfuse/client@beta
OTEL Tracing Setup
To capture traces, you need to set up the OpenTelemetry SDK and register the LangfuseSpanProcessor
. This processor receives all created spans and sends them to Langfuse.
Here is a minimal setup for a Node.js application. You should import this code at the very entry point of your application, before any other modules are loaded.
import { NodeSDK } from "@opentelemetry/sdk-node";
import { LangfuseSpanProcessor } from "@langfuse/otel";
const sdk = new NodeSDK({
// This is the Langfuse OTel Span Processor
spanProcessors: [new LangfuseSpanProcessor()],
// Add any other OpenTelemetry instrumentations
instrumentations: [],
});
sdk.start();
At the entry point of your application, import the setup file:
import "./instrumentation"; // Must be the first import
// Your application code here...
The LangfuseSpanProcessor
is configured via environment variables, but you can also pass them as constructor arguments:
LANGFUSE_PUBLIC_KEY
: Your Langfuse public key.LANGFUSE_SECRET_KEY
: Your Langfuse secret key.LANGFUSE_BASE_URL
: The URL of your Langfuse instance (e.g.,https://cloud.langfuse.com
).
For instrumenting applications in other environments like AWS Lambda, Cloudflare Workers, or Vercel Functions, please refer to the official OpenTelemetry documentation for JavaScript for environment-specific setup instructions. The key is to register the LangfuseSpanProcessor
with the appropriate OTEL SDK for your runtime.
Basic Tracing
Langfuse represents each step in your application as an “observation.” There are three types of observations:
- Spans: Any arbitrary operation, like a tool call, a data processing step, or an API call.
- Generations: LLM calls. These have special fields for model parameters, usage, and cost.
- Events: A single point in time, like a user clicking a button or a specific event occurring.
Manual Observations
The core tracing functions (startSpan
, startGeneration
, createEvent
) give you full control over creating observations.
When you call one of these functions, the new observation is automatically linked as a child of the currently active operation in the OpenTelemetry context. However, it does not make this new observation the active one. This means any further operations you trace will still be linked to the original parent, not the one you just created.
To create nested observations manually, use the methods on the returned object (e.g., parentSpan.startSpan(...)
).
import { startSpan } from "@langfuse/tracing";
// Start a root span for a user request
const span = startSpan("user-request", {
input: { query: "What is the capital of France?" }
});
// Create a nested span for a tool call
const toolCall = span.startSpan("fetch-weather", {
input: { city: "Paris" },
});
// Simulate work and end the tool call span
await new Promise(resolve => setTimeout(resolve, 100));
toolCall.end({ output: { temperature: "15°C" } });
// Create a nested generation for the LLM call
const generation = span.startGeneration("llm-call", {
model: "gpt-4",
input: [{ role: "user", content: "What is the capital of France?" }],
});
generation.end({
output: { content: "The capital of France is Paris." },
usageDetails: { input: 10, output: 5 },
});
// End the root span
span.end({ output: "Successfully answered user request." });
If you use startSpan()
or startGeneration()
, you are responsible for calling .end()
on the returned observation object. Failure to do so will result in incomplete or missing observations in Langfuse.
Automatic Context Management with Callbacks
To simplify nesting and context management, you can use startActiveSpan
and startActiveGeneration
. These functions take a callback and automatically manage the observation’s lifecycle and the OpenTelemetry context. Any observation created inside the callback will automatically be nested under the active observation, and the observation will be ended when the callback finishes.
This is the recommended approach for most use cases as it prevents context leakage and ensures observations are properly ended.
import { startActiveSpan, startGeneration } from "@langfuse/tracing";
await startActiveSpan("user-request", async (span) => {
span.update({
input: { query: "What is the capital of France?" }
});
// This generation will automatically be a child of "user-request"
const generation = startGeneration("llm-call", {
model: "gpt-4",
input: [{ role: "user", content: "What is the capital of France?" }],
});
// ... LLM call logic ...
generation.end({
output: { content: "The capital of France is Paris." },
usageDetails: { input: 10, output: 5 },
});
span.update({ output: "Successfully answered." });
});
observe
Wrapper
The observe
wrapper is a powerful tool for tracing existing functions without modifying their internal logic. It acts as a decorator that automatically creates a span or generation around the function call. You can use the updateActiveSpan
and updateActiveGeneration
functions to add attributes to the observation from within the wrapped function.
import { observe, updateActiveSpan } from "@langfuse/tracing";
// An existing function
async function fetchData(source: string) {
updateActiveSpan({ metadata: { source: "API" } })
// ... logic to fetch data
return { data: `some data from ${source}` };
}
// Wrap the function to trace it
const tracedFetchData = observe(fetchData);
// Now, every time you call tracedFetchData, a span is created.
// Its input and output are automatically populated with the
// function's arguments and return value.
const result = await tracedFetchData("API");
You can configure the observe
wrapper by passing an options object as the second argument:
Option | Description | Default |
---|---|---|
name | The name of the observation. | The original function’s name. |
asType | The type of observation to create (span or generation ). | "span" |
captureInput | Whether to capture the function’s arguments as the input of the observation. | true |
captureOutput | Whether to capture the function’s return value or thrown error as the output of the observation. | true |
Updating Traces
Often, you might not have all the information about a trace (like a userId
or sessionId
) when you start it. The SDK lets you add or update trace-level attributes at any point during its execution.
.updateTrace()
on an observation
When you create an observation manually with startSpan
or startGeneration
, the returned object has an .updateTrace()
method. You can call this at any time before the root span ends to apply attributes to the entire trace.
import { startSpan } from "@langfuse/tracing";
// Start a trace without knowing the user yet
const rootSpan = startSpan("data-processing");
// ... some initial steps ...
// Later, once the user is authenticated, update the trace
const userId = "user-123";
const sessionId = "session-abc";
rootSpan.updateTrace({
userId: userId,
sessionId: sessionId,
tags: ["authenticated-user"],
metadata: { plan: "premium" },
});
// ... continue with the rest of the trace ...
const generation = rootSpan.startGeneration("llm-call");
generation.end();
rootSpan.end();
updateActiveTrace()
When you’re inside a callback from startActiveSpan
, startActiveGeneration
, or a function wrapped with observe
, you might not have a direct reference to an observation object. In these cases, use the updateActiveTrace()
function. It automatically finds the currently active trace in the context and applies the new attributes.
import { startActiveSpan, updateActiveTrace } from "@langfuse/tracing";
await startActiveSpan("user-request", async (span) => {
// Initial part of the request
span.update({ input: { path: "/api/process" } });
// Simulate fetching user data
await new Promise(resolve => setTimeout(resolve, 50));
const user = { id: "user-5678", name: "Jane Doe" };
// Update the active trace with the user's information
updateActiveTrace({
userId: user.id,
metadata: { userName: user.name },
});
// ... continue logic ...
span.end({ output: { status: "success" } });
});
Trace and Observation IDs
In Langfuse, every trace and observation has a unique identifier. Understanding their format and how to set them is useful for integrating with other systems.
- Trace IDs are 32-character lowercase hexadecimal strings, representing 16 bytes of data
- Observation IDs (also known as Span IDs in OpenTelemetry) are 16-character lowercase hexadecimal strings, representing 8 bytes
While the SDK handles ID generation automatically, you may manually set them to align with external systems or create specific trace structures. This is done using the parentSpanContext
option in tracing methods.
When starting a new trace by setting a traceId
, you must also provide an arbitrary parent-spanId
for the parent observation. The parent span ID value is irrelevant as long as it is a valid 16-hexchar string as the span does not actually exist but is only used for trace ID inheritance of the created observation.
You can create valid, deterministic trace IDs from a seed string using createTraceId
. This is useful for correlating Langfuse traces with IDs from external systems, like a support ticket ID.
import { createTraceId, startSpan } from "@langfuse/tracing";
const externalId = "support-ticket-54321";
// Generate a valid, deterministic traceId from the external ID
const langfuseTraceId = await createTraceId(externalId);
// You can now start a new trace with this ID
const rootSpan = startSpan("process-ticket", {}, {
parentSpanContext: {
traceId: langfuseTraceId,
spanId: "0123456789abcdef", // A valid 16 hexchar string; value is irrelevant as parent span does not exist but only used for inheritance
traceFlags: 1 // mark trace as sampled
}
});
// Later, you can regenerate the same traceId to score or retrieve the trace
const scoringTraceId = await createTraceId(externalId);
// scoringTraceId will be the same as langfuseTraceId
Flushing
In short-lived environments like serverless functions (e.g., Vercel Functions, AWS Lambda), you must explicitly flush the traces before the process exits. The LangfuseSpanProcessor
buffers events and sends them in batches, so a final flush ensures no data is lost.
You can export the processor from your OTEL SDK setup file.
import { NodeSDK } from "@opentelemetry/sdk-node";
import { LangfuseSpanProcessor } from "@langfuse/otel";
// Export the processor to be able to flush it
export const langfuseSpanProcessor = new LangfuseSpanProcessor();
const sdk = new NodeSDK({
spanProcessors: [langfuseSpanProcessor],
});
sdk.start();
Then, in your serverless function handler, call forceFlush()
before the function exits.
import { langfuseSpanProcessor } from "./instrumentation";
export async function handler(event, context) {
// ... your application logic ...
// Flush before exiting
await langfuseSpanProcessor.forceFlush();
}
Integrations
OpenAI
The @langfuse/openai
package provides a wrapper to automatically trace calls to the OpenAI SDK.
Installation:
npm install @langfuse/openai@beta
Usage:
The observeOpenAI
function wraps your OpenAI client instance. All subsequent API calls made with the wrapped client will be traced as generations. If there’s no active trace, a new one will be created automatically.
import { OpenAI } from "openai";
import { observeOpenAI } from "@langfuse/openai";
// Instantiate the OpenAI client as usual
const openai = new OpenAI();
// Wrap it with Langfuse
const tracedOpenAI = observeOpenAI(openai, {
// Pass trace-level attributes that will be applied to all calls
traceName: "my-openai-trace",
sessionId: "user-session-123",
userId: "user-abc",
tags: ["openai-integration"],
});
// Use the wrapped client just like the original
const completion = await tracedOpenAI.chat.completions.create({
model: "gpt-4",
messages: [{ role: "user", content: "What is OpenTelemetry?" }],
});
LangChain
The @langfuse/langchain
package offers a CallbackHandler
to integrate Langfuse tracing into your LangChain applications.
Installation:
npm install @langfuse/core@beta @langfuse/langchain@beta
Usage:
Instantiate the CallbackHandler
and pass it to your chain’s .invoke()
or .stream()
method in the callbacks
array. All operations within the chain will be traced as nested observations.
import { ChatOpenAI } from "@langchain/openai";
import { ChatPromptTemplate } from "@langchain/core/prompts";
import { CallbackHandler } from "@langfuse/langchain";
// 1. Initialize the Langfuse callback handler
const langfuseHandler = new CallbackHandler({
sessionId: "user-session-123",
userId: "user-abc",
tags: ["langchain-test"],
});
// 2. Define your chain
const model = new ChatOpenAI({ model: "gpt-4o" });
const prompt = ChatPromptTemplate.fromTemplate("Tell me a joke about {topic}.");
const chain = prompt.pipe(model);
// 3. Add the handler to the callbacks array
const result = await chain.invoke(
{ topic: "developers" },
{
callbacks: [langfuseHandler],
// This becomes the trace name if no active OTEL span is in the context
runName: "joke-generator",
}
);
console.log(result.content);
Vercel AI SDK
The Vercel AI SDK is natively instrumented with OpenTelemetry. If you have configured your OTEL Setup with the LangfuseSpanProcessor
, traces from the Vercel AI SDK will be captured automatically without any additional code.
Please refer to the Vercel AI SDK documentation on Telemetry for instructions on how to enable it.
Other Third-Party Integrations
Many LLM and data libraries are built with OpenTelemetry support. If a library you use supports OTEL, you just need to ensure the LangfuseSpanProcessor
is registered in your OTEL setup. All traces generated by that library will automatically be sent to Langfuse.
Scoring
You can score traces and observations via the langfuse.score
methods on the Langfuse client. This is useful for automated evaluations or for capturing user feedback. See the custom scores documentation for a detailed reference.
import { LangfuseClient } from "@langfuse/client";
import { startSpan, startActiveSpan } from "@langfuse/tracing";
const langfuse = new LangfuseClient();
// First, create a trace and an observation
const rootSpan = startSpan("my-trace");
const generation = rootSpan.startGeneration("my-generation");
generation.end();
rootSpan.end();
// Score the specific generation (an observation)
langfuse.score.observation(generation, {
name: "accuracy",
value: 1,
comment: "The answer was factually correct.",
});
// Score the entire trace
langfuse.score.trace(rootSpan, {
name: "user-satisfaction",
value: 0.9,
comment: "User was happy with the overall result.",
});
// You can also score the currently active observation or trace
startActiveSpan("another-trace", (span) => {
langfuse.score.activeObservation({
name: "latency-score",
value: 0.95,
});
langfuse.score.activeTrace({
name: "quality-score",
value: 0.88,
});
});
// Finally, ensure scores are sent to the server
await langfuse.flush();
Datasets
You can run your models or chains on a Langfuse dataset to evaluate their performance. The SDK provides a way to link each execution to a specific dataset item, creating a Dataset Run in Langfuse. See the datasets documentation for a detailed reference.
The workflow is:
- Fetch the dataset using
langfuse.dataset.get()
. - Iterate through the
dataset.items
. - For each item, execute your model/chain logic, creating a trace.
- Link the trace to the dataset item using the
item.link()
method.
import { LangfuseClient } from "@langfuse/client";
import { startSpan } from "@langfuse/tracing";
const langfuse = new LangfuseClient();
// 1. Fetch the dataset
const dataset = await langfuse.dataset.get("my-evaluation-dataset");
const runName = "my-test-run-v1";
// 2. Iterate and process each item
for (const item of dataset.items) {
// 3. Start a rootSpan for this execution
const rootSpan = startSpan("my-chain", { input: item.input });
// 4. Link the trace to the dataset item for this run
await item.link(rootSpan, runName);
// ... your model/chain execution logic
rootSpan.end({ output: "model-output" });
}
// Flush all traces and links
await langfuse.flush();
Prompts
You can manage prompts in the Langfuse UI and fetch them in your application code. This allows you to version, test, and deploy prompts without changing your application code. See the prompt management documentation for a detailed reference.
Use langfuse.prompt.get()
for caching and resilience
It is highly recommended to use langfuse.prompt.get()
to fetch prompts. This method includes built-in caching to reduce latency and a fallback mechanism for resilience. The direct API wrapper, langfuse.api.prompts.get()
, does not offer these features.
The langfuse.prompt.get()
method fetches a prompt and returns a prompt client with a .compile()
method to inject variables.
import { OpenAI } from "openai";
import { LangfuseClient } from "@langfuse/client";
import { observeOpenAI } from "@langfuse/openai";
const langfuse = new LangfuseClient();
const openai = observeOpenAI(new OpenAI());
// 1. Fetch the production version of the "joke-prompt"
const jokePrompt = await langfuse.prompt.get("joke-prompt");
// 2. Compile the prompt with variables
const compiledPrompt = jokePrompt.compile({ topic: "developers" });
// 3. Use the compiled prompt in your LLM call
const response = await openai.chat.completions.create({
model: "gpt-4o",
messages: [{ role: "user", content: compiledPrompt }],
langfusePrompt: jokePrompt // This links the generation with the prompt
});
Configuration
You can configure fetching behavior with the second argument of langfuse.prompt.get()
:
Option | Description | Default |
---|---|---|
version | Fetch a specific version of the prompt. | undefined |
label | Fetch the prompt with a specific label. | "production" |
cacheTtlSeconds | Duration to cache the prompt in-memory. Set to 0 to disable. | 60 |
fallback | A default prompt (string or object) to use if the API is unreachable or the prompt is not found. | undefined |
Fallback Example
To make your application more resilient on the first prompt fetch when no cache is available, provide a fallback prompt.
const prompt = await langfuse.prompt.get("non-existent-prompt", {
fallback: {
prompt: "This is a fallback for {{topic}}.",
config: { model: "gpt-3.5-turbo", temperature: 0.8 },
},
});
// The application can proceed even if the prompt couldn't be fetched
const compiled = prompt.compile({ topic: "resilience" });
// compiled -> "This is a fallback for resilience."
REST API Wrapper
The SDK client provides a fully-typed wrapper around the Langfuse Public API, available under the langfuse.api
namespace. This allows you to programmatically fetch traces, sessions, scores, and more.
import { LangfuseClient } from "@langfuse/client";
const langfuse = new LangfuseClient();
// Example: Fetch the 10 most recent traces for a specific user
const traces = await langfuse.api.trace.list({
userId: "user-123",
limit: 10,
});
console.log(traces.data);
Advanced Configuration
Logging
You can configure the global SDK logger to control the verbosity of log output. This is useful for debugging.
In code:
import { configureGlobalLogger, LogLevel } from "@langfuse/core";
// Set the log level to DEBUG to see all log messages
configureGlobalLogger({ level: LogLevel.DEBUG });
Available log levels are DEBUG
, INFO
, WARN
, and ERROR
.
Via environment variable:
You can also set the log level using the LANGFUSE_LOG_LEVEL
environment variable.
export LANGFUSE_LOG_LEVEL="DEBUG"
Masking
To prevent sensitive data from being sent to Langfuse, you can provide a mask
function to the LangfuseSpanProcessor
. This function will be applied to the input
, output
, and metadata
of every observation.
The function receives an object { data }
, where data
is the stringified JSON of the attribute’s value. It should return the masked data.
import { NodeSDK } from "@opentelemetry/sdk-node";
import { LangfuseSpanProcessor } from "@langfuse/otel";
const spanProcessor = new LangfuseSpanProcessor({
mask: ({ data }) => {
// A simple regex to mask credit card numbers
const maskedData = data.replace(
/\b\d{4}[- ]?\d{4}[- ]?\d{4}[- ]?\d{4}\b/g,
"***MASKED_CREDIT_CARD***"
);
return maskedData;
},
});
const sdk = new NodeSDK({
spanProcessors: [spanProcessor],
});
sdk.start();
Sampling
Langfuse respects OpenTelemetry’s sampling decisions. You can configure a sampler in your OTEL SDK to control which traces are sent to Langfuse. This is useful for managing costs and reducing noise in high-volume applications.
Here is an example of how to configure a TraceIdRatioBasedSampler
to send only 20% of traces:
import { NodeSDK } from "@opentelemetry/sdk-node";
import { LangfuseSpanProcessor } from "@langfuse/otel";
import { TraceIdRatioBasedSampler } from "@opentelemetry/sdk-trace-base";
const sdk = new NodeSDK({
// Sample 20% of all traces
sampler: new TraceIdRatioBasedSampler(0.2),
spanProcessors: [new LangfuseSpanProcessor()],
});
sdk.start();
For more advanced sampling strategies, refer to the OpenTelemetry JS Sampling Documentation.
Filtering Spans
You can provide a predicate function shouldExportSpan
to the LangfuseSpanProcessor
to decide on a per-span basis whether it should be exported to Langfuse.
Filtering spans may break the parent-child relationships in your traces. For example, if you filter out a parent span but keep its children, you may see “orphaned” observations in the Langfuse UI. Consider the impact on trace structure when configuring shouldExportSpan
.
import { NodeSDK } from "@opentelemetry/sdk-node";
import { LangfuseSpanProcessor, ShouldExportSpan } from "@langfuse/otel";
// Example: Filter out all spans from the 'express' instrumentation
const shouldExportSpan: ShouldExportSpan = ({ otelSpan }) =>
otelSpan.instrumentationScope.name !== "express";
const sdk = new NodeSDK({
spanProcessors: [new LangfuseSpanProcessor({ shouldExportSpan })],
});
sdk.start();
Multi-project Setup (Experimental)
You can configure the SDK to send traces to multiple Langfuse projects. This is useful for multi-tenant applications or for sending traces to different environments. Simply register multiple LangfuseSpanProcessor
instances, each with its own credentials.
The multi-project setup is experimental and may change in the future. Please open a GitHub issue if you have any feedback.
import { NodeSDK } from "@opentelemetry/sdk-node";
import { LangfuseSpanProcessor } from "@langfuse/otel";
const sdk = new NodeSDK({
spanProcessors: [
new LangfuseSpanProcessor({
publicKey: "pk-lf-public-key-project-1",
secretKey: "sk-lf-secret-key-project-1",
}),
new LangfuseSpanProcessor({
publicKey: "pk-lf-public-key-project-2",
secretKey: "sk-lf-secret-key-project-2",
}),
],
});
sdk.start();
This configuration will send every trace to both projects. You can also configure a custom shouldExportSpan
filter for each processor to control which traces go to which project.
Reference
You can find the full SDK reference here.
Upgrade from v3
Tracing
The v4 SDK tracing is a major rewrite based on OpenTelemetry and introduces several breaking changes.
- OTEL-based Architecture: The SDK is now built on top of OpenTelemetry. An OpenTelemetry Setup is required now and done by registering the
LangfuseSpanProcessor
with an OpenTelemetryNodeSDK
. - New Tracing Functions: The
langfuse.trace()
,langfuse.span()
, andlangfuse.generation()
methods have been replaced bystartSpan
,startGeneration
,startActiveSpan
, etc., from the@langfuse/tracing
package. - Separation of Concerns:
- The
@langfuse/tracing
and@langfuse/otel
packages are for tracing. - The
@langfuse/client
package and theLangfuseClient
class are now only for non-tracing features like scoring, prompt management, and datasets.
- The
See the docs above for details on each.
Prompt Management
- Import: The import of the Langfuse client is now:
import { LangfuseClient } from "@langfuse/client";
- Usage: The usage of the Langfuse client is now:
const langfuse = new LangfuseClient();
const prompt = await langfuse.prompt.get("my-prompt");
const compiledPrompt = prompt.compile({ topic: "developers" });
const response = await openai.chat.completions.create({
model: "gpt-4o",
messages: [{ role: "user", content: compiledPrompt }],
});
version
is now an optional property of the second argument oflangfuse.prompt.get()
.
const prompt = await langfuse.prompt.get("my-prompt", { version: "1.0" });
OpenAI integration
- Import: The import of the OpenAI integration is now:
import { observeOpenAI } from "@langfuse/openai";
- You can set the
environment
andrelease
now via theLANGFUSE_TRACING_ENVIRONMENT
andLANGFUSE_TRACING_RELEASE
environment variables.
Langchain integration
- Import: The import of the Langchain integration is now:
import { CallbackHandler } from "@langfuse/langchain";
- You can set the
environment
andrelease
now via theLANGFUSE_TRACING_ENVIRONMENT
andLANGFUSE_TRACING_RELEASE
environment variables.
langfuseClient.getTraceUrl
- method is now asynchronous and returns a promise
const traceUrl = await langfuseClient.getTraceUrl(traceId);
Scoring
See scoring section above for new scoring methods.
Datasets
See datasets section above for new dataset methods.