# Multi-Modality and Attachments Langfuse supports multi-modal traces including **text, images, audio, and other attachments**. By default, **[base64 encoded data URIs](https://developer.mozilla.org/en-US/docs/Web/URI/Schemes/data#syntax) are handled automatically by the Langfuse SDKs**. They are extracted from the payloads commonly used in multi-modal LLMs, uploaded to Langfuse's object storage, and linked to the trace. This also works if you: 1. Reference media files via external URLs. 2. Customize the handling of media files in the SDKs via the `LangfuseMedia` class. 3. Integrate via the Langfuse API directly. Learn more on how to get started and how this works under the hood below. _Examples_ ImagesAudioAttachments ## Availability ### Langfuse Cloud Multi-modal attachments on Langfuse Cloud are currently free on Langfuse Cloud. We reserve the option to roll out a new pricing metric to account for the additional storage and compute costs associated with large multi-modal traces in the near-term future. ### Self-hosting Multi-modal attachments are available today. You need to configure your own object storage bucket via the Langfuse environment variables (`LANGFUSE_S3_MEDIA_UPLOAD_*`). See self-hosting documentation for details on these environment variables. S3-compatible APIs are supported across all major cloud providers and can be self-hosted via minio. Note that the configured storage bucket must have a publicly resolvable hostname to support direct uploads via our SDKs and media asset fetching directly from the browser. ## Supported media formats Langfuse supports a wide range of media types, including: - **Images**: .png, .jpg, .webp, .gif - **Audio**: .mp3, .wav, .ogg - **Video**: .mp4, .webm, .mov - **Text & code**: .txt, .md, .html, .csv - **Documents**: .pdf, .docx, .xlsx, .pptx - **Data & archives**: .json, .xml, .zip Full list of supported MIME types | Category | MIME type | File extension | |---------------------|-------------------------------------------------------------------------------------|---------------------------------| | Images | `image/png` | `.png` | | Images | `image/jpeg`, `image/jpg` | `.jpg`, `.jpeg` | | Images | `image/webp` | `.webp` | | Images | `image/gif` | `.gif` | | Images | `image/svg+xml` | `.svg` | | Images | `image/tiff` | `.tiff` | | Images | `image/bmp` | `.bmp` | | Images | `image/avif` | `.avif` | | Images | `image/heic` | `.heic` | | Audio | `audio/mpeg`, `audio/mp3` | `.mp3` | | Audio | `audio/wav` | `.wav` | | Audio | `audio/ogg` | `.ogg` | | Audio | `audio/oga` | `.oga` | | Audio | `audio/aac` | `.aac` | | Audio | `audio/mp4` | `.m4a` | | Audio | `audio/flac` | `.flac` | | Audio | `audio/opus` | `.opus` | | Audio | `audio/webm` | `.weba` | | Video | `video/mp4` | `.mp4` | | Video | `video/webm` | `.webm` | | Video | `video/ogg` | `.ogv` | | Video | `video/mpeg` | `.mpeg` | | Video | `video/quicktime` | `.mov` | | Video | `video/x-msvideo` | `.avi` | | Video | `video/x-matroska` | `.mkv` | | Text & code | `text/plain` | `.txt` | | Text & code | `text/html` | `.html` | | Text & code | `text/css` | `.css` | | Text & code | `text/csv` | `.csv` | | Text & code | `text/markdown` | `.md` | | Text & code | `text/x-python` | `.py` | | Text & code | `application/javascript` | `.js` | | Text & code | `text/x-typescript` | `.ts` | | Text & code | `application/x-yaml` | `.yaml` | | Documents | `application/pdf` | `.pdf` | | Documents | `application/msword` | `.doc` | | Documents | `application/vnd.openxmlformats-officedocument.wordprocessingml.document` | `.docx` | | Documents | `application/vnd.ms-excel` | `.xls` | | Documents | `application/vnd.openxmlformats-officedocument.spreadsheetml.sheet` | `.xlsx` | | Documents | `application/vnd.openxmlformats-officedocument.presentationml.presentation` | `.pptx` | | Documents | `application/rtf` | `.rtf` | | Data & archives | `application/json` | `.json` | | Data & archives | `application/x-ndjson` | `.jsonl` | | Data & archives | `application/xml` | `.xml` | | Data & archives | `application/vnd.apache.parquet` | `.parquet` | | Data & archives | `application/zip` | `.zip` | | Data & archives | `application/gzip` | `.gz` | | Data & archives | `application/x-tar` | `.tar` | | Data & archives | `application/x-7z-compressed` | `.7z` | | Data & archives | `application/octet-stream` | `.bin` | If you require support for additional file types, please let us know in our [GitHub Discussion](https://github.com/orgs/langfuse/discussions/3004) where we're actively gathering feedback on multi-modal support. ## Get Started ### Base64 data URI encoded media If you use base64 encoded images, audio, or other files in your LLM applications, upgrade to the latest version of the Langfuse SDKs. The Langfuse SDKs automatically detect and handle base64 encoded media by extracting it, uploading it separately as a Langfuse Media file, and including a reference in the trace. This works with standard Data URI ( [MDN](https://developer.mozilla.org/en-US/docs/Web/URI/Schemes/data#syntax)) formatted media (like those used by OpenAI and other LLMs). This [notebook](/content/guides/cookbook/example_multi_modal_traces/index.html) includes a couple of examples using the OpenAI SDK and LangChain. ### External media (URLs) Langfuse supports in-line rendering of media files via URLs if they follow common formats. In this case, the media file is not uploaded to Langfuse's object storage but simply rendered in the UI directly from the source. Supported formats: Markdown imagesOpenAI content parts ``` ``` ``` { "content": [ { "role": "system", "content": "You are an AI trained to describe and interpret images. Describe the main objects and actions in the image." }, { "role": "user", "content": [ { "type": "text", "text": "What's happening in this image?" }, { "type": "image_url", "image_url": { "url": "https://example.com/image.jpg" } } ] } ] } ``` ### Custom attachments If you want to have more control or your media is not base64 encoded, you can upload arbitrary media attachments to Langfuse via the SDKs using the new `LangfuseMedia` class. Wrap media with LangfuseMedia before including it in trace inputs, outputs, metadata, or dataset items. See the multi-modal documentation for examples. **Python SDK** ```python from langfuse import get_client, observe, propagate_attributes from langfuse.media import LangfuseMedia # Create a LangfuseMedia object from a file with open("static/bitcoin.pdf", "rb") as pdf_file: pdf_bytes = pdf_file.read() # Wrap media in LangfuseMedia class pdf_media = LangfuseMedia(content_bytes=pdf_bytes, content_type="application/pdf") # Using with the decorator @observe() def process_document(): langfuse = get_client() # Propagate metadata (including media) to all child observations with propagate_attributes( metadata={"document": pdf_media} ): pass # Or update the current span langfuse.update_current_span( input={"document": pdf_media} ) ``` **JavaScript/TypeScript SDK** ```javascript import fs from "fs"; import { LangfuseMedia } from "@langfuse/core"; // Wrap media in LangfuseMedia class const wrappedMedia = new LangfuseMedia({ source: "bytes", contentBytes: fs.readFileSync(new URL("./bitcoin.pdf", import.meta.url)), contentType: "application/pdf", }); // Optionally, access media via wrappedMedia.obj console.log(wrappedMedia.obj); // Include media in any trace or observation const span3 = startObservation("media-pdf-generation"); const generation3 = span3.startObservation('llm-call', { model: 'gpt-4', input: wrappedMedia, }, {asType: "generation"}); generation3.end(); span3.end(); ``` ### API If you use the API directly to log traces to Langfuse, you need to follow these steps: ### Upload media to Langfuse 1. If you use base64 encoded media: you need to extract it from the trace payloads similar to how the Langfuse SDKs do it. 2. Initialize the upload and get a `mediaId` and `presignedURL`: [`POST /api/public/media`](https://api.reference.langfuse.com/#tag/media/post/api/public/media). 3. Upload media file: `PUT [presignedURL]`. See this [end-to-end example](/content/guides/cookbook/example_multi_modal_traces#custom-via-api/index.html) (Python) on how to use the API directly to upload media files. ### Add reference to mediaId in trace/observation Use the [Langfuse Media Token](/content/docs/observability/features/multi-modality#media-token/index.html) to reference the `mediaId` in the trace or observation `input`, `output`, or `metadata`. ## How does it work? When using media files (that are not referenced via external URLs), Langfuse handles them in the following way: ### 1. Media Upload Process #### Detection and Extraction - Langfuse supports media files in traces and observations on `input`, `output`, and `metadata` fields - SDKs separate media from tracing data client-side for performance optimization - Media files are uploaded directly to object storage (AWS S3 or compatible) - Original media content is replaced with a reference string #### Security and Optimization - Uploads use presigned URLs with content validation (content length, content type, content SHA256 hash) - Deduplication: Files are simply replaced by their `mediaId` reference string if already uploaded - File uniqueness determined by project, content type, and content SHA256 hash #### Implementation Details - Python SDK: Background thread handling for non-blocking execution - JS/TS SDKs: Asynchronous, non-blocking implementation - API support for direct uploads (see [guide](/content/guides/cookbook/example_multi_modal_traces#custom-via-api/index.html)) ### 2. Media Reference System The base64 data URIs and the wrapped `LangfuseMedia` objects in Langfuse traces are replaced by references to the `mediaId` in the following standardized token format, which helps reconstruct the original payload if needed: ``` @@@langfuseMedia:type={MIME_TYPE}|id={LANGFUSE_MEDIA_ID}|source={SOURCE_TYPE}@@@ ``` - `MIME_TYPE`: MIME type of the media file, e.g., `image/jpeg` - `LANGFUSE_MEDIA_ID`: ID of the media file in Langfuse's object storage - `SOURCE_TYPE`: Source type of the media file, can be `base64_data_uri`, `bytes`, or `file` Based on this token, the Langfuse UI can automatically detect the `mediaId` and render the media file inline. The `LangfuseMedia` class provides utility functions to extract the `mediaId` from the reference string. For multi-modal datasets, use [Experiments via SDK](/content/docs/evaluation/experiments/experiments-via-sdk#multimodal-experiments/index.html) to fetch dataset items with resolved `LangfuseMediaReference` objects and pass the media into your model provider. ### 3. Resolving Media References When dealing with traces, observations, or dataset items that include media references, you can convert them back to their base64 data URI format using the `resolve_media_references` utility method provided by the Langfuse client. This is particularly useful for reinserting the original content during fine-tuning, dataset runs, or replaying a generation. The utility method traverses the parsed object and returns a deep copy with all media reference strings replaced by the corresponding base64 data URI representations. **Python SDK** ```python from langfuse import get_client # Initialize Langfuse client langfuse = get_client() # Example object with media references obj = { "image": "@@@langfuseMedia:type=image/jpeg|id=some-uuid|source=bytes@@@", "nested": { "pdf": "@@@langfuseMedia:type=application/pdf|id=some-other-uuid|source=bytes@@@" } } # Resolve media references to base64 data URIs resolved_obj = langfuse.resolve_media_references( obj=obj, resolve_with="base64_data_uri" ) # Result: # { # "image": "data:image/jpeg;base64,/9j/4AAQSkZJRg...", # "nested": { # "pdf": "data:application/pdf;base64,JVBERi0xLjcK..." # } # } ``` **JavaScript SDK** ```javascript import { LangfuseClient } from "@langfuse/client"; const langfuse = new LangfuseClient() // Example object with media references const obj = { image: "@@@langfuseMedia:type=image/jpeg|id=some-uuid|source=bytes@@@", nested: { pdf: "@@@langfuseMedia:type=application/pdf|id=some-other-uuid|source=bytes@@@", }, }; // Resolve media references to base64 data URIs const resolvedTrace = await langfuse.resolveMediaReferences({ obj: obj, resolveWith: "base64DataUri", }); // Result: // { // image: "data:image/jpeg;base64,/9j/4AAQSkZJRg...", // nested: { // pdf: "data:application/pdf;base64,JVBERi0xLjcK..." // } // } ``` ### GitHub Discussions [Enable images in prompt placeholders](https://github.com/orgs/langfuse/discussions/9435 "Langfuse Ideas: Enable images in prompt placeholders") [Disable S3 Media Upload for Base64 Images via Python Client Configuration](https://github.com/orgs/langfuse/discussions/6560 "Langfuse Ideas: Disable S3 Media Upload for Base64 Images via Python Client Configuration") [Support for more modalities and base64 content](https://github.com/orgs/langfuse/discussions/3004 "Langfuse Ideas: Support for more modalities and base64 content") [How can we see traces of MULTIMODAL llms over langfuse ?](https://github.com/orgs/langfuse/discussions/2510 "Langfuse Ideas: How can we see traces of MULTIMODAL llms over langfuse ?") [Add Images in Tracing](https://github.com/orgs/langfuse/discussions/1208 "Langfuse Ideas: Add Images in Tracing") [Supporting OpenAI Vision](https://github.com/orgs/langfuse/discussions/982 "Langfuse Ideas: Supporting OpenAI Vision") [Langfuse support for Audio prompt input / output](https://github.com/orgs/langfuse/discussions/11740 "Langfuse Support: Langfuse support for Audio prompt input / output") [Manually creating file names for images saved in Minio](https://github.com/orgs/langfuse/discussions/5898 "Langfuse Support: Manually creating file names for images saved in Minio") [Langfuse - not capturing audio tokens information (gpt-4o-audio-preview)](https://github.com/orgs/langfuse/discussions/5205 "Langfuse Support: Langfuse - not capturing audio tokens information (gpt-4o-audio-preview)") [LangFuse Low Level SDK does not render images by passing the link](https://github.com/orgs/langfuse/discussions/3477 "Langfuse Support: LangFuse Low Level SDK does not render images by passing the link") GitHubSupportGitHubIdeas Upvotes - [1votes\ Langfuse support for Audio prompt input / output\ g00dsec•1/27/2026•\ 2Resolved](https://github.com/orgs/langfuse/discussions/11740) - [1votes\ Manually creating file names for images saved in Minio\ jg435•3/6/2025•\ 2](https://github.com/orgs/langfuse/discussions/5898) - [1votes\ Langfuse - not capturing audio tokens information (gpt-4o-audio-preview)\ chi1337•1/24/2025•\ 2Resolved](https://github.com/orgs/langfuse/discussions/5205) - [1votes\ LangFuse Low Level SDK does not render images by passing the link\ VikramShenoy97•9/25/2024•\ 1](https://github.com/orgs/langfuse/discussions/3477) Discussions last updated: 7/18/2026, 2:42:32 AM (55 hours ago)