Databricks - Mem0

Databricks Vector Search

Databricks Vector Search is a serverless similarity search engine that allows you to store a vector representation of your data, including metadata, in a vector database. With Vector Search, you can create auto-updating vector search indexes from Delta tables managed by Unity Catalog and query them with a simple API to return the most similar vectors.

Usage

Python

import os
from mem0 import Memory

config = {
    "vector_store": {
        "provider": "databricks",
        "config": {
            "workspace_url": "https://your-workspace.databricks.com",
            "access_token": "your-access-token",
            "endpoint_name": "your-vector-search-endpoint",
            "catalog": "your_catalog",
            "schema": "your_schema",
            "table_name": "your_table",
            "collection_name": "your_index_name",
            "embedding_dimension": 1536
        }
    }
}

m = Memory.from_config(config)
m.add(messages, user_id="alice", metadata={"category": "movies"})

TypeScript

// Requires the Databricks SQL driver (peer dependency): pnpm add @databricks/sql
import { Memory } from 'mem0ai/oss';

const config = {
  vectorStore: {
    provider: 'databricks',
    config: {
      workspaceUrl: 'https://your-workspace.databricks.com',
      httpPath: '/sql/1.0/warehouses/your-warehouse-id',
      accessToken: 'your-access-token',
      catalog: 'your_catalog',
      schema: 'your_schema',
      tableName: 'your_table',
      collectionName: 'your_index_name',
      embeddingModelDims: 1536,
    },
  },
};

const memory = new Memory(config);
await memory.add(messages, { userId: "alice", metadata: { category: "movies" } });

Config

Here are the parameters available for configuring Databricks Vector Search:

Parameter Description Default Value
workspace_url The URL of your Databricks workspace Required
access_token Personal Access Token for authentication None
client_id Service principal client ID (alternative to access_token) None
client_secret Service principal client secret (required with client_id) None
endpoint_name Name of the Vector Search endpoint Required
catalog Unity Catalog catalog name Required
schema Unity Catalog schema name Required
table_name Source Delta table name Required
collection_name Vector search index name mem0
index_type Index type: DELTA_SYNC or DIRECT_ACCESS DELTA_SYNC
embedding_dimension Dimension of self-managed embeddings 1536

Authentication

Databricks Vector Search supports two authentication methods:

Service Principal (Recommended for Production)

config = {
    "vector_store": {
        "provider": "databricks",
        "config": {
            "workspace_url": "https://your-workspace.databricks.com",
            "client_id": "your-service-principal-id",
            "client_secret": "your-service-principal-secret",
            "endpoint_name": "your-endpoint",
            "catalog": "your_catalog",
            "schema": "your_schema",
            "table_name": "your_table",
            "collection_name": "your_index_name",
        }
    }
}

Personal Access Token (for Development)

config = {
    "vector_store": {
        "provider": "databricks",
        "config": {
            "workspace_url": "https://your-workspace.databricks.com",
            "access_token": "your-personal-access-token",
            "endpoint_name": "your-endpoint",
            "catalog": "your_catalog",
            "schema": "your_schema",
            "table_name": "your_table",
            "collection_name": "your_index_name",
        }
    }
}

Important Notes