Skip to content

Reading Foundry's LevelDB Databases

What is LevelDB?

LevelDB is a fast key-value storage library written by Google. Foundry VTT uses it to store compendium data for modules and systems. The data is stored in a binary format that requires special tools or libraries to read.

Database File Structure

packs/[pack-name]/
├── *.ldb           # SSTable files containing actual data
├── CURRENT         # Points to current MANIFEST file
├── LOCK           # Prevents concurrent access
├── LOG            # Write-ahead log for crash recovery
└── MANIFEST-*     # Lists all SSTable files in the database

Data Format Inside LevelDB

Each entry in the database contains: - Key: Usually a UUID or unique identifier (string) - Value: JSON object with the Foundry document data

Example value structure for a spell: 

{
  "_id": "unique-id-here",
  "name": "Fireball",
  "type": "spell",
  "system": {
    "level": 3,
    "school": "evocation",
    "components": {
      "verbal": true,
      "somatic": true,
      "material": true
    },
    "description": {
      "value": "<p>Spell description HTML...</p>"
    }
  },
  "flags": {},
  "img": "icons/magic/fire/explosion.webp"
}

Methods to Read LevelDB

Option 1: Python with plyvel

import plyvel
import json

# Open the database
db = plyvel.DB('/path/to/packs/spells/')

# Iterate through all entries
for key, value in db:
    data = json.loads(value)
    print(f"ID: {key.decode()}")
    print(f"Name: {data.get('name', 'Unknown')}")

# Get specific entry
specific = db.get(b'some-id-here')
if specific:
    data = json.loads(specific)

db.close()

Option 2: Node.js with level

const level = require('level');
const db = level('/path/to/packs/spells/');

// Read all entries
db.createReadStream()
  .on('data', function (data) {
    const key = data.key.toString();
    const value = JSON.parse(data.value);
    console.log(`${key}: ${value.name}`);
  });

// Get specific entry
db.get('some-id-here', function (err, value) {
  if (!err) {
    const data = JSON.parse(value);
    console.log(data);
  }
});

Option 3: Command-line with ldbtools

# Install (varies by system)
apt-get install ldbtools  # Debian/Ubuntu
brew install leveldb      # macOS

# Dump all keys
ldbdump /path/to/packs/spells/

# Export to readable format
ldbdump --hex /path/to/packs/spells/ > spells.txt

Option 4: Using Foundry's API (when server is running)

// In Foundry's console or a macro
const pack = game.packs.get("dnd-players-handbook.spells");
const documents = await pack.getDocuments();
documents.forEach(doc => {
  console.log(doc.name, doc.system);
});

1. Python Script for Extraction

Create a Python script that can: - Open any Foundry LevelDB pack - Extract all entries as JSON - Save to structured JSON files for easy reference - Support searching and filtering

2. Installation Requirements

# Install Python LevelDB binding
pip install plyvel

# Note: Requires LevelDB library
# Ubuntu/Debian: apt-get install libleveldb-dev
# macOS: brew install leveldb
# Arch: pacman -S leveldb

3. Basic Extraction Script Structure

#!/usr/bin/env python3
import plyvel
import json
import sys
from pathlib import Path

def extract_pack(pack_path, output_file):
    """Extract all entries from a Foundry pack"""
    db = plyvel.DB(str(pack_path))
    entries = []

    for key, value in db:
        try:
            data = json.loads(value)
            entries.append(data)
        except json.JSONDecodeError:
            print(f"Failed to parse: {key}")

    db.close()

    with open(output_file, 'w') as f:
        json.dump(entries, f, indent=2)

    return len(entries)

# Usage
if __name__ == "__main__":
    pack_path = Path("/mnt/data/foundry/Data/modules/dnd-players-handbook/packs/spells")
    output = Path("~/docs/dnd/foundry-data/extracted/phb-spells.json")
    count = extract_pack(pack_path, output)
    print(f"Extracted {count} entries")

Common Document Types

Item Documents (spells, equipment, feats, classes)

  • Has type field indicating subtype
  • Contains system object with game mechanics
  • Includes description.value with HTML content

Actor Documents (characters, NPCs)

  • Contains ability scores, HP, AC
  • Lists items, spells, features
  • Includes token configuration

JournalEntry Documents (rules, lore)

  • Contains pages array with content
  • Supports different page types (text, image, video)
  • May have nested table of contents

RollTable Documents

  • Contains formula for rolling
  • Lists possible results with ranges
  • May reference other documents

Challenges and Considerations

  1. Binary Format: Direct file reading won't work without LevelDB library
  2. Concurrent Access: Don't read while Foundry is writing
  3. Large Databases: Some packs may have hundreds of entries
  4. HTML Content: Descriptions often contain HTML that needs parsing
  5. Cross-references: Documents may reference others by ID
  6. Updates: Module updates may change database structure

Next Steps

  1. Install plyvel or another LevelDB library
  2. Create extraction scripts for each pack type
  3. Build a search/reference system
  4. Consider caching extracted data for performance