Skip to content

configuration.md

Latest commit

 

History

History
638 lines (464 loc) · 27.1 KB

configuration.md

File metadata and controls

638 lines (464 loc) · 27.1 KB

CLI Configuration guide

Important

🚀 Respect copyrights and support content creators; use only the sources you own.

This guide introduces you to configuring data transformations using the Command Line Interface (CLI). Whether you're new to command line tools or an experienced user, you'll find helpful information on utilizing configuration files to tailor your experience.

Table of Contents

Overview

The CLI can be set up using JSON or YAML files. These files allow you to specify your preferences and can be used alongside or in place of command-line options. For examples of configuration file structures in both formats, see examples/config.

Note

📝 JSON and YAML are both file formats for storing data in useful and human-readable ways.

  • JSON: If you want to know why the {} and [] are used in the ways that they are you can read about json objects and arrays here).
  • YAML: A format where indentation (spaces at the beginning of lines) is important. Learn about YAML's specification.

The following examples are presented in JSON.

Important

Check your work! If you are getting errors reading your config file, don't rely on your eyes, use a linter!

Basic configuration example

Below is a straightforward config.json file. In this format, settings are noted in a "key": "value" structure.

{
    "sources": {
        "book": [
            "DMG",
            "PHB",
            "MM"
        ]
    },
    "paths": {
        "compendium": "z_compendium/",
        "rules": "z_compendium/rules"
    }
}

This example performs two basic functions:

  1. Select Input Sources: The sources key lists the sources to be included, identified by their source identifiers.
  2. Define Vault Paths: The paths key sets the destination paths for the compendium and rules content. These paths are relative to the output directory set in the CLI command with -o.

Warning

Windows Users: Replace any \ with / in paths in JSON or YAML files

Advanced configuration example

Here's a more comprehensive config.json file.

{
    "sources": {
        "adventure": [
            "LMoP",
            "LoX"
        ],
        "book": [
            "PHB"
        ],
        "reference": [
            "AI",
            "DMG",
            "TCE",
            "ESK",
            "DIP",
            "XGE",
            "FTD",
            "MM",
            "MTF",
            "VGM"
        ],
        "homebrew": [
            "homebrew/creature/MCDM Productions; Flee, Mortals!.json"
        ]
    },
    "paths": {
        "rules": "/compendium/rules/"
    },
    "excludePattern": [
        "race\\|.*\\|dmg"
    ],
    "exclude": [
        "monster|expert|dc",
        "monster|expert|sdw",
        "monster|expert|slw"
    ],
    "include": [
        "race|changeling|mpmm"
    ],
    "reprintBehavior": "newest",
    "useDiceRoller": true,
    "tagPrefix": "ttrpg-cli",
    "template": {
        "background": "examples/templates/tools5e/images-background2md.txt",
        "monster": "examples/templates/tools5e/monster2md-scores.txt"
    }
}

Additional capabilities:

  1. Select input sources: The sources key is used to select included sources (full text from two adventures and a book, reference content from a slew of other official sources, and one homebrew source).
  2. Define Vault Paths: The path sets the vault path destination for rules content.
  3. Targeted exclusion: excludePattern and exclude leaves out specific content.
  4. Targeted inclusion: The include specifies content that is always included.
  5. Reprint behavior: Only the latest/newest version of a resource should be emitted (this is the default).
  6. Use the dice roller plugin: The useDiceRoller key enables the dice roller plugin.
  7. Tag prefix: The tagPrefix key sets the prefix for tags generated by the CLI.
  8. Templates: The template key specifies the templates to use for different types of content.

Warning

Windows Users: Replace any \ in your paths with '/' in your JSON and YAML files.

Source identifiers

🚀 Remember: Only include content you legally own.

Sources in 5eTools and Pf2eTools are referenced by unique identifiers. Find the identifiers for your sources in the Source Map.

Content is classified as a book or adventure (shown as the third column in the source map). Use this classification when specifying your sources.

Some sources are split into multiple files, in which case, you will need to specify each identifier separately. For example, Tales from the Yawning Portal is split into seven files. Content appears using any one of the seven (TftYP-*), in addition to TftYP for common content. If you want to include all of them, you will need to specify each identifier separately.

If you're expecting to see content from a book or adventure and it's not showing up, run the CLI with the --index option, and check the all-index.json file to see which source identifier you should be using.

Specify content with sources

🔥 Version 3.x or SNAPSHOT ONLY. If you're using a 2.x version of the CLI, use the legacy version

The CLI can emit content from a source in two ways:

  • "full text": notes for all content and reference data from your sources. When including the full text, use the book, adventure, or homebrew key as appropriate for the source material.
  • "reference only": only emit reference notes (spells, classes, etc.). Use the reference key to include reference content from books or adventures.

With that in mind, specify your sources in this way:

"sources": {
    "adventure": [
        "WBtW"
    ],
    "book": [
        "PHB"
    ],
    "reference": [
        "MPMM"
    ]
}

The above example that will include full text for the Player's Handbook (a book, PHB) and The Wild Beyond the Witchlight (an adventure, WBtW), but will only create reference notes (backgrounds, cults/boons, races, traps/hazards) from Mordenkainen Presents: Monsters of the Multiverse (MPMM).

Tip

You only need to list your source once.

Homebrew

Tip

🍺 You only need the particular file you wish to import.

Homebrew data is different from the 5etools or Pf2eTools data. Each homebrew file is a complete reference. If you compare it to cooking: the 5etools and Pf2eTools mirror repositories are organized by ingredient (all of the carrots, all of the onions, ... ); homebrew data is organized by prepared meal / complete receipe.

Support your content creators! Only use homebrew that you own.

To include Homebrew in your notes, specify the path to the homebrew json file in a homebrew section inside of sources.

For example, if you wanted to use Benjamin Huffman's popular homebrewed Pugilist class:

  1. Download a copy of the Pugilist json file.

    Save this file to a well-known location on your computer. It is probably easiest if it sits next your 5eTools or Pf2eTools directory.

  2. Add the path to this file to a homebrew section under sources:

    {
  "sources": {
    "homebrew": [
        "path/to/Benjamin Huffman; Pugilist.json"
    ]
  }
}

In the above example, path/to/ is a placeholder. If you use a relative path, it will be resolved relative to the current working directory1. An absolute path2 will also work.

There are a few ways to figure out the path to a file:

  • You may be able to drag and drop the file into the terminal window.
  • You may have the ability to right-click on the file and select "Copy Path".

Warning

Windows Users: Replace any \ with / in paths in JSON or YAML files

Additional notes about homebrew

Homebrew json files are not rigorously validated. There may be errors when importing. I've done what I can to make the errors clear, or to highlight the suspect json, but I can't catch everything.

Here are some examples of what you may see, and how to fix them:

  • Unable to find image 'img/bestiary/MM/Green Hag.jpg' (or similar)

    This kind of path refers to an "internal" (meaning part of the base 5e corpus of stuff) image. These paths are computed relative to a known base.

    Recent releases of 5eTools use a different repository structure, see Copying "internal" images), and images have, by and large, been converted from .jpg or .png to .webp. Fixing this kind of error is usually a case of fixing the path.

  • Unknown spell school Curse in sources[spell|ventus|wandsnwizards]; similar for item types, item properties, conditions, skills, abilities, etc.

    This kind of error could be caused by a missing companion file (check dependencies listed in the meta information at the top of the homebrew file) or a missing definition.

    • If you find the missing definition (or it is already present in the homebrew file), and the error persists, please ask for CLI help first so I can make sure there isn't a bug in the CLI preventing it from being read.
    • If the data really is missing from the homebrew json, please report it in 5eTools #brew-conversion.

  • You may see messages about missing fields or badly formed tables.

Reporting content errors to 5eTools

Note

The lovely folks at 5eTools don't understand the CLI, and they don't need to. If you report an issue, keep the details focused on the JSON content (typo, missed definition, etc.). If you aren't sure, ask for CLI help first.

If you can fix the error by fixing the json content, please report the error in the 5eTools discord channel.

  1. Identify the homebrew source that contains the error in the homebrew manager.
  2. Click the "View Converters" button on the right to find the converter(s).
  3. Tag them in the #⁠brew-conversion channel, describing what you fixed. If you can't find the right user to tag, or if no one is listed as a converter, post the report anyway but make sure you mention this!

Use the following form for your report.

**Brew:**   Write the name of the homebrew source here  
**Converter:**   @converter's-discord-ID   
**Issue:**   Describe the issue here in clear, concise terms (e.g. "the Red-Spotted Gurgler is listed as having AC 15, when it should be 16")  
**Steps for reproduction:**   If you have to do something specific to make the error appear, describe them here. 
> I go to X and click on Y 
> I expected Z, but instead ...

For that last part, you may need to do some digging. Do not report the error using CLI exception messages. Stick to the observed missing links or errors in the data.

Specify target vault paths (paths)

The paths key specifies vault path for generated content.

  • New directories are made if they aren't already present.
  • Paths are relative to the CLI's designated output location (-o), which correlates to the root of your Obsidian vault.

Example:

  "paths": {
    "compendium": "/compendium/",
    "rules": "/rules/"
  }

Tip

The leading slash is optional. It marks a path starting from the root of your Obsidian vault.

5eTools and Pf2eTools content is organized differently, but in general, information is organized as follows:

  • compendium: backgrounds, classes, items, spells, monsters, etc.
  • rules: conditions, weapon properties, variant rules, etc.

Warning

Do not reorganize or edit the generated content. Tuck generated content away in your vault and use it as read-only reference material. It should be cheap and easy to re-run the tool (add more content, errata, etc.). See Recommendations for more information.

Refine content choices

You can use the following configuration to exclude or include specific data.

Just as source material has an identifier, so does each piece of data. The Monster Manual has the identifier MM. Each monster in the Monster Manual has its own key, such as monster|black dragon wyrmling|mm or item|drow +1 armor|mm.

The CLI --index option compiles two lists of data keys:

  • all-index.json: Lists all discovered data keys.
  • src-index.json: Lists the data keys after source filters (adventure, book, reference, and the config options below) have been applied.

Note

Emitted types: You can check some common emitted types to include or exclude here.

Excluding content matching an excludePattern

This option allows you to exclude data entries based on regular expression matching patterns.

Note: A pipe (|) is a special character in regular expressions, and must be escaped.

  • JSON

    "excludePattern": [
    "race\\|.*\\|dmg"
]

  • YAML

    excludePattern:
  - race\|.*\|dmg

Excluding specific content with exclude

Specify the data keys you want to omit.

"exclude": [
    "monster|expert|dc",
    ...
]

Including specific content with include

Specify the data keys you want to include.

"include": [
    "race|changeling|mpmm"
]

This approach is ideal for content acquired in parts, like individual items from D&D Beyond.

Reprint behavior

🔥 Version 3.x or SNAPSHOT ONLY.

Content is often reprinted or updated in later sources or editions. This setting lets you control how reprinted or revised content is handled when generating notes.

  "reprintBehavior": "newest"

This setting has 3 possible values:

  • newest (default): Only includes notes for the most recent version of reprinted content.

  • edition: Focuses on preserving content across incompatible editions (especially for 5e rules).

    Example: The edition check will preserve 2014 edition-specific class and subclass definitions. Other resources (that are not different across editions) will follow the reprints to include new content.

  • all: Includes notes for all reprinted versions from enabled sources

In most cases, you will get the most recent version of the resource that is included, as most resources do not have substantial changes across editions.

For example, trap|pits|dmg is reprinted as trap|hidden pit|xdmg. If both versions are included by your configuration, you will only get a note for the XDMG version unless reprintBehavior is set to all or you have an explicit include rule that preserves the DMG version (in which case, you'll get both).

Troubleshooting reprint behavior

If the behavior isn’t what you expect, run with the --log option and check the log file. The log will show whether a specific key was kept or dropped and explain why.

To ensure a specific resource is included, add its key to the include filter instead of relying on reprint behavior.

Races as species

If you prefer the term "species" over "race" (as used in newer D&D editions), set racesAsSpecies to true.

  "racesAsSpecies": true

This changes the output directory from races/ to species/, tags from race/... to species/..., and the CSS class from json5e-race to json5e-species. It does not affect internal indexing or source data processing.

Only emit referenced tables

By default, the CLI emits all table notes from included sources. Set onlyReferencedTables to true to restrict output to tables that are actually linked from other included content (monsters, spells, adventures, etc.).

  "onlyReferencedTables": true

When enabled, a table note is written only if:

  • something in the rendered output links to it (via a {@table} tag or inline table matching), or
  • it is explicitly named in an include filter.

Unreferenced tables are logged as (drop | unreferenced) when run with --log.

Use the dice roller plugin

The CLI can generate notes that include inline dice rolls. To enable this feature, set the useDiceRoller attribute to true.

Render with Fantasy Statblocks

If you are using the Fantasy Statblocks plugin to render your statblocks, set yamlStatblocks to true. This will remove backticks and other formatting from statblock text.

Tag prefix

The tagPrefix key sets the prefix for tags generated by the CLI. This is useful if you want to distinguish between tags generated by the CLI and tags you've created yourself.

For example, the CLI generates tags like compendium/src/phb and spell/level/1. If you set tagPrefix to 5e-cli, the tags will be 5e-cli/compendium/src/phb and 5e-cli/spell/level/1.

Templates

The CLI uses the Qute Templating Engine to render markdown output. Use the template attribute in your configuration file to specify the templates you want to use for different types of content.

  "template": {
    "background": "examples/templates/tools5e/images-background2md.txt",
    "monster": "examples/templates/tools5e/monster2md-scores.txt"
  }

[!TIP] The key used to specify a template corresponds to the type of template being used. You can find the list of valid template keys in the source code (look for templateKeys).

  • Valid templates for 5etools: background, class, deck, deity, feat, hazard, index.txt, item, monster, note, object, psionic, race, reward, spell, subclass, vehicle.
  • Valid templates for Pf2eTools: ability, action, affliction, archetype, background, book, deity, feat, hazard, inline-ability, inline-affliction, inline-attack, item, note, ritual, spell, trait.

Customizing templates

Documentation is generated for template attributes.

Not everything is customizable. Some indenting, organizing, formatting, and linking is easier to do consistently while rendering big blobs of text.

See the examples templates for reference.

Images

The CLI can copy images referenced in the content to your vault. This is useful if you want to use the content offline or if you want to ensure that images are available in your vault.

  • Internal images are part of the 5eTools or Pf2e tools corpus of content. They are referenced by computed path (like tokens) or by media references marked as "internal".
  • "External" images are usually marked in the Json source as "external" and are referenced by a URL.

Copying internal images

5eTools internal images are stored in a separate repository.

By default, images are not downloaded. Links reference the remote location, requiring internet access to view. This is the safest, fastest option.

To copy internal images into your vault:

  • Option 1: Download on demand

    "images": {
    "copyInternal": true
}

    The CLI downloads each image individually as needed. This is many separate requests and may be slow.

  • Option 2: Use a local clone (faster)

    "images": {
    "copyInternal": true,
    "internalRoot": "5etools-img"
}

    The CLI reads images from a local directory. Relative paths resolve from the current working directory1; absolute paths2 also work. You'll get an error if the directory doesn't exist.

Note

If not copying images, delete both copyInternal and internalRoot attributes.

If using internalRoot, ensure the path exists. http or file URLs work best; otherwise the CLI creates a file link from the specified path.

Without copying, images must be accessible from your vault to display in Obsidian.

Copying external images

External images are marked as "external" in the JSON source and typically use http://, https://, or file://3 URLs.

By default, external images are not downloaded. Links reference the remote location, requiring internet access to view. This is the safest, fastest option.

To copy external images into your vault:

"images": {
    "copyExternal": true
}

The CLI will download each external image it hasn't seen before.

Fallback paths

🧪 Experimental feature. Report issues if problems occur.

If an image reference is broken or fails to copy, you can specify a fallback path:

"images": {
    "fallbackPaths": {
        "img/bestiary/MM/Green Hag.jpg": "img/bestiary/MM/Green Hag.webp"
    }
}
  • Key (original path): Must match exactly what the JSON source specifies. For external/homebrew images, check the JSON source. Internal images may be harder to identify.
  • Value (replacement path): Must be a valid local file path2 or URL3.

Customizing the default source

Warning

🔥 You can truly make a mess with this setting. Change these values with care, and inspect the result carefully in a test vault.

  • This will change generated file names. It will break links.
  • If you have content generated with a different defaultSource configuration, completely remove it before copying freshly generated content into your vault ^[You could also use something like rsync that will remove extraneous files].

If you're only running 5e 2024 content (as an example), you may want to change the "default" source for items, etc. to be XPHB or XDMG or XMM, such that those files do not have the additional notations (e.g. the file name suffix, or the additional source designation in the monster name).

Change the default source for an index type in the sources block:

"sources": {
    "defaultSource": {
        "monster": "XMM"
    }
}

Create a map of a content type to the default source. In general, this is the same key you would use to assign a template. If you open up the generated index, the first segment of a key is its type, for example, trap|collapsing roof|dmg is a trap. Some types are grouped because of tight inter-relationships, like cards and decks.

Emitted type Default Source Includes (note)
background PHB
classtype PHB subclass, class feature, subclass feature
deck DMG card
deity PHB
disease DMG
facility XDMG (bastion)
feat PHB
item DMG item group, magic variant
monster MM legendary group (bestiary)
object DMG
optfeature PHB
psionic UATheMysticClass
race PHB subrace (species)
reward DMG
spell PHB
table DMG table group
trap DMG hazard
variantrule DMG
vehicle GoS

Migrating from, full-source, and convert

Older configurations looked a little different. Updating to the new format should be straightforward.

For comparison, the following examples of older configurations use the same values as the example above.

"from": [
    "MPMM"
],
"full-source": {
    "adventure": [
        "WBtW"
    ],
    "book": [
        "PHB"
    ],
    "homebrew": [
        ...
    ]
}

OR

"from": [
    "MPMM"
],
"convert": {
    "adventure": [
        "WBtW"
    ],
    "book": [
        "PHB"
    ],
    "homebrew": [
        ...
    ]
}

Footnotes

  1. The working directory is the directory you were in (in the terminal) when you launched the CLI. See https://en.wikipedia.org/wiki/Working_directory for more information ↩ ↩2 ↩3

  2. Example/explanation of absolute vs. relative path: https://stackoverflow.com/a/10288252. If you're using relative paths with the CLI, they should be relative to the working directory (see 1). ↩ ↩2 ↩3

  3. A URL is a uniform resource locator, more information at https://en.wikipedia.org/wiki/URL. ↩ ↩2