arrow-left

All pages
gitbookPowered by GitBook
1 of 1

Loading...

Directory

Storing a collection in multiple files within a directory

hashtag
Introduction

This page describes how to set up persistent file storage for a collection using Nabu, where each document is stored within a file under a specific directory on the file system.

Directory storage requires the Nabu storage engine

hashtag
Usage

hashtag
Statically created

To configure

hashtag
Dynamically created

The collection stored in 'content/myCollection' where each document will reside in a yaml file that derives its name from the _id of the document.


hashtag
Reuse across database

By defining a custom I/O rule for the storage engine, we can reuse the same configuration across multiple collections.

Alternatively we can set the default I/O rule and skip the storageEngine directive entirely.


hashtag
Parameters

hashtag
Path

path: string

Path to the directory where files are stored

hashtag
Extension

extension: string

File extension (including the dot)

hashtag
Format

format: string | Document

File format. The current valid file formats include:

  • format: 'json'

  • format: 'yaml'

If we want to access YAML front matter, format also accepts the following configuration.

hashtag
Merge Stat

mergeStat?: Document

Include file information for each document when it's loaded from the file system. Expressions in merge are computed against the underlying file information (lstat). Consider the following example for how to include the path of the file.

The following fields are available for merging:

path, dev, mode, nlink, uid, gid, rdev, blksize, ino, size, blocks, atimeMs, mtimeMs, ctimeMs, birthtimeMs, atime, mtime, ctime, birthtime

Any fields defined in the merge will be pruned before writing them to disk

hashtag
Construct

construct?: Document

Add additional fields to the document. This stage is performed after merge is completed. Expressions in construct are computed against the actual, loaded document.

Consider the following example where we read markdown files with YAML front matter. The markdown content will be stored under a field named markdown and we use construct to add an additional field html that contains the converted output.

Any fields defined in the construct stage will be pruned before writing them to disk

hashtag
Default

default?: Document

Set default values for fields that are not present in the stored content.

collections:
  myCollection:
    storageEngine:
      directory:
        path: ./content/myCollection
        extension: .yaml
        format: yaml
Tashmet.connect(store.proxy()).then(async tashmet => {
  const collection = await tashmet.db('myDb').createCollection('myCollection', {
    storageEngine: {
      directory: {
        path: 'content/myCollection',
        extension: '.yaml',
        format: 'yaml'
      }
    }
  });
});
const store = Nabu
  .configure({})
  .use(mingo())
  .io('yaml', (ns, options) => ({
    directory: {
      path: `${ns.db}/${ns.collection}`,
      extension: '.yaml',
      format: 'yaml'
    }
  }))
  .bootstrap();

Tashmet.connect(store.proxy()).then(async tashmet => {
  const collection = await tashmet.db('myDb').createCollection('myCollection', {
    storageEngine: 'yaml'
  });
});
const store = Nabu
  .configure({
    defaultIO: 'yaml'
  })
  // ...

Tashmet.connect(store.proxy()).then(async tashmet => {
  const collection = await tashmet.db('myDb').createCollection('myCollection');
});
{
  // ...
  extension: '.yaml'
}
// YAML with front matter
{
  // ...
  format: {
    yaml: {
      frontMatter: true,
      contentKey: 'content' 
    }
  }
}
// Include file path
{
  // ...
  mergeStat: {
    path: '$path'
  }
}
{
  path: '/content',
  extension: '.md',
  format: {
    yaml: {
      frontMatter: true,
      contentKey: 'markdown'
    }
  },
  construct: {
    html: {
      $markdownToHtml: '$markdown'
    }
  }
}
{
  path: '/content/posts',
  extension: '.md',
  format: {
    yaml: {
      frontMatter: true,
      contentKey: 'markdown'
    }
  },
  default: {
    slug: '$_id'
  }
}