Initialize MongoDB from a Declarative Configuration

Overview

While writing a Node.js application that uses MongoDB, I realized that I was maintaining a custom script for initializing my baseline database schema and content. I thought to myself, “Wouldn’t it be nice to simply edit a configuration, rather than continually updating a script that is doing the same things over and over, by copying, pasting, and tweaking the additional code?” From there, I searched NPM, and found no good options that would do this. I thought it would be something useful to build, not only for me, but also for others working with these two technologies together. Hence, the module mongodb-igniter was born.

I have built and published the module, and included the README content below for your reading pleasure. I hope it is helpful for some of my fellow programmers out there.

GitHub Respository: https://github.com/obsessiveprogrammer/mongodb-igniter

mongodb-igniter

A declarative MongoDB initializer

NPM

Join the chat at https://gitter.im/mongodb-igniter/Lobby

Description

The purpose of mongodb-igniter is to provide an easy way to initialize your application’s baseline MongoDB database content from a declarative object format using Node.js. It maximizes the use of Node.js’s async capabilities to process the initialization very quickly, and returns all results in a neatly organized array of objects. Both the DB declaration and results object formats leverage the MongoDB native driver object types, for simplicity. It can initiate its own MongoDB connection, or use an existing one. The module source is written in TypeScript and exposes all relevant types for consumption by IDEs and/or the TypeScript language.

Features

  • Create a MongoDB database connection or use an existing one
  • Create collections with any MongoDB supported options
  • Create/update indexes with any MongoDB supported options
  • Create/replace documents with any MongoDB supported options (uses replaceOne)

Compatibility

The current version uses the Node.js MongoDB native driver version 2.2, and therefore supports up to MongoDB 3.4. An update to the version 3.0 driver and support for MongoDB 3.6 is forthcoming.

For more information on Node.js MongoDB native driver compatibility, see: https://docs.mongodb.com/ecosystem/drivers/driver-compatibility-reference/#node-js-driver-compatibility

Installation

These instructions assume you have already installed Node.js, which includes NPM.

To install the module, use the following command:

npm install mongodb-igniter

or, if you have created a project with a package.json file:

npm install mongodb-igniter --save

Basic Usage

Create a Database Declaration

First create a configuration object for your database, collections, indexes, and documents using the following property specifications (click the links for more detailed information on the types):

DbDeclaration: The overall declaration for the database configuration

Property Type Description
db string or Promise<Db> MongoDB connection URI or Promise which resolves to MongoDB Db type. When a connection URI is used, mongo-igniter will open the connection, and close it when finished. If you use a pre-existing Promise<Db>, it will not close the connection, so be sure to close the connection in your own code.
options MongoClientOptions Optional: Options for the MongoClient.connect() method
collections CollectionDeclaration[] This is an array of a custom type in this module. Details are in the next table.

CollectionDeclaration: A declaration of a collection to create within the database

Property Type Description
name string The name of the collection
options CollectionCreateOptions Optional: Options for the Db.createCollection() method
indexes IndexDeclaration[] Optional: This is an array of a custom type in this module. Details are in the next table.
documents DocumentDeclaration[] Optional: This is an array of a custom type in this module. Details are two tables down.

IndexDeclaration: A declaration of an index to create within a collection

Property Type Description
keys string or Object This can be the name of a field to create an ascending index on with default options, or an object which describes the index. See the fieldOrSpec parameter of the Collection.createIndex() method for more information on the object format
options IndexOptions Optional: Options for the Collection.createIndex() method

DocumentDeclaration: A declaration of a document to create or replace within a collection. mongo-igniter uses the Node.js MongoDB native driver’s Collection.replaceOne() method for all declared documents.

Property Type Description
filter Object An object which describes the filter used to locate the document to replace, if it exists. See the filter parameter of the Collection.replaceOne() method for more information on the object format
data Object The document data to insert or replace an existing document with
options ReplaceOneOptions Optional: Options for the Collection.replaceOne() method

Example

The following is an example of a Database Declaration:

const dbDeclaration = {
  db: 'mongodb://localhost:27017/testdb', // Remember, you can also use an existing connection of type Promise<Db> here
  options: { appname: 'mongodb-igniter' },
  collections: [
    {
      name: 'clients',
      indexes: [
        {
          keys: { clientIdHash: 1 },
          options: {
            unique: true,
            background: true
          }
        }
      ],
      documents: [
        {
          filter: { name: 'Test Client for Mobile' },
          data: {
            name: "Test Client for Mobile",
            clientIdHash: "rdZBlx68P07di+3XZ8hMROh+GrcN9ccO2W0+cZa39jI=",
            clientSecretHash: "$2a$10$8MTa8EaiqsxBSCqdgIv0O.g1jOEyAGm5RmZzEaPB3DxY0/wcZUhzq",
            trustedClient: true
          },
          options: {
            upsert: true
          }
        },
        {
          filter: { name: 'Test Client for Web' },
          data: {
            name: "Test Client for Web",
            clientIdHash: "jqcQMI4QllIFRyTDuirdu3TVQ2r6rjXR4gfUlsYtCG4=",
            clientSecretHash: "$2a$10$gVDmGvOBrOjk.TF6z1TkkuljtkQNH6Ktxs8/mN9qhce3J8vVEWcum",
            redirectUri: "http://localhost:3000/login",
            trustedClient: true
          },
          options: {
            upsert: true
          }
        }
      ]
    },
    {
      name: 'tokens',
      indexes: [
        {
          keys: { userId: 1, clientIdHash: 1 },
          options: {
            name: 'userId_1_clientIdHash_1',
            unique: true,
            background: true
          }
        },
        {
          keys: { expirationDate: 1 },
          options: {
            name: 'expirationDate_1',
            expireAfterSeconds: 31536000, // 1 year in seconds
            background: true
          }
        }
      ]
    },
    {
      name: 'sessions',
      indexes: [
        {
          keys: { expires: 1 },
          options: {
            expireAfterSeconds: 0,
            background: true
          }
        }
      ]
    },
    {
      name: 'users',
      indexes: [
        {
          keys: { email: 1 },
          options: {
            unique: true,
            background: true
          }
        }
      ]
    },
    {
      name: 'logs',
      options: {
        capped: true,
        size: 1048576 // Capped at 1MB in bytes
      }
    }
  ]
};
Initialize the Database

Once you have the database configuration declared, you can use it to initialize the database. This module was created “callback-hell”-free, so the initialization function returns a Promise.

Examples

Using Promise.then()-style syntax, you could do the following:

const MongoDbIgniter = require('mongodb-igniter');

MongoDbIgniter.initializeDb(dbDeclaration)
  .then(result => console.log('MongoDB initialization complete'))
  .catch(err => console.log(`MongoDB initialization failed.  Error: ${err.message}`));

or using async/await:

const MongoDbIgniter = require('mongodb-igniter');

async function igniteMongoDB() {
  try {
    await MongoDbIgniter.initializeDb(dbDeclaration);
    console.log('MongoDB initialization complete');
  } catch (err) {
    console.log(`MongoDB initialization failed.  Error: ${err.message}`);
  }
}

igniteMongoDB();

This is all you need for basic usage. Depending on your use case, you may want to log or analyze the results of the initialization operations. See the next section for more information on this.

Results

mongodb-igniter returns results for all operations it performs in a neatly organized array of objects. The result is an array of collection initialization results (CollectionInitializationResult[]), as described below:

CollectionInitializationResult: The result of operations on a single initialized collection

Property Type Description
collectionName string The name of the initialized collection
indexNames string[] This property will only exist if there are indexes initialized. It is an array of the initialized index names.
documentInitializationResults ReplaceWriteOpResult[] This property will only exist if there are documents initialized. It is an array of ReplaceWriteOpResult, which is the return object from the Collection.replaceOne() method.

For a more detailed example of module usage and processing these result objects, see the TypeScript test folder or the JavaScript test folder.

Author: Tom Bonanno

Tom Bonanno is a Staff Systems Engineer at VMware, specializing in vRealize Automation and vRealize Orchestrator. He currently helps large enterprise customers understand the value of VMware products by demonstrating and proving their technical abilities on a deep level. He often designs and builds creative solutions to customers' problems, and shares them with the world via blog articles and other community sites and tools. In fact, his deep-seated passion is software development, including building custom extensions and integrations for the IT world. Tom attended Fairleigh Dickinson University, Florham Park campus, for mathematics and computer science. His 17 years of experience includes system administration, IT management, software implementation consulting, and technical software pre-sales engineering. On his own time, he invests in himself and his passion by learning new programming languages and developing applications and new functionality for existing software. Tom has also been known to do some Android modding and theming on the side. He continues to strive to keep up on the latest trends in technology, as both a provider and a consumer.