Database

Concepts

In KeystoneJS, your data schema and models are controlled by Lists, and documents in your database are often called Items.

To define a data model, you create a new keystone.List, and pass it list options.

You then add fields to the list. Behind the scenes, a Keystone List will create a mongoose schema, and add the appropriate paths to it for the fields you define.

The schema is accessible, allowing you to plug in other mongoose functionality like virtuals, methods and pre / post hooks.

When you have finished setting up your List, call list.register() to initialise it and register it with Keystone.

To query your data, you use the list.model (which is a mongoose model).

List Items are mongoose documents. To create new items, use new list.model() and when you're ready to save it (or to save changes to an existing Item), call item.save().

Lists

Usage

new keystone.List(key[, options]);

The syntax for creating a Keystone List is very similar to the syntax for creating a Mongoose Schema, with the exception of the constructor, which is var MyList = new keystone.List(key, options).

Once you have created a new List, add fields to it using MyList.add(fields), where fields is an object of keys (for field paths) and values (for field types, or options).

Fields are defined by an object with a type property, which must be a valid Field Type or basic data type. Using the object syntax you can specify additional options for the field. Common field options and field-type-specific options are detailed in the fields documentation.

When all the fields and options have been set on the list, call MyList.register() to register the list with Keystone and finalise its configuration.

Example

A simple Post model for a blog might look like this:

var keystone = require('keystone'),
    Types = keystone.Field.Types;
 
var Post = new keystone.List('Post', {
    autokey: { path: 'slug', from: 'title', unique: true },
    map: { name: 'title' },
    defaultSort: '-createdAt'
});
 
Post.add({
    title: { type: String, required: true },
    state: { type: Types.Select, options: 'draft, published, archived', default: 'draft' },
    author: { type: Types.Relationship, ref: 'User' },
    createdAt: { type: Date, default: Date.now },
    publishedAt: Date,
    image: { type: Types.CloudinaryImage },
    content: {
        brief: { type: Types.Html, wysiwyg: true, height: 150 },
        extended: { type: Types.Html, wysiwyg: true, height: 400 }
    }
});
 
Post.defaultColumns = 'title, state|20%, author, publishedAt|15%'
Post.register();

This example implements the optional map, autokey and defaultSort options, described below.

It also specifies title, state, author and publishedAt as the default columns to display in the Admin UI, with state and publishedAt being given column widths.

The author field is a relationship with the User model, as described in the getting started guide.

Options

Lists support the following options:

var item = new List.model();
item.set({ field1: 'value1', field2: 'value2' });
item._req_user = req.user;
item.save();

If you're wondering how to control which navigation area Lists are categorised under in the Admin UI, check out the nav option in the KeystoneJS Configuration docs.

Drilldown example

The drilldown option is a nice way to improve the usability of the Admin UI by providing context to the item a user is currently editing.

By default, the drilldown will just show the list that the item belongs to.

You can, however, set it to a Relationship field in the schema, and it will display the item currently stored in that relationship field.

If there would be several relationships that may be relevant to display in the drilldown list, you can separate their paths with spaces.

var Post = new keystone.List('Post', {
    autokey: { path: 'slug', from: 'title', unique: true },
    map: { name: 'title' },
    defaultSort: '-createdAt',
    drilldown: 'author' // author is defined as a Relationship field in the example above
});

Inheritance example

The inheritance option can be used to allow a list to inherit its fields from another list using Mongoose model discriminators

Parent lists may not themselves inherit from other lists.

var keystone = require('keystone');
 
var BasePage = new keystone.List('BasePage', {
   map: { name: 'title' },
	    autokey: { path: 'slug', from: 'title', unique: true },
	});
BasePage.add(
	{
		title: { type: String, required: true },
		slug: { type: String, readonly: true },
	}
);
BasePage.register();
 
var ChildPage = new keystone.List('ChildPage', { inherits: BasePage });
ChildPage.add({ child_content: { type: String, readonly: true } });
ChildPage.register();

Schema Plugins

You can specify virtuals, methods, statics as well as pre and post hooks for your Lists using the schema. You can also use mongoose plugins from the plugins website.

For example, in our Post list above, we might want to automatically set the publishedAt value when the state is changed to published (but only if it hasn't already been set).

We might also want to add a method to check whether the post is published, rather than checking the state field value directly.

Before calling Post.register(), we would add the following code:

Post.schema.methods.isPublished = function() {
    return this.state == 'published';
}
 
Post.schema.pre('save', function(next) {
    if (this.isModified('state') && this.isPublished() && !this.publishedAt) {
        this.publishedAt = new Date();
    }
    next();
});

Querying Data

To query data, you can use any of the mongoose query methods on the list.model.

For example: to load the last 5 posts with the state published, populating the linked author, sorted by reverse published date:

var keystone = require('keystone'),
    Post = keystone.list('Post');
 
Post.model.find()
    .where('state', 'published')
    .populate('author')
    .sort('-publishedAt')
    .limit(5)
    .exec(function(err, posts) {
        // do something with posts
    });

Promises

There exists another way to work with events in Javascript that is included in mongoose query methods. Instead of passing a callback to the exec method, we can use what it returns: a Promise. Promises are very useful for clean chaining of events with propagation of error.

For example: load 100 posts, then do something asynchronous, then do something with result:

var keystone = require('keystone'),
    Post = keystone.list('Post');
 
Post.model.find()
    .limit(100)
    .exec()
    .then(function (posts) { //first promise fulfilled
        //return another async promise
    }, function (err) { //first promise rejected
        throw err;
    }).then(function (result) { //second promise fulfilled
        //do something with final results
    }, function (err) { //something happened
        //catch the error, it can be thrown by any promise in the chain
        console.log(err);
    });

Pagination Querying

To query data with pagination, you can use List.paginate(),it returns a query object, just as List.model.find() would. It supports the options

• page - page to start at
• perPage - number of results to return per page
• maxPages - optional, causes the page calculation to omit pages from the beginning/middle/end(useful if you have lots of pages, and do not want them to wrap over several lines).

For example: to load the posts with the maxPages 10 and perPage 10,which state is published, populating the linked author and categories, sorted by reverse published date:

var keystone = require('keystone'),
    Post = keystone.list('Post');
 
 Post.paginate({
		page: req.query.page || 1,
		perPage: 10,
		maxPages: 10
	})
	.where('state', 'published')
	.sort('-publishedDate')
	.populate('author categories')
	.exec(function(err, results) {
		locals.data.posts = results;
		next(err);
	});

When you call exec on a paginated query, it will return a lot of metadata along with the results:

• total: all matching results (not just on this page)
• results: array of results for this page
• currentPage: the index of the current page
• totalPages: the total number of pages
• pages: array of pages to display
• previous: index of the previous page, false if at the first page
• next: index of the next page, false if at the last page
• first: the index of the first result included
• last: index of the last result included

Creating Items

To create new items, again use the mongoose model:

var keystone = require('keystone'),
    Post = keystone.list('Post');
 
var newPost = new Post.model({
    title: 'New Post'
});
 
if (shouldBePublished) {
    newPost.state = 'published';
}
 
newPost.save(function(err) {
    // post has been saved	
});

newPost.slug == 'new-post';

Deleting Items

To delete items, first load the data, then use the remove method:

var keystone = require('keystone'),
    Post = keystone.list('Post');
 
Post.model.findById(postId)
    .remove(function(err) {
        // post has been deleted
    });

Headings

Define headings to display within the flow of your documents. Headings can be defined as a String or Object and can depend on another field value for display.

Person.add(
	'User', 
	{ name: { type: Types.Name, required: true, index: true, initial: true } }, 
	'Permissions', 
	{ isAdmin: { type: Boolean, label: 'Can access Keystone', index: true } },
	// header object
	{ heading: 'Activities' }, 
	{ place: { type: Types.Select, options: ['GT', 'UGA'] } },
	// header with dependsOn
	{ heading: "GT Activities", dependsOn: { place: 'GT' } },
	{ type: { type: Types.Select, options: ['ZC', 'MP'], dependsOn: { place: 'GT'} }
);

Fields

When adding fields to Lists, you can either specify basic data types or Keystone Field Types.

Overview

Keystone Fields allow you to easily add rich, functional fields to your application's models. They are designed to describe not just the structure of your data, but also the intention of your data. They provide:

• Rich controls in Keystone's Admin UI
• Complex data types; e.g. the location field stores several strings and an GeoJSON lng/lat point
• Formatting and validation methods
• Additional virtual properties; e.g. the name field provides a name.full virtual which concatenates the stored name.first and name.last
• Underscore methods; e.g. the password field provides a password.compare method for testing against the encrypted hash
• Metadata about how fields relate to each other; e.g. which fields depend on certain values in other fields

Basic data types are mapped to their corresponding Keystone field types:

Field Options

All field types support several common options, which can specify database settings (such as index and default), or can provide information for Keystone's Admin UI (such as label).

Fields can be nested inside objects, as in mongoose schemas.

All mongoose schema type options are passed to the mongoose schema, so you can also use any options mongoose supports.

Common field options include:

Conditional Fields

To improve the usability of the Admin UI, it is possible to hide fields when no value is set, or depending on the value of other fields.

first: { type: String },
// Will show if first === "value1", "1" or "2"
second: { type: String, dependsOn: { first: ['value1', '1', 2] } },
// Will show if first == "value1"
third: { type: String, dependsOn: { first: 'value1' } }

Generated values and watching fields

Keystone's fields support a simple syntax for configuring dynamically updated fields. You can set a field to update its value whenever:

• The item is saved
• The value of any other field (or fields) changes
• The value of any other field (or fields) changes to a specific value

To use the watching functionaliy, set the following two options:

function () {
    return this.total<=this.totalreceived ? true:false;
}

function (callback) { // BEWARE: MUST be called "callback" to allow asynchronous execution
	list.model.findById(this.createdBy).exec(function(err, user){
		callback(err, user.name + "-" + Date.now());
	});
}

Underscore Methods

Some field types include helpful underscore methods, which are available on the item at the field's path preceded by an underscore.

For example: use the format underscore method of the createdAt DateTime field of the Posts List (above) like this

var keystone = require('keystone'),
    Post = keystone.list('Post');
 
Post.model.findById(postId).exec(function(err, post) {
   console.log(post._.createdAt.format('Do MMMM YYYY')); // 25th August 2013
});

Relationships

Keystone enhances MongoDB's ability to store the ObjectIDs of related documents in a field (or many related ObjectIDs in an Array) with support for Relationship fields and Definitions in Models.

Relationship Fields

ObjectId or Array — Displayed as an auto-suggest field in the Admin UI

Stores references to ObjectIDs from another Model in an ObjectID field or array to create one-many or many-many relationships.

Specify the related Model using the ref option. For a many-many relationship, set the many option to true.

For example, if you wanted to link a Post model to a single Author and many PostCategories, you would do it like this:

Post.add({
    author: { type: Types.Relationship, ref: 'User' },
    categories: { type: Types.Relationship, ref: 'PostCategory', many: true }
});

Relationship Filters

You can filter a relationship field using the filters option.

The filters option is an object of key/value pairs, in which the keys correspond to the fields of the related model to be filtered, and the values will either be literals or field names in the current model, the value of which will be used to filter the relationship.

In the example below, the author field will only allow selection of a User whose group field is equal to 'admin'.

Post.add({
    title: { type: String, required: true },
    category: { type: Types.Select, options: 'user, editor, admin', default: 'user' },
    author: { type: Types.Relationship, ref: 'User', filters: { group: 'admin' } }
});

You can also filter by the value of another field on the model. You do this setting the value of the filter to the name of the field, prefixed by a colon (:).

In the example below, the author field will only allow selection of a User whose group field is equal to the value of the category field of the Post model.

Post.add({
    title: { type: String, required: true },
    category: { type: Types.Select, options: 'user, editor, admin', default: 'user' },
    author: { type: Types.Relationship, ref: 'User', filters: { group: ':category' } }
});

Finally, you can also filter by the current model's _id field.

In the example below, the bestPost field will only allow selection of a Post whose author field is equal to the _id of the current document.

User.add({
    name: { type: String, required: true },
    group: { type: Types.Select, options: 'user, editor, admin', default: 'user' },
    bestPost: { type: Types.Relationship, ref: 'Post', filters: { author: ':_id' } }
});

You can only set filters on one-many relationships (i.e. when the many option is NOT set to true).

Populating related data in queries

You can populate related data for relationship fields thanks to Mongoose's populate functionality. To populate the author and category documents when loading a Post from the example above, you would do this:

Post.model.findOne().populate('author categories').exec(function(err, post) {
    // the author is a fully populated User document
    console.log(post.author.name);
});

Note that if no ObjectId is stored, or an invalid ObjectId is stored (e.g. a document has been deleted), author will be undefined in the example above.

Relationship Definitions

What if, in the example above, you wanted to see a list of the Posts by each Author? Because the relationship field is on the Post, you need to tell the Author (and the PostCategory) Model that it is being referred to. Doing so allows the Admin UI to represent the relationship from both sides.

You do this by calling the relationship method on the Model like this:

User.relationship({ path: 'posts', ref: 'Post', refPath: 'author' });

As you can see, the options provided to the relationship method mirror those of the relationship field it refers to.

Relationship definitions are optional; if you leave them out, the relationships simply won't be displayed in the Admin UI from the other side of the relationship. The relationship field will still work as expected.

Loading related items

Filtering one-to-many related items is easy; simply specify the ID of the item you wish to filter on like any other value:

Post.model.find().where('author', author.id).exec(function(err, posts) {
    // ...
});

To filter many-to-many related items, use an in condition and specify one (or more) ids as an array:

Post.model.find().where('categories').in([category.id]).exec(function(err, posts) {
    // ...
});

Field Types

Boolean

Boolean — Displayed as a checkbox in the Admin UI

{ type: Types.Boolean }

Code

String — Displayed with CodeMirror in the Admin UI.

{ type: Types.Code, height: 180, language: json }

Color

Color — Displayed as a text field with a color picker

{ type: Types.Color }

Date

Date — Displayed as a date picker in the Admin UI

Input should either be a valid Date, or a string in the format YYYY-MM-DD (can be blank unless field is required)

To default Date fields to the current time, set the default option to Date.now

{ type: Types.Date }

item.createdDate = Date.now();
item._.createdDate.format(); // returns today's date using the default format string
item._.createdDate.parse('2013-12-04'); // returns a moment object with the parsed date
item._.createdDate.format('YYYY-MM-DD'); // returns '2013-12-04'

Datetime

Datetime — Displayed as a date and time picker in the Admin UI

Input should either be a valid Date, or a string in the format YYYY-MM-DD (can be blank unless field is required)

To default Date fields to the current time, set the default option to Date.now

{ type: Types.Datetime, default: Date.now }

Email

String — Displayed as a text field in the Admin UI

Input must look like a valid email address (can be blank unless field is required)

{ type: Types.Email, displayGravatar: true }

item.email = "demo@keystonejs.com";
item._.email.gravatarUrl(); // "//www.gravatar.com/avatar/74a0071e5f3a7107b570b7d4a1a7619d?s=80&d=identicon&r=g"
item._.email.gravatarUrl(200,'mm','r'); // "//www.gravatar.com/avatar/74a0071e5f3a7107b570b7d4a1a7619d?s=200&d=mm&r=r"

Html

String — Displayed as a text field or WYSIWYG Editor in the Admin UI.

{ type: Types.Html, wysiwyg: true }

See the Admin UI Options for global configuration options that can be used to customise the WYSIWYG editor.

Key

String — Displayed as a text field in the Admin UI

Automatically converts input to a valid key (no spaces or special characters). White space is replaced with a separator.

{ type: Types.Key }

Location

Object — Displayed as a combination of fields in the Admin UI

Contains a standard set of strings for storing an address, and a longitude / latitude point with a 2dsphere index.

Also provides autocomplete functionality using Google's Places API (requires a Google Maps API Key to be provided, must only be used in accordance with Google's terms of service).

Google Places integration requires the google api key option to be set for Keystone. See the Google configuration documentation for details.

{ type: Types.Location, defaults: { country: 'Australia' } }

Note: the schema paths are based on Australian address formats, and should be updated to be more appropriate for other international formats. If you have feedback on how the structure should be internationalised, please open a ticket.

Markdown

Object — Displayed as a textarea field in the Admin UI

{ type: Types.Markdown }

{ type: Types.Markdown, height: 200 }

{ type: Types.Markdown, toolbarOptions: { hiddenButtons: 'H1,H6,Code' } }

Page.add({ content: Types.Markdown });
 
var page = new Page.model();
page.content.md = "# Hello World";
page.content.html == "<h1>Hello World</h1>";
 
// or...
 
Page.fields.content.updateItem(page, "* list item");
page.fields.content.format(page) == "<ul><li>list item</ul></li>";

Money

Number — Displayed as a number field in the Admin UI

Input should either be a valid Number, or a string that can be converted to a number (leading symbols are allowed; can be blank unless field is required).

{ type: Types.Money }

{ type: Types.Money, format: '$0,0.00' }

{ type: Types.Money, currency: 'en-gb' }

Name

Object — Displayed as firstname lastname fields in the Admin UI

{ type: Types.Name }

Number

Number — Displayed as a number field in the Admin UI

Input should either be a valid Number, or a string that can be converted to a number (can be blank unless field is required)

{ type: Types.Number }

Password

String — Displayed as a password field in the Admin UI, with a 'change' button.

Passwords are automatically encrypted with bcrypt, and expose a method to compare a string to the encrypted hash.

The encryption happens with a pre-save hook added to the schema, so passwords set will not be encrypted until an item has been saved to the database.

{ type: Types.Password }

Select

String or Number — Displayed as a select field in the Admin UI

Similar to Enum in other Frameworks.

{ type: Types.Select, options: 'first, second, third' }

{ type: Types.Select, numeric: true, options: [{ value: 1, label: 'One' }, { value: 2, label: 'Two' }] }

{ type: Types.Select, required: true, options: 'first, second', emptyOption: false }

{ type: Types.Select, options: ['first', 'second', { value: 'third', label: 'The third one' }] }

{ type: Types.Select, options: [
    { value: 'first', label: 'The first option', custom: 'value' },
    { value: 'second', label: 'Second' }
]}

MyList.add({ state: { type: Types.Select, options: 'draft, published, archived', default: 'draft' });
 
MyList.fields.state.values == 'draft,published,archived';
MyList.fields.state.labels == { draft: 'Draft', published: 'Published', archived: 'Archived' };
MyList.fields.state.ops == [
    { value: 'draft', label: 'Draft' },
    { value: 'published', label: 'Published' },
    { value: 'archived', label: 'Archived' }
];
MyList.fields.state.map == {
    draft: { value: 'draft', label: 'Draft' },
    published: { value: 'published', label: 'Published' },
    archived: { value: 'archived', label: 'Archived' }
};
 
var item = new MyList.model();
item.state == 'draft';
item.stateLabel == 'Draft';
item.stateData == { value: 'draft', label: 'Draft' };
item.stateOptions == MyList.fields.state.ops;
item.stateOptionsMap == MyList.fields.state.map;

Text

String — Displayed as a text field in the Admin UI

{ type: Types.Text }

Textarea

String — Displayed as a textarea field in the Admin UI

{ type: Types.Textarea }

Url

String — Displayed as a text field in the Admin UI.

{ type: Types.Url }

item.url = "https://keystonejs.com";
item._.url.format(); // "keystonejs.com"

Storage

AzureFile

Object — Displayed as an file upload field in the Admin UI

Automatically manages files stored in Windows Azure Storage, including uploading and deleting.

{ type: Types.AzureFile }

{ type: Types.AzureFile, filenameFormatter: function(item, filename) {
	return item._id + require('path').extname(filename);
} }

{ type: Types.AzureFile, containerFormatter: containerFormatter: function(item, filename) {
	return item.modelProperty;
} }

CloudinaryImage

Object — Displayed as an image upload field in the Admin UI

Automatically manages images stored in Cloudinary, including uploading, resizing and deleting.

See the Cloudinary configuration documentation for details on how to set up Cloudinary in KeystoneJS.

{ type: Types.CloudinaryImage }

{ type: Types.CloudinaryImage, publicID: 'slug' }

{ type: Types.CloudinaryImage, folder: 'path/to/image' }

{ type: Types.CloudinaryImage, autoCleanup : true }

{ type: Types.CloudinaryImage, select : true }

{ type: Types.CloudinaryImage, select: true, selectPrefix: 'path/to/images' }

Remember that if you are uploading images to a CloudinaryImage field using an HTML form, you need to specify enctype="multipart/form-data" in your form tag.

CloudinaryImages

Array — Displayed as a series of images, and an upload field in the Admin UI

Stores multiple images in a array as a nested Schema, each of which expose the same methods as the cloudinaryimage field.

{ type: Types.CloudinaryImages }

{ type: Types.CloudinaryImages, folder: 'path/to/image' }

Embedly

Object — Displayed as read-only data in the Admin UI

Automatically retrieves data from the Embedly API about the value of another field (specified with the from option).

It stores the retrieved data (which includes the provider, media type, full URL, HTML embed code, width, height, thumbnail picture and more).

The api call to retrieve the data is implemented as a pre-save hook, and is only triggered if the from path value has changed.

See the Embed.ly configuration documentation for details on how to set up Embed.ly in KeystoneJS.

{ type: Types.Embedly, from: 'path' }

LocalFile

Object — Displayed as a file upload field in the Admin UI

Stores files on the local file system.

{ type: Types.LocalFile }

{
	type: Types.LocalFile,
	dest: '/data/files',
	prefix: '/files/',
	filename: function(item, file){
		return item.id + '.' + file.extension
	}
}

{
	type: Types.LocalFile,
	dest: '/data/files',
	prefix: '/files/',
	format: function(item, file){
		return '<img src="/files/'+file.filename+'" style="max-width: 300px">'
	}
}

S3 File

Object — Displayed as an file upload field in the Admin UI

Automatically manages files stored in Amazon S3, including uploading and deleting.

{ type: Types.S3File }

{
	type: Types.S3File,
	filename: function(item, filename){
		// prefix file name with object id
		return item._id + '-' + filename;
	}
}

{ 
	type: Types.S3File, 
	headers: { 
		'x-amz-meta-Cache-Control' : 'max-age=' + (60 * 15),
		'x-amz-meta-X-Custom-Header' : 'Object Option'
	} 
}

{ 
	type: Types.S3File, 
	headers: [
		{ name: 'x-amz-meta-Cache-Control', value: 'max-age=' + (60 * 15) },
		{ name: 'x-amz-meta-X-Custom-Header', value: 'Array Option' }
	]
}

{ 
	type: Types.S3File, 
	headers: function (item, file) {
		var headers = [];
		headers.push({ name: 'x-amz-meta-Cache-Control', value: 'max-age=' + item.maxAge });
		headers.push({ name: 'x-amz-meta-X-Custom-Header', value: 'Computed Option (Array)' });
		return headers;
	}
}
// or 
{ 
	type: Types.S3File, 
	headers: function (item, file){
		var headers = {};
		headers['x-amz-meta-Cache-Control'] = 'max-age=' + item.maxAge;
		headers['x-amz-meta-X-Custom-Header'] = 'Computed Option (Object)';
		return headers;
	}
}

{
	type: Types.S3File,
	format: function(item, file){
		return '<pre>'+JSON.stringify(file, false, 2)+'</pre>'+
					'<img src="'+file.url+'" style="max-width: 300px">'
	}
}

More examples

See the Examples page for projects that demonstrate real-world usage of the various list options and field types.