Tag: json

While it rhymes with bacon, it’s not at all the same.

There are a lot of reasons you might want a JSON file output from your static site (I like Hugo). Maybe you’re using Hugo to build out the backend of an API. Maybe you want to have it include a search function. Today I’m going to show you how to have a JSON file created with a complete site archive. The end goal of this example is to have a searchable JSON file that you can use with Lunrjs or Solarjs or anything else of that ilk.

The Old Way: Node

Since I was initially doing this to integrate Hugo with Lunr.js, I spent some time wondering how I could make a JSON file and I ran into Lunr Hugo, a fork of Hugo Lunr but with YAML support (which I needed). I actually use a private fork of that, because I wanted to change what it saved, but this is enough to get everyone started.

To use it, you install it via Node:

npm install lunr-hugo

Then you add the scripts to your Node package file (normally called package.json):

  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "index": "lunr-hugo -i \"site/content/posts/**\" -o site/static/js/search.json"
  },

Change the value of “site/content/” as you see fit. Once installed you can build the index by typing npm run index and it makes the file in the right location.

The obvious downside to this is I have to run it outside of my normal build process.

Another Old Way: Grunt

This idea come from Seb, one of the lead developers for Hugo, and he uses a Grunt script to do this. First you have to install node and things via this command:

npm install --save-dev grunt string toml conzole

Next you make a Gruntfile.js file like this:

var toml = require("toml");
var S = require("string");

var CONTENT_PATH_PREFIX = "site/content";

module.exports = function(grunt) {

    grunt.registerTask("lunr-index", function() {

        grunt.log.writeln("Build pages index");

        var indexPages = function() {
            var pagesIndex = [];
            grunt.file.recurse(CONTENT_PATH_PREFIX, function(abspath, rootdir, subdir, filename) {
                grunt.verbose.writeln("Parse file:",abspath);
                pagesIndex.push(processFile(abspath, filename));
            });

            return pagesIndex;
        };

        var processFile = function(abspath, filename) {
            var pageIndex;

            if (S(filename).endsWith(".html")) {
                pageIndex = processHTMLFile(abspath, filename);
            } else {
                pageIndex = processMDFile(abspath, filename);
            }

            return pageIndex;
        };

        var processHTMLFile = function(abspath, filename) {
            var content = grunt.file.read(abspath);
            var pageName = S(filename).chompRight(".html").s;
            var href = S(abspath)
                .chompLeft(CONTENT_PATH_PREFIX).s;
            return {
                title: pageName,
                href: href,
                content: S(content).trim().stripTags().stripPunctuation().s
            };
        };

        var processMDFile = function(abspath, filename) {
            var content = grunt.file.read(abspath);
            var pageIndex;
            // First separate the Front Matter from the content and parse it
            content = content.split("+++");
            var frontMatter;
            try {
                frontMatter = toml.parse(content[1].trim());
            } catch (e) {
                conzole.failed(e.message);
            }

            var href = S(abspath).chompLeft(CONTENT_PATH_PREFIX).chompRight(".md").s;
            // href for index.md files stops at the folder name
            if (filename === "index.md") {
                href = S(abspath).chompLeft(CONTENT_PATH_PREFIX).chompRight(filename).s;
            }

            // Build Lunr index for this page
            pageIndex = {
                title: frontMatter.title,
                tags: frontMatter.tags,
                href: href,
                content: S(content[2]).trim().stripTags().stripPunctuation().s
            };

            return pageIndex;
        };

        grunt.file.write("site/static/js/lunr/PagesIndex.json", JSON.stringify(indexPages()));
        grunt.log.ok("Index built");
    });
};

Take note of where it’s saving the files. site/static/js/lunr/PagesIndex.json That’s works for Seb because his set setup has everything Hugo in a /site/ folder.

To build the file, type grunt lunr-index and off you go.

The New Way: Output Formats

All of that sounded really annoying, right? I mean, it’s great but you have structure your site to separate Hugo from the Node folders, and you have to run all those steps outside of Hugo.

Well there’s good news. You can have this all automatically done if you have Hugo 0.20.0 or greater. In the recent releases, Hugo introduced Output Formats. The extra formats let you spit out your code with RSS feeds, AMP, or (yes) JSON formatting automatically.

In this example, since I only want to make a master index file with everything, I can do it by telling Hugo that I want my home page, and only my home page, to have a JSON output. In order to do this, I put the following in my config.toml file:

[outputs]
	home = [ "HTML", "JSON"]
	page = [ "HTML"]

If I wanted to have it on more pages, I could do that too. I don’t.

Next I made a file in my layouts folder called index.json:

{{- $.Scratch.Add "index" slice -}}
{{- range where .Site.Pages "Type" "not in"  (slice "page" "json") -}}
{{- $.Scratch.Add "index" (dict "uri" .Permalink "title" .Title "content" .Plain "tags" .Params.tags "categories" .Params.tags) -}}
{{- end -}}
{{- $.Scratch.Get "index" | jsonify -}}

To generate the file, just run a build and it makes a file called index.json in the site root.

How do you statically build JSON Files?

Do you have a trick or an idea of how to make building JSON files better? Leave a comment and let me know!