Half-Elf on Tech

Thoughts From a Professional Lesbian

Author: Ipstenu (Mika Epstein)

  • Looping Hugo

    Looping Hugo

    I’m ending the year with something non-WordPress. But it has a weird JSON related journey.

    I asked three questions at the end of my last post:

    1. How do I properly mimic a for-loop with range?
    2. Can I make a responsive gallery out of Hugo?
    3. Can I power a Hugo site with WordPress JSON?

    This is the answer to the first one. However in the solving, I made a breakthrough in my head about how one calls data from JSON and structures it.

    But I’m getting ahead of myself, becuase changing a for loop from Jekyll to Hugo was harder than I expected. Way harder.

    One of the things I check in some posts is what the rating is. I set a parameter called ‘rating’ and if it’s there, I ran a quick check to determine how many stars to show:

    <strong>Rating:</strong> {% if page.rating %}{% for i in (1..page.rating) %}<i style="color:gold;" class="fa fa-star" name="star"></i>{% endfor %}{% if 5 > page.rating %}{% assign greystar = 5 | minus: page.rating %}{% for i in (1..greystar) %}<i style="color:grey;" class="fa fa-star" name="star"></i>{% endfor %}{% endif %}{% else %}<em>Not Available</em>{% endif %}
    

    I was pretty damn proud of that loop. Check the rating, output a gold star for each number over 0 and a grey star for every number between the rating and 5. It’s simple but it’s quick.

    Transposing from Liquid to GoLang was not as hard as all that. {% %} became {{ }} and {% endfor %} became {{ end }} and, at first, that was the easy stuff. The majority of my logic was a one-to-one translation. Change page.rating to .Params.rating and so on and so forth.

    The order of the if’s was strange to me, in that it became {{ if eq A B }} (except when it wasn’t) and I was used to thinking {% if A == B %} — it was fairly easy to overcome. In short order, all my simple if-checks were done.

    But those damn loops! I had the ugliest “if A == 1, show 1 gold and 4 grey” configuration. And worse, I had cases where I was checking “If there’s an entry in the Filmography for this show, get the rating from that and not the page itself.” The Filmography file was a straight-forward JSON file, you see (told you JSON was involved).

    Back to the first problem. I knew if I wanted a shortcode to say “Get me a list of all pages, by date, where section is news” I did this:

    {{ range where $.Page.Site.Pages.ByDate "Section" "news" }}

    And that $.Page.Site.Pages.ByDate call changed depending on what template I was on and what I was calling. For a shortcode I had to pass though page to site to pages. On a template I could do .Data.Pages.ByTitle (no $ needed either). I’m still at the trial and error stage of figuring out which call and where, but I know I’ll get there. It’s just mastering new syntax and understanding where I’m calling from and what variables are available to me.

    That’s really okay. It took me years to visualize the interrelations with WordPress themes and functions and plugins, and even then sometimes I’ll output an array in ugly, unformatted ways just to read through and make sure I understand what I’ve got at my disposal. This is normal for most of us.

    And it was fairly clear that if I wanted a shortcode to call the data file and not a list of Section pages, there was code for that: $.Page.Site.Data.filmography (back to Filmography again). And that worked fine. Right up until I wanted to say “For all values where X equals Foo…” and I was back to my loop-headache.

    Now. GoLang does have a for loop!

    func main() {
        sum := 0
        for i := 0; i < 10; i++ {
            sum += i
        }
        fmt.Println(sum)
    }
    

    And I though that I could just use that. Nope! So I asked myself why did {{ for i in (1..Params.rating) }} fail?

    First of all, that ‘in’ should be := instead. But even so, when I ran it I got “ERROR: function “for” not defined.” That means there is no for function. And no loop. After looking at how I was iterating in other places, I did this:

    {{ range $index, $element := .Params.rating }} 
       ONE
    {{ end }}
    

    ERROR: 2015/12/12 template: theme/partials/rating.html:3:26: executing “theme/partials/rating.html” at <.Params.rating>: range can’t iterate over 3 in theme/partials/rating.html

    The 3, in that case, had to do with a rating of 3. This makes a little sense, since you’re supposed to use range to iterate over a map, array or slice. A number is none of those.

    Thankfully I found a ticket talking about adding a loop function to Hugo that was addressing what I was trying to do and finally I had an answer:

        {{ range $star, seq .Params.rating }}
            <i style="color:gold;" class="fa fa-star" name="star"></i>
        {{ end }}
    
        {{ $greystar := sub 5 .Params.rating }}
    
        {{ range $star, seq $greystar }}
            <i style="color:grey;" class="fa fa-star" name="star"></i>
        {{ end }}
    

    The idea there is that seq makes a sequence of the value of .Params.rating for me. If the value is 3, I get an array of [1,2,3] to work with. And that’s something range can itterate over!

    Except…

    at <sub 5 .Params.rating>: error calling sub: Can’t apply the operator to the values in theme/partials/rating.html

    Now the amusing thing here is that the code worked! Even if it wasn’t seeing .Params.rating as an integer, it did the math. I suspect it’s related to this old bug, where variables from front matter are strings and not integers. Except that he said it worked in YAML and that’s what I’m using.

    This was the most perplexing issue. How is a number not a number if it’s being numbered? And then I noticed that I was able to make a sequence, so clearly at that point it knew it was a number. And then I did this:

        {{ $goldstar := seq .Params.rating }}
        {{ $goldstar := len $goldstar }}
    

    And yes it works.

    I’m basically saying “Hey, how many ones are in the number X?” and saving that as a number. There’s no real reason I can comprehend why it failed to work, but there it is. I’ll file a bug as soon as I can figure out how to explain that in a way that doesn’t make me sound crazy.

    How did all this teach me about JSON? You’ll have to wait a few days.

  • Porting Jekyll to Hugo

    Porting Jekyll to Hugo

    Importing content is always the bane of moving systems.

    Now one of the things Hugo has is a cool command called $ hugo import jekyll – and yes, it does exactly what you think it does. This was interesting to me since I wondered if I could switch my library from Jekyll to Hugo and yes, yes I can.

    The site I have on Jekyll has a few more posts than the first one I built on Hugo. This one has around 1500 pages. Excited, I ran the command:

    Importing...
    Congratulations! 0 posts imported!
    

    That’s not right. I spent a few hours trying to run it in various permutations, but I was never able to get it to run right. Thankfully, I didn’t need to since I could just copy my content over. There was no translation needed, as I was already using yaml headers in my .md files so that actually was a pretty painless transition.

    The theme on the other hand… Well. I had a complex site with data files and weird junk being thrown around. That was just learning how to translate for-loops for ranges, with some where clauses tossed in.

    Was it hard? Yes. But that’s not what you’re really asking.

    Was it worth it?

    Excuse me while I whip this out:

    Change detected, rebuilding site
    2015-12-11 21:42 -0800
    0 draft content
    0 future content
    1479 pages created
    0 paginator pages created
    16 categories created
    66 tags created
    in 4611 ms
    

    Yes. The site builds faster, which is a win. It also creates tags and categories that can automatically cross link. I can make ‘sections’ without having to define them in my config file, just by making folders. I can spin up templates quickly. I have a little more redundancy than I’d like, but as I master GoLang, this becomes less and less.

    Build time ranges from 11737 ms to 2116 ms. Milliseconds. They were into the minutes with Jekyll, even though Jekyll 3 was notably faster than version 2. It was still taking me about 2 minutes to rebuild the Jekyll site, even with incremental builds working. Plus I had to build and rsync every time, in addition to Git pushing. Now I just Git and done.

    Oh and JSON works remotely out of the box. I just have to get better at my range calls now.

    What’s Different?

    Making a ‘static front page’ with Hugo is harder. In fact there’s still an open trac ticket on it. I was able to use an idea I already had, a custom post param called “type” for tagging ‘index’ posts in the news pages so I didn’t get a list of my indexes when I wanted to list all news from 2002. I leveraged that and tagged my primary index ‘mainindex’ and put a quick check in my index.html in my theme:

    {{ range .Data.Pages }}
        {{if eq .Type "mainindex" }} 
            {{ partial "content.html" . }}
        {{end}}
    {{ end }}
    

    Instead of doing a normal loop where, like a blog it shows me posts in reverse order, it just showed that specific content.

    But…

    If there’s a file in that Section with the type of ‘Index’ then it shows that. Of course… While index.md doesn’t get used in the main content folder, it does in Sections. If I have a file named index.md in a section, that makes index.html like I expected. Since I still wanted lists in the majority of cases, I tweaked the content of my default list file (list.html) so that I could always show a ‘main’ file:

        {{ range .Data.Pages.ByTitle }}
            {{ if eq .Type "index" }}
                {{ .Content }}
            {{ end }}
        {{ end }}
    
        <ul>
        {{ range .Data.Pages.ByTitle }}
            {{ if ne .Type "index" }}
                <li><a href="{{ .Permalink }}">{{ .Title }}</a></li>
            {{ end }}
        {{ end }}
        </ul>
    

    Then I renamed all the index.md files in the Sections to main.md but this created another issue. Now I had two URLs that worked. I could have put this into the template for the section to avoid this, but I want to separate content and theme as much as humanly possible. WordPress habits. For now, I’ve left this alone. If someone is clever enough to go to a non-linked, non-sitemaped page instead of the index, I think my SEO will live. It’s more than I wanted to play around with loops.

    Speaking of the loops and lists, getting a list of a ‘sub section’ is not currently possible either. Rather, they do work, but there’s no default templates for sub-sections. This was a small problem since I structured my site like this:

    .
    └── content
        ├── news
        |   ├── index.md          // /news/
        |   ├── 2001
        |   |   ├── loch-ness.md  // /news/2001/loch-ness/
        |   |   └── monster.md    // /news/2001/monster/
        |   └── 2002
        |       ├── nessie.md     // /news/2002/nessie/
        |       └── spock.md      // /news/2002/spock/
        └── posts
    

    Now, I don’t need a custom template for each sub-section, but I did want to use my custom list template for each sub-section. To work around this, I made a template in /layout/section/news.html that had this:

    	{{ range .Data.Pages.ByTitle }}
    		{{ if and (eq .Type "index") (ne .Params.topic "index") }}
    			{{ .Content }}
    		{{ end }}
    	{{ end }}
    

    I know this is weird. The Type of index means “I am the main index file of the section” while the topic index means “I am an indexing file and not a normal news article. This gives me my main index of /news/ giving me a custom content page with information like I wanted, using the main.md format I use in all sections.

    Then in each sub-section, I use a very simple index.md file:

    ---
    title: News Articles (1992)
    author: Mika Epstein
    layout: news
    topic: index
    date: 1992-01-01
    permalink: /news/1992/
    categories: ["News"]
    tags: ["1992"]
    ---
    
    {{< news >}}
    

    That weird bit in the content is a shortcode ala Hugo, which calls the file /layouts/shortcode/news.html that has this:

    {{ $date := $.Page.Date.Format "2006" }}
    
    <table class="infobox">
    
        <thead><tr>
            <th>Date</th>
            <th>Source</th>
        </tr></thead>
    
        <tbody>
        {{ range where $.Page.Site.Pages.ByDate "Section" "news" }}
            {{ $thisdate := .Date.Format "2006" }}
    
            {{ if and (eq $date $thisdate) (ne .Params.topic "index") }}
    
                <tr>
                    <td>{{ .Date.Format "01 January" }}</td>
                    <td><a href="{{ .Permalink }}">{{ .Title }}</a></td>
                </tr>
    
            {{ end }}
        {{ end }}
        </tbody>
    </table>
    

    That spits out a table of all the pages in that section by date.

    What’s Next?

    It took a lot to get here, or so it may seem. This took me two nights to work out. That’s it. So now I have to figure out the next steps that have vexed me:

    1. How do I properly mimic a for-loop with range?
    2. Can I make a responsive gallery out of Hugo?
    3. Can I power a Hugo site with WordPress JSON?

    And that’s a whole ‘nother story.

  • Mailbag: Hugo from iPads

    Mailbag: Hugo from iPads

    So I’ve done the whole Hugo thing and it’s great and it totally works for me. That 1% itch left from Jekyll is gone. So queue the inevitable…

    But what about mobile posting?!

    Why on earth the planet is obsessed with posting everything from their phone, I don’t know. Things like Instagram and Twitter make it easy for us to ‘communicate’ (and I use that loosely) and post photos of our lunch. And yes, the iOS app for the iPhone means I can ‘live blog’ but, to be honest, I hate it on my iPhone.

    My iPad though… I love posting from that.

    And yes, yes I can post to my Hugo run site from my iPhone or iPad. Remember, I can push my new content by running a git push command. The server will build, sync, and clean up on its own. All I need to do it is a Git app on my iPad that doesn’t suck

    Working Copy does not suck.

    It’s an iOS app that pulls my git repository to my iPad (the whole thing, so make sure you have room). Then I can edit files, commit them, and push. Hugo on the server does the rest, just like it would from my desktop.

    Working Copy is different from many other Git related apps because can hook into any git repository. I’m not using GitHub for this project. I want to do 100% self hosting, and that means no GitHub. Also no BitBucket. These tools are fine for what they are, and in fact I pay GitHub for some private repos. But I wanted my repo on my server, in part so I could do exactly what I’m doing.

    The tool is incredibly simple. It’s a familiar file navigator, tap through the folders. Tap edit to edit, make changes to the post, hit done. There’s even a preview feature that mostly works. Some of my Markdown files are very weird.

    If I wanted to edit in Byword, which is what I do most of my writing in for non-novel related things, I can share and go back and forth. I’d love it if the WordPress iOS app (or Calypso) did that. Write in Byword, send to other app. But even a copy/paste is simple enough. In this case, I can write in Byword and share to Working Copy. The interface takes the title of my document and let’s me pick the folder.

    Once I’m done with my edits, I commit my changes. Then I can push, or not. There’s an option to commit+push, but it crashed my app. A skim of reviews showed this happened to others. The iOS interface can be a bit tetchy so having this be two steps doesn’t bother me.

    I did find it odd that there was no button to push. I had to go back to the main folder, swipe-left, and press the orange push button. But that, and the weird crash, were it for annoyances.

    While free to download, it’s $9.99 for unlimited deployments. It was free for 3 weeks, which I tried out over WordCamp US and found simple and perfect for my workflow.

  • Consenting to Collection

    Consenting to Collection

    Collecting information on users is something every program wants to do. Doing it is easier said than done. (more…)

  • Disclosure

    Disclosure

    One of the myriad reasons I push back on WordPress plugins is because someone didn’t disclose enough information about ‘phoning home.’ Phoning Home is simple concept that comes down to this “Don’t send data from the plugin users back to yourself (or anyone else), unless you’re a service. And if you are a service, make this clear in the readme.”

    Simple, right? It’s totally easy to understand that we want you to tell people “Hey, if you use this plugin, it will report back to my servers…” Much of the time, this is obvious. A Facebook connection plugin, logically, contacts Facebook. Embedding YouTube playlists contacts YouTube. Those sorts of things we don’t worry about, though if you have your plugin say “This will pull data from YouTube” then it’s better. Sometimes this is less obvious, like if you use geoplugin.net to determine locations. Your plugin probably has nothing to do with that domain, but you still have to tell people about it so they know.

    But why do they need to know?

    First of all, this is the basic email I’ll send you if I see you’re not explaining the phone-home in your plugin:

    Plugins that send data to other servers, call js from other servers, and/or require passwords and APIs to function are required to have a full and complete Readme so we can make sure you’re providing the users with all the information they need before they install your plugin. Our goal with this is to make sure everyone knows what they’re installing and what they need to do before they install it. No surprises.

    This is especially important if your plugin is making calls back to your own servers. For the most part, we do not permit offloading of images or code, however in the case where you are providing a service (like Disqus or Akismet or Twitter), we permit it. The catch is you have to actually explain this to the layman in your read me, so they know where data is going.

    Clearly there are some basic reasons, like we should know where our data is going for our own safety. There are also some surprising reasons to people who don’t think about these things, like legal ones. You’re calling out to other servers? What if my company legally can’t do business with them? Then we have the tin-foil hat reasons, like I don’t want to do business with Google so I don’t want to have Google JS in my plugin.

    All that sounds pretty basic. And some of it is super obvious. If you’re making an app to communicate with Facebook, then it’s logically going to send data to Facebook. None of that surprises anyone, nor should it. With a service, one simply has to be up front about what the product does, what services it connects to, and why.

    “This plugin pushes your comments from your Facebook page to your blog, matching users by email addresses with their Facebook accounts (if found).”

    I just made that up. But it’s upfront, it’s honest, it’s direct, and it’s clear what’s being sent and where and why.

    There’s another aspect to this, however, something that is far trickier and more complex. What happens when your existing tool adds a service?

    The obvious answer is that you need to disclose this change to our users. As long as the users know what they’re getting into, then you are golden. The complex answer, the one I can’t really tell you a one perfect answer for, is how you might do this.

    Why is this hard? It’s hard because there is no one right way to tell users about the change. There is no one perfect way to make sure users read the information. There is no one way to get all the data to all the people who need to know about the information.

    And there sure as hell isn’t one way to make sure no one will complain about any of that.

    When you make a change to the paradigm of what your tool does, taking a stand alone tool and adding in a service, you have to consider that a percentage of your users will rebel. This is simply because all those wonderful things you do about disclosing the service for new users have to be transitioned in a meaningful and logical way to existing users. And there is just no way to do that perfectly.

    This doesn’t mean you shouldn’t try. This means you have to be creative and innovation and a little ‘in your face’ about the change. You have to give users an option, before the service kicks in, to say if they want their data shared.

  • Mailbag: Wasting Time

    Mailbag: Wasting Time

    Today’s question comes from a Slack DM tossed my way about learning and possibly discarding Jekyll.

    Don’t you think you’re wasting your time learning all these nonWordPress things?

    Nope! Every single thing I’ve learned and discarded has improved my skill set.

    Let’s take MediaWiki. Learning that taught me templating in a way that I never would have understood in WordPress. It also taught me about the perils of your ‘own’ language instead of HTML. While I’ve come to like Markdown, you still have to know HTML to make Markdown really work, because you need to understand what it is you’re writing.

    And Jekyll? I learned a lot about importing and exporting data between ‘languages’ that don’t like each other. I also learned a lot about deployment of static content. Anything but FTP, right? Jekyll had me writing my own deployment scripts.

    Now that I’m looking at Hugo, really not much has changed. I’ve learned GoLang, which is not all that different from things I already knew. But it’s expanding how I think about the logic structures. Hugo’s got an up on Jekyll in a lot of ways, like how easy it is to make loops for traditional blogs. Also it can handle remote JSON a little better, which sets me up for what’s next.

    You see, all of this work, all this learning, is going to come back to WordPress.

    What I want to do is manage my site in WordPress and have that output the JSON (via that awesome new JSON API I’ve been learning), which will in turn be dynamically called when I want to build my static HTML site.

    For sites you’re not updating daily, or even weekly, this might be the magic you need. Everyone writes on WordPress. Someone has a command (or even a script) to run that collects everything from JSON and dumps it to Hugo, which generates the site for you to proof and then push.

    Version controlling the content.

    Let the users write in Wordpress while you bask in the glory of your static site that is, pretty much, unhackable as a CMS. I mean… what are you going to do to my HTML?

    See?

    It’s all WordPress.