Half-Elf on Tech

Thoughts From a Professional Lesbian

Tag: zenphoto

  • Making HTTPS Everywhere

    Making HTTPS Everywhere

    With the advent of Let’s Encrypt, introducing free and easy SSL certificates for everyone, and the fact that Plesk, cPanel, and home grown Panels like DreamHost’s all providing easy ways to install certs, renew them, and support them, we’re finally inching our way to the dream of HTTPS Everywhere.

    Why HTTPS?

    The S makes it secure, and the green lock on a browser tells a person that their visit is safer, encrypted, and obscures sensitive data. It means a visit is confidential. It means the site is the real site. It cannot be easily monitored, modified, or impersonated.

    While this blog has no sensitive data of yours, it does accept (require) your email when you leave a comment. You don’t want everyone knowing that, I suspect. You probably don’t want everyone grabbing your IP.

    Why do we care about security in general? Because nothing is non-sensitive anymore. Everything we do and say on the Internet can be used against us. Entering in your mother’s maiden name on a form over HTTP? Someone can snipe that and use it to steal your identity. Use the same password on multiple accounts, one of which is HTTP? Your code can be stolen. The list goes on and one.

    How I Turned This Site HTTPS Everywhere

    Every single domain on ipstenu.org is now https. Everyone either has the Let’s Encrypt certificate or a Comodo one. First I turned on Let’s Encrypt. Then I used WP-CLI to search and replace my urls:

    $ wp search-replace http://halfelf https://halfelf
    $ wp search-replace http://ipstenu https://ipstenu
    

    And so on and so forth down the line.

    Next I checked my mu-plugins folder and my content folder to make sure none of my home grown code was hardcoding in http (it wasn’t), and updated my wp-config.php to include this:

    define('FORCE_SSL_ADMIN', true);
    define('FORCE_SSL_LOGIN', true);
    

    That probably wasn’t required but why not? Finally I tossed this into my .htaccess:

    # Force non WWW and SSL for everyone.
    <IfModule mod_rewrite.c>
    	RewriteEngine On
    
    	RewriteCond %{HTTP_HOST} ^www\.(.*)$ [NC]
    	RewriteRule ^(.*)$ https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]
    
    	RewriteCond %{HTTPS} off
    	RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]
    <IfModule mod_rewrite.c>
    

    Really. That’s all it took to swap it all to https everywhere here.

    Gotchas

    Not all my plugins were happy about this.

    Most were, actually, which was nice, but a couple did some incredibly stupid things with hardcoded http resources. Fixing them for myself is trivial. For others… I recommend WordPress HTTPS or Really Simple SSL, both of which will let you force https for all URLs or block the http ones.

    For the most part, with WordPress, you don’t need to worry about this. In recent years, the ability to force SSL from within WP itself has gotten better and better. The problem has always been our themes and plugins.

    Other than that, it’s been pretty smooth going.

    Non WordPress

    But… what about my non-WordPress sites? Yeah, you know I have them.

    Well my ZenPhoto20 site doesn’t run any extensions, so I just checked the box for using SSL and went on my way. I’d cleverly written all my themeing with protocol-less URLs (\\example.com\my\path\file.css). While that’s really an anti-pattern, and https should be used whenever possible, I had everything in one file so I search/replace’d that and it was done.

    My Hugo site required two changes to a config file. It looked like this:

      StaticURL = "https://example-static.net"
      HomeURL = "https://example.net"
    

    The reason I did that was so my templates could look like this:

    <link rel="shortcut icon" href="{{ .Site.Params.StaticURL }}/images/favicon.ico">
    

    Once I saved the variables in my config, I could push the site (which automatically rebuilds and deploys) and be done.

  • Not Ditching ZenPhoto After All

    Not Ditching ZenPhoto After All

    When we last left our heros… We had taken a SQL database, converted it to JSON, split it into 885 separate JSON files, renamed and moved them based on the folder path previously assigned.

    That took a while, but now we’re ready for the fun!

    Convert them to MD

    I already know how to do this one. Run grunt-mustache-render!

    One problem. It didn’t like recursive folders. Pause for laughter. Thankfully I had done a cp and not an mv on my files with Mike’s script, so I just changed that to handle moving .md files instead of .json and it all went fine.

    Turn on Hugo

    I had a theme already so I copied it over and then started making my changes. I needed the layout to be a little different.

    First I needed a list of all the sections and files and I needed to grab all the items that were an ‘Index’ file and format them specially:

    <div id="albums">
    {{ range $section, $taxonomy := .Site.Sections }}{{ range $taxonomy.Pages }}{{if eq .Type "index" }}
    		<div class="indexalbum">
    			<div class="thumb"><a href="{{ .Permalink}}" title="View album: {{ replace $section "-" " " | title }}">IMAGE HERE</a></div>
    			<div class="albumdesc"><h3><a href="{{ .Permalink}}" title="View album: {{ replace $section "-" " " | title }}">{{ replace $section "-" " " | title }}</a></h3>
                {{ with .Params.desc }} {{ . | safeHTML}} {{ end }}</div>
    			<p style="clear: both; "></p>
    		</div>
    {{ end }}{{ end }}{{ end }}
    </div>
    

    I felt very pleased with myself at this point. Obviously I would need to replace “IMAGE HERE” with an image, but I had the basic idea down.

    Inside each post, there was a shortcode mentioned before, {{< gallery >}}, and this is where my drama started. What I wanted I thought would be (fairly) easy. Get a list of all the folders where this index.md file was and list them similar to what I had done before. They were sub-sections (in Hugo Terms), but as it happens, Hugo doesn’t have a way to handle those! There are no templates for subsections. Now this in and of itself wasn’t a deal breaker. I could made do with that shortcode except…

    Limitations

    The thing I really do like about ZenPhoto is that I can drop my new media in a folder and magically knows what’s up. Add a new folder? It knows it’s there. With Hugo (or Jekyll) I found it was incredible hard to do something as ‘simple’ as getting a list of all the items in a folder. With Jekyll I’d need a plugin (which I am certainly not adverse to). Hugo has a readDir call, but it’s limited to the working directory and I wanted to read the images in a CDN folder (on the same server, but still).

    Perhaps ironically, it’s easier to run a gallery with Hugo and Jekyll if you host the images off your own domain. It goes against my goal of having a self-hosted site, though. But for now, it’s not possibly to do what I wanted.

    Do I feel like I wasted time?

    No! In fact, I’ve learned a lot of things that tell me where the foibles are in my plans.

    ustwo actually did make a React powered site and in looking at their source code I see they did it by making the site ‘all one page.’ This is something I can understand today and wrap my head around. Because the alternative would be something like this:

    1. Write a post
    2. Run a script to grab all the pages via the JSON API as individual .json files
    3. Run a script to convert them into .md
    4. Build static site (test and then deploy)

    It’s not actually that horrible when I put it that way. It means everyone could write on WordPress and, when they’re ready to deploy, I just run my script, push to my dev site, validate, and push to live.

    But. That loses a lot of what makes WordPress cool. Editing posts on the fly, pushing live updates, and scheduling posts all becomes a nightmare. For my Gallery idea, it means that WordPress remains a viable option except for the fact that I can’t easily move from ZenPhoto to WordPress and WordPress behaves like a blog when I don’t want it to be one.

    I’m all about using WordPress for things other than blogging but the sad truth is it’s still structured like a blog. Custom Post Types are doing a great job, and a lot of what I do on my estore site is similar to how I’d want to handle a gallery. I could do posts as posts and organize them as categories, for example, and structure the whole theme to handle that…

    Or I could just stay on ZenPhoto, which does all that out of the box.

    And yes. That’s what I’m doing.

  • Ditching ZenPhoto for Hugo?

    Ditching ZenPhoto for Hugo?

    This started out as a 230 word bullet point list of how to do things. It’s now two entire posts!

    (Almost) Leaving ZenPhoto

    I have to preface this by telling you that I didn’t in the end. Yeah. I know. I planned to. And here’s why:

    I like ZenPhoto a lot. It’s a great Gallery tool, and after Gallery2 went to Gallery3 and I hated it, I was quite in love with them. But. They’ve started to inch towards being a general CMS and the whole ZenPhoto/ZenPhoto20 split left a weird taste in my mouth.

    Plus the DB I have is already 65 megs, which has started to behave a little oddly. Now I have 38,534 images in 885 albums, and to a degree that makes sense. If you think about it in WordPress terms, I have 885 taxonomies (tags and categories), and 38,534 posts. Given that each post in this example would be a sentence at most (describing the image), the WordPress DB would be around 15megs based on some napkin math I did. Validating this, there’s a WordPress Joke DB with 40k Jokes and it’s only 14 megs.

    Oh and did I mention that ZenPhoto has no export feature? I guess there’s a reason the first PHP script I wrote was a simple Gallery. All I really want is to display images in a structured format.

    But. As I said before, I did not leave ZenPhoto! Why not? Because I was unable to do what I wanted to in pure Hugo, and because porting everything over to WordPress is incredibly hard and complex.

    Why Not WordPress?

    WordPress has a lot of awesome advantages. It auto generates thumbnails for one, which Hugo does not, so porting the gallery over would save me one level of complication. But I have 38,534 images to import. No matter how you slice that, I’d have to import them and associate them with the right post. Probably manually.

    With a static generator, I can very easily assign a variable for each album to say ‘get images from this folder and display them.’ And if there are folders, list each folder as a link to another page. By contrast, I would have to use a plugin like NextGen gallery to do this in WordPress, and while I do like the plugin, it’s got the same issue that ZenPhoto has for me. It’s doing too much. And, to be honest, part of this process is to limit my potential security issues. Adding in really big, really complex plugins is the opposite of that, no matter how rigorously reviewed they are.

    Besides, if I hated it, I knew I could import MD files into WordPress. Someone already did it with Jekyll after all.

    Get and Grep the Data

    No matter what I needed the data. Thankfully I knew how to do this with phpMyAdmin already. I dropped the _albums table in JSON format. Why JSON? Well as I’d already learned, I could generate a Markdown file from a JSON pretty easily. Of course, I got stumped on having it loop through the JSON file, but I have a plan for this.

    On export, albums.json was 643 KB while images.json was 61.4 MB. And they were all one line. I’m not using images.json just yet.

    I tossed it into BBedit and scrubbed the data, removing everything I didn’t need. This did not change the file size at first, since I also made the JSON human readable. There was a lot of gripping here, but I took each entry from 16-22 items down to six: tags, categories, folder path, title, description, date. And the file went down to 143 KB. One-sixth, more or less.

    Finally I had to split each entry because, as I’d learned, trying to loop through a nested JSON array in grunt is complicated. So fine, I’ll do it manually. But how? Answer: By loving linux. You can split files!

    $ split -a 6 -l 8 ../albums.json album-
    

    There’s also csplit, which is contextual split. Either way, I now have 885 album-xxx.json files. Whew.

    Rename and Move the Files

    I messed around with linux commands to move files based on their folder paths and rename them based on the last variable. I dreaded having to just move them and then going to mess with the renames. So after two days, I asked Twitter and was saved by Mike Little and Kailey Lampert. Between them I learned that there was a jq tool for messing with JSON (homebrew install it) and that a shell script is a godsend.

    #! /bin/sh
    
    # awk script:
    # set field seperator to :
    # strip comma from field 2
    # strip quotes from field 2
    # strip lead space from field 2
    # strip trail space from field 2
    # print field 2
    
    for FILE in `ls *.json` ; do
        NAME=`cat $FILE | grep folder | awk 'BEGIN { FS= ":" } { 
            gsub( ",", "", $2 );   \
            gsub( "\"", "", $2 );  \
            gsub(/^[ \t]+/,"",$2); \
            gsub(/[ \t]+$/,"",$2); \
            print $2 
        }'`
        
        mkdir -p $NAME;
        cp "$FILE" "$NAME/index.json"
    done
    

    I ended up changing it to reduce future broken URLs. But basically everything except the mkdir comes from Mike. He’s my hero.

    To Be Continued

    Once I had all the files, it was time to change them all to Markdown files and whip out my templates!

    But you’ll need to wait for that because this post got really long.

  • Static Content Subdomain

    Static Content Subdomain

    I use a lot of different tools to run my websites, and over time I’ve learned what I want is to have my static content, the files that are uploaded and are images, stored separately from my apps. So while I have the basic folders on my domain (wordpress, wiki, gallery) I have a special subdomain called static.example.com for all those images and videos.

    There are a few reasons I do this. First, I like having my images separate. Second, it allows me to establish a cookie-free subdomain for images and that shuts up YSlow’s check.

    Create The Subdomain

    Do this however your host allows. Keep in mind that some don’t allow you to traverse domain folders. If your host creates your domain as /home/user/example.com and subdomains as /home/user/static.example.com you may have to fight a little more with things depending on your setup. If possible, I prefer to put the subdomain folder inside the main web root.

    If you’re using cPanel, by default you get your static subdomain installed at /home/user/public_html/static which is how I like it. This is perfectly accessible by all things but it’s also browsable at example.com/static/ and we don’t want that. Applying a little .htaccess magic will solve this.

    # CDN
    <If "%{HTTP_HOST} == 'example.com' ">
            RedirectMatch ^/static/(.*)$ http://static.example.com/$1
    </If>
    

    Now we’re ready to go!

    Move WordPress Uploads

    This used to be really easy. Go to Settings -> Media and change things. But we removed that to stop people from blowing themselves up. Now there are a couple ways about it. I jumped right over to editing the options by going to wp-admin/options.php and look for upload_path and upload_url_path.

    Setting image location options

    I change upload_path to /home/example/public_html/static/wordpress which is where I’ve moved all my images. Then upload_url_path becomes http://static.example.com/wordpress and I’m done except for fixing my old posts. It’s actually pretty neat that once I put those paths in, the Media Settings page lists them as editable.

    Fixing the old posts takes a little trick though, and you’ll have to search/replace your posts via the database:

    UPDATE wp_posts SET post_content = REPLACE(post_content,'http://example.com/wp-content/uploads/','http://static.example.com/wordpress/');
    

    Or in wp-cli:

    wp search-replace http://example.com/wordpress/wp-content/uploads http://example.com/wordpress
    

    The gotcha here is that since I use SSL for my administration, I had to set up a new certificate for the static domain. Not a big deal right now since I can set up a self-signed, or use StartSSL until Let’s Encrypt is off the ground. It is something to consider though.

    Move ZenPhoto Uploads

    I have to start by warning you that Zenphoto doesn’t like this. When you install it, it puts your images in an albums folder, in the Zenphoto gallery install. This isn’t so bad, but you actually can move it around. You have to look in your zenphoto.cfg.php file (found in zp-data). The default location for your albums is defined by this:

    $conf['album_folder'] = '/albums/';
    $conf['album_folder_class'] = 'std';
    

    Since I want it in the static location, I tell it my folder path based on ‘web root’ and that its ‘in_webpath’ (which tells ZenPhoto to look in the root and not relative), by changing that section to this:

    $conf['album_folder'] = '/static/gallery/albums/';
    $conf['album_folder_class'] = 'in_webpath';
    

    But that means my URLs for images become http://example.com/static/gallery/albums... and I wanted http://static.example.com/gallery/albums... instead. Thankfully the .htaccess rule I used at the beginning of all this covers me there. Looking into this, I understand this is the case because unlike MediaWiki or WordPress, ZenPhoto only has one ‘location’ setting. The other two have path and URL.

    MediaWiki

    This was … weird. Technically all you have to do is set up the folders and change the following values in LocalSettings.php:

    $wgUploadPath       = "/static/wiki";
    $wgUploadDirectory  = "/home/example/public_html/static/wiki/";
    

    The thing that’s weird is that the documentation says you can do this:

    $wgUploadPath       = "http://static.example.com/wiki";
    

    And when you do, the image URLs properly call from the domain name. They just won’t load. When you dig deeper, it turns out that it’s caused by the settings for responsive images. The way it puts in srcset doesn’t seem to like this. So for now I’ve disabled it and my setup is this:

    $wgUploadPath       = "http://static.example.com/wiki";
    $wgUploadDirectory  = "/home/example/public_html/static/wiki/";
    $wgResponsiveImages = false;
    

    End Result?

    All my uploaded content is on my ‘static’ subdomain, separate from everything else, which makes version control even easier. Also now if I ever decide to move things off to a CDN, I’m pretty well set up.

    The real reason I do this is that while some of my content is uploaded via the content management systems I use (WordPress, ZenPhoto, etc), the majority is not. ZenPhoto, for example, is faster to FTP up a gig of images than it is to use a PHP tool. Ditto videos. And because of them, it’s nice to have a separate location I can give access to without allowing someone full rights on all my tools.

  • ZenPhoto and ColorBox

    ZenPhoto and ColorBox

    A color boxI use ZenPhoto for a gallery on a site that has a pretty hefty (gigs) gallery with many albums and subalbums. It’s too big for WordPress, in my experience, and so I picked up ZenPhoto as sort of the WP of the gallery world. Not knocking WP, it’s great for text, but sorting and organizing images are a hassle. The flip side to this is that getting straight directions on how to do anything in ZenPhoto makes me bang my head on the wall.

    See, WordPress has a lot of people involved, so the forums are filled with people who’ve been there before. And these people come from a varied array of talents, so some are designers, some programers, and some users. This means the documentation, while lacking in many respects, is actually a pretty awesome display of crowd-sourcing when you compare it to other web apps. The worst part is there’s no perfect way to replicate this dynamic. ZenPhoto is still relatively young, even though it’s only a year younger than nine year old WordPress! MediaWiki (at 11) is older than both, but ‘behaves’ more like the middle child, if you really want to break your head on things.

    It’s a lot to do with goals, and you can’t knock any one tool for the other. They have their places. I would never try to blog on MediaWiki, nor would I put a seriously hard-core gallery on WordPress. ZenPhoto has branched out into ZenPage, a simple CMS, but personally I’d rather see them optimize the hell out of their back end, which could use some UI love. Still, a lot of its simplicity is why I chose to use it instead of, say, Gallery or Coppermine.

    But the help is still lacking, so today was a bit of a wrangling and head bashing.

    What I want is, you’d think, straightforward: How do I edit the default theme of ZenPhoto to include ColorBox? If you ask this on the ZenPhoto forums, you get an understandably annoyed mod saying ‘This has been asked before.’ I feel for them, but as a mod and a user, I look at that and think ‘If people keep asking and you can’t give them a link to how to do it, something’s not right.’

    The directions I found in the forums never worked, but it wasn’t long before I realized why. There were simple typos. So here’s how you can turn on ColorBox for ZenPhoto.

    ZenPhoto

    1. Activate the Plugin

    This is a duh moment, but go Admin -> Plugins and check ColorBox. You do not need slideshow.

    2. Make sure ColorBox is on for your theme

    Go to Admin -> Options -> Plugins and click on ColorBox. Then find your theme and make sure that the pages you want to run ColorBox on are checked. I only wanted it to run on albums, so that’s all I checked.

    3. Edit your theme

    This is where everyone’s directions fell apart for me. Since I only want it on albums, I went to my default theme copy and set my image section to look like this:

            <div id="images">
            <?php while (next_image()): ?>
    		<div class="image"><div class="imagethumb">
    		<a href="<?php echo html_encode(getDefaultSizedImage());?>" rel="showcase" title="<?php echo getBareImageTitle();?>"><?php printImageThumb(getAnnotatedImageTitle()); ?></a>
    		</div></div>
    		<?php endwhile; ?>
    

    Make special note of your classes and rel here! In specific, notice how that I have two divs for image and then imagethumb? While either one will work, I made a note of imagethumb, since it was a little more specific. Also I made a note of the rel in my image itself, in this case rel=”showcase”

    Then back up before I close my head section, I added this:

    	<script type="text/javascript">
    	// <!-- <!&#91;CDATA&#91;
    	$(document).ready(function(){
    	$(".colorbox").colorbox({inline:true, href:"#imagethumb"});
    	$("a&#91;rel='showcase'&#93;").colorbox({transition:"none", height:700, width:"75%" });});
    	// &#93;&#93;&gt; -->
    	</script>
    

    See how I’m using the showcase and the imagethumb? That’s why I needed those.

    4. Customize

    Everyone says ‘Read the directions!‘ but when you look at them, they’re written for people who know jQuery. I don’t. So when I don’t know what I’m doing, I make a list of what I want. By the way, yes, it irritates me when directions are ‘too techy.’ You can’t know where people are in their understanding of things, and you can’t expect everyone to be amazing at everything. I was very close to appealing to anyone who owed me a favor for help before the end of this.

    No set height

    That’s as easy as removing height:700 from my js.

    Force colorbox to treat my cached image as an image

    Just add photo:true to the js. I had to do this because my server renders the images via a php file (to redirect to cache) and this was causing funny problems. It’s a known issue, though, so one I figured out how to search for ‘ColorBox is making my images show up as gibberish!’ I found the answer.

    Put a link to the full sized image

    And here began my headache. If I put in this (where I used to have the height code):

    title:function () { return "To view full size, " + "click here!".link(this.href);}

    … then my link goes to the getDefaultSizedImage() size (which is a max width of 540px for my theme) and that isn’t what I want. I could change it to getFullImageURL(), but then colorbox loads the fullsized image, and that’s just a little silly and bad for bandwidth. I spent the next hour reading up on jQuery to understand that I really wanted to pass data through. Finally I struck about the notion that I could make a new variable in my href.

    full=<?php echo html_encode(getFullImageURL()); ?>

    This makes a link to the full-sized image. And then I changed this.href to $(this).attr('full')

    In the end, it really wasn’t hard, but nowhere were all the pieces laid out in a way I understood. I’m happy with how it all turned out and the site now behaves like it’s 2012.