Half-Elf on Tech

Thoughts From a Professional Lesbian

Tag: Codeship

  • AutoDeploy: Github to DreamObjects

    AutoDeploy: Github to DreamObjects

    There are a lot of things that Github makes easy. One of them is a great way to keep versions of data publicly accessible, trackable, and deployable.

    Well. Not so much that last one.

    Automated deployment with Github can be incredibly convoluted, twisted, and perplexing. If you don’t speak high geek, it’s worse. Thankfully there are services like Codeship that let you ship code, run tests, and more importantly, run some hefty commands.

    When the day came that I wanted to keep some JSON files on Github for version control, but deploy them to the cloud, I knew what to do. I was already doing a version of the process, in reverse, for my Remoteless SVG images, after all.

    Create Your Repository

    Now I’m using Github, but Codeship supports Bitbucket or Gitlab, so that’s also an option. Once you’ve made your repository, commit your code but don’t push anything yet! We have to make a couple decisions first.

    First: Branches.

    I have two philosophies of this. If I’m working on something like a plugin or a theme or an app that needs versioned releases, I’m going to name my branches REL_1.2.3 and so on. If, instead, I’m maintaining a theme or plugin for personal use, I have a development branch that I use for all normal dev work. I may spin up a special branch for some new feature, but it always goes into that dev branch first.

    Second: Deployments.

    The reason I use that branch system is that when I push to the dev branch, it pushes code to the dev server.

    Schnieder from One Day at a Time (2017) going 'Mind blown'
    Mind? Blown.

    Once you decide how you want your set up to deploy, and when, you’re ready to Ship That Code.

    Gettit? Right…

    Setup Codeship

    If you’ve never setup Codeship before, I explain how you do that in my post about Deploying from Github with Codeship. The special sauce here is your Custom Script.

    Before you write the script, add in Environment Variables for AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY with those names specifically. This way you won’t have to write a mess of custom code.

    pip install 
    awscli aws s3 --endpoint-url https://objects-us-east-1.dream.io sync ~/clone/ s3://bucket-name/

    The important bits:

    • Endpoint URL is my S3-esque host
    • bucket-name is my bucket name…

    Push Your Code

    Once that’s saved, you can push code and it’ll automatically kick the code to the cloud.

    A lot of this is obviously a backwards implementation of code I already described, but sometimes we need something explained in the straightforward way to help us move forward.

    Enjoy your deploys!

  • Remoteless SVG

    Remoteless SVG

    I spent about half a year working on being able to remote load my SVG, and I wanted to for a few reasons:

    1. Having only one place to update my icons is more efficient
    2. They don’t need to be in code versioning control (like GitHub)
    3. Licensing (not all are free)

    I solved those problems when I finally figured out SVG Injection (via javascript) and CORS to allow the scripts from my objects.

    It’s not all lollipops and sunshine

    However. There was, and is, one really big issue. The icons being remote mean if the remote server isn’t accessible but my site is, I got no icons. Now, the smart dev in me knows that I should have a check. If the server is up, use the icons remotely. Otherwise, use a fallback.

    There was one big, and in the end insurmountable, problem with that. There was no way to do that without slowing my site down, which was the headache in the first place. If I check the uptime for every icon on a page, the site went from a 1 second load to up to 20 seconds. In the best solution, I added 2-5 second load to every page checking if the server was up on every page load. Which is kind of the problem with wp_remote_get() in the first place.

    Make ’em local again

    This meant the ‘best’ solution was to make them local. Again. Yaaay.

    But I really didn’t want to have them stashed in GitHub. I wanted my ‘base’ to be the cloud storage, and then that needed to sync. Normally I would use rsync for that but you can’t rsync from CEPH storage.

    Don’t panic. There are answers.

    Boto-Rsync

    If you’re on DreamPress or any DreamHost server, they’ve forked boto-rsync and (if you set up a .boto file) you can do this:

    boto-rsync --endpoint objects-us-west-1.dream.io s3://my-cool-icons/ /home/my_user/mydomain.com/wp-content/uploads/my-cool-icons/

    Using cron or some other scheduling tool, setting up a server to run that at regular intervals isn’t too terrible.

    AWS CLI

    There’s also the option of AWS CLI, which is very similar, only it’s actually maintained. The advantage with boto-rsync is that it’s installed on the servers at DreamHost. If you have AWS CLI on your server, you can do this:

    aws s3 --endpoint-url https://objects-us-west-1.dream.io sync s3://my-cool-icons/ /home/my_user/mydomain.com/wp-content/uploads/my-cool-icons/

    Keep in mind, you need to configure this either with aws configure or an ~/.aws/credentials files.

    Pushing from the outside inside

    You may have noticed that, unlike rsync, I can’t tell it to push to another server. That is, I can say “sync files from server A to server B” with rsync and I can’t with either boto-rsync or the AWS CLI.

    And no, you just can’t.

    Unless you use rclone. Which is a little messy but totally do-able.

    There’s one other option though… I’ve been using Codeship to push code. That is, every time my GitHub repos update, it triggers a Codeship deploy. And in that deploy I usually just say “Rsync a git clone, k’thnx’bai.”

    Now I tell it this too:

    # Download from S3
pip install awscli
aws s3 --endpoint-url https://objects-us-west-1.dream.io sync s3://my-cool-icons/ ~/my-cool-icons/
# Copy up to Server
rsync -avz -e "ssh" ~/lezpress-icons/ USER@DOMAIN:/home/USER/DOMAIN/wp-content/uploads/my-cool-icons/ --delete

    I will note there’s one extra trick. You have to add Environment Variables for AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY on Codeship.

    This works for me since there are few times I push an icon without also making some code changes to the server code anyway.

    Hopefully if gives you some directions.

  • Deploying from GitHub with Codeship

    Deploying from GitHub with Codeship

    For the longest time I self-hosted all my git repositories not so much because I enjoyed doing so but because there were limited options for easily pushing code from git to my servers. Invariably you will end up using an intermediary because GitHub and their peers have no real reason nor inclination to make those things easy for you. After all, if they can keep you in their systems, more money to ’em.

    And while that’s perfectly understandable and logical, it’s annoying. And if you’re on a budget and not a Git Expert, it’s extremely frustrating. The deployment API was written in High Level Geek, and I found it a headache to decipher and test.

    Thankfully there are tools for that, and one of them in Codeship. Codeship is free for 100 builds a month, and unlike DeployHQ, it uses rsync, which meshes with my preferred way of moving data.

    The Plan

    My plan is simple. I want to push code to a Github Repository and, when it’s a push to master, it would rsync the files over to my server. This will allow multiple people to work on code, among other things.

    The Setup

    First make an account with Codeship. You can log in as Github which is useful since you’ll want to connect your Github account with Codeship anyway. When you create a project, you’ll be prompted which SCM you want to use:

    Create a Project in Codeship

    I’m using Github, but it’s nice to see GitLab in there as well. Once you pick your SCM, paste in the clone URL.

    Examples:

    • git@github.com:<username>/<repository_name>.git
    • https://github.com/<username>/<repository_name>.git
    • https://github.com/codeship/<repository_name>

    Finally you can pick Codeship Pro or Basic – I picked Basic because it required the least amount of know-how. Not that it’s easy, but I don’t have to configure the server or anything annoying. In fact, Basic is so basic, than you can just accept the default setup commands and go. Which is what I did.

    The Settings

    Once you’ve done all that, you’re on a new screen that tells you to push code. Of course, you can’t do that until you set up a couple more things. Like tell it where to push the code.

    Click on “Project Settings” and go to the General tab. You’ll need to get that SSH key and add it to your server’s ~/.ssh/authorized_keys to allow passwordless deployment. I strongly recommend that.

    Now you can click Deploy and add a Deployment Pipeline. Pick ‘master’ unless you’re using something else.

    Now you have to add a deployment to your pipeline. There are a lot of options here, but for my plan of an rsync, the choice is “Custom Script”. Since I’m pushing code to my DreamPress server, the rsync looks like this:

    # DreamPress
rsync -aCz -e "ssh" ~/clone/ wp_e413xh@mysite.dream.press:/home/wp_e414xh/mysite.dream.press/wp-content/themes/my-theme-name/ --delete

    You can customize your rsync commands however you want. I like mine to delete files I’ve removed.

    Deploy the Ships

    Everything is set up so go back to your project page and push some code to Github

    Magic happens.

    The code deploys.

    Huzzah.

    On the free version you only get 100 private builds and five private projects. Mine are public (so’s the Github repo for that matter) so it doesn’t matter. The only downside is only getting one push at a time, but since they take less than five minutes, it’s perfectly acceptable for a free solution.