post

Easy wp-cli automation with wp-bootstrap

I’ve created a PHP Composer package to provide an easier way to automate WordPress installations using wp-cli. If you are unfamiliar with wp-cli and composer. I suggest you read up on composer here, wp-cli here and the rationale of automation in my book or in my blog series on the subject.

Another great resources for learning more about using WordPress, git and Composer together is to check out the roots.io post on the subject as well as their Bedrock project.

What is wp-bootstrap

Wp-bootstrap is a composer package that adds a bunch of commands to your environment that helps you set up WordPress and themes, plugins and settings in a consistent manner using configuration files rather than scripts or worse, point and click installations. Wp-bootstrap depends on two files that it expects to find in your project root folder:

  1. appsettings.json that contains settings, themes and plugins that your WordPress site needs to use. It also has a way to manage some of the content of your application. This file is meant to be a part of your application and managed in a source code control system.
  2. localsettings.json that contains settings that are specific to each environment. This includes database name, password, WordPress installation path and more. This file is meant to be unique to each environment that your site runs in (development, staging, production etc) and is not supposed to be managed by source code control.

By combining these two files wp-bootstrap is able to setup a WordPress installation from scratch, let’s see how.

INSTALLING wp-bootstrap

{
    "require": {
        "eriktorsner/wp-bootstrap": "0.2.*"
    }
}
$ composer update
$ vendor/bin/wpbootstrap wp-init-composer

To include wp-bootstrap to your project, just add the above lines to your composer.json file. Or if you prefer, use this command:

$ composer require eriktorsner/wp-bootstrap
$ vendor/bin/wpbootstrap wp-init-composer

By running the command “vendor/bin/wpbootstrap wp-init-composer” to add the wp-bootstrap commands to composer so that you can call wp-bootstrap easier, this step is not strictly needed.

LOCALSETTINGS.JSON

In addition to installing wp-bootstrap, you also need to have a localsettings to tell wp-bootstrap where to find your database, some of the credentials and where to install WordPress (where the web server expects to serve the files from). Note that your’re not really supposed to install WordPress in the same folder as your configuration files. Here’s a sample localsettings.json file:

{
    "environment": "development",
    "url": "www.wordpressapp.local",
    "dbhost": "localhost",
    "dbname": "wordpress",
    "dbuser": "wordpress",
    "dbpass": "wordpress",
    "wpuser": "admin",
    "wppass": "admin",
    "wppath": "/vagrant/www/wordpress-default";
}

I think it’s fairly self explanatory, but here it goes:

  • environment is typically one of the values “development”, “staging” or “production”. A description of the environment that this installation is
  • url The url for this installation
  • dbhost, dbname, dbuser and dbpass are the database credentials. Wp-bootstrap assumes that a database already exists and is accessible using those credentials
  • wpuser, wppass are the credentials for the default WordPress admin user.
  • wppath is the path where WordPress will be installed.

APPSETTINGS.JSON

The last thing you need to add is an application settings file. The very minimum file you need to provide is:

{
    "title": "TestingComposer approach";
}

Title is the only mandatory field in this file, but it’s far from the only one.

Section: plugins:

This section consists of two sub arrays “standard” and “local”. Each array contains plugin names that should be installed and activated on the target WordPress site.

  • standard Fetches plugins from the official WordPress repository. If a specific version is needed, specify the version using a colon and the version identifier i.e if-menu:0.2.1
  • local A list of plugins in your local project folder. Plugins are expected to be located in folder projectroot/wp-content/plugins/. Local plugins are symlinked into place in the wp-content folder of the WordPress installation specified by wppath in localsettings.json

Section: themes

Similar to the plugins section but for themes.

  • standard Fetches themes from the official WordPress repository. If a specific version is needed, specify the version using a colon and the version identifier i.e footheme:1.1
  • local A list of themes in your local project folder. The themes are expected to be located in folder projectroot/wp-content/themes/. Local themes are symlinked into place in the wp-content folder of the WordPress installation specified by wppath in localsettings.json
  • active A string specifying what theme to activate.

Section: settings

A list of settings that will be applied to the WordPress installation using the wp-cli command “option update %s”. Currently only supports simple scalar values (strings and integers)

Using wp-bootstrap

The easiest way to use wp-bootstrap is to simply call the binary file that is added to your vendor/bin subfolder. Using the sample files above, you could do this:

# Install WordPress
$ vendor/bin/wpbootstrap wp-install

# alternate:
$ composer wp-install

After running this command, you should have a fully working WordPress installation, accessible via the url specified in your localsettings.json file. The title of the site should match whatever you specified as the title attribute in the appsettings.json file.

More Settings

So far we’ve managed to reproduce what wp-cli can do in three separate commands, good but perhaps not that great. The real power of wp-bootstrap lies in extending appsettings.json with some more settings. Here’s a slightly more advanced example:

{
	"title": "TestingComposer approach",
	"plugins": {
		"standard": [
			"if-menu:0.21",
			"baw-login-logout-menu",
			"wp-cfm",
			"google-analyticator",
			"wpmandrill"
		],
		"local": [
			"wordpressapp"
		]
	},
	"themes": {
		"standard": [
			"agama"
		],
		"active": "agama"
	},
	"settings": {
		"blogname": "New title 2",
		"blogdescription": "The next tagline"
	}
}

Using this file, we can customize the WordPress installation with a new command:

# Install plugins, themes etc.
$ vendor/bin/wpbootstrap wp-setup

# alternate
$ composer wp-setup

Let’s walk through this:

  • In the Plugins section, we specify a number of standard plugins that will be fetched, installed and activated from the WordPress plugin repository. The if-menu plugins will be installed with version 0.21, but the rest of that list will just be whatever version that is the latest in the repository
  • We also install and activate a local plugin that will be symlinked from [project root]/wp-content/plugins into the WordPress installation. This is so that we can include our own plugins developed specifically for this project.
  • The themes section works very similar, the theme agama is added to the installation from the WordPress theme repository and it’s also set as the active theme.
  • And finally, the settings “blogname” and “blogdescription” are overwritten with the values specified. These names corresponds to what the settings are called in the wp_options database table.

There are more settings…

If you’re curious enough to try this, I suggest you head on over to the github page for this project. It got some additional settings that you should read up on, especially if you’re keen to manage pages, menus and images in a similar fashion.

I’m also very curious to hear what you think about this, don’t hesitate to let me know in the comments. You can also reach out to me on Twitter with comments or questions.

 

WordPress DevOps – The book

I’ve written an ebook on this subject. Released in September this year on Leanpub.com. Expect a 100+ tightly written pages where we walk through the creation of the skeleton of a WordPress based Saas application, connected to Stripe and Paypal with a working deployment process that takes content into account. Just add your billion dollar idea. Jump on over to Leanpub to get your copy.

WordPress DevOps - Strategies for developing and deploying with WordPress

WordPress DevOps – Strategies for developing and deploying with WordPress

 

[wysija_form id=”3″]

post

WordPress configuration management

This is the 4th post in a series of developing for WordPress in a DevOps friendly way. The other articles:

  1. Introduction to WordPress and DevOps
  2. Developing with WordPress and Vagrant
  3. Grunt Automation for WordPress developer
  4. WordPress configuration management

In the previous posts in this series we’ve been looking at how to work with Vagrant and WordPress and how to automate setup work using Grunt and WP-CLI. In this posts, we’ll look a little bit on how we can transfer various settings between the WordPress database and into text files in the source code tree. Because as soon as we have the settings in files, we can easily transfer those settings to a target system such as a staging environment or the live site.

There’s a Git repository available with code for this post here: https://github.com/eriktorsner/wpconfigurationmanagement I suggest you clone or download the code to your development computer to make it easier to follow along.

WordPress settings

A Setting is any of the small or large things you change in the Settings menu in the WordPress admin area. The number of blog posts shown on the first page is a setting and so is the site title and the default date format.

Settings

Some of the well known settings from the WordPress admin area

 

WordPress uses the database to store settings into a table named wp_options (at least as long as you are using the standard wp_ prefix). Most of the settings that a core WordPress installation will put into the wp_options table are simple scalar values, a string or a numeric value. But some of the settings consists of larger more complex structs like an array or even arrays containing objects containing arrays. The internal WordPress code makes sure that pretty much anything you can represent in a PHP variable can be stored as a setting. If needed, WordPress uses the native PHP function serialize() to convert any given variable to a text representation suitable for database storage and then unserialize() to convert it back when needed.

Serializing

The concept of taking a complex structure and transforming it into a text representation is called seriallization, so no wonder the PHP functions are named as they are. In our ambitions to automate the setup of WordPress, we want to serialize some settings to a text file that we can keep in our source code repository. When we first install a development version of our WordPress site in a new environment, or when we want to deploy it to another environment, we want to take that text file with serialized settings and update the database. To do this we will continue to build on the WordPress skeleton from the previous posts, you’ll find it in the Github repository for this post. To get started with this code repository, please refer to the first post to learn a little bit about Vagrant and how we’re using it. A few notes:

  • This repo uses an evolved version of the Vagrantfile. Only the rows you need to change are present, everything else is moved in to a file in the vagrant/ sub folder.
  • No need to change anytning inside the vagrant/ sub folder, everything it’s taken care of during provisioning.

 We’re going to talk about two different approaches for managing settings in a text file. The first approach is using Gruntfile.js itself, the other approach uses the free plugin WP-CFM.

Keeping settings in Gruntfile.js

The most straight forward way would be to use Gruntfile and Wp-cli. The settings themselves will be in the form of Wp-cli statements that will be automatically inserted into the target WordPress installation when the Grunt task wp-setup is executed.


grunt.registerTask('wp-setup', '', function() {
	ls = getLocalsettings();
	wpcmd = 'wp --path=' + ls.wppath + ' --allow-root ';
	
	// some standard plugins
	stdplugins = ['google-analyticator', 'wp-cfm'];
	for(i=0;i<stdplugins.length;i++) {
		name = stdplugins[i];		
		shell.exec(wpcmd + 'plugin install --activate ' + name);
	}

	shell.exec(wpcmd + 'option update blogname "New blog title"');
	shell.exec(wpcmd + 'option update blogdescription "experimental tagline"');
	shell.exec(wpcmd + 'option update posts_per_page "20"');

})

The three highlighted rows will update the Site title (blogname), the tag line (blogdescription) and the ‘Blog pages show at most’ setting (posts_per_page) that are found in WordPress Admin. This is a very manual form of serializing settings to disk and it’s easy to get started with. If you only have a few of them that you need to set, this approach will work but it has a few downsides.

Obviously, the manual work involved in writing the wp-cli statements in your Grunt file is not very optimal. Easy to forget individual settings, easy to lose the overview etc. The other downside is that changes you make in WordPress admin has no way of automatically find their way into the Gruntfile. You’re in for a lot of manual work. But there is another way.

WP-CFM

WP-CFM is a free plugin available in the WordPress plugin repository. It’s name stands for WordPress Configuration Management and it does exactly that: Configuration Management. WP-CFM uses a concept call bundels. A bundle is a group of individual WordPress settings. WP-CFM can Push a bundle to file, meaning that it will read the value of each of the settings in the bundle from the database and store them in a file. And later, WP-CFM can Pull that file back in and restore those values in the database.

In the Gruntfile above I took the liberty of installing and activating wp-cfm in the WordPress installation in the Vagrant box, so if you’re following along you should already have it in WordPress. This plugin adds a menu item under settings named WP-CFM. The main WP-CFM screen looks like this:

WP-CFM admin screen

We want to create a bundle with the same three setting that we handled in the Grunt task above. So we’ll go ahead and click ‘Add Bundle’, name it ‘settings’ and select the individual settings items we need:

Creating a new bundle in WP-CFM

Creating a new bundle in WP-CFM (note, the posts_per_page setting is under the fold in this image)

After the bundle is created and saved, we can click ‘Push’ to store those three settings to disk. You should end up with a text file in /vagrant/www/wordpress-default/wp-content/config named settings.json:

{
    "blogdescription": "experimental tagline",
    "blogname": "New blog title",
    "posts_per_page": "20",
    ".label": "settings";
}

As you can see, WP-CFM uses a JSON file to store the individual values and stores the file in a predictable location. For fun, we can change the “blogname” setting inside this file and try the Pull button (you probably will need to reload the page for the Pull button to become active). Once the updated settings has been pulled into the database and you’ve reloaded the page, you should see that whatever value you selected for “blogname” now is in the page title.

A little automation

The one great thing with WP-CFM that I didn’t mention yet is that it was written with support for Wp-cli right out of the box. WP-CFM adds a command verb to Wp-cli named ‘config’ with sub commands push and pull. We just need to call these two commands from our Gruntfile to automate things. As usual,  a little bit of plumbing is also needed as well, best explained with code, so let’s have a look at the two new Grunt tasks that we’ve added for this post:


grunt.registerTask('wp-export', '', function() {
	ls = getLocalsettings();
	wpcmd = 'wp --path=' + ls.wppath + ' --allow-root ';
	pwd = shell.pwd();

	shell.mkdir('-p', pwd + '/config');

	// push settings from DB to file
	src = ls.wppath + '/wp-content/config/settings.json';
	trg = pwd + '/config/settings.json';
	shell.exec(wpcmd + 'config push settings');
	shell.cp('-f', src, trg);
});

grunt.registerTask('wp-import', '', function() {
	ls = getLocalsettings();
	wpcmd = 'wp --path=' + ls.wppath + ' --allow-root ';
	pwd = shell.pwd();

	shell.mkdir('-p', ls.wppath + '/wp-content/config');
	
	src = pwd + '/config/settings.json';
	trg = ls.wppath + '/wp-content/config/settings.json';
	shell.cp('-f', src, trg);
	shell.exec(wpcmd + 'config pull settings');
});

  • Line 53, define the task wp-export
  • Line 58, create the /vagrant/config folder unless it already exists
  • Line 63, run the wp-cli command ‘config push’ for the bundle named ‘settings’
  • Line 64, copy the settings file into our local folder /vagrant/config where it will be under source code control

 

  • Line 67, define the taks wp-import
  • Line 72, create the folder www/wordpress-default/wp-content/config unless it already exists
  • Line 76, copy the settings.json file from the project folder into the correct folder in the WordPress installation (where WP-CFM will expect to find it)
  • Line 77, run the wp-cli command ‘config pull’ for the bundle named settings so that each settings is pulled from the file into the database

Since the WP-CFM plugin was build for automation from the start, our work is reduced to managing the settings file so that we can keep in under source code control. With these two Grunt tasks, we’ve now automated the process for storing WordPress settings in the source code control system.

A few pitfalls

Using WP-CFM together with Grunt and wp-cli takes us a long way in terms of creating a fully automated process for managing WordPress.  However, this process isn’t really perfect yet. There are a few issues to look out for when it comes to WP-CFM:

  • Some settings might store the absolute URL for a post or page on the WordPress installation. If that setting is pulled from disk to the database unchanged, the live site might end up with URL references to the development version of the site
  • Some settings contains references to a page, a post or a menu item using the internal database id and the database ID might differ between the development version and the production version.

These two issues as well as the challenge of moving actual content (pages, images etc) from development to production is going to be covered in future posts in this series. So stay tuned!

 

WordPress DevOps – The book

I’ve written an ebook on this subject. Released in September this year on Leanpub.com. Expect a 100+ tightly written pages where we walk through the creation of the skeleton of a WordPress based Saas application, connected to Stripe and Paypal with a working deployment process that takes content into account. Just add your billion dollar idea. Jump on over to Leanpub to get your copy.

WordPress DevOps - Strategies for developing and deploying with WordPress

WordPress DevOps – Strategies for developing and deploying with WordPress

 

 

 

 

 

 

 

[wysija_form id=”3″]

post

Grunt Automation for WordPress developers

This is the third article in a series of developing for WordPress in a DevOps friendly way. The previous articles:

  1. Introduction to WordPress and DevOps
  2. Developing with WordPress and Vagrant
  3. Grunt Automation for WordPress developers
  4. WordPress configuration management

In the previous two posts we’ve had a look at the various tools we need to create an automated workflow for developing and a little bit about how Vagrant can be used to create a better development environment. We’ve also briefly touched on using Grunt and wp-cli to automate setting up WordPress inside the environment. In this post we’ll dive deeper into how to use Grunt to automate even more of the WordPress setup.

There’s a Git repository available with code for this post here: https://github.com/eriktorsner/gruntautomationforwp I suggest you clone or download the code to your development computer to make it easier to follow along.

What’s our goal?

In short. We want to run a single command at the command line to install WordPress, all our wanted plugins and other vital pieces of configuration. The end goal is to automate it to the extent that it can be part of a fully automated, non-interactive, deployment process.

Looking back at the first post in this series I talked a bit about DevOps and how it’s a bit of a challenge to make WordPress a good fit for it. Automation is one of the cornerstones within DevOps and historically WordPress has been more focused on getting up and running quick than on supporting an automated workflow.

Well, that changed entirely when wp-cli came onto the scene a few years back. Wp-cli stands for WordPress command line interface and is a project that aims to enable automation of WordPress management. Wp-cli makes it possible to automate installation, setup, theme and plugin installations and a lot of other common WordPress management tasks like modify settings or even add pages and posts.

The other part of the automation work will be carried out by Grunt. Grunt is a “modern JavaScript task runner” and allows us to create tasks that can be executed from the command line. So we can create a task “wp-install” that uses wp-cli to install WordPress and run that task from the command line using a simple “grunt wp-install”. The most common use case for Grunt is to run things like css and javascript minification or compile SASS file to css. But we’ll use Grunt mainly as a task runner for DevOps related tasks. That doesn’t stop you from using Grunt for those other things as well which may be a very good idea for your project.

Creating the environment

In the previous post, we covered how to get started with Vagrant. To follow along in this post I’m assuming that you’ve read that post and have installed the prerequisites for running Vagrant on your development computer. I’ve made a separate Github repository available for this post, so from now on when I’m talking about specific files, I’m referring to the ones you can find in the git repo at https://github.com/eriktorsner/gruntautomationforwp. If you have any problems following these steps, be sure to read up on the previous post and post any questions in the comments below if you’re still having issues.

A quick checklist to get you started with this Vagrant box

  1. Edit Vagrantfile (hostname, dns names and ip-numner) to fit in your environment
  2. copy localsettings.json.template to localsettings.json
  3. Run “vagrant up” to initialize the new machine”
  4. Go to folder /vagrant inside the virtual machine and run “npm install”

Wp-cli + Grunt

To show what Wp-cli can do in terms of automation, we’ll use it to perform the following tasks:

  1. Install WordPress
  2. Install and activate a few plugins from the official WordPress repo
  3. Change a few settings

As we’ll be doing this by automating wp-cli via Grunt. Let’s have a look at the important parts of the Grunt file that comes with this post (see the top of the post for information on how to get it). The first thing to notice is on 32 where we define the function getLocalSettings, a simple helper function.


function getLocalsettings(test) { 
    ls = grunt.file.readJSON('localsettings.json'); 
    if(ls.wppath === undefined) ls.wppath = shell.pwd() + '/www/wordpress-default'; 
    return ls; 
}

We’ll use this function in each of our Grunt tasks that deals with wp-cli so that we can get the parameters we define in localsettings.json into action. Let’s look at the settings file to get a hunch of what kind of parameters we’re including


{
    "environment": "development",
    "url": "www.gruntautomationforwp.local",
    "dbhost": "localhost",
    "dbname": "wordpress",
    "dbuser": "wordpress",
    "dbpass": "wordpress",
    "wpuser": "admin",
    "wppass": "admin",
    "wppath": "/vagrant/www/wordpress-default"
}

 

The idea is to have anything that is specific for this specific machine should be a parameter in this file. And the point of having settings in a json format is that json can be easily accessed both from javascript code like in the Gruntfile and in PHP code as well.

Let’s look at the first task defined, the default task:


grunt.registerTask('default', 'Log some stuff.', function() {
    ls = getLocalsettings();
    grunt.log.write('localsettings successfully read URL=' + ls.url).ok();
});

It will simply read the settings file and output the URL parameter from it, so it’s a simple test. Let’s try it within the Vagrant machine:

$ cd /vagrant
$ grunt
Running "default" task
localsettings successfully read URL=www.gruntautomationforwp.localOK

Done, without errors.

The next task to look at is the wp-install task, it’s a bit more interesting:

 /*
 * Installing WP
 */
 grunt.registerTask('wp-install', '', function() {
     ls = getLocalsettings();
     wpcmd = 'wp --path=' + ls.wppath + ' --allow-root ';

     shell.mkdir('-p', ls.wppath);

     if(!shell.test('-e', ls.wppath + '/wp-config.php')) {
         shell.exec(wpcmd + 'core download --force');
         shell.exec(wpcmd + 'core config --dbname=' + ls.dbname + ' --   dbuser=' + ls.dbuser + ' --dbpass=' + ls.dbpass + ' --quiet');
         shell.exec(wpcmd + 'core install --url=' + ls.url + ' --title="WordPress App" --admin_name=' + ls.wpuser + ' --admin_email="admin@local.dev" --admin_password="' + ls.wppass + '"');
     } else {
         grunt.log.write('Wordpress is already installed').ok();
     }
 });
  • Line 18 is just loading the localsettings as we saw above
  • Line 19 creates the base string for the wp-cli command. Note that we’re adding the switch –allow-root since this task is likely to be executed by root in a different environment like production. We’re also adding the –path switch which will instruct wp-cli to install WordPress in a sub folder.
  • In line 23 we’re checking if WordPress is already installed or not.
  • Lines 24 to 26 are executing the wp-cli commands “core download”, “core config” and “core install”.

There are a few interesting things to note. The first is obviously that after running this task we’re going to have a fully functioning WordPress installation in a sub folder. All parameters we needed are kept in a separate file which is fundamentally good for our efforts to make this a fully automated WordPress install. We’ve also managed to install WordPress in a sub folder that is explicitly kept out of source code control.

The reason for that is that WordPress in not our code, it’s a depencency to us. We want to avoid adding dependencies in our source code tree (look inside .gitignore, we’re explicitly ignoring the www sub folder) for several reasons. But the most important reason is not size or bloat, it’s the ability to handle upgrades well. WordPress and it’s extensions are upgraded frequently with major and minor upgrades. If you add all the source code for WordPress, themes and plugins into your own source code tree, you will be stuck with the task of upgrading them once new versions arrive, and they do quite often.

A clean separation between your own code and the dependencies means that you can easily test your code with the newest versions of each dependency. If your code breaks as a result of upgrading one of the third party plugins, the clean separation and automated setup makes it super easy to test and fix any such problems before you upgrade anything in production. That’s a big deal.

So, back to our example, let’s run the wp-install task:

$ grunt wp-install
Running "wp-install" task
Downloading WordPress 4.3.1 (en_US)...
Success: WordPress downloaded.
sh: 1: /usr/sbin/sendmail: not found
Success: WordPress installed successfully.

Adding more dependencies

Obviously, we’re going to need more dependencies. Let’s add a few plugins from the WordPress plugin repository. This is done with the wp-cli command “plugin install”. Let’s look at the next task in Gruntfile.js:

/*
* Setting up WP
* 
*/
grunt.registerTask('wp-setup', '', function() {
    ls = getLocalsettings();
    wpcmd = 'wp --path=' + ls.wppath + ' --allow-root ';
	
    // some standard plugins
    stdplugins = ['if-menu', 'baw-login-logout-menu','google-analyticator'];
    for(i=0;i<stdplugins.length;i++) {
        name = stdplugins[i];		
        shell.exec(wpcmd + 'plugin install --activate ' + name);
    }
})
  • Lines 37-38 are similar to the wp-install task
  • Line 41 sets up an array of plugins we want to install
  • Lines 42-45 iterates that array and creates a wp-cli command that installs and activates all the plugins in the array.

So no more manual plugin installations in the WordPress admin area! Let’s run the Grunt wp-setup task:

$ grunt wp-setup
Running "wp-setup" task
Installing If Menu (0.3)
Downloading install package from https://downloads.wordpress.org/plugin/if-menu.zip...
Unpacking the package...
Installing the plugin...
Plugin installed successfully.
Success: Translations updates are not needed for the 'English (US)' locale.
Activating 'if-menu'...
Success: Plugin 'if-menu' activated.
Installing Login Logout Menu (1.3.3)
Downloading install package from https://downloads.wordpress.org/plugin/baw-login-logout-menu.zip...
Unpacking the package...
Installing the plugin...
Plugin installed successfully.
Success: Translations updates are not needed for the 'English (US)' locale.
Activating 'baw-login-logout-menu'...
Success: Plugin 'baw-login-logout-menu' activated.
Installing Google Analyticator (6.4.9.6)
Downloading install package from https://downloads.wordpress.org/plugin/google-analyticator.6.4.9.6.zip...
Unpacking the package...
Installing the plugin...
Plugin installed successfully.
Success: Translations updates are not needed for the 'English (US)' locale.
Activating 'google-analyticator'...
Success: Plugin 'google-analyticator' activated.

Done, without errors.

Working with settings

The last part I’m going to cover in this post is a little bit on how to manipulate WordPress settings. Wp-cli is not the only option we have to get settings from a text file (code) into a live WordPress installation, but since wp-cli can do this, let’s have a look at how.

Most settings in WordPress are stored in a table named wp_options (assuming you’ve kept the standard “wp_” table name prefix). In order to change any of those settings via wp-cli, we first need to figure out what the various settings are called. Wp-cli have a sub command “option” that lets us list, add, delete or modify settings. For instance, if we want to change the settings “Site title” and “Tagline” we could use wp-cli to list all options and search for the options we want to find. In the wp-install task, we set the string “WordPress App” as the title, lets search for it:

$ cd /vagrant/www/wordpress-default/
$ wp option list | grep 'WordPress App'
blogname	WordPress App

Ok, there’s only one single row in the options table with that string, so we can safely assume that the option called “blogname” corresponds to Site title. Let’s look for the tagline, we haven’t changed it yet in our test installation, so it should still read “Just another WordPress site”:

$ wp option list | grep 'ust another WordPress site'
blogdescription	Just another WordPress site

Right. The parameter tagline that we can control in the WordPress admin is found in the option named “blogdescription”.

Knowing what the options are called makes it possible to set them via wp-cli. We can add a few lines to the wp-setup task:

    shell.exec(wpcmd + 'option update blogname "Another title"');
    shell.exec(wpcmd + 'option update blogdescription "experimental tagline");

Running the wp-setup task again, we should now see:

$ grunt wp-setup
Running "wp-setup" task
Warning: if-menu: Plugin already installed.
Activating 'if-menu'...
Warning: Plugin 'if-menu' is already active.
Warning: baw-login-logout-menu: Plugin already installed.
Activating 'baw-login-logout-menu'...
Warning: Plugin 'baw-login-logout-menu' is already active.
Warning: google-analyticator: Plugin already installed.
Activating 'google-analyticator'...
Warning: Plugin 'google-analyticator' is already active.
Success: Updated 'blogname' option.
Success: Updated 'blogdescription' option.

Done, without errors.

Note that wp-cli won’t attempt to reinstall the plugins that are already activated but instead just sends a warning saying that it’s already there. The last few lines tells us that the blogname and blogdescription settings are now updated to their new values. Mission accomplished.

 

Summary

In this post, we’ve taken another few steps towards gettings WordPress under our control. We’ve discussed the reasons why we want to treat WordPress itself as well as any third party extensions as dependencies rather than a part of our own application. Not in the sense that the code we develop would work well without WordPress and the plugins, but because we need to keep them out of our code base for upgrading and testing reasons.

In future installments in this series, we’ll dive into dealing with content and try to find ways to make WordPress separate content and content. If that sounds weird or if you’re just the curious anyway, stay tuned for the next episode. In the mean time, share insights, questions or other feedback in the comments below.

WordPress DevOps – The book

I’ve written an ebook on this subject. Released in September this year on Leanpub.com. Expect a 100+ tightly written pages where we walk through the creation of the skeleton of a WordPress based Saas application, connected to Stripe and Paypal with a working deployment process that takes content into account. Just add your billion dollar idea. Jump on over to Leanpub to get your copy.

WordPress DevOps - Strategies for developing and deploying with WordPress

WordPress DevOps – Strategies for developing and deploying with WordPress

 

 

 

 

 

 

 

[wysija_form id=”3″]

eBook on WordPress development and deployment

Advanced WordPress development

Last weeks I’ve been busy finishing up an eBook with the kind of straight forward title: WordPress DevOps – Strategies for developing and deploying with WordPress It’s a 100 page guide covering how You can get WordPress to work a little bit better in a proper development process, covering automation, testing and deployment.

If you’re interested, click here or on the book cover above to head on over to Leanpub and sign up for it. You’ll get an email as soon as it’s available for download. And yes, it will be available in pdf, epub and mobi formats. When you do sign up, please share your email address with me, I’d love to be able to stay in touch with you.

post

WordPress management – looking for beta testers

logo2_300x200

One of the projects I’m working on is called Remote Control Panel for WordPress and is getting ready for beta testing. Or to be honest, we’re still in the middle of functional testing so we could still be a couple of weeks away. But we still think it’s time to start recruiting beta testers for this service. Our goal is to launch the beta test first half of February.

So what is Remote Control Panel?

Remote Control Panel is a freemium service that will help users manage multiple WordPress installations. In plain English, that means that our service will:

  • Do continuous backups of your WordPress site, all files and the complete database.
  • Keep backups for a year. Allowing you to restore from any given point in time
  • Keep track of all your plugins and themes and alert you when there’s an upgrade available.
  • Provide a one-click interface to upgrade plugins and themes on all your WordPress sites at the same time.

What will I get for testing it?

Well, we hope that the most important thing we deliver is your next favorite tool for backup and management, but apart from that, all our Beta testers that provide feedback will have the first 12 months free after official launch. Even if we’re planning to offer a free entry level version of the product with limited storage capacity, the premium version will offer more storage and more frequent backups. At the very least, you’ll get $179 (14.95/month) worth of premium backup service for helping us out with the beta testing.

What do I need?

At this point, we’re not picky at all. If you’re the admin of one or more WordPress installations, you’re welcome to apply for the beta testing program.

Where do I sign up?

Head on over to https//remotecontrolpanel.net to sign up for the beta test. Many thanks in advance.

A slow host is a bad host

Just a quick post about a discovery I just made. When using the Load Time Profiler plugin to measure load time, it’s possible to compare two hosting environments by looking at the first part of the load process, before any plugins or theme files are loaded. The last point such point measured by the plugin is post_exec_register_theme_directory that is added to the profiling data just before plugins are about to get loaded.

On a really fast host running Apache2 without any PHP enhancements but with plenty of RAM and fast SSD disks, this event is done 18.4 ms into the process. This host happens to be my laptop. But please do note, the load time is measured entirely on the server side, so there’s no advantage of being on the same physical computer.

On a really (really) slow host, a business account on Swedish hosting company Binero, the same exact point is reached after 220 ms(!). I’ve reported this to their support, but they seem to think that it’s completely in order so it’s not a matter of a single misconfigured server. They really are that slow. As a reference, a similar hosting provider, Loopia, is about 25% quicker at 165 ms.

220 versus 18.4, that’s more than a ten times the load time. Actually, when looking at the total load time, including plugins and the theme, the Binero hosted account will sometimes consume more than 10 seconds before the page is sent back to the browser. Yes, we’re looking at moving that site to another home in the near future and being able to profile the load time behaviour was exactly the kind of data point needed to make that decision. With that kind of performance, there’s simply no other quick fix available.

Now, go test your hosting provider and please let me know what you find in the comments below. Download the plugin here.

 

post

WordPress profiler

memory_added

This post is part two of a mini series where I (1) explain why keeping track of WordPress load timing is important and (2) discuss how WordPress load time profiling can be done using a newly created plugin. If you want to skip the chatter and go straight to the plugin, you can download it here.

I’ve updated my load time profiling plugin a little bit over the weekend. There are now two new features:

  1. The output will show you memory consumption at each log point
  2. The plugin will trace time spent in each function called via an ‘add_action’ callback.

Lets talk about both these two a bit and why they make sense

Memory consumption

There are a few reasons that you want your WordPress installation to have as low memory consumption as possible.

First of all, there is a limit. Sometimes on some hosts, you will get an error that tells you that your PHP script tried to allocate more memory than is available and it will halt execution. This is not a specific WordPress error, but since WordPress is built on PHP, you’ll get it sometimes. When troubleshooting this type of error, it’s not always obvious what code that is causing the problem. Even if Plugin A uses a lot of memory and pushes memory consumption right to the limit, it will sometimes load just fine anyway. But a little later, when Plugin B is initializing, your total memory consumption may hit the limit and processing will halt. When that happens, finding the problematic piece of code can be very time consuming, not to mention frustrating.

The other reason is that allocating memory pretty much always takes a lot of time, memory allocation is simply an expensive operation. Not that each call to emalloc in itself is all that expensive, but when a script gradually built up memory consumption there have usually been quite a few emalloc calls under the hood. So the mere act of consuming memory also costs a lot of CPU cycles.

The last reason is a matter of scalability. A script that uses 256Mb of memory will occupy the server RAM with… well, 256Mb. If your webserver only have 1024Mb available it can only make room for four 256Mb scripts running at the exact same time. If the script on average takes 2 seconds to finish it means that the maximum requests/sec that your server can handle will be heavily reduced. So you want your WordPress requests to consume as little RAM memory as possible simply because it means that the server can fit more requests into memory at any given point of time.

All the three above reasons makes it important to be able to track how your WordPress installation uses memory. I hope you find it useful.

Hooked functions

One of the features that makes WordPress such a wonderful product is the ability for a theme or plugin to hook into it. In practical terms it means that a plugin or theme can request to be notified via a callback whenever WordPress hits a certain point in it’s execution, in WordPress lingo, such a callback is referred to as an action. For instance, most plugins uses a hook for the Init event that fires early in the startup process and allows a plugin to perform certain initialization before the page starts to render. Plugins and themes can register that they want a callback for the Init event by simply calling the add_action function. More than one plugin can hook up to the same event and when this happens WordPress will just execute them one by one.

A potential problem with callbacks is that all plugins are not written with performance and memory efficiency in mind. On the contrary, some plugins will spend ages doing expensive initialization tasks and consuming a lot of memory for no particular reason at all. For the end user this means that the requested page is delayed until some plugin is done doing unneeded work. One example is the very popular NexGen Gallery plugin that on initialization will load a lot of PHP files into memory and perform some quite expensive initialization work on every single request, regardless if the resulting web page will actually display a gallery.

In the first version of my Load time profiler plugin you could track how long time WordPress took to complete a few of the actions, but it was not possible to see how long each individual callback would take. In this version, I’ve added code to track this so that for instance when the Init action is fired, you can see the time consumed by each individual callback (click image to enlarge)

init_breakdown

One tiny drawback

The traditional way to do profiling would be to run the code in a specific profiling tool that needs to be installed on the same server as your code is running on. In the case of WordPress, the challenge is that many production installations are running in hosted environments where the web developers can’t just go and install debugging and profiling extensions at will. This plugin is really an attempt to overcome this limitation by actually modifying how WordPress is executed in a controlled way. Three core WordPress files are copied to the plugin directory and are modified to (1) collect data on timing and memory consumption and (2) alter the include paths a little to avoid the standard version of those files. Feel free to check the modified files, they are stored under /path/to/plugins/loadtimeprofiler/resources and have the suffix _custom.php

The drawback of this should be fairly obvious, since the code is modified, it’s not the exact same as in production and therefore, the results will differ ever so slightly between what’s reported and reality. So as long as you keep this in mind and use the tool sensibly, I think you can draw some

Download

I swear I will start the process to get the plugin into the official repository at wordpress.org asap, in the meantime click here to download the current version of the plugin.

I’m eager to hear back from your. I’ll answer technical questions or listen to your critique. Whatever feedback you think is appropriate. Use the comments below to let me know what you think.

 

WordPress load time analysis

Load time profiler ‹ Walltool — WordPress

UPDATE Dec 8th 2013: I’ve updated the plugin even more. Read about it in part 2.

UPDATE Dec 3rd 2013: While testing the plugin on a few installations, I discovered a couple of bugs. If you downloaded the plugin on Dec 2nd 2013, you may want to try the improved version available below.

I’ve been working a lot with WordPress lately and doing so I’ve sometimes needed to trouble shoot slow WordPress installations. Naturally, I’ve googled for things like “wordpress performance”, “wordpress is too slow” and similar phrases. Almost always, those searches will take me to a page listing the top-10 things you should do to speed up your WordPress website. Those lists of tips are usually quite good, you’ll typically learn that you should install a caching plugin, clean up your database and get rid of unneeded plugins etc. Mostly quite useful information…. but with some important exceptions.

Installing a caching plugin such as W3 Total Cache will often solve most of your performance problems as long as you have mostly a “read only” website. However, if your website is inherently slow when used without any caching, your users will notice it and suffer from it whenever they do anything that can’t be cached as easily, like posting comments or doing custom searches. The most scary example  I’ve experienced was an e-commerce website that was screaming fast as long as you were just browsing products (good) but was terribly slow when adding things to the cart (bad), entering a coupon code (worse) or checking out (flat out terribly bad). In that case, it didn’t matter much that the front page got a 90/100 score on Google Speed since the users had a terrible experience just as they were about to place their orders. So sometimes, a WordPress site need performance beyond what you can get from a good caching plugin.

Maybe it’s time for a little clarification. When people talk about performance it’s common that a few important concepts are mixed up. For the remainder of this post, when I say performance I mean the time needed to create the main HTML document that WordPress generates and that IS the page. Specifically, I don’t talk about the size of or number of other resources (images, css-files, js-files etc) that your page needs. Those are almost always static and the webserver will seldom require the PHP  scripting engine to serve them to your users, and if your webserver actually is too slow, static files can easily be moved to a CDN anyway.

When a user requests a page from (or makes an Ajax call to) your Wordpres site and that request can’t be cached, the entire WordPress system needs to initialize in order to generate the response. For the sake of argument, let’s divide the response generation process into two phases:

  1. Internal WordPress init. In this phase. WordPress will load itself into the PHP process memory, it will set up the database connection and read various options from the database. It will initialize your theme, meaning it loads the theme’s function.php into memory and allow it to register the hooks it want. It will also load the main php-file from all your active plugins and they will also typically register some hooks.
  2. Theme processing When phase 1 is ready, your theme will be called into action. If it’s a standard post or page, Wordpres will typically load your single.php or page.php and let it execute it’s code. That code will call internal WP functions such as ‘wp_head()’ or get_bloginfo() to do it’s job and sometimes, calling an internal function will call back into your theme’s function.php, it all depends on the theme.

During these two phases, you really don’t have a lot of information about what WordPress spends the most time with and that makes it terribly difficult to know what you can do to optimize it. I found this very frustrating so at one point, I started hacking a few of the files in WordPress core in to insert debug information about what it’s doing. Specifically, I was inserting profiling code into wp-settings.php like this:

require( ABSPATH . WPINC . '/version.php' ); // <--- existing line
$profiledata['post_require_version.php'] = microtime(TRUE); // <--- my line

Then in my footer.php, I’d output the content of the $profiledata array into HTML to review it. Naturally, modifying core files like that gets messy quite fast. You can easily break the entire site by accident  and any changes will be overwritten on the next WordPress upgrade. So even if I believe that I was gathering useful information this way, I needed to do it cleaner. So, (since this is WordPress) I wrote a plugin, I now have a working plugin that will do the following using a bunch of Ajax calls from an admin-page:

  1. Copy wp-settings and wp-config into a separate plugin folder
  2. Rewrite wp-settings and wp-config so that they contain profiling code as shown above and some modified path information.
  3. Via a special bootstrap php file, load wordpress using the modified versions of wp-settings and wp-config and at the end, output the profile data using json
  4. Submit the data back to the server for storage, and render an HTML table with all the profiling data.

This is what it looks like:

Load time profiler ‹ Walltool — WordPress

And the complete content of the analysis can be downloaded here (tab separated text)

One interesting thing is that this is not limited to just displaying internal WordPress load time analysis, if you want to, you can add additional profiling statements into your own theme code and have them displayed in the same table. I’d even say it’s the preferred thing to do. One thing that I’ve seen so far is that in a lot of cases, the WordPress internal stuff takes roughly 40-55% of the total load time, so to get the full story of what happens during load time.

I’ve just finished writing this plugin so it’s really quite rough at the moment. It’s not been tested with a lot of WordPress versions and the UI is not at all what I’d want it to be. I’m in the process of adding it to the WordPress repository, should you want to look at it before that, feel free to download it from here:

Download Loadtimeprofiler

Feedback, comments questions? Use the comments below.

 

 

post

WordPress file permissions

 

In order for WordPress to be able to install a plugin and plugins or themes automatically there are a number of conditions that have to be met. If all those conditions aren’t met, one-click installations or upgrades won’t happen, instead, whenever you try to upgrade, WordPress will show you the FTP credentials input form. If you’re anything like me, you hate it.

I sometimes run into this problem. My first instinct is to check the obvious file permissions. Will the web server have write access to all the important places. As long as we’re only talking about plugins and themes, important places means the wp-content folder. When I’m certain that the web server have write access, I typically try again and successfully upgrade my plugin.

Every once in a while, installing or upgrading still won’t work even if I’m 100% certain that WordPress should be able to write everywhere it needs to. I end up searching for a solution for about 10 minutes, give up and resort to manually uploading plugins via ssh and get on with my life. Today I decided to find out the root cause of this problem and solve it. Writing this blog post about it servers as much as a ‘note to self’ as assistance to anyone else that trouble shoots this without finding a solution.

The rules

So, the rules for WordPress to be able to install and upgrade plugins and themes:

  1. The web server needs to have write access to the wp-content folder. For example on a Debian based system (i.e Ubuntu), this will be user ‘www-data’, on RedHat/Fedora, it’s typically user ‘httpd’ (plese correct me here if I’m wrong). WordPress will test this by writing a temporary file to wp-content and then remove it. There are plenty of blog posts, howtos and forum posts about this. They usually points back to this article: http://codex.wordpress.org/Changing_File_Permissions
  2. The files in wp-admin needs to be owned by the web server user. WordPress will test this by using PHP function getmyuid() to check if the owner of the currently running PHP script is that same as the owner of the newly created temporary file. If it’s not the same, WordPress will select another method of installation or upgrade.

Rule #2 is what typically gets me. Whenever I move an existing WordPress installation to a new home, I’m sometimes (obviously) not careful with setting file permissions and file ownership and end up in this situation. Rule #1 is extremely intuitive, checking for write permission is close to second nature. But Rule #2,  checking file ownership in wp-admin… well, I’d even say it’s slightly unintuitive. If anything is worth protecting it should be the admin area, and being more restrictive with file ownership and permissions under wp-admin would even kind of make sense.

Anyway. Comments, questions or other feedback. Please post a comment below.

 

[wysija_form id=”3″]