Write Better Plugins and Themes with Codeception

If you’re a developer, then you know that one of the quickest ways to damage your reputation and loose users is releasing a faulty theme or plugin. I regularly meet developers who are very confident of the quality of their work, but who take no action to actually verify it. And, honestly?, I have something to confess—I was one of them too! Luckily, I learned we can do much, much better… provided we use the right tools.

Today, I’d like to present you Codeception, a PHP testing framework that will help you guarantee (up to some extent) the quality of your work. I’ve been using Codeception for the past four months and I came to love it. In fact, a few weeks back I even shared my experience with Codeception in WordCamp Europe 2015!

The Importance of Quality

Quality matters. We don’t buy poor-quality things, but those that are great! Now, if I asked you what the heck is “quality“, what would you answer?

According to the Merriam-Webster dictionary:

How good or bad something is.
A high level of value or excellence.

In my opinion, that doesn’t help too much. Since this is a WordPress-related blog, what about taking a look at the Codex and looking for an answer there?

WordPress Coding Standards
WordPress Guidelines for plugins and themes
Plugin and Theme Reviews

That’s more helpful! But I think there’s more of it.

For me, quality is the idea of being good, looking nice, feeling right, working properly… Can we address all these aspects when writing a plugin? Of course! And not only we can, but we must. High quality plugins (and themes) should:

Work as expected.
Have no bugs or issues.
Be perfectly integrated in WordPress.
Follow the coding standards and recommended guidelines.
Be efficient.
And probably more!

The Development Cycle

When creating a plugin from scratch, or when editing a plugin to add some new functionalities, we always follow the same process:

We have a set of (new) requirements.
We write some code to implement one of those requirements.
We try it out.
If it didn’t work as expected, we fix it and go back to 3.
Once things work properly, we keep adding new code until all the requirements are met.
We check that the overall behavior of the plugin is as expected.
We release the plugin.
BAM! We have a bug.

The most important step in the previous cycle is the sixth. Right before releasing the new version of the plugin, we have to manually check that things work as expected. I used to do that at Nelio, and the results were not always successful. But what does “check” meaning?

Sometimes, adding new code to a plugin or refactoring some old code means that things that used to work will not longer work. For instance, a login screen might not let users login anymore, or a JavaScript file might contain a syntax error that we missed, or we forgot to change the URL of our backend servers from Test to Live. Whatever the error is, you should be able to detect it before the new version is released. In other words, it should be you, the developer, the one who detects the error, not your users!

In Nelio, we used to have a tiny checklist of all the things that had to be tested before each release of our plugin. During the first versions of the plugin, the list was pretty small and, with just a few minutes of testing, we could assert that everything was just fine. But as the plugin grew in complexity, so did the checklist, and manually testing everything quickly became a pain in the ass.

Hard Work by AlmostInfamous — Work can be hard sometimes… Picture by AlmostInfamous.

We needed to test our work faster and in a more reliable way. How? The answer lied in Automated Tests.

Automated Tests

An Automated Test checks that a certain functionality works as expected without any manual intervention. If we create multiple tests, then we have a test suite that can be run by our testing software.

Automated Tests usually have three stages:

Arrange. We setup the environment, so that when the test is run, the initial state is properly set.
Act. We perform one or more actions that will change the initial state.
Assert. Finally, we simply need to compare the result we obtained from steps 1 and 2 with the result we expected.

Let’s see an example. Suppose we have a plugin with a form that asks users for their telephone number. A telephone number should be a set of digits and/or dots. As the developers of the plugin, we added the following PHP function:

function validateTelNumber( $num ) {
  return preg_match( '/^[0-9]+$/', $num );
}

which tests whether a certain $num is a valid telephone number. Now, a possible test would be the following one:

// Arrange
$num = ‘934123456’;
// Act
$result = validateTelNumber( $num );
// Assert
assertTrue( $result );

If we execute the previous test, it will PASS—that is, everything will be just fine. The telephone number is a set of digits, which is exactly what the regular expression of the validateTelNumber function accepts.

If, for some reason, we now change the function in a way such that it no longer accepts the telephone number 934123456, the test will fail and we’ll quickly know something went wrong. Great, isn’t it? Actually, the idea of testing is not new. If you can be confident that your last release/commit didn’t break your plugin, you’ll probably sleep better 😉

Unit Testing

Unit Testing is a method for verifying that individual units of source code are fit for use. The example I presented above is a Unit Test. The test checks that a single function returns the expected result given a certain input.

And the great part is: WordPress already uses Unit Testing extensively! The Core of WordPress has plenty of unit tests that ensure the quality of a base WordPress installation.

Now, Testing does not cover all the scenarios. It’s possible that a certain component passes all the tests we’ve written and, yet, it contains a bug. For instance, in the previous example we said that “a telephone number should be a set of digits and/or dots”, which means that the telephone number 934.123.456 should also be valid. Let’s write the test for that:

// Arrange
$num = ‘934.123.456’;
// Act
$result = validateTelNumber( $num );
// Assert
assertTrue( $result );

Yet, if we run this test, it will FAIL. Why? Because the regular expression we coded only accepts digits as a telephone number (the dot chars are not accepted). Had we written this test before, we’d have discovered the bug earlier and we’d have fixed it before releasing!

The great thing about automated tests is that, even though we missed that detail, we now have two tests that check (hopefully) all the possible scenarios! From now on, tests have our back 🙂

Unfortunately, Unit Testing only covers some of the issues our users might face. Unit Tests are focused on individual units and the internal structure of our code. They tell us nothing about the plugin as a whole. Do the components work altogether? Do we satisfy users’ expectations?

Nelio Unlocker

Switch to WordPress safely while keeping your design and content

Improve your SEO today and boost your site speed by converting your pages into HTML, CSS, and WordPress standards. With zero technical knowledge required, you’ll only pay for what you need.

Try It Now

Functional and Acceptance Testing

Functional and Acceptance Tests go one step further and test the plugin (or application) as a whole. They replicate users’ and developers’ experience and are driven by expectation logic.

This type of tests are part of a movement called Behavior-Driven Development (BDD). As we can read in the previous link:

Behaviour-driven development is an “outside-in” methodology. It starts at the outside by identifying business outcomes, and then drills down into the feature set that will achieve those outcomes. Each feature is captured as a “story”, which defines the scope of the feature along with its acceptance criteria.

I really recommend you read that article thoroughly. It’s extremely well written and gives some very useful insights. There, you’ll learn that, essentially, BDD provides a structure for a story. Stories are the first step towards appropriate testing, and should contain all of the elements described in the following template:

Story Title
As a role
I want to feature
So that benefit
Acceptance Criteria
Scenario Name
Given context
When event
Then outcome

For example, in WordPress you have the following Story and Acceptance Criteria:

Story An author (John) publishes a post
As an author (John)
I want to publish a post
So that new content is available in my blog
Acceptance Criteria
Scenario 1 – John has a draft post
Given a draft post in the Dashboard
and the post’s author is John
and the post’s title is X
When John accesses the posts page in the Dashboard
and publishes the post
Then the post’s status is no longer Draft
and the Latest Posts page contains X

As you can now see, this type of tests follow the same schema we presented before: (a) arrange, (b) act, and (c) assert. Now, what’s great about them is the fact that stage (b) consists in replicating users’ actions. Therefore, as we shall see in just a few lines, writing acceptance tests is super easy and, moreover, they can be easily understood by our customers (thus adding more value to our work!).

Functional Tests lie in between Unit Tests and Acceptance Tests (but, in my opinion, they’re closer to Acceptance Tests). With them, you’ll reproduce developers’ experience. That is, you’ll be able to interact with your plugin as a regular user (or developer testing it) would, but you’ll have finer access to some lower details (such as HTTP requests, reading the database or the file system, and so on).

Codeception

We just saw that there are plenty of ways to test a plugin. The most popular (which we reviewed first) is Unit Testing. It’s popular because it’s fast to implement and super fast to test. Unfortunately, Unit Tests are not enough for guaranteeing that your plugin is working. To test the behavior of your application as a whole, you should write, as described above, functional or acceptance tests.

Codeception is a PHP Testing Framework designed to work just out of the box. Codeception covers the three types of tests we just reviewed:

Unit Tests,
Functional Tests, and
Acceptance Tests

Let’s see how the WordPress post publishing example we saw before would look like in Codeception:

<?php
$I = new AcceptanceTester();
// Let's assume we're logged in
$I->amOnPage( '/wp-admin/index.php' );
$I->click( 'Posts' );
$I->moveMouseOver( '#the-list tr:first-child' );
$I->click( 'edit' )
$I->waitForElementVisible( 'Publish' )
$I->click( 'Publish' )
$I->waitForElementVisible( '#misc-publishing-actions' );
$I->see( 'Status: Publish', '#misc-publishing-actions' );
$I->amOnPage( '/blog' );
$I->see( '...' );

Once the test is ready, we can execute it using Codeception and check the results:

Automated Tests with Codeception — Example of an automated test being executed by Codeception.

Codeception offers you several actions for interacting with the browser:

amOnPage
click
fillField
selectOption
submitForm
checkOption
and many others

as well as several actions for asserting the results we expect:

see
seeInLink
seeInTitle
seeInField
dontSee
…

If you want to learn more about what Codeception offers, just take a look at their documentation. It’s very helpful and also a great starting point.

Using WebDriver

Another interesting thing that can be covered with Codeception is available thanks to WebDriver.

The WebDriver API [is] a platform and language-neutral interface and associated wire protocol that allows programs or scripts to introspect into, and control the behaviour of, a web browser. The WebDriver API is primarily intended to allow developers to write tests that automate a browser from a separate controlling process, but may also be implemented in such a way as to allow in-browser scripts to control a—possibly separate—browser.
WebDriver Spec at W3C

Using Codeception along with WebDriver, you’ll be able to run your tests on real browsers. In other words, you’ll be able to start a new instance of Firefox, Chrome, or Internet Explorer and control them from your tests! This is great, because now you can test real behavior on real drivers.

I find it specially useful when one of our users opens a support ticket and reports a bug that only occurs on a specific browser (actually, you can create acceptance tests for all the issues a user reports, so that you can make sure that this issue is never ever introduced again in your plugin 😀 ). Just follow this steps:

A user reports an issue.
Try to reproduce the error locally.
Once you confirmed the issue, create an acceptance test. You know the test will fail (there’s an issue, right?) but, anyway, write it.
Fix the issue.
Run the test and… voilà! Now it should PASS, and you can be more confident about the quality of your plugin.

WebDriver, however, has some cons you have to remember. The major caveat is its speed—starting and stopping real browsers for each test takes its time, and real browsers are much slower than simulated browsers (such as PhpBrowser or PhantomJS). Also, you’ll need to have the browsers installed in your testing machine for running the tests, and sometimes this is simply impossible (running Internet Explorer on a Linux machine? Not happening!).

Some Tips and Tricks

Finally, I’d like to share a few tips and tricks I learned along the way. If you use them from the very beginning, they’ll certainly save you some precious time in the future:

PageObjects. As you might foresee, writing tests that interact with a user interface entails some complication, especially when there are some “tiny” changes (such as changing element IDs or classes). In order to overcome this issue, you can use PageObjects, which add a layer between your tests and the actual User Interface. If the UI changes, you simply need to update the references in the PageObject and your tests will continue working.
StepObjects. When the interactions are more complex, or when there’s a set of actions that are repeated over and over again, you can use StepObjects. Things like logging in and out from WordPress, publishing a (simple) post, and stuff like that, can be defined in a StepObject for easing its (re)usage.

Remember

Good code is rare. Automated tests are even rarer. The key to excellence? Seek for the former using the latter! Always test your work and go for the next level of quality.

A couple of weeks ago I covered our experience in WordCamp Europe where I shared my presentation about Codeception. Anyway, here it is again:

Featured Image by Dystopos.