Monday, May 7, 2018

Multi-Platform BDD with Cucumber and Appium (Part 2)

Previously, on Battlestar Galactica...

In the last post, we set up a simple BDD test for an Android app. We defined some general BaseSteps that allowed us to look for text on the screen and click it. In this exercise we are going to abstract this out so that we can perform the same test on multiple platforms using Guice injection.

Source code for this version with the changes from Part 1 is available on the WITH_GUICE branch on GitHub.

As you saw in the last article, the biding between step definitions and feature files is handled via Java annotations. By adding the cucumber-guice library we can provide a platform strategy that will let us define these common step definitions across platforms based on execution time parameters.

If you are like me, you are going to be wondering about the decisions that have been make here from an architecture standpoint. So let's cover some of the weirdness here...

  1. cucumber-guice doesn't seem to know how to introspect abstract classes for step definitions. You need a concrete class for that.
  2. If you have two concrete classes with matching @Override methods, it is going to yell at you about duplicate step definitions.
  3. I have elected to overcome these difficulties by using a "Strategy" pattern. We have preserved our BaseSteps class from the previous example, but now it simply delegates calls to a strategy implementation @Inject-ed when it is created.
  4. We're going to spend more time looking at design patterns for BDD tests in the next post, but for now we just want to add muti-platform support to our testing scenario.
All that said, let's get started...

The first thing we want to do us update our dependencies in the integration module from the previous example we are adding two lines here:

dependencies {
    cucumberRuntime 'info.cukes:cucumber-java:1.2.5'
    cucumberCompile 'info.cukes:cucumber-java:1.2.5'
    cucumberCompile 'info.cukes:cucumber-guice:1.2.5'
    cucumberCompile 'com.google.inject:guice:4.2.0'
    cucumberCompile 'junit:junit:4.12'
    cucumberCompile 'io.appium:java-client:3.3.0'
}


The cucumber-guice dependency is new. Because, for some FSM damned reason, the cucumber-guice module doesn't have a declared dependency on Guice, we also add a dependency on Guice. I really got nothing here. You have to do it or it just won't work.

Now we have the option to inject dependencies into our step definitions. If all you want is some simple state definitions, you can do that with the regular javax.inject annotation, or the Cucumber annotations, but we will get to that. Right now, though, we want to change our execution based on platform, so we need to modify our build.gradle to pass a new parameter to the test execution.

cucumber {
    formats = ['pretty', 'json:build/cucumber.json', 'junit:build/cucumber.xml']
    jvmOptions {
        maxHeapSize = '512m'
        environment 'apk', rootProject.project(":app").buildDir.getAbsolutePath() +
                "/outputs/apk/debug/app-debug.apk"
        environment 'platform', System.getProperty("platform") == null ?
                "android" :
                System.getProperty("platform");
    }
}

So we are just going to read a system property here and default to android if it is undefined, then pass that value to the execution environment for Cucumber.

Now, we have a bit of boilerplate to create. Again going to interesting design decisions, cucumber-guice doesn't allow you to simply declare modules to include. However, you can provide a factory class for the Guice injector it will use. Just so you understand the subtleties of this, lets be clear about what is happening here: Cucumber scans the classpath for method implementations with a step definition (@Given, @When, @Then), and then with cucumber-guice, asks the Injector for an implementation of that class. This is why you can't have multiple simple implementations of a step definition floating around, and why we have opted for a strategy pattern to provide multi-platform implementations. Again, we will look at this in more detail in the next post. All this hand-waving aside, we need to create an InjectorSource implementation:

public class ConfiguredInjectorSource implements InjectorSource {
    @Override
    public Injector getInjector() {
        return Guice.createInjector(CucumberModules.SCENARIO,
           new CucumberModule());
    }
}

When we create our Guice injector, we need to include the SCENARIO module. This defines a @ScenarioScoped annotation that we can use later when creating our injected classes. We aren't going to use it for this exercise, but it is required to bootstrap cucumber-guice. Here we have added this default module and our module.

Finally, we need to declare the CucumberModule class. We are going to read the environment variable we defined in our Gradle file and do an implementation swap.

public class CucumberModule extends AbstractModule {

    private static final Logger LOGGER = Logger.getLogger(
       CucumberModule.class.getCanonicalName()
    );
    @Override
    protected void configure() {
        String platform = System.getenv("platform");
        LOGGER.info("Configuring run for platform: "+platform);
        switch(platform){
            case "android":
                bind(BaseStepsStrategy.class)
                   .to(AndroidBaseSteps.class)
                   .in(Singleton.class);
                break;
            case "ios":
                bind(BaseStepsStrategy.class)
                   .to(IOSBaseSteps.class)
                   .in(Singleton.class);
                break;
            default:
                throw new RuntimeException(
                   "Unknown platform environment variable: " +
                   platform);
        }
    }
}


This feels like a crazy number of lines for what we are doing here, but you know, cope. We read the environment variable, we log it so the user knows what is going on, then we do a switch around the BaseStepsStrategy to provide an implementation.

This is all well and good, but our test execution is still going to depend on BaseSteps from the previous exercise. So what we are going to do is rework this class to delegate to our strategy implementations, declare itself as a @Singleton, and get the strategy implementation injected into it.

@Singleton
public class BaseSteps {
    private final BaseStepsStrategy strategy;
    @Inject
    public BaseSteps(BaseStepsStrategy baseSteps){
        this.strategy = baseSteps;
    }

    @Given("I have launched the application")
    public  void startApp() throws IOException{
        strategy.startApp();
    }

    @When("I click the \"(.*)\" button")
    public void clickByText(String text){
        strategy.clickByText(text);
    }

    @Then("the \"(.*)\" is gone")
    public void assertMissing(String text){
        strategy.assertMissing(text);
    }
}

What about all the code we had from the previous exercise? Well, we're gonna copy and paste that into our AndroidBaseSteps class, but first, we're gonna create a generic abstract strategy here.

public abstract class BaseStepsStrategy<T extends WebDriver> {

    private T driver;
    BaseStepsStrategy() throws MalformedURLException {
        this.driver = createDriver();
    }

    protected abstract T createDriver() throws MalformedURLException;
    T getDriver(){
        return this.driver;
    }

    public abstract void startApp() throws IOException;
    public abstract void clickByText(String text);
    public abstract void assertMissing(String text);
}

I know that WebDriver seems like a weird thing to extend from here, but MobileDriver for Appium actually extends from WebDriver -- I guess no one has gone back and just made "AppDriver" as a thing from the Selenium group.  Because we want the "driver" to continue to be a singleton in the runtime, we have kicked it out to a createDriver() method. The reason is obvious when we create our AndroidBaseSteps class.

public class AndroidBaseSteps extends BaseStepsStrategy<AndroidDriver> {

    public AndroidBaseSteps() throws MalformedURLException {
        super();    }

    @Override
    protected AndroidDriver createDriver() throws MalformedURLException {
        File app = new File(System.getenv("apk"));
        DesiredCapabilities capabilities = new DesiredCapabilities();
        capabilities.setCapability("deviceName","Android Emulator");
        capabilities.setCapability("app", app.getAbsolutePath());
        capabilities.setCapability("appPackage", "net.kebernet.appium_cucumber");
        capabilities.setCapability("appActivity", ".MainActivity");
        return new AndroidDriver<>(new URL("http://127.0.0.1:4723/wd/hub"),
           capabilities);
    }

    @Override
    public void startApp() throws IOException {
        getDriver().resetApp();
    }

    @Override
    public void clickByText(String text) {
        getDriver().findElementByAndroidUIAutomator(
                      "new UiSelector().textContains(\""+text+"\")")
                    .click();
    }

    @Override
    public void assertMissing(String text) {
        MobileElement element = null;
        try {
            element = (MobileElement) getDriver()
               .findElementByAndroidUIAutomator(
                   "new UiSelector().textContains(\"" + text + "\")");
        } catch(NoSuchElementException e){
            //expected exception;
        }
        assertTrue(element == null);
    }
}


Aside from a bit of boilerplate, this looks pretty much exactly like out BaseSteps class from the first example. We are creating the driver, by delegation, in the constructor, but since this is a generic implementation, we can use getDriver() everywhere and know we are starting with an AndroidDriver. We do have a new cast in the assertMissing() method, but that is a small price to pay.

We are now back to a "known good" state for our application and tests. Now, let's look at making our "integration" suite work with iOS.

We will start with our iOS app, that is basically the same as our Android app...

Next we need to implement our IOSBaseSteps class. This is going to look largely like our Android version we already have, only this time we are going to use the XPath selector. We are also adding a new configuration to the environment configuration in the build.gradle to pass a path to our .app build.

public class IOSBaseSteps extends BaseStepsStrategy<IOSDriver<MobileElement>> {


    public IOSBaseSteps() throws MalformedURLException {
        super();
    }

    @Override
    protected IOSDriver<MobileElement> createDriver() throws MalformedURLException {
        File app = new File(System.getenv("app"));
        DesiredCapabilities capabilities = new DesiredCapabilities();
        capabilities.setCapability(MobileCapabilityType.PLATFORM_NAME, "iOS");
        capabilities.setCapability(MobileCapabilityType.PLATFORM_VERSION, "11.3");
        capabilities.setCapability(MobileCapabilityType.DEVICE_NAME, "iPhone Simulator");
        capabilities.setCapability(MobileCapabilityType.APP, app.getAbsolutePath());
        return new IOSDriver<>(new URL("http://127.0.0.1:4723/wd/hub"), capabilities);
    }

    @Override
    public void startApp() throws IOException {
        getDriver().resetApp();
    }

    @Override
    public void clickByText(String text) {
        MobileElement element = getDriver().findElementByXPath(
              "//*[contains(@label, '"
              + text + "')]");
        element.click();
    }

    @Override
    public void assertMissing(String text) {
        MobileElement element = null;
        try {
            element = getDriver().findElementByXPath(
               "//*[contains(@label, '"
               + text + "')]");
        } catch(NoSuchElementException e){
            //expected execption;
        }
        assertNull(element);
    }

And our change to the build.gradle file:

cucumber {
    formats = ['pretty', 'json:build/cucumber.json', 'junit:build/cucumber.xml']
    jvmOptions {
        maxHeapSize = '512m'
        environment 'apk', rootProject.project(":app").buildDir.getAbsolutePath() +
                "/outputs/apk/debug/app-debug.apk"
        environment 'platform', System.getProperty("platform") == null ?
                "android" :
                System.getProperty("platform")

        environment 'app', rootProject.projectDir.getAbsolutePath() +
                "/ios/DerivedData/appium-cucumber/Build/Products/" +
                "Debug-iphonesimulator/appium-cucumber.app"
    }
}

Now we can run our sample tests on the iOS app:

$../gradlew -Dplatform=ios cucumber



> Task :integration:cucumber 
Gradle now uses separate output directories for each JVM language, but this build assumes a single directory for all classes from a source set. This behaviour has been deprecated and is scheduled to be removed in Gradle 5.0
May 07, 2018 12:10:39 PM cucumber.inject.CucumberModule configure
INFO: Configuring run for platform: ios
Feature: Click the button
  Clicking buttons is clever

  Scenario: I see a button and click it.  # features/hello.feature:4
    Given I have launched the application # BaseSteps.startApp()
    When I click the "Click Me" button    # BaseSteps.clickByText(String)
    Then the "Click Me" is gone           # BaseSteps.assertMissing(String)

1 Scenarios (1 passed)r
3 Steps (3 passed)
2m37.075s



BUILD SUCCESSFUL in 2m 57s

Friday, May 4, 2018

BDD For Android Hello World


Today I am going to demonstrate a very basic bootstrap of BDD for Android using Cucumber and Appium.

You can find all of the code on GitHub, natch.

The first step is to get Appium installed and working. I found this to be the best startup instructions, but it is a little old, so YMMV.

First, the app. We have a very simple Android app here with a button. You click it, and it goes away.


Now, we want to test our app. We need to set up a new Gradle module parallel to our application project, as the plugin environment we need to make this work isn't going to play well with the Android plugin. Here I have called the project "integration" to indicate this for performing integration testing.

Lets break down the build.gradle real fast:

Apply our plugins:

buildscript {
    repositories {
        maven {
            url "http://repo.bodar.com"
        }
        maven {
            url "https://plugins.gradle.org/m2/"
        }
    }
    dependencies {
        classpath "com.github.samueltbrown:gradle-cucumber-plugin:0.9"
    }
}
plugins {
    id 'java'
    id "com.github.samueltbrown.cucumber" version "0.9"
    id 'idea'
}

Next we need to make sure our app gets built for the Cucumber plugin to run.

tasks.cucumber.dependsOn(":app:assembleDebug")


And point the cucumber plugin to the debug build of the application, so we can find it. Here we are going to pass an environment variable pointing to the APK file we want to test.

cucumber {
    formats = ['pretty', 'json:build/cucumber.json', 'junit:build/cucumber.xml']
    jvmOptions {
        maxHeapSize = '512m'
        environment 'apk', rootProject.project(":app").buildDir.getAbsolutePath() +
                "/outputs/apk/debug/app-debug.apk"    
     }
}

repositories {
    jcenter()
    maven {
        url 'https://repository-saucelabs.forge.cloudbees.com/release'
    }
}

Next we set up the dependencies we need for the Cucumber environment. These come in as two new Gradle configuration scopes. Finally, we are just going to use a little config magic to make sure IntelliJ IDEA can properly resolve the dependencies while we are editing these things.

dependencies {
    cucumberRuntime 'info.cukes:cucumber-java:1.2.5'
    cucumberCompile 'info.cukes:cucumber-java:1.2.5'
    cucumberCompile 'junit:junit:4.12'
    cucumberCompile 'io.appium:java-client:3.3.0'
}

idea {
    module {
        testSourceDirs += file('src/cucumber/java')
        scopes.TEST.plus.add(configurations.cucumberCompile)
    }
}

sourceCompatibility = "1.8"
targetCompatibility = "1.8"

Cucumber allows us to write test directives using a natural language format. The idea here is that the natural language syntax is easier to maintain as your requirements change. Our sample .feature file in src/cucumber/resources looks like so:


Feature: Click the button
  Clicking buttons is clever

  Scenario: I see a button and click it.
    Given I have launched the application
    When I click the "Click Me" button
    Then the "Click Me" is gone

The first line is the name of the feature we are testing with this file. The second line is just a descriptor. It is then followed by 1..n "Scenarios" that are basically sequences of test steps. Here, the scenario is named "I see a button and click it."

Next we have a number of steps. These are in the format of "Given/When/Then". Given steps are basically there to establish the baseline for the test. Here, we are just launching the app, but you could have a "Give" that navigates to a page, or logs in, or whatever.  When steps are basically your interaction operations. Then steps are your assertion operations.

These steps, however, are going to require some code to work. So lets get into that. Cucumber is going to look for your steps to be defined in some Java classes with methods annotated with the step definition. You can use regular expressions to pull data from the step line, as well as some other {} -type templating.

So let's look at the first one: "Given I have launched the application." This is a simple method call which we annotate with the @Given annotation...


public class BaseSteps {
    protected AndroidDriver<MobileElement> driver;
    @Given("I have launched the application")
    public void startApp() throws IOException {
        File app = new File(System.getenv("apk"));
        DesiredCapabilities capabilities = new DesiredCapabilities();
        capabilities.setCapability("deviceName","Android Emulator");
        capabilities.setCapability("app", app.getAbsolutePath());
        capabilities.setCapability("appPackage", "net.kebernet.appium_cucumber");
        capabilities.setCapability("appActivity", ".MainActivity");
        driver = new AndroidDriver<>(new URL("http://127.0.0.1:4723/wd/hub"),
                   capabilities);
        driver.resetApp();
    }

Here we start by reading the apk environment variable we configured in the Gradle file, configure the DesiredCapabilities object, and create an AndroidDriver.  For prophylactic reasons, we do a resetApp() call to just kill and restart the app so we know we have a good state. This will connect to the Appium node.js server, which in turn will install the apk file on the running emulator and spin it up.

Next we want to click our button, so let's look at that one.

@When("I click the \"(.*)\" button")
public void clickByText(String text){
    driver.findElementByAndroidUIAutomator(
              "new UiSelector().textContains(\"" +
              text + "\")"
           ).click();
}

Here we are selecting the text inside the quotation marks using a regex selector. Then we take the text and go to the driver and use findElementByAndroidUIAutomator() to select it. Here we are constructing a small Java snippet in a String to perform the UI selection on the device, then we call click() to perform the action.

Finally, we want to validate the behavior: that the button goes away. To do that we look for the button again, but expect it to be gone...
@Then("the \"(.*)\" is gone")
public void assertMissing(String text){
    MobileElement element = null;
    try {
        element = driver.findElementByAndroidUIAutomator(
               "new UiSelector().textContains(\"" + 
                text + "\")"
               );
    } catch(NoSuchElementException e) {
        //expected exception;
    }

    assertTrue(element == null);
}

If the driver fails to select the element, it will throw a NoSuchElementException, but we want that to happen here.

Finally we run our feature suite by doing gradle cucumber:


Executing tasks: [cucumber]

Configuration on demand is an incubating feature.
:app:preBuild UP-TO-DATE
:app:preDebugBuild UP-TO-DATE
:app:compileDebugAidl UP-TO-DATE
:app:compileDebugRenderscript UP-TO-DATE
:app:checkDebugManifest UP-TO-DATE
:app:generateDebugBuildConfig UP-TO-DATE
:app:prepareLintJar UP-TO-DATE
:app:mainApkListPersistenceDebug UP-TO-DATE
:app:generateDebugResValues UP-TO-DATE
:app:generateDebugResources UP-TO-DATE
:app:mergeDebugResources UP-TO-DATE
:app:createDebugCompatibleScreenManifests UP-TO-DATE
:app:processDebugManifest UP-TO-DATE
:app:splitsDiscoveryTaskDebug UP-TO-DATE
:app:processDebugResources UP-TO-DATE
:app:generateDebugSources UP-TO-DATE
:app:javaPreCompileDebug UP-TO-DATE
:app:compileDebugJavaWithJavac UP-TO-DATE
:app:compileDebugNdk NO-SOURCE
:app:compileDebugSources UP-TO-DATE
:app:mergeDebugShaders UP-TO-DATE
:app:compileDebugShaders UP-TO-DATE
:app:generateDebugAssets UP-TO-DATE
:app:mergeDebugAssets UP-TO-DATE
:app:transformClassesWithDexBuilderForDebug UP-TO-DATE
:app:transformDexArchiveWithExternalLibsDexMergerForDebug UP-TO-DATE
:app:transformDexArchiveWithDexMergerForDebug UP-TO-DATE
:app:mergeDebugJniLibFolders UP-TO-DATE
:app:transformNativeLibsWithMergeJniLibsForDebug UP-TO-DATE
:app:processDebugJavaRes NO-SOURCE
:app:transformResourcesWithMergeJavaResForDebug UP-TO-DATE
:app:validateSigningDebug UP-TO-DATE
:app:packageDebug UP-TO-DATE
:app:assembleDebug UP-TO-DATE
:integration:compileJava NO-SOURCE
:integration:processResources NO-SOURCE
:integration:classes UP-TO-DATE
:integration:jar UP-TO-DATE
:integration:assemble UP-TO-DATE
:integration:compileTestJava NO-SOURCE
:integration:processTestResources NO-SOURCE
:integration:testClasses UP-TO-DATE
:integration:compileCucumberJava
:integration:processCucumberResources UP-TO-DATE
:integration:cucumberClasses
:integration:cucumber
Gradle now uses separate output directories for each JVM language, but this build assumes a single directory for all classes from a source set. This behaviour has been deprecated and is scheduled to be removed in Gradle 5.0
Feature: Click the button
  Clicking buttons is clever

  Scenario: I see a button and click it.  # features/hello.feature:4
    Given I have launched the application # BaseSteps.startApp()
    When I click the "Click Me" button    # BaseSteps.clickByText(String)
    Then the "Click Me" is gone           # BaseSteps.assertMissing(String)

1 Scenarios (1 passed)
3 Steps (3 passed)
0m12.263s


BUILD SUCCESSFUL in 16s
30 actionable tasks: 2 executed, 28 up-to-date
1:18:07 PM: Task execution finished 'cucumber'.

Wednesday, April 11, 2018

MQTT Sucks (First Post)

So I saw that HackADay is having a chat about MQTT.

That seems as good as anything for an inaugural post on.

TL;DR: MQTT sucks.

The problem with the spec, as it is, is a simple one: there isn't enough spec there.

1. Authentication


With the spec basically having the equivalent of Basic Auth as your only option, every MQTT endpoint system ends up coming up with some idiot way to work around it. Sometimes it is an SSL key, sometimes it is a bearer token (put in either of the user/pass fields chosen at random). Unlike HTTP, MQTT is supposed to be about limited devices, but by leaving so much open to interpretation, you are basically burning firmware for a particular devices that is going to be nailed to a particular MQTT endpoint implementation.

2. Message Format


Much like authentication, the lack of any real delineation of message format in the spec means that everything does something different. If you want to use ThingsBoard, you need one channel hierarchy and message format for a device. If you want to use Google or Amazon's, you have to use their particular message format. Again, you are nailing your device to a particular endpoint implementation, or you end up having to do an MQTT -> MQTT transformation layer for anything that you want.

3. The Current Crop of Servers


As noted at the end of the last bit, unless you want to be nailed to one particular way of doing things, you end up having do build a proxy layer to adapt from one MQTT to another MQTT. Of course, though, none of the current endpoint implementations support doing this from within the server, so you have to cobble together something on your own, which is just ridiculous. Even Google and Amazon that already have powerful data pipeline tools don't let you accept a message and then transform it within their data pipeline tools; the message has to be perfectly formatted before they will receive it in the first damned place.

We really need a new spec, or at least a clarification of the original that deals with the following:

  1. Authentication. AuthN should really be done with encryption keys, but the protocol needs a way for a client to request provisioning access from an endpoint with a pre-existing key, rather than provisioning a key on an endpoint and having to put it onto a device. 
  2. We need a standard format for at least a channel specification. That specification should include the concept of "this" as a hierarchy param, an separate channels that imply sequenced data vs configuration or updates to static data. That is:
    • /clients/this/configuration <- {ipAddress="10.10.10.10" name="weatherstation1"}
    • /clients/this/sequence <- { "temperatureK"="300.15" }
    • Other standard channels could be defined, but it should be assume that each device should listen on /clients/this/configuration for provisioning and configuration updates.
    Then when someone wants to subscribe to events from "weatherStation1" they can subscribe to /clients/[provisioned synthetic id]/sequence.
  3. While MQTT as simple TCP and over websockets is already pretty standard, it makes no sense at all to me that IP Multicast isn't an option. Given the generally restrictive network environments lots of IoT type systems want to play in, it makes much more sense for a hub device on a given network environment to be able to grab messages out of "the air" and store and forward them onto a next leg network system.
MQTT isn't the only IoT protocol going on, but it is the one with the most traction right now. It is just too incompatible between environments to be REALLY useful. It wouldn't take much at all to get some standards around channel/message structure to solve most of these problems.