How to write Robot Framework tests for Plone

Require plone.app.robotframework¶

Update setup.py to require plone.app.robotframework:

                        extras_require={
   'test': [
        ...
        'plone.app.robotframework',
    ],
},

                      

All you need is plone.app.robotframework. It will require the rest (selenium, robotframework, robotframework-selenium2library and also robotsuite).

Note

Selenium-bindings for Python use Firefox as the default browser. Unless you know how to configure other browsers to work with Selenium you should have Firefox installed in your system

Define functional testing layer¶

Plone add-on testing requires defining a custom test layer, which setups Plone-sandbox, the dependent add-ons and any custom configuration required by the tests.

Update your src/my/product/testing.py to include:

                        from plone.testing import z2
from plone.app.testing import FunctionalTesting
from plone.app.robotframework.testing import AUTOLOGIN_LIBRARY_FIXTURE

MY_PRODUCT_ROBOT_TESTING = FunctionalTesting(
   bases=(MY_PRODUCT_FIXTURE, AUTOLOGIN_LIBRARY_FIXTURE,
          z2.ZSERVER_FIXTURE),
    name="MyProduct:Robot")

                      

Note

AUTOLOGIN_LIBRARY_FIXTURE is optional, but it will allow you to write faster Selenium tests, because tests don't need to spend time on login forms. Also note that the order of the bases matters.

If you don't have any testing layers for your product yet, or want to know more about them, please read plone.app.testing-documentation.

Install Robot-tools¶

plone.app.robotframework ships with two main helper scripts for writing tests:

bin/robot-server starts a temporary Plone site with the given test layer set up
bin/robot executes Robot Framework's pybot-runner so that it will run the given test suite against the running robot-server, ensuring that tests will be run in isolation (database is cleaned between the tests)

Update buildout.cfg:

                        [buildout]
parts =
    ...
    robot

[robot]
recipe = zc.recipe.egg
eggs =
    Pillow
    ${test:eggs}
    plone.app.robotframework

                      

Note

Robot-tools are optional, but will ease and speed up your test development.

Start test server¶

Once the buildout with Robot-tools is run, start the test server with:

$ bin/robot-server my.product.testing.MY_PRODUCT_ROBOT_TESTING

Once the test server has started, there should be a test Plone-site served at http://localhost:55001/plone/ (by default). This allows you to play with the sandbox while writing the tests.

Note

The default admin user for plone.app.testing-based Plone-sandbox is admin and password is secret.

Write your first test suite¶

Robot tests are written in test suites, which are plain text files, usually ending with .robot (and older ones with .txt).

The first test can be written anywhere in the filesystem.

For example, a test_hello.robot:

                         *** Settings ***

  Force Tags  wip-not_in_docs

  Resource  plone/app/robotframework/selenium.robot

  Test Setup  Open test browser
  Test Teardown  Close all browsers

*** Test Cases ***

  Plone is installed
    Go to  ${PLONE_URL}
    Page should contain  Powered by Plone

Robot is all about running test clauses called keywords (or, to be more exact, keyword calls with parameters). Every test case may contain one or more keywords, which are run sequentially -- usually until the first of them fails. Keywords are separated from their arguments (and arguments from each other) using at least two spaces.

Keywords are defined in keyword libraries and as user keywords. Keyword libraries can be Python libraries or XML-RPC-services. User keywords are just lists of test clauses reusing existing keywords or other user keywords. User keywords are described in the test suite, or imported from resource files.

Here is a more complicated example with some user keywords in action:

                        *** Settings ***

 Force Tags  wip-not_in_docs

 Resource  plone/app/robotframework/selenium.robot

 Library  Remote  ${PLONE_URL}/RobotRemote

 Test Setup  Open test browser
 Test Teardown  Close all browsers

 *** Variables ***

 ${ADMIN_ROLE}  Site Administrator

 *** Test Cases ***

 Site Administrator can access control panel
     Given I'm logged in as a '${ADMIN_ROLE}'
      When I open the personal menu
      Then I see the Site Setup -link

 *** Keywords ***

 I'm logged in as a '${ROLE}'
     Enable autologin as  ${ROLE}
     Go to  ${PLONE_URL}

 I open the personal menu
     Click link  css=#user-name

 I see the Site Setup -link
     Element should be visible  css=#personaltools-plone_setup

                      

Please, stop for a while end read the example above again. Once you understand how you can stack keyword calls with user keywords, you are ready to unleash the power of Robot Framework all the way to building your own domain specific test language.

Note

We use .robot as the Robot Framework test suite file extension to make it easier for developers to configure Robot Framework syntax highlighting for their editors (otherwise .txt would work also).

Run your first test suite¶

Once the bin/robot-server has been started and a test suite has been written, the new test suite can be run with bin/robot:

$ bin/robot test_hello.robot

Note

bin/robot is mostly just a wrapper for Robot Framework's pybot test runner, but it does inject necessary options to enable plone.testing's test isolation for Plone when used together with bin/robot-server.

Integrate with Zope-testrunner¶

It's often convenient to run Robot tests with other Plone tests (e.g. on Jenkins or Travis-CI). To achieve that, we integrate Robot tests to be run with other tests so that all tests can be run with zope.testrunner.

For zope.testrunner integration, create src/my/product/tests/test_robot.py:

                        import unittest

import robotsuite
from my.product.testing import MY_PRODUCT_ROBOT_TESTING
from plone.testing import layered


def test_suite():
    suite = unittest.TestSuite()
    suite.addTests([
        layered(robotsuite.RobotTestSuite('test_hello.robot'),
                layer=MY_PRODUCT_ROBOT_TESTING),
    ])
    return suite

                      

Note

For this to work and zope.testrunner to discover your robot test suite, remember to move test_hello.robot under my/product/tests.

RobotSuite is our package for wrapping Robot Framework tests into Python unittest compatible test cases. It's good to know that this registration pattern is the same as how doctest-suites are registered to support zope.testrunner's layers (see https://pypi.python.org/pypi/plone.testing for layered doctest examples).

Running tests with zope.testrunner¶

Once your robot test have been integrated with zope.testrunner using test_robot.py-module (or any other module returning RobotTestSuite), you can list your integrated robot test cases with command:

$ bin/test --list-tests

And run robot tests cases with all other test cases with command:

$ bin/test

You can filter robot test using -t-argument for zope.testrunner*:

$ bin/test -t robot

And it's also possible to filter test by Robot Framework tags:

$ bin/test -t \#mytag

Or exclude matching tests from being run:

$ bin/test -t \!robot

How to write more tests¶

The most difficult part in writing robot tests with Selenium-keywords is to know the application you are testing: which link to click when and to which field to input test data.

At first, you should have a brief idea about the available keywords:

Then, learn to use pause test execution to make it easier to figure out, what to do next:

                        *** Settings ***

Force Tags  wip-not_in_docs

Resource  plone/app/robotframework/selenium.robot

Library  Remote  ${PLONE_URL}/RobotRemote

Test Setup  Open test browser
Test Teardown  Close all browsers

*** Test Cases ***

Let me think what to do next
    Enable autologin as  Site Administrator
    Go to  ${PLONE_URL}

    Import library  Dialogs
    Pause execution

Robot Framework ships with a few selected standard libraries. One of them is the Dialogs-library, which provides a very useful keyword: Pause execution. By importing Dialogs-library (while developing the test) and adding the Pause execution keyword, you can pause the test at any point to make it possible to figure out what to do next. (Dialogs depend on TkInter-library.)

Note

Be sure to remove Import libary and Pause execution keyword calls before committing your tests to avoid pausing your tests on CI.

Note

plone.app.robotframework ships with an optional collection of Plone-specific user keywords, which already include Pause keyword as a shortcut for Pause execution keywords. You can include and use the collection with:

                          *** Settings ***

Force Tags  wip-not_in_docs

...

Resource  plone/app/robotframework/keywords.robot

*** Test Cases ***

Let me think what to do next
    ...
    Pause

How to write Robot Framework tests for Plone¶

Require plone.app.robotframework¶

Define functional testing layer¶

Install Robot-tools¶

Start test server¶

Write your first test suite¶

Run your first test suite¶

Integrate with Zope-testrunner¶

Running tests with zope.testrunner¶

How to write more tests¶