Testing images

Cekit is able to run behave based tests for images. We suggest you read the Behave documentation before reading this chapter.

An image can be tested by running:

$ cekit test

Test options

  • --test-wip – only run tests tagged with the @wip tag.
  • --test-steps-url – a git repository url containing steps for tests.
  • --tag altname – overrides the name of the Image used for testing to altname. Only the first occurrence of this argument is honoured.
  • --test-name – part of the Scenario name to be executed

About Tests

Behave tests are defined in two separate parts: steps and features.

You can place the files defining tests in a tests directory next to the image descriptor, module descriptor or in a root of a git repository which contains the modules.

The tests directory is structured as follows:

tests/features
tests/features/amq.feature
tests/steps
tests/steps/custom_steps.py

The tests/features directory is the place where you can drop your behave features.

The tests/steps directory is optional and contains custom steps for the specific image/module.

We strongly recommend that a test is written for every feature that is added to the image.

Cekit comes with a list of build-in steps that are available for use in tests. See the steps repository.

Where necessary we encourage people to add or extend these steps.

Tags

Cekit selects which tests to run via the tags mechanism. Here are several examples of ways ways that tags could be used for managing tests across a set of related images:

  1. Product tags

    Tags based on image names. Cekit derives two test tag names from the name of the Image being tested. The whole image name is converted into one tag, and everything before the first ‘/’ character is converted into another. Example: If you are testing the jboss-eap-7/eap7 image, tests will be invoked with tags @jboss-eap-7 and @jboss-eap-7/eap7.

    If --tag is specified, then the argument is used in place of the Image name for the process above. Example If you provided --tag foo/bar, then the tags used would be @foo and @foo/bar.

  2. Wip tags

    This is very special behavior used mainly in development. Its purpose is to to limit the tests to be run to a subset you are working on. To achieve this you should mark your in-development test scenarios with the @wip tag and run cekit test --test-wip. All other scenarios not tagged @wip will be ignored.

  3. The @ci tag

    If cekit is not running as a user called jenkins, the tag @ci is added to the list of ignored tags, meaning any tests tagged @ci are ignored and not executed.

    The purpose of this behavior is to ease specifying tests that are only executed when run within Jenkins CI.

Running specific test

Cekit enables you to run specific Scenario only. To do it you need to run Cekit with --test-name <name of the tests> command line argument.

Example: If you have following Scenario in your feature files:

Scenario: Check custom debug port is available
When container is started with env
| variable   | value |
| DEBUG      | true  |
| DEBUG_PORT | 8798  |
Then check that port 8798 is open

Then you can instruct Cekit to run this test in a following way:

$ cekit test --test-name 'Check custom debug port is available'

Note

--test-name switch can be specified multiple times and only the Scenarios matching all of the names are executed.

Skipping tests

If there is a particular test which needs to be temporally disabled, you can use @ignore tag to disable it.

For example to disable following Scenario:

Scenario: Check custom debug port is available
When container is started with env
| variable   | value |
| DEBUG      | true  |
| DEBUG_PORT | 8798  |
Then check that port 8798 is open

You need to tag it with @ignore tag in a following way:

@ignore
Scenario: Check custom debug port is available
When container is started with env
| variable   | value |
| DEBUG      | true  |
| DEBUG_PORT | 8798  |
Then check that port 8798 is open