Actions and Asserts

Tests in Cicada are composed of Actions and Asserts. Each test cycle proceeds as follows:

  1. Run each action in the actions list
  2. Run each assert in the asserts list
  3. If there are remaining asserts, repeat the cycle without the ones that have passed until all have passed or a timeout is reached

This cycle can be tweaked to better suit your needs. See the API for more details

Actions

Actions are used to make something happen to a service in situations where we don't necessarily care about the result.

Suppose we wanted to create 100 Jeff's:

description: Actions example
tests:
  - name: add-jeff
    description: Creates 100 Jeffs
    runner: rest-runner
    actions:
      - type: POST
        executionsPerCycle: 100
        params:
          url: "http://api:8080/members"
          body:
            name: Jeff

Each action call made to a rest-runner will return the following body:

{
  "status_code": 200,
  "headers": {
    "Content-Type": "application/json",
    "Content-Length": "33"
  },
  "body": {
    "id": 1,
    "name": "jeff"
  },
  "text": "{\n  \"id\": 1, \n  \"name\": \"jeff\"\n}\n",
  "runtime": 19.056
}

A list of result for each execution will get stored in the state container at state['add-jeff-100']['POST0']['results']

Outputs

Outputs are used for storing additional data for an action.

Let's say we wanted to determine the average runtime per action execution:

description: Actions example with outputs
tests:
  - name: add-jeff-100
    description: Creates 100 Jeffs
    runner: rest-runner
    actions:
      - type: POST
        executionsPerCycle: 100
        params:
          url: "http://api:8080/members"
          body:
            name: Jeff
        outputs:
        # Used to add state outside of runners 
        - name: average-runtime
          template: >
            {% set runtimes = [] %}
            {% for result in results %}
              {% do runtimes.append(result['runtime']) %}
            {% end %}
            value: {{ runtimes|sum / runtimes|count }}

This will add an output named average-runtime under state['add-jeff-100']['actions']['POST0']['outputs']

Notice that the template uses a special variable called results. This refers to the results generated by the current action, since it isn't added to the state container until after the action finishes

Asserts

Asserts are essentially actions where the runner will check the results

This example makes sure that 100 Jeff's were added to the database:

description: Asserts example
tests:
  - name: check-100-jeffs
    description: Checks that 100 Jeffs were created
    dependencies:
      - add-jeff-100
    runner: sql-runner
    assert:
      - type: EqualsRows
        params:
          method: SQLQuery
          actionParams:
            query: "select count(*) as cnt from members where name='jeff'"
          expected:
            rows:
              - cnt: 100

Assert Result

The runner will return a message of this format and store it under state['check-100-jeffs']['asserts']['EqualsRows0']:

{
    "passed": True,
    "description": "passed",
    "expected": "100",
    "actual": "100"
}
← StateRunners →