Using Aptori in Continuous Integration

To maximize benefit of Aptori's dynamic application testing, you can easily run Sift as part of your continuous integration to get feedback about every change committed to your source code repository. Running Sift on every merge request ensures that new defects are not released into production, and reduces remediation time by identifying the code change that introduced a defect.

Sift can be executed in any continuous integration platform. This guide introduces general concepts for running Sift in a continuous integration pipeline. Refer to detailed guides below for particular continuous integration platforms.

• GitHub
• GitLab

Requirements

The requirements to run Sift in a CI pipeline are the same as running Sift on the CLI. You will need:

• Network access to the running application to test
• Sift configuration
• Aptori Platform key

Dynamic Application Testing

Sift is a dynamic application test tool that generates and executes tests for the network API of an application. In order to use Sift, the application under test must be running at a URL that is accessible to Sift. Some techniques for running the application under test are:

1. Launch an test instance of the application in the CI environment.
2. Deploy a temporary test instance to a cluster (e.g., GitLab Review App).
3. Test the application in a UAT (User Acceptance Testing) or staging environment after the code changes have been merged and deployed.

Techniques 1 and 2 are useful for running Sift in a pre-merge manner. The third technique is often used when run Sift to test your application in a post-merge manner.

Once you know where your application will be running, you can create the configuration for Sift.

Sift Configuration

The easiest way to get started is to create a configuration in Aptori Platform. In the UI, you can view the Sift command to run with the configuration stored in the Aptori Platform. The command will look like: sift run --config-id xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.

Alternately, you can maintain the Sift configuration as a file in your source code repository. You can download a configuration from the Aptori Platform, or create the file from scratch (see Sift Configuration). We suggest storing the configuration in a file named .aptori/sift.yaml in a directory at the root of your source code repository. This .aptori directoy can also contain scripts used in the configuration for dynamic authorization.

Platform Key

An Aptori Platform key is required by Sift to be able to report test results to the Aptori Platform. You created an Aptori Platform key when onboarding your application into the Aptori Platform, and used the key to run Sift CLI. You can create a new Platform key for CI/CD.

When running Sift in CI, the most convenient and secure way to provide the Platform key to Sift is by setting environment variable SIFT_PLATFORM_KEY. Most CI platforms allow you to store secrets and pass those secrets as environment variables in a CI job. Do not store the Aptori Platform key (or any secrets) in your source code repository.

Running Sift

The easiest way to run Sift in CI is to use the Sift container. Many CI platforms use a container as the execution environment for a job. The Sift container contains the Sift CLI. The exact method of executing Sift will vary by CI platform, but in general, you will specify your Sift configuration as a command-line argument and your Aptori Platform key in an environment variable.

Running Sift can be as simple as:

sift run --config-id xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

or

sift run --config .aptori/sift.yaml

In some cases, you may create a test instance of your application with a URL that changes in each CI pipeline. If you have a dynamic URL for the test instance of your application. You can override the URL value in the configuration using the CLI flag --target-url.

You may also want to use Labels to specify details about the code commit and CI pipeline. Use the --labels flag to add such details. For example, GitLab provides environment variables with details about the CI job:

# Environment variables used by this example are populated in GitLab CI jobs.
sift run --config .aptori/sift.yaml \
    --label="commit=${CI_COMMIT_SHA}" \
    --label="job=${CI_JOB_ID}" \
    --label="pipeline=${CI_PIPELINE_IID}"

Frequenly Asked Questions

Why does Sift report an error that "InitiateRun failed"?

If Sift is unable to create a Run in the Platform, then it will display an error. The following error indicates that the platform key was invalid, expired, or missing.

Error: platform: InitiateRun failed: status="Unauthorized" message="Session has expired"

Can I use a self-signed certificate in my application?

Yes. If you are using HTTPS with a self-signed TLS certificate, then you can configure Sift to ignore certificate validation by passing the -k/--insecure CLI flag, as in sift run --insecure ....

How do I override the configured target URL of the application?

You can override the URL of a target by passing the CLI flag --target-url to the sift run command. This is useful to be able to use the same Sift configuration for testing applications that are running with a dynamically assigned hostname or IP address.

How do I override values in the Sift configuration when there is not a CLI flag to override a setting?

You can use Template Variables to pass values from environment variables into the Sift configuration.

For example, consider an application that requires a Referrer header to be populated with the URL of the application, and the target application URL is a dynamically assigned in the test environment. Suppose the target URL is stored in environment variable APPLICATION_URL.

In the configuration, define both the target URL and the value of the additional Referrer header to use a Template Variable that gets the URL from environment variable APPLICATION_URL.

targets:
  - api: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
    url: '{{.Env.APPLICATION_URL}}'
    header:
      Referer: '{{.Env.APPLICATION_URL}}'

And run Sift with the APPLICATION_URL environment variable set:

export APPLICATION_URL="https://dynamic123.example.com:8080"
sift run --config .aptori/sift.yaml