itobey.dev

Running SoapUI Tests Automatically in a GitOps / Argo CD Deployment Pipeline

Published Updated 7 months ago

Disclaimer This article does not present a fully productized, drop-in solution. It describes one concrete way of running SoapUI tests in an Argo CD–based GitOps environment, based on real-world requirements and constraints. The approach and implementation will likely require adjustments to fit other organizations, infrastructures, or security models. All relevant example files (Dockerfile, scripts, and Kubernetes resources) are available on GitHub

Automated integration testing is a critical part of any mature delivery pipeline. In GitOps-driven environments—especially those based on Argo CD—the challenge is not whether to run tests, but how to run them reliably, repeatably, and declaratively as part of the deployment lifecycle.

In this post, we’ll walk through a practical approach to automatically executing SoapUI tests as an Argo CD PostSync hook, using:

  • A custom Docker image built on top of the official SoapUI TestRunner image
  • Maven-based SoapUI projects published to Artifactory
  • A custom entrypoint that downloads, executes, reports, and notifies
  • Native Kubernetes Jobs triggered by Argo CD hooks

The result is a clean GitOps-compatible solution where tests are versioned, traceable, and tightly coupled to the deployed application.

Why Not Use the Official SoapUI Image Directly?

SmartBear provides an official soapuios-testrunner Docker image, which is great for basic execution. However, in real-world CI/CD and GitOps scenarios, it quickly shows limitations:

  • No built-in mail notification
  • No easy way to download test artifacts dynamically
  • EntryPoint logic exits the container early, making composition difficult
  • Limited extensibility for enterprise environments

To address these gaps, I built a thin extension image that preserves the official behavior but adds:

  • Artifactory-based test retrieval
  • Email notifications with test summaries and attachments
  • Better control over exit codes

High-Level Architecture

Here’s the flow at a high level:

  1. Developers create SoapUI test suites in a Maven module next to the application
  2. The test module is packaged as a JAR and pushed to Artifactory
  3. Argo CD deploys the application
  4. A PostSync hook Job runs the custom SoapUI TestRunner image
  5. The container:
    • Downloads the SoapUI project from Artifactory
    • Executes the tests
    • Collects reports
    • Sends email notifications
    • Exits with the correct status

For Argo CD even fails are considered a successful deployment, since my goal was to not interfere with the deployment itself. This can be changed however so Argo CD handles failures with SoapUI tests as a failed deployment.

Maven-Based SoapUI Test Projects

To make tests first-class citizens of the delivery process, SoapUI projects are treated like any other build artifact.

Key conventions:

  • SoapUI tests live in a dedicated Maven module (e.g. my-app-soapui-tests)
  • The module packages exactly one SoapUI XML project
  • The resulting artifact is published to Artifactory as a JAR

This enables:

  • Versioned test suites
  • Reproducibility
  • Clear coupling between app version and test version

Extending the Official SoapUI Image

Custom Dockerfile

We start from the official image and layer in what we need:

FROM docker.artifactory.ww-intern.de/smartbear/soapuios-testrunner:5.9.0

USER root

# Install required utilities
RUN apt-get update && \
    apt-get install -y \
        ca-certificates \
        openssl \
        libssl1.1 \
        curl \
        zip \
        file \
        unzip && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

# Copy custom scripts
COPY scripts/* ./
RUN chmod +x ./customEntrypoint.sh && \
    chmod +x ./EntryPoint.sh && \
    useradd -u 4444 -r -g root -m -d /home/runner -s /sbin/nologin -c "Runner user" runner && \
    chmod -R 770 /usr/local/SmartBear/* && \
    chgrp -R 0 /usr/local/SmartBear && \
    chmod -R g+rwX /usr/local/SmartBear

USER runner

ENTRYPOINT ["./customEntrypoint.sh"]

Key points:

  • We retain the official SoapUI runtime
  • We install minimal tooling (curl, zip, unzip) for Artifactory and reporting
  • We replace the entrypoint with our own orchestrator

Overriding the Original SoapUI EntryPoint

The original SoapUI EntryPoint.sh exits the container internally, which breaks composability.

To fix this, we copy and modify it:

  • Remove exit statements
  • Preserve exit codes in environment variables
  • Allow it to be called from a parent script
if [ -d "$MOUNTED_PROJECT_DIR" ]; then
  cp -a $MOUNTED_PROJECT_DIR/. $PROJECT_DIR
fi

if [ -d "$MOUNTED_EXT_DIR" ]; then
  cp -a $MOUNTED_EXT_DIR/. $SOAPUI_DIR/bin/ext
fi

sed -i "s|COMMAND_LINE|$COMMAND_LINE|" ./RunProject.sh
sed -i "s|%project%|$PROJECT_DIR|g" ./RunProject.sh
sed -i "s|%reports%|$REPORTS_DIR|g" ./RunProject.sh

./RunProject.sh

export EXIT_CODE=$?

This allows us to wrap the SoapUI execution in higher-level logic without losing status information.

The Custom Entrypoint: Orchestrating Test Execution

The core of this solution is a custom container entrypoint that wraps the official SoapUI TestRunner logic and adds the functionality required for a GitOps-driven deployment workflow.

Instead of executing SoapUI directly, the container starts with customEntrypoint.sh, which orchestrates the full test lifecycle in a controlled and extensible way.

Responsibilities of the Custom Entrypoint

The script performs four steps:

1. Download the SoapUI Project from Artifactory

SoapUI projects are published as Maven artifacts to Artifactory. The entrypoint resolves the Maven coordinates provided via environment variables (groupId, artifactId, version) into a download URL, retrieves the JAR, and extracts the SoapUI XML project.

To avoid ambiguous executions, the script enforces that exactly one SoapUI XML file is present in the artifact. If this condition is not met, execution fails immediately.

2. Execute Tests Using the Official SoapUI Logic

Instead of re-implementing the test execution logic, the custom entrypoint delegates to a modified copy of SoapUI’s original EntryPoint.sh.

The original script contains hard exit statements that would terminate the container prematurely. These statements are removed so that:

  • The SoapUI runner can be invoked as a subprocess
  • The exit code can be captured and forwarded correctly

This preserves the official behavior while making it composable.

3. Collect Results and Send Notifications

During execution, the console output is filtered to extract a concise test summary, which is written to a temporary summary file.

After execution:

  • All generated reports are bundled into a ZIP archive
  • An optional email notification is sent containing:
    • The test summary
    • The report archive as an attachment

Email delivery is fully optional and controlled via environment variables, allowing the same image to be reused across environments.

4. Propagate the Correct Exit Code

Finally, the custom entrypoint exits with the same exit code as the SoapUI TestRunner.

This is critical for Argo CD integration: a non-zero exit code causes the PostSync hook to fail, which in turn marks the deployment as unsuccessful.

From Argo CD’s perspective, the Job is a simple Kubernetes resource. All complexity—artifact retrieval, execution, reporting, and notifications—is handled inside the container, making this approach both GitOps-friendly and easy to operate.

Configuration for Helm

The following values.yaml shows all configuration parameters for the Kubernetes resources.

soapui:
  # Job / chart naming
  name: soapui-tests

  # Job behavior
  ttlSecondsAfterFinished: 1200
  backoffLimit: 0

  # Optional override for SoapUI endpoint
  # forcedEndpoint: https://api.example.com

  # Optional: run only specific suite or case
  # testSuite: "My Test Suite"
  # testCase: "My Test Case"

  # Custom SoapUI properties (-G key=value)
  customProperties: []
  #  - env=dev
  #  - timeout=30

  # Maven artifact configuration
  maven:
    groupId: "com.example"
    artifactId: "soapui-tests"
    version: "1.0.0" # defaults to container.image.tag if unset

  # Credentials configuration
  credentials:
    # Artifactory credentials (required)
    artifactoryUser: "artifactory-user"
    artifactoryToken: "artifactory-token"

    # Optional SoapUI credentials
    # soapuiUser: "soapui-user"
    # soapuiPassword: "soapui-password"

  # Optional secret name override for SoapUI credentials
  # secretName: soapui-credentials

  # Email notification configuration
  email:
    enabled: false
    recipients: []
    #  - [email protected]
    #  - [email protected]

  # Resource requests & limits
  resources:
    limits:
      memory: "256Mi"
    requests:
      memory: "64Mi"
      cpu: "100m"

  # Scheduling options
  nodeSelector: {}
  #  disktype: ssd

  tolerations: []
  #  - key: "dedicated"
  #    operator: "Equal"
  #    value: "soapui"
  #    effect: "NoSchedule"

  affinity: {}

Endpoint Configuration Strategies

The SoapUI test execution supports two complementary strategies for configuring service endpoints.

The preferred approach is to define the service endpoint as a SoapUI project property that references an environment variable (for example via ${#env#SERVICE_ENDPOINT}). This approach keeps the SoapUI project environment-agnostic while still allowing fine-grained control over endpoints per deployment. It works well with authentication flows such as OAuth, since SoapUI retains full control over how endpoints are resolved internally.

Forced Endpoint Override

As an alternative, the setup supports a forced endpoint configuration.

When a forced endpoint is defined:

  • All endpoints used by SoapUI are rewritten at runtime
  • Every request is routed to the forced endpoint, regardless of how it is defined in the SoapUI project
  • No project-level endpoint configuration is required

This is useful for simple, uniform environments where all requests must be directed to a single base URL.

The forced endpoint mechanism also overrides endpoints used internally by SoapUI, including those involved in OAuth token flows. Because of this, it must not be used with OAuth-based authentication, as token acquisition will fail.

Credential Handling

The SoapUI test setup distinguishes clearly between infrastructure credentials and test-level credentials, ensuring both security and flexibility.

Artifactory Credentials

Artifactory credentials are used exclusively to authenticate against Artifactory in order to download the JAR file that contains the SoapUI test project.

SoapUI Test Credentials (soapuiUser / soapuiPassword)

For convenience, the setup provides two optional default credentials:

  • soapuiUser
  • soapuiPassword

These map directly to predefined environment variables that can be referenced inside the SoapUI project. Important: The SoapUI test suite must explicitly reference these environment variables. If they are not defined in the project, these credentials will have no effect.

You may use your own environment variables which can then be set as a customProperty in the values.yaml.

Comments

Comments are hosted on comments.itobey.dev. Loading them sets cookies and shares your IP address with that service.