Parallel Integration Tests With Random Ports On Jenkins

A common problem when doing end-to-end tests is colliding ports on you buildmachine with parallel execution. With the technology stack of Docker, Jenkins and Gradle I’ll demonstrate one solution I use in my current project to start the backend with a random port and use it in the test execution afterwards.

Our situation is the following: You want to execute a test which starts your backend application and fires against it from the outside. That way you can make sure that your whole infrastructure of the backend is working, including the servlet container for example. But the problem lies in the small detail that such tests usually take a while to execute and therefore lengthen your job runtime while blocking a single port over the time of execution. So the simplest solution by just starting the backend on a fixed port does not scale well for larger teams where a bunch of jobs are executed all the time. Therefore we’d think about starting the backend with randomly assigned ports. The graphic below demonstrates what’s happening on a single executor on Jenkins.

Our example backend is a very simple service written with Spring Boot and Kotlin which has one endpoint we want to test against:

backend/src/main/kotlin/de/held/randomport/backend/ExampleController.kt:

@RestController
class ExampleController {

	@GetMapping("example")
	fun exampleEndpoint() = "foo"

}

The test should happen through another application, because we want to test against the actual jar of our backend. So we create another application, called integration-test that only consists of a single unit test written with JUnit in Java:

integration-test/src/test/java/de/held/randomport/integrationtest/BackendIntegrationTest.java:

@Test
public void TestBackendExampleEndpoint() throws Exception {
    HttpClient httpClient = HttpClient.newHttpClient();
    HttpRequest request = HttpRequest.newBuilder()
            .GET()
            .uri(new URI(backendUrl + "/example"))
            .build();
    HttpResponse<String> response = httpClient.send(request, BodyHandlers.ofString());

    Assertions.assertEquals("foo", response.body());
}

As you can see the test is quite simple and performs a GET request against the /example endpoint we defined earlier to check if the expected body is received. The hard problem to solve lies in the backendUrl variable. We need to point the test to the correct address with the correct port the server started on. For local testing this is easy, it’s just the configured port of the backend application, but as soon as we want to execute the test on Jenkins or any other buildmachine we want to be independent of the port to avoid conflicts with other builds.

So first we need the functionality to start the backend with a random port. For that purpose we use docker-compose. The configuration is simple again:

docker-compose.yml:

version: '3'
services:
  backend:
    build:
      dockerfile: backend/Dockerfile
      context: .
    ports:
      - "8080"

By configuring the service with ports "8080" docker-compose will take an available port for us to expose and maps it on the port 8080 inside of the container, which is the port we configured for our backend application.

Starting our setup with docker-compose up shows us that we assign a random port.

docker ps command after starting the backend shows a random port

As a next step we need to pass this port to our integration-test. To do so we use the docker-compose plugin from avast. With it we have the possibility to start our container with gradle and receive information about it afterwards which we can pass through system properties to the integration-test.

integration-test/build.gradle:

dockerCompose.isRequiredBy(test)

dockerCompose {
	useComposeFiles = ['../docker-compose.yml']
	captureContainersOutput = true
}

test.doFirst {
    // exposes "${serviceName}_HOST" and "${serviceName}_TCP_${exposedPort}" environment variables
    // for example exposes "WEB_HOST" and "WEB_TCP_80" environment variables for service named `web` with exposed port `80`
	dockerCompose.exposeAsSystemProperties(test) 
}

Within our integration-test we can read the system properties and assign it to a field which is accessible in our test:

integration-test/src/test/java/de/held/randomport/integrationtest/BackendIntegrationTest.java:

public class BackendIntegrationTest {

	private static String backendUrl;

	@BeforeAll
	public static void ReadBackendUrl() {
		String backendHost = System.getProperty("backend.host");
		String backendPort = System.getProperty("backend.tcp.8080");
		backendUrl = "http://" + backendHost + ":" + backendPort;
	}
	
	(...)
}	

And that is it already. When we execute ./gradlew integration-test:test we start the docker container with a random port and the test picks it up. This can be easily executed on any build machine without interfering other builds.

warning “Warning” When you want to run a setup like this I recommend configuring an additional cronjob that cleans up dangling containers from time to time. It can happen that for some random reason a container might not stop properly.