How to run multiple Symfony Panther clients in parallel

Symfony Panther is a great PHP-based library for browser testing and scraping websites.

Recently I worked on a project where I had to scrape a Single Page Application. To speed things up, I wanted to queue a separate job for each page I needed to scrape, so that they could scrape in parallel, with each job using a new Panther instance/client.

However, I got the following exception:

RuntimeException: The port 9515 is already in use

 
Scroll to full solution

1. How to run multiple clients in parallel

The exception I quoted above already gives away what the problem is. Each Symfony Panther client needs an available TCP port to run on. Symfony Panther by default picks the TCP port 9515. When you start up a second client, it tries to run on the same port as the first client, resulting in the exception above.

To fix this, we need to do 2 things:

1.1 Get an available TCP port

In the next step (1.2) we use this function to get an available TCP port we can use for the Symfony Panther client.
Note: you need the PHP sockets extension enabled.

For Windows, make sure php_sockets is in your ext directory, and then add extension=sockets to your php.ini file.

function getAvailablePort(): int
{
    // When providing '0' as port, the OS picks a random available port
    $socket = socket_create_listen(0);

    socket_getsockname($socket, $address, $port);

    socket_close($socket);

    return $port;
}

1.2 Provide the available TCP port when creating your Symfony Panther instance

Now you simply provide the available TCP port as an option when creating your Symfony Panther client. This works Chrome, but also Firefox.

$availablePort = getAvailablePort();

// Note: This also works for other clients, e.g. 'createFirefoxClient'
$client = \Symfony\Component\Panther\Client::createChromeClient(
	options: [
    	'port' => $availablePort,
    ],
);

2. Full solution

function getAvailablePort(): int
{
    // When providing '0' as port, the OS picks a random available port
    $socket = socket_create_listen(0);

    socket_getsockname($socket, $address, $port);

    socket_close($socket);

    return $port;
}

$availablePort = getAvailablePort();

// Note: This also works for other clients, e.g. 'createFirefoxClient'
$client = \Symfony\Component\Panther\Client::createChromeClient(
	options: [
    	'port' => $availablePort,
    ],
);