There are two ways to perform tasks in any program you write. The tasks will be executed one after the other in sequence or they will be executed in parallel without waiting for the completion of a previous activity. The first method of performing tasks is called synchronous execution and the second is called asynchronous execution.
Sometimes, the tasks or instructions need to be done sequentially like when extracting headlines from a scraped web page. The scraping of the web page must take place before any extraction takes place.
However, there are situations where you may want to perform tasks asynchronously. For example, let’s say you want to extract the header from 20 different web pages. Instead of waiting for one page to be scrapped and extracted before proceeding to the next, you can run multiple requests in parallel without waiting for the first request to complete.
In this tutorial we will learn how to perform multiple tasks in parallel in PHP using the Spatie asynchronous library.
Setting up space on Windows
The Spatie asynchronous library actually provides an easy to use wrapper around PHP’s PCNTL extension. However, the PCNTL extension is not available for Windows. This means that you can only use the library in a UNIX environment.
Fortunately, it’s easy to get around this by simply installing Linux on Windows with WSL. Don’t worry, it looks a lot more complicated than it actually is. All you need to do is run the following command after running PowerShell or Windows Command Prompt in administrator mode.
wsl --install
The above command will install Ubuntu as the default Linux distribution which is fine for our purpose. Once the installation process is complete, you can open Ubuntu from the Start menu. Provide a username and password. This new Linux account will be considered an administrator and will allow you to run sudo administrative commands.
I recommend that you install Visual Studio Code if it is not already installed. Within Visual Studio Code, you should also consider installing the remote WSL extension to make it easier to edit files that reside in WSL or the Windows file system without worrying about cross-platform issues.
You should now run the following command while in the Ubuntu environment.
code .
This will install a shim server that will allow WSL and VSCode to communicate with each other. You will also need to install Composer to simplify installation, upgrade and libraries.
Once the development environment is set up, you can create a file tasks directory while inside Ubuntu by running the following command:
mkdir tasks
Now execute the change directory command to get inside tasks.
cd tasks
Inside the tasks directory, we can finally install the file space / asic package by running the following command:
composer require spatie/async
Verify correct installation
Let’s say you are using this library in an environment where the PHP PCNTL extension is not installed. In that case, the library will automatically execute the code synchronously as a fallback.
One way to check if we are running code in an environment that supports asynchronous processes is to use the file isSupported() method from the library, which returns a Boolean value. The return value would be true whether the code can run asynchronously.
Create a file called test.php inside the task directory and add the following code.
<?php
require_once('vendor/autoload.php');
use SpatieAsyncPool;
$pool = Pool::create();
if($pool->isSupported()) {
echo 'We can run asynchronous code!';
} else {
echo 'Something is wrong!';
}
?>
If everything is set up correctly, you should also get We can run asynchronous code! as output when executing the preceding code.
Execution of requests in parallel
The library uses the symfony/process component to create and manage various child processes. Since the library can create multiple child processes, it is capable of running PHP scripts in parallel. This allows you to run multiple independent synchronous tasks in parallel and greatly reduce the time it takes to complete them all.
One thing you need to be aware of when running processes in parallel is not to generate many at the same time. This can cause an unexpected application crash.
Luckily, space / asynchronous deals with this with some support methods of the Pool class. This method add() can manage all the desired processes by scheduling and executing them optimally.
Different processes will take different amounts of time to complete. It is ideal to wait for all processes in a pool to complete before continuing further without accidentally killing any child processes. This task is managed by the wait() method.
Let’s say you want to run another code after a particular child process has exited and fired a success event. You can do this with the help of then() function.
We will now write some code that will create 10 different text files. To make a comparison, we’ll start by writing the code to run synchronously and then update it to run asynchronously.
Here is the synchronous code:
<?php
for($i = 1; $i <= 10; $i++) {
$file_name = "file_$i.txt";
$content = bin2hex(random_bytes(2048));
file_put_contents($file_name, $content);
echo "Generated file: $file_name".PHP_EOL;
}
?>
The code above gives the following output:
Generated file: file_1.txt Generated file: file_2.txt Generated file: file_3.txt Generated file: file_4.txt Generated file: file_5.txt Generated file: file_6.txt Generated file: file_7.txt Generated file: file_8.txt Generated file: file_9.txt Generated file: file_10.txt
The content of each file is just a random hexadecimal string 4096 bytes long. Here is an example:
841bda21ae704ecd05ad64ccb4fb029c6c6e8bc590eda828e2080d9f9f842c1f39883fd8e837325655184219ed92d3a9ca356b96c4a0edeb751d7270f8c1b3b949975ab9786289870a3f3cb7501..... and so on
We will now rewrite the code so that it runs asynchronously. Here’s what it will look like:
<?php
require_once('vendor/autoload.php');
use SpatieAsyncPool;
$pool = Pool::create();
for($i = 1; $i <= 10; $i++) {
$pool->add(function() use ($i) {
$file_name = "file_$i.txt";
$content = bin2hex(random_bytes(2048));
file_put_contents($file_name, $content);
return $file_name;
})->then(function ($file_name) {
echo "Generated file: $file_name".PHP_EOL;
});
}
$pool->wait();
?>
The above code will generate the following output:
Generated file: file_5.txt Generated file: file_6.txt Generated file: file_1.txt Generated file: file_9.txt Generated file: file_8.txt Generated file: file_2.txt Generated file: file_4.txt Generated file: file_3.txt Generated file: file_10.txt Generated file: file_7.txt
As you can see, the files are not generated in a sequential order when we run the code asynchronously. In other words, file_5.txt he didn’t have to wait file_1.txt to generate. We produce the file name inside the file then() function as soon as its success event is triggered.
Another alternative to using methods add() And wait() is to use functions async() And await(). Our code will look like this with the use of these functions:
<?php
require_once('vendor/autoload.php');
use SpatieAsyncPool;
$pool = Pool::create();
for($i = 1; $i <= 10; $i++) {
$pool[] = async(function() use ($i) {
$file_name = "file_$i.txt";
$content = bin2hex(random_bytes(2048));
file_put_contents($file_name, $content);
return $file_name;
})->then(function ($file_name) {
echo "Generated file: $file_name".PHP_EOL;
});
}
await($pool);
?>
Using event listeners
In the previous section, we created many child processes and added them to ours Pool class to run asynchronously. Several processes within the pool run independently of each other. This meant we needed a way to tell when a certain task was completed. The success event is triggered when a task has been successful. At this point we are free to run some other piece of code using the then() function.
However, the processes will not always run correctly. In some cases, they will fail or time out without completing the ongoing task. You can handle exceptions by providing a callback with the file catch() function and timeout by providing a callback with the timeout() function.
Let’s use all these concepts together to write some code that verifies the Collatz conjecture. The conjecture tells us that if an even number returns its half as the next term and an odd number returns 3 times itself + 1 as the next term, you will eventually end up on 1. For example, the sequence for 14 will be 14> 7> 22 > 11> 34> 17> 52> 26> 13> 40> 20> 10> 5> 16> 8> 4> 2> 1.
We will run ten iterations in our code where we will choose a random number at each step. Since the conjecture only deals with positive numbers, we will throw an exception whenever the random number is less than 1. Here is our code:
<?php
require_once('vendor/autoload.php');
use SpatieAsyncPool;
$pool = Pool::create();
for($i = 0; $i < 10; $i++) {
$pool->add(function() use ($i) {
$orig_num = $num = mt_rand(-10000, 100000);
if($i == 0) {
$orig_num = $num = 75128138247;
}
$count = 0;
if($num < 1) {
throw new Exception("Conjecture not applicable on $orig_num.");
}
while($num != 1) {
if($num%2 == 0) {
$num /= 2;
} else {
$num = 3*$num + 1;
}
$count++;
}
return [$orig_num, $count];
})->then(function ($output) {
echo "".$output[0]." reduced to 1 in ". $output[1] ." steps.". PHP_EOL;
})->catch(function($e) {
echo "Caught Exception ". $e->getMessage() . PHP_EOL;
})->timeout(function() {
echo "Process took too long n";
});
}
?>
$pool->wait();
Since the conjecture states that every positive number will eventually become 1, our code will eventually come out of while loop and returns the original number as well as the iterations needed to reach 1. We also throw an exception if the number is less than 1 because the conjecture only applies to positive numbers.
Try running the code a few times and you are sure to run into exceptions. Here is my output:
47443 reduced to 1 in 75 steps. 75128138247 reduced to 1 in 1228 steps. 44961 reduced to 1 in 62 steps. 28545 reduced to 1 in 59 steps. 53756 reduced to 1 in 246 steps. Caught Exception Conjecture not applicable on -8059. 39324 reduced to 1 in 106 steps. Caught Exception Conjecture not applicable on -7991. 97972 reduced to 1 in 190 steps. 71809 reduced to 1 in 94 steps.
You may have noticed that we passed a very large number during the first iteration of the loop. It took 1228 steps to reach 1. However, it was still fast enough to escape the timeout condition.
Pool configuration options
Let’s say you are doing something where you want results within a certain time or you abandon the task at hand. For example, you only want to calculate steps if they take less than 0.01 seconds to complete. How can this constraint be respected?
This is where the pool configuration options come in handy. There are four useful methods available to you.
-
concurrency()determines the maximum amount of processes that can run concurrently. This is set to 20 by default. -
timeout()determines how long a process runs within the pool before times out. The default is 300 seconds. -
sleepTime()determines how often the cycle should check the status of a process. the default is 50000 microseconds. -
autoload()specifies the autoloader that should be used by several sub-processes.
In our case, we will set the timeout value to 0.01 seconds. All we need to do is add the following line before creating our loop.
$pool->timeout(0.01);
If you rerun the code from the previous section with this change, you will notice that some numbers now expire before reaching the value 1. In real life, you can use this option to terminate processes such as reading the contents of a large file if it takes too long time.
Final thoughts
We have discussed many concepts in this tutorial. We started by learning how parallel processing and asynchronous code execution can help us get things done faster. Next, we learned how to configure WSL in Windows to use the asynchronous library. Once the installation was successful, we saw how to create multiple files with parallel processing.
Finally, we learned about different event listeners and how to use the pool configuration options to make sure our processes run with certain constraints. To practice, you should try to figure out how to run multiple processes in parallel to quickly edit images in PHP.
