System-wide waiting queue in PHP

The task is to find a way that enables php calls to an external application to be processed one at a time in the order they were entered. The context here is that I have, on a web server, an application that is being called by a PHP script. The execution of the application is quite costly in terms of CPU performance. The whole process is initiated by users (visitors to the homepage) so there may be several simultaneous calls. Unchecked execution of these calls may result in undesirable behaviour. Therefore I need a fifo queue for <em>every</em> PHP process on the system that wants to call this external application.
1 answer

PHP blocking fifo queue

I have put together a few functions in PHP that allow processes to enter into a queue and wait their turn. This queue is file based and thus accessible to all processes on the system. The use case is e.g. a function call or processing of a file that should not have more than one instance on the whole system.

There are two files, one that contains the actual queue (queue.php) and a lock for accessing the queue (queue.lock). All the functions are part of a bigger PHP object, hence the private modifiers and occasional $this references.

As of now I can't get any kind of indention to work here, sorry for that. Now for the function that lets a process put his ID into the queue:


private function push_to_queue()
{
$queue = Array();
$fh = fopen($this->folder."/queue.lock","w");
$start_time = microtime(true);

do
{
$haveLock = flock($fh, LOCK_EX);
if(!$haveLock) usleep(rand(0,100)*10000);
}
while(!$haveLock && (microtime(true)-$start_time) < 10000);

if($haveLock)
{
include($this->folder."/queue.php");

if($queue == null) $queue = Array($this->id);
else array_push($queue, $this->id);

$save = '$queue = unserialize(\''.serialize($queue).'\') ?>';
file_put_contents($this->folder."/queue.php", $save);
}

fclose($fh);
}

Next the function which lets a process remove his id from the queue:

private function remove_from_queue()
{
$fh = fopen($this->folder."/queue.lock","w");

$start_time = microtime();

do
{
$haveLock = flock($fh, LOCK_EX);
if(!$haveLock) usleep(rand(0,100)*1000);
}
while(!$haveLock && (microtime(true)-$start_time) < 10000);

if($haveLock)
{
include($this->folder."/queue.php");

$key = array_search($this->id,$queue);

if($key !== FALSE)
{
unset($queue[$key]);
$queue = array_values($queue);
}

$save = '$queue = unserialize(\''.serialize($queue).'\') ?>';
file_put_contents($this->folder."/queue.php", $save);
}

fclose($fh);
}

And last a function for reading the queue:

private function load_queue()
{
$fh = fopen($this->folder."/queue.lock","w");

$start_time = microtime(true);

do
{
$haveLock = flock($fh, LOCK_SH);
if(!$haveLock) usleep(rand(0,100)*1000);
}
while(!$haveLock && (microtime(true)-$start_time) < 10000);

if($haveLock)
{
include($this->folder."/queue.php");
return $queue;
}

fclose($fh);
}

Usage for the functions currently looks like this:

$this->push_to_queue();

$sleep_time = 0;

$queue = $this->load_queue();

while($queue[0] != $this->id)
{
usleep(500000);
$sleep_time++;

if($sleep_time >= 120)
{
$return = Array("error" => "queue_timeout");
$this->make_log("[WARNING] Request timed out in queue.");
return $return;
}

$queue = $this->load_queue();
}

// DO STUFF

$this->remove_from_queue();

Caveats: Obviously there could be a lot of disk access with the file based queue.

Taggings: