Updated README and fixed responsibility mixup. Also, added stability by cleaning the memory block on startup.
This commit is contained in:
Joop Schilder
parent
a5b5a58d06
commit
f29d1a0d7a
80
README.md
80
README.md
|
|
@ -1,52 +1,67 @@
|
|||
# joopschilder/php-async
|
||||
Asynchronous PHP callable processing with return values via SysV shared memory.<br/>
|
||||
Requires the `php-sysvshm` extension.<br/>
|
||||
Works with PHP >= 5.3 due to `shm_attach(...)` returning a resource instead of an int.<br/><br/>
|
||||
<b>Note:</b> This package should not be used in a CGI environment, as each PHP runtime that makes a call on the async functions WILL
|
||||
create an 8MB shared memory block in RAM. If you run out of memory: you have been warned.
|
||||
<br/>
|
||||
<b>Note:</b> This project is an experiment. It is, however, available on packagist.
|
||||
If you think your project lacks witchcraft and black magic, just add this package to your `composer.json`:
|
||||
## Introduction
|
||||
This package provides functions to run callables asynchronously in PHP. Return values are shared via System-V shared memory.<br/>
|
||||
To use this package, you'll need PHP >= 7.0.0 with `ext-sysvshm` and `ext-pcntl`.<br/>
|
||||
You should consider the state of this package to be <i>experimental</i>.<br/><br/>
|
||||
<b>Note:</b> This package should not be used in a CGI environment.
|
||||
The key that is used to access the block of shared memory is created based on the inode information of one of the source files.
|
||||
This means that, whenever multiple instances (processes) from the same project source are created, they will try to use the same block of memory and collisions will occur.
|
||||
I might swap the `ftok()` call for a random string generator somewhere down the road.<br/><br/>
|
||||
<b>Note:</b> It is possible (but discouraged) to change the amount of available shared memory.
|
||||
If you wish to do so, it's as simple as calling either `Runtime::_setSharedMemorySizeMB(<amount of MB>);` or `Runtime::_setSharedMemorySizeB(<amount of bytes>);`.<br/>
|
||||
If you want to use 32MB for example, call `Runtime::_setSharedMemorySizeMB(32);`.<br/>
|
||||
Be sure to make this call before using any of the asynchronous functionalities.
|
||||
## Installation
|
||||
This package is available on <a href="https://packagist.org/packages/joopschilder/php-async">Packagist</a>
|
||||
and can be installed using <a href="https://getcomposer.org/">Composer</a>:<br/>
|
||||
```bash
|
||||
$ composer require joopschilder/async-php
|
||||
```
|
||||
It's also possible to manually add it to your `composer.json`:
|
||||
```json
|
||||
{
|
||||
"require": {
|
||||
"joopschilder/php-async": "dev-master"
|
||||
}
|
||||
}
|
||||
```
|
||||
```
|
||||
|
||||
## Usage
|
||||
#### Functions
|
||||
The library exposes three functions in the global namespace that provide indirect access to the class `Asynchronous`:
|
||||
* `async(callable $function, ...$parameters)` to run something asynchronously, giving back a `Promise`;
|
||||
* `async_wait_all()` to wait for all currently running jobs to finish;
|
||||
* `async_reap_zombies()` to clean up any zombie processes during runtime if any exist;
|
||||
|
||||
## Examples
|
||||
|
||||
### Promises
|
||||
#### Promises
|
||||
Whenever you call `async(...)`, a `Promise` instance is returned.<br/>
|
||||
A `Promise` is considered to be resolved when the function it belongs to returned a value or finished execution.
|
||||
To block execution until a promise is resolved, simply call the `resolve()` method on the promise.
|
||||
It's possible to check whether the promise has been resolved in a non-blocking way by calling the `isResolved()` method.<br/>
|
||||
You can actually return anything that is serializable in PHP: objects, arrays, strings, you name it.
|
||||
```php
|
||||
<?php
|
||||
require_once __DIR__ . '/vendor/autoload.php';
|
||||
|
||||
$promise = async(function() {
|
||||
sleep(random_int(1, 5));
|
||||
return getmypid();
|
||||
sleep(random_int(1, 5));
|
||||
return getmypid();
|
||||
});
|
||||
|
||||
// ... some other work
|
||||
// ... do some other work
|
||||
|
||||
$promise->resolve();
|
||||
$pid = $promise->getValue();
|
||||
printf("Me (%d) and %d have worked very hard!\n", getmypid(), $pid);
|
||||
```
|
||||
The shutdown handler and destructors should take care of the rest.
|
||||
|
||||
|
||||
|
||||
### Asynchronous curl requests
|
||||
... though you should probably look into curl multi handles for this: <a href="http://php.net/manual/en/function.curl-multi-init.php">curl_multi_init() on PHP.net</a>.
|
||||
The shutdown handler and destructors should take care of the cleanup.<br/>
|
||||
#### Asynchronous curl requests
|
||||
... though you should probably look into curl multi handles for this: <a href="http://php.net/manual/en/function.curl-multi-init.php">curl_multi_init()</a>.
|
||||
|
||||
```php
|
||||
<?php
|
||||
require_once __DIR__ . '/vendor/autoload.php';
|
||||
|
||||
// Create the body for the process
|
||||
// Create the body for the process...
|
||||
$process = function(string $url) {
|
||||
$handle = curl_init($url);
|
||||
curl_setopt($handle, CURLOPT_FOLLOWLOCATION, 1);
|
||||
|
|
@ -56,21 +71,30 @@ $process = function(string $url) {
|
|||
file_put_contents(uniqid('download_'), $response);
|
||||
};
|
||||
|
||||
// Define some urls we want to download
|
||||
// Define some urls we want to download...
|
||||
$urls = [
|
||||
'example.com',
|
||||
'example2.com',
|
||||
'some.other.domain'
|
||||
];
|
||||
|
||||
// And away we go!
|
||||
// And there we go.
|
||||
foreach($urls as $url)
|
||||
async($process, $url);
|
||||
```
|
||||
That's all there is to it.
|
||||
|
||||
## Tips
|
||||
If you're on a UNIX system, you can make use of the tools `ipcs` and `ipcrm` to monitor and manage the shared memory blocks.<br/>
|
||||
To track what's happening in real time, I like to use:<br/>
|
||||
```bash
|
||||
$ watch -n 1 "ipcs -m --human && ipcs -m -p && ipcs -m -t && ipcs -m -u"
|
||||
```
|
||||
<br/>To clean all 'unused' shared memory blocks (they might remain resident in RAM if your program terminated unexpectedly):<br/>
|
||||
```bash
|
||||
$ ipcrm -a
|
||||
```
|
||||
|
||||
## What's next?
|
||||
- Refactoring
|
||||
- More functionality (maybe)
|
||||
- Improving stability
|
||||
- Add a diagram that explains this witchcraft
|
||||
- Add an explaining diagram
|
||||
|
|
@ -14,7 +14,6 @@
|
|||
},
|
||||
"require": {
|
||||
"ext-pcntl": "*",
|
||||
"ext-sysvshm": "*",
|
||||
"ext-posix": "*"
|
||||
"ext-sysvshm": "*"
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -8,25 +8,15 @@ namespace JoopSchilder\Asynchronous;
|
|||
*/
|
||||
class Asynchronous
|
||||
{
|
||||
public const BLOCK_SIZE_MB = 8;
|
||||
private const BLOCK_SIZE_BYTES = self::BLOCK_SIZE_MB * (1024 ** 2);
|
||||
|
||||
/** @var Asynchronous|null */
|
||||
private static $instance;
|
||||
|
||||
/** @var int */
|
||||
private static $key = 0;
|
||||
|
||||
|
||||
/** @var int[] */
|
||||
private $children = [];
|
||||
|
||||
/** @var resource */
|
||||
private $shm;
|
||||
|
||||
/** @var int */
|
||||
private $shmKey;
|
||||
|
||||
|
||||
/**
|
||||
* @param callable $function
|
||||
|
|
@ -39,7 +29,7 @@ class Asynchronous
|
|||
* Prepare for fork
|
||||
*/
|
||||
$instance = self::getInstance();
|
||||
$key = self::generatePromiseKey();
|
||||
$promiseKey = Promise::generatePromiseKey();
|
||||
|
||||
/*
|
||||
* Fork the parent
|
||||
|
|
@ -61,7 +51,7 @@ class Asynchronous
|
|||
if ($pid > 0) {
|
||||
$instance->children[] = $pid;
|
||||
|
||||
return new Promise($key);
|
||||
return new Promise($promiseKey);
|
||||
}
|
||||
|
||||
/*
|
||||
|
|
@ -75,17 +65,17 @@ class Asynchronous
|
|||
* On failure, write a default response to the block in order for
|
||||
* the Promise to be able to resolve.
|
||||
*/
|
||||
Runtime::markChild();
|
||||
Runtime::markAsChild();
|
||||
$instance->_attachToShm();
|
||||
|
||||
try {
|
||||
$response = call_user_func($function, ...$parameters);
|
||||
shm_put_var($instance->shm, $key, $response ?? Promise::RESPONSE_NONE);
|
||||
shm_put_var($instance->shm, $promiseKey, $response ?? Promise::RESPONSE_NONE);
|
||||
|
||||
exit(0);
|
||||
|
||||
} catch (\Throwable $throwable) {
|
||||
shm_put_var($instance->shm, $key, Promise::RESPONSE_ERROR);
|
||||
shm_put_var($instance->shm, $promiseKey, Promise::RESPONSE_ERROR);
|
||||
|
||||
exit(1);
|
||||
}
|
||||
|
|
@ -144,14 +134,11 @@ class Asynchronous
|
|||
private function __construct()
|
||||
{
|
||||
/*
|
||||
* Use the filename as an identifier to create the
|
||||
* System V IPC key.
|
||||
* The reason we do this is for when the shm block
|
||||
* already exists. We attach, remove, detach and reattach
|
||||
* to ensure a clean state.
|
||||
*/
|
||||
if ($this->shmKey == null)
|
||||
$this->shmKey = ftok(__FILE__, 't');
|
||||
|
||||
Promise::__setShmKey($this->shmKey);
|
||||
$this->_attachToShm();
|
||||
$this->_attachToShm(); // Attach
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
@ -192,7 +179,7 @@ class Asynchronous
|
|||
*/
|
||||
private function _attachToShm()
|
||||
{
|
||||
$this->shm = shm_attach($this->shmKey, self::BLOCK_SIZE_BYTES);
|
||||
$this->shm = shm_attach(Runtime::getSharedMemoryKey(), Runtime::getSharedMemorySize());
|
||||
|
||||
return $this;
|
||||
}
|
||||
|
|
@ -217,29 +204,6 @@ class Asynchronous
|
|||
}
|
||||
|
||||
|
||||
/**
|
||||
* @return int
|
||||
*/
|
||||
private static function generatePromiseKey()
|
||||
{
|
||||
/*
|
||||
* Get the current key.
|
||||
*/
|
||||
$promiseKey = self::$key;
|
||||
|
||||
/*
|
||||
* Reset the key to 0 if the upper bound of
|
||||
* 9.999.999 is reached (Windows limit for
|
||||
* shm keys).
|
||||
*/
|
||||
self::$key++;
|
||||
if (self::$key > 99999999)
|
||||
self::$key = 0;
|
||||
|
||||
return $promiseKey;
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
*
|
||||
*/
|
||||
|
|
|
|||
|
|
@ -15,8 +15,7 @@ class Promise
|
|||
public const RESPONSE_ERROR = '__PROMISE_RESPONSE_ERROR__';
|
||||
|
||||
/** @var int */
|
||||
private static $shmKey;
|
||||
|
||||
private static $generatedKey = 0;
|
||||
|
||||
/** @var resource */
|
||||
private $shm;
|
||||
|
|
@ -27,16 +26,27 @@ class Promise
|
|||
/** @var mixed|null */
|
||||
private $value;
|
||||
|
||||
|
||||
/**
|
||||
* @param int $shmKey
|
||||
* @return int
|
||||
*/
|
||||
public static function __setShmKey(int $shmKey)
|
||||
public static function generatePromiseKey()
|
||||
{
|
||||
/*
|
||||
* Should be done only once: when the Asynchronous class
|
||||
* has created a key that will be used for IPC.
|
||||
* Get the current key.
|
||||
*/
|
||||
self::$shmKey = $shmKey;
|
||||
$promiseKey = self::$generatedKey;
|
||||
|
||||
/*
|
||||
* Reset the key to 0 if the upper bound of
|
||||
* 9.999.999 is reached (Windows limit for
|
||||
* shm keys).
|
||||
*/
|
||||
self::$generatedKey++;
|
||||
if (self::$generatedKey > 99999999)
|
||||
self::$generatedKey = 0;
|
||||
|
||||
return $promiseKey;
|
||||
}
|
||||
|
||||
|
||||
|
|
@ -48,19 +58,9 @@ class Promise
|
|||
{
|
||||
$this->key = $key;
|
||||
$this->value = null;
|
||||
$this->shm = shm_attach(self::$shmKey);
|
||||
$this->shm = shm_attach(Runtime::getSharedMemoryKey());
|
||||
}
|
||||
|
||||
/**
|
||||
* @return $this
|
||||
*/
|
||||
private function attempt()
|
||||
{
|
||||
if (shm_has_var($this->shm, $this->key))
|
||||
$this->value = shm_get_var($this->shm, $this->key);
|
||||
|
||||
return $this;
|
||||
}
|
||||
|
||||
/**
|
||||
* @return bool
|
||||
|
|
@ -137,4 +137,15 @@ class Promise
|
|||
shm_detach($this->shm);
|
||||
|
||||
}
|
||||
|
||||
/**
|
||||
* @return $this
|
||||
*/
|
||||
private function attempt()
|
||||
{
|
||||
if (shm_has_var($this->shm, $this->key))
|
||||
$this->value = shm_get_var($this->shm, $this->key);
|
||||
|
||||
return $this;
|
||||
}
|
||||
}
|
||||
|
|
@ -13,16 +13,54 @@ namespace JoopSchilder\Asynchronous;
|
|||
*/
|
||||
class Runtime
|
||||
{
|
||||
/** @var int */
|
||||
public const INITIAL_SHM_SIZE_MB = 16;
|
||||
|
||||
/** @var int */
|
||||
private static $sharedMemKey = null;
|
||||
|
||||
/** @var int */
|
||||
private static $sharedMemSize = self::INITIAL_SHM_SIZE_MB * (1024 ** 2);
|
||||
|
||||
/** @var bool */
|
||||
private static $inParentRuntime = true;
|
||||
|
||||
|
||||
/**
|
||||
*
|
||||
* Runtime constructor.
|
||||
*/
|
||||
public static function markChild()
|
||||
private function __construct()
|
||||
{
|
||||
self::$inParentRuntime = false;
|
||||
}
|
||||
|
||||
|
||||
/*
|
||||
* Public
|
||||
*/
|
||||
|
||||
|
||||
/**
|
||||
* @return int
|
||||
*/
|
||||
public static function getSharedMemorySize()
|
||||
{
|
||||
return self::$sharedMemSize;
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* @return int
|
||||
*/
|
||||
public static function getSharedMemoryKey()
|
||||
{
|
||||
/*
|
||||
* Use the filename as an identifier to create the
|
||||
* System V IPC key.
|
||||
*/
|
||||
if (is_null(self::$sharedMemKey))
|
||||
self::$sharedMemKey = ftok(__FILE__, 't');
|
||||
|
||||
return self::$sharedMemKey;
|
||||
}
|
||||
|
||||
|
||||
|
|
@ -34,4 +72,36 @@ class Runtime
|
|||
return !self::$inParentRuntime;
|
||||
}
|
||||
|
||||
|
||||
/*
|
||||
* 'Semi' private.
|
||||
* To be used by internal classes.
|
||||
*/
|
||||
|
||||
|
||||
/**
|
||||
* @param int $size_mb
|
||||
*/
|
||||
public static function _setSharedMemorySizeMB(int $size_mb)
|
||||
{
|
||||
self::$sharedMemSize = abs($size_mb) * (1024 ** 2);
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* @param int $size_b
|
||||
*/
|
||||
public static function _setSharedMemorySizeB(int $size_b)
|
||||
{
|
||||
self::$sharedMemSize = $size_b;
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
*
|
||||
*/
|
||||
public static function markAsChild()
|
||||
{
|
||||
self::$inParentRuntime = false;
|
||||
}
|
||||
}
|
||||
Loading…
Reference in New Issue