Background case
madisonrightnow.com is a collection of near-real-time information about a metropolitan area. Traffic, weather, parking lot usage, and loads of web cam images are displayed on one page. Data come from a wide variety of sources: images come from cameras, weather data comes from a web API, and parking lot utilization come from good ol’ fashioned page scraping. All these transactions are triggered when a user wants to view the page. It is a challenge to get near-real-time information to the client from so many sources without a lot of pre-processing on the server end, and without the client having to make connections to a myriad of hosts to get images and other data. I found that asking the client to load all this makes the page too slow to load.
The solution is to build a server-side caching mechanism, so all the data and images are ready to go when the page is hit, and so that all transfers are between madisonrightnow.com and the client without having to wait on anybody else. This can cause the data served to be a little older, but is much better than asking the client to open connections to dozens of hosts and wait for every one to completely reply before the page can render. Under the caching scheme I developed, all data on the page are loaded from my server in order to give the user a smooth experience when expanding UI panels after the client renders it.
When a user requests madisonrightnow.com, the main (index.php) script gives the client URLs for the cached data when it serves the HTML. At this time it must also check the age of the cached files to check if they need to be updated. The “freshness” age is arbitrary but I chose 8 minutes in order to stay under the limits of my API accounts and avoid pounding on web servers from my host for page scraping and camera images. No matter how many times the site is visited, it will not request external data any more often than every 8 minutes.
The code
The cache update script is separate from the main script and is called by it when the page loads. The following code is from the main script which calls the background task to simultaneously run on a separate thread. It receives no result and routes any output to /dev/null. If you have a case where you need to blindly run a PHP script from another script, you could do it like this:
$execCmd = "/usr/bin/php -f ~/www/update.php";
$outputFile = '/dev/null';
$pidFile = '/dev/null';
exec(sprintf("%s > %s 2>&1 & echo $! >> %s", $execCmd, $outputFile, $pidFile));This executes PHP at the command line on Linux servers and ignores the output.