Feed generator for Cablegate mirrors

This code has been released under the public domain. Updates may or may not occur. Keep an eye on this page for new updates.
Latest version: December 7 2010

Download

Click here to download the code.

Installation

Place the script anywhere BUT NOT IN YOUR PUBLIC_HTML DIRECTORY! As long as you configure the script correctly (top of the PHP file) it will be able to find your mirror. On some Linux systems you may need to install the php5-cli package to enable command-line PHP. Simply set a cronjob for the script to be executed every hour (Google for more information). You will need to have this command executed every hour: php /path/to/parse_wl.php where you change the path to reflect the location on your server.

APIs

APIs will be coming soon, for PHP and VB.net. These API's will cache a list of mirrors, and compare server staleness so that the entire API will work in a decentralized fashion.

Developer instructions

Despite no API being available yet, it's already possible to use the data provided by this script. The following files are created by this script:
* total.num : a file that only has a number indicating the total amount of released cables on that mirror.
* feed.xml : an XML-formatted feed with ALL cables on that mirror. This file may be large.
* feed.rss : an RSS-formatted feed with the latest 100 cables. This amount of cables may be changed by the server administrator.

To check if a mirror generates feeds, simply poll the server whether the file total.num exists (in the root of the mirror address). If it does, the server is feed-enabled. You can then use this number to compare staleness amongst mirrors. For example, if a mirror is at www.wikileaks.org, you would check if http://www.wikileaks.org/total.num exists. For the XML file from one of the mirrors, click here (LARGE!). You can use this XML file to see the structure of the XML files. feed.xml and rss.xml can be found in the root as well.