Hack 99 Cache Auction Data to Improve API Efficiency


Reduce the number of API calls your program makes and work within your daily API allotment.

One of the requirements of certification, as described in [Hack #82], is that your application or script does not make more calls or retrieve more data than is absolutely necessary. This is typically accomplished in any of three ways:

  • Restricting the result set to a specific date range, as described in [Hack #85].

  • Download only new entries by comparing the current total of entries with the total you recorded the last time the call was used, as described in [Hack #96].

  • Caching retrieved data so that it doesn't have to be retrieved again, as described in this hack.

Which data you cache and how you do it depends on the type of data you're working with.

8.19.1 Caching Input

Probably the most useful place to start is by recording the item numbers of all auctions you're currently selling. Assuming you're using GetItem to upload your listings to eBay (see [Hack #88]), you can simply save the resulting item number in a file, like this:

open (OUTFILE,">>$localdir/auctionlist.txt");
  print OUTFILE "$rsp->{Item}[0]{Id},$rsp->{Item}[0]{EndTime}\n";
close (OUTFILE);

Eventually, the file will look like this:

4500207651,2005-07-15 20:43:32
4500207783,2005-07-16 08:14:18
4500208002,2005-07-18 19:00:31

with each line containing one item number and one end date, separated by a comma. Then, instead of using the GetSellerList API call found in many hacks in this chapter, you can simply load the list, like this:

open (INFILE,"$localdir/auctionlist.txt");
  my %items = map { split(/,/, $_) } <INFILE>;
close (INFILE);
chomp %items;

which will create a hash of item numbers and end times. To fill an array with all the item numbers, use the keys function:

@itemnumbers = keys %items;

To retrieve the end date for a single auction, reference the hash like this:

$endtime = $items{'4500207783'}

Naturally, if you want to store more than just item numbers and end times, you'll need a more complex storage system, but the methodology will be the same.

Not only will this approach significantly reduce your API usage, it will end up being a lot faster than repeatedly downloading large amounts of data from eBay.

8.19.2 Caching Output

Another case where caching can reduce API usage and improve performance is when it comes to the output your program generates. For example, the script in [Hack #94] generates a web page with a listing of your currently running auctions (with pictures). But instead of generating the same page every time, you can redirect the output to a static HTML file and then run the script every so often to generate the file.

You can schedule the script to run at regular intervals or, better yet, run the script every time an auction starts or ends (using, among other things, the notifications discussed in [Hack #93]).

A simple conversion to the gallery.pl script will make it cache its own output. First, add this line immediately before line [1]:

open (OUTFILE,">/some_directory/gallery.html");

where /some_directory is the full path to the gallery.html file. Next, remove line [1] entirely:

print "Content-type: text/html\n\n";

Then, you'll need to convert all print statements in the script so that their output is redirected to the open file. For example, change:

print "<html><script type=\"text/javascript\">\n";


print OUTFILE "<html><script type=\"text/javascript\">\n";

Finally, at the end of the script, after line [3], close the file like this:

close (OUTFILE);

That should do it. All that's left to do at this point is to change the URL in the <iframe> tag to reflect the address of the static page, like this:

<iframe src="http://www.ebayhacks.com/gallery.html" 

With this approach, your customers see a cached list of auctions every time your custom gallery is loaded. Not only will this eliminate the unnecessary API calls, but the page will load much faster as well.