Blog posts tagged "gzip"

Magpie, PHP 4.3.2 Memory Error, an Update

June 25th, 2003

Steve tracked down the root of the problem that is causing MagpieRSS to run out of memory after upgrading to 4.3.2. It lies in my (apparently dodgy) patch to Snoopy to add gzip support.

Temporary Solution

To turn off gzip encoding (and thereby avoid this bug) add:

define('MAGPIEUSEGZIP', false);

to the top of your script.


After much hair pulling trying to get gzip encoding working in PHP, I found this magic recipe in comments attached to gzencode:

$results = substr($results, 10); $results = gzinflate($results);

Seems you had to strip off the gzip header to make it work. This was confirmed when I found Simon’s httpClient.

Apparently this is changed in 4.3.2, and stripping the first 10 characters leads eventually (through a yet untraced path) to an out of memory error.

Next Steps

Does anyone have any info about this? Any suggestions on a good way to support 4.3.2 and pre-4.3.2 PHPs? Gzip encoding really isn’t an optional feature in a modern aggregator, so turning it off is a short term solution.

Update 6/25 2:16

Okay, so I’ve got PHP 4.2.3 locally now (via Marc Liyanage’s excellent package), and I’m seeing similar behaviour. I don’t actually get an out of memory error, but I can see memory usage spike (with iPulse) and I get a warning about a gzinflate buffer error. Unforunately removing “$results = substr($results, 10);” does not fix the problem for me. So currently the only solution is to turn gzip encoding off :(

Curiouser and Curiouser

So removing the substr() call changed the error message form “gzinflate(): buffer error”, to “gzinflate(): data error”. However only for certain websites! Files served with gzip encoding from some sites (like mine) work, but from others, it doesn’t. Ack.

Tagged: Uncategorized , , ,

MagpieRSS 0.5.1

June 12th, 2003

MagpieRSS 0.5.1 is out with a fix for the gzip silent failure bug that showed up when attempting to fetch an RSS file with gzip encoding when PHP had been built without zlib support. Also include random other small clean up.

Btw, that also means Magpie now runs on OS X’s stock Apache/PHP.

Tagged: Uncategorized , ,

Magpie GZIP Bug

May 30th, 2003

MagpieRSS 0.5 can fail without errors when fetching a RSS feed using GZIP encoding if your PHP doesn’t support gzinflate() (i.e. it hasn’t been built with --with-zlib)

As of 0.5 sends an “Accept-encoding: gzip” header by default when fetching RSS files. You turn this feature off by adding define(MAGPIEUSEGZIP, 0) to the top of your script.

A patched version of the locally maintained extlib/ that checks function_exists(gzinflate) is available in CVS. This fix seems to work, I haven’t had a chance to test it widely. It is a busy week, but I’ll try to have a final fix, and new version out in the next couple of days.

(sidenote: It seems silly to fork Snoopy, and give it a new name over one minor change like adding gzip support however development on Snoopy has stopped and its confusing this maintaing a local version because I don’t want to imply there is a bug with the real Snoopy, just my version. What do you think?)

Tagged: Uncategorized , ,

MagpieRSS 0.5: Now with GZIP support

May 12th, 2003

MagpieRSS 0.5 just hit Sourceforge. This is not the promised 0.5 with its mythical new parser, this is a much more modest release that adds support for HTTP gzip encoding to get Magpie off Bill and Ted‘s bad aggregator list.(flattering to make the short list really)

Also if you’re interested, a patch to Snoopy that gives it gzip encoding support.

Tagged: Uncategorized , ,

Gzip Encoding

April 28th, 2003

Magpie has been “outed” for not supporting GZIP encoding. I’m thinking of switching from Snoopy to Simon’s HttpClient in order to get gzip, and a few other features. Has anyone ever played with it? Snoopy has been solid, and error free, and while development stopped on it a while ago, this is sometimes a good thing. HttpClient on the other hand looks to be undergoing very active development, and I like some of the design decisions I’m seeing.

One problem, is it just me, or is there no way to send arbitrary headers?(like If-None-Match, to pick a purely hypothetical example) That would seem very strange, but quick glance threw the code seems to confirm this.

Tagged: Uncategorized ,