Back in May I started looking at making feed fetching in /reader more efficient and explored the options for running PHP processes in parallel. Unfortunately, the common methods (async, pthreads, parallel) are not supported by my hosting.
The library I'm using to consume feeds uses cURL (if installed, file_get_contents()
if not) to grab the feed contents but doing this repeatedly for multiple feeds takes time – with the 29 feeds I had it was taking around 18 seconds to do a full refresh.
I recently came across 'cURL multi' 1 which allows the processing of multiple cURL handles asynchronously so wondered if I could make that work here. On Wednesday evening, as is my way, I started overcomplicating matters thinking I was going to have to rewrite the library. Tired and confused I went to bed wondering how I was going to achieve this.
Yesterday, however, I realised that, rather than need to rewrite things, I could just make a couple of the private functions public and call the library one stage further in. Instead of the library making single cURL requests, I could do the multi requests first and pass the results to the (now public) later functions.
The upshot is that a full refresh now takes less than three seconds for 31 feeds – a sixth of the time it was previously taking! As I add more feeds the time savings will only get larger.
-
I'm still learning my way around PHP. I haven't got to classes yet, using them confuses my old brain. ↩