A bit about my recent work:
As the web dev for the Allied Media Conference, I’ve been tasked with creating a system for participants to liveblog about all the sessions at this year’s conference. I decided on setting up a three-part system: a tweet searcher+archiver, a JSON server, and the existing 2010 AMC and USSF discussion message board software.
- Tweet searcher+archiver: a node.js app that reads a filtered twitter feed from the Streaming Twitter API (using a version of twitter-node, separates out relevant hashtags (each relating to a conference session) and stores them in a MySQL db. (80% complete)
- Tweet server: another node.js app, this time a HTTP server using djangode, that responds to queries for certain hashtags and returns a JSON stream of tweets with that tag, or alternatively returns a “last tweet” timestamp for comparison on the frontend. (10% complete)
- Frontend: a modified Drupal forum setup with a special field on topics that offers the relevant hashtag as an element on the page. Javascript by PaulH will form a HTTP request using that hashtag to retrieve tweets from the tweet server and, based on the twitter timestamp, intertwine them with the topic replies. (80% complete)
This may seem like a complicated system. Why not just draw tweets using clientside javascript during pageload? Why use bespoke node.js servers instead of doing a Drupal implementation or buying cloud services? Respectively, temporariness and efficiency. With our setup, the twitter conversation isn’t temporary anymore: we can archive the conversation and guarantee that it lasts for future viewers (no need to rely on twitter to keep on serving tweets). It’s also more efficient: for one thing, it’s only one stream request from twitter (maybe two if i want to do redundancy) instead of a bunch of in-browser requests, and the streaming API seems to be more timely and reliable. Node.js lends itself very well to this type of system.
While the current implementation is definitely off-the-cuff hackish, I’m going to work on generalizing the code and will release it on github afterwards. If anyone is interested in collaboration, please comment!
Staging magento sites: multiple sites running APC == broken
Well, it’s been a wild webhosting ride recently. I’ve been automating dev/staging/live production procedures with my Drupal and Magento clients. In a few days I’ll publish a comprehensive blogpost about this whole project, replete with nginx, php, php-fpm, etc configuration goodies.
For now, here’s one tidbit that I just can’t hold back:
I ran into an interesting problem with running multiple Magento sites on the same server: the new Magento devsite I set up would unfailingly read the old site’s db configuration information (from app/etc/local.xml) 9/10 times after restarting nginx and php5-fpm. And to make things even more confusing, on some restarts, it’d read the devsite db configuration and so would the live site!
After trying every single possible iteration of php5-fpm and nginx options that I thought might be relevant, I remembered that I had set up Magento to use APC for caching, and changed the prefix:
It took a LONG time to track down this issue. I had forgotten that I had apc cache running on this site, and I never would’ve guessed that it would mix up different site’s caches. The search was even more complicated because I am not as confident with my nginx/php-fpm skills as i am with apache/mod_php. Serves me right for blindly implementing the magento apc cache without understanding what the various xml config options mean.
After finding the problem on my own, I searched to see if anyone else has experienced this and found a post on the magento board. Wish I’d seen that 6 hours ago.
I’m going to add an APC-prefix-changing sed operation to my magento dev/staging/production scripts now.