Drupal Planet

Case Study in Migrating XML to Drupal using Migrate

Sorry for the lack of posts as of late -- a massive upgrade operation at $DAYJOB has had me out of commission for a few weeks. Also, I've had the great fortune to be able to be part of a migration to Drupal which exposed me to migrate and friends. Yes, I said "great fortune" in the same sentence with "migration" without using a negative - that's just how awesome this module is.

My first impression when looking at the documentation for migrate was that it didn't seem complete. While it's true that the documentation could be better (what module couldn't use better documentation?), the problem is that no two migrations are alike.  Because of this, the best documentation is not going to be written by the module authors, it will be written by the module users - they are the ones that come up with the recipies to fill the cookbook.  There are several good reasons why there aren't many recipes available:

  • Developers don't like doing migrations.  It can be painful, and often takes quite a bit of time.
  • Users don't like migrations.  They see a migration of data as something easily done, and they often get sticker shock when presented with estimates for a large migration.
  • Migration code is written in a flurry before the site is active.  Right before launch, development crescendos, and then is often never used again (because no two migrations are the same).

This being my first migration, I vowed that I would document my experience, because I learned so much from it.  In this particular migration, we had to migrate a huge XML file into about 2,200 nodes in 3 content types.  Read on for my contribution to the cookbook!

5
Your rating: None Average: 5 (2 votes)

My Thoughts and Ramblings on DrupalConSF 2010

DrupalCon SF 2010I had the great pleasure of attending my first DrupalCon this week.  Held in downtown San Francisco at the Moscone Center, it was my opinion that this was Drupal's "homecoming".  While Drupal wasn't "born" in San Francisco, it seems to be the city that has the strongest following.  The attendance numbers didn't lie - I'm pretty sure they broke 3,000 geeks attendees.  I made this trip solo -- I only knew three people that were going, and those three were only acquaintances I'd met via email/IM a few months before.  When I left, I didn't come home with "leads" or "contacts", I came home with friends and role models, many of whom I plan on staying in touch with.  I met most of the authors of the Drupal books I've read, associated faces to the podcasts and RSS feeds I subscribe to, and I even had the opportunity to quickly say thanks to Dries and shake his hand.

For those who didn't know, archive.org has made the sessions available for download, so be sure to check those out.  Read on for my "takeaways" from DCSF2010.

4
Your rating: None Average: 4 (4 votes)

New Linode StackScript: Pantheon Mercury (High Performance Drupal in 10 Minutes or Less)

For those who might not know, Pantheon Mercury is:

... a drop-in replacement for your Drupal website hosting service that delivers break-through performance. Mercury can serve two-hundred times more pages per second and generate pages three times faster than standard hosting services.

Mercury achieves this by using open-source technologies like so many ingredients of a complex dish - a little Varnish here, a dash of Memcached there, a hint of the Alternative PHP Cache, a healthy dose of Tomcat and Solr, all based upon the Pressflow distribution of Drupal. None of it is anything you couldn't do yourself -- many before Chapter Three had done it actually. However, they were the first to tie it all together using BCFG2, and release an Amazon EC2 AMI image of it.

As word spread, many liked the idea of Mercury, but wanted to brew their own non-EC2 instance. While they posted a wiki article on how to do it yourself, they went to work on native support for RackSpace. When I read Josh Koenig's post on the Linode blog stating he wanted to bring Mercury to Linode, I made a mental note. Some time passed, I became much more involved in Drupal, and I decided to volunteer to write the StackScript. Josh said okay, and put me in touch with Greg Coit, their resident sysadmin, and we went to work.

Fast forward a couple weeks, and we've announced a beta! The StackScript is quite complete - it supports Ubuntu Jaunty and Karmic, and can use the current stable branch or the soon-to-be-released 1.1 development branch. Once Lucid is released, we'll test to make sure it works there as well.

I want to thank Greg for all his help. We found some bugs in Ubuntu, some quirks in the memcached init script, and fixed many bugs and added some features to their BCFG2 bazaar repo. Thanks also go out to Josh for his oversight and guidance. It was a great time, a great learning experience, and I came out of it with some new colleagues (and some free beers at DrupalConSF).

Feel free to read up on my experiences with Linode, and if you like what you see, click on one of the many links to Linode from my blog. If you sign up and stay a customer for 90 days (trust me, you will), I'll get $20 credited to my account. Feel free to comment below about the StackScript and let me know about any issues you might find.

0
Your rating: None

Tracking Drupal's Outbound HTTP Requests using tcpdump

While working on tweaking performance for a client, I was able to shave 7 seconds of PHP execution time time off the homepage load. The cause was eventually tracked down to calls out to TinyURL for every node being rendered. The core problem came from the service links module. We were able to fix it by disabling short URL's in the module, but the problem has been addressed in the current pre-release 2.x branch by using caching. We might have ended up discovering this by disabling module after module one at a time, but that would have taken forever.

4
Your rating: None Average: 4 (1 vote)

Drupal StackScript for RH Derivatives on Linode (Instant Drupal!)

StackScripts are a relatively new offering from Linode that allow users to build their own installation script by "stacking" previously existing scripts together to build the machine you want. You can keep your StackScript to yourself, or publish it for the world to use. Deploying a distribution with a StackScript takes only about 5 minutes, afterwards you have a fully configured system with applications up and running. Here's a sneak-peek at a my Drupal StackScript for RH Derivatives deployment just before launch:

0
Your rating: None

Assign Different Values to Different Nodes via One Action in Views Bulk Operations

The Views Bulk Operations module (a.k.a. VBO), is a godsend for busy Drupal site administrators. Don't just take my word for it - Lullabot wrote a chapter about it in O'Reilly's Using Drupal, it's included in the Open Atrium Drupal distribution, and it's even used on Drupal.org! Out of the box, VBO does a lot to streamline the things you do everyday, so that you spend less time doing them. A perfect example is bulk content moderation - with a few clicks of the mouse, you can mark a huge amount of comments as spam. You can even enable batch processing with a single click of a mouse so that you can literally do thousands of these without timing out.

VBO was attractive enough that we decided to offload the bulk/batch operations of Node Gallery to VBO. Integration for the most part was surprisingly easy - VBO "speaks" in Drupal Actions, so by writing actions, we were writing integration with VBO.

There's one undocumented case where VBO can be used that was critical for us. Most VBO actions you will find perform one action to a set of nodes, one at a time. Often times, that one action is to set a value of some sort on said nodes. In the case of Node Gallery, we wanted to be able to assign different weight values (used for sorting) to a bunch of nodes. The key here is that we aren't assigning a value of '2' to all selected node's weight, we want to assign a weight of 2 to node #1, 3 to node #2, 8 to node #3, and so on. While not straightforward, it's definitely achievable.

4.666665
Your rating: None Average: 4.7 (3 votes)

HipHop PHP and Drupal

So, Facebook has released HipHop PHP - a PHP-to-C++ converter. While the name is stupid, the idea is not. 100% of their developers know PHP, I would guess that less than 5% of them are proficient at C++. So, HipHop takes their PHP code, and converts it to compiled C++ -- in turn, they get a huge boost in performance and get to keep their existing developers. HipHop is also it's own webserver too - fun!

My first thought was: I wonder what this could mean for Drupal? Well, David Struass, a maintainer of Pressflow (a set of patches for Drupal performance and scalability) put up a blog post about what it would take for Pressflow and Drupal to become HipHop-friendly. Exciting times!

3.666665
Your rating: None Average: 3.7 (3 votes)

Display a CCK Filefield or Imagefield Upload Widget on Your Own Custom Form

Took a fair amount of googling around to find the solution to this one. With the Node Gallery 3.x branch, we needed a way to quickly add an image to an existing gallery. We could have displayed the whole node form, but there's a lot of things on that form that we can just use the defaults for 99% of the time. We need just three fields filled in: Title, Caption, and the imagefield itself.

4.4
Your rating: None Average: 4.4 (5 votes)

Drupal, meet Hudson; Hudson, Drupal...

At $work, we use Hudson extensively, and it rocks. For those who don't know already, Hudson is an implementation of Continuous Integration that is remarkably easy to use. I wrote about my first impressions of Hudson previously. Hudson's original audience was Java developers using Ant or Maven, but with plugins and some hacking, we can make it do some things for us as module contributors to Drupal.

4
Your rating: None Average: 4 (8 votes)
Syndicate content