« Duff metadata and out of date databases | Main | Ubuntu image on Amazon EC2 »

Replication

Musicbrainz uses a Postgres contributed script called dbmirror to handle replication. dbmirror was designed to work between two live instances, They’ve done quite a nice job to decouple this - packages are regularly written to the musicbrainz ftp area containing data suitable for dbmirror slaves.

I wanted replication to be controllable within the sleevenotez application architecture, and more to the point I didn’t want to have to schedule a bunch of cron jobs running perl scripts on our development environments. I’ve ported the dbmirror slave part to twisted with a MusicbrainzReplicationService running on startup that does the full process of replication - ftping to the server, pulling down the packages, unpacking them, parsing them and updating the database.

It works very nicely, and seems to be working as well, more to the point. I’ll be releasing this bit back to the community once I’m sure it’s bug free - it might be useful for others later.

Post a comment

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)