it’s email migration time again!

That’s right, folks. After the grueling 72hr dovecot-to-gmail (Google Apps) migration ~1.5 years ago, it’s that time of the decade where we are going to do it in reverse!

In fairness to Google, their migration API back then was weak (or non-existent?).  Mails have to be sent one by one via SMTP.  Lucky for us, Gmail-to-IMAP isnt that painful, thanks to imapsync.

imapsync is a tool for facilitating incremental recursive IMAP transfers from one mailbox to another. It is useful for mailbox migration, and reduces the amount of data transferred by only copying messages that are not present on both servers. Read, unread, and deleted flags are preserved, and the process can be stopped and resumed. The original messages can optionally be deleted after a successful transfer.

Usage is very simple:

./imapsync –host1 imap.gmail.com –port1 993 –user1 user@gmail.com –passfile1 /home/user/passwordfile –ssl1 \

–host2 mail.domain.com –port2 993 –user2 user –passfile2 /home/user/passwordfile –ssl2

It’s safer to use –passfile than –password so that your password does not show in ‘ps’, in case there are other users in the server.  imapsync is written in perl and requires additional modules:

Mail::IMAPClient
IO::Socket
IO::Socket::SSL
Digest::MD5
Digest::HMAC_MD5
Term::ReadKey
Date::Manip

Your IMAP folders may not appear on your mail client. You may have to manually subscribe to your folders.

Internal DNS on Amazon EC2

While looking for a way to name instances and update the /etc/hosts file, I came across Tim Dysinger’s blog entry.  It’s a ruby script that parses AWS metadata to get the internal IP address of the running instances in your AWS account and its respective keypair – perfect for /etc/hosts entries.  Obviously, the caveat here is that you need to generate 1 unique ssh private key for every instance you are running because key == internal dns name.  You’ll be managing a lot of ssh keys in the end.

This is perfect for situations where your instances interact with each other (load balancers in front of app servers with database servers at the back?).  It’s very handy calling your instances lb0, app23  or db3 instead of something like ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com.  Automatic, too.  If you’re running the script as a cron job, your /etc/hosts file gets updated if you run new instances.

#!/usr/bin/env ruby
%w(optparse rubygems EC2 resolv pp).each { l require l }
options = {}
parser = OptionParser.new do p
  p.banner = "Usage: hosts [options]"
  p.on("-a", "--access-key USER", "The user's AWS access key ID.") do aki
    options[:access_key_id] = aki
  end
  p.on("-s",
       "--secret-key PASSWORD",
       "The user's AWS secret access key.") do sak
    options[:secret_access_key] = sak
  end
  p.on_tail("-h", "--help", "Show this message") {
    puts(p)
    exit
  }
  p.parse!(ARGV) rescue puts(p)
end
if options.key?(:access_key_id) and options.key?(:secret_access_key)
  puts "127.0.0.1 localhost"
  EC2::Base.new(options).describe_instances.reservationSet.item.each do r
    r.instancesSet.item.each do i
      if i.instanceState.name =~ /running/
        puts(Resolv::DNS.new.getaddress(i.privateDnsName).to_s +
             " #{i.keyName}.ec2 #{i.keyName}")
      end
    end
  end
else
  puts(parser)
  exit(1)
end

Just pass on your AWS access key and secret key as parameters, pipe it to /etc/hosts and you’re good to go.