IMAP Tools


Updated: November 4, 2008

Over the last few years I have written a number of tools and scripts for use with IMAP servers (
what is IMAP? ). I've given copies to to various people some of whom have encouraged me to make them generally available. So, for what it's worth, here they are. Note that all tools are offered "as is" with no promise of support or guarantee that they will actually work for you.    Download imap_tools_V1-26.tar.gz

Rick Sanders   rfs9999@earthlink.net

Notes:

  • SSL Support
  • IMAP debugging and tracing

    Recent Changes
    o 10/27/2008 - imapfilter.pl. Added wildcard mailbox support. Removed -D argument (remote IMAP server) and moved the remote access control info to the rules files. See description of imapfilter below for details.
    o 10/23/2008 - imapsync.pl. Added SSL support.
    o 10/22/2008 - imapfilter.pl. Added -D argument for copying filtered messages to a remote IMAP server
    o 10/20/2008 - imapcopy.pl. Added -r argument which clears the \Deleted flag on msgs being copied.
    o 10/20/2008 - IMAPtoMbox.pl. Added -r argument which when used with -M < mailbox list > includes all submailboxes of the ones in the mailbox list.

    Name Description SSL ?
    imap_tools_V1.26.tar.gz Tar file containing all of the tools -
    imapcopy.pl Copies messages and mailboxes from one IMAP server to another. More Yes
    pop3toimap.pl Copies POP3 messages to an IMAP server. More Yes
    migrateIMAP.pl Mass migration of messages for a set of users from one IMAP server to another. More Yes
    imapfilter.pl Moves messages from one IMAP mailbox to another based on a set of rules. More Yes
    imapCapability.pl Queries an IMAP server for a list of features it supports. More Yes
    imapdump.pl Writes IMAP messages to local files. More Yes
    imapPing.pl Pings an IMAP server. More Yes
    imapsync.pl Synchronizes two IMAP accounts. More Yes
    IMAPtoMbox.pl Copies messages from IMAP server to Mbox format. More No
    MboxtoIMAP.pl Copies messages from Mbox format to IMAP server. More Yes
    mbxIMAPsync.pl Synchronizes mbx and IMAP accounts. More No
    purgeMbx.pl Purges an IMAP mailbox. More Yes
    trash.pl Moves messages marked "Deleted" to trash mailbox. More No
    mozillatoIMAP.pl Copies Mozilla messages to an IMAP server. More No
    delIMAPdups.pl Deletes duplicate IMAP messages. More Yes

    More info:
    1. imapcopy.pl
      imapcopy is a utility for copying a user's messages from one IMAP server to another.   
      It logs into one server with your username & password, builds a list of your 
      mailboxes and the messages in them, then logs into the other server with your 
      username/password on that machine and copies the mailboxes and messages to it.  
      The mailbox (folder) structure is reproduced on the other server.  The IMAP status 
      flags are preserved (eg SEEN, DELETED, etc) along with the dates on the original messages.
                                                                           
      Usage:  imapcopy.pl -S host1/user1/password1 -D host2/user2/password2    
                                                                           
      Optional arguments:                                               
         -d debug                                                      
         -L logfile                                                    
         -m <mailbox list> (copy only certain mailboxes, eg "INBOX,Drafts,etc")
         -r clears the Deleted flag from copied messages
      
      You can select a port number other than the standard IMAP port (143) by 
      specifying the host value as host:port, eg localhost:993.
      
      Top

    2. pop3toimap.pl
      
      pop3toimap.pl is a tool for copying a user's messages from a POP3  
      server to an IMAP4 server.  pop3toimap.pl makes a POP3 connection  
      to the POP3 host and logs in with the user's name and password.    
      It makes an IMAP connection to the IMAP host and logs in with the  
      user's IMAP username and password.  pop3toimap.pl then fetches     
      each message from the user's POP3 account and copies it to the     
      user's IMAP account.                                               
                                                                         
      The usernames and passwords are supplied via a text file specified 
      by the -i argument.  The format of the file of users is:           
      
      popUsername password imapUsername password    
      
      Usage: pop3toimap.pl -P POP3host -I IMAP host -i input_file
      
      Optional arguments:
          -d debug mode
          -n notify e-mail address
          -r delete POP3 message after copying to IMAP
          -L logfile
          -h print this message
      
      
      Top

    3. imapdump.pl
      imapdump.pl will copy messages from an IMAP server and place them under
      a directory the value of which you supply with the -f argument.  The
      name of each file is the message number associated with it in the IMAP
      server message store.  When the script finishes the file structure will
      look something like this:
      
       /tmp/mailboxes/INBOX
            1 2 3 4 5 .....
       /tmp/mailboxes/drafts/
            1 2
       /tmp/mailboxes/save/2002
            1 2 3 4 5 6 7 8 9 a b c d e f 10 11 ...
       /tmp/mailboxes/save/2003
       etc 
      
      Usage: imapdump.pl  -f /tmp/mailboxes  -S localhost/rick/mypassword
      
      Some optional arguments are:
        -d debug
        -L <logfile>
        -m <mailbox list>  For example "INBOX,drafts,save/2003" will copy only the
      messages in the indicated folders.
      
      Top

    4. imapfilter.pl
      imapfilter allows you to scan folders on an IMAP server and move selected
      messages to different folders based on the contents of designated
      message header fields, eg Subject, From, Date, etc.
      
      imapfilter also supports the use of regular expressions in the
      matching rules.  For instance, the following rule will move
      all messages from a sender named "Jack" from the inbox to
      the wastebasket.
      
      From     ^Jack*   Inbox  Wastebasket
      
      Some examples of filter rules:
      
      #  Header field  match  source-mailbox   destination-mailbox.
      #  Note that mailboxes are case-sensitive but filter criteria are not.
      
      Subject  test$   Inbox Junk
      Importance High junk Inbox
      From  *abc.com Inbox Wastebasket
      X-Priority 3 Inbox Wastebasket
      
      Usage: imapfilter -S host/rfs/mypassword -r rules.dat -L imapfilter.log
      
      Optionally the destination mailbox can belong to another user on the local server
      or on a remote IMAP server.  In the following two examples the first rule moves
      messages to a local mailbox belonging to another user while the second moves
      messages to a remote IMAP server.
      
      Subject *home*     INBOX  localhost:INBOX 
      Subject *testing*  INBOX  remotehost.xyz.com:INBOX
      
      Access control info must be supplied in the rules file as shown below:
      
      RemoteServer: localhost/user1/password1
      RemoteServer: remotehost.xyz.com/user2/password2
      
      Wild-card mailboxes.  You can specify a '*' character in place of the local
      mailbox name in a rule.  This causes imapfilter to search the messages in each mailbox
      for matches.  For example:
      
      Subject *sales*     *   filtered_messages
      
      Top

    5. imapPing.pl
      
      imapPing is a tool for getting a rough feel for the response times
      from an IMAP server.  It connects to the server and then performs
      some basic IMAP operations on a user's  account and displays the
      time as each one is executed.  The operations are:
      
        1.  Connect to the IMAP server
        2.  Log in with a username and password
        3.  Get a list of mailboxes
        4.  Select the INBOX
        5.  Get a list of messages in the INBOX
        6.  Log off the server
      
      Usage: imapPing.pl -h imap.server.com -u rfs9999 -p XXXXX
      
      Top

    6. imapsync.pl
      
      imapsync is a tool for synchronizing user accounts on a pair of
      IMAP servers called the "source" and "destination" hosts.   
      You supply it with the host/userid/password on each host (and 
      optionally a list of mailboxes).  imapsync does the following:
       
      1.  Adds any messages on the 1st host which aren't on the 2nd 
      2.  Deletes any messages from the 2nd which aren't on the 1st 
      3.  Sets the message flags on the 2nd to match the 1st's flags
      
      imapsync works like this:
      
      1.  Log into the source host using the user name and password
      2.  Get a list of all mailboxes belonging to the user and a 
          list of messages in each mailbox.
      3.  Do the same for the destination host
      4.  For each message on the source see if the message exists on 
          the destination.  If not, add it to the destination.
      5.  For each message on the destination see if the message exists 
          on the source.  If not, delete it from the destination (eg, 
          set the \Deleted flag and expunge the mailbox).
      6.  For each message on the source compare the message flags with 
          the message flags on the destination.  If they don't match 
          update the flags on the destination.
       
      Some notes:
       
      a.  By default all of the user's mailboxes are processed but you 
          can limit that by providing a list of mailboxes in which case 
          only those will be examined (eg -m "Inbox, Drafts, Actions").
       
      b.  The Message-Id is used for identifying messages in an IMAP account.  
          A consequence of this is that if you have multiple copies of a 
          message in a source mailbox only one copy will be reproduced on 
          the destination host.
       
      Usage: imapsync -S sourcehost/username/password -D desthost/username/password
       
      Other (optional) arguments are:
       
        -L logfile  (if omitted the output goes to STDOUT)
        -d debug mode
        -m mailbox list (eg "Inbox, Drafts, Actions")
       
      
      Top

    7. IMAPtoMbox.pl
      
      IMAPtoMbox will copy mailboxes and messages from an IMAP server to the local
      server (or PC) in the Unix mbox format.  The mailbox hierarchy (nested mailboxes)
      is preserved.
       
      To run IMAPtoMbox you supply the name of the IMAP server, your username & password 
      along with a pointer to the directory on the local host where the mailboxes are to be 
      created.  The local mailbox is given the same name as the IMAP mailbox.  For example:
       
      Usage: IMAPtoMbox -i server/user/pwd -m mailbox_directory
      
      Optional arguments:
       
      -L logfile
      
      For example:
      ./IMAPtoMbox                                      \
              -i mail.abc.com/rick.sanders/mypassword   \
              -m  /home/rfs                             \
              -L IMAPtoMbox.log
       
      The -i argument supplies the imap server name / the username / user's password.    
      See the usage() notes for additional arguments, such as -M to copy only certain IMAP 
      mailboxes, -d to turn on debugging output and -o user to set the ownership of 
      the mailbox files. When used with -M the -r argument will copy all submailboxes
      of the ones in the mailbox list.
      
      Top

    8. MboxtoIMAP.pl
      
      MboxtoIMAP.pl is used to copy the contents of Unix mailfiles to IMAP               
      mailboxes.  It parses the mailfiles into separate messages which are 
      inserted into the corresponding IMAP mailbox.           
      
      Usage: MbxtoIMAP.pl -m mailfile -i server/user/pwd
      
      Optional arguments:
      
      -L logfile
      -d debug
      -r <range of messages to copy>  for example, 1-500 or 25-75
      -R remove messages from mailfile after copying to the IMAP server
      
      Top

    9. mbxIMAPsync.pl
      
      mbxIMAPsync is used to synchronize the contents of a Unix      
      mailfile with an IMAP mailbox.  The user supplies the location   
      & name of the Unix mailbox (eg /var/mail/rfs) and the hostname,   
      username, & password of the IMAP account along with the name      
      of the IMAP mailbox.  For example:                                
                                                                           
      Usage:  mbxIMAPsync.pl -f /var/mail/rfs -i imapsrv/rfs/mypass -m INBOX  
                                                                           
      mbxIMAPsync compares the messages in the mailfile with those in   
      the IMAP mailbox by Message-Id and adds the ones in the mailfile  
      which are not in the IMAP mailbox.  Then it looks for messages    
      in the IMAP mailbox which are not in the mailfile and removes     
      them from the IMAP mailbox.      
      
      Top

    10. purgeMbx.pl
      
      This script deletes all of the messages in a user's IMAP          
      mailbox.                                                          
                                                                        
      purgeMbx.pl is called like this:                                  
          ./purgeMbx.pl -s host/user/password -m mailbox               
                                                                        
      Note that the mailbox name is case-sensitive.                     
                                                                        
      Optional arguments:                                               
          -d debug                                                      
          -L logfile       
      
      
      Top

    11. trash.pl
      trash.pl opens a connection to an IMAP server, logs in with 
      a username and password, searches mailboxes for messages marked 
      for deletion, and moves them to the trash folder.  The name of
      the trash folder is supplied with the -t argument.
       
      For example:
      
      Usage: trash.pl -S server/user/pwd -t "Wastebasket"
      
      Optional arguments:
      
      -m mailbox_list (eg "Inbox, Drafts, Notes". Default is all mailboxes)
      -e  empty the trash can
      -L  logfile
      -d debug
           
      For example:
      
          trash.pl starting
          Checking mailboxes for deleted messages...
             Moved 2 messages from Drafts to Wastebasket
             Moved 4 messages from Sent Items to Wastebasket
             Moved 45 messages from INBOX to Wastebasket
           
          51 messages were moved to Wastebasket
          The Wastebasket mailbox has been emptied
      
      
      Top

    12. mozillatoIMAP.pl
      
      mozillatoIMAP is a tool for copying messages from a Mozilla mailfile 
      to an IMAP server.  You supply the script with the name of the IMAP
      server along with your username and password.  The -m argument is
      used to indicate the root directory where the Mozilla mailfiles
      reside.  For example:
      
      "/DOCUMENTS AND SETTINGS/RICK/APPLICATION DATA/Mozilla/Profiles/default/dlss3fw9.slt/Mail/mail.earthlink.net"
      
      Usage: mozillaToImap.pl -i host/username/password -m <mozilla root>
      
      Optional arguments:
      
      -L logfile
      -d debug mode 
      -I show IMAP protocol exchanges
      
      Note that mozillatoIMAP does not map Mozilla message status codes to
      IMAP flags. There is a comment in the code pointing to a URL with a
      table of Mozilla codes if someone wants to implement them.
      
      
      Top

    13. delIMAPdups.pl
      
      delIMAPdups looks for duplicate IMAP messages in a mailbox by    
      finding messages in each mailbox that have the same Message   
      ID.  When a duplicate message is found the DELETED flag is set.  
      If the -p argument has been supplied then an EXPUNGE operation   
      is executed on the mailbox in which the message resides to remove
      it.
                                                                       
      Note that delIMAPdups does not check for duplicate copies of     
      messages across multiple mailboxes since it is often useful to   
      cross-file messages in multiple mailboxes.                       
                                                                       
      Usage:  delIMAPdups -S host/user/password                         
      
      Optional Arguments:
      
      -m <mailbox list>                       
      -p delete the duplicates
      -M <mailbox>   Move duplicate msgs to this mailbox instead of deleting them.
      -L <logfile>                
      -d debug
      
      Top

    14. migrateIMAP.pl
      migrateIMAP is a utility for mass migration of messages for a 
      set of users from one IMAP server to another.
                                                                           
      Usage: migrateIMAP -S host1 -D host2 -i <users file>
                                                                           
      Optional arguments:                                               
      
      -n maxChildren
      -d debug                                                      
      -L logfile                                                    
      
      The user list file (-i <file>) should contain entries like this:              
          sourceUser1:password:destinationUser1:password               
          sourceUser2:password:destinationUser2:password               
          etc                                                           
      
      The maxChildren argument specifies how many child processes will be executed
      in parallel (the default is 2).  The optimum number of children depends on
      the system's resources but usually 4 or 5 maximizes the performance.  Beyond
      those numbers there is too much contention for resources to obtain additional
      improvement.
      
      Note:  migrateIMAP is intended for use on Linux/Unix servers since it takes
      advantage of the fork() system call to run multiple, simultaneous child processes
      for higher performance.  fork is not available on Windows so if you are running
      on a Windows platform you should use migrateIMAP-win.pl instead.
      
      Top

    15. migrateIMAP-win.pl
      migrateIMAP-win is basically the same as migrateIMAP.pl except it does not
      support multiple child processes via fork().
                                                                           
      Usage: migrateIMAP-win -S host1 -D host2 -i <users file>
                                                                           
      Optional arguments:                                               
      
      -d debug                                                      
      -L logfile                                                    
      
      Top

    16. imapCapability.pl
      imapCapability queries an IMAP server and reports which IMAP services
      it supports.
                                                                           
      Usage: imapCapability.pl -h <host> -u <username> -p <password>
                                                                           
      Optional arguments:                                               
      
      -d debug                                                      
      
      
      # ./imapCapability.pl -h localhost:1430 -u rfs9999 -p mysecret
      
      The server supports the following IMAP capabilities:
      
      IMAP4 IMAP4REV1 ACL NAMESPACE UIDPLUS IDLE LITERAL+ QUOTA 
      ID MULTIAPPEND LISTEXT CHILDREN BINARY LOGIN-REFERRALS 
      UNSELECT STARTTLS AUTH=LOGIN AUTH=PLAIN AUTH=CRAM-MD5 
      AUTH=DIGEST-MD5 AUTH=GSSAPI AUTH=MSN AUTH=NTLM
      
      Please refer to the IMAP RFCs for an explanation of these capabilities.
      
      Top

    Note: SSL Support

    As of January 10, 2008 many of these utilities support SSL connections and the plan is to add this to the remainder in the near future.

    SSL support is implemented using openSSL and the IO::Socket::SSL Perl module. Both must be installed on the system where you are running these tools.

    If you specify port 993 (995 for POP3) then an SSL connection is initiated. If the port number is 143 (110 for POP3) then it will try a non-SSL connection. With any other value the port will be tested to see if it supports SSL. If so, SSL will be used to make the connection; otherwise a non-SSL connection will be made.

    Top

    IMAP Tracing and Debugging

    The -d argument will log additional information about what the tool is doing which may be helpful in diagnosing problems. Most of these scripts accept the -I argument which causes logging of the IMAP protocol exchanges between the script and the IMAP server. This can be useful when troubleshooting problems but you probably don't want to use it during normal operations because of the amount of data that gets written to the logfile.

    For example:

    11-08-2007.03:28:18 2114 >> 1 EXAMINE "Inbox"
    11-08-2007.03:28:18 2114 << * FLAGS (\Answered \Flagged \Deleted \Seen \Draft $MDNSent $
    Hidden $Media $Forwarded Junk $Label1 $Label2 $Label3)
    11-08-2007.03:28:18 2114 << * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Dra
    ft $MDNSent $Hidden $Media $Forwarded Junk $Label1 $Label2 $Label3)] limited
    11-08-2007.03:28:18 2114 << * 86 EXISTS
    11-08-2007.03:28:18 2114 << * 0 RECENT
    11-08-2007.03:28:18 2114 << * OK [UIDNEXT 360] predicted next UID
    11-08-2007.03:28:18 2114 << * OK [UIDVALIDITY 298417783] UIDs valid
    11-08-2007.03:28:18 2114 << 1 OK [READ-ONLY] EXAMINE completed
    
    Top