Migration speed

In certain situations, users may notice that process of both gathering and uploading data takes longer than expected considering the network bandwidth. Multiple factors may impact speed of the migration.

Installing directly on the source server works best

First of all, software installation location may have impact on the migration speed. CodeTwo strongly recommends installing the migration software directly on the source Exchange Server. Installation on other servers or workstations is fully supported, however, in this scenario the migration speed might be affected by the fact that data extracted from source Exchange mailboxes must be transferred over the local network to another machine where CodeTwo software is installed. This usually is not an issue but in some cases might be. Also, some organizations set up different Internet access speeds for their Exchange Server machine(s) and different for users' workstations. Installing directly on the server you avoid stumbling on such a problem.

The above advice, however, might not be the best one in two cases:

  • The source server machine is very old, slow computer with small RAM amount (e.g. 2GB). If this is how your source server looks like you may stumble upon problems related to the source Exchange Server machine performance.
  • The source server is Exchange Server 2003. This version of Exchange comes with its own, old MAPI libraries pre-installed which cannot be removed. CodeTwo software is optimized for MapiCDO libraries which cannot be used on Exchange 2003.

In any of the above scenarios we recommend to install the software on a workstation or dedicated server with better hardware and newer operating system.

Info

When migrating from IMAP source you can install the software anywhere you want. Source server side installation is crucial only for migration from Exchange Server via MAPI as MAPI can be accessed only locally. IMAP on the other hand, is designed to be used over the Internet so IMAP sources can be accessed from workstations outside of the source environment. So, when migrating from IMAP source, check which Internet connection is slower - to the source server or to the target server and preferably install within the environment to which outside connection is slower. This will let you avoid using that slow connection.

Better hardware = faster migration

The software does not have any special hardware requirements, but there are a few hardware improvements that may considerably speed up the migration process. First of all, the network card should be as fast as 1Gbps. However, this is rather obvious, and on the other hand, simply swapping a network adapter may not necessarily be enough to increase the network throughput, as other slow network devices might be bottlenecks. Less apparent obstacle could be the processor architecture. The whole software is optimized for a 64-bit architecture as 64-bit systems allow addressing more than 4GB of RAM (32-bit systems are supported too but will probably work slower). And here we arrive at another discussion point which would be the size of your RAM. The software can work with just 2GB, but it performs much better with at least 4GB. More RAM allows loading more items to the memory and speed up their processing. Last but not least, one more word about the CPU. If the network speed is not a problem and there are no other obvious obstacles, the migration speed will directly depend on the number of logical processors (*) in the system. More of those allow more migration threads to be executed at once, hence faster migration. You can also manually increase the number of threads in concurrency settings, regardless of the number of logical processors, but doing so may not necessarily yield the expected results.

(*) Logical processors would be: the number of actual CPUs, multiplied by the number of CPU cores in each of the CPUs, and multiplied by two, assuming processors feature Hyper Threading technology.

Internet connection upload speed is often a bottleneck

Despite the high throughput of your network adapter, the speed of upload is certainly limited by your Internet Services Provider (ISP). All of the ISPs offer contracts with asymmetric connections (the upload speed is considerably slower than download speed), when talking about your Internet connections speed you must be sure you are referring to the upload value. Symmetric links (upload = download) are often offered for business clients but there is another fine print catch - offered speeds are maximum possible speeds and not guaranteed ones. For more information, please check your Internet access contract or discuss that with your ISP directly.

Coexistence of other MAPI-using software might be a problem

Source Exchange Server data is accessed by CodeTwo software using MAPI libraries. It might be a good idea to make sure there is no other MAPI-using software enabled during the migration. It is not really required to disable third party MAPI-using applications in the same environment, but if the other software performs many Exchange operations via MAPI the same time you migrate this can affect the migration speed as both applications will have to share Exchange's time.

You actually migrate 30% more data than you have

Every single MAPI-extracted item is converted by CodeTwo software into the Fast Transfer Stream (FTS) format, applicable for a particular target Exchange Server version. The software calls EWS API methods on the target server, to allow native access to target Exchange's database and migration without necessity to install any additional applications on the target server. All calculations of the items’ sizes are based on final FTS binary files. The communication between your machine and the target server is done using the SOAP protocol, which works in the Request – Response mode. Prior to item dispatch, its FTS buffer has to be transformed to the Base64 format. Also, additional XML file with entries required by SOAP protocol is added to the transferred data stream. In a result, the size of every item is increased by 30% on average. Be aware that batches of items cannot be compressed due to SOAP protocol limitations.

Lots of small items migrate slower than big items

By default, every batch sent to the target server, contains 150 items regardless of their overall size and requires opening a new connection. Consequently, a batch may be completed very quickly, containing e.g. 150 small text-only messages with total size of only 3MB. This may result in pulling many small batches within short time range and in effect lead to performance issues on target server. After receiving a batch, every item requires individual processing by the Exchange Server. Therefore, you may observe that the migration speed slows down after some time when migrating lots of small items as the target Exchange is trying to cope with processing them. Migrating batches of small items is also slower in general due to a fact that both ends of the transmission exchange so called handshakes very often (i.e. the time spent by both source and target server on establishing and closing a connection for every batch separately).

Target Exchange's EWS throttling affects the speed too

We have also observed that despite the target server is capable of receiving many batches at the beginning, the migration process may start to slow down after some time. This behavior is normal and is due to target Exchange Server applying throttling policies.

Troubleshooting

If you believe that your migration process is too slow, please consider following the steps below:

  1. Move the migration software to another machine. If it was installed directly on the source Exchange Server, perhaps its hardware performance is too low - find a faster machine and install there. If it was installed on a workstation, maybe you have stumbled upon a local network related issues - do the opposite, install directly on the source server. In any case, before installing somewhere else, familiarize yourself with our Knowledge Base article on how to move installation safely.
  2. Perform a staged migration. Instead of migrating all data at once, you can migrate items only from e.g. last 90 days. Thanks to that, your users will be able to start working on the new server without any interruption, while you can comfortably finish migrating older items later.
  3. In the case of on-premises Exchange Server, you can apply optimized throttling policies to increase the speed. However, as those cannot be modified on Office 365, please consider contacting Microsoft to discuss such a possibility. Contact CodeTwo Support first, to discuss how to get in touch with Microsoft.
  4. CodeTwo software comes with the migration parameters preconfigured by the developers to maximize the migration speed based on our research, tests, our own experience, as well as suggestions from our clients. However, it is difficult to expect exactly the same set of parameters to be actually optimal for all possible scenarios. Therefore you may want play around the software settings to suit your needs, see our Knowledge Base article on that.
  5. As a last resort, you can go for “one at a time” migration mode to track down and resolve any potential issues which are interrupting and slowing down the migration process.
  6. If none of the above helps, contact CodeTwo Support following steps from this article.

Was this information useful?