Profiling: how the program influences your server and email flow

In this article you will learn about the effect our software has on email flow in Exchange environments.

• Overview and summary of results
• Specification of the test environment
• Testing procedure
• Results

Overview

The aim of these tests was to analyze how CodeTwo Exchange Rules Pro influences email flow and how much load the software puts on an Exchange server in terms of CPU and RAM usage. To realize this objective, the following elements were prepared:

• a test Exchange environment,
• a package of sample emails in the EML format,
• dedicated tools for email processing and server load (CPU & RAM) measurements.

4 different configurations of the environment (both with and without CodeTwo Exchange Rules Pro) were tested. To simulate different mail flow scenarios, emails were sent with the speed of 5, 10 and 20 mails per second (mps). You might want to know how these numbers are related to mail flow in a typical organization. For example, the speed of 20 mails per second means that:

• in an organization with 500 AD users: each user sends 144 emails per hour (i.e. 1 message every 25 seconds);
• in an organization with 1000 AD users: each user sends 72 emails per hour (1 message every 50 seconds).

Summary of results

In our tests, CodeTwo Exchange Rules Pro:

• increased the average email processing time by up to 0.27 s, which is virtually not visible for the end user (show details);
• had no visible effect on email delivery speed (show details);
• was measured to increase average CPU load by 6.2% and average RAM utilization by 2% when 10 signature rules were defined in the program (show details).

Read on to learn more about the environment we used during the tests, and view all the results in greater detail.

Environment

Specification

The environment used for testing was based on MS Exchange 2016 (Tab. 1.).

Mail flow configurations

For the purpose of this study, 5 configurations of the test environment were prepared:

1. MS Exchange Server only - in this setup, no Exchange Server transport rules were defined and CodeTwo Exchange Rules Pro was not running,
2. Exchange + 1 Exchange transport rule - here, one Exchange transport rule was defined as in Fig. 1. It appended an HTML disclaimer for all senders. The disclaimer contained several placeholders and a hyperlinked image. CodeTwo Exchange Rules Pro was not running.

Fig. 1. The test Exchange transport rule.

3. Exchange + CodeTwo Exchange Rules Pro + 1 ER Pro signature rule - in this configuration, only a single CodeTwo Exchange Rules Pro signature rule was configured (there were no Exchange transport rules). The rule inserted an HTML signature with placeholders and an embedded image (see Fig. 2.) if the message sender was internal (which was true for all emails sent in this environment). The Sent Items Update service was disabled.

Fig. 2. One of the test CodeTwo Exchange Rules Pro signature rules.

4. Exchange + CodeTwo Exchange Rules Pro + 10 ER Pro signature rules - in this configuration was the same as configuration 3 above, but 10 CodeTwo Exchange Rules Pro rules were configured. Those rules inserted various signatures to messages sent in the environment. The content of each signature was similar - a few placeholders and an embedded picture, like the rule shown in Fig. 2. The conditions for those 10 rules were defined in such a way that 1 rule was applied to each email. There were no Exchange transport rules; the SIU service was disabled.
5. Exchange + CodeTwo Exchange Rules Pro + 1 ER Pro signature rule + SIU - the configuration was the same as configuration 3 above, but the Sent Items Update service was enabled. This setup was used for the Sent Items Update tests (see the next section for details).

Test subject – sample emails

A collection of 70 sample emails in the EML format was prepared for testing. The package included new emails and replies, which varied in content (tables, hidden and visible attachments, etc.) and size: there were 65 EML files between 4 KB and 100 KB, 4 EMLs in the 100-126 KB range and a single EML file of 1 MB.

Testing procedure

Testing was based on internal messaging in the Exchange environment described earlier. In total, 3 types of experiments were performed. They all focused on measurements of server load and email processing times in various configurations of the test environment (see the summary in Tab. 2.)

A dedicated application picked random emails from the collection of samples. For each message, the program assigned a unique 36-character identifier (MessageId) and randomly selected 1 sender and 1 recipient from the 1000 users available in the test environment. The program then copied these messages to an Exchange Pickup directory (Pickup folder) located on the Exchange server, where they were subjected to further processing. Learn more about the Transport service on Mailbox servers

Three types of tests were performed:

• in test 1, the application copied emails to the Pickup directory with the speed of 10 mps (mails per second); mail flow configurations 1-4 were tested;
• in test 2, the application copied emails to the Pickup directory with the speed of 20 mps; mail flow configurations 1-4 were tested;
• in test 3, messages were sent (with the speed of 5 mps) via Exchange Web Services (EWS) instead of the Pickup directory, so that these messages could be delivered to the Sent Items folder and then processed by the Sent Items Update service of CodeTwo Exchange Rules Pro. Mail flow configuration 5 was used.

During each 1 hour long test the program measured the CPU and RAM usage on the server. Another dedicated application analyzed email processing times and calculated average delivery speed.

Measurement of CPU and RAM usage

CPU load was measured via the Windows Performance Counter: % Processor Time whereas RAM usage was measured with the Working Set - Private Performance Counter. CPU and RAM measurements were performed for the following processes (components) of CodeTwo Exchange Rules Pro:

• Exchange Rules Service (CodeTwo.ER.Engine.Service)
• Sent Items Update (CodeTwo.ER.SentItemsUpdate.Service)
• the Attendant service (CodeTwo.ER.Attendant.Service)

and for the following Microsoft Exchange Server process: Edge Transport (EdgeTransport).

Measurement of email processing time and delivery speed

A dedicated application analyzed email processing time. The program retrieved the necessary data from Exchange Server, by running  Exchange Management Shell and tracking a message via its MessageId identifier (Fig. 3.). Email processing time was calculated as a time difference between the PICKUP RECEIVE and SMTP SEND events. The average email processing time was obtained by adding the processing times of all messages together and dividing them by the total number of these messages.

Fig. 3. Analysis of the processing time of an email.

Exchange Server can process several messages at the same time. That is why average email delivery speed cannot be calculated directly from the average email processing time. To calculate the average delivery speed of messages (in mps - mails per second), the total number of messages delivered during a single test (see Fig. 4.) was divided by the duration of this test (i.e. 3600 seconds).

Fig. 4. Checking the number of messages delivered during a single test.

Results

This section summarizes the results for all the tests. For details, see tables 3-5 at the end of this section.

• Email processing time
• Average delivery speed
• CPU load
• RAM usage

Email processing time

The results of email processing time measurements for tests 1 and 2 are shown in Fig. 5. (for more details, see Tab. 3. and 4. at the end of this article). When emails were sent with the speed of 10 mails per second, it took 1.242 s (on average) for Exchange Server to process an internal message without any transport rules or CodeTwo software. Installation of CodeTwo Exchange Rules Pro and defining 10 signature rules in the program increased this time to 1.507 s. For 20 mps, the corresponding times were 2.233 s (Exchange only) and 2.477 s (CodeTwo Exchange Rules Pro + 10 rules). The increase in the processing time caused by our software is practically not observable for the user. There is a 1-second difference in the average email processing time when moving from 10 to 20 mps, but it is caused by the heavier load on the server (because more messages are sent), not by our application.

Fig. 5. Average email processing times.

Email delivery speed

Here, we investigated if emails are delivered with the same speed as the speed with which they were submitted for delivery, by counting messages delivered during a single 1 hour long test. When emails were sent with the speed of 10 mps, they were delivered with the speed of 9.99 mps (Tab. 3.). For 20 mps, the measured delivery speed was 19.99 mps (Tab. 4.). It is clearly visible that increasing the number of sent messages does not influence their delivery. The small difference between the speed of sending and delivering comes from the fact that messages that were sent near the end of a test were delivered shortly after the test, and thus they were not taken into calculations.

It was also observed that CodeTwo Exchange Rules Pro does not influence email delivery - the average delivery speed was the same no matter if the program was installed or not.

CPU load

The results of the CPU usage measurements for tests 1 and 2 are shown in Fig. 6. (for even more details, see Tab. 3. and 4. at the end of this article). When emails were sent with 10 mps, the CPU utilization was 2.81% for the EdgeTransport process. If one Exchange transport rule was working, this increased to 3.56%. Installing and running CodeTwo Exchange Rules Pro with 10 rules changed the CPU usage to 6.02% (this includes the additional load on the EdgeTransport service and 3 services of our software). When messages were submitted for delivery with 20 mps, the processor load was 5.83% for Exchange Server (EdgeTransport) only, and increased to 12.03% when CodeTwo Exchange Rules Pro with 10 rules was running.

Fig. 6. Average CPU usage.

When emails were sent with 5 mps (via EWS), with CodeTwo Exchange Rules Pro running with the Sent Items Update service enabled, the software's load on CPU was 4.4% in total: 2.72% for EdgeTransport, 1.36% for the SIU service and 0.32% for the other CodeTwo services (Fig. 7.). For more details, see Tab. 5. at the end of this article.

Fig. 7. CPU usage by individual MS Exchange and CodeTwo processes.

RAM usage

When messages were sent with 10 mps (Fig. 8.), Exchange Server (EdgeTransport) used 727 MB on average, which was 4.4% of the whole memory (16 GB). Installing and running CodeTwo Exchange Rules Pro with 1 rule increased this value to 1018 MB (including the additional load on the EdgeTransport service and 3 services of our software), which corresponds to a rise of 1.8% in the usage of the whole available memory. When 10 signature rules were configured, the average RAM usage was 859 MB (0.8% increase in the total memory usage). When emails were sent with 20 mps (Fig. 8., right), EdgeTransport used 701 MB / 4.3% on average. With our software installed and running, the total RAM utilization increased by 1.9% (to 1023 MB) for 1 signature rule and by 1.7% (to 992 MB) for 10 signature rules. For more details, see Tab. 3. and 4. at the end of this article.

Fig. 8. Average RAM usage.

When messages were sent with 5 mps via EWS and CodeTwo Exchange Rules Pro (with the Sent Items Update service) was running, memory usage was 460 MB (2.81% of the available RAM) for Exchange Server (EdgeTransport), 85 MB (0.52%) for SIU, and 77 MB (0.47%) for the rest of CodeTwo processes, as shown in Fig. 9. For more details, see Tab. 5. at the end of this article.

Fig. 9. RAM usage by individual MS Exchange and CodeTwo processes.

Below you can find tables with detailed results from tests 1-3.