SIP Test Tools

If you have spent some time on this website you likely have come across numerous guides to installing your own IP PBX phone system. When you install a system it needs to be tested. You could go out and buy loads of phones and set them up on your system and make some calls but there is a much better way, use a SIP tester. SIP test tools give you the ability to perform a load test on your phone system, sending thousands of calls using the SIP protocol with nothing more than a standard PC. It is a very inexpensive way to test your system, you don't even need an actual phone.

There are a few companies that design SIP tester tools and in this article we will highlight a few of them while focusing on one in particular. We often use this tool to test out a phone system when we are writing a new PBX tutorial. We will also highlight some tools to use for basic line quality, line speed testing and ongoing VoIP monitoring.

SIP Tester Options

There are a few options available to you for testing SIP calls, and it really depends on the type of testing required and the platform you wish to use. We will break this down into three sections, the first is a basic line quality test the second is an ongoing monitoring service and the third is for SIP load generation.

1: Line Quality Test and Line Speed Test

If you are having call quality problems, a simple line quality test or line speed test is often sufficient to pin-point problems and aid your troubleshooting. Often these kind of issues are related to the quality of your Internet connection.

At WhichVoIP.com we actually provide a line quality and speed test tool for our visitors to use, and it is free of charge. It will first test your line speed, in the uplink and downlink directions. This is important in order to know your available bandwidth, thus giving you an estimation of how many calls you can run over your Internet connection at any given time (assume 70kbits/second per SIP call). Next it will test the line quality, by simulating a SIP call and monitoring for packet loss, latencies, jitter and overall consistency of service. Finally it will produce an estimated MOS score, which is a subjective measurement of audio quality.

2: VoIP Monitoring

If you require an ongoing line quality VoIP monitoring service there are a number of companies that offer this, and for this purpose of this article we will focus on VoIP Spear and Appneta.

• VoIP Spear

  VoIP Spear is a mid-level monitoring solution for most Small and Medium sized Businesses (SMBs). Typically this kind of service does not require any installation, extra hardware, or even additional staff. It is an online VoIP testing system that assesses crucial performance parameters 24x7x365, and provides you with easy to understand reports and alerts to let you know when your service is performing below par. You just plug in your endpoints and preferred testing location, and you're good to go.

  VoIP Spear monitors key characteristics of your Internet connection to determine VoIP quality. The control panel gives you access to reports that chart your MOS, latency, and packet loss in 3 hour, 6 hour, 24 hour and 1 week periods. You can easily pinpoint the cause of problems when quality impairments occur. It is an affordable network monitoring service for most SMBs that are serious about monitoring the quality of their phone service. Below you will find a snapshot of the charts you can monitor.

  
• Appneta

  On the upper end of the spectrum are services like Appneta's PathView, which is part of their cloud-based full-stack application performance management solutions. PathView is great for large corporations that operate their own network of servers across several locations (view chart below as an example of the location monitoring). It is a very good, feature rich, VoIP monitoring solution - if you can afford it! Yes, this solution is expensive and also hard to install hence the reason this solution typically is more applicable to larger enterprises that have full time IT staff in place to manage it.

  PathView is expensive and requires complicated setup, management and maintenance. At $5,000 per server, it is more suited to large operations where stringent monitoring of all servers is necessary.

  

3: SIP Load and Packet Loss Test

If you have a phone system, such as an IP PBX, you will need to load test it with SIP calls in order to determine how the system performs under heavy traffic load. In our minds there are two tools to consider here but our focus will be on one of them only as it is much simpler to use, in our opinion.

• SIPp: SIPp is a free, open-source, Linux based, SIP load tester. This is a powerful traffic stress testing utility which also sends audio or video over the Real-time Transport Protocol (RTP) and can handle thousands of calls, provided you have the throughput capabilities. The only issue for us is that it is complex to learn and there is no Graphical User Interface (GUI) for it so everything is command driven from a terminal window and through editing configuration files.
• StarTrinity: StarTrinity is a full featured SIP load generator for Windows. Unfortunately there is no Linux version but the Windows application is very powerful and intuitive to use. It can be used for load generation and monitoring and also supports RTP streams. We also like that it has timer scheduling capabilities for running tests but also for running a monitoring function, for example send a call through the phone system every morning and alert me if it fails. Our focus for the remainder of this article will be on the StarTrinity SIP application.

StarTrinity SIP Tester

First a quick note on pricing. It should be noted that StarTrinity has a freeware license which has a limitation of 150 calls and 50 concurrent calls. If you need more, licenses are available for purchase and are relatively inexpensive. For example, for unlimited calls and 50 concurrent calls, the license fee is a one-off fee of $180, at this time of writing. This includes 6 months of technical support, which has been stellar in my experience.

I will now walk you through the steps necessary to run some traffic tests through a PBX and also indicate some ways to analyze the system in terms of CPU and RAM utilization.

1: Download and Install

This step is self-explanatory and should be a straightforward process since it is Windows based. Follow the link in the last section for StarTrinity and click on the download ZIP link. Unzip the downloaded file by double clicking on it and follow the on-screen instructions.

Once installed, double click on the application and you will be presented with the StarTrinity GUI.

2: Setup Registrations

Now you need to create the simulated phone extensions and register them with your phone system. To do this, perform the following actions:

• PBX Extensions: Create phone extensions on your PBX for testing purposes. This is left for you to do, after all we have no idea which phone system you are currently using. Examples on setting up phone extensions for the most common PBXs can be found in our Business Articles section. For the purpose of this article we decided to do a load test on a FusionPBX/FreeSwitch PBX. We created 100 phone extensions across two tenants.
• StarTrinity UAC Registrations: In StarTrinity create the 100 simulated phone extensions in the User Agent Client (UAC) Registrations section (or however many you wish to run for your test). These behave just like a SIP based IP phone or soft phone in that they try and register to the PBX on the extensions you created in the step above (make sure passwords and registrar host information marry up with your PBX). Figure 1 shows the UAC Registration screen (10 extensions shown). Make sure they register with your phone system successfully, they should also highlight green in the status column of the StarTrinity UAC registration window.



Figure 1: StarTrinity UAC Registrations

3: Configure Outbound Calling

Configure the outbound calling options as per your needs. For the purpose of this test, we set the outbound calls as highlighted in Figure 2. The important parameters are as follows:

• Interval Between Calls - Set to 100ms. This allows 10 calls per second to be made.
• Concurrent Outgoing Calls - Set to 50. Basically this will set the limit at 50 outgoing calls which equates to 100 extensions since a single concurrent call is between 2 extensions. Alternatively set to 100 but change to Incoming + Outgoing.
• Least Busy UAC Registrations - This ensures an even load between extensions.
• Codec - Set to G711U, although if your PBX supports G729 (and has the license) it may be prudent to set this to limit the bandwidth necessary for heavy loads. Running even 50 concurrent calls eats up a lot of your Internet bandwidth.
• Call Termination - Terminate call after 60 seconds. Play RTP audio for this length of time. If not answered after 20 seconds, terminate the call.
• Burst mode - One call at a time is fine since we have set it for 10 calls per second already.



Figure 2: StarTrinity Outbound Calling Configuration

4: Configure Inbound Calls

Configure the inbound calling options based on your needs. We set the inbound calls as highlighted in Figure 3. The important parameters are as follows:

• SIP Port Range - Set appropriately with respect to your PBX, 5060 is standard for SIP.
• Registration Authorization - Check this box for security reasons.
• Rejection Probability - Clear to 0%. This is important as for some reason this defaults to something other than 0% which means you will get errors injected into your phone system. I wasted some time troubleshooting this before realizing errors were being injected into my system, due to this setting.
• Codec - Set to G711U, unless you followed the notes above regarding G729.
• Limit Call Duration - Limit to 60 seconds.



Figure 3: StarTrinity Inbound Calling Configuration

5: Start Test and Monitor Results

Go back to the Outbound Configuration window and click on START test. Alternatively you can create a time schedule, if you prefer.

In the bottom log window you should be able to see calls starting up in the Current Calls tab. Also check your PBX to verify that calls are indeed coming through your phone system.

While the calls are being switched in and out of your PBX, click on the Reports/Statistics tab to see the performance metrics for your phone system. We ran our 50 concurrent SIP test calls, between our 100 extensions, for one hour and the results can be observed below in Figure 4. Note that this is licensed software that we paid for since our test is larger and is run for longer than the freeware software license allows.



Figure 4: SIP Test Call Results

You can see from Figure 4 that the performance of the system was impressive. One failed call out of 3000 outbound calls, and realistically this is likely due to some throughput issues on our Internet. StarTrinity classes this as 6000 calls since it is handling outgoing and incoming calls, though I would consider this to be 3000 actual phone calls. The other important parameters to check are as follows:

• Packet Loss - Our SIP test calls are sent using User Datagram Protocol (UDP) and there is no packet retry for UDP. It is therefore vital that packet loss is held to an absolute minimum as otherwise you will observe call quality problems, since packets will be dropped. In this test we see close to zero packet loss, which is very good.
• Jitter - Jitter is another determining factor for voice quality. Too much jitter (greater than 50ms) can result in serious audio issues. Most phones have jitter buffers to help here to some extent. In this test, jitter is impressively low.
• MOS - MOS stands for Mean Opinion Score and it is a subjective measurement of voice quality. Anything between 4 and 5 is deemed to be high quality.

Overall, the results are excellent for our test. Now let's monitor how our server is doing during the test.

6: Server Performance

So we are running our test and the results look great. The next variable at play is the server. How is it handling this traffic load and how much can we scale our system.

The FreeSwitch/FusionPBX is running on a Linux server. Here are the important specifications:

• Linux - Running Debian 8.1 on a Virtual Machine (VM).
• PBX - FreeSwitch 1.4 and FusionPBX v4.0.
• 8 Cores - We have 8 cores at our disposal running at over 2GHz per core on an Intel Central Processing Unit (CPU).
• 4GB RAM - We have 4GBytes of RAM in our system.

Linux has some very nice monitoring tools to use in order to check CPU and RAM utilization. On Debian, run the "top" command as per below and have this running during the traffic test:

top -d 2

 

Now press "SHIFT-i" which will turn OFF Irix mode and the CPU usage is divided by the number of cores, thus showing an average value. Also press '1' which will break down the CPU cores into 8 unique elements.

In Figure 5 you can see the performance of our server while running the 50 SIP call test described above.



Figure 5: CPU and RAM Utilization

Looking at the Load Averages which show the average CPU loads over 1 minute, 5 minutes and 15 minutes respectively, the results look great. A quick note on this. A one core CPU with full utilization would be 1.0, therefore the full load on 8 cores would be 8.0. At an average of 0.5, we are only using about 6% CPU utilization. We do need margin here though, as there were a few spikes occasionally at around 1.0 due to other processes running, for example firewalls.

With this in mind, we could probably run over 300 concurrent calls on this system and still have margin to spare.

Final Thoughts

In this article we highlighted some useful SIP tools. On the monitoring side these tools can perform packet loss tests, line speed tests and an overall line quality test to evaluate how suitable your Internet connection is for SIP calls. On the traffic load side we showed you how to run a large number of simulated calls to check the performance of your phone system and to test the limits regarding CPU and RAM utilization.

If you have any questions or comments we would love to hear them. Simply use the comment form below and we will respond promptly.