Take a behind the scenes take a look at how we performance-tested our internet hosting towards a number of the largest WordPress hosts on the net.

What began as a easy in-house train to see how our internet hosting measured up, rapidly changed into an interesting journey of self-discovery.

A journey we’ve determined to share with you, pricey weblog readers.

In spite of everything, we pleasure ourselves on honesty and integrity spherical these elements. And as soon as we determined we’d convey you alongside for the journey – one of many most important objectives (other than kicking a**!) was to be utterly open and clear. Each with the outcomes printed, and our testing strategies.

That approach you’ll be able to belief every thing is legit and nothing has been swayed in our favor (which advantages nobody BTW).

In order that’s what you’re getting on this article.

An inside look into how considered one of our in-house specialists examined WPMU DEV internet hosting towards a number of the hottest platforms within the biz.

Comply with alongside and be at liberty to recreate our methodology for your self.

*BTW, the entire instruments talked about on this article are utterly free!

Right here’s the way it all went down…

A creative take on all the hosting companies WPMU DEV will be up against.
Dev Man has his work minimize out for him on this battle of the hosts.

Step one was to clearly create accounts with the internet hosting suppliers we wished to pit WPMU DEV towards.

Talking of, listed here are the courageous internet hosting suppliers DEV battled on this comparability (you’ll acknowledge ALL of them, no host dodging right here):

To make the testing as truthful as doable, we in contrast the bottom stage plans of every internet hosting supplier.

We additionally used the similar primary take a look at web site and added it to every internet hosting plan.

Right here’s a peek on the take a look at web site we used (canine lovers put together to “awwww”):

We examined each host with this straightforward (and darn cute!) pet web site.

Time to get [host] testing!

Now for the enjoyable half.

As soon as we’d established the essential (and truthful) comparability factors, it was time to start out the testing course of.

We wished to see how every internet hosting server carried out beneath stress. In spite of everything, the very last thing you need is your server to fail in case you have a sudden inflow of tourists.

We additionally wished to check the pace of every host, because it’s essential to serve your shoppers in a well timed method or they may get annoyed and click on away.

So we ran two major efficiency exams on every host:

  1. A internet hosting load take a look at.
  2. A pace (TTFB) take a look at.

Right here’s how each exams unfolded, beginning with the internet hosting load take a look at:

Testing what number of parallel customers every internet hosting server might deal with.

For this load take a look at we used “https://loader.io/” a free load testing service that means that you can stress take a look at your web-apps and APIs with 1000’s of concurrent connections.

We used Loader.io to run tests on each host
Loader.io is the place to go to if you wish to see what your host is de facto made from.

Loader.io means that you can run three totally different sorts of exams:

1.”Purchasers per take a look at” – You specify the full variety of shoppers to attach over the period of the take a look at.

The clients per test which Loader.io allows you to run

2.”Purchasers per second” – Much like “Purchasers per take a look at”, however as an alternative of specifying the full, you specify the variety of shoppers to start out every second.

The clients per second test that Loader.io allows you to run

3.”Preserve shopper load” – This take a look at means that you can specify a from and a to worth for shoppers.

A look at the maintain load client test that Loader.io allows you to run

Since we have been aiming to check how every internet hosting server coped beneath consumer stress – we selected to run the “Preserve shopper load” take a look at.

As talked about, this take a look at works by permitting you to specify a from and a to worth.

What this implies is that when you specify “0” and “2000” for instance, the take a look at will begin with Zero shoppers and enhance as much as 2,000 simultaneous shoppers by the top.

Setting the shopper load take a look at boundaries.

When working every load take a look at, we set a max restrict of 5000 shoppers. We discovered this to be an acceptable restrict – as most hosts didn’t find yourself reaching 1000 shoppers anyway.

The entire exams ran for five minutes and the error failure was set to 1% as quickly as errors began to look. These errors embrace timeouts, 400/500, and community errors (all accumulating to 1%).

We selected 1% because the lowest doable worth so the take a look at would cease instantly and provides essentially the most correct studying of max parallel shoppers.

That is essential as a result of if we had the fail setting at 50% for instance, parallel shopper numbers could be loads increased, however solely as a result of extra customers are being allowed (because of the increased error setting).

When in actuality they shouldn’t depend, as they’d’ve acquired an error response – which means they have been primarily misplaced guests.

The measurements we took into consideration.

With this specific take a look at, we have been most involved with the “Response Rely,” and “Parallel Consumer” metrics.

Response Rely exhibits you the general success/failed responses:

A look at the response count metric which is part of the Loader.io test

Parallel Purchasers measures the quantity of customers the server can deal with at one time earlier than maxing out:

The parallel clients metric which is an integral part of the hosting load test

Why is the variety of parallel shoppers a server can deal with essential?

Earlier than we proceed let’s break down this concept of “parallel shoppers” a bit of additional…

In easy phrases, max parallel shoppers is the quantity of people that can ship the primary HTTP request to your web site at precisely the identical time.

For instance, let’s say your max variety of parallel shoppers was 50. This implies 50 individuals can entry the location at the very same time earlier than the server crashes.

So if 60 individuals attempt to entry on the similar time, the server will restart and present an inside server error for the subsequent couple of minutes whereas it will get again up and working – which means you’ll lose guests

Right here’s an excellent analogy we like to make use of:

“For those who desire to have a bar serving beer to 10 shoppers after which closing it down as a result of the 11th began a hearth, effective by us.”

“We’d desire a bar that serves 140 individuals in a well timed method. Even when it’s a tad slower.”

Mainly, it’s value having a bunch with the next parallel shopper quantity (even when the response time is a bit of slower) as a result of having much less parallel shopper functionality places you at extra danger of your server failing and dropping guests.

Watch a simulation of one of many load exams.

One other cool factor about Loader.io is it permits you to watch a simulation of every take a look at and the way it all went down.

Watch an instance of how WPMU DEV’s load take a look at turned out right here.

In addition to working the take a look at, Loader.io additionally means that you can watch a simulation of how the take a look at unfolds.

Subsequent, we put the pace of every host to the take a look at.

To take a look at pace we used KeyCDN’s efficiency testing instrument.

In a nutshell, the instrument exams and measures the efficiency of any URL from 10 totally different places from all over the world.

There isn’t loads to the take a look at itself, merely paste within the URL you need to take a look at and hit the button. Keep in mind it’s additionally free, so you need to use it to your personal testing.

A look at the tool we used to test each host's speed and TTFB
KeyCDN’s efficiency testing instrument offered a easy approach to take a look at every host’s TTFB.

The outcomes you get again then give a breakdown of the loading instances and HTTP response headers. As under:

An example of the results you get back from this performance test
A pleasant breakdown of your host’s pace and efficiency by location.

Wanting on the desk above, the metric we have been most occupied with for this take a look at was “TTFB.”

TTFB measures the time from a shopper making an HTTP request, to then receiving the primary byte of knowledge from the server.

The large downside with evaluating TTFB outcomes…

The one downside is, TTFB (or the pace of a bunch typically) isn’t so simple to check. It’s because the pace will fluctuate relying on the placement of the hosts server in relation to the consumer.

With WPMU DEV internet hosting for instance, our server is positioned in The Netherlands, which implies the TTFB studying from Amsterdam is at all times going to be the strongest.

So with a view to be truthful to all of the hosts concerned, we selected to current the TTFB readings in two other ways:

  1. ”Common TTFB” (Geo optimized) – This was the bottom (A.Ok.A greatest) TTFB studying out of all of the places examined.
  2. ”Common TTFB” (Throughout all places) – The typical TTFB time throughout all of the examined places.

Levelling the enjoying subject even additional.

One other essential facet about our testing is the actual fact all exams have been run WITHOUT taking caching into consideration.

Mainly this implies we examined the internet hosting servers themselves, not factoring in any caching or CDN implementations every host could have. This was performed by forcing WP to be logged-in so every thing is by-passed.

Why we expect it’s higher to check with out caching (or a CDN) enabled.

Sorry Dev Man, no caching allowed
Sorry Dev Man, no caching allowed with these exams.

In our opinion, evaluating full web page cache efficiency isn’t a good suggestion in a state of affairs like this.

We imagine this to be true for a few causes:

  1. Bypassing cache means that you can take a look at the efficiency of the internet hosting servers themselves. That is essential because it means you don’t need to depend on caching mechanisms (extra on why that is essential under).
  2. Testing with cache doesn’t take “dynamic” web site actions into consideration.

Any internet hosting platform can put a CDN in entrance of their web site, inform it to cache every thing, after which declare to present you insanely quick and scalable websites.

The issue is, this isn’t normally sensible in the actual world as WordPress, and plenty of of its plugins are supposed to be dynamic.

For instance, caching is an effective way to hurry up easy websites or pages. Like an “About Web page” – which seldom adjustments and for essentially the most half wouldn’t have a lot dwell or dynamic motion taking place.

Examine this with a full-blown eCommerce retailer that’s consistently performing dynamic actions (dwell checkout course of and so forth.) which bypass cache and hit your server straight.

That’s why you’ll usually hear of (or expertise) eCommerce shops having points throughout huge gross sales or promotions. Their servers aren’t ready (or haven’t been stress examined!) and might’t deal with all of the simultaneous dynamic motion taking place.

Mainly, your pal Mr. Cache isn’t at all times going to be there to avoid wasting you, so it’s higher to view it as an additional advantage, and to nonetheless guarantee your server goes to have the ability to cope by itself.

So how did WPMU DEV fare towards a few of hottest WordPress hosts on the net?

Tune in to half 2 of this text to seek out out!

Yep sorry, we selected to be like all of your fav Netflix exhibits and go away you with an excellent ol’ cliffhanger (it’s a superb little trick actually).

Later within the week we’ll have the total outcomes of our testing for you.

Till then, I’d be remiss if I didn’t invite you to try our internet hosting plans, or take a WPMU DEV membership (Incl. 1 Bronze stage hosted web site) for a free 7 day trial.

That approach you’ll be able to see how our internet hosting performs for your self and run your individual exams following our methodology (or every other you favor).

For those who’d quite look forward to the outcomes earlier than you give DEV internet hosting a strive, that’s cool too.

See you on the subsequent one for the reveal.