• Thanks for stopping by. Logging in to a registered account will remove all generic ads. Please reach out with any questions or concerns.

Army.ca's Server Infrastructure Changes

Mike Bobbitt

Administrator
Staff member
Owner
Directing Staff
Reaction score
55
Points
960
Summary

Our Internet service provider is changing their hosting options, and the new choices do not work well for us. We need to come up with a plan to move our services within the next 60 days.

Background

Army.ca is currently on a hosted server at Slicehost. Slicehost was purchased by Rackspace and as part of that consolidation they are no longer offering the hosting plan we use. Their new 'cloud' approach will cost us more and give us less, so it's not an appealing option. What's more, they are migrating all existing customers (that's us) over to the new cloud server architecture.

This conversion process will be complete NLT May 2012.

Rather than be stuffed into a solution that doesn't fit our needs, I thought I would use this as an opportunity to assess our requirements and resources to come up with a plan. I have outlined a couple of high level options for consideration.

For comparison, our current setup includes:

  • CPU: Quad-Core AMD Opteron(tm) Processor 2374 HE
  • RAM: 3 Gb
  • Traffic limit: 1,800 Gb/month
  • No backups
  • Disk space: 113 Gb
  • Throughput: Unknown (slicehost doesn't post this) but at least 200k/s
  • We own but a 'slice' of a virtual server
  • Monthly cost: $200 USD
  • No contract

A quick analysis on this comes up with the following important points:

  • Initially the CPU was adequate, though in the last 6+ months, the server load has moved from "frequently idle" to "usually busy". Not a critical issue but as we continue to grow we may start to see performance degrade slightly.
  • 3 Gb RAM is sometimes not enough - we could benefit from more and while it's not critical, we would see performance improvements with more.
  • 1,800 Gb of traffic/month is far more than we need, we typically use around 100 Gb/mo.
  • I am conducting my own backups, but there are performance drawbacks to doing so with a remotely hosted server.
  • 113 Gb of disk space is adequate. We currently use roughly 70% of that, with moderate growth over time.
  • Our throughput has rarely been an issue; usually some other part of the server tops out first.
  • Virtual servers are great for the hosting company as management and uptime are improved. However this often comes at the cost of performance, as your server is competing for physical resources with other virtual servers. This technology has improved drastically over the years, but sometimes problems on one 'slice' can spill to another.
  • Our availability (uptime) with Slicehost has been excellent. With a few exceptions, most of the outages have been at my hand.
  • Slicehost has no contract, if I cancelled today my bill would be prorated for the portion of the month I used the service.

Option 1 - Find another hosted solution

Pros:
  • No startup costs.
  • No hardware maintenance.

Cons:
  • Likely requires a contract, so we will be tied to a single host for 1-2 years. I tend to like to avoid this type of commitment as it locks us in to a host that may not end up suiting our needs.
  • Research is required to find an appropriate host may be extensive.
  • No console/physical access to our server. If something goes wrong it could be difficult to diagnose and fix.

Option 2 - Run a dedicated server/connection

Pros:
  • Easy management. In the past, managing a server that is in a datacenter has had it's issues. Mostly when I bring it down and can't bring it back up again!
  • Local backups are easy, fast and do not sap our performance.
  • We can get a significantly more powerful server to help handle load and growth.
  • Monthly costs are lower - I estimate half of a typical hosted service.

Cons:
  • Initial capital outlay required for new server. I estimate that a decent server will cost us $1,500, though more research is required on my part.
  • Hardware maintenance - if something goes wrong, we need to fix it.
  • A local fast connection is required, with good upload/download speeds. Bell FiberOP may fit the bill here, but is only now being rolled out to my area.
  • Uptime could be an issue; luckily I still have a 2200 VA UPS to help keep things going, at least for a short time.

These lists are far from exhaustive, and are meant more as preliminary notes at this stage.

Request

There are a lot of technically minded users here and as this change will have a big impact on how we operate, I wanted to open it up for discussion. Some folks may have been through a similar process recently, know where to find a good deal on hardware, have experience with a good hosting company, etc.

Let's get the conversation rolling!


Thanks
Mike
 

vonGarvin

Army.ca Legend
Subscriber
Reaction score
17
Points
430
I'm a Technoviking, but I'm no techno-geek, so though the music dances to me, I'm a slave to the computer.  So, my feedback is purely logic based on your original post:


Mike Bobbitt said:
Option 1 - Find another hosted solution

Pros:
  • No startup costs.
  • No hardware maintenance.

Cons:
  • Likely requires a contract, so we will be tied to a single host for 1-2 years. I tend to like to avoid this type of commitment as
  • Research is required to find an appropriate host may be extensive.
  • No console/physical access to our server. If something goes wrong it could be difficult to diagnose and fix.

Option 2 - Run a dedicated server/connection

Pros:
  • Easy management. In the past, managing a server that is in a datacenter has had it's issues. Mostly when I bring it down and can't bring it back up again!
  • Local backups are easy, fast and do not sap our performance.
  • We can get a significantly more powerful server to help handle load and growth.
  • Monthly costs are lower - I estimate half of a typical hosted service.

Cons:
  • Initial capital outlay required for new server. I estimate that a decent server will cost us $1,500, though more research is required on my part.
  • Hardware maintenance - if something goes wrong, we need to fix it.
  • A local fast connection is required, with good upload/download speeds. Bell FiberOP may fit the bill here, but is only now being rolled out to my area.
  • Uptime could be an issue; luckily I still have a 2200 VA UPS to help keep things going, at least for a short time.

These lists are far from exhaustive, and are meant more as preliminary notes at this stage.
Of these two options, I would go with option 2.  The kicker for me was this:
If something goes wrong it could be difficult to diagnose and fix.

Hardware maintenance - if something goes wrong, we need to fix it.

I would change the second into a "pro" and restate is this:  if something goes wrong, we can fix it.

 

GAP

Army.ca Legend
Subscriber
Donor
Mentor
Reaction score
3
Points
380
Not if he's away....server's are backed up using Raid, etc. to reduce data loss.
 

2010newbie

Full Member
Reaction score
0
Points
0
I've used this company before and they were always reliable and cost-effective.

http://www.bluegenesis.ca/

 

agc

Full Member
Reaction score
0
Points
0
I'm certainly no expert, but you might wish to investigate Amazon EC2. http://aws.amazon.com/ec2/pricing/
 

Gunner98

Army.ca Veteran
Subscriber
Donor
Mentor
Reaction score
0
Points
0
Mike,

Wouldn't an option be to close the shop and go hunting, fishing and spend time with your family?  Think of the free time you and the rest of us would have.  Just sayin'! >:D
 

Zoomie

Army.ca Veteran
Mentor
Reaction score
20
Points
430
I still fixated on the part where Mike mentions that we only currently use 100GB/month of the allowable 1800GB/month.  I don't think the Mods can use the whole bandwidth argument anymore - new rule for the server - don't use the SEARCH function.
 

armyvern

Army.ca Myth
Subscriber
Mentor
Reaction score
15
Points
430
Zoomie said:
I still fixated on the part where Mike mentions that we only currently use 100GB/month of the allowable 1800GB/month.  I don't think the Mods can use the whole bandwidth argument anymore - new rule for the server - don't use the SEARCH function.

Yeah, but that was way back when I was a mod too. Mike was running off his server vice the virtual and we all used to scream, "Mike, come back off vacation dammit!!" Uhhmmmm, good old days.  >:D
 

PuckChaser

Army.ca Fixture
Staff member
Directing Staff
Mentor
Reaction score
299
Points
1,030
I really think an offsite host is going to be the best option here. They will have the racks, the backups, and the connection. Your post was cut off, what is the reasoning behind not wanting a 1 year contract?
 

captloadie

Sr. Member
Subscriber
Reaction score
16
Points
230
The offsite option, if with a reliable company, should be the best option. Yes, having everything in house is nice, but in this case, it is literally in house (Mike's that is). If Mike is away, and things go pear shaped, what happens? If Mike finally realizes that there is more than just Milnet.ca to life, what happens then?

An offsite option should give Mike more flexibility, like assigning other trusted users access to do admin functions when he is away, and the ability to hand over the keys forever, if he ever wants to. It also means that if for some reason a legal issue arose that required the servers be taken into custody, he doesn't have his personal space invaded. (Alright, this last one is out there, but ehh, who knows what the man who's name shall not be mentioned on this site might do.)

My  :2c:
 

Mike Bobbitt

Administrator
Staff member
Owner
Directing Staff
Reaction score
55
Points
960
Thanks guys, I appreciate the feedback. All good points and will be considered. Keep the feedback coming!

2010newbie said:
I've used this company before and they were always reliable and cost-effective.

http://www.bluegenesis.ca/

Thanks, I hadn't head of Blue Genesis. They look like a decent hosting company, but I'm pretty fussy about our needs. Specifically, I don't want a shared account on a server, I prefer to have full control (insert maniacal laugh here). It looks like Blue Genesis doesn't give me my own server with root and console access to mould in my own image. Do you know what their throughput is like? (I.E. how much simultaneous traffic they can support; their actual speed provisioned to users?)

I ask because I have a DreamHost account that has unlimited monthly traffic, but the connection speed is so slow it is unusable for a 'big' task like Army.ca. (It's also a shared account - no root, no Mike. :))

agc said:
I'm certainly no expert, but you might wish to investigate Amazon EC2. http://aws.amazon.com/ec2/pricing/

The last time I checked on this, it wasn't available in Canada, so I dismissed it this round. Looks like that has changed. I have signed up and will start investigating. Overall I have heard very good things about EC2 so I will be paying close attention here!

Zoomie said:
I still fixated on the part where Mike mentions that we only currently use 100GB/month of the allowable 1800GB/month.  I don't think the Mods can use the whole bandwidth argument anymore - new rule for the server - don't use the SEARCH function.

It's more about conservation of sanity than bandwidth. ;) All kidding aside, while this is mostly a leftover from the days of resource constraints, it is still a 'best practice.' Don't forget, while they allow us to use 1,800 Gb per month, our connection speed may not allow that in practice. See my note above about DreamHost's unlimited traffic; that account would crawl under Army.ca's load because the pipe is not big enough to serve our traffic needs. An analogy would be if someone gave you a lawn tractor and said "don't drive it over 50,000km per month." Probably can't hit that limit even if you drove it at 'top speed' 24/7. In short: you're still competing for traffic with every other user for limited traffic delivery, regardless of what our monthly cap is.

PuckChaser said:
I really think an offsite host is going to be the best option here. They will have the racks, the backups, and the connection. Your post was cut off, what is the reasoning behind not wanting a 1 year contract?

Sorry, fixed that line:

Mike Bobbitt said:
Likely requires a contract, so we will be tied to a single host for 1-2 years. I tend to like to avoid this type of commitment as it locks us in to a host that may not end up suiting our needs.
 

Prefect

Guest
Reaction score
0
Points
0
You should definitely check out iweb.com. They operate 4 datacentres in Montreal.

I've had my server colocated with them for the last 5 years and have had very few issues. (Uptime on my system currently stands at 475days.) They have stopped offering colocation services for individual servers but their dedicated server prices are reasonable.

Here is an example of what they call a "Smart Server". (Virtualized for ease of remote management but the server itself is only used by one person.)

CPU: Core 2 Quad 2.66ghz
Mem: 4GB
HDD: 2x 1TB SATA2 (RAID 1)
Connection: 100mbit
Bandwidth: 10,000GB
IPs: 5 usable in a /29 block
OS: CentOS/Debian/Ubuntu

Price: $159 + taxes/month

As for actual throughput, from my experience as a colocation customer it is always a full 100mbit up/down with no exceptions.
 

Mike Bobbitt

Administrator
Staff member
Owner
Directing Staff
Reaction score
55
Points
960
Excellent. good to know. And it keeps it Canadian too, which is nice. I'll definitely check into them as well.
 

josh54243

Member
Reaction score
0
Points
0
Throwing my hat into iWeb.com as well. Used both their dedicated and shared services and was a pleasant experience.
 

wildman0101

Sr. Member
Mentor
Fallen Comrade
Reaction score
0
Points
0
Mike. Give netzero a call. 1800-560-7429. They got some good package deal's
both personal/web. Might be just what your lookin for. Cheer's mate and best
regard's. Scoty B
 

Mike Bobbitt

Administrator
Staff member
Owner
Directing Staff
Reaction score
55
Points
960
A quick update: I have settled on Option 1 (new host) as it looks like there are some good options available. The cost and extra work of a local setup is probably too much.

I have not yet settled on a host, but thanks to the suggestions here, I have a short list to work from.

As part of this transition, we will be moving from Fedora to Ubuntu, which is going to be a big change for me. The next major revision of Ubuntu comes out in about 3 weeks, so I will wait and use that as a starting point.

More to follow.
 

Mike Bobbitt

Administrator
Staff member
Owner
Directing Staff
Reaction score
55
Points
960
Another update, I have settled on Amazon's EC2 service as it has the most flexibility and the best price, depending on the package we end up with.

The current package I am looking at is $600 up front with a small monthly fee for 3 years. This gets us a server comparable to the one we use today.

I'm trying to find a way to get a much improved server; it's costs are $1,200 USD up front but about the same monthly costs as the above package. This gets us on a server that is significantly better than the one we are using now. I'm crunching some numbers to see if we can make that option work. In the long run, it is a much better deal for us and allows future growth.

I'll provide an update shortly.


Cheers
Mike
 

GAP

Army.ca Legend
Subscriber
Donor
Mentor
Reaction score
3
Points
380
Many here would not be adverse to "kicking in" per se if it substantially upgrades the server.....just saying..... ;D
 
Top