Site Outages
|
Joe Kriz |
Posted on 10/24/07 - 11:57 AM
|
Site Owner
Personal Page
Personal Album
Photo Albums
Project Albums
Posts: 11447
Comments:
452
Joined: 03/18/05
|
Just wanted to give everyone a little information about all of our site outages the past 6 days.
Our Hosting service has been upgrading their hardware and software.
Unfortunately, this has NOT gone well.
We were down again yesterday, Tuesday, from 3 PM to about 4 AM this morning...
NOT good.
Our Hosting service supports 125,000 customers and I assume all of them were just as upset as we are.
It's not an easy thing switching hosting services as all the files and databases have to be transferred to the new site plus the domain name takes a day or two to propagate over the Internet. We really hope we don't need to change hosting services but if the outages continue, we will have no choice as this much down time is unacceptable........
Lets hope that they have these growth and upgrade problems under control and there are NO MORE outages.
Joe Kriz
|
|
|
|
sraab928 |
Posted on 10/24/07 - 4:25 PM
|
Member
Project Albums
Posts: 1282
Comments:
13
Joined: 12/27/05
|
Sometimes you just have to deal with it. Its a reality with some hosting services. I'm sure its tough on their end too. Hopefully it will be all over soon an it will get back to normal.
Scott
1974 21' Revenge w Mercury 200 ProXS V8 |
|
|
|
kamie |
Posted on 10/24/07 - 4:47 PM
|
Member
Personal Page
Project Albums
Posts: 2975
Comments:
3
Joined: 11/04/05
|
I am assuming that with all the outages this month is free and probably next month as well? I would have my service rep and their boss on a conference call, ASAP!
Yes, I have been on those calls, not pretty
|
|
|
|
Joe Kriz |
Posted on 10/24/07 - 5:04 PM
|
Site Owner
Personal Page
Personal Album
Photo Albums
Project Albums
Posts: 11447
Comments:
452
Joined: 03/18/05
|
I would imagine that "some heads are going to roll" at there company.
Yes, they gave us 2 months free service... However, if it is always down, what good is 2 months of NO service.......???
What is strange to me is that they are supposed to have a built-in redundancy....
This to me means if one set of servers or routers go down, then the second set keeps the customers up and running.
However, this was not the case as we were down completely, so where was the redundancy????
Beside the outages, we were happy with their service.
Lots of room to grow and a lot of nice features available....
I have shell shock now and any time there is a moments glitch, I get a little nervous....
Time will tell....
|
|
|
|
Phil T |
Posted on 10/24/07 - 5:21 PM
|
Administrator
Personal Page
Personal Album
Project Albums
Posts: 7042
Comments:
6
Joined: 03/26/05
|
Prior to being doing daddy-daycare, I was a Project Manager in IT (10+ yrs) and can tell you there are always problems in any "redundant" system. Some big and some huge.
Upgrades or migrations for the web and back office are ALWAYS bumpy. It is only a question of the degree.
I presume their "plan" has fallen apart due to unforseen issues. Now it is just a question of how long it will take them to get the process finished. If this is the last outage, I take that as a good indicator that they have good techs. Could be a third party (Fiber provider, hardware manufacturer etc.) and out of their direct control. Seen many projects have much worse issues.
If the company has listened, expressed appreciation and apologized, that is a good sign.
Think hard before you bail. A known foe is better than a stranger.
Edited by Phil T on 10/24/07 - 5:21 PM
1992 Outrage 17 I
2019 E-TEC 90, Viper 17 2+
2018 Load Rite Elite 18280096VT |
|
|
|
cdnwhaler |
Posted on 10/24/07 - 5:36 PM
|
Member
Personal Page
Posts: 200
Comments:
0
Joined: 04/01/05
|
While I'm as peeved as most are I have to admit Casco Bay Outrage's comments do make some sense.
"Think hard before you bail. A known foe is better than a stranger"
And I'm typing as person who has very little patience these days
Pete
1984 Outrage 22 |
|
|
|
Joe Kriz |
Posted on 10/24/07 - 5:37 PM
|
Site Owner
Personal Page
Personal Album
Photo Albums
Project Albums
Posts: 11447
Comments:
452
Joined: 03/18/05
|
We are hanging in there for now...
When I get frustrated and fed up, it makes me think of the 2 Vultures sitting in the tree top.
One vulture said to the other: Be patient, something will come along.
The other vulture replied: Patient my rear, I want to kill something.....
I'm sure it can't be easy taking care of all the equipment and hearing from 125,000 customers who don't have any patience with outages. I know I wouldn't want to have their job(s)...
|
|
|
|
arthureld |
Posted on 10/24/07 - 5:48 PM
|
Member
Personal Page
Posts: 735
Comments:
7
Joined: 02/15/07
|
It's probably a good time to make a backup. I need to follow my own advice.
I switched hosting services once and it wasn't a big deal for me, but my site wasn't being used by anyone but me at the time.
If you do switch, it would probably be best to have the site up on the old and the new hosting service until you are sure everything is working.
By the way, I'm not an IT guy. I'm just a nut that likes to mess with web development. I also teach a online CADCAM class that I put together with Moodle.
|
|
|
|
kamie |
Posted on 10/24/07 - 6:47 PM
|
Member
Personal Page
Project Albums
Posts: 2975
Comments:
3
Joined: 11/04/05
|
Joe Kriz wrote:
What is strange to me is that they are supposed to have a built-in redundancy....
This to me means if one set of servers or routers go down, then the second set keeps the customers up and running.
However, this was not the case as we were down completely, so where was the redundancy????
Joe,
your right 2 free months would be nothing if you were down for those entire months. But if your down for 12 hours out of 1440 it's not so bad a percentage (99.16%) depending on their stated guarantee. I would also have them go over the root cause with you and yes they should be doing one to see what happened. Ask them exactly what components are redundant in their infrastructure and which are not. I have seen systems fail, I have seen a software upgrade to one switch of a redundant pair take both down, I have seen a database on the primary server have issues, only to fail over to DR and have different issues on the DR side. You can do belt and suspenders but you still have the risk that both will fail. You can do two belts and two suspenders and your risk of failure is lower but can you afford the cost for that lower risk, and the more redundancy you have the bigger the price tag.
|
|
|