Members MattACaster Posted October 1, 2009 Members Share Posted October 1, 2009 A critical server went down at 3am last night. Luckily I got the backup fired so I didn't have to drive it to work. Got in this morning to find both hard drives in the server fried. So much for that mirrored RAID array for a backup. Link to comment Share on other sites More sharing options...
Members mcr23 Posted October 1, 2009 Members Share Posted October 1, 2009 Should have gotton a Mac Link to comment Share on other sites More sharing options...
Members NixerX Posted October 1, 2009 Members Share Posted October 1, 2009 Love it. What caused it? Spike? Link to comment Share on other sites More sharing options...
Members MattACaster Posted October 1, 2009 Author Members Share Posted October 1, 2009 Should have gotton a Mac Link to comment Share on other sites More sharing options...
Members MattACaster Posted October 1, 2009 Author Members Share Posted October 1, 2009 Love it. What caused it? Spike? I have no idea actually. Everything else seems to be running fine. The PC itself seems Ok when I pop another drive into it. Both drives failed the drive diagnostics within the first 5 seconds of the test. Link to comment Share on other sites More sharing options...
Members Aibric Posted October 1, 2009 Members Share Posted October 1, 2009 Need more redundancy. Link to comment Share on other sites More sharing options...
Members NixerX Posted October 1, 2009 Members Share Posted October 1, 2009 Damn. SCSI drives usually have a 5yr warranty on them. maybe you could send'em back and put the new ones on ebay Hows the UPS its plugged into? Id have your electrician do a ground test. Link to comment Share on other sites More sharing options...
Members MattACaster Posted October 1, 2009 Author Members Share Posted October 1, 2009 Need more redundancy. This server processes text files from our IBM iSeries then pretties them up when they print. It's for work orders, invoices and the paper work the truck drivers needs to leave. The backup server picked up the slack enough to get through the night. It's not as fast as the main server and only processes what the company needs to limp by. I figured redundant hard drives were enough.... Link to comment Share on other sites More sharing options...
Members MattACaster Posted October 1, 2009 Author Members Share Posted October 1, 2009 Damn. SCSI drives usually have a 5yr warranty on them. maybe you could send'em back and put the new ones on ebay Hows the UPS its plugged into? Id have your electrician do a ground test. These are straight SATA drives. The app it runs isn't HD intensive. Just a pair of Seagate 160GB drives. The UPS is wired to the outlets that all the servers are plugged into. I would think we would see more issues with other things if it was a UPS issue? Link to comment Share on other sites More sharing options...
Members Aibric Posted October 1, 2009 Members Share Posted October 1, 2009 This server processes text files from our IBM iSeries then pretties them up when they print. It's for work orders, invoices and the paper work the truck drivers needs to leave. The backup server picked up the slack enough to get through the night. It's not as fast as the main server and only processes what the company needs to limp by. I figured redundant hard drives were enough.... Well that's pretty good, then. Slowing down a bit is much better than shutting down. The only other thing I would think you could do is upgrade to a backup server that is an actual duplicate of the main machine to have a true standy failover server. They would have their bases covered pretty well then. Link to comment Share on other sites More sharing options...
Members NixerX Posted October 1, 2009 Members Share Posted October 1, 2009 These are straight SATA drives. The app it runs isn't HD intensive. Just a pair of Seagate 160GB drives.The UPS is wired to the outlets that all the servers are plugged into. I would think we would see more issues with other things if it was a UPS issue? Im just sayin there's a reason the all went at once. that is extremely unusual. Link to comment Share on other sites More sharing options...
Members mcr23 Posted October 1, 2009 Members Share Posted October 1, 2009 should have gotton the ultra? Link to comment Share on other sites More sharing options...
Members Dolf Posted October 1, 2009 Members Share Posted October 1, 2009 We do daily drive mirrored backups. A different drive for every day, mirrored at 10pm. Hotswappable, takes 4 minutes tops to change a drive out. Push come to shove, we'd have to have 6 drives fail simultaneously to lose all of our data. Link to comment Share on other sites More sharing options...
Members MattACaster Posted October 1, 2009 Author Members Share Posted October 1, 2009 Well that's pretty good, then. Slowing down a bit is much better than shutting down. The only other thing I would think you could do is upgrade to a backup server that is an actual duplicate of the main machine to have a true standy failover server. They would have their bases covered pretty well then. Same here. While I don't really have a budget, this company doesn't like spending a lot of money on IT. I do what I gotta do to get by. I've been pushing to move to something like Citrix recently just to get all out of other facilities (most of which don't have onsite IT staff) on the same page as far as software and updates. So far, there is little interest from the higher ups. Link to comment Share on other sites More sharing options...
Members MattACaster Posted October 1, 2009 Author Members Share Posted October 1, 2009 Im just sayin there's a reason the all went at once. that is extremely unusual. No, I totally agree. I think I'll have the maintenance guys come in and take a look at the electrical work in here. Thanks for the advice. Link to comment Share on other sites More sharing options...
Members NixerX Posted October 1, 2009 Members Share Posted October 1, 2009 No, I totally agree. I think I'll have the maintenance guys come in and take a look at the electrical work in here. Thanks for the advice. NP! Anytime IT broseph! And why the {censored} dosent this {censored} happen in the middle of the day?! Link to comment Share on other sites More sharing options...
Members MattACaster Posted October 1, 2009 Author Members Share Posted October 1, 2009 NP! Anytime IT broseph! And why the {censored} dosent this {censored} happen in the middle of the day?! No kidding. I got a wakeup call at 11pm because someones printer was out of paper. I got a call at 1am because no one told the night shift people that when it's end of the month, the iSeries is down longer and night when it does it's backups. Then again at 3am for this whole thing. I'm {censored}ing exhausted. Link to comment Share on other sites More sharing options...
Moderators justcrash Posted October 1, 2009 Moderators Share Posted October 1, 2009 Same here. While I don't really have a budget, this company doesn't like spending a lot of money on IT. I do what I gotta do to get by. I've been pushing to move to something like Citrix recently just to get all out of other facilities (most of which don't have onsite IT staff) on the same page as far as software and updates. So far, there is little interest from the higher ups. Dude, I work in a public school system... it's silly the hoops I have to jump through to get anything. As of now, it's been 6 or 7 months since we had a major server crash, and yet, magically, STILL NO FUNDS FOR A BACK UP SERVER. I want to get a duplicate serve like the one I just built, for a failover and there is also talk of going with a VoIP solution (our phones are all hodge podged) but that seems to be what we excell at... talk. Link to comment Share on other sites More sharing options...
Members NixerX Posted October 1, 2009 Members Share Posted October 1, 2009 Dude, I work in a public school system... it's silly the hoops I have to jump through to get anything. As of now, it's been 6 or 7 months since we had a major server crash, and yet, magically, STILL NO FUNDS FOR A BACK UP SERVER. I want to get a duplicate serve like the one I just built, for a failover and there is also talk of going with a VoIP solution (our phones are all hodge podged) but that seems to be what we excell at... talk. JC I just plain feel bad for you man.. you just wanna do whats right for the place and your getting financially neutered. Link to comment Share on other sites More sharing options...
Members MattACaster Posted October 1, 2009 Author Members Share Posted October 1, 2009 Dude, I work in a public school system... it's silly the hoops I have to jump through to get anything. As of now, it's been 6 or 7 months since we had a major server crash, and yet, magically, STILL NO FUNDS FOR A BACK UP SERVER. I want to get a duplicate serve like the one I just built, for a failover and there is also talk of going with a VoIP solution (our phones are all hodge podged) but that seems to be what we excell at... talk. I'd like to think, for what I have to work with, I have a pretty good system. Backup tapes for the 100% critical stuff and all my iSeries support servers are run in Virtuals so if something dies, I can just move that virtual to another box and, at worst, I have to divide resources up more so things run slower. God, I hate our new phone system. We have unified messaging so all faxes and voice mail go to Outlook. Except the client you have to install to get Outlook to do this is complete garbage. They have to be reinstlled on what seems like a weekly basis. Plus, I had almost nothing to do with the install of the servers for this. I have no idea how they work and they are not a part of my domain. Anytime something does work with them, I just go.... Link to comment Share on other sites More sharing options...
Moderators justcrash Posted October 1, 2009 Moderators Share Posted October 1, 2009 JC I just plain feel bad for you man.. you just wanna do whats right for the place and your getting financially neutered. I really am. The bitch of it is, when the next server crashes, the "server guy" will be to blame, me, even though I was warning them since before the LAST crash. Ah well... when life gives you lemons and all that. Link to comment Share on other sites More sharing options...
Members MattACaster Posted October 1, 2009 Author Members Share Posted October 1, 2009 I bet you are. I was up till 1 IPL ing our iseries after a good dose of PTF's ( gold star if you know what those are ) Im glad I work for a small bank....I dont have on call, weekend....ive done my time with that JC I just plain feel bad for you man.. you just wanna do whats right for the place and your getting financially neutered. Yeah, I know all about that... Luckily my company is flexible with hours. I'm hourly so they don't really care when I work as long as I get done what I need to do. Plus, time and a half. Link to comment Share on other sites More sharing options...
Moderators justcrash Posted October 1, 2009 Moderators Share Posted October 1, 2009 I'd like to think, for what I have to work with, I have a pretty good system. Backup tapes for the 100% critical stuff and all my iSeries support servers are run in Virtuals so if something dies, I can just move that virtual to another box and, at worst, I have to divide resources up more so things run slower. God, I hate our new phone system. We have unified messaging so all faxes and voice mail go to Outlook. Except the client you have to install to get Outlook to do this is complete garbage. They have to be reinstlled on what seems like a weekly basis. Plus, I had almost nothing to do with the install of the servers for this. I have no idea how they work and they are not a part of my domain. Anytime something does work with them, I just go.... Ours is 100x better than it was when I started but yeah, no back up = What kind of phone system did you go with. Right now one school can't just call an extension to get to another schoo, they have to dial the main number, like a parent, only 6 lines at the HS, but every teacher wants their own phone line , if the voice mail goes down at one schoo, we have to drive across town to ANOTHER school to reset just that other schools voice mail... sense... we make none. Link to comment Share on other sites More sharing options...
Members MattACaster Posted October 1, 2009 Author Members Share Posted October 1, 2009 Oh! Awesome phone system story... So for the longest time, we've been fighting with space issues on the Exchange server. Trying to get people to keep their mailboxes clean because we were constantly filling up the HDs and the company didn't want to buy new ones. So, our Oshkosh branch recently got a new phone system with the unified messaging (voicemail and faxes come to email). When they are talking about this, my department throught it would be like the system we have here in Chicago. Messages are kept on another server but still show up in Exchange. NO, THEY ARE KEPT ON THE EXCHANGE SERVER AND NO ONE TOLD US!!! All of the suddent Exchange shuts down one night because there is literally no space left on it. PLUS, the mailstore outgrew my backup tapes!!! {censored}. I've never been so pissed in my whole life... Link to comment Share on other sites More sharing options...
Members NixerX Posted October 1, 2009 Members Share Posted October 1, 2009 Yeah, I know all about that... Luckily my company is flexible with hours. I'm hourly so they don't really care when I work as long as I get done what I need to do. Plus, time and a half. Fun fun. I got back ups tonight so it will be another late one...we are doing outlink so our AS400 will go bye bye.... what software do you guys run? we run Jack Henry Link to comment Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.