Omega Owners Forum

Chat Area => General Discussion Area => Topic started by: TheBoy on 24 February 2009, 21:26:52

Title: Typical
Post by: TheBoy on 24 February 2009, 21:26:52
Arghhhh!

Just rebooted one of my brother's servers at his shop, not come back.  Bloody Lights-Out board has stopped responding as well....

I'm guessing I won't be Mr Popular when he gets in tomorrow morning ::)
Title: Re: Typical
Post by: The Red Baron on 24 February 2009, 21:28:03
I think not :-[
Title: Re: Typical
Post by: doog on 24 February 2009, 21:39:02
deny it
you never touched it etc ......


Doug
Title: Re: Typical
Post by: TheBoy on 24 February 2009, 21:50:28
Quote
deny it
you never touched it etc ......


Doug
He knows I was planning to do some work tonight - I'm already not Mr Popular as a new software drop (I write all his Point of Sale software) has been, if I was being polite, "problematic"

And now his rather Citrix server is toasted ::)
Title: Re: Typical
Post by: TheBoy on 24 February 2009, 22:02:47
Phew  :D

Took longer than expected to come back due to a failed disk (seems to be having an adverse effect on performance, more than I'd expect), and it had to remove and install the new software as part of the GPO - and the failed disk made this take ages.

I'd given up on it, just but luck I retried it. Result.

So he'll bang in a new disk tomorrow, and pull the power on the lights-out board to reset it, and all good :D
Title: Re: Typical
Post by: cem_devecioglu on 24 February 2009, 22:07:18
very close..you are lucky.. :y
Title: Re: Typical
Post by: TheBoy on 24 February 2009, 22:13:01
Quote
very close..you are lucky.. :y
Yeah.  I was going to call him and tell him to get in early to kick it, but I'm not available in the morning as am busy doing a DNS infrastructure change out, which if it goes wrong, will affect 8m UK users...
Title: Re: Typical
Post by: Entwood on 24 February 2009, 22:14:25
Quote
Quote
very close..you are lucky.. :y
Yeah.  I was going to call him and tell him to get in early to kick it, but I'm not available in the morning as am busy doing a DNS infrastructure change out, which if it goes wrong, will affect 8m UK users...

Go on .. you know you want to ... really .......  :D :D :D :D :D
Title: Re: Typical
Post by: Gaffers on 24 February 2009, 22:18:01
Quote
Quote
Quote
very close..you are lucky.. :y
Yeah.  I was going to call him and tell him to get in early to kick it, but I'm not available in the morning as am busy doing a DNS infrastructure change out, which if it goes wrong, will affect 8m UK users...

Go on .. you know you want to ... really .......  :D :D :D :D :D


Seeings as it wont affect Germany.............  ::) ::) ::) ::) ::)
Title: Re: Typical
Post by: cem_devecioglu on 24 February 2009, 22:20:04
Quote
Quote
very close..you are lucky.. :y
Yeah.  I was going to call him and tell him to get in early to kick it, but I'm not available in the morning as am busy doing a DNS infrastructure change out, which if it goes wrong, will affect 8m UK users...

 :o :o

go sleep early.. really dont want to be in your place :P
Title: Re: Typical
Post by: TheBoy on 24 February 2009, 22:28:04
Quote
Quote
Quote
very close..you are lucky.. :y
Yeah.  I was going to call him and tell him to get in early to kick it, but I'm not available in the morning as am busy doing a DNS infrastructure change out, which if it goes wrong, will affect 8m UK users...

 :o :o

go sleep early.. really dont want to be in your place :P
maybe i shouldn't have cracked open another stella ::)
Title: Re: Typical
Post by: TheBoy on 24 February 2009, 22:29:09
Quote
Quote
Quote
Quote
very close..you are lucky.. :y
Yeah.  I was going to call him and tell him to get in early to kick it, but I'm not available in the morning as am busy doing a DNS infrastructure change out, which if it goes wrong, will affect 8m UK users...

Go on .. you know you want to ... really .......  :D :D :D :D :D


Seeings as it wont affect Germany.............  ::) ::) ::) ::) ::)
my affect you - not 100% sure of network layout and infrastructure on that part
Title: Re: Typical
Post by: stuart30 on 24 February 2009, 22:33:21
Quote
Quote
very close..you are lucky.. :y
Yeah.  I was going to call him and tell him to get in early to kick it, but I'm not available in the morning as am busy doing a DNS infrastructure change out, which if it goes wrong, will affect 8m UK users...

Sounds like you work for a similar company i look after.....i really dont see the problem if the data centre over heats....least it keeps me warm. ;D
Title: Re: Typical
Post by: Gaffers on 24 February 2009, 22:34:06
Quote
Quote
Quote
Quote
Quote
very close..you are lucky.. :y
Yeah.  I was going to call him and tell him to get in early to kick it, but I'm not available in the morning as am busy doing a DNS infrastructure change out, which if it goes wrong, will affect 8m UK users...

Go on .. you know you want to ... really .......  :D :D :D :D :D


Seeings as it wont affect Germany.............  ::) ::) ::) ::) ::)
my affect you - not 100% sure of network layout and infrastructure on that part

Pull a sickie then  ;D ;D ;D ;D ;D
Title: Re: Typical
Post by: amigov6 on 24 February 2009, 22:34:23
Err, yeah...ok. Hav'nt got a clue what you mean. Tell him a big boy did it & run away...then phone your legal team!!!! ::)
Title: Re: Typical
Post by: TheBoy on 24 February 2009, 22:38:24
Quote
Quote
Quote
very close..you are lucky.. :y
Yeah.  I was going to call him and tell him to get in early to kick it, but I'm not available in the morning as am busy doing a DNS infrastructure change out, which if it goes wrong, will affect 8m UK users...

Sounds like you work for a similar company i look after.....i really dont see the problem if the data centre over heats....least it keeps me warm. ;D
Very rare we have environmental issues at any of our UK datacentres. Don't have many dealings with non-UK ones so can't comment.
Title: Re: Typical
Post by: stuart30 on 24 February 2009, 22:45:17
Quote
Quote
Quote
Quote
very close..you are lucky.. :y
Yeah.  I was going to call him and tell him to get in early to kick it, but I'm not available in the morning as am busy doing a DNS infrastructure change out, which if it goes wrong, will affect 8m UK users...

Sounds like you work for a similar company i look after.....i really dont see the problem if the data centre over heats....least it keeps me warm. ;D
Very rare we have environmental issues at any of our UK datacentres. Don't have many dealings with non-UK ones so can't comment.

No we dont now days now its housed in special room...way beyond me. :)

Mind you was funny when the leccy board tried too disconnect mains power (before the generator was up and running).....think blood would have been spilt if they had. ;D ;D
Title: Re: Typical
Post by: JamesV6CDX on 24 February 2009, 22:53:01
Quote
Quote
Quote
Quote
very close..you are lucky.. :y
Yeah.  I was going to call him and tell him to get in early to kick it, but I'm not available in the morning as am busy doing a DNS infrastructure change out, which if it goes wrong, will affect 8m UK users...

Sounds like you work for a similar company i look after.....i really dont see the problem if the data centre over heats....least it keeps me warm. ;D
Very rare we have environmental issues at any of our UK datacentres. Don't have many dealings with non-UK ones so can't comment.

Although by comparison to some it's not a huge DataCentre - but I used to work for a business focused ISP in their NOC  / Comms, before I took my current job in the NHS.

I'll never forget I was alone on a nightshift, the power failed, and the generator never started. Turns out the changeover panel had fried.

I managed to get the Jenny manually fired up, when the UPS's were reading less than 1 minute before the DC lost power.

Close call  ;D ;D ;D
Title: Re: Typical
Post by: TheBoy on 24 February 2009, 22:53:34
Quote
Quote
Quote
Quote
Quote
very close..you are lucky.. :y
Yeah.  I was going to call him and tell him to get in early to kick it, but I'm not available in the morning as am busy doing a DNS infrastructure change out, which if it goes wrong, will affect 8m UK users...

Sounds like you work for a similar company i look after.....i really dont see the problem if the data centre over heats....least it keeps me warm. ;D
Very rare we have environmental issues at any of our UK datacentres. Don't have many dealings with non-UK ones so can't comment.

No we dont now days now its housed in special room...way beyond me. :)

Mind you was funny when the leccy board tried too disconnect mains power (before the generator was up and running).....think blood would have been spilt if they had. ;D ;D
The nearest datacenter to me - ie the one across the road ::) has enough battery power to run for about an hour on UPS (1000+ servers), and each of the 6 generators onsite can alledgedly power a small town.  Some critical stuff in there  :-X. Most of the other datacenters I don't get to too often, do most stuff remotely, from the comfort of my own home or from the office if I go in ::)

We lost one of our Internet DCs went offline a few years ago (a pdu went down when its partner was offline having scheduled maintenence).  Oh, there was hell to pay over than one - felt sorry for the sparkies, it wasn't their fault....
Title: Re: Typical
Post by: JamesV6CDX on 24 February 2009, 22:54:54
On the subject of environmental issues, the internal air handlers were awful, always breaking down. The external chillers were in an open car park and always switched off by local kids, for devilment!!

The ISP was too tight to build an enclosure around them.

How they ever get away with advertising that DC as N+1 is beyond me - it's most certainly not!
Title: Re: Typical
Post by: JamesV6CDX on 24 February 2009, 22:56:02
Quote
Quote
Quote
Quote
Quote
Quote
very close..you are lucky.. :y
Yeah.  I was going to call him and tell him to get in early to kick it, but I'm not available in the morning as am busy doing a DNS infrastructure change out, which if it goes wrong, will affect 8m UK users...

Sounds like you work for a similar company i look after.....i really dont see the problem if the data centre over heats....least it keeps me warm. ;D
Very rare we have environmental issues at any of our UK datacentres. Don't have many dealings with non-UK ones so can't comment.

No we dont now days now its housed in special room...way beyond me. :)

Mind you was funny when the leccy board tried too disconnect mains power (before the generator was up and running).....think blood would have been spilt if they had. ;D ;D
The nearest datacenter to me - ie the one across the road ::) has enough battery power to run for about an hour on UPS (1000+ servers), and each of the 6 generators onsite can alledgedly power a small town.  Some critical stuff in there  :-X. Most of the other datacenters I don't get to too often, do most stuff remotely, from the comfort of my own home or from the office if I go in ::)

We lost one of our Internet DCs went offline a few years ago (a pdu went down when its partner was offline having scheduled maintenence).  Oh, there was hell to pay over than one - felt sorry for the sparkies, it wasn't their fault....


Not Telecity Redbus / Sov House?
Title: Re: Typical
Post by: TheBoy on 24 February 2009, 22:57:04
Quote
Quote
Quote
Quote
Quote
very close..you are lucky.. :y
Yeah.  I was going to call him and tell him to get in early to kick it, but I'm not available in the morning as am busy doing a DNS infrastructure change out, which if it goes wrong, will affect 8m UK users...

Sounds like you work for a similar company i look after.....i really dont see the problem if the data centre over heats....least it keeps me warm. ;D
Very rare we have environmental issues at any of our UK datacentres. Don't have many dealings with non-UK ones so can't comment.

Although by comparison to some it's not a huge DataCentre - but I used to work for a business focused ISP in their NOC  / Comms, before I took my current job in the NHS.

I'll never forget I was alone on a nightshift, the power failed, and the generator never started. Turns out the changeover panel had fried.

I managed to get the Jenny manually fired up, when the UPS's were reading less than 1 minute before the DC lost power.

Close call  ;D ;D ;D
I remember when I worked for BT, there was some work being done on a concentrator (a vital part of the regional phone network), all stuff had been offloaded to another one. It was some power test.  Sparkies connected it up wrong way round and killed it, but also a load of other stuff.  Whole area out for several hours ;D
Title: Re: Typical
Post by: stuart30 on 24 February 2009, 22:57:17
Well i used too wonder why the IT geeks umm guys used too flap and panic so much when a server went down ot generator was being looked at until someone mentioned that 1 server alone generated something like 250k hourly....if it wasn't for the fact he was the data centre manager id have laughed.

Kinda too things a bit more serious after that. ;D
Title: Re: Typical
Post by: TheBoy on 24 February 2009, 22:58:01
Quote
Quote
Quote
Quote
Quote
Quote
Quote
very close..you are lucky.. :y
Yeah.  I was going to call him and tell him to get in early to kick it, but I'm not available in the morning as am busy doing a DNS infrastructure change out, which if it goes wrong, will affect 8m UK users...

Sounds like you work for a similar company i look after.....i really dont see the problem if the data centre over heats....least it keeps me warm. ;D
Very rare we have environmental issues at any of our UK datacentres. Don't have many dealings with non-UK ones so can't comment.

No we dont now days now its housed in special room...way beyond me. :)

Mind you was funny when the leccy board tried too disconnect mains power (before the generator was up and running).....think blood would have been spilt if they had. ;D ;D
The nearest datacenter to me - ie the one across the road ::) has enough battery power to run for about an hour on UPS (1000+ servers), and each of the 6 generators onsite can alledgedly power a small town.  Some critical stuff in there  :-X. Most of the other datacenters I don't get to too often, do most stuff remotely, from the comfort of my own home or from the office if I go in ::)

We lost one of our Internet DCs went offline a few years ago (a pdu went down when its partner was offline having scheduled maintenence).  Oh, there was hell to pay over than one - felt sorry for the sparkies, it wasn't their fault....


Not Telecity Redbus / Sov House?
Nope...
Title: Re: Typical
Post by: TheBoy on 24 February 2009, 23:00:15
Quote
Well i used too wonder why the IT geeks umm guys used too flap and panic so much when a server went down ot generator was being looked at until someone mentioned that 1 server alone generated something like 250k hourly....if it wasn't for the fact he was the data centre manager id have laughed.

Kinda too things a bit more serious after that. ;D
One of the platforms I used to look after - which many ISP workers here will be very familiar with - for a few hours every month used to generate £1m per minute apparently :o

No wonder the pager went ape when that platform fell over...
Title: Re: Typical
Post by: Kevin Wood on 24 February 2009, 23:03:34
Quote
I managed to get the Jenny manually fired up, when the UPS's were reading less than 1 minute before the DC lost power.

Close call  ;D ;D ;D

As someone once said - always carry a spare crank sensor... ;D

Kevin
Title: Re: Typical
Post by: stuart30 on 24 February 2009, 23:08:56
Quote
Quote
Well i used too wonder why the IT geeks umm guys used too flap and panic so much when a server went down ot generator was being looked at until someone mentioned that 1 server alone generated something like 250k hourly....if it wasn't for the fact he was the data centre manager id have laughed.

Kinda too things a bit more serious after that. ;D
One of the platforms I used to look after - which many ISP workers here will be very familiar with - for a few hours every month used to generate £1m per minute apparently :o

No wonder the pager went ape when that platform fell over...

Frightening really.... :)

If these servers (blades) what ever there called were stolen is the data accessible....only ask as these lot hold software for large financial institutions and billing software ect.

Not planning on robbing the place just curious.
Title: Re: Typical
Post by: TheBoy on 24 February 2009, 23:14:29
Quote
Quote
Quote
Well i used too wonder why the IT geeks umm guys used too flap and panic so much when a server went down ot generator was being looked at until someone mentioned that 1 server alone generated something like 250k hourly....if it wasn't for the fact he was the data centre manager id have laughed.

Kinda too things a bit more serious after that. ;D
One of the platforms I used to look after - which many ISP workers here will be very familiar with - for a few hours every month used to generate £1m per minute apparently :o

No wonder the pager went ape when that platform fell over...

Frightening really.... :)

If these servers (blades) what ever there called were stolen is the data accessible....only ask as these lot hold software for large financial institutions and billing software ect.

Not planning on robbing the place just curious.
Generally with blades, the storage will be SAN - and the 'LUN' is spread over many physical disks, so unless you lift the whole SAN (which will be several cabinets worth), you're safe.  Though you'd have to get past the physical site security.


The risk assessments show the biggest intentional threats for our sites are from terrorist incidents, so UK critical stuff is geographically resilient.
Title: Re: Typical
Post by: stuart30 on 24 February 2009, 23:52:37
Quote
Quote
Quote
Quote
Well i used too wonder why the IT geeks umm guys used too flap and panic so much when a server went down ot generator was being looked at until someone mentioned that 1 server alone generated something like 250k hourly....if it wasn't for the fact he was the data centre manager id have laughed.

Kinda too things a bit more serious after that. ;D
One of the platforms I used to look after - which many ISP workers here will be very familiar with - for a few hours every month used to generate £1m per minute apparently :o

No wonder the pager went ape when that platform fell over...

Frightening really.... :)

If these servers (blades) what ever there called were stolen is the data accessible....only ask as these lot hold software for large financial institutions and billing software ect.

Not planning on robbing the place just curious.
Generally with blades, the storage will be SAN - and the 'LUN' is spread over many physical disks, so unless you lift the whole SAN (which will be several cabinets worth), you're safe.  Though you'd have to get past the physical site security.


The risk assessments show the biggest intentional threats for our sites are from terrorist incidents, so UK critical stuff is geographically resilient.

Im assuming there called blades...about 6in deep and 3 foot square from memory and racks and racks of them.

Always wonderd why they went too the extent too physicaly protect them.....mind you the physical site security well we dont get paid enough too argue. ;D

Title: Re: Typical
Post by: TheBoy on 25 February 2009, 08:30:30
Quote
Quote
Quote
Quote
Quote
Well i used too wonder why the IT geeks umm guys used too flap and panic so much when a server went down ot generator was being looked at until someone mentioned that 1 server alone generated something like 250k hourly....if it wasn't for the fact he was the data centre manager id have laughed.

Kinda too things a bit more serious after that. ;D
One of the platforms I used to look after - which many ISP workers here will be very familiar with - for a few hours every month used to generate £1m per minute apparently :o

No wonder the pager went ape when that platform fell over...

Frightening really.... :)

If these servers (blades) what ever there called were stolen is the data accessible....only ask as these lot hold software for large financial institutions and billing software ect.

Not planning on robbing the place just curious.
Generally with blades, the storage will be SAN - and the 'LUN' is spread over many physical disks, so unless you lift the whole SAN (which will be several cabinets worth), you're safe.  Though you'd have to get past the physical site security.


The risk assessments show the biggest intentional threats for our sites are from terrorist incidents, so UK critical stuff is geographically resilient.

Im assuming there called blades...about 6in deep and 3 foot square from memory and racks and racks of them.

Always wonderd why they went too the extent too physicaly protect them.....mind you the physical site security well we dont get paid enough too argue. ;D

yup. thats a blade :y

our places the physical security is enough so the security guards can call for assistance before the scumbags can get in
Title: Re: Typical
Post by: TheBoy on 25 February 2009, 08:32:46
Quote
Quote
Quote
Quote
very close..you are lucky.. :y
Yeah.  I was going to call him and tell him to get in early to kick it, but I'm not available in the morning as am busy doing a DNS infrastructure change out, which if it goes wrong, will affect 8m UK users...

 :o :o

go sleep early.. really dont want to be in your place :P
maybe i shouldn't have cracked open another stella ::)
On reflection, I might have overdone it with that last stella ;D