Omega Owners Forum
Chat Area => General Discussion Area => Topic started by: MR MISTER on 25 November 2013, 15:57:35
-
...114 seconds. Fickin asthmatic server. ;D
-
And..why is my username showing twice ???
-
Page created in 25.668 seconds with 12 queries.
I guess it's a bit poorly then :(
-
Page created in 0.045 seconds with 18 queries.
Guess it's woken up now ;D
-
Yep same here - was like watching paint dry, other sites/forums working at normal speed - but seems ok now :y
-
Yeah, that ongoing problem reared its head around 2:30 and 4pm this afternoon, as before, no clues as to why.
I hate spending money on the offchance it will fix it :'(
-
Yeah, that ongoing problem reared its head around 2:30 and 4pm this afternoon, as before, no clues as to why.
I hate spending money on the offchance it will fix it :'(
You have caught that from your mate tunnie :y
-
Yeah, that ongoing problem reared its head around 2:30 and 4pm this afternoon, as before, no clues as to why.
I hate spending money on the offchance it will fix it :'(
You have caught that from your mate tunnie :y
Trouble is, its quite large sums (£100s) on an offchance it may resolve it. Or about £1500 that I know will resolve it.
-
Yeah, that ongoing problem reared its head around 2:30 and 4pm this afternoon, as before, no clues as to why.
I hate spending money on the offchance it will fix it :'(
No clues at all?
-
Yeah, that ongoing problem reared its head around 2:30 and 4pm this afternoon, as before, no clues as to why.
I hate spending money on the offchance it will fix it :'(
No clues at all?
Nope. I was watching it do it on Saturday. Everthing running smoothly, then all of a sudden, disk latency jumping up to over 100ms for ages. Disk throughput at the time was peaking at approx 25MBps, which is fairly low.
Array diags found no hardware errors with controller, or no preemtive failure signs on the disks.
No clues in logs of hypervisor or VMs, only that processes are aborting due to disks not responding.
-
Should add, at 4pm today, latency jumped to 200-225ms but fell to normal sub 10ms levels after about 3 mins, which was a short enough time for everything to recover.
-
I have no experience of hardware raid or VMs therefore I am probably way off the mark but
1. does the VM have a way of capping the disc transfer rate of each virtual machine?
2. could you be the target of a DNS attack ?
-
I have no experience of hardware raid or VMs therefore I am probably way off the mark but
1. does the VM have a way of capping the disc transfer rate of each virtual machine?
2. could you be the target of a DNS attack ?
1) Not capping, but prioritising. All VMs have same priority to the disks, thus one VM shouldn't easily be able to bring down others.
2) The only DNS available to the Internet is an authoritive one, not caching one, thus doesn't allow recursive lookups, thus DNS/Reflection type attacks are less successful. Said DNS server is not seeing increased traffic, and easily fits all zones in RAM, so not touching the disks much. DNS/Reflection attacks are more to swallow bandwidth rather than server resources.
Been looking for eveidence of any attacks, not found anything so far...
-
I have no experience of hardware raid or VMs therefore I am probably way off the mark but
1. does the VM have a way of capping the disc transfer rate of each virtual machine?
2. could you be the target of a DNS attack ?
Oops, been a long crappy weekend visiting the sick :(
What I meant to ask was: could you be the target of a DoS attack?........... (which you answered anyway :y )
-
I have no experience of hardware raid or VMs therefore I am probably way off the mark but
1. does the VM have a way of capping the disc transfer rate of each virtual machine?
2. could you be the target of a DNS attack ?
1) Not capping, but prioritising. All VMs have same priority to the disks, thus one VM shouldn't easily be able to bring down others.
2) The only DNS available to the Internet is an authoritive one, not caching one, thus doesn't allow recursive lookups, thus DNS/Reflection type attacks are less successful. Said DNS server is not seeing increased traffic, and easily fits all zones in RAM, so not touching the disks much. DNS/Reflection attacks are more to swallow bandwidth rather than server resources.
Been looking for eveidence of any attacks, not found anything so far...
I wonder if there is some 'undocumented feature' that trys to be helpful (when you aren't expecting it)
-
How many VMs are you running on that processor/server/disc array?
Is there some housekeeping task that kicks in occasionally?
-
How many VMs are you running on that processor/server/disc array?
"Several"
Is there some housekeeping task that kicks in occasionally?
As said, disk throughput is not going up when it happens, its just disk speed seems to drop dramatically (ie, latency goes up, ie, it takes longer to process a disk request).
CPU utilisation shows nothing odd either before or during the glitches, though difficult to tell for sure, as by its nature with "several" VMs, its very spikey. Certainly at no point are any of the Xeon's cores looking stressed.
-
Yeah, that ongoing problem reared its head around 2:30 and 4pm this afternoon, as before, no clues as to why.
I hate spending money on the offchance it will fix it :'(
You have caught that from your mate tunnie :y
Trouble is, its quite large sums (£100s) on an offchance it may resolve it. Or about £1500 that I know will resolve it.
Get everyone here to chip in,why should you have to shoulder it. ???
-
Is anything happening to power supplies .. both external (mains voltage drop??) or internal (12v rails ??) as I've had problems with discs in the past overreacting to to fluctuating power by changing speed ... lets face it, other than fans there'll be nowt else physically moving in your set up, so they'll probably be using most, and be most sensitive to, power ??
Just a thought .. nowt else .. :(
-
Yeah, that ongoing problem reared its head around 2:30 and 4pm this afternoon, as before, no clues as to why.
I hate spending money on the offchance it will fix it :'(
You have caught that from your mate tunnie :y
Trouble is, its quite large sums (£100s) on an offchance it may resolve it. Or about £1500 that I know will resolve it.
Get everyone here to chip in,why should you have to shoulder it. ???
Because its my hardware, not OOF's property.
-
Is anything happening to power supplies .. both external (mains voltage drop??) or internal (12v rails ??) as I've had problems with discs in the past overreacting to to fluctuating power by changing speed ... lets face it, other than fans there'll be nowt else physically moving in your set up, so they'll probably be using most, and be most sensitive to, power ??
Just a thought .. nowt else .. :(
I'd say the electronics would be more sensitive :-\. However, being server class hardware, the server's internal environment is constantly monitored by the management processor, and nothing has been logged except when I stuck a pen in one of the fans to test the monitoring a few months ago ;D
Server's power is protected by a UPS, which will deal with (and log) brownouts and surges. Brakkers power is pretty clean TBH.
-
Is OOF the only VM displaying this behaviour? Is OOF the only VM using simple machines ?
Do the 'several' VMs run the same operating system? or have you got a mix?
-
Is OOF the only VM displaying this behaviour?
No
Is OOF the only VM using simple machines ?
All 3 OOF VMs run SMF, nothing else does.
Do the 'several' VMs run the same operating system? or have you got a mix?
Windows, Solaris and Linux. Linux seem worse affected, but thats expected (as Linux is shite). Linux VMs are Debian and CentOS, so from seperate base distros (Debian from, well, Debian, CentOS is a Redhat derivitive).
-
Is OOF the only VM displaying this behaviour?
No
Is OOF the only VM using simple machines ?
All 3 OOF VMs run SMF, nothing else does.
Do the 'several' VMs run the same operating system? or have you got a mix?
Windows, Solaris and Linux. Linux seem worse affected, but thats expected (as Linux is shite). Linux VMs are Debian and CentOS, so from seperate base distros (Debian from, well, Debian, CentOS is a Redhat derivitive).
;D ;D ;D Yer what?
-
Is OOF the only VM displaying this behaviour?
No
Is OOF the only VM using simple machines ?
All 3 OOF VMs run SMF, nothing else does.
Do the 'several' VMs run the same operating system? or have you got a mix?
Windows, Solaris and Linux. Linux seem worse affected, but thats expected (as Linux is shite). Linux VMs are Debian and CentOS, so from seperate base distros (Debian from, well, Debian, CentOS is a Redhat derivitive).
;D ;D ;D Yer what?
When the Jargon comes out I just sit back and watch in wonder, I have absolutely no idea what they are saying........ :D :D
-
;D ;D ;D Yer what?
In laymans terms, its fooked
-
;D ;D ;D Yer what?
In laymans terms, its fooked
We understand that...... :D :D :y
-
When the Jargon comes out I just sit back and watch in wonder, I have absolutely no idea what they are saying........ :D :D
What jargon?
OOF = well....
VM = Virtaul Machine
SMF = The software that runs the site - Simple Machines Forum
-
When it runs OK it's really fast. Just now and again....
It's like being called out to the dreaded intermittent fault.
-
When the Jargon comes out I just sit back and watch in wonder, I have absolutely no idea what they are saying........ :D :D
What jargon?
OOF = well....
VM = Virtaul Machine
SMF = The software that runs the site - Simple Machines Forum
I knew that one!....... ;) ;)
-
Whoever invented Virtualisation (well, VMWare, Parallels, etc etc) should be hung, drawn and quartered.
JMHO ;D
-
As said, disk throughput is not going up when it happens, its just disk speed seems to drop dramatically (ie, latency goes up, ie, it takes longer to process a disk request).
CPU utilisation shows nothing odd either before or during the glitches, though difficult to tell for sure, as by its nature with "several" VMs, its very spikey. Certainly at no point are any of the Xeon's cores looking stressed.
Sounds to me like the RAID controller's having to stop and scratch its' head every so often. Does it report any stats about what it's doing?
I wonder if it's getting errors on one of the devices and retrying? I had a PERC3 that would stop and think every so often. I had a couple of corrupted files, then eventually found out that every time I MD5'ed a partition on it, I got a different answer yet it reported nothing unusual! That went in the bin with immediate effect.
-
Yeah, that ongoing problem reared its head around 2:30 and 4pm this afternoon, as before, no clues as to why.
I hate spending money on the offchance it will fix it :'(
You have caught that from your mate tunnie :y
Trouble is, its quite large sums (£100s) on an offchance it may resolve it. Or about £1500 that I know will resolve it.
Let's say that there are a hundred regular contributors to the forum, and whether we admit it or not, would all be devastated if it rolled over...
:-\
I know that it's nearly Christmas, blahblahblah, but would it really kill us to chip in £15 each to get this sorted :-\
-
Yeah, that ongoing problem reared its head around 2:30 and 4pm this afternoon, as before, no clues as to why.
I hate spending money on the offchance it will fix it :'(
You have caught that from your mate tunnie :y
Trouble is, its quite large sums (£100s) on an offchance it may resolve it. Or about £1500 that I know will resolve it.
Let's say that there are a hundred regular contributors to the forum, and whether we admit it or not, would all be devastated if it rolled over...
:-\
I know that it's nearly Christmas, blahblahblah, but would it really kill us to chip in £15 each to get this sorted :-\
Great idea, but it's been put out numerous times before .. but TB is just simply against it and will not change his mind. He has his reasons which we must respect, even though we might disagree with them.
-
Yeah, that ongoing problem reared its head around 2:30 and 4pm this afternoon, as before, no clues as to why.
I hate spending money on the offchance it will fix it :'(
You have caught that from your mate tunnie :y
Trouble is, its quite large sums (£100s) on an offchance it may resolve it. Or about £1500 that I know will resolve it.
Let's say that there are a hundred regular contributors to the forum, and whether we admit it or not, would all be devastated if it rolled over...
:-\
I know that it's nearly Christmas, blahblahblah, but would it really kill us to chip in £15 each to get this sorted :-\
Great idea, but it's been put out numerous times before .. but TB is just simply against it and will not change his mind. He has his reasons which we must respect, even though we might disagree with them.
As I indicated in another thread, only way I could see this work is if no one was given recognition for donations, it has to be totally anonymous.
-
As said, disk throughput is not going up when it happens, its just disk speed seems to drop dramatically (ie, latency goes up, ie, it takes longer to process a disk request).
CPU utilisation shows nothing odd either before or during the glitches, though difficult to tell for sure, as by its nature with "several" VMs, its very spikey. Certainly at no point are any of the Xeon's cores looking stressed.
Sounds to me like the RAID controller's having to stop and scratch its' head every so often. Does it report any stats about what it's doing?
I wonder if it's getting errors on one of the devices and retrying? I had a PERC3 that would stop and think every so often. I had a couple of corrupted files, then eventually found out that every time I MD5'ed a partition on it, I got a different answer yet it reported nothing unusual! That went in the bin with immediate effect.
I did some offline Array Diagnostics (or whatever wankword HP have come up with now, I think its now branded Smart Storage Administrator, no wonder I had to do a double take to pick the right option), no reported errors on the array card, and array card was reporting the underlying disks as healthy, with all the SMART parameters well within spec.
The HP Smart Array controllers are pretty robust, although I have seen a problem where it refused to rebuild a disk, which happened on the previous Proliant we had, as you may recall. Blowing the array away and rebuilding from scratch was the solution provided by HP, which worked, along with a firmware upgrade to prevent it happening again.
Given the cost, it would be a leap of faith to replace the card based on diags so far.
-
Whoever invented Virtualisation (well, VMWare, Parallels, etc etc) should be hung, drawn and quartered.
JMHO ;D
It has its advantages... ...if you don't overallocate ::)
-
Yeah, that ongoing problem reared its head around 2:30 and 4pm this afternoon, as before, no clues as to why.
I hate spending money on the offchance it will fix it :'(
You have caught that from your mate tunnie :y
Trouble is, its quite large sums (£100s) on an offchance it may resolve it. Or about £1500 that I know will resolve it.
Let's say that there are a hundred regular contributors to the forum, and whether we admit it or not, would all be devastated if it rolled over...
:-\
I know that it's nearly Christmas, blahblahblah, but would it really kill us to chip in £15 each to get this sorted :-\
Not viable, not fair, thus not gonna happen. End of.
-
Yeah, that ongoing problem reared its head around 2:30 and 4pm this afternoon, as before, no clues as to why.
I hate spending money on the offchance it will fix it :'(
You have caught that from your mate tunnie :y
Trouble is, its quite large sums (£100s) on an offchance it may resolve it. Or about £1500 that I know will resolve it.
Let's say that there are a hundred regular contributors to the forum, and whether we admit it or not, would all be devastated if it rolled over...
:-\
I know that it's nearly Christmas, blahblahblah, but would it really kill us to chip in £15 each to get this sorted :-\
Not viable, not fair, thus not gonna happen. End of.
An impression of obligatory contribution (or membership/subscription by any other name) was/is neither my point nor intent :-\
Rather a voluntary and annonymous contribution to sorting any hardware issues that you might be facing :y I understand your your devotion to the principles behind the Forum, respect those principles and actually agree with them. The poll that I instigated should, hopefully, indicate a willingness to bear some of the burden by the guests that we all ultimately are...
The poll has, touch wood, another 6.5 days to run and should be seen as a sign of encouragement.
-
42 offers can't be wrong.
But maybe TB feels he may not want to host the site forever. Might look funny if he gives up ownership. If that's the right word.
There's probably as many reasons for as against, I suppose.
Anyway, hopefully he'll take some comfort from knowing the offers there. ;)