I'll post later when I get a chance and have counted how many more grey hairs I've accrued over the last 24 hours!
we all know you do a great job so it is not necessary to apologise for the occasional hiccup.
I hope you are an eccentric millionaire who does not have to keep a job as well
Aargh, I typed a reply then my phone battery went and I lost the lot, so here goes again!
Yes, if only I was an eccentric millionaire, mojo! I'd snap up that £20K PVL for us without a flinch, and be able to pay a top web design company to run this website! Maybe one day my premium bonds will come up!
Anyway, back to yesterday. Everything was working fine until about 11 or so yesterday morning, then the error appeared. I hadn't done anything to the site's infrastructure, but I wondered if the increasing size of the database necessitated an extension of php memory allocation in one of the configuration files, but no joy there. Then I noticed that other sites on the server suffered from the identical issue, so it seemed to be something related to the server as opposed to anything here. I suspected maybe there was a server software upgrade that precipitated an incompatibility issue, and got in touch with the hosting company.
I spent 30 minutes waiting in a queue for an advisor on the phone (thankfully it was an 0800 number!) and they raised a ticket and hoped the matter would be resolved within 24 hours, so the matter was outwith my control, and it was a matter of waiting. But thankfully it has been resolved without too much delay!
As to what went wrong, I am still none the wiser. At least it didn't result from us, and judging by this email I just got from the hosting company, it looks like it may have been some sort of software error:
"...our system resolved the issue at 11:40 am today, they ran a fix on the webserver which has fixed the issue, they haven't given me any more information than that they have fixed it."
Rest assured that issues like these are 'merely' errors affecting access to, and display of, content in the database: there is no threat to the integrity of the site's data and files. I take regular back ups of the database (which contains the site's content) and if ever there was a catastrophic failure and the data on the server was somehow entirely wiped or otherwise lost, the website could readily be restored by reinstalling the site software and reuploading the database and sundry graphics and images (albeit a time consuming task).
But hopefully all is well now, and it'll not happen again!
Lovely to see the site back up, Ive really missed it being there, thanks for sorting out the problems, this is my all time favourite website.
Glad normal service is resumed! It was a sleepless night last night wondering how long it would take to sort things out!
I was meaning to do an email to update everyone, but I was out last night and didn't get a chance.
We still have space on the panspeople.net website, so maybe in future I can paste a notice there by way of updating everyone if there are problems here.
I keep forgetting about our Facebook and Twitter accounts!
Make sure you bookmark those, so in the event of any spanners in the works here, I can post updates on the Facebook page.
I don't think you have to be a member to see the Facebook page, unlike with personal pages there.
If you haven't been there for a while, please do check it out, as Peter's been doing a marvellous job sprucing it up!
A great relief to see the website active again. I was very worried yesterday.
Good to see the site up and running again and thanks for the explanation PanFan. You don't realise how much you value something until it's gone I suppose. Sorry to hear you're not independently wealthy though!
PP.com's on Facebook -come take a look. Ha,ha sounds like a good advertising jingle?.Pleased all is up & running again.Thanks Panfan for all your hard work. Servers & Spammers - a threat to the well being of all websites.
Hi everyone - apologies the loss of service over the last couple of days. It appears to have been the same issue that affected the site in February, and thankfully the hosting company was able to resolve it without too much delay.
However, one outtage was enough, but for the same thing to happen in less than two months makes me think there is some systemic issue with the management of the server, and I have asked them to look into the matter, so prevent this happening a third time. We have never had a problem since the migration from PP.net to PP.com, until the beginning of February. I sincerely hope they'll prevent any further problems, but I told them if there are I may have to look into using another company. They get a lot of business from me, so I hope they get the message.
When I got a chance, I posted a message on our Facebook and Twitter pages, so please do bookmark those now so that if this happens again, so can get a status report. But with this sort of thing it's a matter of reporting it to technical support and waiting until they can sort it.
As I say, I'll keep at them to find out what the problem is and what steps they'll be taking to prevent it happening again.
Just to reiterate what I said after the last outtage, don't worry about the integrity of the site, as this was a PHP memory issue and had no impact on the database or other site files.
Apologies to all again, and let's pick up where we left off!
Good to know it's all backed up.
I meant to say, I noticed both times this happened on a Thursday morning. I wonder that they do then that affects the servers?!
I hope it's not a case of fingers crossed each Thursday morning in case things go pear shaped again!
It's so good to see you again, maybe it's time to really get motoring with those pending dancer interviews, answering questions posed by FMers here.
I don't know where Panfan's based in this world but it was about 05:10 yesterday morning when I tried visiting this website & seeing this "out of memory" error for the first time & not being able to access any of the normal contents of pp.com!
I posted a thread in OftD informing of this website going off-line, but anyways they had brief outages yesterday as well.
I guess that stuff happens!
I didn't mind the site being off for a day. That left me with some time to do other less important things!
At least being a recurrence of what happened before it wasn't quite as nerve wracking, but still stressful enough! I am in Scotland, PBP, but it was already out by the time I logged on.
Apologies for the delay in the interview schedule. Unfortunately due to my aunt's illness I've fallen a bit behind schedule due to time limitations, so all I can do is ask you to bear with me as the next few months will be rather difficult time wise. I'm hoping to have Lulu's up in about a fortnight or so. With the amount of things that needs to be dealt with on a daily basis, I can easily spend 6-9 hours a day on the site and still struggle to catch up! So although it may take a while, we will get there!
I got an email from the tech support guy with the hosting company. He said: 'they did come back to me saying that there was a rogue database query which was using all available memory on the mysql server which they found, killed off the process and restarted the server which resolved the issue'.
I trying to get them to see if there's a way of sandboxing this sort of thing, as we don't want it happening again.
The previous day to the outtage when I hit one of the topic links in the right hand column I got the out of memory error message. I thought oh no its happening again, but when I clicked on my link to pp.com in my favourites, it took back into the functioning website again.
I can't now remember which link it was that caused it, but if the rogue database query is generated by a particular link, it may be possible to locate the source of the problem and understand more about how it is generated.
This piece of information may not help in this case because of my failing memory. But maybe if the users keep an eye out and note down the link if a particular link produces a tempory glitch like I experienced. A quick message to Panfan might allow the problem to be rectified before it produces a complete outage.
Only a thought, what do you think Panfan does this sound like it might be useful?
Thanks Simon, yes I wanted to find out where the query originated, and asked the tech guy to see if it originated here. I don't think so, as a number of sites went out (we're on a shared server), but hopefully we'll find out.
Good thinking, and might be worth doing. Can't say I'd ever noticed such a problem before the outage, and I'm on here, um, a lot!
I should check my site error log file to look for evidence too - funny I forgot about that!
Looks like the rogue query didn't have anything to do with us, according to this message I got this morning: 'I don't think you need to look at anything you have done from your side, as the system team haven't mentioned it. It looked like there was an issue on the server itself as it affected all our customers.'
I was just wondering, websites that break down like this one did, in many cases do they just say that they are "down for maintainance" just to save face!
It often happen's with Digitalspy, & only the homepage can be accessed!
Just to add, that I've got a theory about this.
Thanks to the TOTP re-runs on BBC Four, many more people are (re)discovering just how good Legs & Co. et al were, including, I daresay a whole new generation of fans like me, who don't really remember them from the original TOTP broadcasts.
We looked for stuff on the web about the TOTP dancers, & found pp.com & OFtD, consequently these sites are getting more hits, & so the servers are being put under more strain than ever.
It was bound to cause outages, you obviously need better servers that can handle more traffic!
Actually that's one for the pp.com improvements thread!
Thanks for your thoughts, PBP.
It would be great to have an Apologies for Temporary Loss of Service message or some such, but sadly when these errors happened I was as stuck as you guys, and couldn't get past the PHP memory error. All I could do was sit and wait for the tech support to restart the server software.
As far as I can see, there would be two options to get such a message up. The error results affects a dynamically generated content site using PHP, but wouldn't affect a plain HTML page, so I could temporally disable the index.PHP page and replace it with an index.htm page with apology, but that requires manually changing it via FTP, and has the disadvantage of us then not knowing when normal service was resumed, as I'd have to keep deleting the index.htm file periodically from the server to see if the index.PHP worked! So the problem may have been resolved but no one would be able to access the site because I hadn't managed to delete that file.
The other option is to replace the site logo with a loss of service graphic, which could be removed when things were back to normal, but maybe again not so practical.
It's a shame I can't do more: I guess when the other site you mention goes pear shaped, it's a site internal error that doesn't affect the administrator's ability to manage the site: with us the problem was on the server, effectively disabling my ability to do anything.
It would be great to think it's traffic that caused it, but we'd have to get tens of thousands of hits a day for any impact. It looked like a single database query that went awry, using up all the memory allocation. Depending on how the site grows (if we get too popular, lol), maybe one day there might be a need to invest in a dedicated server, but that involves a fair bit of extra dosh, of course!
But thanks for your comments: I'll try to see that the company can do to prevent a recurrence, as that would be best!
It's One For The Dads having real problems tonight, Ryan also on here tonight has already mentioned it over there, I couldn't get any OftDs pages to re-load just now!
Just been backwards and forwards between the two sites for the last 20 minutes and have not noticed any problems. Maybe they have fixed it now.
Every now and again on there, mostly at weekends, Nabble ( who provide the forum ) do maintence work and this slows things up some what. Its a bit of a drag i know but usually sorts it self out. At the moment its fine. Just hope its okay for my weekly quiz tommorrow.
Once again, welcome back on-line!
It took a little more time than usual, but it was nice to see our friends at OFTD were so concerned about it
What a reiief! Will be home in an hour or so and fill you in then!
Are we sorted?! Is this really it?!
No sooner had I posted on Saturday than we were out again. I think they rebooted the server, then it went off again after 45 mins. Besides emailing I was on the phone to them first thing this morning.
My apologies for this loss of service. After many trouble free years using this hosting company, I'm seriously losing faith in them and wondering what I'm paying them for. I have asked them to investigate what is happening and to ensure there is no recurrence (not holding my breath).
Being off for a day is one thing, but over three and a half is taking the mickey. I alerted them by email as soon as we went offline on Thursday (19:30), as off course the support lines were shut. I got an acknowledgement finally at Friday lunchtime, then the first reboot wasn't till Saturday afternoon. It only gives the impression of a stretched skeletal staff, struggling to cope.
I will write them a letter of complaint about all this, and how they're at risk of losing a hitherto loyal customer, and that they'll be leaving me with no option but to take my custom elsewhere. (in practical terms that would be a total pain, but I might be left with no choice...)
Anyway, service resumed, I hope! Let's hope we can get through next Thursday without the server going off again!
I probably jinxed it all when I posted a 'welcome back' message, when you were briefly back on-air as it were, see earlier in theis thread.
I do hope you're all sorted now, you've been a hot topic of conversation over @ OftDs, I also hope that if you're forced to use an alternative server, that the consequential service interruption is as brief as possible!
Good to see the website up and running again PanFan and thanks again for all your efforts. I realise now how much I miss cruising around on here as I must have seen that error page about a hundred times over the weekend. Let's hope this is now behind us but these issues beyond your control must be a total headache for you.
Great to have the website back, it feels like its been away for months.
Hope you give the Hosting compant a piece of your mind.
Welcome back online to Panfan and all at PP.com. Three and a half days is a long time to be off, i certainly wont be moaning about 15 minute nabble crashes on OFTDs no more. Its been great to talk to a few of the guys here who have popped over the road to see us these last few days.
So can i say thanks to Simon D, Scrooge McDuck and Flight 14 for becoming members there also. I hope my site and Panspeople.com both prosper and continue in our quest of promoting and appreciating our fabulous dancers and the dances they performed.
Thanks everyone for your wishes: it's great to know we were missed!
I am so pleased to see normal service has been resumed. I suppose that hosting companies cannot completely guard against system crashes. But as this has happened more than once is sounds like they should have procedurers in place that will indentify when a problem has occured so they can implement a reboot as quickly as possible. They should be ringing you Panfan to apologize for the brief loss of service and to tell you that the system has now been rebooted and normal service will resume at a specified time. The fact that you have to call/email them suggests that there system has no integrity checks built in.
I would also like to thank Ryan for being a wonderful host at OftDs while PP.com was off line.
Panfan, you might like to use their forum to vent your frustration while things are not working. There is a dedicated thread that PBP started that would be ideal for the purpose. I am sure that Ryan/YMG would welcome this too as there was quite a bit of concern and well wishing going on over there.
Of course a warm welcome back from me as well! I really hope you can solve this problem permanently soon.
As others have said already, we did miss you but at the same time enjoyed a good time at OFTD, and it is not just my hope but my firm belief that our two sites will get closer together.
I'll have to put some backup measures in place. I can't send out an update message via this site software (as far as I know), and I can't send more that 200 emails a day without the address being automatically suspended as a potential source of spam! So I'll have to fire up the old forum software to use for notifications and as a way of touching base.
I am a little confused at your last post Panfan. As you talk about sending out more than 200 emailos, I assume that you are talking about a back up plan for communicatIng with members when pp.com is offline. I thought you has Facebook and Twitter as the back-up for this. Most of us know to have a look there and it is the first place I head if things have gone pear shaped, followed by OftDs.
However if you are concerned that not everybody knows about the Facebook page, could it be possibe to include in the front page that is still visible when a crash has occured a link to the Facebook page? Besides the box with the out of memory error message in I can see the PP.com logo. Maybe it could also have in case of problems visit: http://www.facebook.com/pages/Pans-People/207746622599781 for updates.
I can't do anything to the front page when this happens. All I could do is remove the index.php and replace it with a static index.htm page, which I could tailor. The only problem is that then I wouldn't know when it was sorted! (besides the fact that this has to be done manually with Dreamweaver on my desktop computer, so if I was away from home, it could be hours until changed back.)
Thanks for your suggestions though!
The pp.net site is still functional and maybe I would post on it. I just wondered about Facebook as I don't know how many here look there (you don't need to be on FB to view it).
As far as the 200 emails goes, I had the idea of sending out newsletters but I found out that I'm only allowed to send our 200 a day, and as we have about 500 members now, I'd have to stagger them, which is about of a pain. I think I'll register the active members at least using the old forum software, which allows a message to be sent to all members from within that software, thus bypassing the 200 a day problem.
Good to see you panfan and everyone else on pp.com back online
I was not suggesting that you do anything to the front page when it happens. I was suggesting that the message is there on a permanent basis. The logo is displayed when a server crash has happened, so if you were to locate the message in the same template that is being read and not effected hopefully people would see this message when a crash occurs.
It would probably take some playing around to find a position that is fairly inobtrusive and does not wreck your design in normal use. But when the content box shrinks down to a small error box sudenly leaps out an hits you between the eyeballs.
I don't know if it will be possible in practice, but it may be worth a quick test.
The logo is about the only thing that displays before the memory over run. Not even the full template loads. All I could do is replace the logo with someetjibg else, but again this is an FTP job from the desktop. Not sure if I could replace it with anything practical.
I'm going to hassle the company to try to get its act together so I hope we don't have a repeat!
I wonder Panfan-could the exceptionally long delay in rebooting be due to that fact that this occured over a weekend?.I could imagine the webserver staff knock off early on Fridays & drag themselves back into work 9am on Monday mornings?.I suppose in a way we all do it really.But they should have sufficient cover but of course we know that even the cover is usually thin on the ground & just earning overtime & not fully trained etc.
Anyway Panfan thankyou for all the efforts you made & it's wonderful to be back.
Thanks to Ryan because we were able to discuss the Pan's People.Com closedown on OFTDs.I must say that really was very much OFTD's to the rescue & many of us PPDC members - Simon - Rusty - PbP - Scrooge - Suefan -Andy -Glenn -Doc & others 'reconvened' our meetings over at OFTDs.
Panfan - I have been 10 years on another website & it's crashed just as much as on here.Down for 2 or 3 days at a time.So these things do happen.But you are paying for a professional back up service & you are so right to make your views known to them that the reboot took far too long & this is not good enough.
Anyway It feels grrreeeeaaaaaat to be back on here......
Hi Peter, yes likely the weekend played a part. The technical cover would likely be reduced, but I do know they have some cover. When we changed from pp.net to pp.com last April, there was a configuration error that prevented the site showing, and I was surprised to get an email on the Sunday which sorted it out.
Thanks also to our friends across the road for their concern. I must say not a few people have suggested to me that I reciprocate the extension of friendliness by visiting OFTD and inviting their members to join us too, as a matter of fairness, lol!
I've still to get a chance to write a letter to the hosting company- I figured writing rather than emailing would be more formal.
The three outages that occurred before all occurred on a Thursday , so fingers crossed we can make it to midnight without the lights going out!
Hi all. I've been having a struggling trying to update the site software, but got it sorted now. Apologies for being off so long. I tried to send a message to all members via the site, but it seems to have overtaxed the memory and I don't know hue many got out so if you didn't hear I apologise.
At least that intermittent problem we had last spring seems to have been resolved though!
Great to see your forum back where it belongs. :)
I, we've all missed you and your forum since its absence.
Good to see the site back up and running Panfan, it was missed and a thank you from me for the work you and the other administrators put in.
I'll second that!
I did read the circulated message, thanks for that, I thought that you were gone for ever, judging by the message on the web host's holding page
'This site has been suspended'
The screen that had appeared during previous outages would've sufficed, at least then we'd have had a better idea that you'd be back, but hopefully you're here to stay now.
Just to add my relief at seeing the site back in action and your message PanFan
There are currently 2 users online.
Get Started on PansPeople.com
All Recent Content
Today's Popular Content
To Do List