Tag Archives: search

Report on search index outage

Hi — as promised, I’m passing along the details from our search index provider about the outage we had this morning.
They said they were having issues with memory leakage, which necessitated a full cluster restart. They are investigating what caused that, and in the meantime have added lots of physical memory to the cluster to prevent another occurrence of the outage.
I’ll let you know when we hear anything else.
Thanks,
Doug.

Advertisements
Tagged , ,

Search index service is back up

Hi — it looks like the search index service is back up.
We’ll let you know what our provider says as soon as they get back to us — I imagine they’re pretty busy making sure things are all in order at the moment, but we’ll keep you posted as soon as we hear from them.
In the meantime, we’re sorry once again for the outage and the inconvenience it caused.
Thanks,
Doug.

Tagged , ,

Search index service down

Hello — our search index service has fallen over, which means that large parts of the site are currently broken. We’ve contacted our search index service provider, and they will be scurrying to fix this as quickly as they can.
In the meantime, we’re looking into constructing an alternative index of our own, in case it takes them a while.
Hopefully our alternative won’t be needed, or even have the time to be completed, because our provider will have resolved things quickly.
We’re really sorry about this, and will keep you posted as to progress, here.
Thanks,
Doug.

Tagged , ,

Folksy offline briefly at 11:36

Hi — the site went down for around 3-5 minutes at 11:36. This was due to our search service falling over. We’ve raised urgent tickets with our search service provider to establish what exactly happened, but in the meantime we’re back up, and everything looks good.

We’re sorry for any inconvenience,

   Doug.

Tagged , , ,

Maintenance mode for around 15 minutes, 07:30 today

Hi — we’re making a couple of small changes to the search index today, that unfortunately require completely rebuilding it.

This will mean that the site needs to go into maintenance mode, possibly for around 15 minutes.

We’ll be looking to do this work at 07:15.

Sorry for any inconvenience this causes, and thanks for bearing with us.

   Doug.

Tagged , , ,

Search Problems

We’re currently experiencing some problems with the search service which means search isn’t currently returning any results. We’re looking into the problem at the moment.

UPDATE : The search index has been rebuilt and the problem is now fixed.

Tagged

Search outage

We just experienced a problem with the search component which lead to other issues on the site such as an inability to list or edit items on the site.  The service has been restarted and service should be back to normal.  The problem persisted for approximately 15 minutes.

Tagged ,

Search Failure

Our search component just failed, meaning users may have experienced problems searching and taking other actions on the site.  We’ve added further monitoring and automated restart processes to minimise disruption if this happens again in the future.

Tagged ,

Search reboot

We just rebooted the search service on Folksy, customers will have experienced approx 2 minutes outage whilst this was in progress.

Tagged ,

Search problems

We’ve been experiencing some search problems over the last 24 hours, we’ve restarted the search service and will continue to monitor the situation.

Tagged
Advertisements