Capsule CRM - System Status

Investigating

Wed, 18 Apr 2012 05:12:00 +0100

5:10AM BST - Investigating

We’re investing a problem with Capsule. We’ll have the system back online as soon as we can.

6:10AM BST - Downtime Update

We’re working to restore Capsule as quickly as possible. Our primary database server has failed and we are in the process of failing over to our replica. We’re just waiting now for a new slave to come online before we enable access to the system again. We’ll have the system back online as soon as we can.

6:35AM BST - Downtime Update

We have the primary database back online but it’s taking longer than normal to bring up the new slave database. Our ETA for having the system back online is 9:00AM BST.

Very sorry for the disruption. We’ve had all hands on deck this morning to have the system back online as soon as we can.

8:20AM BST - We’re back online

Capsule is back online. Very sorry for the disruption this morning. We had a hardware failure on the primary database server which we’ve recovered from without data loss. The experience this morning has raised some ideas internally that we’ll implement to make recovery from a full hardware failure faster next time around.

Scheduled Maintenance: Sunday 1st April

Fri, 30 Mar 2012 16:49:00 +0100

The system will be unavailable for up to 60 minutes on Sunday 1st April 2012 starting 08:00 BST (see other time zones) while we perform System Maintenance.

UPDATE:

Slight change of Plans. Down time will now commence at 10:00BST (see other time zones)

UPDATE: 1050 BST

Maintenance is taking a little longer than planned. Planning on being back by 1115 BST

UPDATE: 1100 BST

Scheduled maintenance for Capsule is now complete. Thanks for waiting.

Investigating

Thu, 15 Mar 2012 09:42:00 +0000

9:35AM GMT Investigating

We’re investing a problem with Capsule. We’ll have the system back online as soon as we can.

9:46AM GMT Update

We’re up and running again; sorry for the hassle while this was getting sorted out. There was a networking connectivity issue with our hosting provider which has now been resolved.

Scheduled Maintenance: Sunday 11th December

Thu, 08 Dec 2011 11:05:00 +0000

Capsule will be unavailable for up to 60 minutes on Sunday 11th December 2011 starting 8:00 GMT (see other time zones).

Update 09:05 BST

Maintenance is complete and we’re back online. Thanks for waiting.

Scheduled Maintenance: Sunday 20th November

Thu, 17 Nov 2011 11:18:00 +0000

Capsule will be unavailable for up to 60 minutes on Sunday 20th November 2011 starting 7:00 BST (see other time zones).

Update 8:05am

Maintenance is taking a little longer than planned. Will update with eta soon

Update 08:32 BST

Maintenance is complete and we’re back online. Thanks for waiting.

Scheduled Maintenance: Sunday 16th October

Fri, 14 Oct 2011 09:54:00 +0100

Capsule will be unavailable for up to 60 minutes on Sunday 16th October 2011 starting 7:00 BST (see other time zones).

Update 07:57 BST

Maintenance is complete and we’re back online. Thanks for waiting.

Scheduled Maintenance: Sunday 9th October

Wed, 05 Oct 2011 09:17:00 +0100

Capsule will be unavailable for up to 60 minutes on Sunday 9th October 2011 starting 7:00 BST (see other time zones).

Update 07:38 BST

Maintenance is complete and we’re back online. Thanks for waiting.

Scheduled Maintenance: Sunday 28th August

Sat, 27 Aug 2011 10:40:00 +0100

Capsule will be unavailable for up to 30 minutes on Sunday 28th August 2011 starting 15:30 BST (see other time zones).

Update 16:00 BST

Maintenance is complete and we’re back online. Thanks for waiting.

Scheduled Maintenance: Saturday 27th August

Thu, 25 Aug 2011 11:26:00 +0100

Capsule will be unavailable for up to 4 hours on Saturday 27th August 2011 starting 6:00am BST (see other time zones) while we move the database to new servers.

Update 7:45am BST

Maintenance is complete and we’re back online. Thanks for waiting.

Investigating

Tue, 09 Aug 2011 16:43:00 +0100

We’re seeing a slow patch at the moment and are investigating the problem with Amazon. We’ll get it resolved as quickly as we can. Sorry for the slow performance currently.

Update 22:00 BST

We’re back to full speed again after a successful fix by Amazon. Sorry once again for the slow patches earlier.

Outage - resolved

Tue, 09 Aug 2011 04:24:00 +0100

4.05am BST

Unfortunately Capsule was unavailable between about 3.30am and 3.55am this morning. This was the result of a network failure at our hosting provider (Amazon EC2). We’re back online and looking to see if there are any lessons to be learnt from the failure, although it appears it affected all services using the Amazon EC2 east region.

Very sorry for the outage.

Scheduled Maintenance: Sunday 24th July

Wed, 20 Jul 2011 11:41:00 +0100

Capsule will be unavailable for up to 2 hours on Sunday 24th July 2011 starting 6:30am BST (3:30pm Sydney, 5:30pm Wellington) while we perform a software and database update.

Update 7:45am BST

Maintenance is complete and we’re back online. Thanks for waiting.

Investigating

Wed, 13 Jul 2011 00:26:00 +0100

12:26am BST / 4.26pm PDT

We’re investing a problem with Capsule. We’ll have the system back online as soon as we can.

Update 12:40am BST / 4.40pm PDT

We had an unresponsive storage volume for a short period of time which came right at 12.31am BST. We’ll check in with Amazon to establish the cause, but everything is back to normal now. Sorry for the downtime this evening.

Investigating

Wed, 29 Jun 2011 08:26:00 +0100

8.25am BST

We had a problem with a backup processes this morning which has disabled the site. We expect to have Capsule back online very shortly.

Update 8.28am BST

Capsule is back online again. Very sorry for the short period of downtime this morning.

Scheduled Maintenance: Sunday 3rd July

Tue, 28 Jun 2011 15:01:00 +0100

Capsule will be unavailable for up to 3 hours on Sunday 3rd July 2011 starting 6:30am BST (3:30pm Sydney, 5:30pm Wellington) while we perform a software and database update.

Update 9:26am BST

Maintenance is complete and we’re back online. Thanks for waiting.

Scheduled Maintenance: Wednesday 8th June

Tue, 07 Jun 2011 19:42:00 +0100

Capsule will be unavailable for up to 30 minutes on Wednesday 8th June 2011 starting 6:30am BST (3:30pm Sydney, 5:30pm Wellington) while we perform a software and database update.

Update 7.00am BST

Maintenance is complete and we’re back online. Thanks for waiting.

Short Notice Maintenance

Tue, 24 May 2011 23:30:00 +0100

Capsule will be unavailable for 4 hours from 5:00 BST Wednesday, 25 May (21:00 PDT, midnight EDT, 14:00 AU EST, 16:00 NZT).

This is to move the database into a faster storage environment. We are very sorry for the short notice and duration of the downtime, but the performance of the system over the past 24 hours has been degraded as a result of a performance issue in our current storage environment. The maintenance will enable us to bring the system back up to speed.

Update 7.15 BST

We are making progress. Unfortunately the transfer is taking longer than anticipated. Based on current transfer rates we are expecting a further 2.5 hours downtime. That pushes the ETA out to 11.30 BST.

This is frustrating because we tested the process twice yesterday and completed the process each time in a 4 hour window. This morning the performance of storage environment has slowed further hence the delay in moving the data over.

We know this is also frustrating for our customers. I’m sorry that we aren’t able to make this happen more quickly. We are fully focused right now on getting the transfer done and bringing Capsule back online as quickly as possible.

Update 10.05 BST

The first part of the process is complete. We are now warming up the servers in the new storage environment. We are still on track to bring Capsule back online at 11.30.

Update 11.25 BST

We’re back online. We know the downtime this morning isn’t acceptable and we are putting changes in place to enable much faster failover if we get into this situation again.

Investigating slow performance

Mon, 23 May 2011 20:26:00 +0100

We’re investigating reports of slow performance.

Update 23 May, 21:15 GMT

We had a spike up on the latency of a storage volume. We’ve raised a support case with our infrastructure provider (Amazon Web Services)

Update 23 May, 21:30 GMT

Performance is back to normal.

Update 24 May, 05.15 GMT

We are continuing to monitor performance as we are seeing some elevated latencies from time to time. We are working with Amazon to resolve.

Update 24 May, 22:10 GMT

Unfortunately we are seeing a slow down on the storage volume which is affecting the service. We are working to restore as quickly as we can. Sorry once again.

Service running slowly

Wed, 11 May 2011 16:31:00 +0100

Sorry about the slow performance we’re experiencing at the moment. We’re working on it and should be back to normal shortly.

Update 16:50 BST. Performance is back to normal.

Scheduled Downtime: Sunday, 1st may 08:00-09:00 BST

Sat, 30 Apr 2011 15:26:00 +0100

The system will be unavailable Sunday, 1st May between 8.00am - 9am BST.

Update 9.02am BST

Maintenance is complete and we’re back online. Thanks for waiting.