Last July 31, 2015, we were affected by an outage from Amazon Web Services (AWS). A lot of our backend services were affected like generating reports and sending clockings.
Around 1:06pm SGT we started getting elevated error rates from the AWS API. The root cause as described by our vendor:
The root cause was a networking component failure that resulted in a saturated core service component. Service was restored by clearing the caches on the core component, and gradually shifting traffic to the system.
We restored and replayed clockings done on TeamClocks Android, iOS and Chrome as all these apps have offline support.
The outage lasted from 1:06pm to 4:00pm. Full data recovery was completed at 6:05pm.
If you have any questions about this outage, please do not hestitate to contact us by clicking here .
— Vincent Paca
