Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Majority of AWS us-east-1 down/having issues, Heroku included
32 points by robbiet480 on Sept 20, 2015 | hide | past | favorite | 13 comments
DynamoDB has been down for about 25 minutes now and has taken other services with it, like Lambda, Cloudwatch, EC2, ASG, SQS. I'm assuming the 503's on Medium, TheNextWeb and The Guardian are related. Lots of people on Twitter confirming.

AWS has updated their status page to say they are looking into increased error rates on DynamoDB and increased APIs errors on EC2 as of 3AM PDT.

Full list of services affected:

* ASG

* CloudFormation

* CloudTrail

* Cloudwatch

* CodeCommit

* CodePipeline

* Config

* Directory Service

* DynamoDB

* EC2

* Elastic MapReduce

* Lambda

* Mobile Analytics

* OpsWorks

* RDS

* Redshift

* SES

* SNS

* SQS

* SWS

Heroku is now also reporting they are having issues because of AWS: https://status.heroku.com/incidents/811#update-2808

Status page is http://status.aws.amazon.com/



I'll take the opportunity to once again complain about how AWS's status page is a UX clusterfuck.

Pretty much the entire us-east-1 region is unreliable, that needs to be glaring from the page. Instead, I need to browse hundreds of happy green checkmarks to find out what is/n't broken by their unreasonable standard.

What a fucking joke.


Not even consistent...

Amazon DynamoDB (N. Virginia) Increased API error rates : Red icon.

Amazon Redshift (N. Virginia) Increased API Error Rates : Green icon with check mark.

Two recommendations:

- No green with check mark (info). Change that to yellow/orange. At this point you are not fooling anyone that green with a check mark is not yellow. There can be no green if it is service impacting (reduced availability, performance, broken, etc - anything other than the norm)

- Default sort should be by status : sort order is red-yellow-green. Need to know whats broken up-top.


Goes without saying that Heroku is a great example of how it should be done -

https://status.heroku.com/


Unfortunately I'd have to disagree. Production applications belonging to paying customers are completely down - I don't think "Elevated Build and API Error Rates" quite does it justice.

Note: The applications in question have had zero interactions (deploys, restarts, scaling, etc.) which might have caused their dynos to be unallocated/re-provisioned within Heroku's own service.


<3


Wrote a simple script to hide all the green statuses https://gist.github.com/vijaydev/c0a670e79dcf7657cb2e


We're seeing the same problems with DynamoDB... it started around 2:20 AM PDT (1 hour ago). We're not having any problems with EC2—what kind of problems are you seeing? I hope it's just affecting the API and not running instances.

Side note: Medium posted an article last week about how most of their data is stored in DynamoDB, so I'm sure the Medium outage is related.


Right now I can't load Lambda, CloudWatch or SQS. People on Twitter reporting massive ASG issues that caused their deployments to scale up massively suddenly.


Same here... PagerDuty started blowing up a little while ago only to discover we're seeing issues in SQS and Dynamo (plus ASG).. Damnit


....and now Dynamo has gone to full-blown "RED" status on the AWS status page. :sigh:


Just spoke to AWS Premium Support around 4:35 AM. They don't have any more information than what's on the status page, basically. I was hoping to get an ETA for a resolution, but no such luck.


AWS Premium support is a joke. I never got a reply under 1 hour time, like they promised.


Our app is mostly dependent on AWS's DynamoDB and we're facing problems with it. I can confirm the outrage of it.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: