Apigee Release Schedule

Apigee Edge: Tue-Thu 12am to 4am in Singapore, Central European, and US Eastern time zones (learn more)

Read the release notes to learn what is new.

API Services Disruption
Incident Report for Apigee
Postmortem

Impact Start Time (UTC): 2018-Oct-2 00:16 AM

Impact End  Time (UTC): 2018-Oct-2 00:58 AM

A capacity related incident affecting the Apigee NAT gateway tier in the US-East region caused a service impacting issue for API runtime traffic across multiple customers.  No other regions were impacted, and all other services were working as expected during this time. During the outage time period, CPU resources on the NAT gateway units were exhausted due to an inordinate increase in API traffic.  The NAT servers were functional and processing some traffic, but given the load signature, a high amount of packet loss was registered causing severe request processing latency and/or an increase in HTTP 503 error rates for the duration of the outage.  The source of the traffic increase was isolated and corrective actions taken to mitigate impact. A full RCA has been completed and follow-up items prioritized to help prevent the risk of recurrence.

If you were impacted by this incident and would like a copy of the more detailed Incident Report, please open a P3 Support case to Apigee Support.

Posted Oct 09, 2018 - 16:04 PDT

Resolved
This incident has been resolved.
Posted Oct 02, 2018 - 09:45 PDT
Update
All services in the Apigee US-East region have been fully restored. If you continue to see any service impact, please log a support ticket with Apigee Support.
Posted Oct 01, 2018 - 21:09 PDT
Monitoring
Primary actions to restore services have been completed and are observing evidence of service recovery. We are in the process of validating full service restoration. Additional updates to follow.
Posted Oct 01, 2018 - 18:45 PDT
Identified
We are still in the process of implementing the fix to resolve this issue and will keep you posted on the progress shortly.
Posted Oct 01, 2018 - 18:23 PDT
Update
We are continuing to investigate this issue.
Posted Oct 01, 2018 - 18:09 PDT
Investigating
We are currently experiencing a disruption with API Services for the US-East region. During this time, customers may experience delays in their API response and/or increased error rates. We apologize for any impact this is causing and are working to restore services.
Posted Oct 01, 2018 - 17:55 PDT
This incident affected: API Services, Management Services UI, and Management Services API.