An investigation / iFly
Where were all these requests coming from?
A traffic spike. No source in the logs. One trace back to the caller.
Illustrative request flow from another team’s Lambda through our service to iFly. All investigation findings are listed beside this diagram.
Illustrative trace
- 01
Over the limit.
Requests to iFly exceeded our 250 TPS contract.
- 02
No source in sight.
AWS logs couldn’t identify the team. Request IDs had no source prefix.
- 03
Follow the caller.
I used Dynatrace to trace the requests to another team’s Lambda.
- 04
A deployment backfill.
Historical data had been prefilled during a deployment.
- 05
Make the source visible.
We shared the findings with the integration team and pushed for company-wide source prefixes in request IDs.




