Something we keep doing in this industry is writing our own API client packages/modules. We build a lot of backend services, and most would agree that it's worth extracting some kind of standard functionality for it. Then, all you have to do is take on that package as a dependency and hammer out some boilerplate code and you're well on your way to sprint satisfaction.
When the next sprint comes around, someone has opened a ticket about a possible bug in the HTTP client code related to a production incident. So, you pile on some story points and try to mentally and emotionally prepare yourself to backpedal a bit to see what's going on.
I know what you probably found. Some setting on the standard library HTTP client. A timeout, or a botched header. Unpooled connections. How about a missing retry? A regrettably ambitious retry? Am I right? Ask me how I know. Welp, I've been there a few times.
If I had a woefully incomplete, unsorted list of things to remember when you take a stab at writing an API client, it might look something like this:
- Service Discovery
- Load Balancing
- Timeouts and Expirations
- Retries
- Rate limiting
- Connection pooling
- Circuit breaking
- Failure detection
- Metrics and tracing
- Interrupts
- Context propagation
Feel free to break out this list when someone sends you a pull request to review where they're braving into this familiar and error prone territory.