Understanding the Essentials: Idempotency

As a developer, you are always wanting things to go smoothly, but in the world of REST APIs, that isn’t always the case. Client failures, networking errors, or just poorly written code can all spell trouble for an API call (and your end user experience). Luckily, idempotency is here to help! It can be a little confusing at first, but once you have a handle on it, you’ll see that it is an important aspect of designing and using mission critical systems.

Idempo-whatnow?

Idempotence is the property of certain operations in mathematics and computer science whereby they can be applied multiple times without changing the result beyond the initial application. — Wikipedia

In the context of a developer working with REST APIs, your application is sending requests and getting back responses with some other service on the internet. When that API is idempotent guarantee that when you send an identical request you will always get an identical response. What do I mean by identical? The headers, body, authorization, everything needs to be exactly the same.

Not all REST APIs use idempotency, most of the time it is only necessary for requests that influence the state of something, or you really don’t want to accidentally run twice. For many APIs, this is limited to POST or PUT requests, whereas most “state neutral” requests do not add the complexity to their interfaces. In most cases, with Square’s APIs you will see this in the context of Transactions where you want to keep the state of bank accounts in order, and prevent accidental money movement.

Idempotency with Payment APIs

For our idempotent APIs, there is a dedicated body parameter called idempotency_key to help you keep track of your requests. This is a string that you include in the request body. When we see the request you make with that specific key, we can ensure that we will process the requested operation only once successfully. When you send a request with the same idempotency_key we’ll send back the current successful response. If you are familiar with the Etag header it is another method to help prevent accidental state changes but with a different mechanism.

It is important to note that getting back a “successful” response in the context of idempotency doesn’t necessarily mean that you are getting a 200 OK. Even if you get back a 401 or 500 or something else indicating that your requested operation didn’t work, you managed to get a response indicating that whatever was sent did not succeed. The intention here is to handle ambiguous moments where an operation was attempted but its unclear what the result was, so you retry it with the same idempotency_key. You can safely retry the request without concern of doing the same operation more than once by just retrying with the same idempotency_key.

You might be thinking that there are lots of good use cases where you want to be making lots of similar requests; how does making the same API call twice differ from making a single request that gets repeated from network stutter?You’ll just need to make sure that for each transaction you want to make, you have a unique idempotency_key for the request.

“Where do I find the idempotency key?”

Where are you supposed to get that unique value for each request? Well, it isn’t something you can look up in your developer dashboard like your access token. You’ll need to decide the best way to set the idempotency_key with your application architecture. Common examples are using the current timestamp or a UUID for the idempotency_key and while it is true that these are generally unique, they have some drawbacks. Consider an e-commerce site with a single page front end that sends the card data (in this case a card nonce) to a backend asynchronously to make a charge. What happens if your user clicks that charge button twice, sending two requests to your backend? If you are dynamically creating the idempotency key each time your backend code runs, then you would accidentally be charging your customer twice (in reality you can’t charge the same nonce twice, but that is besides the point).

A better strategy for setting your idempotency_key might be to use something like a session id, a hash of the username & date, a transaction id that you are using in another system, or even a uuid that is assigned when a user starts a session instead of at the time of the request. That way you might guarantee that every unique idempotency key that you make is a for a unique purchase but you’ll also have the information to catch problems before the request when you might accidentally making duplicate charges. There isn’t a value that works for everyone, but spend some time to think about how applications will be making idempotent requests before you architect your system.

In many cases, working with payments for the first time can be your first introduction to idempotency, but that doesn’t mean it should be a bad one. Idempotency is a powerful tool in helping your apps and services be more resilient against the unexpected and uncontrollable. If you want to learn more, you might want to check out the Square documentation or this video about cows (the premier work on the subject).

Still confused?

Don't worry, watch this deep dive episode on idempotency that covers its history, use in REST, and even some live coding.

If you want to keep up to date with the rest of our content, be sure to follow this blog & our Twitter account, and sign up for our developer newsletter! We also have a Slack community for connecting with and talking to other developers implementing Square APIs.