Unfortunately, you are correct, there is no way to really “cancel” the payment once they’re redirected to the checkout API. They can either hit back, or just close out of the page at that point, but there is no way to be redirected back to your web page without paying.
If you want to create a better flow, I would suggest looking into the Square Payment Form, which allows you to fully have it embedded in your own site, so you can control everything that happens. As for the original request, I will update this to be a feature request since it doesn’t technically exist.
the addition of a cancel or back button would be extremely helpful in Square hosted checkout.
both stripe and paypal allow this on their hosted checkout pages
and return either a success_url/return_url for successful charge
or a cancel_url when cancel/back button is selected.