How does Square manage its Sandbox Catalog space requirements?

Questions
#1

Hello there,

I recently found out that even when I delete CatalogItem and CatalogItemVariation instances in my Sandbox account, that those instances are actually “soft” - deleted; they still exist on the server, but they have the is_deleted flag set to true. Listing the Catalog still shows those items there.

This is surprising to me because especially in Sandbox environments, people (me) run testing pipelines all the time, which can create, edit, delete thousands of products at a given time. If Square never actually deletes those instances, I can totally imagine your repositories becoming intractably huge.

So I was wondering how Square manages its space requirements and whether there are any background processes that just flush phantom CatalogObjects after a while. Perhaps the rules are different in production environments?

#2

You are right, in the catalog we never actually delete anything. Objects just get marked as deleted. We don’t have a process to clean up the deleted objects because we have some things on our side that use the historical versions of the items. Soon we are actually launching the ability for 3rd parties to access the catalog history as well.
Only a few merchants have encountered issues with deleted objects building up enough to cause performance issues, which resulted in us manually deleting their catalog history. Usually that is because of run away integrations that continuously spam new objects. A few million deleted objects should be ok though. And to address the total storage requirements, our databases are sharded so that we do not run out of space. The sandbox environment for items is just a clone of the prod environment, so we could do a reshard there if necessary.

#3

Thank you for the info. It is amazing how much “junk” we can fit in modern DBs, on-demand or otherwise.

#4

Hi. I wonder why do i have different behavior. I delete all the catalog objects

seller_client.catalog.batch_delete_catalog_objects(
  body: { object_ids: items_ids },
)

then I retrieve all the objects

seller_client.catalog.list_catalog.data.objects

but get an empty array in the result
Has anything changed?

If I manually remove an item from the Square UI I get the same result: it seem like the item gets completely removed, not soft-deleted.

Am I missing something?

#5

if you use SearchCatalogObjects endpoint, there is an optional include_deleted_objects parameter, which if you set to true, you will see the “deleted” objects.

2 Likes