Background Job Processing for Responsiveness
When one merchant imports 50,000 products, it shouldn't slow down a merchant sync for 5 products. I implemented "Fair-Share" queueing.
- Queue Separation: I created
high,default, andbulkqueues to ensure small tasks are never stuck behind large imports. - Retries with Jitter: For failed API calls, I use exponential backoff with jitter to prevent our app from "DDOSing" Shopify when a service comes back online.
- Dead Letter Queues: Any job that fails 5 times is moved to a manual review queue for our support team to investigate.