← All cases

Retail · supermarkets in two countries · 2020 — 2021

Making the website's stock match the store's

Five microservices connected through Pub/Sub that carry stock from the merchandising system to the e‑commerce site, in two countries.

The problem

When a supermarket's website says a product is in stock and it isn't, the problem reaches the customer's door. Here, stock travelled from the merchandising system to the e‑commerce site through a middleware.

Merchandisingon-premiseIntakePub/SubGCPStorageAssortmentQueriesGrouping and sendingE-commerce
From merchandising to e-commerce, in steps that scale separately.

What I did

I built the new integration from scratch with a teammate, to whom I handed down the architecture: five Node.js microservices, one on-premise and four on GCP, connected through Pub/Sub.

The first one receives every stock change and publishes it as an event. The others store it while discarding stale data —⁠in an asynchronous flow messages don't arrive in order⁠—, group it by country, store and type before sending it to the e‑commerce platform, and answer queries by product and store.

I followed the whole delivery through: QA, the system documentation and the handover to the support team.

What happened

It replaced the previous integration and went live in both countries.

During development, the QA team stress-tested it with Artillery: it injected over 900,000 events in ten minutes, at about 1,100 per second. The ingestion service reached 1,300 transactions per second and answered 100% of them successfully.

The test also showed why splitting the pieces paid off. Storing that volume took about 40 minutes, limited by the test environment's database, and meanwhile the events waited in Pub/Sub: ingestion kept answering at the same pace and none were lost.

What I took away

Delivering doesn't end at the production release: it carries on into QA, into support and into keeping an eye on what breaks.

Node.js · Koa · MongoDB · GCP Pub/Sub · Kubernetes