One of the more annoying side effects of connecting systems together is duplicates. The same invoice booked twice, the same order shipped twice, the same customer showing up in your CRM under three different records. It looks trivial — until you start counting the hours wasted on cleanup and the money spent on extra couriers.
Duplicates don't come from "the system just being stupid". They come from a very specific mechanism nobody mentions during the demo. And the good news is you can eliminate them without rebuilding the whole integration — as long as you understand where they actually come from.
Where duplicates come from
A classic integration between two systems — say an online shop and an ERP — works roughly like this: the shop sends "here's a new order, no. 12345" to the ERP. The ERP replies "got it". One order, one confirmation, done.
The trouble starts when the confirmation doesn't arrive. The network hiccupped for a second, the ERP was briefly unresponsive, a timeout, whatever. The shop has no idea whether the order was accepted or not. It has two options: assume it wasn't (and resend), or assume it was (and let it go). Almost every implementation picks the first — better to send twice than lose a customer's order.
And that's your duplicate. The ERP actually accepted the first attempt — the reply just didn't get back. The second attempt hits the ERP as a brand new order. Nothing stops it from saving both.
The same mechanism plays out anywhere systems meet: bank → ERP, ERP → WMS, courier → shop, POS → ledger. Anywhere "someone sends, someone receives, and there's a network in between" that occasionally stalls.
Idempotency — the concept in three sentences
The fix has a technical name — idempotency — and sounds harder than it is in practice. The idea: send the same operation ten times, and be sure the result is the same as if you'd sent it once. Like a light switch: the first press turns the light on, the next nine change nothing.
In practice it works like this: the sender (shop, courier, bank) attaches a unique number — the idempotency key — to every operation. The receiver (ERP, WMS) remembers which keys it has already processed. If the same key shows up again, it just replies "got it, thanks" and doesn't duplicate anything. Stripe and PayU use the same trick for payment transactions.
It's one of those things you have to design in at integration build time — bolting it on afterwards costs more. But it's always doable.
Example: an auto parts wholesaler
One of our clients — a 22-person auto parts wholesaler in Greater Poland — had a setup like this. Comarch Optima as the ERP, BaseLinker as the bridge to Allegro and a Shoper shop, a lightweight WMS in the warehouse. Orders from Allegro flowed to Optima via BaseLinker, then down to the warehouse for shipping.
On average, 14 duplicate orders per month. Not catastrophic against total volume, but a real operational headache. The accountant spent about 6 hours a month finding and reversing duplicates. The warehouse guy occasionally shipped the same box twice — courier cost, plus the effort of getting the goods back from the customer, plus the "very sorry" phone call. Customers saw two card charges and reached for the phone. Over a year, that added up to dozens of hours of management time spent apologising.
Diagnosis took us half a day with the logs. BaseLinker was pushing orders into Optima via a middleware layer (custom, written 4 years earlier by someone who had since left the company). On timeout, the middleware retried — with no marker saying "this is the same operation as before". Optima happily saved the second copy.
The fix was boring. We added an idempotency key to the middleware, built from baselinker_order_id + created_at. Optima got a simple database view that checked whether the key had already been seen. As a safety net for earlier sins, a nightly job hunted for identical orders inside a 24-hour window and raised an alert.
Over the next 3 months — zero duplicates. Boring, but effective.
What you can do this week
Even without budget for a full integration overhaul, you can limit the damage with a few quick moves:
- Measure the problem. Ask accounting how many reversed invoices and orders you had last quarter. Until you have a number, no one will take it seriously.
- Turn on alerts. A simple report — "orders from the same customer for the same amount within a 15-minute window" — catches most duplicates before they hit the warehouse. It's a SQL report in Optima, enova or Subiekt.
- Check the bridge settings. Many mid-tier bridges (BaseLinker, Selly, and similar) have a "don't resend the same order within X minutes" option. It's often disabled by default.
- Ask your middleware vendor about idempotency. If the integration was built by someone external, there's one question worth asking: "does your code use an idempotency key on this integration?" If the answer is "what's that?", you've found your reason for the duplicates.
- Don't try to fix it in the ERP. The middleware/bridge is what should guarantee once-only delivery. Cramming dedup logic into the ERP tends to end in tangled accounting rules.
When to bring in help
If duplicates still show up after those steps — or the integration is generating whole cascades of secondary damage (double warehouse reservations, double shipments, double credit notes) — that's a sign it's worth adding a proper idempotency key. This isn't a revolution: for most integrations between popular systems (Comarch Optima, enova365, Subiekt GT, Microsoft Dynamics AX, BaseLinker, Shoper, WooCommerce), it's a few working days of effort, without touching the source systems themselves.
Duplicates are boring. But the cost of handling them grows with your volume — and at some point they cost more than rebuilding the integration from scratch.
Idempotency is one of those things nobody mentions during the demo, but it decides whether your integration will feel like a lifesaver a year later — or a permanent source of correction work.