An operation is idempotent when running it twice leaves the world as it was after running it once. Deleting a file is idempotent. Sending a message is not, and that asymmetry is where most duplicate-message bugs live.
Where you actually need it
- Webhook handlers, because delivery is at-least-once and a retry is normal.
- Anything you retry after a timeout, because a timeout does not tell you whether the write landed.
- Anything a queue can redeliver, which is every queue.
How to get it, cheaply
Keep a table of the ids you have already acted on and insert into it with a unique constraint before you do the work. If the insert fails, somebody already handled this one. That is the whole technique; it needs no library and it survives the restart in the middle that a memory-based version does not.
Insert before, not after. Inserting afterwards leaves a window between doing the thing and recording that you did it, and a crash in that window is exactly the case the table was meant to cover.
What to use as the id
Whatever the sender calls the event, if it gives you one. If it does not, build a key from the fields that make the event what it is — the account, the chat and the message, say — and hash it. A key derived from a timestamp is not a key: the retry has a different one.
Idempotency is not deduplication in the UI. By the time a duplicate reaches a screen you have already sent the second message. The check belongs at the top of the handler, before anything leaves.