Last year (2024), I had the chance to work on a payment system. While working on it, I noticed that synchronous approaches are impractical for long-running processes—common in production-scale applications.
The Problem with Synchronous Processing
What's a Synchronous Approach?
A synchronous process executes from start to finish in a single blocking call. The client waits and receives either a success or failure response immediately.
func CreateTransfer(ctx context.Context, request CreateTransferRequest) (CreateTransferResponse, error) { // 1. Validate user permissions // 2. Verify destination account validity // 3. Check sender's balance // 4. Authorize (hold) funds // 5. Call payment gateway (slow!) // 6. Update balances based on gateway response // 7. Record transaction in DB // 8. Return final result}
Why Synchronous Approach Fail?
- Resource Bottlenecks: each call holds a connection thread (means more costly)
- Timeout Risks: network instability may interrupt long-running calls
- Poor Scalability: Handling 100 RPS for a long-running process can be very expensive
The Asynchronous Solution
instead of processing it in a single-go, lets split the process...
Step 1: Immediate Acceptance
func CreateTransfer(ctx context.Context, request CreateTransferRequest) (CreateTransferResponse, error) { // 1. Persist request to DB (status=PENDING) // 2. Publish to message queue (e.g., Kafka) // 3. Return 202 Accepted with status=PROCESSING return CreateTransferResponse{ Status: "PROCESSING", Message: "Transaction queued", RequestID: requestID, }, nil}
Step 2: Background Processing
func ProcessTransferWorker(ctx context.Context) { for { msg := queue.Consume() // 1. Validate all business rules // 2. Execute payment gateway call // 3. Update DB status (SUCCESS/FAILED) if err := process(msg); err != nil { metrics.RecordFailure() continue } }}
ok, thats simple right, but there is a question how the client know when the transaction is either completed or failed ?
Step 3: Retrieve transaction status
// GET /transactions/{id}func GetStatus(ctx context.Context, id string) (StatusResponse, error) { tx, err := db.GetTransaction(id) if err != nil { return nil, ErrNotFound } return StatusResponse{ ID: tx.ID, Status: tx.Status, Updated: tx.UpdatedAt, }, nil}
Sounds simple enough, right? But there's another approach we can take—without creating an extra endpoint (and you know, more endpoints mean more things to build, handle, and maintain 😅). Let’s keep that for another post.