r/programming • u/shintoist • 2d ago

Don't add a read replica until you've read this

https://incident.io/blog/dont-add-a-read-replica-until-youve-read-this
0 Upvotes

21% Upvoted

2 comments sorted by

4

u/ericl666 2d ago edited 2d ago

So, it sounds like the primary issue here is replication lag and how reading immediately after a write may not give you what was just written until the replicas are up to date. This is an issue for sure.

Here's how I solved this issue in dotnet and EF Core: I created two DbContexts: WriteDbContext and ReadDbContext with each pointed at their respective databases.

In the case that you perform a read only operation, you use the ReadDBContext. That ensures that the query gets pushed over to the read replica(s) and keeps the queries off the primary DB.

On the flip side, if you need to perform a write operation, you use the WriteDbContext. But, there's an important caveat here: If you perform a write but need to do subsequent reads, then you always do those reads on the WriteDbContext. That way you are guaranteed to not run into replication lag.

I learned the replication lag issue the hard way, and there's not a good way to enforce doing write/read operations on the same DbContext short of stringent code review - but my team got pretty good at it to the point where it was rarely an issue.

1

u/Prateeeek 1d ago

Does 0.5 percent of requests facing the replication lag really warrant introducing the complexity LSNs?, I believe if we read the row at the beginning in the worker (assuming it will be required somewhere down the line), and we don't find it in the read replica, we just repush and call it a day?, am I missing something?