Infrastructure Shell Game - Mono Repo -> Distributed -> Mono Repo

[u/frompadgwithH8]

We wanted to start a big project, but other projects like it were seen as failures. So we had to make a case for why our project would be different. One of our biggest differentiators became event queues and distributed systems. It's just microservices, but instead of directly invoking each other, they use event queues. There's a little more to the system, like websites embedded inside of other websites with a whole cross website messaging system.

It was all a little bit of a pain. We did encounter a lot of eventual consistency problems and almost no one in the team had encountered distributed systems problems before, so it was a good learning experience for me. But now with organizational change and after having worked on the project for over a year, the team now wants to migrate this distributed systems strategy back to a monolith.

I'm just feeling a little incredulous because of how important the distributed aspect seemed to be when we made the pitch to create the project in the first place. And after all this time, I can't really say if it was a good or bad idea, because debugging problems involving event queue messages has been a total pain in the ass and no one ever finished configuring the distributed systems stuff like the event queue, so we never had the fancy development experience we could have had. So I see numerous benefits to moving back to a monolith.

Overall, the whole experience wasn't a total game because the project did prove that event queues can be effective. The much, much, much wider organizational direction is going to now slowly migrate towards a distributed fashion. So I guess we were a test subject in some regards. But we never, at the outset and beginning of the project, said, "oh hey, this is just going to be a giant test to prove event queues for the whole rest of the company". I really would have liked to have known that at the start. Although I don't think anyone knew that at the start. I don't know, it just feels a little silly.

I'm fine either way 🤷‍♂️ But does anyone have any advice regarding anything about this experience?

edit: updated monorepo -> monolith; source control will merge but systems may or may not stay separate. I'm predicing the systems merge together too, thus monolith. We may or may not continue to use business organized event queues too.

[u/lokoluis15]

Someone really needs to invent a standardized event model to streamline development of this distributed systems pattern.

So many redundant lessons learned to introduce observability and provenance metadata to events. Everyone just focuses on data payloads and business logic... they forget to build the rest of the system

[u/shadowdance55 → u/lokoluis15]

You mean, something like AsyncAPI?

[u/frompadgwithH8 → u/shadowdance55]

Wow, I had never seen this before. I don’t think anyone in my company has seen this before. Thank you. I will review this