From 8 Years to 6 Months: Orchestrating AI Agents for Large-Scale Refactoring

Let's play a game. We're going to play Jenga. But our game is going to have a little bit of a different rule set. We're going to build a tower for a decade. We're going to have some of the blocks glued together and some of the towers are going to be tied together with strings.

This is our daily reality because we've been building our client-side monolith for a decade now. It has over 10,000 selectors, actions, reducers, over 40,000 files, and, of course, over a decade of work. We desperately wanted to break it apart. And we estimated it. We decided it's going to take over eight years of work to do that. That's not realistic for us.

We were able to harness AI to do everything for us. And we're here to tell you the story on how we did it. My name is Amit. I'm a tech lead at Monday. And I'm Alon. And I'm a team lead at Monday. And this is MorphX, our AI tool that is breaking our client-side monolith day and night. And to give you a bit of context on where this project is coming from, let me tell you a few words about Monday.

So Monday is a global software company that helps customers scale their business impact and expand what teams can deliver, and now with one AI platform for agents and people to work together. And before we talk about Monday's monolith, let me tell you a little bit of what kind of a web application Monday is. It is not a trivial pages application. So, imagine a page and add some micro-frontends to it and then add a micro-frontend for navigation and some context here and a heavy stated up there.

In addition to all of that, sessions tend to last a long time at Monday. And tabs tend to stay open for several days, for three and more days with a lot of interaction and a lot of dependencies between all of these components and micro-frontends. So our client-side is a sophisticated, complex piece of software running on your browser and some of its dependencies and services have been built and maintained for over than ten years with a lot of Monday developers and external developers contributing code creating this massive piece of software.

This is a real example of a selector that we have. You can see it depends on so many other selectors and each one of them may depend on so many other selectors and each one of them depends on services and utilities and a lot more files, resulting in the fact that this single selector can depend on hundreds of files.

So, what we came out to do. We wanted to break the core, the very hearts and brains of our client-side monolith and doing so while keeping some ground rules. We wanted zero production incidents. We wanted to create no behavioral changes in our application. We wanted to convert everything to TypeScript along the way and to keep the very minimal human friction in the process.

What is that Morphix that is breaking our monolith? To the average developer at Monday, Morphix is creating random pull requests at random times and sends these pull requests to them to review and merge. In this example, you can see a Slack message. Morphix sent these two developers telling them it had done a lot of work for them and now it's up to them to review and merge it.

Now how to pick the right file to extract. Just like in the Jenga game, you need to know which brick you can extract without breaking the entire tower. We don't want our product to crash. But if you ever played Jenga, you know that eventually the tower must fall. When extracting files, if you just pick a random file and you want to take it out of the monolith to extract it somewhere else, you know you can't just take it out because it has dependencies and it has other blockers that must be solved before that specific file must be handled.

But if we could just take out the bricks from the top of the tower, that would be much easier game to play. And eventually, for our monolith, we found how to do so. We created the Morphix analyzer which is a static deterministic tool using an AST under the hood that traverses the entire code base and builds a huge dependency tree. And its leaves are just like the top of the tower. It is free for extraction. And after the analyzer runs, we take all of the data, all of the migration data, the dependencies, complexity, and all of the other relevant data and we just threw it to a Monday board.

And now, with the board, Morphix and us have clear visibility on the extraction progress, status, and order, creating an amazing human agent collaboration where human and agents are working together on the same board towards the same goal. And with this board as a sink in place, we now enable Morphix to run parallelly extracting multiple files simultaneously. So let's see how we can extract a single file, a single block out of the monolith. We just go to Claude, of course, and we say hey, Claude, please take this file out of the monolith. And as we said, we want to also convert it to TypeScript. This is what the prompt looked like, like a one-liner.

And Morphix runs autonomously with no human. So we weren't able to do it this way because we needed to one-shot it, right? We need it to work every single time, thousands of thousands of times. And we also wanted more stuff. For example, we wanted to add a feature flag to the prompt. And we wanted to add some more context about the guidelines and standards and how we write code at Monday to the prompt. And we added more and more things. And we were still not able to one-shot it consistently.

So it turns out that making the prompt larger and larger is not the way to go. The other way around, we split it into microprompts. Each one of those microprompts has its own dedicated task and dedicated job that it does really well. And this way we can one-shot it more consistently. We built it in this way that each step needs to succeed because the following step relies on it to succeed. That way if the first step, for example, didn't work so well, then probably the rest will fail as well.

But that raised the question, how do we know when a step is done, right? When a prompt really succeeded? Because it's AI. AI can tell you, yes, it worked. But sometimes that's not the case, right? We created this mechanism. Here we let AI do the work. We run a set of validations. And those validations happen deterministically after the AI runs. I don't just tell the AI in the prompt run lint when you're done.

No. We run it. We make sure that 100% of the time it runs. And if anything fails, we will retry and retry again and again with proper feedback about what went wrong and let's try the task again. For example, we have this converter type script, right? And we have these two validations. First is the lint. As I said, we run it. And we not only let AI run lint, we run the bash command and make sure that we give the proper feedback. And we also use AI as a judge.

We spin up another cloud with a reviewer prompt that reviews the task, knows what the task is, make sure to know if it succeeded or not. And it's a fresh context. So that way we know for sure that the job really is done. Now, we also understood that not all work necessarily needs AI. For example, moving constants, we need to create a good setup for the AI to succeed. If we give it so many distractions like moving constants around, copying files, it will just eat up context, it will be expensive, it will cause hallucinations.

So we made sure to use deterministic operations whenever we could for very simple tasks, creating a really great setup for AI to succeed and also it's free. So it reduces costs and makes sure we get the best results possible. So we wrapped everything in a nice builder. And this builder, as I said, is the way we define migrations. We have steps. And each of the steps will move to the next step only when all validations succeed and the job really is done. And this way we know for sure that the migrations worked and ran.

Now, the obvious question we asked ourselves is, are we overcomplicating this? Because this is a whole lot, right? Like the builder and the validations and everything. So the short answer is no. Because we are migrating the core of Monday. It's a big company. We have over $1 billion of revenue. And 100% of our users are using this code. If anything breaks, the blast radius is absolutely huge. So in order to succeed every time, thousands of times, we needed safety control. And our safety mechanisms are those. The ones I'm gonna show you right now.

First, we wanted a safe release. We need to wrap in a feature flag. But we're moving a file out of the monolith. So in order to wrap in a feature flag, we need to change all of the imports. Which is kind of clunky and weird and we can miss some occurrences. So we use the following mechanism. We take a file and we convert it to a folder with an index.js file. Now JavaScript knows to resolve it automatically exactly the same.

And then the feature flag is in a single place. You can see in the index.js file, there's a single toggle for the new and all implementations. And that way we can safely release our changes to production. It caused a nice side effect. Instead of changing the entire import throughout the entire repository, which is hundreds of hundreds of files, we have a very concise number of files. Easy to review. Easy to be confident about your release. And that way it's manageable to us humans. But we do need to change the imports at some point, right? So we moved it to the cleanup PR. Cleanup PRs were huge. Hundreds of files. But all of the changes are single lines.

Very simple to review. You breathe through it. You do it fairly quickly and we are confident about it. So, we also wanted zero behavioral changes. Because when you move such sensitive code, we wanted to maintain all of the existing behaviors, all of the existing bugs. Not create anything new. Because any new bugs can be very severe.

What we did is we tested it. We added unit tests for each file we extract. And all of the unit tests are not to test the logic inside the file. It's to test the new implementation against the old implementation. To make sure that we have zero behavioral changes. To do that, we created this utility. Code results are the same. We check if the results are the same.

If one throws an error, then the other will too. If one returns undefined, for example, then the other will too. And we abuse those tests. We create a lot of tests to make sure that the behavior doesn't change. On top of that, sometimes AI can find a bug. Or not do the job just right. Or can't decide on a challenge it needs. It over complicates things.

And we needed AI to escalate to a human whenever it needs some help. So, we created this mechanism. AI can add a human to do comment and this comment is blocked in the PR. Now, the process doesn't stop. Because if AI couldn't decide, it can escalate to a human and then a human will resolve it. But the file will not be stuck because each file will unleash more and more files. It will unclog more files out of the monolith.

So, that way, AI can escalate to a human. A human resolves it after it's merged. Then the PR can be merged as well and the file can be extracted out of the monolith. Amazing, right?

Now, let's zoom out a little bit and look at the entire flow. Morphx picks the file out of the dependency tree and creates the pull request of extracting it. Then ships it to the developer to review and merge it.

Once merged, Morphx takes the control back and releases the flag automatically. After that, it creates a cleanup PR, deleting the old file automatically and sends it back to the developer for review and merge. The cycle continues, allowing Morphx to rebuild the dependency tree and move to the next file.

So, we've done all of that work to be able to safely extract a single file out of our monolith. And with that, we felt we have everything figured out. We have the ability to extract files. We know the scope of work. We sat down to calculate everything. We listed all of the files to be extracted and all of the relevant teams. And we spread the work across the coming six months.

And what we came up with was 155 files a week with two PRs per file, one for the migration and one for the cleanup, resulting in 310 PRs a week. And we're done. And then we really felt that we've reached what we came out to do. The plan started to work. We were invited to share our story to the entire company. We had VP meetings and champagnes and limousines and everything was great and everything was running smooth. Yeah. We only needed the human developers to review 310 PRs a week and we're done.

Yeah. Right. We only needed the human developers to review 310 PRs a week. Do you think it could work? I don't. Well, no, it didn't. Very soon people started delaying the work. They started questioning if that effort really was worth their time. We actually received more ambitious demands from our leadership. And new features on Monday seemed more appealing to the developers rather than reviewing the pull request that Morphix had generated for them.

So in that time, we decided that we need to fully unleash Morphix to allow it to automatically merge any PR it finds that is good enough to merge. If it passes its review and requirements, it is able to automatically merge the PR without waiting for anyone. And the results were visible. What you can see is the change before the auto-merge flow and after the auto-merge flow.

But the success did not come only from tech. Right? It was everything on its own. It was really good at creating the PRs, reviewing them, and merging them. But we also needed humans because the future of agents has humans with it. First, we needed to listen. We listened to our developers, we got feedback, and we created a very short feedback loop, getting feedback, fixing it, getting feedback, and fixing it.

We also removed the coding from our developers, but we kept the ownership. So that means that developers own the code that Morphix releases to production, which is crucial. And we made Morphix fit into our organization. We made Morphix not merge on weekends. We know the developer workdays. We don't want any interruptions. We want it to be as smooth and seamless as possible. And we created direct communication channels. Morphix sends Slack messages. I merge this. I release this. This cleanup is already done. And that way, the developer really knows what Morphix did for them. And we built trust. We created a new software development lifecycle inside our company. This is the first time it was ever done in the company. And we built trust in the R&D to make sure that Morphix really does the work and developers now trust it.

To summarize, we want to share some of our key learnings from this project. First split into micro prompts. That way, dedicated tasks and very specific tasks just work better. Also, your results will be the same as your workflow. Deterministic workflow will always give you good, deterministic, consistent results. Create a good setup before you let AI do the task. Like we do with constants and we're copying files before AI goes to do the task, it helped us a lot. It reduced costs, and it creates way better results.

Every autonomous AI project needs a human in the loop mechanism. Because if AI reaches a halt or a dilemma it can't solve, you cannot put the brakes on your project. You need to move on and you need humans to unclog it sometimes. And using our very own Monday board. This was actually a game changer. Instead of using a database or anything, this way, imagine the entire Monday R&D working on the same board with Morphix, transferring ownership, keeping good communication throughout the entire company and the entire R&D, and this was huge for us. It was really collaboration between Morphix and the entire R&D.

Going fully agentic and letting Morphix really do the work relied a lot on humans, on the culture, making Morphix fit into our organization and our needs was a game changer here and it was a lot of work and a lot of feedback. And we are really happy we made it work. So we also wrote a blog post, which I will move to in a minute because I see some of you are taking pictures of the learnings, if you want to also scan the QR code. And that's it. Thank you so much for listening.

PR reviews were initially a bottleneck that required significant effort to streamline. The team implemented various automation techniques to prompt and guide reviewers, focusing their attention on specific areas for review. Meetings were conducted to ensure teams understood what needed reviewing, excluding unnecessary checks such as tests or old implementations. Despite extensive optimizations, the bottleneck persisted until the decision was made to automate PR merging, eliminating the bottleneck from the process.

MorphX encapsulates the Cloud Code SDK, enhancing its capabilities to read files, analyze issues, and facilitate workflows. The core functionality, including the builder, deterministic workflows, and validations, mirrors the iterative human-AI collaboration process. Leveraging Cloud Code and Abstract Syntax Tree (AST) for deterministic operations, MorphX embodies the principles of continuous code review and feedback loops.

Breaking down the monorepo was a strategic initiative that required convincing the leadership of its value. While the company managed various repositories for micro frontends, the main monolith presented a significant challenge. The decision to address the monolith issue had been a long-term goal for the company, representing a critical step in enhancing efficiency and reducing delays in development.

They don't need to review the old implementation. They just need to review a few places. And we've worked on that for a long time. But eventually, it was the bottleneck, and that is when we decided that we need to crack the way to allow it to automatically merge the PRs, to remove that bottleneck exactly from the process.

Awesome. Thank you. And I've had a question that is could you please share a little bit about what's inside MorphX? Yeah, so we're actually wrapping Cloud Code SDK. We started before we even had Cloud Code but Cloud Code really gave us good results because it's very independent, right? It can read files. It can understand problems. And most of the magic that happens in MorphX, we actually extend here like the builder, deterministic workflows, creating the validations, and looping back around. We wanted to imitate the way that we work in our day to day because we work you get the code from the AI, you review it, you give feedback, and you do the same over and over again. I think the core principles and the main things were here, but it is using Cloud Code and we are using AST for all of the deterministic operations. AST is like a code model library for the ones who don't know it. Yeah. Great, thank you.

It sounds like it was quite a comprehensive process of figuring that out and lots of iterations, I'm sure. One question is how did you convince your leadership that splitting up your monorepo was worth it? Did you have any data that you presented in order to initiate this process? Yeah, well, that's like, it's an interesting question. So breaking our huge monolith, in addition to the monolith that we have, we do have like a lot of other repositories for micro frontends and other monorepos and more. But the old main monolith was like a goal that we had before I even joined the company. It was like a golden initiative for the company to handle it because it is delaying the work. It reduces the velocity of development. And it was something that was targeted to be handed a long time ago. And each time that someone tried to tackle it, they reached other things that prevented it from happening. And I can say that the root of it was the fact that, as I said, the application is complex and a lot of different components there react to each other. And it caused us to be in a situation that in order for one team to extract their code from the monolith, they had to wait for another team to extract their code in order for all of their data and components and everything to be out of the monolith and to be able to use it. So that was like the main reason. And it was like something that was measured a long time ago, even again before I even joined the company. And it was like fun to us that we didn't have that challenge of proving it. But it was a lot of work done by other people before we joined it. Great to know.

It sounds like, yeah, it's a whole team effort, right? Another quite a juicy question. Did the auto-merge you implemented without human review result in any significant bugs or incidents? So actually before the auto-merge we had one incident, which was funny because in the human escalation a human decided to fix a bug and then there was the incident. But after the auto-merge we added very strict review guidelines. We review in two ways. One MorphX reviews with AI as a judge, like we did for the validations. We reviewed the entire PR. In addition to that, we review with deterministic operations like ASTs. We check, for example, that all the symbols in the old file are the same as the symbols in the new file. And a lot of those checks. So it was a lot of work. We also did dry runs. We sent messages to ourselves. This PR is going to be merged. Then we review it on our own, getting confidence knowing, okay, no incidents will happen, no incidents will happen. And eventually when we unleashed MorphX, no incidents really happened because we did the dry runs and we checked everything that it really works with, more and more review guidelines. And that way we were able to really reduce the risk to a minimum. Okay, awesome. That sounds really cool. And it's funny that you mentioned that it was a human in the loop that did cause an incident with that one.

So another question is, did MorphX do any complex refactoring or was the idea to, sorry it's disappeared, to simply scope and code to code splitting and the TypeScript migration? So when we took that challenge, we said that we are not here to fix the codebase and we are here to break the monolith. And if we will take every side quest possible, we will never progress and we will never finish it. So we took that decision not to improve the code and not do refactoring other than the things that we mentioned. And MorphX, for example, it also adds a comment when it finds a bug, but it is not handling it. It is saying, I see something problematic here. I'm not touching it right now, but take it into your own consideration later on. We did find a few things like in the specific dependencies of our monolith that must be handled and MorphX cannot extract it on its own. And it flagged us and we saw in the board that the amount of items available for MorphX is reducing. We had the visibility for that and MorphX was like alerting us that it has blocked files, that it cannot extract. We found the relevant people, whether it's us or other teams, they untangled that specific file and then MorphX was free to continue. Awesome.

Thank you. One that's a little bit more about that process, the things that MorphX had to do. Somebody has asked, if you are extracting file by file, are you not preserving the wires between the Jenga pieces and not solving the real problem, which is simplifying complexity? We are preserving the lines because we extract it to separate packages. We do have package dependencies and we have an entire infrastructure of state management in Monday. We have this thing called Vulkan. We manage all state and everything inside of it. We are able to keep the dependencies between the different blocks and product while still extracting it to different code bases to make sure that we keep everything as clean as possible and as logically separated as we can. Awesome. We've got time for one more question. What did the AI usage costs look like? Running many prompts for a long time must be costly. Yes. Actually, I think that from day one, we added visibility for the cost. As Amit said, we started it before CloudCode was there. The root developer of the project, he created something like CloudCode by himself before we started. He was sending API requests to Gemini and getting responses. We just collected the costs. Now, what we are doing is that every migration, we are collecting the cost of it. I can share that I think for most of them, it's several dollars, like three, four dollars a migration. We added caps to it that if it is trying and iterating the validation process for more than a few times or if the cost reached a certain amount, it stops. Again, it alerts us. It's telling us that something is wrong because it should not be that costly. As I said, it's like four or five dollars per file to be extracted and compared to the time and effort that it will take a developer, then we're taking that price. The limits were crucial and also the code mods. Because we use a lot of deterministic operations, we reduce the cost dramatically. It was a game changer when we introduced this. We actually see the prices go down. This was also awesome for us. Amazing. Thank you so much. Thank you. Thank you.