Great Migrations: Upgrading a Component Library at Scale

All right. So, thank you for joining me today for this talk about great migrations, upgrading this Component Library thingy. And what I'm going to tell you or the story that I'm going to be sharing is something that works for us, for our organization, and it doesn't necessarily directly apply to your organization, but hopefully you can take some inspiration or some parts of it to apply within your own company or organization.

And what it says, Component Library, here at the title, it's basically about the entire environment that we have because everything that we have is built in Vue and Nuxt. So we'll be upgrading and migrating everything basically during this talk, if you bear with me. So let's get started.

So my name is Johan. As you can see, I'm a front-end developer. I work for Yumbo. I'll explain a bit later about what that is. You can find me on Twitter, you can find me on Mastodon or on my personal webpage. If you have any questions related to this topic or any other topic, feel free to reach out to me after this presentation or just chat with me regarding these topics.

Now, this whole presentation is sort of inspired by BBC's series about nature's great events, and in this particular case, the great migration episode. You can see here a picture of a young David Attenborough wrangling something that resembles a lizard or something. I was watching these series when I was younger, usually with my dad, so I have very fond memories of this series, and that's why I think it has a nice analogy to what we are doing in software development.

We will be seeing how the pros do it in nature, migrating, and what we can learn from it in our day-to-day job. So, I picked a couple of migrations that stand out for various reasons, and let's take a look at them. I think the most famous one is of course the wildebeest migration. I've written some things down about it because I don't know all of the details by heart, but this is the biggest migration there is. It's actually a super-herd made of wildebeests, zebras, and gazelles, and they are with a million or more than a million beasts migrating. They sort of take a journey that's 1800 miles or 2900 kilometers and it's a round-trip from Tanzania to Kenya and back. And while they are doing this, with the massive scale that they have, they actually play a role in forming or adjusting the landscape. So that's how big this is. And they have to traverse crocodile-infested rivers. They have to overcome droughts. They're preyed upon by lions, and what they're actually doing is they rear calves while they are migrating. If you take a look at the analogy, they are actually pushing stuff to production while migrating. I think this is really an interesting and impressive feat.

And then we have the caribou or reindeer, whatever you want to call them, and they have the longest migration path that we have on land. So they cover up to 3100 miles or 5000 kilometers each year and they move from their rentering grounds to their calving grounds in spring. And then back to their wintering grounds during fall in sort of a circular pattern. So the timing and distance about those migrations can vary on certain factors such as the weather patterns, availability of food and water, and risk of predatories or predators. So this is the longest migration.

And then we have, this is peak collaboration, we have the fire ants in the forests of Brazil, the rainforests, and these can form up to bodies of more than 100,000 ants clinging together and they do this in order to traverse water. And this is an interesting way of moving about, right? So they are experts at collaborating and the ants at the bottom of this body, they need to cling on for dear life in order to avoid being submerged while the ones at the top, they run the risk of being eaten by birds and other predators. And then there is the final bit of the puzzle. This is the elusive pack of developers and they typically move about in scrub teams of various sizes and they have to deal with other dangers such as moving requirements, Friday deployments, and bug infestations, and get up outages. So what we will do is we will investigate patterns for optimizing migration patterns and see if there's anything we can learn from nature. Let's first start with a definition. So what do we mean with a migration? And for the sake of this talk, let's consider this. So it is a period because it doesn't happen instantly, it takes time and effort of moving. Well, the moving could be anything from ants to people or even code.

Large volumes because it's not about just one gazelle looking what's on the other side of the valley, what's going on there. It's about a group. Otherwise, it doesn't really make a lot of sense. And we go from point A to point B, because we have a job to do at point B, right? So this is the definition that we will be using.

And then there's the question, so why do we migrate? And if you look at the wildebeests, they have a very good reason. They follow the rate to lush feeding grounds. And if they wouldn't, the environment where they are living in wouldn't be able to sustain the herd or support the numbers. So they will die a horrible death. And the software analogy is we want to have support of our software of course. So if we take a look at the life of a wildebeest, it's not really a pretty life, but it's in constant movement. So let's see how we can apply this to us. And of course it's not actually about following lunch and coffee. It's about the ability to quickly push new features to production and have some stability in environment to release confidently.

Now this stability thing is sort of the opposite of migrating and there's a risk here just as the wildebeest face, those wildebeest would probably be better off if they would just stay in their feeding grounds. They don't have to cross those rivers. They have little risk if they would stay, but they have a reason to do this and that's all about resources. For them, it's food. And for us developers, it's the support of the platform, as I said. So combine that support with an improved developer experience and the ability to create new stability. That's a good reason to migrate. So that's why we're doing this.

So after this silly analogy, let's investigate. And what we will be doing is we're going to observe a migratory pattern in a relatively isolated environment, and that is the wild place of YUMBO. So if you're not from the Netherlands or Belgium, you will probably have never heard of this or us, but we are a grocery chain and we do a lot of online groceries. We're actually in the top of the Netherlands and Belgium. And we develop a lot of our software in-house. And where we are doing this is the YUMBO tech campus or JTC as we call it, and there are over 450 developers that are collaborating on building our digital products. And this is not the herd. These are the stewards of the herd, and they need to make sure that our herd of software or code gets to point B safely. So now that we've met the stewards of the herd, it's time to take a look at the herd and the landscape.

Now, we have among things a big e-commerce monolithic engine, and that's been around for over a decade, I think. And what we've been doing over time is we started to offload a bit of the responsibility of the e-commerce engine into more specific Vue.js and Nuxt applications. And we started to get involved with Vue at the beginning of the V2 release, so we're doing this for quite some time now. And we gradually added Nuxt to also mitigate or migrate a bit from the monolith to its specific domain or applications and groups.

And what we have here is a team setup where teams are individually responsible for their own domain. And if we zoom in a bit more, then we can take a look at what a team setup might look like. So, this could be a typical team setup. It might differ in complexity from team to team, but this is what you would normally expect. So, there is a Nuxt application at the center. That could also be a Vue application. And that depends on some microservices. That could be RESTful or GraphQL or Apollo. We don't really care because that's part of the team's responsibility. And then in terms of e-commerce, most of the Nuxt applications depend on a couple of standardized modules and plugins we developed, and they deal with sort of the generic stuff, such as analytics and tracking. And so, they just help out the Nuxt application with generic stuff. And then in terms of facilitating to the customer needs, if you serve the base, we also include some micro applications in order to fulfill the e-commerce process. And one example of this could be the Basket, which is an isolated application, which is just attached to the UI of that Nuxt application. And then lastly, we have a component library. And that just holds hundreds of components that you build or that you use in order to build UIs, which is I think a sensible approach. If you imagine this setup with different types of complexities spread between the company, then you sort of have a decent view of what our herd looks like. So this is the herd that we need to move, because this is all done in Vue 2 with Nuxt 2. So we need to upgrade. We need to migrate. And this is because, of course, last year Vue 3 and Nuxt 3 were announced, but also the end of life of Vue 2 was announced. So that's when we run out of resources, more or less. So this is the reason that we need to migrate. And now that we have a very good reason of doing this, we need sort of a general direction or a migration path. And with all of our knowledge combined, we started to sort of plot a migration combined we started to sort of plot a migration path based on the dependencies and the bottlenecks that we could identify. And in terms of the wildebeest analogy, bottlenecks could be considered the river crossings of rivers. We need to cross in order to move forward.

And one of the first and more important bottlenecks or river crossings we have to do are those micro frontends. So what we did in the past, what we're currently doing migrating away from this, is we have these micro frontends and they are basically view applications that we attach to the DOM. And they are using VueX in order to mutate and modify the internal workings of that micro application, like adding product to a basket, removing, or updating the quantity. And that's something we need to get rid of, because we don't want to support VueX in Vue 3 environments anymore. So what in this case, the team who was responsible for the basket is doing, is they are actually removing the whole VueX API. And of course, having a VueX API for public operations can be considered somewhat of a risky idea, but it worked for us. But they are moving this to more of a tech agnostic way to interact with the basket. Internally, they are moving to Pina or to Apollo Clients, if I'm not mistaken. But basically they're removing the VueX dependency, which allows everybody else to upgrade as well, because as long as they have VueX, everybody is dependent on this. So we are already crossing the stream at the moment.

And then we have, just like this Auxbacker is supporting. This is a water buffalo, actually, not a wildebeest. But just like this Auxbacker plays a supporting role on our NUXT applications, we have modules. And those modules, as I said, they deal with stuff like gathering analytics or making sure that the SEO is up to spec. And what we did here is, because not everybody is depending on those, we have gathered the teams who are using these modules and plugins and each individual team sat together, and they either rewritten the module and plugin to be compatible with NUXT 3, in this case, or writing wrappers, or just rewriting it entirely so that they can all move forward. So again, we're crossing another river here.

And then finally, this is sort of a challenge because this caterpillar represents Vue 2, and it needs to morph in a beautiful Vue 3-compliant butterfly, the issue being that we need to keep the caterpillar alive until everybody else has been able to migrate. So there's a challenge here, and we're going to take a look at the component library more specifically.

So our component library. This is something that we built a couple of years ago, and it contains hundreds of components. We built it in Vue, of course. We use Storybook for documentation. So just a sensible approach. And the way that we are maintaining and collaborating on our component library is what we call a distributed approach. This means that every developer who has a stake in a component library is able to provide a contribution or to collaborate on adding new features, doing maintenance, and just keeping it in a really good shape. And I'll discuss these four topics further on.

So let's first take a look at this investment. So I think this is the most important factor or decision that we made, and that is that all users can actually modify the library. And this is important because I think that this creates an investment in the library, and that makes sure that everybody cares about the library. Sort of a nurturing, if you look at this picture, making sure that everything is in tip-top shape and is responsibility. And I think this is key for all of the other things that we do. And then we have like it sometimes happens that stewardship for some reason leave the herd. And if you do this, then if you do this with a small team, I have to say, then there is a risk of having a lot of knowledge leaving your herd or your company or organization. And again, with having this distributed approach, we also have distributed the knowledge about our component library, which means that if somebody leaves for whatever reason, that the knowledge drain isn't as big and that the risk is mitigated actually in between all of those other contributors. And it also creates sort of an encouragement of everybody to keep sharing their knowledge as well, which I think is also key in making sure that we keep everything up to date and have everybody on the same page as well. And then just as these wildebeests are crossing the stream, what they do with their body is somebody moves forward, and that that body then creates space for another wildebeest to move into that gap and to move forward as well. So they are blocking the stream to help others get across. And I don't want to focus on the blocking part here, but what happens here is that they create space for other teams to move forward or other wildebeests to move forward. And it's not much different with our distributed model. So each team here contributes and moves forward which creates space for other teams to move forward and accelerate as well. I think this is also a key benefit of having this approach. And once we have crossed that river, our whole herd can move forward. And so we wait until a large body of code or animals has moved across before we cross the next challenge or face the next problem. And in addition, any improvement that we do is immediately shared across the entire organization which makes everybody, even BOs happy, everybody benefits. So once you cross that river, everybody can move forward. We could adjust more of the component library from Vue 2 into Vue 3 because we needed to also support Vue 2.

And what we did here is we used Vue Demi in order to bring our Vue 2 application as close to Vue 3 as possible. And then we started to place a new package that is Vue 3 compatible next to the Vue 2 package so that we can support both. Now, if we look at where we are right now, we're not there yet, but we've already cleared a lot of crocodile-infested rivers in this case. What we're doing or what we learned is that we are going to start new projects if they have no dependencies on legacy software in Vue 3 and NUXT 3 already to make sure that those are already prepared for our next step.

We're also making sure that if we have any APIs that are dependent on a certain tech stack, such as the VueX case, in the case of the basket, we're now making sure to make that a more tech-agnostic approach so that we have the APIs ready, but everything that is more tech-related is done inside of the application, so it doesn't have any external dependencies anymore. We're currently in the process of upgrading those micro-applications as well as the component library, so we're well underway. This means that our herd has found green pastures again, overcoming the challenges as a team. We're not quite there yet, though, but we're closing the gap.

I think it's good to share these takeaways that we have, that we have learned during our journey. There's this common direction. It is really important to know where you want to go, and also what you need to do in order to get there. So this really helped us in plotting our entire path, and those increments, they're basically making sure that we have the deliverables along the way, and they add value once they are released or complete. You can think about those plugins that are already NUX 3 capable, which make sure that we can already start building NUX 3 projects for new scaffold-up projects. And making sure that nobody gets left behind basically means that we are supporting everything that is still dependent on Vue 2 until our last team or application is ready to make the jump to Vue 3. And this is like, this is why we are doing this with the component library, to support both of them, because it's just, in the end, everything that we deliver, we do this for the customer. And the customer actually doesn't really care if we are using Vue 2 or Vue 3, he just wants to buy his groceries. So we are making sure that our teams are able to deliver those features. And this distributed approach that I explained a bit about is that makes sure that we can continue to deliver features to production. This is like the rearing of the cows in the wilderness herd while working on our migration. So we're migrating and delivering at the same time. And I think that's really important also to make sure that you're not blocking any features while you're doing this because it's such a big endeavor. I'd like to close with an inspirational quote by Sir David Attenborough himself. I hope that you've also taken a bit of inspiration from this talk and that you can apply some of our learnings in your own migration path. And of course, this quote is not necessarily about the natural world. It's about front-end development as well, which is also a great source of excitement. And what also was a great source of excitement for me was giving this talk to you and being able to share our story. As I said before, if you want to reach out to me, feel free to contact me via any means possible and I will be happy to talk to you.