Compilers, User Interfaces & the Rest of Us

Okay, good morning, everyone. It is great to be here. First time in JS Nation US. Loving it so far. And yeah, let's talk about compilers, user interfaces, and the rest of us. That is me. I work for Medaglia. I'm a GDE in Web Technologies, and you can find me pretty much everywhere as Wide Accommodator.

Before we start, I'd like to share a little bit of a preamble or a story behind this talk and behind me and compilers. That's a story that started 10 years ago with friends and printers, and yeah, you got that right. So I would do a raise your hand thing, but I cannot see you, but there's this thing called ZPL, which stands for Zebra Programming Language. And that's a language that basically is used to program things for the Zebra labelers. And that's somewhat what the code looks like. So it's a bunch of instructions where you're basically telling what needs to be printed and where, positioning and stuff.

And as you can probably imagine, it's really, really hard to write that. And then I had a really, really good friend of mine, Marcelo, working with it at the time. And he was tired of that. So he created what you see on the side of the screen, which is basically a language that was inspired by Haskell and other functional languages. It was declarative, modular, immutable, and all of that. And it would compile that into ZPL. And you're probably feeling right now I was really, really mind blown by that because I was like, how can someone use a compiler to turn that thing into that other thing? That was really, really amazing.

So I started flirting a bit with the world of compilers myself. Besides Frontend, at the time I was working with Swift and Objective-C. Swift is open source, and I tried to write a proposal to add features to it. Learning about DSLs, ASTs, and APIs changed my perspective. In 2017, reading Tondale's piece 'Compilers are the New Frameworks' made me realize the importance of understanding compilers in web development's future.

In 2017, Dominic Ganaway predicted JS frameworks evolving into JS compilers. Fast forward to 2022, Ryan Corneado emphasized the significance of compilers in reactive programming. The broad application of compilers and static analysis in tools like formatters, bundlers, and type checkers became apparent. Despite the vast possibilities and use cases of compilers, I will focus on just three of them, starting with React and its compiler's data flow understanding and mutability analysis.

The React compiler's approach to understanding data flow and mutability involves advanced techniques like Elias analysis and static single assignment form. Elias analysis helps identify variables referring to the same data, optimizing code generation. The React compiler playground showcases different steps, including addressing Elias analysis for emitting and grouping mutations efficiently. Static single assignment form, another vital technique, is commonly known as SSA form in compiler theory, enhancing optimization and efficiency in code generation.

Also in 2017, Dominic Ganaway, the author of Inferno and more recently Ripple and also a former member of React and SvelteCore teams, expressed his thoughts on the evolution of JS frameworks into JS compilers. Moving to 2022, Ryan Corneado highlighted the significance of compilers in reactive programming. Compilers and static analysis play a crucial role in various tools like formatters, bundlers, and type checkers, impacting different aspects of web development.

The React compiler's documentation delves into the understanding of data flow and mutability in code, emphasizing the incorporation of advanced techniques like Elias analysis and SSA form. Elias analysis aids in optimizing code generation by identifying variables referring to the same data. The React compiler playground demonstrates the application of Elias analysis in emitting and grouping mutations effectively. SSA form, another essential technique, enhances optimization and efficiency in code generation, simplifying variable definitions.

Type inference and the algorithm W are key components inspired by the Hindley-Milner type system, prevalent in functional languages like Haskell and OCaml. The React compiler integrates type inference techniques influenced by the Hindley-Milner type system, showcasing a deep connection to classical type systems. Beyond React, Svelte is renowned for its compile-time reactivity, offering a unique perspective on compiler functionality. Exploring the phases behind the Svelte compiler provides insights into its compile-time processes and reactivity mechanisms.

And the reason why I find it interesting is because there's actually a lot of CS behind the React compiler. For example, there's this thing called Elias analysis, and it's one of the many techniques in the world of programming languages theory and compiler, which is basically a set of techniques that try to determine if variables are talking about the same underlying data. This is really helpful because it helps your compiler generate efficient and optimized code. Checking the React compiler playground reveals different steps addressing Elias analysis, emitting effects, collapsing effects into per-value mutable changes, and grouping mutating places into reactive scopes. Another vital technique is static single assignment form, also known as SSA form, where every variable has a single definition, simplifying code maintenance and enabling powerful optimization techniques.

Type inference and the algorithm W, part of the Hindley-Milner type system popular in functional languages like Haskell and OCaml, are key components in the React compiler. The React compiler integrates type inference techniques inspired by the Hindley-Milner type system, enhancing its classical type system connection. Svelte, known for compile-time reactivity, offers unique insights into compiler functionality. Exploring the Svelte compiler phases provides understanding of its compile-time processes and reactivity mechanisms.

Svelte, a framework distinct from React, introduces features like Invalidate in versions 3 and 4 and Template effect and Delegate in Svelte 5, enhancing reactive state management and dynamic expression handling. Reflecting on older advancements, Opalanguage, introduced around 2011, enabled writing programs compiling to Node.js and JavaScript for client-server communication automation. The concept of co-locating client and server code in a single file emerged as an essential discussion around code extraction according to Misko, highlighting the evolving landscape of DX for Frameworks.

It's part of this thing called Hindley-Milner type system, which is one of the most classical type systems, and it's very popular in functional languages like Haskell and OCaml. And guess what also inspired the React compiler. So actually, if you check the source of the React compiler, you're going to see this type inference pass that's heavily inspired by the equation model, the type equation model defined by Hindley-Milner type systems. But this is JS Nation, so let's talk about other things that are not React, like Svelte. Svelte has been long known for its what's called compile-time reactivity.

And again, I'm not a Svelte developer on a daily basis, and I'm not going to talk a lot about Svelte, but you can definitely check out the source of this Svelte compiler and the different phases behind the compiler. And experiment with the playground. It doesn't matter which version of Svelte actually. You can check version 3 and 4 before Reunions, for example, and you're going to see this thing called Invalidate, and this is basically used to mark variables as dirty and schedule an update for that component. And the newest version of Svelte, too, like Svelte 5, you're going to see things like Template effect and Delegate, and those are used for handling reactive state, event listeners, and handling dynamic expressions and templates, and a whole bunch of other stuff.

But again, I feel like I'm talking about a lot of things that you already know, so let's go a little bit more back in time. Around 2011, we had this thing called Opalanguage, and it didn't get a lot of traction, a lot of people never heard about it. But it allowed you to write entire programs in this language, and it would compile to Node.js on the server and regular JavaScript on the client, and the compiler would automate all the communication between both of them. So you would have those signatures like client and server for each part. And this is really, really interesting, because it brings the whole discussion around code extraction. As Misko was saying, this was like two or three years ago, he was basically talking how co-locating client and server code in a single file is the next battleground for DX for Frameworks.

And of course, he was talking about SolidStorm, QUIC, and other meta-frameworks. But this whole code extraction challenge is actually not that new. So with Opalanguage, we were trying to solve the same problems that we're trying to solve today. So we have use server, use client, and even for infrastructure, like use cache. And on SolidStorm too, so that's not only a Next.js thing, but we got use server and SolidStorm, and even on mobile, so Lynx and React Lynx, the new competitors to React Native by Bytance, they also have compiler programs to denotate where code runs, like on the background for that or on the main UI thread of your phone.

But it doesn't stop there. Compilers are also in our browsers. So since 2010, V8, for example, has had this thing called Crankshaft, which was an optimization compiler that ran inside V8. And throughout the years, they added many, many more optimization steps to their multi-tier pipeline, like Ignition, Sparkplug, Liftoff, Maglev, and many different others. But that's not just a V8 thing. In VS Core, for example, they also have a multi-tier pipeline with Faster Than Light JIT, or Just In Time, or B-Free JIT 2. And I don't know about you, but all of this really, really blows my mind.

But I understand how it's easy to sometimes think, okay, I am not Apple, I'm not Meta, I'm not Google, I don't need to build complex compiler infrastructure at my daily basis. So that's why I have the second half of this talk, which is basically compilers for the rest of us. And I want to kind of speed around different scenarios in my recent life as a front-end developer slash JavaScript developer, where compilers and static analysis helped me modernize, generate, or even reason slash understand hundreds, thousands, or even millions of lines of code.

So the first example is about modernizing. And this is probably the one that has been the common denominator throughout the last couple of years for me. I am a React developer, and if you're a React developer, you probably remember, for example, when we got React 16 and it had some breaking changes, like prop types. Prop types was moved from the main React package into a separate package. But more recently, with React 19, we also had some breaking changes, and if you wanted to use React 19, you had some migration steps. And in both cases, what has been a common denominator are codemods.

And most of the React codemods, they operate on top of this tool by method called jsCodeShift. And jsCodeShift basically allows you to write AST transformers that will run on top of your code. So a Hello World example with jsCodeShift would look like this. Basically it takes a file, it reads the source of the file, and then it gives you some utilities. It's a Hello World, so all I'm doing here is finding variables that are called full and renaming them to var. And you're going to find lots of other public, huge case studies of using codemods, like Netlify and call.com, they have published case studies under the site of codemod.com.

Also, open source case studies, node.js, they've been recently pushing for migration steps using codemods, and also Nuxt, PMPM, and Ember. Not to mention that more recently we got the codemod registry, where you can find a lot of open source, they have more than 600 available codemods addressing all sorts of technologies you can think of. And another thing is, three, four years ago when I was writing codemods, I used a lot of this tool called AST Explorer that basically helps you visualize what your JSX or TSX code turns into. But nowadays codemods, they also got this codemods studio that you can even use AI agents to help you write AST transformers.

If you want to hear more about this large scale migration, I have another talk where I talk more about it, but TLDR here is with static analysis in my company we could have a gradual rollout of two entirely different versions of the same codebase, but that was feature-flag protected, and each version had different versions of dependencies and entire different strategies used for bundling. Because we were automating a lot using compilers.

Another case is, when you, for some reason, you need to build types from JSON samples. Let's say you're not working with GraphQL or TRPC or you don't have a swagger contract, you can generate TypeScript types on top of, there's a bunch of tools out there that will help you build TypeScript definitions from JSON types. We ended up using QuickType, which is great, but one thing that always happens with this kind of tools is that there's only as far as you can go with static analysis, and sometimes the output of these tools is not going to be exactly what you need in your frontend application, for example.

So to address that, we use tsmorph, which is a wrapper around the TypeScript compiler API, and a quick example of things you can do with tsmorph when it comes to refactoring is, let's say you have an interface that's not following the code standards of your team or of your company, and you can map that. The way you can do this kind of transformation is using tsmorph's AST utilities. This is one example that you can do in less than 20 lines of AST manipulation, and that's all using, for example, here, the utilities around interfaces, but that would be the same for type alliances, type unions, and all sorts of different things.

You can also find online a bunch of interesting case studies of running large-scale migrations on top of tsmorph. Even more, one recent case I do have is one of a really, really good friend of mine. He watched an early version of the same talk, and he actually solved the problem in his company using tsmorph to generate typings based on backend-frontend contracts to avoid backend changes breaking the frontend of his applications. So unlimited possibilities here.

Last case, understanding millions of lines of code. For this one, I'd like, actually, to talk about this thing called Hiram's Law. The term was coined by a Google engineer called Hiram Wright, and it says, when you depend on an API, you also depend on its implementation details, which is funny, because that's the opposite purpose of an API, right? But an interesting way to understand that is with this XKCD piece.

It's like a release notes of something, so it starts, so we fixed this. The CPU no longer overheats when you hold down spacebar, and then there is a user who complains, okay, this update broke my workflow. My control key is hard to reach, so I hold spacebar instead, and I configured Emacs to interpret a rapid temperature rise as control key. And they move on and say, look, my setup works for me, so just add an option to re-enable this spacebar heating. And besides this being actually funny, the model here is every change breaks someone else's workflow. And you can see the side effects of Hiram's Law in React, too.

For example, when you have a GitHub issue asking, if you're not a React developer, one of the internals of React is called Syncrete internals do not use or yield befired. And then there's a GitHub issue asking if it's safe for that to be used. So again, besides this being funny, this actually is a problem because when they had React 19, they had to address this, and the migration guy is saying, hey, you might be using some libraries that actually are using those internals that are not meant to be used. So that's an example with React.

With me, Hiram's Law comes in a different flavor. So I mentioned I work for Medalli. We are a CX company. And I work mostly with surveys, and we have a classic quiz week kind of capability where you can drag and drop elements to build surveys. And one of the things we allow people to do is actually write JavaScript and CSS or even HTML code to customize the look and feel of their surveys. When you allow people to write arbitrary code, they can get really creative.

One thing we started doing was running static analysis to solve some problems that were coming from people's creativity. One example is jQuery. And you're probably familiar with the performance impacts of using jQuery nowadays, and also the process impacts if you're using legacy versions of jQuery that have open CVs and all that. What a lot of people don't know is how popular jQuery still is. So as of August 22, it's still used by 77% of the top 10 most popular websites.

So we started doing... We started running Babel on top of JavaScript snippets created by our users to understand how they were using jQuery so that we could decommission that. Because we're very, very used to Babel as part of our build pipeline. But Babel actually ships with a powerful parser and traverser that you can build many utilities to understand your code. So we have, like, hundreds and hundreds of lines of Babel code trying to understand the many different possibilities of using jQuery. Like simple... Using simple built-in APIs or extending it with custom APIs, or using open source jQuery plugins and whatnot.

And on top of that, we could build this sort of dashboard where, actually, we can see how people are using jQuery and even help them migrate depending on how they're using it. And because this is a compiler talk, this is one tiny POC I built of actually compiling jQuery to native JavaScript APIs. So this is compiling an AJAX call to a native fetch API in the browser. So all things considered, static analysis here enabled metrics around user-generated content. So not our code, but someone else's code running in our platform, based on plugin composition. Because you can write plugin to understand any kind of thing. Like jQuery, Angular, or specific functions, and all of that.

And the actual data is a single source of truth. And because of that, consistency is guaranteed across all the API endpoints and the React application, because we're actually generating type definitions based on the output of the static analyzers. And overall, we get very, very rapid integration with instantly typed APIs, and enhanced AJAX whenever we need to implement any new metrics. Because those are all composable plugins, you can use anything to analyze code. We can use, for example, CheerIO to understand HTML code, or even other tools like ASTGraph, which is a very, very brilliant tool built on top of Rust, to also understand how people are into query code.

We can use, for example, CheerIO to understand HTML code, or even other tools like ASTGraph, which is a very, very brilliant tool built on top of Rust, to also understand how people are into query code.

Whew, that was a lot. So finally, some closing thoughts. First thing is, I feel like there's this whole stigma around compilers, that they are this hard thing that we take in CSGrad, and they're just for systems programming. But that's not true. They're deeply embedded in how we build, optimize, and reason about user interfaces.

It doesn't matter if we're talking about just transforming JSX, or full-on UI compilers like Marco, and similar stuff. We depend on them more than we realize already. And there's this old CS saying that goes like, everything is either a compiler, a database, or a combination of both. And you can actually find really good examples of people abstracting things that we usually don't think as compilers, but actually are compilers.

So if lots of things are compilers, I think there's a lot of room for us to be really creative about it. And it doesn't matter if that's, for example, what Middenland does, where they use compilers to inject instrumentation that will capture information about your code in runtime, and then feed that into an LLM that will tell you how to optimize the react performance of your code. Or if that's what Astro does, where the Astro compiler actually splits your code based on directives, and allows you to solve hydration problems in many different fashions. Or what Mitosis by Builder.io does, which is also very complex, because they transform JSX components into components for Angular, React, Quake, Vue, Veltas, Solid, and whatnot. And these three last examples, they might have sounded too complex, and they might also fall into this umbrella of you're not Facebook, why would you bother? But again, we can find examples we can relate to. Like this tool by Rangle, they basically run a static analysis to understand how their design system is being used by their team.

So I really think that understanding compilers can make us better JavaScript developers, or front-end developers. So again, we don't need to write one, but just having an overall idea of how they work, they can help us better evaluate frameworks, debug edge cases, and even influence our architectural decisions in our projects. And I found this the other day on Reddit, and it's a post that goes like this. There is really no better education on complex data structures and data processing and data types itself than compiler design. With the possible exception of network programming, I'd say all software development can be seen as a sort of subset of compiler design, and you won't have the complete picture if you don't understand it. And, for me, that's really interesting because we can abstract the compiler as a pipeline of sometimes even relative simple transformations from one data to the next form of data.

So to wrap it up, the title of the talk is Compilers, User Interfaces, and the Rest of Us. But probably the main takeaway is actually compilers, aesthetic analysis, and all of that, they are for the rest of us, and we can start using them today. The slides for this session are available in this QR code and other sessions I have on internals of tools and browsers and whatnot. And I think we still have time for some questions, so thanks a lot for having me today. Yeah. I would love to know. So as someone like myself who does not have a background in computer science, are there any resources you would recommend for developers who are looking to learn more about compilers or maybe start their compiler deep dive? Yeah. So there's a lot of free online courses from MIT, Stanford, and other universities out there, which are basically subsets of their CS courses you would take. Also reading how those frameworks are using, for example, Sakya, who used to be on the React compiler team at Meta, he writes a lot on how React compiler uses those concepts. And the same thing about Marco, for example, Ryan just mentioned how he hosts lots of livestreams. So in some of these livestreams, they actually compile the talk about compiling that. So yeah, reading the classics, also some books like Structure and Interpretation of Computer Programs and all of that and seeing how this is all applied in front end tools. Nice. I love it. And I also love it. Anybody who thinks that there's not a lot going on on the front end is kidding themselves.

And I also love it. Anybody who thinks that there's not a lot going on on the front end is kidding themselves. There's so much. It's so deep. Right? Yeah. There's so much going on. Yeah. So if folks have more questions, please put them in the Slido. Again, you can scan the QR code and join the Q&A session with 1117 as the code. So yeah. And I'm curious, you know, there's a lot, a lot of tools. And like you said, you had a slide up there with like 80 million logos. I'm like, I wish I had four hours to talk about all of this, which if folks do want to keep the conversation going, Mateusz will be in a speaker Q&A room after this talk.

But would love to know, you know, do you see all of these tools as sort of equally powerful in the ecosystem? Are there, are there forerunner tools that you think are going to become more and more important, the more static analysis and compilation kind of becomes part of front end workflows? Yeah. I think, I don't think they're competing in a sense, for example, Zout and Solid are two really good examples of frame libraries slash frameworks employing reactivity. And they have very, very different approaches to it. For example, a lot of Solids reactivity magic lies in runtime reactivity, and they use compilers and static analysis for other stuff that are not reactivity per se, whereas Zout uses a lot for reactivity. So I think that the react compiler is not, is using it a lot for memorization and optimizing that. So I think different tools will find different creative use cases. And what Midden does, for example, that is basically using compiler to generate insights and then give it to AI and et cetera. So I think that it's not like they're competing, it's just that they are finding different levels of interesting use cases. Gotcha. And yeah, I think we have another question also related to many of the, or one of the many components of all of this, which is WOSM, the web assembly.

What do you see as sort of the future of that? Will there be more widespread adoption with this whole push around front end compilation? I think that we'll have more and more widespread adoption, but not necessarily because of compilers per se and this whole movement around compilers. I think that it will have, because it's growing more and more like and the few missing pieces we needed to build entire apps in WOSM are now getting solved. Like the web as a platform is evolving. So for that, I think that it will lead to a more widespread adoption of WOSM. And I think that also we may end up having some creative cases around compilers and WOSM, but I'm not sure if there would be a direct correlation in that sense. But yeah, hard to tell to be honest. Gotcha, gotcha.

And I guess to maybe wrap up our conversation on, you know, all the various tools and everything that's out there, the ocean of them as Owen asked, how do you know which one is best for your project? It's the classic, it depends. So like in the free, actually, that's why I try to speed run through different use cases because in the first one, for example, I just needed to migrate code bases. So that was a clear call for code mods and JS code shift. In the other one, I started using Babel to understand code. But as I was exploring analysis with Babel, that's when I heard about ASC grab, for example. And I started experimenting with ASC grab for some of my plugins. So try to understand what each of these tools do and what are you trying to achieve and work on proofs of concepts and all that too and check open source out there. Awesome, awesome.

All right, well, we have a whole stream of questions coming in, but we're already a little bit over time. So we're going to give Mateus another huge round of applause. Woohoo! Woohoo! Woohoo!