The Forest for the (Abstract Syntax) Trees

Hi, my name's Matt. I work at Shopify on the Hydrogen team. Hydrogen is an opinionated React framework and SDK for building fast and dynamic Shopify-powered custom storefronts. We are currently under private beta, but look out for a developer preview in the coming weeks. I'm actually not going to touch on the details about how Hydrogen works specifically, but you can watch this demo with Toby, the Shopify CEO, from our Unite conference earlier this year.

So like any new framework or product, the time it takes to get up and running is crucial to the developer experience. With so many competing frameworks, all with a much longer tenure, we knew this would be an important consideration for anyone choosing to adopt Hydrogen. Create X scripts, such as Create React App or Create Next App, are pretty standard for project bootstrapping. But still, many of these are currently limited to cloning a template directory. This is not ideal as it doesn't provide much in the way of configurations, and every new configuration comes with the maintenance overhead of a completely new template, and likely, a lot of duplicate code. The chances that any defaults within these templates will be configured exactly to your preferences is highly unlikely. You might want to add Tailwind or style components or Storybook, or you might install your own internal library. In these cases, you're left on your own to stitch together your final configuration for multiple examples or duplicating previous projects, all before writing a single line of actual code.

We wanted to give developers a better place to start. A command line interface that is more sophisticated, extensible and walks with developers as they build their custom commerce experience. A CLI specific to hydrogen development that will do more than bootstrapping projects, but that you would call out to ongoing as you develop your project and capable of an infinite number of configurations without the additional overhead. Here's a high level mental model of how this works. First, we gather information about the project either by arguments in the CLI, a configuration file or prompts for input. Next, we parse the existing project if there is one or start from a base hydrogen project and turn it into a JSON tree like representation. This is what is referred to as an abstract syntax tree or AST. We then apply transformation functions to move nodes of the JSON around, effectively manipulating the code based on the inputs we gathered in the first stage. And finally, we generate new code from the JSON and write that back to disk. These stages, parsing, transformation and generation are common to most compilers. And we're just using Babbel under the hood to do most of the heavy lifting. For those not familiar, Babbel is a tool that is most well known for being one of the first ways we began modifying code written using feature JavaScript features to a more browser compatible syntax. Each stage of the compiler is dealing with a different concern. We don't use loads of template files such as EJS or Handlebar Templates, and we are operating only on the AST and the transform functions are input code agnostic. This is because it uses a common code pattern known as the visitor pattern that allows for many unique and unrelated operations on the AST object. A main function will start by walking or traversing the tree and match nodes with transformation functions. These functions only care about the specific nodes they operate on and can care less about the rest of the code outside of those points of interest.

Because each operation is isolated, they behave like the component primitive in React and we can group common operations into a shared package. This makes composing a series of common transforms trivial and the syntax is very approachable. There are also some great tools such as astexplorer.net or the Babel Playground that can help you write and visualize AST transforms.

Imagine being able to add a progressive web app support with one command, or for example, if you wrote a transformation that added internationalization with all the appropriate error handling and fallbacks in place. At a basic level, we might import a provider and hook, provide the hook some unique configuration based on your project, and wrap the other app in that provider, passing the result of the hook as a prop. Of course, we would also install the package and all the easy things, but being able to manipulate code in this way can lead to some pretty powerful and creative opportunities.

With Hydrogen, we want to be an opinionated framework when it matters most, but there are certain add-ons in your project that may not make sense for you right at the beginning. Should you decide later that the project requires it, we can provide a turnkey process data down the road and with all the best defaults out of the box. So what are we opinionated about? Well, we care about image performance, request caching, and using server-side rendering and React server components to provide non-blocking responses and eliminating client-side waterfalls. At Shopify, we are passionate about commerce and we want to give the merchant developers on our platform to building blocks to build highly performant storefronts using Hydrogen.

In that world, we are opinionated about the difficult but undifferentiated aspects of development and want to provide a developer experience that makes web development fun again. Think of this as us trying to raise the floor on the area as we can so that you can more easily raise the ceiling within the types of creative and custom commerce experiences you develop. Take React server components. Probably not something you've worked a great deal with yet and perhaps you're not sure about what a component should be client-only, server-only or shared and what primitives are available in each of those. Well, using the same parsing, transforming and generating process I've outlined earlier, we can again walk with you. We can inspect for common footguns while using React server components and either guide you to the correct usage or just change the code for you. We can provide migrations baked directly in the CLI that will automatically look at your project for breaking changes in new versions of Hydrogen and optionally just apply a migration automatically. We also audit your project for accessibility and security, conformance of web vitals and any framework violations. We also use the CLI internally to power our integration tests, generating unique configurations of Hydrogen apps on the fly that we test in headless browsers for breakages. This protects ourselves from releasing new versions with unknown issues, and also has the added benefit of us dogfooding our own tools. So, this has been a quick dive under the hood of how we are leveraging compilers with everything from bootstrapping to enforcing best practices, to ongoing maintenance support, and even internal testing. With the Hydrogen CLI, each of these interactions culminates in a powerful and fun developer experience.

Thanks for watching my talk. And if you have any questions or want to shoot me a message, I'm not so big on social media, but you can find me as kardagram in most places.