Hard GraphQL Problems at Shopify

Rate this content
Bookmark

At Shopify scale, we solve some pretty hard problems. In this workshop, five different speakers will outline some of the challenges we’ve faced, and how we’ve overcome them.

Table of contents:
1 - The infamous "N+1" problem: Jonathan Baker - Let's talk about what it is, why it is a problem, and how Shopify handles it at scale across several GraphQL APIs.
2 - Contextualizing GraphQL APIs: Alex Ackerman - How and why we decided to use directives. I’ll share what directives are, which directives are available out of the box, and how to create custom directives.
3 - Faster GraphQL queries for mobile clients: Theo Ben Hassen - As your mobile app grows, so will your GraphQL queries. In this talk, I will go over diverse strategies to make your queries faster and more effective.
4 - Building tomorrow’s product today: Greg MacWilliam - How Shopify adopts future features in today’s code.
5 - Managing large APIs effectively: Rebecca Friedman - We have thousands of developers at Shopify. Let’s take a look at how we’re ensuring the quality and consistency of our GraphQL APIs with so many contributors.

This workshop has been presented at GraphQL Galaxy 2021, check out the latest edition of this Tech Conference.

FAQ

Shopify's versioning system allows for API changes to be made in a controlled manner, ensuring backward compatibility and smooth transitions for developers. It helps manage and schedule API changes, alerting developers about deprecations and necessary updates through version releases.

Shopify deprecates fields by marking them and gradually phasing them out. They provide warnings and updates through their API health dashboard and developer changelogs, giving developers time to adapt before the fields are removed in future API versions.

Shopify optimizes GraphQL query performance on mobile by implementing strategies like pagination, parallel querying, and careful management of query costs. They focus on minimizing data loads and optimizing data retrieval times to enhance user experiences on mobile devices.

Shopify ensures the accuracy of GraphQL queries by using a combination of version control, schema validation, and proactive monitoring of API usage. They manage breaking changes by providing versioned APIs, allowing developers to choose when to adopt new changes and adjust their applications accordingly.

The GraphQL Schema Bot at Shopify automates the detection of common GraphQL issues in schema changes. It helps maintain high-quality API design by flagging potential problems and inconsistencies, thereby reducing manual review workload and ensuring API design standards are met.

Shopify handles nullability by carefully considering whether fields should be nullable or non-nullable based on the likelihood of being able to always return a value. They use nullability to provide flexibility in their schema and to avoid breaking changes that might impact client applications.

Using directives in Shopify's GraphQL APIs allows for more flexible and dynamic query behavior, enabling context-specific customizations such as regional pricing or conditional data fetching. Directives help manage runtime execution and type validation behavior, enhancing API functionality and developer control.

Rebecca Friedman
Rebecca Friedman
Jonathan Baker
Jonathan Baker
Alex Ackerman
Alex Ackerman
Théo Ben Hassen
Théo Ben Hassen
 Greg MacWilliam
Greg MacWilliam
164 min
01 Dec, 2021

Comments

Sign in or register to post your comment.

Video Summary and Transcription

The Workshop covered various topics related to GraphQL at Shopify, including the Nplusone problem, batch loading with lazy resolvers, contextualizing GraphQL APIs with directives, optimizing GraphQL for mobile, managing large GraphQL APIs, and handling deprecated fields and nullability. The speakers shared insights and solutions to these challenges, such as using Active Record Preloading, GraphQL Batch library, and custom directives. They also discussed the importance of measuring performance, pagination design, and parallel querying for mobile optimization. Overall, the Workshop provided valuable information on improving GraphQL performance, API design, and versioning strategies.

1. Introduction to Workshop

Short description:

Thank you all for joining us for the workshop. Over the next three hours, five different speakers will discuss the problems they've tackled with GraphQL at Shopify. Each session will be 20 minutes long, followed by a short break and an hour-long Q&A session. Feel free to ask questions in the chat throughout the workshop.

So, once again, thank you to everyone who's been able to join us here. I truly appreciate your time. We all do, all our speakers, all of Shopify. Thank you very much for being there and joining us for the workshop.

So this is how the next three hours will look like. It will be a bit different from what we are used to, but I hope it's good different. So hopefully you can also share that feedback with us at the end of this, but we are going to have five different sessions by five different speakers. All of them are engineers here at Shopify. And all of these sessions will actually be talking about had problems that we've had to tackle with GraphQL here at Shopify. So we hopefully hope that that will shed some light and you're going to get some insight into how the teams were able to look at the problems and you know, the various methods they use to tackle the problem.

So each session or each talk will be 20 minutes long. And at the tail end of that, we are going to have a break, a short break, a 10 minute break, you know, and then after that we are going to have a one hour office hours, or a mail session, if you may wish. So if you do have questions, just keep on dropping them here on chat, we'll make sure we take note of them. And at the tail end of it during the last hour during the office hours, we are going answer your questions. But also if you have a very specific question to any of the sessions as they roll, they are pre recorded, but our speakers are here live, you can definitely get to ask your questions on chat, and they're going to take care of that. So I hope that is fine. Sounds fine and makes sense.

2. Introduction to Jonathan's Session

Short description:

Our first session is by Jonathan, a staff developer at Shopify, focusing on customer storefront applications. Jonathan is here to clarify any questions during the pre-recorded session. Let's get started!

So without much further ado, I think our very first session is by Jonathan. Jonathan is here. Jonathan, I don't know if you want to take just a moment say hi to you know the people who are joining us before we get to a session on? Hello, everyone. I'm Jonathan, as you will see in the talk I'm a staff developer here at Shopify, focusing on our customer storefront applications. So that's from a GraphQL standpoint, that's the storefront API, which is one of two of our public GraphQL APIs and welcome to be here.

Fantastic. Tell us one thing about you that you don't know probably let me make it much more closer home, what's your favorite or your current read or favorite blog or newsletter right now? Shopify's engineering blog. All right, thank you so much, Jonathan. So Jonathan is our first speaker, as I mentioned, all our sessions are pre-recorded but feel free to drop your questions as the session runs. Jonathan is actually here to just help clarify anything just in case we get to that. So let me see. I will share my screen here, share sound as well. All right, fantastic. Let me see. Give me a thumbs up, actually, if you're able to get the sound up here.

QnA

Watch more workshops on topic

Build with SvelteKit and GraphQL
GraphQL Galaxy 2021GraphQL Galaxy 2021
140 min
Build with SvelteKit and GraphQL
Top Content
Featured WorkshopFree
Scott Spence
Scott Spence
Have you ever thought about building something that doesn't require a lot of boilerplate with a tiny bundle size? In this workshop, Scott Spence will go from hello world to covering routing and using endpoints in SvelteKit. You'll set up a backend GraphQL API then use GraphQL queries with SvelteKit to display the GraphQL API data. You'll build a fast secure project that uses SvelteKit's features, then deploy it as a fully static site. This course is for the Svelte curious who haven't had extensive experience with SvelteKit and want a deeper understanding of how to use it in practical applications.

Table of contents:
- Kick-off and Svelte introduction
- Initialise frontend project
- Tour of the SvelteKit skeleton project
- Configure backend project
- Query Data with GraphQL
- Fetching data to the frontend with GraphQL
- Styling
- Svelte directives
- Routing in SvelteKit
- Endpoints in SvelteKit
- Deploying to Netlify
- Navigation
- Mutations in GraphCMS
- Sending GraphQL Mutations via SvelteKit
- Q&A
React at Scale with Nx
React Summit 2023React Summit 2023
145 min
React at Scale with Nx
Top Content
Featured WorkshopFree
Isaac Mann
Isaac Mann
We're going to be using Nx and some its plugins to accelerate the development of this app.
Some of the things you'll learn:- Generating a pristine Nx workspace- Generating frontend React apps and backend APIs inside your workspace, with pre-configured proxies- Creating shared libs for re-using code- Generating new routed components with all the routes pre-configured by Nx and ready to go- How to organize code in a monorepo- Easily move libs around your folder structure- Creating Storybook stories and e2e Cypress tests for your components
Table of contents: - Lab 1 - Generate an empty workspace- Lab 2 - Generate a React app- Lab 3 - Executors- Lab 3.1 - Migrations- Lab 4 - Generate a component lib- Lab 5 - Generate a utility lib- Lab 6 - Generate a route lib- Lab 7 - Add an Express API- Lab 8 - Displaying a full game in the routed game-detail component- Lab 9 - Generate a type lib that the API and frontend can share- Lab 10 - Generate Storybook stories for the shared ui component- Lab 11 - E2E test the shared component
Build Modern Applications Using GraphQL and Javascript
Node Congress 2024Node Congress 2024
152 min
Build Modern Applications Using GraphQL and Javascript
Featured Workshop
Emanuel Scirlet
Miguel Henriques
2 authors
Come and learn how you can supercharge your modern and secure applications using GraphQL and Javascript. In this workshop we will build a GraphQL API and we will demonstrate the benefits of the query language for APIs and what use cases that are fit for it. Basic Javascript knowledge required.
End-To-End Type Safety with React, GraphQL & Prisma
React Advanced Conference 2022React Advanced Conference 2022
95 min
End-To-End Type Safety with React, GraphQL & Prisma
Featured WorkshopFree
Sabin Adams
Sabin Adams
In this workshop, you will get a first-hand look at what end-to-end type safety is and why it is important. To accomplish this, you’ll be building a GraphQL API using modern, relevant tools which will be consumed by a React client.
Prerequisites: - Node.js installed on your machine (12.2.X / 14.X)- It is recommended (but not required) to use VS Code for the practical tasks- An IDE installed (VSCode recommended)- (Good to have)*A basic understanding of Node.js, React, and TypeScript
GraphQL for React Developers
GraphQL Galaxy 2022GraphQL Galaxy 2022
112 min
GraphQL for React Developers
Featured Workshop
Roy Derks
Roy Derks
There are many advantages to using GraphQL as a datasource for frontend development, compared to REST APIs. We developers in example need to write a lot of imperative code to retrieve data to display in our applications and handle state. With GraphQL you cannot only decrease the amount of code needed around data fetching and state-management you'll also get increased flexibility, better performance and most of all an improved developer experience. In this workshop you'll learn how GraphQL can improve your work as a frontend developer and how to handle GraphQL in your frontend React application.
Build a Headless WordPress App with Next.js and WPGraphQL
React Summit 2022React Summit 2022
173 min
Build a Headless WordPress App with Next.js and WPGraphQL
Top Content
WorkshopFree
Kellen Mace
Kellen Mace
In this workshop, you’ll learn how to build a Next.js app that uses Apollo Client to fetch data from a headless WordPress backend and use it to render the pages of your app. You’ll learn when you should consider a headless WordPress architecture, how to turn a WordPress backend into a GraphQL server, how to compose queries using the GraphiQL IDE, how to colocate GraphQL fragments with your components, and more.

Check out more articles and videos

We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career

From GraphQL Zero to GraphQL Hero with RedwoodJS
GraphQL Galaxy 2021GraphQL Galaxy 2021
32 min
From GraphQL Zero to GraphQL Hero with RedwoodJS
Top Content
Tom Pressenwurter introduces Redwood.js, a full stack app framework for building GraphQL APIs easily and maintainably. He demonstrates a Redwood.js application with a React-based front end and a Node.js API. Redwood.js offers a simplified folder structure and schema for organizing the application. It provides easy data manipulation and CRUD operations through GraphQL functions. Redwood.js allows for easy implementation of new queries and directives, including authentication and limiting access to data. It is a stable and production-ready framework that integrates well with other front-end technologies.
Local State and Server Cache: Finding a Balance
Vue.js London Live 2021Vue.js London Live 2021
24 min
Local State and Server Cache: Finding a Balance
Top Content
This Talk discusses handling local state in software development, particularly when dealing with asynchronous behavior and API requests. It explores the challenges of managing global state and the need for actions when handling server data. The Talk also highlights the issue of fetching data not in Vuex and the challenges of keeping data up-to-date in Vuex. It mentions alternative tools like Apollo Client and React Query for handling local state. The Talk concludes with a discussion on GitLab going public and the celebration that followed.
A Framework for Managing Technical Debt
TechLead Conference 2023TechLead Conference 2023
35 min
A Framework for Managing Technical Debt
Top Content
Today's Talk discusses the importance of managing technical debt through refactoring practices, prioritization, and planning. Successful refactoring requires establishing guidelines, maintaining an inventory, and implementing a process. Celebrating success and ensuring resilience are key to building a strong refactoring culture. Visibility, support, and transparent communication are crucial for addressing technical debt effectively. The team's responsibilities, operating style, and availability should be transparent to product managers.
Debugging JS
React Summit 2023React Summit 2023
24 min
Debugging JS
Top Content
Watch video: Debugging JS
Debugging JavaScript is a crucial skill that is often overlooked in the industry. It is important to understand the problem, reproduce the issue, and identify the root cause. Having a variety of debugging tools and techniques, such as console methods and graphical debuggers, is beneficial. Replay is a time-traveling debugger for JavaScript that allows users to record and inspect bugs. It works with Redux, plain React, and even minified code with the help of source maps.
Principles for Scaling Frontend Application Development
React Summit 2023React Summit 2023
26 min
Principles for Scaling Frontend Application Development
Top Content
Watch video: Principles for Scaling Frontend Application Development
This Talk discusses scaling front-end applications through principles such as tearing down barriers, sharing code in a monorepo, and making it easy to delete code. It also emphasizes incremental migration, embracing lack of knowledge, and eliminating systematic complexity. The Talk highlights the use of automation in code migration and the importance of removing barriers to enable smoother code migration.
Building a Voice-Enabled AI Assistant With Javascript
JSNation 2023JSNation 2023
21 min
Building a Voice-Enabled AI Assistant With Javascript
Top Content
This Talk discusses building a voice-activated AI assistant using web APIs and JavaScript. It covers using the Web Speech API for speech recognition and the speech synthesis API for text to speech. The speaker demonstrates how to communicate with the Open AI API and handle the response. The Talk also explores enabling speech recognition and addressing the user. The speaker concludes by mentioning the possibility of creating a product out of the project and using Tauri for native desktop-like experiences.