Let's Talk GraphQL With Your Services

Hi, welcome everyone. So this is Roy at GraphQL Galaxy with Let's Talk GraphQL with your services. So what does this talk about? So during today, I will present you some use cases for GraphQL also with existing services. So let's have a look at legacy codes. It might look like this, right? Or is this actually PASTA? I'm not sure if you know, but what was the last time you tried talking with PASTA? For me it was never. So let's assume we're going to be looking at existing services as legacy code.

So a little bit about myself. So my name is Roy. You can find me on Twitter with Github team. Currently, I work for a renewable energy company called Vanderbrand, previously I worked for a lot of different companies, including the city of Amsterdam, most lately, where we created open source projects for the city. And also, you can find me online on YouTube either with conference videos from previous times or some of my books about React or React Native. And in the topic of today, my most current book about Fullstack GraphQL. So this is a book about how you can build GraphQL servers and clients with React, Node.js and TypeScript. So if you have any questions afterwards, make sure to get it.

So for today, we're going to be looking at how you can talk GraphQL with existing services. So suppose you start working on this great project, you would just get hired at a new job, or maybe a new project starts at your current company. And this one is called the e-commerce platform of the future. Probably you've been approached by recruiters that had something similar like this, like hey, we're going to build the e-commerce platform of the future, we want you to be part of it. Well, let's assume you fell into this trick and you started working at a company that is going to build this platform. So this might be the stack, right? So we have, on the left, you can see an example UI and on the right, you can see technologies. So it's using JavaScript and React, which all seems fine. And then it's using a REST API with a database and a database as a back-end. So, maybe after seeing this, you're thinking it's an e-commerce platform, but it might not be the e-commerce platform of the future. If you think by now, that's okay. If you think it's still the e-commerce platform of the future, that's also okay, because we might be able to find another way to use this platform and make it future-proof. Or you might also think the REST API and the database together are like the pretty bowl of spaghetti we saw before, which is the legacy code that we're going to be interacting with. So still, we have some legacy code back end, we have JavaScript and React, and we somehow want to connect all this together with GraphQL, because if it's using REST, it's going to lead to several problems, right? Because this might be the REST API, you have three different requests, or maybe even more, and all these requests will be called by your UI. So every single bits and pieces of our UI is using a separate REST API call, which in the end is going to be troublesome for people that are using mobile internet. And these often are directly correlated to a database. So we have a REST API and we have database, and those are closely linked together.

So every table in your database will have a separate API call that will work like this in the UI. And all these API calls will also return some JSON, that might be a lot of JSON, and you're actually using small pieces of this JSON. So once you've seen this, you're probably thinking we should start using GraphQL for this, and that's exactly what we're going to be looking at today. So we don't want to do this, having all these separate requests that return a lot of JSON, it all needs to be going into our UI, that needs to be parsed, it needs to be normalized. That's something you absolutely want to prevent.

So this can be done more efficiently, like we already saw, and this is of course with GraphQL. But the service isn't using GraphQL already. So let's try and find a way to have this backend service interact using GraphQL with our frontend UI. And let's do this in a way that we don't have to touch the legacy code or the PASTA or Spaghetti or whatsoever that's living there on the back. So what we want to prevent is having to touch anything that has been built by previous developers.

So you have multiple options to do this. And I believe one of the best options is using existing libraries, because it's going to get rid of a lot of stuff. It's going to prevent you from making mistakes, it's going to prevent you from having to maintain another service. And for this, I would definitely advise you to use GraphQL Mesh. I believe Uri Goldstein is also a speaker today, or maybe he already presented. I'm not sure. At least Uri will be probably will be talking a bit about GraphQL Mesh and how you can use it. And if you want to stay away from making any custom code, it's definitely a way to go. But for today, we're going to be looking at a different approach. And this is by creating your own data layer or data access layer. And using this layer, you'll be able to use GraphQL for your backend services without having to change those services. And it's a similar to what GraphQL Mesh does, except we're going to be owning the entire data flow. We're going to be owning all the logic in there. And in case you're not familiar, a data access layer, or data layer is a layer of a computer program that provides simplified access to the data of that surface.

So what we'll be doing, we'll be building a data layer that provides simplified access using GraphQL over our legacy code. So this will help us do a lot of different things like caching, like monitoring, but also it's going to prevent us from making mistakes while making changes to legacy code that runs, but somehow nobody knows how it runs, which often is the case with legacy code, of course. So this is the question you might ask yourself, why would I be creating a layer on top of a surface that already exists, that runs perfectly fine? Why should we do that? Because is my API already a data layer? Because probably your legacy code already provides an API with an interface that's going to give you all the data that you need. But we're going to be creating a data layer on top of a data layer. You might feel like inception right now because somehow we're creating something on top of something, or maybe it's already inside something. So something to think about for the rest of this talk.

So how do we get from a GraphQL document to data? Because that's what we want, right? We want to send the GraphQL document to our server, and we want the server to return the data from our legacy code. And it might look like this, right? So we saw the previously that the three separate REST API calls, and what we want to do actually instead of sending those REST API calls, we want to send the query and we want to return this data in a way that we formatted it, because that's what GraphQL does.

So somehow we want to be going from a document, which you see on the right, to the data that we have on the left. And this is perfectly fine because this is what GraphQL does, and this is how GraphQL is going to do it for us. So let's see how we can actually implement that. It's by doing this.

So the first step is send a GraphQL document, which you can see already graded out. And this GraphQL document will be converted to an abstract syntax tree. So basically what is this? This is a representation of whatever document you're sending. So we're sending a document that contains an operation, which is a query. It contains fields like top level fields, which is products, but also lower level fields like title or relationships or categories. All this will be parsed into an abstract syntax tree that can be used by the GraphQL schema and the GraphQL server to find out how these things match you. So your server will be in your schema. We'll have a vital role in here because there will be information in there about resolvers, about connections, all these things. This will be described in your schema and also in the server in general. And then the final thing that will happen is the resolvers will retrieve the data. So based on the document that you send, it will be converted into an ASD, and that ASD will be matched to your schema. And finally it will end up at your resolvers that will retrieve the data. So the document that you initially sent to the server will be responded with a JSON output with exactly that data. This is briefly how it should work and how it will work with GraphQL. And if you use existing libraries like the GraphQL mesh thing, it will already do this for you without any trouble. But for today, we're going to be looking at how we can build this ourself with the custom code, which in the end, I think, is the way to go. Because abstraction layers are great, but they also give you the risk of ending up with even more legacy code. So it's something you should always think about. Am I going to do this myself, or will I be using an existing library that already does one of the heavy lifting for me? And constructing the data layer around the data works the other way around, right? Because now our starting point was the document, went to the AppSectors syntax tree, went to the schema and the server, and in the end, resolvers got the data. But if you want to have a data layer, actually your data is the starting point, because we need to have the data in order to construct resolvers and the schema, so you actually have a server to send the document to. Because we will need this server, otherwise we can't possibly use GraphQL operations like the query we saw before to get our data. And constructing the data layer, like I told you, it's exactly the other way around. So we have our data as a starting point, and we will end up with a server that can accept documents and that will receive our data in the end. And for this, there are multiple ways you can go.

So you might have heard about code first or resolver first. Usually, they're the same two things. Basically, it means that you don't really construct a schema as your starting point, but you'll be thinking about existing code, existing resolvers, how to handle or retrieve your data. Another way we could do is schema first. So both could be options, like the code or resolver first option, or schema first. And it all depends what you want to see as a starting point. So more about it later. But I think the code and resolver first discussion is a good thing to have, or schema first, whenever you would pick up such a project with your team, because you need to be able to know how to handle the data, how to handle data flows, and whether you'll be working with a code or resolver first solution, or maybe a schema first solution.

And the starting point should always be the source of truth for your data. So if your existing data is a rest API or a legacy API, then this will be the source of truth. So your API might have a swagger definitions that you can use or any other open API definitions. It might return some sort of schema in the form of JSON, maybe with the JSON schema. If it's a database, you might have a migration documents, you may be with MongoDB, you might have Mongoose models, same for any other kind of database.

So the starting point when doing this should be any source of truth for your data. And if you don't have one that's going to be a bit tougher. And if you don't have a source of truth, free data, I would always advise to go schema first, because then your schema will be the source of truth for your data. And if you already have a source of truth, you can use a swagger, open API or Mongoose models to create the resolvers code first.

So the thing we need to do is map the data specification to a GraphQL schema. And this is easier than you think. Because there are packages to do this for, like the GraphQL master cell before. But if there isn't, and I actually I found a use case myself, because quite recently we started working at a salesforce implementation. And if you know salesforce, they have rest APIs, but they don't come with swagger. So they come with their own describe endpoint, which is sort of following the JSON API schema. So we needed to create everything ourself, but we had a source of truth. So that's the important part. Without a source of truth it will be very difficult to do this. And you should always go for a schema first approach. But if you don't, you can just go for a code or resolve a first approach. But the thing you need to do is use the source of truth, whether or not you have it, to specify a graphQL schema, whether or not through code first, which we define the resolve first that create our schema, or by directly creating the schema.

And this might look like this, right? Because previously we saw we have a rest API, which is our legacy project. And we want to be able to have a data layer on top of this rest API to get our data in graphQL. And at the forefront of this database, of this rest API, there's also a database. So there are two ways you can go here. So either you're going to be building on top of your rest API, or you're going to be building on top of your database. And which you should select really depends on a lot of things. But you can imagine that if your rest API is a one-to-one mapping of the database without any normalizations, you should use the database as your source of truth. But if your rest API is having side effects, so instead of just getting data from the database, it will also do normalizations, maybe call other APIs, then we should definitely take the rest API as a source of truth. And if you're lucky, this rest API will have a Swagger definition or adjacent schema.

And if it doesn't, you're going to be actually checking all the endpoints that you want to use and create a schema out of these endpoints. So it might look something like this because we have a rest API with some sort of definition, and you might want to port this to a schema or to some sort of schema definition.

So for this, I actually went with the GraphQL JS approach by using GraphQL object types to define my schema like this. And in here, I can also define resolvers or variables that I want to accept. But it's just like the basic mapping. And this mapping is done with the knowledge that we might not have a source of truth for this API, because a source of truth might be outdated, something you often see with projects that use Swagger. It's a lot of manual work if you don't use the proper libraries.

So let's assume the only thing we have is the request and maybe some minor documentation on our side. So you would have to find a way to map this to a GraphQL object type, so to a schema or to any sort of library that uses a code-first approach. But for this time, we just created manually like this. So an example of my project, we use the Salesforce described endpoints with a JSON schema to create the GraphQL schema out of it. So basically it looks like this. So the mapping is one-to-one and you just insert the fields that you might want to use. And once you've done this, you'll be able to create resolvers to retrieve the data from our original source.

So in this solution, it will be a REST API or my solution could be Salesforce. But even if you're using a database, would maybe Mongo to be a Mongoose, then the resolvers will be retrieving data from a database. So your resolvers should be able to retrieve the data that matches the schema. And basically it looks like this. So I went for an old nice approach with graphical object types to create the schema and it's schema can also include resolvers. So in here you can see I've created one for product that relates to my product REST API endpoints. And in there, there's resolver that will call some sort of class or method or function to get the product based on an idea that I get from resolver. So the resolver is getting some information that's getting the parent object, the arguments, the context, and also the info object. And the info object can also be quite nice to explore more in detail if you're going for this approach, because the info object will always know all the information you have about the AST, so the Abstract Syntax Tree. So it's interesting to have a look at this later. Yeah, so fortunately, we don't have the time for this today, but I believe there's many more things to investigate for that. And I created another one for category, and you can see this using the parent object to get the category for a certain product. So this is about the resolver. So once you've got the schema and you have a source of truth that you've created using the schema, you can also use this schema to create resolvers at once. And if you're going for different solutions, so maybe you're using TypeScript, you might want to go for the code or resolver-first approach, and then the schema will automatically be generated once you create resolvers. So there are multiple ways to approach this.

So I think an important question to ask yourself is, which solution should I choose? So I think I give you a very broad oversight about how you can use existing services and make them talk GraphQL. Of course, I mentioned a library called the GraphQL Mesh, which Yuri is probably mentioning today as well, or maybe already has mentioned it. And this is great if you have an existing source of truth, and you're pretty sure you don't want to make that much customizations to it. Although there are options to add custom resolvers. But in general, I think my approach was, you should go with your existing legacy code, take a source of truth, and then try to find out what you want to use or not want to reuse, and then how you can create a schema out of it. Because in the end, your schema or your resolver should be defining the data model for a GraphQL server. And not all servers really like this approach, because you may be using older things like SOAP. Or if you are using REST and don't have a Swagger definition, then you should do this all by yourself, which I think can also be done manually. It can be automated if you use the response from every endpoint. But it's a very good discussion to have with your team whenever you start working on a project like this. So you want to build a data layer on top of your legacy code, but you don't want to create more legacy code. So if you do it all custom, then you might create a lot of legacy code. But if you invest in an existing library, then you have less customizations that you can do. And also if you want to have side effects in your library, then of course, you need to build a custom one yourself. So these are very good things to discuss beforehand. But hopefully you will get some sort of ID, how you can use a source of truth to create a GraphQL layer on top of it. And for me personally, the approach to Salesforce required a lot of customizations, but in the end, I think it's also a nice approach to use and to be able to use it for all our services. And in the end, as soon as you build a graph on one sort of legacy code, you of course, can use schema stitching or federations to connect the services with all the services as well.

So if you want to learn more about this, I believe we have some time for Q&A, or you can just go to my Twitter page and ask me a question over there. And also make sure to find me on YouTube. And of course, if you want to know more about GraphQL, make sure to find my book online, because there's a lot of cool content on there and you can get the first chapter for free and see whether or not it'll suit you. Or again, just find me online and ask the questions over there. So yes, thanks a lot for your time.

Hi. Hi, Rai! Hey. So, my first idea was actually also Notepad+++. Oh, you've only got three pluses in that one. Yeah, I was just thinking. I think the third plus was when I found out you could also link FTP to it. You could directly push your code to the server. Yeah, that sounds pretty good. Yeah. When you could write to be on production, that was great. In many ways, a simpler time. Yeah, those were the times. It's just refresh production. Yeah, now it's back-to-back. How did you get inside the server room?

So yeah, what inspired you to make this talk about incorporating GraphQL onto a legacy stack? Working with legacy code. I think everyone did. Yeah, for me myself, the last project I did was mostly frontends, and we wanted to change things on the back-end, but the back-end still doesn't want to change anything. So we ended up building something around it. Yeah, I think that's a really common use case. It wasn't until quite recently, actually, that I'd made a GraphQL system from scratch, which wrote directly to a database. It's always been a case of, like, look, we've got seven different disparate systems, and wouldn't it be great if there was a graph to connect them? Also, they all have different authentication. Yeah, that sounds interesting. They're classical problems. Yeah. So what do you think are the most common struggles people have when they're trying to bring a data layer on top of existing services? In your experience, what has been the top thing that causes grief? Like, when implementing it or reasons to implement it? Oh, when implementing it.

I think one of the biggest things is authentication and caching. Authentication, probably because the server that you're wrapping, of course, to understand it and caching because you want to make sure that it doesn't get out of sync. So you want to do some sort of caching but you also don't want your data to be outdated. So if you're working with legacy and they don't provide e-text or something, it's going to be pretty hard to make sure that you have recent data.

Yeah, there's a lot of things. I believe Jiri Goldstein also gave a talk and he created something called GraphQL Mesh which is great for this. And of course you could also use Azure and I believe some other services doing this as well. And they all automate it for you, but then again it's something you might have to pay for and it's also fun to build yourself.

Yeah, that's correct. So performance wise, it might not be the best improvement. But I think what I wanted to show is most of all you can use the developer experience and tooling around GraphQL instead of making the most performant server because yeah to make it most performant you of course have to change the legacy code, which is something you usually can't.

I think it depends on the kind of normalizations you need to do. So previously we did an open source project for the city of Amsterdam. And all the APIs we consumed were also open source. So they were made available for a broad range of things. So some people were using it for machine to machine. Other people were using some parts of it. And the platform rebuild took like 20 of these APIs with like 80 different calls and normalized, merged them all together. So these are things we didn't want to do on the backend, but also things you don't want to do on the front end. Because I don't know, if you have to parse like 20 different API calls, merge the data together, it's really something you want to prevent it from doing. So it really depends on the-

Sorry, I had a phone call to mute. I'm so sorry. Okay. Well, thank you for that question. And sorry for my rudeness.

So it really depends on the- Sorry, I had a phone call to mute. I'm so sorry. Okay. Well, thank you for that question. And sorry for my rudeness. Oh, I should have put that in silent a long time ago. It was actually my iPad that ran because I put my phone on silent. But apparently if you have an iPhone, it also rings, your iPad. Yeah. That's really annoying. And it takes long to go. So when my phone rings and at the end of my AirPods set it's still MacBooks and iPods, iPad, and then the phone going off. Yeah. I'm going to quickly turn this off. So, yeah.

When you sell this idea, I guess I should rephrase this. Do you have any tips for selling this approach, this technology over competing ways of managing your services if you're trying to get a customer interested in this technology stack? Yeah. That's a question with many answers I think. But usually I think like using GraphQL for client-facing applications, I think it's something you have to do. Because if you're trying to force a REST API to a client-facing application, I think it's something... Well, I don't think it has any benefits for everyone.

Wonderful. Thank you for these answers. I have exhausted my pile of questions. And we have no more questions from the audience. So this is your slot at the end. Is there anything you would like to promote, plug, tell the audience? No, not really. I think if you saw my talk, then hopefully, you'll be starting to use GraphQL. And hopefully, you'll be starting to using it from scratch, and just connect your database to it. So you get the optimal experience. I hope so, too. We should all try GraphQL today. All right. Thank you very much. Leave your claps in the chat for Roy. And thank you again for coming on. Cheers.