Mastering Error Handling Node.js

Hi, everyone. I am Liz Fardy, and Head Developer Advocate at a really cool company called Fusebit, focused on integrations for developers. Thanks for attending to my talk. Today I will be talking about mastering error handling in Node.js. This is my Twitter handle, LizFardy23. And if you want to create awesome integrations, check out Fusebit.io Twitter account.

Error handling should be always taken into consideration when coding, because we all want to write code that works. But errors happen in every single application, from small startups to big tech companies. And developers need to decide what to do about those errors. How do we handle it, suppress it, ignore it, maybe notify the users, report to the team, retry? But how many times? Because one does not simply write a code without bugs. And that's a fact.

So, this is the agenda for this talk. First, we're going to see the types of errors, operational errors, and programmer errors, because they're different strategies for the different types of errors. Then, we'll see how to handle those errors. And finally, patterns to improve error handling. So, let's begin.

A big part of programming is just finding and fixing bugs, so it's important to do it like a pro. Sometimes, when trying to fix a bug, we create three more. Or even, we break the whole application. So, if we know the best practices of error handling, it's possible to reduce debugging time, improve the code, detect errors faster, and avoid creating more errors when fixing a bug.

Now, let's first identify the type of errors. We can divide all errors into two broad categories, operational errors and programmer errors. The operational errors are runtime problems experienced by code that is well written. So, the code doesn't have any problems, no bugs. These errors are caused by something else, some external circumstances that can disrupt the flow of the program execution.

Some examples are the system itself, for example, when the system runs out of memory, the system configuration, there is no route to a remote host, maybe because the host is offline or if you're trying to connect to the incorrect port, the network, when you experience sockets hang up, remote service, 500 error, which is the famous internal server error. There is something wrong with the server that is not very specific and is not related to your code, the internet or your computer. It's just an error with the server. Fail to connect to the server, maybe because you don't have internet. Fail to resolve hostname when you have a typo in the hostname that you're trying to connect with. Invalid user input. For example, the user needs to enter an e-mail address but the e-mail is invalid. Or any other invalid input provided by the user. Request an app. For example, the server took way too much time to respond and it fails. A database connection is lost, perhaps due to faulty network connection. This is an example of a system that's operating in our memory, socket hung up, database connection lost, invalid input, 500 error or remote service error, and no WiFi, which can cause fail to connect to the server, error.

When you experience these kind of errors, there is not much code we can do to fix it. File not found is an operational error, but that doesn't necessarily mean anything is wrong. These situations do not arise due to mistakes in the application code, but they must be handled correctly. Otherwise, they could just cause more serious problems. There are five main techniques to handle operational errors. First, retry the operation. If you're experiencing a 500 error or fail to connect to server or the server is overload, you can just wait and retry to connect in a few seconds. Network requests to external service might sometimes fail, even if the request is completely valid. Such issues are normally sporadic, so instead of reporting it or notifying, you can retry the request a few times until it succeeds or it reaches the maximum amount of retries.

If the HTTP status code is 500, 503 or 429, it's a good idea to retry after a few seconds or retry using the exponential backover strategy, which is delaying the follow-up retry and progressively increasing the delay for each consecutive retry.

The second is report the error up to the stack. In many cases, it's a good practice to stop the flow with the program execution, clean up any unfinished process and report the error to the stack so that it can be handled appropriately. This is useful when a function doesn't have enough information to handle the error directly.

The third one, notify the client. For example, if the error is caused by invalid user input, like an incorrect email address, you should let the client know and make sure to include the information that you need to construct the error message that makes sense to the user, like invalid email address. When dealing with external input from users, always assume that the input is bad by default. Validate the input before it's sent to the user so they can address it before starting any error-handling processes.

The next one is abort the program. Another solution is just abort the operation. In cases of unrecoverable system errors, the only thing we can do is to lock the error and terminate the program immediately. Clean up where you started and start again. That's it.

And finally do nothing. Yes, just do nothing. Sometimes, there's nothing you can do about it to operational error. There's nothing to retry or abort. There's no reason to crash the program. An example might be if you're keeping track of a group of remote services using DNS and one of those services falls out of the DNS. There's nothing you can do about it except to log a message and proceed with the remaining services.

Error handling is about what failed and why. The distinction between operational error and programmer errors is important to figure out how to deliver those errors and handle them.

Now let's check the programmer errors, which are the errors where developers have control and can be fixed by code. Programmer errors are just bugs in the program. Those can be always avoided by fixing and changing code or logic.

There are six types of error that can be found in JavaScript. First one is type error. It's an error when an operation can't be performed because the value is not an expected type. For example, when you see kind of a property of undefined or pass a string when an object was inspected or required is not a function. In this case, the variable of the parameter is not a valid type.

You can also get this error when attempting to modify a value that cannot be changed or use a value in an inappropriate way. The second one is syntax error, when you miss in parentheses a semicolon or quotation marks inside a string. All of those are synta errors and they're super annoying.

Sometimes they're so difficult to find because the syntax of the code is incorrect. The next one is reference error. When you reference or call a variable or object that doesn't exist or is outside the scope of the executing code. For example, referencing the variable number where this hasn't been defined.

Those three are the most common, but there are other three that are not that common, but is useful to know. Range error is an error you receive when a numeric value exceeds the allowed range. For example, if I have a numeric range from 10 to 20, numbers that start this range will get a range error. Or when the maximum call stack size is exceeded, likely because of a problem with a recursive function within your JavaScript code. The function keeps calling itself indefinitely. URI error. It happens when the encode URI or the code URI functions are used in incorrect manner. URI is uniform resource identifier. For example, if you want to decode percentage signs, it will throw an URI error. Eval error. When you use eval function in an incorrect manner. This exception is not thrown by JavaScript anymore. Now it's used syntax error instead. That's why I don't have an example here because you will probably never encounter this error. The eval error remains for compatibility.

Now that we know the six types of JavaScript you can find, how to handle them? NodeJS supports several mechanisms for handling errors that occur while an application is running. How these errors are reported and handled depends entirely on the type of error and the style of API that is called. There are four main ways to deliver an error in NodeJS. First one, exceptions. Draw the error for synchronous operations. Callbacks, pass the error to a callback. A function provides specifically for handling errors and the result of asynchronous operations. Promise rejections, pass the error to a reject promise function using async await.

Event emitters, emit an error on an event emitter. Now let's see the first one. Exceptions or throwing the error which is a very common way from functions to deliver errors. All JS script errors are exceptions that immediately generate and throw an error using the standard JavaScript throw mechanism. When you throw an error it becomes an exception and needs to be caught somewhere in the stack using a try catch block. If the error is allowed to populate the stack without being caught it becomes an uncaught exception which causes an application to exit prematurely. As we can see in this example if there is no handler the no choice process will exit immediately. I will show a reference error because z is not defined. Throw delivers an error synchronously, that is in the same context where the function was called. If the caller or the caller's callers use try catch they can catch the error. In none of the colors the program usually crashes.

Callbacks. The second way of handling errors is callbacks. Callbacks are the most basic way of delivering an error asynchronously and because of the synchronous nature of Node.js callbacks are heavily used.

A callback function is processed as an argument to another function and executed when the function has completed. If you have been working with JavaScript for a while you have probably seen the callback a lot. Node.js uses the error first callback pattern in most of its asynchronous methods to ensure that errors are checked correctly before using the results of an operation.

This callback function is usually the last argument to a function that initiates an asynchronous operation. For example, the real fight method here expects a callback function and its last argument and it is called once when an error occurs as a result is available from the operation, for example here at the end in the console log. The first argument is the error. If there is an error it will be available in the error argument, if not results will contain the expected result of the operation. It is important to control the flow of the content of the callback function by always checking for an error before attempting to access the result of the operation. Ignoring errors is unsafe and you should not trust the content of the result before checking for errors.

This is an example of how use callbacks to throw a type error. Any caller of the square function here will need to pass a callback function to access its results or error. Note that the runtime exception will occur if the callback argument is not a function.

Also, we want to avoid at all cost callback hell. A callback, instead of a callback, instead of multiple catch statements can get very messy.

Now callbacks are still used for handling errors asynchronously but they are out of fashion because since Node.js v8, the most popular way of handling errors asynchronously is Async.away, the third way of handling errors.

Promise rejection using Async.away. Promises are the modern way to perform asynchronous operations in Node.js. They're generally preferred to callbacks because this approach has better flow that matches the way we analyze programs, especially with Async.away.

If you use callbacks for Async.error handling they can be converted to promises using the built-in utPromiseify method. For example here's how the FsReadFile method can be made to utilize promises. The readFile variable is a Promisify version of FsReadFile in which promise projections are used to report errors. These errors can be caught by changing the catch methods as shown for example here.

You can also use Promisify's API in an Async function like in this example. This is a predominant way to use promises in a modern JavaScript because the code reads like asynchronous code and the familiarity of try-catch mechanism can be used to handle errors. It is important to use a wait before the asynchronous method so that the promise is settled, fulfilled, or rejected before the function resumes its execution. If the promise rejects, the await expression throws the rejected value which is subsequently caught by the surrounding patch block.

You can utilize promises in asynchronous function by returning a promise from the function and placing the function code in the promise callbacks. If there is an error, reject with an error object. Otherwise, resolve here the promise with the result so it can be accessible in the chain. Then method directly adds the value of the async function when using async await.

Finally, event emitters. For more complicated cases, the function itself can return an event emitter object instead of using a callback, and the caller would expect to listen for error events on the emitter. In other words, return an event emitter from the function and emit the event for both success and failure cases. An example of this code is shown here. The emitCount() function returns a new event emitter that reports both success and failure events in the asynchronous operation. The function increments the count variable and emits a success every second and an error if the count is divisible by four. When count reaches 10, an end event is emitted. This pattern allows the streaming of results as they arrive instead waiting until the entire operation is completed.

Here is how you can listen and react to each event emitted from the emitCount() function. As you can see here, the callback function from each event listener is executed indefinitely as soon as the event is emitted, one second at a time. The error event is a special case in Node.js because if there is no listener for it, the Node.js will crash. This is what happens when you don't put the error event listener and you run the program. Just crashes. For the most part, we can join callbacks and event emitters in the same bucket of asynchronous error delivery. If you want to deliver an error asynchronously, you generally want to use one of the other callbacks or the event emitter, but no both.

Finally, the last session of this talk, which is patterns to improve error handling. The first one is use error classes. Creating a global error class. In this case, call application error. That extends other different error classes that are more specific to what you're doing. In order to have different layers when you handle your errors. One layer is, for example, a database layer. Another one is user-facing error and so on. All of these have very specific error cases. You can mold up those error classes into separate modules, that way you can easily import the module and test against the application error. Anything else will be unexpected errors.

The second button is make message errors and expressive as possible. Return proper errors status and codes. If it's 500 or 104 and just make the errors very descriptive.

The next one is YouTube promisify. Some projects only support callbacks instead of a sync wait and it's not possible to refactor it. So you can use promisify because it's very useful for this case.

The next one is adopt TypeScript. TypeScript is a strongly typed superset of JavaScript. Its primary design goal is to identify constructs likely to be errors without any runtime penalties. By adopting TypeScript in your projects you can eliminate a whole class of programmer errors at compile time. For example it was estimated that 38% of bugs in Airbnb codebase were preventable by TypeScript. When you migrate your entire projects to TypeScript errors like undefined is not a function or syntax error or reference errors should no longer exist in your codebase. Migrating your entire Node.js application to TypeScript can be done incrementally. So you can start getting the rewards immediately in crucial parts of the codebase.

The next one is a stack traces. A stack trace is a list of method calls that the application was in the middle of processing when the exception was drawn. You have all the information for where you awaited your promises. Highly recommended to use as much as a sync weight as you can. If you don't, you will lose valuable debugging information.

Number six, automated testing. The JavaScript language doesn't do much to help you find the mistakes in the logic of your program. So you have to run the program to determine whether it works as suspected. The presence of automated tests makes it far more likely that you will find and fix various programmer errors, especially logic errors. They are also helpful in asserting how a function builds with atypical values. Using a testing framework such as Jest or Mocha is a good way to get started with unit testing for Node.js applications.

Finally, you should know that in JavaScript and Node.js, there is a difference between an error and an exception. An error is an instance of an error class. Errors that might be constructed and then passed directly to another function or term. When you throw an error, it becomes an exception. Here is an example of using an error as an exception.

Throw new error, something that happened. But you can just as well create an error without throwing it. Call back, new error, something that happened. And this is much more common in Node.js because most errors are asynchronous. It's very common to need to catch an error from a synchronous function.

Conclusion, proper error handling is a non-negotiable requirement if you're aiming to write good code and reliable software. By employing the techniques described in this talk, you will be able to handle it correctly. Thanks for watching. Happy coding!

Hey, Liz! Hello again! Yeah, that was an awesome talk. I loved your code. One simply doesn't write code without errors. Yeah, that's totally true. Yeah, so you asked the question, HTTP error 404 is operational or programming error, and 57% of people say it would be both. 28% say it's an operational error with 9% and it changes. But yeah, 4%, it's a programming error. Yeah, that's actually right. If the majority of the people that responded to the poll is correct, because 404 can be both. Usually it can be an operational error because maybe there is a typo in the route. For example, a route is the slash contact and the person typed it slash contacto and that would show up 404, because it's not found. But sometimes, and less frequent, it can also be a programming error because the requirement could say that the route should be contact but the programmer creates a different route by mistake. So, the people is right. You a lot in this talk, so, good job. Yeah, you did a great job explaining all the errors, yay. Okay. So, let's take a look at some questions. So, Arzintype1990 asks, does infrastructure as code and the DevOps approach blur the line or complicate deciphering between operational and programming errors? So, what I've read doing this research and yeah, everywhere is like there's always going to be two types of errors, like, sometimes errors can be both, like, we just saw, or can be both, but there is like this distinction always. So, yeah. So yeah, there will always be distinction depending upon the use case. Yeah. Yeah, depending on the use case, and sometimes one error can be both, can be, yeah.

Okay, so, yeah, totally depends upon the context. I hope they got the question answered. Okay, so we have another question. Yeah, again, I think that is what we just discussed is can error be both operational and programmer error, or is it one of the other? So, yeah. Yeah. Well, yeah, sometimes you have both operational and programming errors as part of the same root problem. For example, if HTTP server tries to use an undefined variable, and it crashes, that's a programmer error. And any client we request in Flags at the time of the crash will see an error, but typically reported as a circuit hangup. Yeah, so it can be basically can be both. Okay, nice, and errors are so fascinating.

So, okay, so and we have a question, is it possible to keep another statement in between try, catch, and finally blocks? No, it is not recommended to include any statements between the section of try and catch and finally blocks since they form one whole unit of the exception handling mechanism, so it's not recommended, yeah, it will throw an error if you do that.

Okay, so talking about these errors, I was wondering what are some kinds of let's say annoying errors that you have come across while coding? Yeah, for example, blocking the event loop can be super annoying because it could be easy to block the event loop because of the single threaded nature of Node.js, but also invoking a callback more than once, I have done that. I'm sure a lot of people have done that too, or expecting callbacks to run synchronously, seems no, no, no, or join errors from inside callbacks. Those are errors that are very common and they're very annoying and it's like oh, I did this again. Or the typical error to just forget, like a semicolon or like a parenthesis or whatever and everything crashes. Yeah, those are annoying or I guess errors in general can be super annoying because sometimes it's like the simplest thing ever and it's just so hard to find and sometimes their errors are super hard and you just like know what to do. So yeah, errors are a big part of programming and I was very excited to give this talk because it's a big part of it and knowing how to handle those errors depending of the use case is very important. So I mean yeah. So there are these different kinds of errors and they are super annoying. So what's your approach like how do you tackle these? How do you debug and what's your approach? There are multiple ways in which we can debug the errors and all. Well the first thing I always do is just console log everything. I'm a big fan of console log. Some people just there's other techniques like async trace. There is yeah there is like different techniques you can also use like an APM or external tools for debugging. Yeah but there's like a whole new different talk how to debug applications. Yeah I mean I always keep reading all these different ways but console log nothing beats console log. Yeah. It's super handy. Okay so I have a question from Metin and I read something this week about a global error catcher something like report error. Did you see this new API? Did you take a look at it this new API? No I didn't see it actually I should check it out.

Thank you Metin. Like now we all know about his new API. Thank you. Yeah. So that's interesting global error catcher yeah. Okay so tell us something. Is there something that you would like to elaborate more in your like talk like you gave a super nice talk. So is there some point we should like to elaborate on which I would like. Yeah. Yeah. Yeah. I would like the audience if they can just in the Q&A the score channel if they can tell me like the errors and what errors they have. What are the least favorite errors or where the easiest errors for them or the most difficult like I would like to read their approaches or never say they can like score if you can answer those questions or Yeah, so just so yeah, do let us know what were some of your errors which you found and yeah, we will take a look and there is another question which is what is the most time you have spent on debugging an error and do you recall it being worse or better than expected? Sorry, can you repeat again? What was the? So what is the most time that you have spent on debugging and error? Okay, okay. For me, I think it was like a whole day. Like I couldn't figure out it was a while ago and I was just spent like a whole day thinking about this error. And then as usual, like sometimes it's really good like to just completely disconnect. And go to bed or weigh or go for a walk or anything and then just come back and you can actually solve the error like magically because you were maybe you come with a different approach or you have a different mindset. So because sometimes when you're stuck in an error and you just try and try and try it's just, it just like in a loop it gets harder. So sometimes disconnected can be odd that maybe I shouldn't include that in my next talk. Like disconnected is also a form of error handling like just. Yes yeah yeah sometimes when you see with fresh eyes again and then you can suddenly have that moment yes this was the area. Yeah. Sometimes like I don't know if that happened to you but sometimes even before like when I was like coding a lot right now it's like I'm developer relation so I do less coding. I could even dream about coding and it's like I would solve the errors when I was sleeping and it's like oh no but yeah. I think so Jay Scaars asks what is your opinion about TypeScript versus tdd? Okay well I really like TypeScript. I think yeah I think it can solve a lot of problems and also test-driven development I guess that's why you may buy tdd. Yeah I mean there is no like big difference. I mean there is like there are two different themes. One is TypeScript which is what I've said before and the test-driven development is just it covers the test case before the software is fully deployed. That's what we have, CI, CD and all that. So I think both of them tackle different things so I would use them both really. Yeah, they have some different use cases and we should be using them both maybe to have less error. I would like to know what type of errors which one is your least favorite or your most rememberable error? I hate the kind of errors in which it was it was very easy, like just at my site and I couldn't see it. I mean I'm debugging the whole thing just going deep into the code like and analyzing lots of stuff yeah this that you know all big stuff and then it turns out to be that single maybe a single loop issue or sort of that a very small issue just just one line change or one word change and you are done. So once I remember I had this very peculiar issue with concat array concatenation and all so like these are always super confusing, some of the functions are returning errors, some of them are mutating and you are using one or the other and then your result is totally going awry. Yeah yeah okay yeah so so we that's all question which we have and thank you Liz once again for sharing all this information with us and audience you can still continue the conversation with Liz, go over to the special chat and join the list speaker room and you can still talk to her.