Web3.js - Past, Present & Future

Hi everyone, it's Nazar here. And today we are going to discuss the Web3JS, one of the most famous and widely used libraries in the Ethereum community. So we will be discussing some past aspects of the library and some future aspects as well as some present milestones that we have achieved currently. So let's get started.

So first of all, we're going to explore the past about the library. Most of you had been using this library a lot, but may be not aware of how this library was actually started. So this Web3js library was started in 2014 by a solo contributor, Jeffrey Bickle, and in 2016 it was moved to another solo maintainer, Samuel Furter. And from 2018, Ethereum JS from the Ethereum Foundation was maintaining it. And since two years, our company ChainSev has been responsible to maintain this library. And if you look on the facts that this library has been maintained for over eight years, it has over 100 contributors overall and have 17,000 more than 1700 pull requests.

And so far 172 versions were released and out of which 90 were the pre-release version. That means this library had very thorough investigation and debugging and testing from the community during the pre-release versions. 90 pre-release versions is a large number. And this library has 600,000 weekly downloads as well. And it's all because of the community who is using and trusting this library.

And there are some fun facts about the library. I have asked so version one of the library web3.js has been living in beta for two years and people have been using v1 beta in the production environment for many years. Because this v1 was in extensive testing and thorough debugging process. And we started to have a v2 rewrite in past and it was a fail and abandoned rewrite attempt because of some aspects I will discuss in further slides. We started v3 rewrite with some extensive features, which never lasted a lot. And since then, v1 is the only version which the whole community has been using so far.

This is quite an interesting aspect as well, and considering that like two rewrite attempts or two version attempts has been failed so far, but it doesn't mean that the library v1 version has been unstable. So v1 has been maintained and supported so far and had been doing it. And if you look at the aspect that v1 lived for the two years in the beta, I think this is the most longest time after another project expressjs version 5, it will last for more than two years.

And there were some challenges we were facing in the web3js version 1. Most of you can understand it because you may have already reported some of the issues or have been also encountered or facing those issues. It was very obscure code base. The code base was being maintained by over 100 contributors I explained earlier. The code base was not using some standard process or standard patterns. The modules inside the code were very tightly coupled. There were a lot of technical challenges to maintain this code and there was a lot of difficulties to introduce new features particularly in the library v1.

And most of the you guys have already been using the typescript. So there were a lot of decoupled types. Decoupled types were in the web3.js v1. And because of that, there were a lot of typescript issues being reported by the community. And if I look on the logs, I can see around 60 over 60 issues been reported of the mismatch or inaccurate types, which was one of the biggest challenge for us. And from the community perspective, the build size became around the bottleneck. And for the dev development, because this library was bundled in a huge single package file and that package file was being bundled in every dev. And this was causing a lot of problem during the dev development by a lot of community as well. And on a larger spectrum, JS community and the toolchain has been advanced a lot. A lot of new frameworks, new patterns and new tools have been introduced in the community in the ecosystem, but Web3 JS could not get way forward because of those challenges I discussed earlier.

So are we going to stick with the version one, or are we going to do something about it? Despite the two attempts or failed attempts, we are still hopeful that we are going to present the current improvements in the library and with that said, this is the new era for the Web3 JS, the version four, and this is not some speculation. It's been under development for over a year now. And we are almost ready to release, hopefully it's been released this month, but you can already explore directly from the GitHub. So version four is going to be a huge, huge, huge milestone for the Web3 JS lifespan and it's going to address a lot of stuff that we've been discussing in the community and all those people have been addressing those things. So version four will come with a native TypeScript support and it will have a reduce size. We have decoupled a lot of code and the complexity has been reduced since we are doing this rewrite completely from scratch, so the code readability has been improved a lot, and we try to make it extensible so that in the future we can add more features very easily and even the community can extend the library for their own use, and we focus on the test coverage for the version four and hopefully once, right now we are working on writing the system test for the library, and within a month when we will release the first alpha for the v4, it will be having a lot higher test coverage compared to the version one, and compared any other competitive library in the ecosystem. So the question is, does all these goals that we defined, rewrite for the version four, are these good enough, or have we achieved it or not? Yes we have achieved all these goals, and the most important thing is that we achieved all these goals with a v1 Parity API. So you don't need to be worried about that your projects are going to break. There could be slight changes that we will be documenting and sharing with the community in the change log or the migration guide, but those changes will not be a lot. Overall, the API of version one remains the same.

We explicitly decided this architecture change, or this pattern, so that a lot of community does not break their projects, and they can easily migrate to the version four, which is more recent.

And the question is like, does it just rewrite of the v1 with the same API or does it include some new features as well or not? Yes, it is. It contains a lot of new features. It's not just the rewrite. Rewrite was the first objective, but during the rewrite, we focused on creating more features within the library so that we can be future safe and we can extend the library very easily in the future.

And some of the features which I'm going to present here, which is very unique for the version one, version four for the Web3.js library is unique in the Ethereum ecosystem. No other package or the utility as of now, which I remember may have introduced such features and hopefully you guys will like them.

So the first thing, the most important feature within our rewrite is this new introduction of the new validator. Because data is so important for any dev particularly and data validation has been an extensive job for the maintainers of the different devs and we decided to create a brand new layer of the declarative validation for the community. And we use this validator within our library as well and it is also available for you to use in your devs.

And so what is a declarative validator? So most of you know that JSON schema or JSON data is kind of a defacto data format for the JS community. JSON schema is also a defacto data standard for validation and communities already adopted it. So we use both of these schemes as a JSON data and the JSON schema and merge them together for the Ethereum community. And you don't need to have no if-else anymore and all of the validation that you're going to do using the JSON schema will be tightly coupled with the Ethereum types. So you don't need to remember the Ethereum native types or compare to the JavaScript native types. It will work seamlessly. And this validator is also compatible with Ethereum ABI schema. So if you have such an ABI, it will also validate the data based on the ABI as well.

Let's jump to some examples. So here you can see some examples. So we have some data and which contains some addresses in an array and some balance, and, if I go with the imperative validation, which we had been doing in previous library versions, you can see that, we have, some loops going on and then we are validating by using function calls and then throwing some errors and same for the balance field that we have, we are particularly validating by using some function call. But if we go with a declarative approach, you can see that we have schema defined for the data and this schema is going to present the whole anatomy of the data and if you notice that the data that we are going to present is very particular for the Ethereum. So we have an address array, we have UN256. These are Ethereum types and we can validate this data based on this schema and there are different formats of the schema and hope you can explore further.

So there is another feature that we introduced in version 4 rewrite and this is going to change the way you look into your data. So the question is that every dApp that you develop is not the same. So the need for every dApp is also different. For example, someone developing an app or using a library where he needs numbers in the beginning. Someone wants numbers in a number or someone wants number to be a number string. Similarly for the bytes.

So there can be different formats of the same data that you can use but the previous version was very strict and very restrictive to the one data format. Which was not counter which was counterintuitive for the different dApps which were using different data formats.

So we tried to cover this as well and we are creating a dynamic data format, which is quite easy. All of the Ethereum web 3js library interfaces are going to return are going to support this custom data formatting. And let's say we have this get balance and by default, this get balance will return you a balance in the hex string format.

But if you want that the balance function should return the balance into different formats, you can define your own data format, specifying the format for the numbers and specifying the format for the bytes and passing these formats to the every function call is the last parameter and you will be get the data in that particular format that you define. And you can use this formatting logic in your own project as well. Our library has been extensively using it.

So one more important feature. Maybe everyone should be relying on it, and I think this is one of the unique features which Web 3.js version 4 is going to introduce, is Typescript for the contracts without any transpiling. So you don't need to transpile anything. All you need is your API declaration in the Typescript project. So there is no need for the transpiling of the APIs or creating or doing some command line execution of any other third-party tools. You just need to copy your API, declare them into the Typescript project, and you will be ready to have a complete typesafe contract instance. Let's say we have this API and all you need is to have this API declared, and the trick is here that you have to declare them as a const, so the Typescript does not infer them as generic types rather have strong types for each declaration. And once you have declared this API as a constant, you can create your contract instance and this contract instance will be completely typesafe and there is no transpiling required. And if you ever change the API or update your API, your types will automatically reflect those change and Typescript can detect the validations on the compile time. This is a unique feature and I would love that we'll be sharing some feedback on it.

And there is another feature that we're going to introduce is easy extensibility for our API. So the Web3 modules have introduced a different or consistent API to support all modules. So when I'm referring to the modules, you may be aware that Web3 has different modules like contracts, Ethereum, SSH, ABI, ENS. So these are different modules. So we introduced a consistent API for all of the modules so that you don't have a different compatibility issues using them.

And you can also create your own modules if you want. And the feature is very simple and like straightforward to introduce, you just need to, we have consistent object or Web3 context and all you need is to declare your component or the module which is extended from the Web3 context and then you have a library instance Web3. You just call them a use and you will get an instance of the component which will have all of the main component of the library link. For example, a provider will be linked a request manager, a subscription manager and a lot more will be linked to your component. And that will easily, so you can use your component easily with the same consistent API which we had been using in the Ethereum or contract or ABI or ENS packages. So this is all from the version four, so the question is like, this is all the features that we're gonna introduce. I think not. There are a lot of future improvements, a lot of refactoring, a lot of things going on in the version four and I will left all those to you that you can explore in the version four and the version four is in the alpha hopefully this month but you don't need to wait it. You can go to the GitHub and check out 4.x branch and it will be all up for you to install directly from the GitHub and use it.