Technical SEO & JavaScript

Hello and welcome everybody out there, I'm super excited to be talking about technical SEO to all of you today. Specifically I want to focus a little bit on Vue.js, but basically none of the things that I talk about are very, very Vue.specific. If you want to learn more about Vue.specific SEO then I'll show you a few resources after the talk.

So when we talk about SEO, then a lot of people are like, what does it even mean? Besides the obvious meaning which is search engine optimization, it seems that it's very non-obvious to a lot of people what this is and a bunch of people think it's basically snake oil and trying to sell links and stuff like that. It's not that. Fundamentally, what SEO is about is making sure that what you are creating, what you are putting online there is actually visible to people looking for it, right? Because if you are going online to seek something that you don't know yet, then you're going through your search engine of choice and you search for whatever it is that you need and you find potential web applications or websites that are serving that specific purpose. To do so, you need to make sure that search engines can actually, you know, find your things and thus people using search engines can actually find the things that you built. And that entails making sure that the robots, which are the search engines, because they are computer programs that consume your content. You need to make sure that these robots actually can process and understand what you're putting out there on the web. And for that, there is a bunch of stuff that needs to be done and a bunch of it is technical, but not all of it is technical.

So SEO is a very broad field, similar to how software engineering is a very broad field. You might do frontend, you might do backend, you might do embedded systems, you might do game development. SEO is not only technical, but we'll focus on the technical side today, as we are all developers.

So when it comes to search engines, search engines fundamentally run a process, and this process can be broken down into multiple steps. The very first step is actually making HTTP requests to your server, to a URL that we found somewhere, and then fetching whatever comes back from your server. Then we need to understand what it is. So is this a website about dogs? Is this a website about cats? Is this a website about boats? And then we would put that into a database, which is called the index. And whenever someone looks for, let's say we make a cat website, if someone looks for cat pictures, then we have lots of websites for cat pictures in our index, and we will need to fetch them. And we need to figure out what is the best result for this user at this time. And that's the ranking process where we figure out where each of the websites in our index goes on the list of results, and then we show these results to users. Now, as developers, we actually have fundamental impact on this process because, if you think about it, both the server side as well as the client side influence if a website or web application can be crawled and indexed. If we can't make requests to certain pieces of content, or if the URL just gives us a 404 or a 500 error, then we can't really take it in and process it further. If we do, but then there's like something in the JavaScript that prevents us from seeing any of the content, then we don't really know what the website is about, and we can't actually put that into the index either. And if it's not in the index, we can't really rank it. A lot of people are focusing on ranking when it comes to SEO, and I don't think that's reasonable because it's just a lot of factors. And ranking is also influenced by where is our user, how many other competitors are there, what are the competitors doing. So there's constant flux in rankings, and I don't think it makes sense to look into those too much. I would focus more on the crawling and indexing side of things, because that's where you can have a lot of impact. Now as I said, the process starts with crawling, which means we discover a URL somewhere, and then we make a request to that URL. URLs take many forms, but these are the typical URLs that you probably have encountered yourself or have provided yourself.

So you have a host, you have a path, and sometimes you have subsections. The thing with these hashes, or fragments as they are called technically, is that they are not meant to address different content. So page should have a lot of content and then part of it is the content that is addressed by subsection. But it's not that if I have page and then I change the fragment that I get completely new content that I wouldn't have seen beforehand if I would have gone to slash page.

Unfortunately, that's what a lot of people are doing. They are using hash-based routing, which is legitimate because it is a lot easier to set up for a local development server. But nonetheless, for SEO purposes, this is not great because a crawler assumes that these two pages are the same as the homepage here, right? The hash about should just be a part of what is already on the homepage on slash. Same with an exclamation mark in there. We have a specific URL crawling scheme for such URLs but we have deprecated it because we found it not to work that well. So definitely make sure to use History API rather than Hash Based Routing. In 2019, 12% of the websites we crawled for the Web Armor Doc unfortunately continue to use fragmented URLs.