Let’s Build a TV Spatial Navigation
From Author:
In this talk, I'll take you through my journey as I joined the team supporting our Smart TVs application and share my experience learning one of the most overlooked but essential pieces of functionality we have.
This talk has been presented at React Advanced Conference 2023, check out the latest edition of this React Conference.
FAQ
The topic of Sergio Avalos's talk is Spatial Navigation for smart TV applications.
Sergio Avalos is a software engineer at Spotify, working on the team behind the Spotify client that runs on smart TVs.
Spatial Navigation is a term used to describe the process of navigating a TV interface using the directional keys on a TV remote control.
A library for Spatial Navigation is needed because smart TVs have different operating systems, and using a web application for the user interface can lose native platform support, including Spatial Navigation.
As of 2023, browser support for Spatial Navigation is still a work in progress. There is a proposal in draft, but it is not yet implemented.
Yes, there is an open-source project provided by Norwegian Media, released in 2019, but it wasn't available when Spotify's smart TV application was initially developed.
Using IDs for navigational elements can be error-prone, difficult to work with dynamic views, and adds extra information unrelated to the application logic.
Sergio Avalos suggests using a hook function that returns a callback for setting the reference of the HTML element and managing focus without relying on static IDs.
Some advanced challenges in Spatial Navigation include handling non-matrix layouts, managing focus on pop-ups, and implementing circular navigation for convenience.
Developers can use the library linked in Sergio Avalos's presentation to start building smart TV applications without developing Spatial Navigation logic from scratch.
Video Transcription
1. Introduction to Spatial Navigation
Welcome to the talk on Spatial Navigation. We'll be discussing the challenges of implementing spatial navigation for TV controls and why a library is needed. The market for smart TVs has multiple brands with their own operating systems, making it necessary to have native applications for each. However, to simplify maintenance, we built a web application for the user interface. Unfortunately, this approach resulted in the loss of native platform support for spatial navigation. Although there is a proposal to provide this functionality in browsers, it is still a work in progress.
Welcome, everyone. Thank you very, very much for joining this talk.
My name is Sergio Avalos, and we're going to be talking about Spatial Navigation. But rather than talking, we're going to be building in.
I'm a software engineer at Spotify and I recently joined about a year ago the team working behind the Spotify client that runs on your smart TV. That means that for this talk, we're not going to be talking about mobile, neither we're going to be talking about desktop. And most importantly, we're not going to be talking about mouse. Instead, we're going to be talking about TV control, that gadget that I bet all of you have in your living rooms.
Special navigation is nothing else, just a fancy name for describing what you do with the TV control when you are pressing the directional keys, the arrow keys for selecting the one application or just navigating to one of them. That got me very curious when I joined the recent team where I'm working because I didn't know that, I mean, I was surprised that one had to create a library for that. So I decided to dig into the code and I was fascinated. Not because the code was amazing, I mean, it was fine, the code, but most importantly because I felt that it was a very interesting problem to solve. So that's what this talk is about. I want to share with you my learning of how I learned about this library and what a better way to learn than just building it ourselves.
But in case you wonder, because that was my first impression, why do we need to build a library for spatial navigation? I mean, isn't it a huge utility that should be provided by the platforms? And the answer is yes, totally. If you're building a native application. Let me try to explain.
The market for smart TVs is quite cement, there are many brands and each of them run their own operating system. That means that you need to have your native application running for each of them. But, just to make our lives easier and reduce the maintenance costs, we decided to build a user interface using a web application that can be loaded in each of the native app. That gave us a great interoperability of shipping the same code to all these native applications. But, it came at the cost of losing the support from the native platform. In that case, it's obviously the spatial navigation.
Then, I was thinking, okay, okay. But, the year is 2023. Shouldn't that be provided by the browser? I mean, the browser, nowadays, is a very sophisticated piece of software. And, the answer is not yet. It's a work in progress. There is a proposal. It's still a draft for building this functionality, but it's not there yet.
2. Improving the Approach to Spatial Navigation
We need to continue waiting. Are there any open-source projects we could use? Norwegian Media released one in 2019, but our application is older. Let's start building it. Wrap each navigational element with an ID and tell them where to go. This approach has caveats: difficult with dynamic views, prone to mistakes, and adds extra information. Let's improve this approach by developing the extra logic to connect TV control with our application.
We need to continue waiting. Then, I was thinking, okay, okay. But, are there any open-source projects out there that we could use? And, actually, there is. Thank you very much, Norwegian Media, for providing this. Unfortunately, they released it in, no, unfortunately, but they released it in 2019, and our application is a little bit older than that. So, we didn't have any back then.
Having answered that question, let's start. Let's start building it. If I ask you just from top of your head, like your intuition, how would you do it? I don't know about you, but for me, it was, I mean, the simplest that I could come up, and I think I read it on a blog from Norwegian Media, and even from Netflix. It's basically, you just wrap each of what I call navigational elements as the element that the user can interact with it, with just an ID, just identify them, and then you tell them where to go. Take for example, the sidebar of our application, the Spotify application. Each of these elements is just a link for the home view, the search, and so on, and like I've explained before, you wrap them with an ID, and in that wrapper, you tell them where to go, so if you are going to the, if you're in the search and you go to the app, then you tell them, go to this ID that is the home.
That approach actually gets the job done, but it obviously have a few caveats as you, I can imagine, you can anticipate. One is that it is difficult to work with dynamic views. Think for example, recommendations, the developer doesn't know what they are going to get. Also, it's ever prone, because the developer is the developers role to add this ID manually, so mistakes can happen. We're humans. And finally, it just adds extra information that is not related to the application, like I said, this is just a utility that should be invisible to the application layer. So let's improve this approach.
For this presentation, I built a very small application app that basically has just two views. It's a welcome. Click on this one. Then you go to another view that renders you just a surprise for whatever. And then you have the go back link. And then you come back to SMBN. It works perfectly well with the mouse, but it doesn't work with the TV control. So this is exactly what we're going to do. We're going to develop the extra logic that we need to connect TV control with our very simple application. Demo application.
Check out more articles and videos
We constantly think of articles and videos that might spark Git people interest / skill us up or help building a stellar career
Workshops on related topic
Table of contents:
1 - The infamous "N+1" problem: Jonathan Baker - Let's talk about what it is, why it is a problem, and how Shopify handles it at scale across several GraphQL APIs.
2 - Contextualizing GraphQL APIs: Alex Ackerman - How and why we decided to use directives. I’ll share what directives are, which directives are available out of the box, and how to create custom directives.
3 - Faster GraphQL queries for mobile clients: Theo Ben Hassen - As your mobile app grows, so will your GraphQL queries. In this talk, I will go over diverse strategies to make your queries faster and more effective.
4 - Building tomorrow’s product today: Greg MacWilliam - How Shopify adopts future features in today’s code.
5 - Managing large APIs effectively: Rebecca Friedman - We have thousands of developers at Shopify. Let’s take a look at how we’re ensuring the quality and consistency of our GraphQL APIs with so many contributors.
We will enhance a full-stack JS application (Node.js backend + Vanilla JS frontend) to authenticate users with One Time Passwords (email) and OAuth, including:
- User authentication – Managing user interactions, returning session / refresh JWTs- Session management and validation – Storing the session securely for subsequent client requests, validating / refreshing sessions
At the end of the workshop, we will also touch on another approach to code authentication using frontend Descope Flows (drag-and-drop workflows), while keeping only session validation in the backend. With this, we will also show how easy it is to enable biometrics and other passwordless authentication methods.
Comments