Shining Offline: The Mobile BelIS-Online Wuppertal Experience

Hi, and welcome to Shining Offline, a best practice talk about us almost struggling and how the offline first principle helped us. I am Thorsten Hell and I work for a small software company called Kismet in the southwest of Germany. Today, I want to share how we found ourselves one step away from disaster with the first mobile urban lighting information system from Wuppertal City Council, and how using the offline first approach saved us, turning a hard time into a smoother, easier experience for our team.

Here's the structure of my talk. I begin with an introduction with one slide about Kismet. Next, I will explain what is the Bayless original system and why did we develop it. Then I show how we transition from iOS to a progressive web app, focusing on a four-dimensional offline first approach. At the end, I will highlight the most important points from my perspective.

Kismet started in 2002 as a spinoff from the EIG at the Saarland University of Applied Science. Our main focus is building software solutions for cities and local government bodies using open-source technologies. We are a 100% remote company and we've been working remotely long before the pandemic.

So, let's talk about Bayless. What is it? Back in 2008, the Wuppertal City Administration asked us to develop an information system for managing the city's lighting. It needed to integrate with the geospatial data infrastructure that we had already helped build. With the old systems, like BS2000 or MSXS, the georeferencing was done manually on printed-out maps. We decided to go with a Java Rich Client application, which was quite common at that time. The really new thing was the embedded map component. The application handled recording and managing every type of object in the lighting infrastructure. This included not just the lights but also poles, switching stations, junction boxes and even cables with wall hooks, which are fixtures attached to buildings to hold ropes and wires in place. Even these little pieces were georeferenced and displayed on the map.

Alongside managing the lighting infrastructure objects, the system also handled work orders, at that time as reports. These reports were printed out, brought to the field, filled out by hand and then entered into the system again, causing a lot of unnecessary effort and inconvenience for both service providers and operators.

In 2015, therefore, the demand for a mobile solution emerged, and we ambitiously had the brilliant idea to build a native iOS app using Swift. We used the native iOS map component. On that, we let users search for objects in the system, display these objects on different iOS background maps, and access detail information for each object, including images. Users could also perform actions on the objects, like taking photos or reporting issues. The app made it possible to load work orders and manage them through protocols tied to each specific object that were somehow related to the work order. The work assignments here range from painting over electrical testing to stability checks and other tasks. In this example, you see the small form the users needed to fill out for a stability test.

We had several issues with this iOS app. After the application was built, it was basically feature complete, so that no bigger development was planned from the city. Moreover, there were no additional Swift projects in the Kismet pipeline, but Swift was constantly changing. Furthermore, we didn't go through the App Store. We used ad hoc distribution, which you can do if you have a deployment count under 100 devices. So we were constantly rolling out updates to adjust certificate files because of new or changed devices and updates related to Swift changes. But one of the biggest challenges though was how our app communicated with the infrastructure. It communicated with the rest of the system as if it would be an intranet application, because that was what we did for years. We often heard the app is not working even when the issue wasn't on our end. Each time we had to first prove we weren't at fault. This was especially challenging with the service level understanding of the city IT department back then. For example, when they would make changes to firewall rules on a Friday at 2 p.m. and then leave for the weekend. So welcome on Elm Street. As you can imagine, we weren't too pleased with how things were going. Eventually, we managed to convince the client to make the switch to a progressive web app.

Luckily by then, we had gained experience with single page applications built using React and Leaflet and had developed various so-called topic maps, which are small apps that are trying to fill the gap between large GIS portals and basic slippy maps. Here are some examples of those topic maps. They reach from maps for environmental monitoring stations over city maps with points of interest to charging station maps for electromobility or heavy rain hazard maps. The app for the environmental alarm team or the citizen application for calculating stormwater fees based on sealed surfaces were also built using these components and many more. So we had the know-how to build these systems and we were excited to use these components we developed in the new Bayliss progressive web app.

The new PWA's design should be based on the existing app but with improvements in usability and with the change that it should use now maps and services from the city's own infrastructure, not the Apple Maps anymore. I will now walk you through the new PWA.

The user experience begins just as before with a log-in screen. Visually the layout remains almost the same as the iOS app. Users can set the focus to search only for objects in the center of the map. They can switch between three different map backgrounds. The background can be made more transparent to enhance the visibility of objects. The first small change behind the scenes is that the positioning search now utilizes the city's street directory. Also the presentation of information has been adjusted and is now always positioned in the same area on the screen. By the using components from the project I mentioned earlier, the topic maps, we were able to surpass the functionality of the iOS app without requiring too much effort.

As a major improvement the newly introduced data sheets now presents the information about the objects much more clearly simply because we have more space to use. However, the core functions had to be retained. Users had to configure their own navigation. Users can still capture photos for each object they find. They can report issues related to an object. They can select and view work orders and select the task they need to perform within such a work order. By eating our own dog food, we managed to address roughly 50% of the issues that we had with the old app. It also meant we no longer relied on that one guy who knew Swift a little. So we could spread the work more evenly.

But what about the rest? Well, as I stated earlier, the offline-first principle helped a lot. We implemented the offline-first principle in four different dimensions, PWA, offline maps, app data and sync instead of post. The PWA part was the easiest. It is pretty standard and you'll find plenty of instructions on it. We created a service worker for our app, allowing it to launch even with no or poor internet connection, adding some meter tags and a manifest file. And, well, it looks and feels like an app. But this is fairly common, so I won't go into detail here. Providing a map background without internet connection is certainly more interesting. After ditching the first idea of using a service worker to intercept the traffic, we found a solution using MapLibreGL app protocol.

Providing a map background without internet connection is certainly more interesting. After ditching the first idea of using a service worker to intercept the traffic, we found a solution using MapLibreGL app protocol. With that great feature, we were able to set a custom load type function. First, we need to store the 30 megabytes of vector tiles of the city of Ubatal in an IndexedDB. And then, with the help of this custom load type function, we can access the tiles without networking.

For a talk on a German conference, I prepared a GitHub project where you can see how this works in detail. The project illustrates the seven steps to an offline available map, including a running instance where you can skip through the steps. In Beelis, here's how offline access to the vector layer looks like. The user simply checks a box to enable it and then the archive is downloaded and saved in the database. That's it. Once done, the background map can be accessed entirely offline without needing an internet connection. You will see it in action later in the presentation.

The next level involves the application data. This means that we need to hold all object data in the browser. With all object data, I mean all the information about lights, cables, work orders, and so on. And to make the search possible, we also need to store a geometry index of all objects. We don't store images though in the offline mode because of the size. Then, when the app is in offline mode, data is no longer loaded directly from the live system but from this application cache or better application offline store. Here's how it looks in the app. Usually, before a user leaves his office to work on the streets, he has to update the cache. This update is a manual process and has to be initiated by the user. With the office Wi-Fi, the whole process takes less than a minute. Once updated, offline mode can be activated within the app.

The final layer of the offline first paradigm in Bayless is sync instead of post. In our communications with the endpoint in the iOS app, we didn't use traditional CRUD operations. Instead, we communicated through specific actions such as reporting a disruption or adding a picture. The challenge was that these actions were directly triggered by the user interface.

In our communications with the endpoint in the iOS app, we didn't use traditional CRUD operations. Instead, we communicated through specific actions such as reporting a disruption or adding a picture. The challenge was that these actions were directly triggered by the user interface. To enable this behavior offline, actions along with their payloads are now written into a table locally. This table is then synchronized with the server in our data center. When the synced table changes on the server, we make the right API calls and write the results back to the table on the server. The established synchronization process ensures that these updates are delivered back to the end device.

The offline core was implemented using RxDB and the Hasura PostgresQL combination. Here you see the schematics. On the left side you see our internet infrastructure with a public API endpoint. In the middle you see what we are calling our offline core, which is located in the data center that we are responsible of. And on the right side you see the end device which is the PWA running on an iPad. With that behavior, we addressed many issues that had to do with reliability. The app was getting more resilient against problems with either mobile connections or the infrastructure of the city or even downtimes of our APIs. But we introduced some new issues.

First of all, we had longer intermediate states. So we needed to visualize changes that exist exclusively on the end device at any time. We managed this with clear markings in the user interface. For example, if a number is normally generated on the server, we simply cannot show it before a roundtrip is made. In that case, we replaced the generated paths with stars. Secondly, we had to deal with potential conflicts. To maintain consistency, we simply execute all requests in their original order as if all services had been online the entire time. This method helps prevent conflicts between offline changes and the live system. The next point was the need for a detailed status feedback. Providing users with feedback on the successful saving of data is essential. Visual indicators in the user interface assist users in understanding the current status of their data, especially in offline mode. The last point is trust. To ease our conscience and provide a safety net, each end device can download the action table. This feature serves as a precaution in case something goes wrong. However, we've never had to use this backup feature, which highlights the stability and reliability of our system.

So what is the main takeaway here? First of all, offline-first doesn't stop just at setting up a progressive web app. The biggest part is it takes a significant effort. But the good news is, if you are going that extra mile, the users are very satisfied.

I want to mention as a side note, even though we are not restricted to them, the iPads remain the devices of choice. So no Android device at the moment in our field work.

Thank you so much for listening and thanks to React Summit US for having me. I hope our journey has been helpful for you all. Feel free to reach me out online if you have any questions. You'll find me, I'm sure. Thanks.