October 25, 2019

Sync localStorage into the Cloud

We build a remote storage interface for the Web Storage API, so you can persist localStorage into a remote key-value store.

The Web Storage API provides a mechanism for developers to persist key-value data associated with the current page’s origin. It’s a significant improvement to using client-side cookies to persist bits of data, typically allowing storage of up to 5 MB of data. The API is simple:

localStorage.setItem(‘color’, ‘blue’)
localStorage.getItem(‘color’) // ‘blue’
localStorage.removeItem(‘color’)

A few examples of where you could use localStorage:

  • Store UI settings (filters, state of the UI) that don’t strictly need to be persisted on the server
  • Data store for offline apps that don’t need a backend
  • Share state across different IFrames on the same page
  • Cache data for offline use or improve performance

In some cases, you may not want to create an API or persistence layer on the server, but still want to save the contents of localStorage into the cloud. Imagine a web app that has a search UI that lets the user select some filters. From a UX point of view, you may want to persist the state of the filter settings the user has chosen so they don’t need to reselect them every time. This could very well be stored in localStorage only and not complicate server-side logic. But what if the user opens the web app from another device? It would improve the user experience if the filter settings were also available there.

In this example, essentially, what we’d like to do is sync localStorage to the cloud so it can be fetched anytime the frontend needs it. Fortunately, the Web Storage API is simple enough that we can create a wrapper object that is API-compatible with it, so it’s easy to swap out the implementation, and also persist it into an online key-value storage service.

However, one downside of the Web Storage API is that it only offers a synchronous API. Using a remote storage service requires an asynchronous network round-trip, which makes designing a fully-compatible API difficult due to restrictions on synchronous network I/O, particularly in browsers. For this reason, instead of hacking together a synchronous API emulation, let’s implement an asynchronous API that is async-aware and Promises-friendly.

Let’s use our own kvdb.io-commonjs module to build it!

class KVdbWebStorage {
  constructor(bucket) {
    this.bucket = bucket
  }

  // When passed a number n, this method will return the name of the nth key in the storage.
  async key(index) {
    return this.bucket.list({skip: index, limit: 1})
      .then(keyName => keyName)
      .catch(err => null)
  }

  // When passed a key name, will return that key's value.
  async getItem(keyName) {
    return this.bucket.get(keyName)
      .then(keyValue => keyValue)
      .catch(err => null)
  }

  // When passed a key name and value, will add that key to the storage, or update that key's value if it already exists.
  async setItem(keyName, keyValue) {
    return this.bucket.set(keyName, keyValue)
  }

  // When passed a key name, will remove that key from the storage.
  async removeItem(keyName) {
    return this.bucket.delete(keyName)
  }

  // Returns an integer representing the number of data items stored in the Storage object.
  async length() {
    return this.bucket.list()
      .then(keys => keys.length)
      .catch(err => -1)
  }

  // When invoked, will empty all keys out of the storage.
  async clear() {
    const keys = await this.bucket.list()
    const waiting = keys.map(key => this.bucket.delete(key))
    return Promise.all(waiting)
  }
}

Using it is as easy as with localStorage, just instantiate it with your KVdb bucket and API key:

const bucket = KVdb.bucket(‘MY_BUCKET’, ‘MY_SECRET’)
const kvdbStorage = new KVdbWebStorage(bucket)

kvdbStorage.setItem(‘foo’, ‘bar’)
  .then(() => kvdbStorage.getItem(‘foo’))
  .then(value => console.log(‘get: ‘, value))
  .catch(err => console.error(‘error: ‘, err))

While it’s possible to treat localStorage as an object and directly set keys on it, it’s recommended to use the setter and getter methods instead, to allow for asynchronous storage backends such as the one we built above.

In fact, the popular localForage library builds on this approach and exposes a localStorage-like interface with support for different drivers, or storage backends such as localStorage, IndexedDB, and custom ones. The latest version of the kvdb.io-commonjs module has built-in support for localForage, making it easy to use KVdb as the backend:

<script src="localforage.js"></script>
<script src="https://unpkg.com/kvdb.io@v1.0"></script>
<script>
KVdb.installLocalForageDriver(localforage)

localforage.config({
  bucket: KVdb.bucket('MY_BUCKET_ID', 'MY_ACCESS_TOKEN')
})

localforage.setDriver([KVdb.LOCALFORAGE_DRIVER])
  .then(() => localForage.setItem('foo', 'bar'))
  .then(() => localForage.getItem('foo'))
  .then(value => alert('value: ' + value))
</script>

In summary, we’ve seen how to implement a Web Storage API wrapper that is backed by KVdb and how minimally disruptive it is to maintain API compatibility with localStorage. And if you’re already using localStorage and want a vendor-neutral approach to persisting it into the cloud, check out localForage and its API.


Tags: Getting Started Web Storage localStorage Client-Side


Previous post
Why You Should Ditch Your Database for a Key-Value DB We explore using a key-value store as your main database instead of a relational database and how it makes building applications simpler and faster.
Next post
Sorting Ordered Data in a Key-Value Store We discuss key naming in the context of lexicographical sort order, which most key-value systems use, and how to design keys accordingly.

Like what you see? Give KVdb a spin.