[w3c/IndexedDB] Backward-compatible schema changes are very hard (#282)

Use case: Support making backwards-compatible changes to the schema without requiring users to reload existing tabs.

Details: Right now, any time you need to make schema changes you have to bump the version number on the database. Generally speaking this requires existing tabs to be reloaded.

This makes a lot of sense when a schema change makes backwards incompatible changes to the database. I.e. where the code running in existing tabs would not be able to correctly read or write data to the database after the schema change.

However most schema changes are likely backwards compatible. For example they add an object store or an index.

In this case existing code in any existing tabs would be able to keep using the database ignoring the new objectStores/indexes.

It is technically possible to deploy code that uses indexedDB already with the current API.

You could for example use the following code to open the database:
```JS
let dbVersion = 1
let db = undefined
let dbPromise = undefined
function openDatabase() {
  if (!dbPromise) {
    dbPromise = new Promise((resolve, reject) => {
      const dbRequest = indexedDB.open("mydb", dbVersion)
      dbRequest.onupgradeneeded = () => { ... }
      dbRequest.onsuccess = () => {
        let db = dbRequest.result
        db.onversionchange = (event) => {
          if (event.newVersion && event.newVersion < 1000) {
            dbVersion = event.newVersion
            db.close()
            dbPromise = undefined
          }
        }
      }
      resolve(db)
    }
  }
  return dbPromise
}
```

In this code the page assume that any versions below 1000 are versions that have a schema compatible with the code in this tab. So if we get a versionchange event to such a version we remember that version number, close and then reopen the database using this new version.

This way when you deploy new IndexedDB-using code, you can make that code upgrade to version 2 if they know version 2 is backwards compatible. If you need to make a backwards incompatible schema change you can use version 1000 which will cause existing tabs to hold on to their database connections and prevent dataloss or data corruption, but will require those tabs to be closed/reloaded.

However this code has at least two subtle bugs:

* If the tab that tries to update the plugin fails to do the update (due to IO error or other bugs), then the existing tabs will now open the database with the updated version number thus causing them to bump the version number. And they will do so without making the schema changes that should go along with that update.
* If the database in the old tab isn't opened upon page load, then it won't receive the version update notifications and so will just receive an error when trying to open the database with the old version number. And even if you do open the database during page load, you are likely racing with a new tab being opened which uses the new version.

Fixing these problems is doable but quite tricky.

Another way to approach this problem is to use BroadcastChannel to send out messages about when a backwards compatible change to the schema is about to happen. This message can contain information about what the new version number is. But again there seems to be tricky edge cases involved in implementing this approach.


What is even worse is that websites are required to deploy this logic with the initial release of their IndexedDB usage. I.e. you are required to realize ahead of time that you should think about how to support both backwards compatible and backwards incompatible version changes.

The reality is likely that most websites do not implement a solution like this when they first roll out IndexedDB. 

It would be great to have some solution in the API specifically for allowing a backward compatible schema change. This could help in several ways:
* Existing connections to the database could remain open and undisrupted (possibly other than that no trasactions could start until the schema change is done)
* Existing tabs which try to open using the "old" schema version could be allowed to succeed
* Users wouldn't have to think about how to handle backward compatible schema changes until they actually make one.

We could even enable websites to treat the first version upgrade as a backward compatible schema change (since any schema is compatible with "database not used at all"). So this could be an opportunity to create schema-changing API which is friendlier than the current API.

If we did this then the version handling in IndexedDB could feel much more smooth. For "backward compatible" changes, including creating the database in the first place, a simpler API could be used.

For backward incompatible changes users would still have to use the current heavy-handed API. But would arguably make more sense since for backward incompatible changes you have to force existing tabs to be reloaded or at least stop using the database.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/w3c/IndexedDB/issues/282