Architecture of optimized sync of JSON data to a (cloud) server

I’m building an application that generates JSON data based on the user’s notes. The notes are rich data that contain task lists and other types of elements inside it, so I use JSON format to store it. It currently saves this data to the browser’s local storage using JSON.stringify.

Now I’m building a feature that saves this data to a cloud server, but since the nature of the notes is that they constantly change, sending a big stringified object to the server at every note update (even if it has a 5-second debounce) seems a bit crazy to me. Depending on the size of the note, I’d have some 300kb calls or more to the server for a simple typo correction. Not good.

So I’ve looked into some services that support rich data JSON storage, such as Google’s Cloud Firestore or Amazon’s DynamoDB. Both look promising, but I’m still trying to wrap my head around my architecture for this. Here’s my plan so far:

  1. I’ll read the JSON from the server and keep a local object.
  2. At every local change, I’ll analyze the difference down the object tree and make some sort of diff string
  3. I’ll send only this diff string so the server can analyze and update its record.

Well, that seems to work. I would send a few bytes requests to the server, and not 300kb. It would require some code to analyze and generate the diff, but ok so far.

Now the server part is where my challenge is. Both Cloud Firestore and DynamoDB have rich data change functions, but it seems that for both, these operations must be pre-established according to the data itself, in other words, you must know the dataset in advance in order to make operations on it. Since my notes’ JSON are a dynamic dataset, I have no way of writing custom updates and inserts on it, it must be dynamically calculated via some sort of diff mechanism.

My questions are:

  1. Is this a good way to approach this problem or is there something I could be doing that would make things better/easier?
  2. Are there any cloud services, NoSQL databases, query languages, or libraries that would allow for this type of optimized dynamic JSON document update architecture?
  3. Would this be a good use case for GraphQL?

Go to Source
Author: raphadko