SQLite offers through its JSON1 extension a lot of capability. Generating JSON outputs from SQL, even complex structured, multi-level JSON is very straightforward.
But what about the opposite use case ? When you have store data as JSON in an SQLite database, and you want to search or filter it ? This is a short exploration of what to expect.
Despite having “lite” in its name, SQLite can handle very large databases quite efficiently. The single largest I am currently dealing with just passed 100 GB, and occasionally when backing it up with the SQLite standard Command Line utility it would take many hours.
After some asking on the sqlite-users list (thanks Clemens Madish & Simon Slavin), and some experimenting, I have gotten the backup time down to 4-5 minutes on a live 100 GB database, which is just fine in my book. Below are the findings and an utility, sql3bak, which wraps the results of the findings.