sql.js size optimization

[lalkmim]

Not exactly a issue, more of a question/idea. I'm trying for a serverless application, and I need to run a few complex queries (tree-like and such). The data would be imported from another source on load time and there would be no need to maintain it, just query. In the end, I would need a sql.js that could:

• create tables (no need to alter/drop);
• insert data (no need to update/delete);
• run complex query on low volume data (few thousand lines at most - no need for views, but temp tables, indexes and subqueries needed).

Do you know if these are achievable on compile time? And would you consider making a sql.js version with such features? If possible, I imagine this could shrink the sql.js a lot.

[lovasoa]

Yes, it would be possible, and I encourage you to try it and make pull requests.
See http://www.sqlite.org/footprint.html#relfootprint for a list of interesting compile-time options.

You will first need to compile the amalgamation (sqlite3.c), and than compile it to sql.js with the right options.

[lovasoa]

But don't expect it to cut the size of sql.js down by more than a half... (I bet you won't pass under 1Mb).

[lalkmim]

Thanks, I'll give it a go during this week.

[lovasoa]

Perfect :)
Don't hesitate to ask for help if you have any problem.

I recently fixed a bug in emscripten that prevented sql.js from compiling, so you might have to use the incoming branch of emscripten ( https://github.com/kripken/emscripten ) for the last part of the compilation.

[lovasoa]

I started some work on this
https://github.com/kripken/sql.js/issues/75

Currently I'm only removing the options that are really not useful in sql.js. You can remove even more options for your read-only use case.

[lalkmim]

Great! I'll probably dig into this tomorrow.

[lalkmim]

By the way, I've built a small library to import from spreadsheets (so far only google spreadsheets) to a sql.js database, all online. It was part of another project. http://github.com/lalkmim/sheets2sqlite

[lovasoa]

Very interesting! I'm always glad to learn about interesting uses of sql.js.

[lovasoa]

You can see the work on the sizeopts branch.