In the next release, we are going to load stdlib from memory instead of from external files, which will make the BuckleScript toolchain more accessible and performant. Loading stdlib from memory
You can try it via
npm i email@example.com
How does it work
When the compiler compiles a module
test.ml, the module
Test will import some modules from stdlib. This is inevitable since even basic operators in BuckleScript, for example
(+), are defined in the Pervasives module, which is part of the stdlib.
Traditionally, the compiler will consult
Pervasives.cmi, which is a binary artifact describing the interface of the Pervasives module and
Pervasives.cmj, which is a binary artifact describing the implementation of the Pervasives module.
Pervasives.cm[ij] and other modules in stdlib are shipped together with the compiler.
This traditional mode has some consequences:
- The compiler is not stand-alone and relocatable. Even if we have the compiler prebuilt for different platforms, we still have to compile stdlib post-installation.
postinstallis supported by npm, but it has various issues against yarn.
In this release, we solve the problem by embedding the binary artifacts into the compiler directly and loading it on demand.
To make this possible, we try to make the binary data platform agnostic and as compact as possible to avoid size bloating. The entrance of loading cmi/cmj has to be adapted to this new way.
So whenever the compiler tries to load a module from stdlib, it will consult a lazy data structure in the compiler itself instead of consulting an external file system.
What's the benefit?
More accessiblity.Package installation now becomes downloading for prebuilt platforms. In the future, we can make it installable from a system package manager as well. The subtle interaction with [yarn reinstall](https://github.com/BuckleScript/bucklescript/issues/2799) is also solved once and for all.
Easy separation between compiler and JS artifacts
The compiler is just one relocatable file. This makes the separation between the compiler and generated JS artifacts easier. The remaining work is mostly to design a convention between compiler and stdlib version schemes.
Yes, better compile performance
A large set of files is not loaded from the file system but rather from memory now!
Fast installation and reinstallation.
Depending on your network speed, the installation is reduced from 15 seconds to 3 seconds. Reinstallation is almost a no-op now.
JS playground is easier to build
We translate the compiler into JS so that developers can play with it in the browser. To make this happen, we used to fake the IO system; this not needed any more since no IO happens when compiling a single file to a string.
Some internal changes
To make this happen, the layout of binaries has been changed to the following structure. It is not recommended that users depend on the layout, but it happens. Here is the new layout:
|-- bsb // node wrapper of bsb.exe |-- bsc // node wrapper of bsc.exe | |-- win32 | |-- bsb.exe | |-- bsc.exe | |---darwin | |-- bsb.exe | |-- bsc.exe | |---linux | |-- bsb.exe | |-- bsc.exe