Fetch seed / RNG / attestation / user keys / appname / appversion from env at startup then stick with it

[lpascal-ledger]

Feature 1

Previously, seed, RNG and app name & version were fetched from the environment each time they were needed.
When several Speculos instances runs simultaneously in the same shell, the last declared env variable would overwrite the environment variable for every Speculos instance, and thus every Speculos instances would use the same value (same seed, same RNG, same app name / version).

Now these information are fetched once when Speculos starts, and stored internally. Successive Speculos instances no longer overwrite these information for former Speculos instances which are still running.

Feature 2

Attestation key and user private keys were not configurable.
Now they can be.

Implementation

These features are implemented in a environment.c module which embeds static variables to stores these values.
This module exposes an init_environment function, which is called at the start of Speculos (early in launcher.c:main, before the emulator is properly started). This function fetches, checks and sets the variables from the environment, or keeps them with their default values if they are not defined.

Module which need these values (bolos/endorsement.c, bolos/cx.c, bolos/os.c, bolos/os_eip2333.c and bolos/os_bip32.c) can use env_get_[seed|rng|user_private_key|user_certificate|app_tag] to access them.

[lpascal-ledger]

I hold this PR for some time as I'm going to have a very similar issue with the RNG.
Still I'm already interested with thoughts from @srasoamiaramanana-ledger as I'm going for a similar approach (only I'm going for an environment.[c|h] instead of seed.[c|h], and I'll rename [get|init]_seed with [get|init]_env_seed)

[xchapron-ledger]

@lpascal-ledger In that case, there is also app name and version which could be handled similarly.

[lpascal-ledger]

@xchapron-ledger This can be reviewed, I'm going to add unit tests but that's functionally ready.

[xchapron-ledger]

Can you remove this now noisy log: https://github.com/LedgerHQ/speculos/blob/master/src/bolos/cx_ec.c#L1655

[xchapron-ledger]

The launch output of Speculos get mixed info. I'm wondering if we can do something simple to fix it to ease readability?

[lpascal-ledger]

$ speculos ~/repos/apps/boilerplate/app/bin/app.elf 
Device model detected from metadata: nanos
17:53:08.094:speculos: Disabling OCR.
Warning: Ignoring XDG_SESSION_TYPE=wayland on Gnome. Use QT_QPA_PLATFORM=wayland to run on Wayland anyway.
[*] speculos launcher revision: 8e9897d
[*] using SDK version 6 on nanos
[*] loading CXLIB from "/home/lpascal/.virtualenvs/speculos/lib/python3.10/site-packages/speculos/cxlib/nanos-cx-2.1.elf"
[*] patching svc instruction at 0x126358
[*] Seed initialized from environment
[*] RNG initialized by default (time(NULL)): '1704905588'
[*] Private key ('ATTESTATION_PRIVATE_KEY') initialized with default value: '0x138fb9b91da745f12977a2b46f0bce2f0418b50fcb76631baf0f08ceefdb5d57'
[*] Private key ('USER_PRIVATE_KEY') initialized with default value: '0xe15e01d47082f0ea4771c99fe312f9d70093c89af47787fdf82e031f6728b710'
[*] User certificate 1 initialized (size: 71)
[*] Private key ('USER_PRIVATE_KEY') initialized with default value: '0xe15e01d47082f0ea4771c99fe312f9d70093c89af47787fdf82e031f6728b710'
 * Serving Flask app 'speculos.api.api'
 * Debug mode: off
17:53:08.302:werkzeug: WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on all addresses (0.0.0.0)
 * Running on http://127.0.0.1:5000
 * Running on http://192.168.1.15:5000
17:53:08.302:werkzeug: Press CTRL+C to quit
QPixmap::scaled: Pixmap is a null pixmap
[*] User certificate 2 initialized (size: 71)
[*] Env app name: 'Boilerplate'
[*] Env app version: '2.1.0'
[*] patching svc instruction at 0x4000204a
[*] patching svc instruction at 0x40002058

What's bothering you? Given the tool and the amount of features, it still seems reasonable to me.

We could drop "default" lines such as [*] RNG initialized by default (time(NULL)): '1704905588', implying it would only print something if not the default behavior?

As some logs comes from the Python server(s?) which are asynchronous vs. the emulator, it's hard to manage some of the logs order.

[xchapron-ledger]

What's bothering you? Given the tool and the amount of features, it still seems reasonable to me.

The fact that there are lines starting with *, [*] , timestamp, .. that are mixed, And would rather have them grouped.
But I'm probably being too picky?

[lpascal-ledger]

Right I see. [*] comes from the emulator, all the rest is from the Flask server and other Python threads. As different programs, I don't know if we could arrange their logs, except by artificially postponing the server startup by one or two seconds.

[lpascal-ledger]

Adding all this crypto stuff may have slowed the emulator startup enough for the Flask API to catchup and log on top of it, so this may be more mixed than it was.

[xchapron-ledger]

Yeap, not sure that postponing is a good thing. This will slow test everywhere...
But I preferred the previous logs...

[xchapron-ledger]

I'd say, let's remove logs that are mostly non necessary, so that output stay relevant, and let's merge.
Seed / RNG / Private Key / Certificate won't be necessary most of the time. Can we make them available only in a specific debug mode?

[lpascal-ledger]

$ speculos ~/repos/apps/boilerplate/app/bin/app.elf 
Device model detected from metadata: nanos
18:32:27.220:speculos: Disabling OCR.
Warning: Ignoring XDG_SESSION_TYPE=wayland on Gnome. Use QT_QPA_PLATFORM=wayland to run on Wayland anyway.
[*] speculos launcher revision: b253aa4
[*] using SDK version 6 on nanos
[*] loading CXLIB from "/home/lpascal/.virtualenvs/speculos/lib/python3.10/site-packages/speculos/cxlib/nanos-cx-2.1.elf"
[*] patching svc instruction at 0x126358
[*] Seed initialized from environment
 * Serving Flask app 'speculos.api.api'
 * Debug mode: off
18:32:27.368:werkzeug: WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on all addresses (0.0.0.0)
 * Running on http://127.0.0.1:5000
 * Running on http://192.168.1.15:5000
18:32:27.368:werkzeug: Press CTRL+C to quit
QPixmap::scaled: Pixmap is a null pixmap
[*] Env app name: 'Boilerplate'
[*] Env app version: '2.1.0'
[*] patching svc instruction at 0x4000204a
[*] patching svc instruction at 0x40002058

I've dropped quite some logs. Smaller, yet mixed up.

There's no logging mechanism in the emulator, only fprintf(stderr, ...), so hard to have DEBUG prints except by adding a whole log level mechanism - which is quite heavy to do.

And TBH, most of the logs here are not that interesting: either the values are the default one, and you can easily get them from the code if needed, or the user set them through the environment, and in this case it probably already has them in a variable or something.