v0.10.6

• small fixes

v0.10.4

Mostly small bugfixes. Such as:

• raise NotImplementedError when trying to use dropout in neural_ode_model/controller

Also changes isort to isort --profile google, and adds

• constant_value_controller_wrapper

v0.9.0

Removes acme as a dependency. Only its base-classes and type definitions were used anyways. This way we no longer require tensorflowand launchpad

This also means that it should work just fine under macOS and Windows, in addition to Linux

v0.8.0

• adds convenience functions save_eqx and load_eqx to better store trained models / controllers
• adds new environment ackermann which is the only environment to not use MuJoCo. It is implemented using equations. It simulates a driving car with Ackermann Steering
• adds new module circus.py that bundles useful reference signal generation functions
• adds a new EnvLoopObserver that de-noises the output / measurement of a noisy environment
• adds convenice function l1_l2_regularisers
• adds ability for dynamic markers using the scene_callback argument of VideoWrapper
• adds ability to periodically reset the controller in the interactive viewer (simple way of ensure that it stays stable even for longer time horizons)

v0.7.0

Breaking:

• renames collect_random_step_source to random_steps_source (better name as it does not really use collect function)
• renames muslce_siso environment to muscle_cocontraction

New:

• adds new environment muscle_asymmetric

v0.6.1

minor bump. only bugfixes

v0.6.0

Breaking changes:

• env.control_timestep becomes env.control_timestep()

• env.ts is no longer an attribute, instead use

from cc.utils.utils import timestep_array_from_env
ts = timestep_array_from_env(env)

New features:

• Adds two new SISO environments rover and muscle. You can check out a small introduction and some animations here

• Reworked the RecordVideoWrapper which is now called VideoWrapper. It no longer writes images to disk but creates videos directly from a list of frames.

• Replaced tree_concat by tree_batch; It's faster

• Adds support for Python=3.10

• Reworked some env-related wrapping logic. TrackTimeWrapper and ControlTimestepWrapper are no longer required

• setup.py now directly uses acme@main. However, acme=0.4.0 still works.

v0.5.0

• Merges PR #1, #2

This allows to create many (different and non colliding) double pendulums as part of a single environment programmatically.
Multiple controllers can be wrapped into one controller which can control the many double pendulums environment.
For feasible reference trajectories the corresponding feedforward controller can be incorporated to visually create the moving reference pendulum position.

• Moves tree.py into seperate repository tree_utils

v0.4.0

Controller training now supports multiple models.

Example:

ModelControllerTrainer(
    model = {
        "model1": fitted_model1,
        "model2": fitted_model2,
    },
    controller=controller,
    # Note the `model1` name
    trackers = [Tracker("model1", "train_mse")]
)

To get a scalar loss value, the TrainingOptionsController have gained a new option called loss_fn_reduce_along_models.
By default it is:

NAME_AND_VALUE = dict[str, float]

def default_loss_fn_reduce_along_models(
    log_of_loss_values: dict[str, NAME_AND_VALUE]
) -> NAME_AND_VALUE:
    flat_logs = batch_concat(log_of_loss_values, 0)
    return {"loss_without_regu": jnp.mean(flat_logs)}

We could provide a custom implementation (extending the above example)

def our_loss_fn_reduce_along_models(log_of_loss_values):
    train_mse1 = log_of_loss_values["model1"]["train_mse"]
    train_mse2 = log_of_loss_values["model2"]["train_mse"]
    return {"my_loss": (train_mse1 + train_mse2) / 2}

Then, we could also track on this metric by using

ModelControllerTrainer(
    model = {
        "model1": fitted_model1,
        "model2": fitted_model2,
    },
    controller=controller,
    controller_train_options=my_options,
    trackers = [Tracker("my_loss")]
)

v0.3.0

Breaking changes

• The functions collect, collect_exhaust_source and sample_feedforward_collect_and_make_source return one additional object. The additional object is a dictionary containing information of the unrolling process of the environment.

# BEFORE
sample = collect(env, controller)
# AFTER
sample, loop_result = collect(env, controller)

• The function make_dataloader expects either SupervisedDataset, SupervisedDatasetWithWeights or UnsupervisedDataset instead of PyTree. These three new dataset objects are simply NamedTuple's.

# BEFORE
make_dataloader((X, y),...)
# AFTER
make_dataloader(SupervisedDataset(X,y),...)

New features

• loop_observer-module. They can be used to record additional trial information such as physical quantities like angles over time. Works with collect, collect_exhaust_source and sample_feedforward_collect_and_make_source. Output is stored in the new dictionary output (see above).

• LOSS_FN_MODEL in combination with SupervisedDatasetWithWeights can be used to weigh the loss term during model training on a per-sample basis.