Skip to content

Commit

Permalink
Update documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
actions-user committed Nov 30, 2023
1 parent 764c7bc commit b89d001
Show file tree
Hide file tree
Showing 85 changed files with 6,053 additions and 23 deletions.
2 changes: 1 addition & 1 deletion stable/.buildinfo
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Sphinx build info version 1
# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
config: 0ed8f9e64e0f14b1eea91837a98334a7
config: da86be50487b2a1819c228ce4d7c1d9c
tags: 645f666f9bcd5a90fca523b33c5a78b7
Binary file modified stable/.doctrees/commands/run.doctree
Binary file not shown.
Binary file modified stable/.doctrees/environment.pickle
Binary file not shown.
Binary file modified stable/.doctrees/methoddocs/application.doctree
Binary file not shown.
Binary file modified stable/.doctrees/methoddocs/exceptions.doctree
Binary file not shown.
Binary file modified stable/.doctrees/methoddocs/middlewares.doctree
Binary file not shown.
Binary file modified stable/.doctrees/methoddocs/runner.doctree
Binary file not shown.
Binary file modified stable/.doctrees/methoddocs/subscriptions.doctree
Binary file not shown.
Binary file modified stable/.doctrees/methoddocs/utils.doctree
Binary file not shown.
Binary file modified stable/.doctrees/userguides/development.doctree
Binary file not shown.
83 changes: 78 additions & 5 deletions stable/_sources/userguides/development.md.txt
Original file line number Diff line number Diff line change
Expand Up @@ -61,19 +61,64 @@ Any errors you raise during this function will get captured by the client, and r

## Startup and Shutdown

If you have heavier resources you want to load during startup, or otherwise perform some data collection prior to starting the bot, you can add a startup function like so:
### Worker Events

If you have heavier resources you want to load during startup, or want to initialize things like database connections, you can add a worker startup function like so:

```py
@app.on_startup()
@app.on_worker_startup()
def handle_on_worker_startup(state):
# Connect to DB, set initial state, etc
...

@app.on_worker_shutdown()
def handle_on_worker_shutdown(state):
# cleanup resources, close connections cleanly, etc
...
```

This function comes a parameter `state` that you can use for storing the results of your startup computation or resources that you have provisioned.
It's import to note that this is useful for ensuring that your workers (of which there can be multiple) have the resources necessary to properly handle any updates you want to make in your handler functions, such as connecting to the Telegram API, an SQL or NoSQL database connection, or something else.
The `state` variable is also useful as this gets made available to each handler method so other stateful quantities can be maintained for other uses.

TODO: Add more information about `state`
It's import to note that this is useful for ensuring that your workers (of which there can be multiple) have the resources necessary to properly handle any updates you want to make in your handler functions, such as connecting to the Telegram API, an SQL or NoSQL database connection, or something else. **This function will run on every worker process**.

*New in 0.2.0*: These events moved from `on_startup()` and `on_shutdown()` for clarity.

#### Worker State

The `state` variable is also useful as this can be made available to each handler method so other stateful quantities can be maintained for other uses. Each distributed worker has its own instance of state.

To access the state from a handler, you must annotate `context` as a dependency like so:

```py
from typing import Annotated
from taskiq import Context, TaskiqDepends

@app.on_(chain.blocks)
def block_handler(block, context: Annotated[Context, TaskiqDepends()]):
# Access state via context.state
...
```

### Application Events

You can also add an application startup and shutdown handler that will be **executed once upon every application startup**. This may be useful for things like processing historical events since the application was shutdown or other one-time actions to perform at startup.

```py
@app.on_startup()
def handle_on_startup(startup_state):
# Process missed events, etc
# process_history(start_block=startup_state.last_block_seen)
# ...or startup_state.last_block_processed
...


@app.on_shutdown()
def handle_on_shutdown():
# Record final state, etc
...
```

*Changed in 0.2.0*: The behavior of the `@app.on_startup()` decorator and handler signature have changed. It is now executed only once upon application startup and worker events have moved on `@app.on_worker_startup()`.

## Running your Application

Expand Down Expand Up @@ -101,6 +146,34 @@ If you configure your application to use a signer, and that signer signs anythin
Always test your applications throughly before deploying.
```

### Distributed Execution

Using only the `silverback run ...` command in a defualt configuration executes everything in one process and the job queue is completely in-memory with a shared state. In some high volume environments, you may want to deploy your Silverback application in a distributed configuration using multiple processes to handle the messages at a higher rate.

The primary components are the client and workers. The client handles Silverback events (blocks and contract event logs) and creates jobs for the workers to process in an asynchronous manner.

For this to work, you must configure a [TaskIQ broker](https://taskiq-python.github.io/guide/architecture-overview.html#broker) capable of distributed processing. For instance, with [`taskiq_redis`](https://github.com/taskiq-python/taskiq-redis) you could do something like this for the client:

```bash
export SILVERBACK_BROKER_CLASS="taskiq_redis:ListQueueBroker"
export SILVERBACK_BROKER_URI="redis://127.0.0.1:6379"

silverback run "example:app" \
--network :mainnet:alchemy \
--runner "silverback.runner:WebsocketRunner"
```

And then the worker process with 2 worker subprocesses:

```bash
export SILVERBACK_BROKER_CLASS="taskiq_redis:ListQueueBroker"
export SILVERBACK_BROKER_URI="redis://127.0.0.1:6379"

silverback worker -w 2 "example:app"
```

This will run one client and 2 workers and all queue data will be go through Redis.

## Testing your Application

TODO: Add backtesting mode w/ `silverback test`
Expand Down
4 changes: 3 additions & 1 deletion stable/commands/run.html
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@
<option value="">Select version...</option>
<option value="latest">latest</option>
<option value="stable">stable</option>
<option value="v0.1.1">v0.1.1</option>
<option value="v0.1.0">v0.1.0</option>
</select>
</div>
Expand Down Expand Up @@ -111,6 +112,7 @@
<h1>run<a class="headerlink" href="#run" title="Permalink to this heading"></a></h1>
<section id="run">
<h2>run<a class="headerlink" href="#run" title="Permalink to this heading"></a></h2>
<p>Run Silverback application client</p>
<div class="highlight-shell notranslate"><div class="highlight"><pre><span></span>run<span class="w"> </span><span class="o">[</span>OPTIONS<span class="o">]</span><span class="w"> </span>PATH
</pre></div>
</div>
Expand All @@ -133,7 +135,7 @@ <h2>run<a class="headerlink" href="#run" title="Permalink to this heading"></
<dd><p>Override the default network and provider. (see <cite>ape networks list</cite> for options)</p>
<dl class="field-list simple">
<dt class="field-odd">Options<span class="colon">:</span></dt>
<dd class="field-odd"><p>:mainnet:geth | ethereum:mainnet:geth | :mainnet | ethereum:mainnet | :goerli:geth | ethereum:goerli:geth | :goerli | ethereum:goerli | :sepolia:geth | ethereum:sepolia:geth | :sepolia | ethereum:sepolia | ::test | :local:test | ethereum::test | ethereum:local:test | ::geth | :local:geth | ethereum::geth | ethereum:local:geth | :local | ethereum:local | ethereum</p>
<dd class="field-odd"><p>:mainnet:geth | ethereum:mainnet:geth | :mainnet | ethereum:mainnet | :goerli:geth | ethereum:goerli:geth | :goerli | ethereum:goerli | :sepolia:geth | ethereum:sepolia:geth | :sepolia | ethereum:sepolia | ::geth | :local:geth | ethereum::geth | ethereum:local:geth | ::test | :local:test | ethereum::test | ethereum:local:test | :local | ethereum:local | ethereum</p>
</dd>
</dl>
</dd></dl>
Expand Down
21 changes: 18 additions & 3 deletions stable/genindex.html
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@
<option value="">Select version...</option>
<option value="latest">latest</option>
<option value="stable">stable</option>
<option value="v0.1.1">v0.1.1</option>
<option value="v0.1.0">v0.1.0</option>
</select>
</div>
Expand Down Expand Up @@ -214,10 +215,14 @@ <h2 id="G">G</h2>
<table style="width: 100%" class="indextable genindextable"><tr>
<td style="width: 33%; vertical-align: top;"><ul>
<li><a href="methoddocs/application.html#silverback.application.SilverbackApp.get_block_handler">get_block_handler() (silverback.application.SilverbackApp method)</a>
</li>
<li><a href="methoddocs/application.html#silverback.application.SilverbackApp.get_event_handler">get_event_handler() (silverback.application.SilverbackApp method)</a>
</li>
</ul></td>
<td style="width: 33%; vertical-align: top;"><ul>
<li><a href="methoddocs/application.html#silverback.application.SilverbackApp.get_event_handler">get_event_handler() (silverback.application.SilverbackApp method)</a>
<li><a href="methoddocs/application.html#silverback.application.SilverbackApp.get_shutdown_handler">get_shutdown_handler() (silverback.application.SilverbackApp method)</a>
</li>
<li><a href="methoddocs/application.html#silverback.application.SilverbackApp.get_startup_handler">get_startup_handler() (silverback.application.SilverbackApp method)</a>
</li>
</ul></td>
</tr></table>
Expand All @@ -226,6 +231,10 @@ <h2 id="H">H</h2>
<table style="width: 100%" class="indextable genindextable"><tr>
<td style="width: 33%; vertical-align: top;"><ul>
<li><a href="methoddocs/exceptions.html#silverback.exceptions.Halt">Halt</a>
</li>
</ul></td>
<td style="width: 33%; vertical-align: top;"><ul>
<li><a href="methoddocs/utils.html#silverback.utils.hexbytes_dict">hexbytes_dict() (in module silverback.utils)</a>
</li>
</ul></td>
</tr></table>
Expand Down Expand Up @@ -280,11 +289,15 @@ <h2 id="O">O</h2>
</li>
<li><a href="methoddocs/middlewares.html#silverback.middlewares.SilverbackMiddleware.on_error">on_error() (silverback.middlewares.SilverbackMiddleware method)</a>
</li>
</ul></td>
<td style="width: 33%; vertical-align: top;"><ul>
<li><a href="methoddocs/application.html#silverback.application.SilverbackApp.on_shutdown">on_shutdown() (silverback.application.SilverbackApp method)</a>
</li>
</ul></td>
<td style="width: 33%; vertical-align: top;"><ul>
<li><a href="methoddocs/application.html#silverback.application.SilverbackApp.on_startup">on_startup() (silverback.application.SilverbackApp method)</a>
</li>
<li><a href="methoddocs/application.html#silverback.application.SilverbackApp.on_worker_shutdown">on_worker_shutdown() (silverback.application.SilverbackApp method)</a>
</li>
<li><a href="methoddocs/application.html#silverback.application.SilverbackApp.on_worker_startup">on_worker_startup() (silverback.application.SilverbackApp method)</a>
</li>
</ul></td>
</tr></table>
Expand All @@ -304,6 +317,8 @@ <h2 id="P">P</h2>
</ul></td>
<td style="width: 33%; vertical-align: top;"><ul>
<li><a href="methoddocs/middlewares.html#silverback.middlewares.SilverbackMiddleware.post_execute">post_execute() (silverback.middlewares.SilverbackMiddleware method)</a>
</li>
<li><a href="methoddocs/middlewares.html#silverback.middlewares.SilverbackMiddleware.post_save">post_save() (silverback.middlewares.SilverbackMiddleware method)</a>
</li>
<li><a href="methoddocs/middlewares.html#silverback.middlewares.SilverbackMiddleware.pre_execute">pre_execute() (silverback.middlewares.SilverbackMiddleware method)</a>
</li>
Expand Down
1 change: 1 addition & 0 deletions stable/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@
<option value="">Select version...</option>
<option value="latest">latest</option>
<option value="stable">stable</option>
<option value="v0.1.1">v0.1.1</option>
<option value="v0.1.0">v0.1.0</option>
</select>
</div>
Expand Down
Loading

0 comments on commit b89d001

Please sign in to comment.