Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Misc upgrades #8

Merged
merged 11 commits into from
Nov 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,6 @@ cython_debug/
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/
.idea/

src/ezmsg/blackrock/__version__.py
13 changes: 6 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,13 @@

Interface for Blackrock Cerebus ecosystem (incl. Neuroport) using `pycbsdk`

NOTE: Currently only supports spikes, but a PR for continuous data as `ezmsg.util.messages.axisarray.AxisArray` wouldn't be a heavy lift and a readily accepted PR :D

## Installation
`pip install git+https://github.com/CerebusOSS/ezmsg-blackrock`

## Dependencies

* `pycbsdk`, which requires Python >=3.9
* `python` >=3.9
* `pycbsdk`
* `ezmsg-sigproc`

## Setup (Development)
Expand All @@ -20,7 +19,7 @@ NOTE: Currently only supports spikes, but a PR for continuous data as `ezmsg.uti

## Setup Notes

__IMPORTANT NOTE:__ _`pycbsdk` is only compatible with Central Neuroport/Cerebus Suite/NSP Firmware `7.0.6+`, built using `cbhwlib`/hardware library/network protocol `3.11+`. Anything older is not supported by `pycbsdk`_. Check this requirement in Central by navigating to `Help > About Central`
__IMPORTANT NOTE:__ _`pycbsdk` is only compatible with Central Neuroport/Cerebus Suite/NSP Firmware `7.0.5+`, built using `cbhwlib`/hardware library/network protocol `3.11+`. Anything older is not supported by `pycbsdk`_. Check this requirement in Central by navigating to `Help > About Central`

Blackrock Neuroport/Cerebus uses UDP multicast traffic to deliver upwards of 30000 packets/sec of data with low latency. There are better ways to do this now, but back when this hardware was made, the engineers were dealing with the limitations of the hardware they had. UDP comes with no packet delivery guarantee, but dropped packets can be catastrophic when interfacing with the Blackrock hardware. As such, do your best to locate the highest quality networking equipment you can find, and have a high tolerance for blaming ethernet adapters and/or your router/switch for errors during troubleshooting.

Expand All @@ -32,7 +31,7 @@ This setup describes a setup with a Legacy NSP, a Windows PC (called "Central PC

Central cannot run on the same PC simultaneously as `ezmsg-blackrock` because `pycbsdk` exclusively binds the port that central would use to communicate with the NSP. The NSP has too many settings to implement in `ezmsg-blackrock`, so in practice, we use Central to configure the NSP then acquire live data thereafter. If you want to, you _can_ use `ezmsg-blackrock` on the Central PC once you've configured the NSP and closed Central. When you do this, the Central PC and the Client PC are the same PC/IP address. Its assumed that this PC is running windows because Central Suite is only available on Windows.

Find the highest quality ethernet (yes, wired) network switch or router that you can. Look for 1+ Gigabit speed ratings (2.5 Gigabit better, 10 Gigabit is best), and features like QoS (Quality of Service) that can guarantee packet delivery. Ensure both the Central PC and the Client PC have high quality ethernet network adapters. Not every ethernet adapter will work, due to packet buffering requirements. High quality ethernet cables are also important; aim for Cat6E or better.
Find the highest quality ethernet (yes, wired) network switch or router that you can. Look for 1+ Gigabit speed ratings (2.5 Gigabit better, 10 Gigabit is best), and features like QoS (Quality of Service) that can guarantee packet delivery. Ensure both the Central PC and the Client PC have high quality ethernet network adapters. Not every ethernet adapter will work, due to packet buffering requirements. High quality ethernet cables are also important; aim for Cat6E or better.

1. Attach the NSP, Central PC, and Client PC to the router or switch using ethernet cables.
1. Blackrock NSP and Gemini Hubs use hard-coded static IP addresses.
Expand Down Expand Up @@ -71,7 +70,7 @@ However, if your intention is for the emulation PC to be a different machine tha

* Use Central to set up the NSP hardware.
* If running Central and `ezmsg-blackrock` on the same PC, shut down Central
* Assuming `pycbsdk` is installed in your python environment (which it should be, considering it's a dependency of `ezmsg-blackrock` that automatically gets installed with a `pip install -e .`), you should have the `pycbsdk_print_rates` command on your path. Test the connection and hardware/software setup using `pycbsdk_print_rates --inst_addr 192.168.137.128 --inst_port 51001 --protocol 3.11`
* Assuming `pycbsdk` is installed in your python environment (which it should be, considering it's a dependency of `ezmsg-blackrock` that automatically gets installed with a `pip install -e .`), you should have the `pycbsdk-rates` command on your path. Test the connection and hardware/software setup using `pycbsdk-rates --inst_addr 192.168.137.128 --inst_port 51001 --protocol 3.11`
* Note that this command line is for Legacy NSP with hard-coded address `192.168.137.128` running on port 51001
* Also note that network protocol is not automatically detected. Our old hardware can only be updated to firmware `7.0.6` which uses hardware library/protocol `3.11`. This can be checked by opening Central and navigating to "About Central" which will tell you which hardware library the firmware is running (assuming your NSP's firmware and Central version match). As of this writing, most Gemini hardware should be using protocol 4.1 (default).

Expand All @@ -82,4 +81,4 @@ However, if your intention is for the emulation PC to be a different machine tha
* Make sure the Central PC and the Client PC can ping each other and the NSP.
* Ensure NSP Firmware matches Central version.
* Ensure the protocol version you specify matches the firmware's protocol version.
* `pycbsdk_print_rates` and Wireshark + [a dissector](https://github.com/CerebusOSS/CerebusWireshark) are your friends. Godspeed.
* `pycbsdk-rates` and Wireshark + [a dissector](https://github.com/CerebusOSS/CerebusWireshark) are your friends. Godspeed.
136 changes: 65 additions & 71 deletions examples/demo.py
Original file line number Diff line number Diff line change
@@ -1,86 +1,80 @@
import sys

import ezmsg.core as ez

from ezmsg.blackrock.nsp import NSPSource, NSPSourceSettings
from ezmsg.util.debuglog import DebugLog
import typer
from typing_extensions import Annotated

if __name__ == "__main__":
import argparse

parser = argparse.ArgumentParser(description="Consume data from NSP")

parser.add_argument(
"--inst_addr",
"-i",
type=str,
default="192.168.137.128",
help="ipv4 address of device. pycbsdk will send control packets to this address. Subnet OK. "
"Use 127.0.0.1 for use with nPlayServer (non-bcast). "
"The default is 0.0.0.0 (IPADDR_ANY) on Mac and Linux. On Windows, known IPs will be searched.",
)

parser.add_argument(
"--inst_port",
type=int,
default=51001,
help="Network port to send control packets."
"Use 51002 for Gemini and 51001 for Legacy NSP.",
)

parser.add_argument(
"--client_addr",
"-c",
type=str,
default="",
help="ipv4 address of this machine's network adapter we will receive packets on. "
"Defaults to INADDR_ANY. If address is provided, assumes Cerebus Subnet.",
)
from ezmsg.blackrock.nsp import NSPSource, NSPSourceSettings

parser.add_argument(
"--client_port",
"-p",
type=int,
default=51002,
help="Network port to receive packets. This should always be 51002.",
)

parser.add_argument(
"--recv_bufsize",
"-b",
type=int,
help=f"UDP socket recv buffer size. "
f"Default: {(8 if sys.platform == 'win32' else 6) * 1024 * 1024}.",
def main(
inst_addr: Annotated[
str,
typer.Option(
help="ipv4 address of device. pycbsdk will send control packets to this address. Subnet OK. Use 127.0.0.1 "
"for use with nPlayServer (non-bcast). The default is 0.0.0.0 (IPADDR_ANY) on Mac and Linux. On "
"Windows, known IPs will be searched."
),
] = "192.168.137.128",
inst_port: Annotated[
int,
typer.Option(
help="Network port to send control packets. Use 51002 for Gemini and 51001 for Legacy NSP."
),
] = 51001,
client_addr: Annotated[
str,
typer.Option(
help="ipv4 address of this machine's network adapter we will receive packets on. Defaults to INADDR_ANY. "
"If address is provided, assumes Cerebus Subnet."
),
] = "",
client_port: Annotated[
int,
typer.Option(
help="Network port to receive packets. This should always be 51002."
),
] = 51002,
recv_bufsize: Annotated[
int, typer.Option(help="UDP socket recv buffer size.")
] = (8 if sys.platform == "win32" else 6) * 1024 * 1024,
protocol: Annotated[
str, typer.Option(help="Protocol Version. 3.11, 4.0, or 4.1 supported.")
] = "3.11",
cont_buffer_dur: Annotated[
float,
typer.Option(
help="Duration of buffer for continuous data. Note: buffer may occupy ~15 MB / second."
),
] = 0.5,
):
source_settings = NSPSourceSettings(
inst_addr,
inst_port,
client_addr,
client_port,
recv_bufsize,
protocol,
cont_buffer_dur,
)

parser.add_argument(
"--protocol",
type=str,
default="3.11",
help="Protocol Version. 3.11, 4.0, or 4.1 supported.",
)
comps = {
"SRC": NSPSource(source_settings),
# TODO: SparseResample(fs=1000.0, max_age=0.005),
# TODO: EventRates(),
"SPKLOG": DebugLog(name="SPK"),
"GRPLOG": DebugLog(name="GRP"),
}

parser.add_argument(
"--cont_smp_group",
type=int,
default=0,
help="Continuous data Sampling Group (1-6) to publish. Set to 0 to ignore continuous data.",
conns = (
(comps["SRC"].OUTPUT_SPIKE, comps["SPKLOG"].INPUT),
(comps["SRC"].OUTPUT_SIGNAL, comps["GRPLOG"].INPUT),
)

parser.add_argument(
"--cont_buffer_dur",
type=float,
default=0.5,
help="Duration of buffer for continuous data. Note: buffer may occupy ~15 MB / second.",
)
ez.run(components=comps, connections=conns)

parser.add_argument(
"--cont_override_config_all",
action="store_true",
help="Set this flag to set all analog channels to cont_smp_group (group 0 will disable continuous data).",
)

source = NSPSource(NSPSourceSettings(**vars(parser.parse_args())))
log = DebugLog()

ez.run(SOURCE=source, LOG=log, connections=((source.OUTPUT_SPIKE, log.INPUT),))
if __name__ == "__main__":
typer.run(main)
74 changes: 74 additions & 0 deletions examples/enable_cont.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
import sys
import time
from typing_extensions import Annotated

from pycbsdk import cbsdk
from pycbsdk.cbhw.packet.common import CBChannelType
import typer


def main(
smp_group: int,
inst_addr: Annotated[
str,
typer.Option(
help="ipv4 address of device. pycbsdk will send control packets to this address. Subnet OK. Use 127.0.0.1 "
"for use with nPlayServer (non-bcast). The default is 0.0.0.0 (IPADDR_ANY) on Mac and Linux. On "
"Windows, known IPs will be searched."
),
] = "192.168.137.128",
inst_port: Annotated[
int,
typer.Option(
help="Network port to send control packets. Use 51002 for Gemini and 51001 for Legacy NSP."
),
] = 51001,
client_addr: Annotated[
str,
typer.Option(
help="ipv4 address of this machine's network adapter we will receive packets on. Defaults to INADDR_ANY. "
"If address is provided, assumes Cerebus Subnet."
),
] = "",
client_port: Annotated[
int,
typer.Option(
help="Network port to receive packets. This should always be 51002."
),
] = 51002,
recv_bufsize: Annotated[
int, typer.Option(help="UDP socket recv buffer size.")
] = (8 if sys.platform == "win32" else 6) * 1024 * 1024,
protocol: Annotated[
str, typer.Option(help="Protocol Version. 3.11, 4.0, or 4.1 supported.")
] = "3.11",
):

params = cbsdk.create_params(
inst_addr=inst_addr,
inst_port=inst_port,
client_addr=client_addr,
client_port=client_port,
recv_bufsize=recv_bufsize,
protocol=protocol,
)
device = cbsdk.NSPDevice(params)
run_level = device.connect(startup_sequence=False)
if not run_level:
raise ValueError(f"Failed to connect to NSP; {params=}")
config = cbsdk.get_config(device, force_refresh=True)
for chid in [
k
for k, v in config["channel_infos"].items()
if config["channel_types"][k]
in (CBChannelType.FrontEnd, CBChannelType.AnalogIn)
]:
_ = cbsdk.set_channel_config(
device, chid, "smpgroup", smp_group
)
# Refresh config
time.sleep(0.5) # Make sure all the config packets have returned.


if __name__ == "__main__":
typer.run(main)
6 changes: 4 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,9 @@ readme = "README.md"
requires-python = ">=3.9"
dynamic = ["version"]
dependencies = [
"ezmsg>=3.5.0",
"pycbsdk>=0.1.3",
"ezmsg-event",
"ezmsg>=3.6.0",
"pycbsdk>=0.2.0",
]

[project.optional-dependencies]
Expand All @@ -34,4 +35,5 @@ packages = ["src/ezmsg"]
[tool.uv]
dev-dependencies = [
"ruff>=0.6.8",
"typer>=0.12.5",
]
Loading