Pluggable design

[jbenet]

(originally at ipfs/notes#37 (comment))

It may make sense to rework how multiaddr-net works into something like this:

n := manet.NewNetwork()
n.Add(manet.IP4)
n.Add(manet.IP6)
n.Add(manet.TCP)
n.Add(manet.UDP)
n.Add(onion.Onion)

l, err := n.Listener("/ip4/1.2.3.4/tcp/1234")
c, err := n.Dial("/onion/<onionaddr>") 

// with shortcuts like:
n := manet.NewThinWaistNetwork()
n.Add(onion.Onion)

"network" is a bit odd name, but does give the idea that these networks are stacked and makes more sense why mulriaddrs work the way they do. "transport" is another possible name, but may not capture the full meaning of "network".

[jbenet]

cc @david415

[jbenet]

cc @whyrusleeping

[cryptix]

Yup, makes sense. I2P and other overlay networks need this as well. I can see how global configuration (like the control port for Tor) can be part of the value that gets added to the "network" but would this allow for passing in configuration of connection specific parameters?

[david415]

@jbenet OK... I'd like to point out a couple of observations about your code samples in ipfs/notes#37 (comment)

onion service key material isn't included in your API design sketch... also tor can be configured with control port authentication... so the tor controller api needs to have even more secret information besides the onion service key private key... and possibly other information.

here's a concise working example of an onion echo server:
https://github.com/david415/oniondialer/blob/master/examples/echo.go

I think it's worth mentioning that Twisted Python has a transport plugin system called Twisted "endpoints"; https://twistedmatrix.com/documents/current/core/howto/endpoints.html One notable difference is that the client versus server plugins are referenced with two different plugin name spaces, and are processed by clientFromString and serverFromString respectively.

After looking at Twisted endpoints, it's clear that the multiaddr usage is essentially equivalent to a "client endpoint descriptor string".... because it's what is announced to the world... and clients use it to connect... but it's not really enough information for a server to begin listening.

Let's look at what our plugin system might look like. We don't have a server endpoint equivalent... and I don't see a compelling reason to create such monstrosities as this server side multiaddr onion:

"/onion//onion-key-file//home/human/myOnionkey/tor-control-port/9151"

So avoiding the ugliness of a "server" multiaddr syntax... What do you think of this?

listenerPlugins.RemoveAllListenerPlugins()
listenerPlugins.Register(listenerType, plugin)

listener := MultiAddrToListener("/onion/<onion-addr>")

In order for the call to MultiAddrToListener to successfully create an onion listener it first needs to have the associated onion service key material. The calling party who registers the onion plugin must also "register" their onion address before using the MultiAddrToListener function.

[jbenet]

@david415 you're totally right

so it would really be more like:

n.Add(onion.NewTransport(onion.ListenerOptions{
    OnionKeyFile:     "echoOnionKey",
    OnionServicePort: 8080,
    LocalAddr:        "127.0.0.1:8080",
    ControlNetwork:   "tcp",
    ControlAddr:      "127.0.0.1:9051",
    AuthCookie:       "/var/lib/tor-cookie/cookie",
})

// dialing out
n.Dial("/onion/<addr>")

// and maybe
n.Listen("/onion/0") // where 0 uses the key in "Onion Key File" ?

[whyrusleeping]

@jbenet I dont like that approach very much. It means that my 'network' is limited to that one set of onion options, when I should be able to specify things in my dial/listen calls.

[jbenet]

@whyrusleeping do you have another API in mind?

this is the goal:

n.Dial(ma.Multiaddr)

but may not be achievable for 100% of the cases

[whyrusleeping]

Yeah.. I dont have a good api in mind. I've been mulling them over and cant really come up with anything I like.

[whyrusleeping]

for it to work right, all of that onion dialer information needs to be embedded into the multiaddr

[jbenet]

not sure if all, i can imagine putting the private key in the addr, but the proxy details still on the transport object (the proxy details are less relevant)... Unless the proxy cannot listen on more than one key.

another option is to set the listener opts on a map on the transport

t := onion.NewTransport()

// add transport to the network
n.Add(t)

// t keeps a map[addr]onion.ListenerOptions
t.AddProxy("<onion-addr>", onion.ListenerOptions{
    OnionKeyFile:     "echoOnionKey",
    OnionServicePort: 8080,
    LocalAddr:        "127.0.0.1:8080",
    ControlNetwork:   "tcp",
    ControlAddr:      "127.0.0.1:9051",
    AuthCookie:       "/var/lib/tor-cookie/cookie",
})

// and than can do
n.Listen("/onion/<onion-addr>")

---

and another option is to just forgo the nice plugging altogether :(

onion.Listen(onion.ListenerOptions{
    OnionKeyFile:     "echoOnionKey",
    OnionServicePort: 8080,
    LocalAddr:        "127.0.0.1:8080",
    ControlNetwork:   "tcp",
    ControlAddr:      "127.0.0.1:9051",
    AuthCookie:       "/var/lib/tor-cookie/cookie",
})

[cryptix]

I'm with @whyrusleeping wrt Configuration. Setting up Listeners beforehand might be doable like this, but configuring outgoing dials (TOR, for instance, allows to specify in which country an exit-node is located. In I2P you can tweak tunnel length and a lot of other parameters per connection, too) beforehand sounds unwieldy, because you simply might not have all the information until user interaction or that data comes from somewhere else.. Readding the transport with new a new config to the network feels ikcy.

for it to work right, all of that onion dialer information needs to be embedded into the multiaddr

I don't see how this is true though... @whyrusleeping did you mean, assuming we want it as simple as this, every Param would have to be in the multiaddr? Aggreed, I wouldn't want that either. Doesn't it mix where to call with how to get there? The latter is up to the calling party IMHO.

[whyrusleeping]

did you mean, assuming we want it as simple as this, every Param would have to be in the multiaddr?

Yeah, thats what i was going for.

After sleeping on it, I think that we might have to go with something like:

func Dial(addr ma.Multiaddr, opts ...interface{}) (Conn, error) {

this allows us to call it as: Dial(addr) for the simple connections, and it also allows us to call: Dial(addr, onion.Options{....})

The same goes for listener

[jbenet]

Agree with @cryptix and @whyrusleeping .

Though, i think @whyrusleeping's point about it being in the multiaddr (or at least that being one way of making it possible) makes sense. One of the goals of multiaddr is to allow single-string configuration of transports, so that we can pass those easily from the CLI, from config files, etc, that may not understand the specialized transports at all.

Maybe, what we're missing here, is that this is not just onion, but really, a specific way of accessing onion, namely through a local tcp proxy. I think this does make sense:

# proxy onion over a local tcp service
/ip4/127.0.0.1/tcp/9051/onion/<onion-addr>

and this makes sense for both dials out and listens.

the tricky part with listens is other options, like AuthCookie and the private key.

/ip4/127.0.0.1/tcp/9051/onion-listen/<private-key>:<auth-cookie>/onion/<public-key>

is starting to get long and redundant.

---

@whyrusleeping i do like the Dial with interface{} options. And probably worth prototyping. Though again, i think being able to have single-string config is valuable.

@david415 i wonder what part of this listener config is really not meant to be part of the user configuration, but rather just a part of the "network stack implementation". what i mean by that is, the multiaddr string does not specify "for network tcp, make sure to use the net pkg's TCPDial function". This happens by virtue of the implementation of tcp in https://github.com/jbenet/go-multiaddr-net/ and it's a specific choice by the package author. People could chose to implement this with a C++ library instead, or with a userland TCP stack, etc. etc. Why is this relevant? because we may be able to separate the ListenerOptions above into three sections:

• usage and user dependent material, determined at runtime per address
• usage and user dependent material, determined at runtime for the whole transport
• implementation dependent material, determined at compile time.
• It is true that the /ip4/127.0.0.1/tcp/9051 proxy transport should probably be determined at runtime. But it may be a "whole transport" thing: instead of having two Tor TCP proxies, there really is just one, and all dials and listens go through it. Is there ever a case to use to different ones from the same binary? would a different network init, as shown above cover this use case?
• What about the AuthKey path? where does that fall? Seems like a runtime concern, but not necessarily per address.
• The PrivateKey should definitely be per address.

---

What about i2p? could we take a look at what connections there would look like, to get some inspiration? And DNS records?

[cryptix]

The length frightens me a little. Also encoding file paths in it.... (Could maybe be map[string]string to nick name them?). And I don't like the idea of having two representations of an address. (Think cat .ipfs/config and ipfs id output.

Though...

/ip4/127.0.0.1/tcp/9051/onion-listen/<private-key>:<auth-cookie>/onion/<public-key>

I just got this idea: There could be a multiaddr proxy that uses ssh intelligently to build forwarding's. Basically socks5 for the dial case but, you know.. Listen at mars from your code? I like the look of this! /ssh/mars/tcp/1337

what i mean by that is, the multiaddr string does not specify "for network tcp, make sure to use the net pkg's TCPDial function". This happens by virtue of the implementation of TCP.

For none-raw sockets, with TCP/UDP you are also at virtue of your systems kernel implementation of how it handles IP, no? I guess this is where implementation and layers rob up against?

What about i2p? could we take a look at what connections there would look like, to get some inspiration?

Well... Fire away :)

I2P is really toiled towards p2p. It uses a kademlia-ish dht for its netdb. You basically dial key hashes in the end, like with ipfs. You don't have numeric tcp/udp ports and just create new keys and endpoints on the go. (You can choose with stream and datagram though - there are lots of options to tweek those.. datagrams with no sender addr for instance.).
One of the funkier features and setups which is also related here are encrypted leaseSets (endpoints), where you can't build a tunnel connection if you only have the key hash but also need a corresponding symmetric key to ask the netdb to build you a tunnel to it.
Otherwise /ip4/127.0.0.1/tcp/9051/sam-stream-listen/ might actually work. (That's how the SAM bridge works. It can hand you new tcp ports locally that end up on the other side.) But do you want to open a previous key or a new one? How long should it be? etc.. Lots of options.
Naming (name.i2p to $keyHash.b32.i2p) is done in each client node with an address book (like /etc/hosts maybe) which can be subscribed to or you ask a jump service which you trust for new ones.
I'd really like to make multiaddr i2p aware, I think its a good fit.

[jbenet]

@cryptix leaseSets sound great actually. TRTTD.

on listen, wouldn't it be:

/ip4/127.0.0.1/tcp/9051/sam/0

?

what would other multiaddrs look like? like the ones you distribute out

[str4d]

Just adding my name to the participant list. I would definitely like to see I2P support (being one of the devs 😜) and am happy to answer questions.

A few points (from reading the previous two comments):

• IDK what a "multiaddr that you distribute out" refers to (I've not read the whole thread), but as far as connecting to a known peer, all you need is the I2P Destination of that peer, which can either be used directly by IPFS (the canonical form is a Base64 string) or referenced in its B32 form (52chars.b32.i2p).
• We have three different Go libraries: ccondom that uses the BOB API, and sam3 and goSam that use the SAM API. I'd recommend using SAM, it is better-supported. IDK which SAM library is better, but @cryptix developed the latter so you'll get better support for that 😉
• I2P does support an equivalent of numeric TCP and UDP ports. I2P's streaming library is its equivalent of TCP, and its datagrams are the equivalent of UDP. In SAM, these are STYLE=STREAM and STYLE=DATAGRAM respectively. The SAM API itself does not support these ports yet, but it will in the upcoming version 3.2 API.

[str4d]

Having read a little more about multiaddr, I would imagine that distributable string multiaddrs for I2P would look like:

• /i2p/spam.i2p/stream - Streaming, default port, uses router's addressbook to look up spam.i2p.
• /i2p/52chars.b32.i2p/dgram/0 - Repliable datagrams, port 0 (explicit default port), looks up Destination corresponding to 52chars.b32.i2p.
• /i2p/B64DESTINATION/raw/2000 - Raw datagrams, port 2000

[jbenet]

thanks @str4d ! and thanks for all the work on I2P.

those multiaddrs LGTM.

[whyrusleeping]

The interface i've been working with for the utp codebase is like this:

type Dialer interface {
    Dial(Multiaddr) (Conn, error)
    Matches(Multiaddr) bool
}

So you can create anything that implements the dialer interface and construct a network with it like so:

n := NewNetworkThings()
n.AddDialer(myutpdialer)
n.AddDialer(mytcpreusedialer)
n.AddDialer(myfallbacknetdialer)

Then when you call dial on the nework, it will find a dialer that can dial the given address, and use it. The primary reason i went with this method is that for utp (and udt and friends) we need to be able to use the listener socket to dial out on, This becomes possible now:

list := NewUtpListener(address)
utpd := UtpDialerFromListener(list)
mynet.AddDialer(utpd)

[david415]

What's the status? Did we settle on a multi-addr format for onion service addresses?

[david415]

It's no big deal if we only use these multiaddrs for clients to connect to... but if we want onion services to work then we must specify the key somehow. Either we put the entire key in the multiaddr or a filename containing the key. Of course in the future when tor prop 224 is resolved the key will be an ed25519 key... so it'll be much shorter.

[david415]

i think this ticket can be closed now.

[whyrusleeping]

@david415 agreed