Irdest developer manual

Welcome to the Irdest developer manual. This book is a collection of documents explaining the concepts, ideas, and protocols of the irdest project. Its source files are part of the main irdest code repo.

Outline

• Social documentation -- how we work together
• Technical introducion -- how does irdest work
• Design guides -- design principles for both code and UI
• Website documentation -- working on the irdest website
• License information -- licenses in use by irdest

Social Documentation

This section outlines the social structure of the irdest project.

Communication

The irdest project uses Matrix as a development and social chat. Feel free to drop by to ask questions or hang out!

Code of Conduct

TLDR: be nice!

We want to foster an open and engaging atmosphere for irdest and the development community around it. Because of this we follow the "Contributor Covenant" code of conduct. A copy of it should have been included in the sources for this book.

How to contribute?

First of all: thank you for wanting to help out :)

The irdest source can be found in our mono repo. We accept submissions via our mailing list, and (in a more limited capacity) via GitLab merge requests. See sections below for details.

Reporting an issue

If you've encountered a problem using Irdest software, we would highly appreciate it if you could tell us about it.

Since we use our own GitLab instance (and don't want to open registrations without verification) it's hard to submit issues via GitLab.

To submit an issue, just write an e-mail to the community mailinglist, in a format like: [BUG] ratman: sometimes crashes when ... or [QUESTION] irdest-proxy: how to set ..., etc. Do please try to first search for an existing e-mail thread in the mail archive though.

Contributions via e-mail

The easiest way to contribute code is via e-mail. This can be done in two ways:

1. Send a patch via git send-email
2. Upload your contributions to a different forge/ repository, and send an e-mail pull request

Contribution via send-email

You can follow the guide at https://git-send-email.io/ to get yourself set up for sending e-mail patches.

For any patch set that touches more than one component, please include a cover-letter to explain the rationale of the changes.

Sending an e-mail pull request

To send a pull-request via e-mail you must first upload your changes to your own copy of the irdest repository. You can host this anywhere that is convenient to you (for example GitLab or Codeberg).

Contributing via GitLab merge requests

If you want an account for development, please say hi in the Matrix channel so we know who you are.

• If a relevant issue exists, please tag in your description
• Include a short description of the accumulative changes
• If you want your history to be rebased/ merged, please clean it up to be useful. Otherwise we will probably squash it.
• Feel free to open a work-in-progress MR as a place to have a discussion about changes or to get feedback.

Submitting an e-mail patch

If you can't contribute via GitLab , you're very welcome to submit your patch via our community mailing list.

The easiest way of doing this is to configure git send-email.

Without git send-email

• Send an e-mail with the title [PATCH]: <your title here>.
• Format your patch with git diff -p
• Don't send HTML e-mail!
• Make sure your line-wrapping is wide enough to allow the patch to stay un-wrapped!

Lorri & direnv

You can enable automatic environment loading when you enter the irdest repository, by configuring lorri and direnv on your system.

 ❤ (uwu) ~/p/code> cd irdest
direnv: loading ~/projects/code/irdest/.envrc
direnv: export +AR +AR_FOR_TARGET +AS +AS_FOR_TARGET +CC
        // ... snip ...
 ❤ (uwu) ~/p/c/irdest> cargo build                           lorri-keep-env-hack-irdest
 ...

Contributor Covenant Code of Conduct

Our Pledge

In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.

Our Standards

Examples of behavior that contributes to creating a positive environment include:

• Using welcoming and inclusive language
• Being respectful of differing viewpoints and experiences
• Gracefully accepting constructive criticism
• Focusing on what is best for the community
• Showing empathy towards other community members

Examples of unacceptable behavior by participants include:

• The use of sexualized language or imagery and unwelcome sexual attention or advances
• Trolling, insulting/derogatory comments, and personal or political attacks
• Public or private harassment
• Publishing others' private information, such as a physical or electronic address, without explicit permission
• Other conduct which could reasonably be considered inappropriate in a professional setting

Our Responsibilities

Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.

Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.

Scope

This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.

Enforcement

Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at kookie@spacekookie.de. All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.

Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.

Attribution

This Code of Conduct is adapted from the Contributor Covenant, version 1.4, available at http://contributor-covenant.org/version/1/4

Irdest release checklist

This page is meant for anyone on the project with release access. Potentially this page should be moved into a Wiki or similar.

1. Make sure that CI passes on develop
2. Update CHANGELOG.md and make sure that it contains all relevant changes for the release.
3. Select a set of packages/ targets to include in the Releases section
4. Bump any relevant version numbers
5. Create a release/{version} tag corresponding to the new ratmand/ libratman version
6. Check the issue tracker and mailing list for issues that are being closed by this release
7. Update any relevant crates to crates.io (via cargo release)
8. Update the website to point to the new bundle download (as soon as release CI passes)
9. Re-deploy the website
10. Write a release description on the release tag
11. Optionally: write an announcement on the mailing list

Technical Documentation

This is the Irdest technical documentation tree; it consists of documents meant to give you an overview into the various components that make up the Irdest project, as well as their inner workings.

This manual is a good starting point both to those interested in hacking on Irdest, and developing applications on top of it.

Introduction

Irdest is a distributed routing system, creating an address space over ed25519 keys. Each address on the network is a public key, backed by a corresponding private key. This means that encryption and message authentication are built into the routing layer of the network. Each physical device can be home to many addresses, used by different applications, and it is not possible from the outside to re-associate a specific device with a specific address.

A lot of traditional networking infrastructures is built up in layers (see OSI model). Similarly, the irdest project replicates some of these layers. BUT... the layers between the OSI model and Irdest don't map directly onto each other and are only meant to illustrate difference in hardware access, user access, and scope.

Following is a short overview of layers in Irdest.

(While the following sections mention the OSI layers it's important to keep in mind that this is by convention. Nothing stops you from implementing a netmod for a high-level protocol like XMPP or a client for a low-level protocol like ARP. Alternatively, you can think of the two layers in Ratman's API architecture as "address scope" and "wire scope")

Network drivers

Network drivers establish and manage connections with peers via different underlying transport mechanisms. A driver (in the irdest jargon called a "netmod") is initialised by the Irdest router Ratman.

This allows the core of Ratman to remain relatively platform agnostic, while letting platform-specific drivers handle any quirks of the connections that are being established. For example, the netmod-datalink module uses NetworkManager to configure a wireless AP/ wireless connections to an existing host without user intervention. Ratman itself doesn't need to understand how to talk to NetworkManager.

Many different drivers can be active on the same device, as long as they are connected to the same router. In the OSI model, this roughly maps to layers 1 & 2.

Currently a driver needs to be specifically added to ratmand and included at compile time. We are working on a dynamic loading mechanism however (either via .so object loading or an IPC socket).

Ratman: the Irdest router

Ratman/ ratmand is a decentralised router daemon. It comes with a small set of utilities such as ratcat (a netcat analog), ratctl (a batctl analog), and a simple management web UI.

Clients communicate with Ratman via a local TCP socket and protobuf envelope schema. For most use-cases we recommend the ratman-client library. Alternative implementations don't currently exist and this API is also extremely unstable, so please be aware of this for the time being!

In the OSI model, this maps roughly to layer 3 and 4.

Integration shims

Currently only one (work in progress) shim exists: irdest-proxy. This layer aims to create interoperability layers between existing IP networks and an Irdest/ Ratman network.

In future many different shims could exist, tunneling Tor traffic through Irdest, or providing apps on mobile devices to take advantage of the "VPN" functionality of the OS.

In the OSI model this maps to layer 5. This is because in Ratman a connection is stateless, and thus no real session state exists. This shim introduces the concept of sessions for the benefit of existing applications that rely on them.

Clients

These are applications that use Ratman for their networking (either natively or via an integration shim).

The Irdest project develops a few of these as demonstrations as to what is possible to do with the Irdest network. While we of course hope that the Irdest application bundle makes onboarding into the network easier, we encourage others to work on top of this platform, or bridge it to other decentralised networking platforms.

Irdest comes with a few command-line applications and Irdest mblog, a Gtk usenet-inspired microblogging app!

What next?

If you are interested in writing an application for Irdest, or porting another application to use Irdest, you should familiarise yourself with the Ratman client SDK first.

If you want to get started hacking on Irdest, check out the "Hacking" section. In either case you may also want to read "Ratman internals"!

• Ratman SDK
• Ratman internals
• Hacking on Irdest
• Bibliography

Ratman client lib

This is the client library used to write applications for Ratman. Currently only a Rust implementation exists.

You can find ratman-client on crates.io and its documentation on docs.rs!

Workflow

There are three main steps to using the client-lib:

1. IPC initialisation
2. Address registration/ login
3. Message sending and receiving

IPC initialisation

By default the IPC socket for Ratman is running on localhost:9020. Many of the Irdest tools allow you to overwrite this socket address, to allow for local testing with multiple routers. We recommend that your application expose this option to users for testing purposes as well!

Address registration

An address for Ratman is associated with a cryptographic key pair. Currently we don't expose the private key from the router to applications (which will probably change in the future!)

When your application is given an address you should store it in your application state somewhere, along with the corresponding auth token. These will be important the next time your application starts. For added security you should encrypt this data with a user password!

Message sending and receiving

Sending messages happens asynchronously, which means that the client lib will not get feedback on if your message has actually been dispatched across a network channel, let alone been received.

Messages can be sent as one of two types: Standard and Flood.

Standard messages have a fixed recipient address and will be routed to the destination where they will leave the network and be processed by the target application (or dropped).

Flood messages are sent to every device and address on the network, to allow for network-wide announcements (this is also how your address announces itself!) To limit the amount of relevant Flood messages an application has to deal with, they are namespaced. The namespace itself is also an Irdest address.

So for example a standard message sent to ECB4-30B9-4416-C403-716F-601F-FC56-9AD3-BD2E-3892-227A-84AD-E6FC-A1CE-0A92-03F6 will be carried across the network until it reaches this exact address.

A flood message sent to ECB4-30B9-4416-C403-716F-601F-FC56-9AD3-BD2E-3892-227A-84AD-E6FC-A1CE-0A92-03F6 will be delivered to all applications that are listening on this namespace.

API Example: irdest-echo

This is a small program demonstrating the most basic usage of the ratman-client SDK. At start-up it registers a new address, listens to any incoming messages, and returns them as they are to the sender.

use ratman_client::{RatmanIpc, Receive_Type};

#[async_std::main]
async fn main() {
    let ipc = RatmanIpc::default()
        .await
        .expect("Failed to connect to Ratman daemon!");

    println!("Listening on address: {}", ipc.address());
    while let Some((tt, msg)) = ipc.next().await {
        // Ignore flood messages
        if tt == Receive_Type::FLOOD {
            continue;
        }

        // Get the message sender identity and reply
        let sender = msg.get_sender();
        ipc.send_to(sender, msg.get_payload()).await.unwrap();
    }
}

Hacking on Irdest

Hey, it's cool that you want to hack on Irdest :) We recommend you install nix to handle dependencies. Depending on the directory you are in you can fetch development dependencies:

$ cd irdest/
$ nix-shell # install the base dependencies
...
$ cd docs/
$ nix-shell # install documentation dependencies

With lorri and direnv installed transitioning from one directory to another will automatically load additional dependencies!

Alternatively, make sure you have the following dependencies installed:

• rustc
• cargo
• rustfmt
• rust-analyzer
• clangStdenv
• pkg-config
• protobuf
• cargo-watch
• binutils
• yarn
• reuse
• jq

Building Ratman

Ratman provides several binaries in the ratman package. You can build the entire package with cargo. By default the ratman-dashboard will be included, which requires you to build the sources with yarn first.

$ cd ratman/dashboard
$ yarn && yarn build
$ cd ../..
$ cargo build -p ratman --all-features

Alternatively you can disable the dashboard feature. Unfortunately cargo doesn't allow selective disabling of features, so you will need to disable all default features, then select a new set of features as follows:

$ cargo build -p ratman --release --disable-default-features \
                        --features "cli datalink inet lan lora upnp"
...
[cargo goes brrr]

Building irdest-echo

irdest-echo is a demo application built specifically to work with Ratman as a networking backend. Build it via the irdest-echo package with cargo.

$ cargo build -p irdest-echo --release
...

Building irdest-mblog

irdest-mblog is probably the most complete user-facing application that is native to the Irdest network. You can build it with Cargo, as long as you have gtk4 installed on your system (or using the Nix environment).

$ cd client/irdest-mblog
$ cargo build --release --bin irdest-mblog-gtk --features "mblog-gtk"
...

What now?

Check the issue tracker for "good first issues" if you are completely new to Irdest, and additionally "help wanted" issues if you already have some experience with the code-base.

Please also don't hesitate to ask us any questions! We're very happy to help :)

This section outlines some internal concepts that are in use by the Ratman routing daemon.

Gossip announcements

Ratman operates on the gossip protocol approach, where each address on the network repeatedly announces itself to other network participants. Based on these announcements the routing tables of passing devices and peers will be updated as needed. This means that no single device will ever have a full view of the network state, but will always know the "direction" a packet needs to be sent in order to make progress towards its destination. This is similar to how existing routing protocols such as BATMAN and BGP work.

A short example

To illustrate this capability, let's look at this simple network graph:

  +--------- [ D ] ----------+
  |                          |
[ A ] ------ [ B ] ------- [ C ]

Node A sends announcements to B an D, which will both proxy it to C. The router at C will use various metrics to decide which link is more stable, and declare it the "primary" for peer A.

When C wants to route a packet to A, it looks up the local interface over which it thinks it can reach A the best (for example D). It then dispatches the packet to D, knowing that this node must be closer to the destination to deliver the packet.

Announcement metadata

As part of the announcement protocol nodes may include metadata in the announcement. According to the current specification draft, it looks as follows:

Announcement {
  origin: {
    timestamp: "2022-09-19 23:40:27+02:00",
  },
  origin_sign: [binary data],
  
  peer: {
    ...
  },
  peer_sign: [binary data],
  
  route: {
    mtu: 1211,
  },
}

For the following section the announcement.route.mtu parameter is especially important!

Message slicing & streaming

An incoming message in Ratman is sliced twice: once into cryptographic blocks via the ERIS encoding, and then again for transport according to the path MTU outlined in the previous metadata section.

When slicing ERIS blocks, frames should be filled completely, with non-overlapping block boundries. This makes sure to not send frames that contain a lot of zero-padding.

This should be implemented as an iterator/ stream, which consumes at iterator/ stream of eris blocks.

ERIS block size = 4 bytes.
Frame size = 5 bytes.

Eris blocks: [1 2 3 4][5 6 7 8][9 10 11]
Frames:      [1 2 3 4 5][6 7 8 9 10][11]

ERIS block size = 12.
Frame size = 5.

Eris blocks: [1 2 3 4 5 6 7 8 9 A B C][D E F 10 11 12 13 14 15 16 17 18]
Frames:      [1 2 3 4 5][6 7 8 9 A][B C D E F][10 11 12 13 14][15 16 17 18]

Selecting frame sizes

The two block sizes supported by ERIS by default are 1kB and 32kB. For small messages these wil create a significant amount of overhead, especially on low-MTU connections.

For these cases we should have small-message optimisations, based on the size of the message, and the path MTU to the recipient.

Messages larger than 32 kB/ 2 kB on a path MTU of <1 kB/ <256 bytes respectively should be rejected by the sending router. We may want to add another small message optimisation between 2kB and 32kB max size messages.

Delay tolerance

This section will be expanded when the implementation of delay tolerance becomes more stable and usable. But in short: messages can be buffered by various nodes across the network when the destination is not reachable. This means that different networks can communicate with each other even when no stable connections between them exist (for example via a sneaker net). This does however require applications to be aware of long delays and handle them gracefully!

Roaming

Because of the distinction between network channels and routing it is easy to roam across network boundries. As long as a channel can be established (even just one-way), packets can be sent through the Irdest network with no knowledge of these network bounds.

It also means that a network can easily be composed of different routing channels. Local UDP discovery, TCP links across the existing internet, Wireless antenna communities, and even phones.

WIP specification

There is a work-in-progress specification available in the wiki!

Network drivers

In Irdest a network driver is called a netmod (short for network module). It is responsible for linking different instances of Ratman together through some network channel.

The decoupling of router and network channel means that Ratman can run on many more devices without explicit support in the Kernel for some kind of networking.

Because interfacing with different networking channels comes with a lot of logical overhead these network modules can become quite complex and require their own framing, addressing, and discovery mechanisms. This section in the manual aims to document the internal structure of each network module to allow future contributors to more easily understand and extend the code in question.

Backplane types

There are three types of network backplanes that Irdest can interact with:

• Broadcast :: only allowing messages to all participants
• Unicast :: only alloing messages to a single participant
• Full range :: allowing both broadcast and unicast message sending

Netmod API

The netmod Endpoint API looks as follows:

#![allow(unused)]
fn main() {
#[async_trait]
trait Endpoint {
    fn msg_size_hint(&self) -> usize;

    async fn peer_mtu(&self, target: Option<Target>) -> Result<usize>;

    async fn send(&self, frame: Frame, target: Target, exclude: Option<u16>) -> Result<()>;

    async fn next(&self) -> Result<(Frame, Target)>;
}
}

(This API is still in flux and needs to be extended in various ways in the future. Please note that this documentation may be out of date. If you notice this being the case, please get in touch with us so we can fix it!)

• msg_size_hint is used to communicate a maximum size per message transfer and will be used to populate the announcement.route.size_hint parameter (on endpoints that support this!)

• peer_mtu is used to determine the immediate hop MTU to a target and will be used to populate the announcement.route.mtu parameter

• send is used to send messages.

  The exclude parameter is important on certain unicast & boardcast backplanes to prevent endless replication of flood messages.

• next is polled by the router in an asynchronous task to receive the next segment from the incoming frame queue.

This API is auto-implemented for all Arc<T> where T: Endpoint.

Internet overlay netmod (inet)

The main way to use Ratman with other people at the moment is via the internet overlay network module inet. It creates peering sessions over the internet and TCP. With that comes a significant amount of connection state logic and routing outside of Ratman, because each instance of inet can be connected with many other instances of inet.

Structure diagram

Following is a class structure diagram for the three main components of the inet driver. Note that Server is a dispatch-type, meaning that after allocation it copies itself to a private task-stack and remains running until the containing application is shut down.

TODO: figure out why the mermaid graph is borked

classDiagram-v2

class InetEndpoint {
    +Arc[Routes] routes
    +ChannelPair channel
    +start( bind )
    +port()
    +add_peers( peers )
    +send( target, frame )
    +send_all( frame )
    +next()
}

class Routes {
    +AtomicU16 latest
    +BTreeMap[Target, Peer] inner
    +next_target()
    +add_peer( target, peer )
    +remove_peer( target )
    +exists( target )
    +get_peer_by_id( target )
    +get_all_valid()
}

class Server {
    +Option[TcpListener] ipv4_listen
    +TcpListener ipv6_listen
    +port()
    +run()
}

Flowchart

While the inet driver doesn't have a lot of type components, their interactions can get quite complex. Furthermore there are some stateless function components that can't be expressed in a traditional class diagram.

digraph G {
  size="8,30!"
  graph [fontname = "Handlee"]
  node [fontname = "Handlee"]
  edge [fontname = "Handlee"]
  splines="polyline"
  bgcolor=transparent
  nodesep=0.5
  

  subgraph cluster_0 {
    color=orange
    node [color=orange]
    
    A [label="InetEndpoint::new"]
    B [label="add_peers()"]
    S [label="Listen for\nincoming connections"]
    C [label="for each peer", shape=Mdiamond]
    D [label="Resolve address"]
    E [label="start_connection()"]
    E1 [label="connect()"]
    E2 [label="handshake()"]
    E3 [label="Add peer\nto Arc<Routes>"]
    E4 [label ="SPAWN peer.run()"]
    E5 [label="setup_cleanuptask()"]
    Z [label="SPAWN restart.recv()"]

    A -> B
    A -> S
    B -> C
    C -> D
    D -> E
    E -> E1
    E1 -> E1 [label="retry"]
    E1 -> E2
    E2 -> E3
    E2 -> E4
    E3 -> E5
    E5 -> Z
    
    label = "Initialisation"
    fontsize = 20
  }
  
  subgraph cluster_1 {
    color=cyan
    label = "Peer Frame Receiver"
    node [color=cyan]
    
    P1 [label="run() loop", shape=Mdiamond]
    P2 [label="Read Frame"]
    P22 [label="Release Mutex"]
    P222 [label="break", style=filled]
    P3 [label="receiver.send()"]
    
    P1 -> P2 [label="Lock Mutex"]
    P2 -> P22 [label="No Data"]
    P22 -> P1 [label="yield_now()"]
    P2 -> P222 [label="Read failed"]
    P2 -> P3 [label="Valid Frame"]
  }
  
  subgraph cluster_2 {
    color=green
    label="Server loop"
    node[color=green]
    
    S1 [label="for each\nincoming", shape=Mdiamond]
    S2 [label="SPAWN\nhandle_stream()"]
    S22 [label="Drop stream"]
    S3 [label="accept_connection()"]
    
    S -> S1
    S1 -> S2 [label="valid"]
    S1 -> S22 [label="invalid"]
    S2 -> S3
    S3 -> E4 [label="valid"]
    S3 -> S22 [label="invalid"]
  }
}

LoRa broadcast driver

The netmod-lora endpoint uses the lora-modem embedded firmware in the background to communicate with the actual radio hardware. Check the user manual on how to set this up.

Irdest LoRa implements its own framing mechanism, incompatible to LoRaWAN. We do set the magic number equivalent bit from LoRaWAN to a different identifier though, which means that LoRaWAN devices should ignore Irdest frames.

LoRa is a broadcast backplane, which means that unicasts can only be implemented via filtering.

More documentation to follow. For more in-progress notes check out this wiki page!

Mesh Router Exchange Protocol (MREP)

Status: pre-draft Start date: 2022-04-05 Latest revision: 2023-09-27

The Irdest router Ratman exchanges user packets as well as routing metadata with neighbouring routers. This communication is facilitated through the "mesh router exchange protocol". It has three scopes.

1. Exchange user data packets
2. Collect connection metrics and transmission metadata
3. Perform routing decisions according to these metrics

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC2119

Basics

The Mesh Router Exchange Protocol specifies different mechanisms for multiple routers to communicate with each other in order to facilitate the flow of messages across a shared network.

Routers are connected with each other via different communication channels (or backplanes), meaning that the routing logic is decoupled from the connection logic. For Ratman, this is done via the Netmod abstraction. Each netmod allows routers to peer with each other over a variety of backplanes, with each requiring different platform specific configuration.

Simultaneously an Irdest router MUST allow connections from client applications, which can register addresses on the network. An address on an Irdest network is an ed25519 public key (32 bytes long).

A client application can register as many addresses as it needs. From a network perspective the relationship between an address and a device can't necessarily be proven.

The terms "Address" and "Id" may be used interchangeably in this specification draft.

Tangent: Ratman architecture

While this specification doesn't explicitly define the interface between an Irdest router and the way it connects networking channels between different instances, several of the mechanisms in this specification require communication between the wire layer and the routing layer. This section shortly outlines the required function endpoints and the data they accept and provide.

(In future this section should become the base for a second specification.)

A networking endpoint is defined via the following trait:

#![allow(unused)]
fn main() {
#[async_trait]
trait Endpoint {
    fn msg_size_hint(&self) -> usize;

    async fn peer_mtu(&self, target: Option<Target>) -> Option<usize>;

    async fn send(&self, frame: Frame, target: Target, exclude: Option<u16>) -> Result<()>;

    async fn next(&self) -> Result<(Frame, Target)>;
}
}

• fn msg_size_hint should return the largest recommended message to be sent over this link at a time. This metric is used to constrain traffic over low-bandwidth connections. If no limit exists, return 0.
• fn peer_mtu queries the MTU value for the peer represented by a target ID. If no target is given the immediate/ broadcast MTU is queried. If no MTU could be determined, return None.
• fn send takes a frame, a target ID, and an exclusion parameter (used to de-duplicate flood messages on certain backplanes). Returns an error if the frame is too big or the peer is busy.
• fn next will be polled by Ratman on an async task to grab the next incoming frame from a particular endpoint

Announcements

Irdest is a gossip based network, meaning that participant addresses need to announce themselves, and rely on intermediary nodes to propagate messages.

Announcements fundamentally allow different network participants to learn of each other's existence without a management intermediary.

Router announcements

Every router on the Irdest network has a unique address. Messages sent to this address MUST conform to the MREP Message specification (Appendix A).

Routers announce their address to other routers they are immediately connected to. However unlike regular Irdest addresses, these announcements SHOULD NOT be propagated, unless explicitly instructed to do so by the sending router. This functionality may be added in a later version of this specification.

Routers announce each other via a specific mechanism in the netmod API. On broadcast backplanes this should happen at the same frequency as the address announcement loop.

On connection (or p2p) backplanes this should only be done when initialising a connection, or whenever a connection was re-established/ recovered.

Every router keeps track of the set of routers connected to it, to enable future queries specified in this protocol draft.

Address announcements

When registering an address, this generates an ed25519 keypair. The private key is stored in the router, whereas the public key is used to announce the address on the network. Ratman does not use address compressions at this time.

The default announcement rate is 2 seconds. Following is an example announcement payload.

#![allow(unused)]
fn main() {
Announcement {
  origin: {
    timestamp: "2022-09-19 23:40:27+02:00",
  },
  origin_sign: [binary data],
  
  peer: {
    ...
  },
  peer_sign: [binary data],
  
  route: {
    mtu: 1211,
  },
}
}

The announcement structure itself does not need to contain the announced address, since it is itself contained in a carrier frame, which contains the sending address.

Three metadata sections are used to communicate authenticity, reliability and scale of connections across a network.

Origin metadata is provided by the origin of the announcement, and MUST thus be signed (via the origin_sign field). As per this specification it MUST contain a timestamp of when the announcement was generated.

Additional announcement origin metadata MAY be included by the originator in the origin section. Routers MUST NOT strip this metadata, since it would break the authenticity verification for the announcement. However announcements with origin metadata fields larger than 1024 bytes MUST be discarded. This limit may be reduced in a future version of this specification.

Peer metadata is provided by the immediate peer that an announcement was propagated via. Let's look at the following example.

A  ->  B  ->  C

In the above example, router A is connected to router B, which is connected to both A and C, and router C, which is only connected to router B. Messages can flow from A to C via B bidirectionally.

An announcement from A to B will contain origin metadata from A, as well as peer metadata from A. However when B replicates the announcement to C, origin metadata will still be from A, but peer metadata from B.

Peer metadata is used to provide additional data about a network address, which may not be generally relevant to the network and instead only spread for a single hop. Peer metadata is signed by the router's address key. For every incoming announcement, routers MUST replace the previous peer metadata section with their own. If no metadata is provided both the peer and peer_sign fields MUST be filled with zeros.

Lastly an announcement contains route metadata, which is not signed and can not be trusted. This field allows routers along the way to deposit helpful markers to other routers in the downstream of a particular announcement.

This may be related to the reliability of connections, the limitation of bandwidth, or that a uni-directional network boundary was crossed. Currently only the mtu field is specified. Additional fields will be added by later revisions of this specification.

Additionally, the route metadata section MUST be pruned if it surpasses 512 bytes in length. Pruning means that only recognized fields will be kept, while unknown keys are discarded.

Route scoring

Consider the following scenario:

A -  B  - C
 \ D - E /

Router A is connected to B and D, router B is connected to A and C, router C is connected to B and E, etc.

Announcements from A will reach C both via the link from B to C and from E to C. Thus, when C wants to send a message to A it needs to consider multiple routes for sending messages. This is called "route scoring" and is implemented via the Route scoring API (see Appendix C).

There are many different approaches that can be taken for route scoring, especially for loop avoidance strategies. By making this component modular, it means that future additions can radically change or overhaul the way that Ratman performs this scoring mechanism, without needing large amounts of re-engineering in the core of Ratman.

This also allows network researchers to integrate their research code into an existing networking and testing platform.

Default greedy strategy

This is the default routing strategy in Ratman.

For any connection/ message pair not qualifying for a small message optimisation strategy, ...

Outline

• For "large" MTUs (> 1200 bytes?) select path of smallest ETD
• Otherwise balance an MTU curve vs ETD curve (don't reward high MTU or low ETD at the cost of the other)

Small message optimisation strategy

This routing strategy is activated when sending messages over a connection that fall under the message/ MTU metric requirements of Low bandwidth modes.

Outline

• Deny messages that are too big
• Use pull routing
• Send the manifest first
• Use a store & forward protocol (up to a limit?)

Security

(I am not a cryptographer and this section will have to be expanded/ reviewed in the future)

All user messages sent through Irdest are encrypted via the ChaCha20 stream cipher, provided by the ERIS block slicing specification which is used to encode user payloads.

An address in Irdest is an public key (also called "address key"), backed by a corresponding secret key. Keys are generated on the ed25519 twisted edwards curve.

Because ChaCha20 is a stream cipher it requires a symmetric secret to work. For this purpose we convert both secret and public keys from the edwards curve representation to the montgomery curve representation, and use this for the x25519 diffie-hellman handshake between the sending address secret key and the recipient address key.

There are two layers of signatures in Irdest. The base layer Frame (defined in Appendix A) contains space for an ed25519 signature. All message payloads are signed by the sending address edward's curve key (currently using ed25519-dalek) and can be verified by any node a message traverses (since the sending address is visible to any network participant).

For user payloads, ERIS guarantees message integrity by verifying block content hashes against the recorded versions in the manifest. This manifest message is also signed via the basic Frame delivery mechanism. Thus user message integrity can be guaranteed.

Message encoding & delivery

User payloads are encoded via ERIS, sliced into carrier frames, and sent towards their destination (see Appendix A on details).

This uses two basic message types: DataFrame and ManifestFrame. Messages in Irdest are sessionless streams, meaning that data is streamed between different Irdest routers, but buffered into complete messages before being exposed to the recipient application.

ERIS specifies a "Read Capability" which for the purposes of Irdest and this spec we are calling the "Manifest".

For a DataFrame the payload of the underlying carrier frame is entirely filled with content from a re-aligned block stream. Frames MUST NOT be padded with trailing zeros to fill the target MTU.

A ManifestFrame contains a binary encoded version of the "Read Capability". If this manifest is too large for the containing .carrcarrier frame, it is split into multiple frames (see Appendix A: Manifest Frame)

Journal sync

Irdest allows devices to connect to each other via short-lived (or "ephemeral") connections. One such application is Android phones, where p2p WiFi connections can only be established with a single other party at a time. Bluetooth mesh groups are possible, but are also significantly limited in the number of active connections.

For this purpose we introduce the "journal sync" mechanism.

An Irdest router MUST contain a journal of content-addressed blocks of data (see Appendix B). Messages are indexed via their content hashes, as well as the recipient information. A journal sync is a uni-directional operation, which should be applied in both directions of the link. What that means is that journals are not so much synced, but propagated.

Let's look at an example to demonstrate the process.

Routers A and B are connected to each other via an ephemeral connection (req_ephemeral_connection is called by a netmod driver which has established the connection).

First both routers exchange a list of known addresses. Future versions of this specification MAY implement some kind of compression or optimisation for this transfer, since routing tables may get quite large.

#![allow(unused)]
fn main() {
SyncScopeRequest {
  addrs: BTreeSet<Address>,
}
}

Outline:

• Exchange list of known addresses (with an optimisation for "last recently used")
• Forward blocks addressed to any of the known addresses
• How to avoid re-transmit loops in a group of phones?
• How to avoid having to send too much data?
• Loops between people who both infrequently see the same peer address? Who gets the frames? Both? (probably)

AGPL compliance

Ratman is licensed under the AGPL-3.0 license and as such needs to be able to provide its own source code.

It is not possible to query the source of a node more than one router edge away from your own since router address announcements do not propagate across the network.

A router MAY at any time send a source request to a connected router. The request is time-stamped to avoid repeated and duplicate requests.

#![allow(unused)]
fn main() {
SourceRequest {
  date: "2022-09-22 03:18:32"
}
}

As a response, the recipient router MUST send a SourceResponse reply. The response doesn't contain the source code. Instead it describes the source that is running. A SourceResponse MUST contain the source_urn field. Every other field is optional, but a router SHOULD still provide them. The note field SHOULD contain a list of patch-names that have been applied to the node, if the number_of_patches is not zero. Otherwise this field SHOULD remain empty.

#![allow(unused)]
fn main() {
SourceResponse {
  version: "0.5",
  number_of_patches: 0,
  source_url: "https://git.irde.st/we/irdest",
  source_urn: "b5bb9d8014a0f9b1d61e21e796d78dccdf1352f23cd32812f4850b878ae4944c",
  note: "Hier könnte Ihre Werbung stehen",
}
}

A recipient of this SourceResponse can now check whether the source code their node is running is the same as the router that responded, by checking the source_urn against their own source version (TODO: specify how this URN is generated).

In case the recipient doesn't already have this source code they can now send a PullRequest to the sending node:

#![allow(unused)]
fn main() {
PullRequest {
  urn: "b5bb9d8014a0f9b1d61e21e796d78dccdf1352f23cd32812f4850b878ae4944c",
}
}

Low bandwidth modes

Some links in an Irdest network may be extremely low bandwidth, for example when using netmod-lora for long range communication. This severely constricts the maximum transfer size (< 255 bytes), on a < 1% duty cycle. This means that the maximum incoming message size MUST be constricted as well.

In these cases the "Small Message Optimisation" (SMO) MUST be used. Following is a table that outlines the selection of encoding block sizes based on the determined path MTU and size-hint (via announcement.route.mtu and announcement.route.size-hint)

For these cases we should have small-message optimisations, based on the size of the message, and the path MTU to the recipient.

Messages larger than 32 kB/ 2 kB on a path MTU of <1 kB/ <256 bytes respectively should be rejected by the sending router. We may want to add another small message optimisation between 2kB and 32kB max size messages.

MTU leap-frogging

A frame may encounter a netmod link that doesn't allow for a sufficiently sized MTU

In some cases, the path MTU information on the sending node was incorrect, and a set of frames will encounter a link that is too low-bandwidth to support their size. In this case the "leap-frogging" protocol should be used.

The first frame in a series that is too large to transmit over a connection will be prepended with this metadata section:

#![allow(unused)]
fn main() {
LinkLeapRequest {
  seq_id: "1D90-C2AB-E50D-A4EC-F88C-BD9E-818B-7006-7D32-BED0-4EEC-83F0-756E-D856-40AA-B611",
  inc_mtu: 1222,
  look_ahead: false,
}
}

• seq_id :: the sequence ID for the incoming set of frames. This identifier is used to determine which frames need to be re-sliced
• inc_mtu is the size of incoming frames

MTU leap-frogging performs a single step of look-ahead. This means that a router receiving a LinkLeapRequest MUST perform an MTU look-ahead if request.look_ahead is set to true (and subsequently set it to false). This means that up to two link MTU limitations can be "skipped over" before having to re-collect into the original frame size and re-slicing.

For an incoming LinkLeapRequest a router MUST spawn a LeapFrameCollector

Appendix A: MREP Message specification

This section of the specification outlines the way that MREP container messages are encoded. Any optional fields that are not present in a particular structure MUST be replaced with a NULL byte.

The basic container type of any message in an Irdest network is a carrier frame, which has the following structure:

#![allow(unused)]
fn main() {
CarrierFrame {
  version: u8,
  modes: u16,
  recipient: [u8; 32] (optional),
  sender: [u8; 32],
  seq_id: [u8; 34] (optional),
  signature: [u8; 32] (optional),
  pl_size: u16,
  payload: [u8, pl_size]
}
}

This message structure is byte aligned.

• version :: indicate which version of the carrier frame format should be parsed. Currently only the value 0x1 is supported
• modes :: a bitfield that specifies what type of content is encoded into the payload
• recipient :: (Optional) recipient address key. May be replaced with a single zero byte if the frame is not addressed (see below).
• sender :: mandatory sender address key
• seq_id :: (Optional) sequence ID for push messaging payloads, mtu-leap protocol, etc
• signature :: (Optional) payload signature, generated by the sending key. May be replaced with a single zero byte if the frame has a payload-internal signature (see below).
• pl_size :: 16 bit unsigned integer indicating the size of the data section. Frame payloads larger than 32kiB are not supported!
• payload :: variable length payload. Encoding is protocol protocol specific and payload MUST be prepended with an 2 byte size indicator.

Importantly, the CarrierFrame does not include a transmission checksum to detect transport errors. This is because some transport channels have a built-in checksum mechanism, and thus the effort would be duplicated. It is up to any netmod to decide whether a transmission checksum is required.

Following is a (work in progress!) overview of valid bitfields. If a field is not listed it is invalid! Routers that encounter an invalid message MUST discard it.

Message payloads are encoded via their own Protobuf schemas (TODO: turn pseudo-code schemas from this specification into protobuf type definitions).

Announcement

Announcement frames are special in that they MUST set the recipient and signature field to a single zero byte. This is because announcements are not addressed, and contain a payload-internal signature system. All other message types handled by this specification MUST include both a recipient and signature!

message Announcement {
    OriginData origin = 1;
    bytes origin_sign = 2;

    PeerData peer = 3;
    bytes peer_sign = 4;

    RouteData route = 5;
}

message OriginData {
    string timestamp = 1;
}

message PeerData { /* tbd */ }

message RouteData {
    uint32 mtu = 1;
    uint32 size_hint = 2;
}

Data Frame

A data frame is already explicitly sliced to fit into a .carrcarrier frame (see "MTU leap-frogging" for how to handle exceptions to this). Therefore the payload content can simply be encoded as a set of bytes.

The .carrcarrier frame knows the size of the payload. Thus no special encoding for data frames is required.

Manifest Frame

Message manifests SHOULD generally fit into a single .carrcarrier frame. This may not be the case on low-bandwidth connections. Additionally, because the manifest has no well-defined representation in the ERIS spec, we need to wrap it in our own encoding schema.

message Manifest {
    string root_urn = 1;
    string root_salt = 2;
}

Netmod peering range

When establishing a peering relationship between two routers their respective netmods MAY have to negotiate some state between them. In most cases this protocol should be simple. In any case, the bitflag range 0000 0000 001x xxxx is reserved for such purposes (in decimal numbers 32 to 63). CarrierFrame's in this modes range MUST not be passed to the router. Instead their payloads SHOULD be parsed by the receiving netmod to influence peering decisions.

It is left up to the netmod implementation to specify how this range is used. Netmods that wish to interact with each other SHOULD coordinate usage of the same frame type flags.

Router peering range

Similar to the netmod peering protocol range, routers have the ability to exchange data with their immediate peers about who they are, where they can route, and any other information that may impact neighbour routing decisions. The bitblag range 0000 0000 01xx xxxx is reserved for such purposes (in decimal numbers 64 to 127). CarrierFrame's in this range MUST NOT be cached in the routing journal, or forwarded to any other peer.

message RouterAnnouncement {

}

SyncScopeRequest

SourceRequest

SourceResponse

PushNotice

DenyNotice

PullRequest

LinkLeapNotice

Appendix B: message routing

Ratman has two message sending capabilities: Push and Pull.

• Push routing is used by default when an active connection to a peer is present.
• Pull routing is used whenever there is no active connection, or for particularly large or static payloads.

Push routing

This is the default routing mode. It is used whenever an active connection is present, or if the sending application didn't provide any additional instructions.

A message stream is encoded into ERIS blocks which are encoded, encrypted, and content addressed. Each block is saved in the Router journal. Lastly a message manifest is generated and signed by the sending key. Blocks are sent to the recipient as they are generated, avoiding having to save the entire message in memory.

Lastly a message manifest is generated which contains the content hashes of each previous block. This manifest is signed by the sending key and also sent to the recipient.

On the receiving side blocks are kept in the journal until the manifest is received, then the message can be verified and decoded for the receiving application.

For messages larger than N MB (tbd), a sending router MUST generate a PushNotice before the final message manifest has been generated.

#![allow(unused)]
fn main() {
PushNotice {
  sender: <Address>,
  recipient: <Address>,
  estimate_size: usize, // size in bytes
}
}

A receiving router MAY accept this notice by simply not responding, or MAY reject the incoming message (for example via an automatic filtering rule). The sequence ID can be obtained from the containing .carrcarrier frame.

#![allow(unused)]
fn main() {
DenyNotice {
  id: <Sequence Id>
}
}

When receiving a DenyNotice a sending router MUST immediately terminate encoding and transmission. Any intermediary router that encounters a DenyNotice, which holds frames in its journal associated with a stream ID MUST remove these frames from their journals.

Pull routing

An incoming message stream is still turned into ERIS blocks which are encoded, encrypted, and content addressed. Each block is saved in the journal as it is generated, but not dispatched. Once the manifest has been created it will be sent towards the recipient peer.

This message routing mode will be used either:

1. When a sending client marks a message as a "Pull Payload"
2. When an active sending stream is interrupted by a broken connection

When the recipient router receives the signed message manifest it MAY generate a set of pull request messages for the sender.

#![allow(unused)]
fn main() {
PullRequest {
  urn: "25660fc21c9b25b7fde985b8ae61b36dedcb8b0192e691eda60dff7c0e5ff00a"
}
}

Appendix C: route scoring API

Consider the following scenario:

A -  B  - C
 \ D - E /

When routing a message from C to A first the routing table will return a set of available routes, with associated metadata:

#![allow(unused)]
fn main() {
RouteData {
  peer: [0, 2],
  meta: {
    mtu: 1222,
    size_hint: None,
    etd: "00:00:31.110",
  }
}
}

• peer :: the tuple of endpoint and target identifiers that identify a routing "direction"
• meta.mtu :: maximum transfer unit of the route
• meta.size_hint :: maximum total message size of the route. None means there is no imposed limit
• meta.etd :: "estimated transfer delay" uses the incoming announcement timestamp and calculates a delay to the current system time. While this metric can be very inaccurate due to different time sync mechanisms or badly configured timezones, a route scoring system may still access it.

The API for route scoring is defined via the RouteScore trait:

#![allow(unused)]
fn main() {
trait RouteScore {
    async fn configure(&self, r: &Router) -> Result<()>; 

    async fn irq_live_announcement(&self, a: &Announcement) -> Result<(), RouteScoreError>;
    
    async fn compute(&self, msg_size: usize, meta: [&RouteData]) -> Result<usize, RouteScoreError>;
}
}

Via the configure flag the router can be set-up to send live announcements to the given route score module by calling Router::req_live_announce(&route_scorer). For any incoming announcement Ratman will then call the irq_live_announcement endpoint with a given announcement frame.

#![allow(unused)]
fn main() {
enum RouteScoreError {
    UpdateFailed(String),
    
    ReSelect(enum Branch {
        Small,
        Delay,
        Trust,
        Neighbour,
    })
}
}

Bibliography

Irdest is not built from scratch and relies on a lot of existing work done over the last few decades of networking research. This section is very work in progress while we re-compile papers, articles, and design documentation from other networking projects.

• Serval mesh datagram protocol
• Serval rhyzome bundle specification
• Serval rhyzome manifest
• peer to peer networking application
• A simple pragmatic approach to mesh routing using BATMAN

Design Guide

This document outlines the basic design guide for irdest components.

Code design

• Use async only if required
• Use named generics instead of impl T in public functions
• Avoid shared state, if a channel will do the same thing

Visual design

To be figured out when we have visual design :')

Style Guide

A unified styleguide is meant to make the Irdest codebase more consistent and easier for external contributors to pick up. There are three separate style guides contained in this document.

• Source code
• Commit messages
• Commit history

When sending contributions, please make sure to check that your submission adheres to these styles!

Source code checklist

• CI enforces formatting via rustfmt so please run it before sending an MR!
• Add the SDX header to any file that are changed significantly/ created! (what is SDX?)
• If a code block (for example a complex match) becomes less readable by letting rustfmt format it you MAY annotate it with #[rustfmt(ignore)]!
• Use named generic parameters in public facing APIs (So A: Into<A> instead of impl Into<A>) as this improves inter-operability of function API types.
• Structuring imports via nested { } blocks is encouraged. You however MUST NOT condense all imports into a single use statement!
  • Generally: try to make the imports look "pretty" and easy to parse for another contributor (this is vague I know - sorry)!

Commit messages

Prefix your commit message with the component name that the commit touches. For example: android: perform some minor housekeeping.

If a commit touches multiple components then you may ommit the component name, HOWEVER consider breaking the commit up into multiple parts if this makes sense.

Commit messages SHOULD be written in present-tense, passive voice. For example: irdest-core: add authentication module or ratman: improve frame collection algorithm complexity.

Valid components

We haven't been the most consistent about this in the past but please try to format your commit messages along one of these component identifiers:

• client/android-vpn
• client/echo
• client/mblog
• docs/developer
• docs/user
• docs/website
• docs
• netmod/datalink
• netmod/inet
• netmod/lan
• netmod/lora
• ratcat
• ratctl
• ratman/client
• ratman/netmod
• ratman/types
• ratman
• util/<crate name>
• ...

Commit history

You MUST NOT use merge commits, either while merging or in your branch history. Rebase your changes on top of the latest develop HEAD regularly to avoid conflicts that can no longer be merged.

This results in the smallest possible commit-change size. Avoid using squash commits whenever possible!

irde.st website

The irdest website is built via the static site generator hugo. Its contents and sources are part of the irdest mono repo.

Building the website

You need to have hugo installed on your system to build the website.

$ hugo build # build the website for deployment
$ hugo serve # serve the website for development

Build with Nix

Alternatively you can use the Nix package manager to build the website. The build process will create a result symlink to the generated site data.

$ nix build -f nix/ irdest-website
...

Website structure

The website structure is somewhat non-linear and uses a lot of hugo template features to support easy text translations. Following is a breakdown of the structure.

• Template
  • irdest-theme folder contains base HTML and CSS templates. The only page not generated via these files is the root page.
  • The root page template can be found in layouts/index.html
• Content
  • Markdown section content can be found in the content directory
  • The root page content is content/indemd and the content/home directory (to allow multi-language versions).