Irdest developer manual

Welcome to the Irdest developer manual. This book is a collection of documents explaining the concepts, ideas, and protocols of the irdest project. Its source files are part of the main irdest code repo.

Outline

• Social documentation -- how we work together
• Technical introducion -- how does irdest work
• Design guides -- design principles for both code and UI
• Website documentation -- working on the irdest website
• License information -- licenses in use by irdest

Social Documentation

This section outlines the social structure of the irdest project.

Communication

The irdest project uses Matrix as a development and social chat. Feel free to drop by to ask questions or hang out!

Code of Conduct

TLDR: be nice!

We want to foster an open and engaging atmosphere for irdest and the development community around it. Because of this we follow the "Contributor Covenant" code of conduct. A copy of it should have been included in the sources for this book.

How to contribute?

First of all: thank you for wanting to help out :)

The irdest source can be found in our mono repo. We accept submissions via our mailing list, and (in a more limited capacity) via GitLab merge requests. See sections below for details.

Reporting an issue

If you've encountered a problem using Irdest software, we would highly appreciate it if you could tell us about it.

Since we use our own GitLab instance (and don't want to open registrations without verification) it's hard to submit issues via GitLab.

To submit an issue, just write an e-mail to the community mailinglist, in a format like: [BUG] ratman: sometimes crashes when ... or [QUESTION] irdest-proxy: how to set ..., etc. Do please try to first search for an existing e-mail thread in the mail archive though.

Contributions via e-mail

The easiest way to contribute code is via e-mail. This can be done in two ways:

1. Send a patch via git send-email
2. Upload your contributions to a different forge/ repository, and send an e-mail pull request

Contribution via send-email

You can follow the guide at https://git-send-email.io/ to get yourself set up for sending e-mail patches.

For any patch set that touches more than one component, please include a cover-letter to explain the rationale of the changes.

Sending an e-mail pull request

To send a pull-request via e-mail you must first upload your changes to your own copy of the irdest repository. You can host this anywhere that is convenient to you (for example GitLab or Codeberg).

Contributing via GitLab merge requests

If you want an account for development, please say hi in the Matrix channel so we know who you are.

• If a relevant issue exists, please tag in your description
• Include a short description of the accumulative changes
• If you want your history to be rebased/ merged, please clean it up to be useful. Otherwise we will probably squash it.
• Feel free to open a work-in-progress MR as a place to have a discussion about changes or to get feedback.

Submitting an e-mail patch

If you can't contribute via GitLab , you're very welcome to submit your patch via our community mailing list.

The easiest way of doing this is to configure git send-email.

Without git send-email

• Send an e-mail with the title [PATCH]: <your title here>.
• Format your patch with git diff -p
• Don't send HTML e-mail!
• Make sure your line-wrapping is wide enough to allow the patch to stay un-wrapped!

Lorri & direnv

You can enable automatic environment loading when you enter the irdest repository, by configuring lorri and direnv on your system.

 ❤ (uwu) ~/p/code> cd irdest
direnv: loading ~/projects/code/irdest/.envrc
direnv: export +AR +AR_FOR_TARGET +AS +AS_FOR_TARGET +CC
        // ... snip ...
 ❤ (uwu) ~/p/c/irdest> cargo build                           lorri-keep-env-hack-irdest
 ...

Contributor Covenant Code of Conduct

Our Pledge

In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.

Our Standards

Examples of behavior that contributes to creating a positive environment include:

• Using welcoming and inclusive language
• Being respectful of differing viewpoints and experiences
• Gracefully accepting constructive criticism
• Focusing on what is best for the community
• Showing empathy towards other community members

Examples of unacceptable behavior by participants include:

• The use of sexualized language or imagery and unwelcome sexual attention or advances
• Trolling, insulting/derogatory comments, and personal or political attacks
• Public or private harassment
• Publishing others' private information, such as a physical or electronic address, without explicit permission
• Other conduct which could reasonably be considered inappropriate in a professional setting

Our Responsibilities

Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.

Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.

Scope

This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.

Enforcement

Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at kookie@spacekookie.de. All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.

Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.

Attribution

This Code of Conduct is adapted from the Contributor Covenant, version 1.4, available at http://contributor-covenant.org/version/1/4

Irdest release checklist

This page is meant for anyone on the project with release access. Potentially this page should be moved into a Wiki or similar.

1. Make sure that CI passes on develop
2. Update CHANGELOG.md and make sure that it contains all relevant changes for the release.
3. Select a set of packages/ targets to include in the Releases section
4. Bump any relevant version numbers
5. Create a release/{version} tag corresponding to the new ratmand/ libratman version
6. Check the issue tracker and mailing list for issues that are being closed by this release
7. Update any relevant crates to crates.io (via cargo release)
8. Update the website to point to the new bundle download (as soon as release CI passes)
9. Re-deploy the website
10. Write a release description on the release tag
11. Optionally: write an announcement on the mailing list

Technical Documentation

This is the Irdest technical documentation tree, which gives a general overview of the various components that make up the Irdest project, as well as their inner workings.

This manual is a good starting point both to those interested in hacking on Irdest, and developing applications on top of it.

Introduction

Irdest is a distributed routing system, creating an address space over ed25519 keys. Each address on the network is a 32 bytes public key, backed by a corresponding private key. This means that both encryption and message authentication are built into the routing layer of the network. Each physical device can be home to many addresses, used by different applications, and it is not possible from the outside to re-associate a specific device with a specific address.

A lot of traditional networking infrastructures is built up in layers (see OSI model). Similarly, the irdest project replicates some of these layers. BUT... the layers between the OSI model and Irdest don't map directly onto each other and are only meant to illustrate difference in hardware access, user access, and scope.

Following is a short overview of layers in Irdest.

(While the following sections mention the OSI layers it's important to keep in mind that this is by convention. Nothing stops you from implementing a netmod for a high-level protocol like XMPP or a client for a low-level protocol like ARP. Alternatively, you can think of the two layers in Ratman's API architecture as "address scope" and "wire scope")

Network drivers

Network drivers establish and manage connections with peers via different underlying transport mechanisms. A driver (in the Irdest jargon called a "netmod") is initialised by the Irdest router "Ratman".

This allows the core of Ratman to remain relatively platform agnostic, while letting platform-specific drivers handle any quirks of the connections that are being established. For example, the netmod-datalink module uses NetworkManager to configure a wireless AP/ wireless connections to an existing host without user intervention. Ratman itself doesn't need to understand how to talk to NetworkManager.

Many different drivers can be active on the same device, as long as they are connected to the same router. In the OSI model, this roughly maps to layers 1 & 2.

Currently a driver needs to be specifically added to ratmand and included at compile time. We are working on a dynamic loading mechanism (either via .so object loading or an IPC socket), but this is still work-in-progress.

Ratman: the Irdest router

Ratman/ ratmand is a decentralised router daemon. It comes with a small set of utilities such as ratcat (a netcat analogue), ratctl (a batctl analogue), and a simple management web UI.

Clients communicate with Ratman via a local TCP socket and Protobuf envelope schema. For most use-cases we recommend the ratman-client library. Alternative implementations don't currently exist and this API is also extremely unstable, so please be aware of this for the time being!

In the OSI model, this maps roughly to layer 3 and 4.

Integration shims

Currently only one (work in progress) shim exists: irdest-proxy. This layer aims to create interoperability layers between existing IP networks and an Irdest/ Ratman network.

Different shims can exist, tunnelling Tor traffic through Irdest, or providing apps on mobile devices to take advantage of the "VPN" functionality of the OS.

In the OSI model this maps to layer 5. This is because in Ratman a connection is stateless, and thus no real session state exists. This shim introduces the concept of sessions for the benefit of existing applications that rely on them.

Clients

These are applications that use Ratman for their networking (either native or via an integration shim).

The Irdest project develops several of these as demonstrations as to what is possible to do with the Irdest network. While we of course hope that the Irdest application bundle makes onboarding into the network easier, we encourage others to work on top of this platform, or bridge it to other decentralised networking platforms.

Irdest comes with a few command-line applications and Irdest mblog, a Gtk usenet-inspired microblogging app!

What next?

If you are interested in writing an application for Irdest, or porting another application to use Irdest, you should familiarise yourself with the Ratman client SDK first.

If you want to get started hacking on Irdest, check out the "Hacking" section. In either case you may also want to read "Ratman internals"!

• Ratman SDK
• Ratman internals
• Hacking on Irdest
• Bibliography

Ratman client lib

This is the client library used to write applications for Ratman. Currently only a Rust implementation exists. The raw protocol is documented in the next chapter.

You can find ratman-client on crates.io and its documentation on docs.rs!

Workflow

There are three main steps to using the client-lib:

1. IPC initialisation
2. Address registration/ login
3. Message sending and receiving

IPC initialisation

By default the IPC socket for Ratman is running on localhost:5852. Many of the Irdest tools allow you to overwrite this socket address, to allow for local testing with multiple routers. We recommend that your application expose this option to users for testing purposes as well!

Address registration

An address for Ratman is associated with a cryptographic key pair. Currently we don't expose the private key from the router to applications (which will probably change in the future!)

When your application is given an address you should store it in your application state somewhere, along with the corresponding address auth token. These will be important the next time your application starts. For privacy reasons you should encrypt this data with a user password!

Message sending and receiving

Every "message" in Irdest is a stream, which is encoded into encrypted data blocks, along with a manifest which contains the root block reference and key. Currently the manifest is not encrypted, meaning that anyone intercepting it will be able to decode the rest of the block stream.

Sending a message stream happens somewhat asynchronously: the sending client will block until the router has received the full message, but there's currently no feedback how encoding or sending is going.

Receiving message streams can happen in two ways: synchronously and asynchronously via subscriptions. A subscription can persist across router restarts and will save missed items for a client to re-play when it next connects. Synchronous receiving blocks the main client socket, so no other commands can be exchanged. A subscription uses a dedicated TCP socket for a client to connect to.

When receiving you first get a "letterhead", which contains who the stream is from, who it is addressed to, it's final size, and additional metadata map currently not used).

Messages can either be sent to one ore multiple Addresses or a Namespace. Address-recipient messages will only be delivered to the devices that have advertised those addresses.

Namespaces are special addresses that any client can subscribe to. Namespace-recipient messages are spread across the whole network and any router/ client that subscribes to it can receive them. Namespaces are a great way for different instances of your application to find each other.

So for example a message sent to the address ECB4-30B9-4416-C403-716F-601F-FC56-9AD3-BD2E-3892-227A-84AD-E6FC-A1CE-0A92-03F6 will be carried across the network until it reaches this exact address.

A message sent to the namespace ECB4-30B9-4416-C403-716F-601F-FC56-9AD3-BD2E-3892-227A-84AD-E6FC-A1CE-0A92-03F6 will be delivered to all applications that are listening on this namespace.

API example

This is a small program demonstrating the most basic usage of the ratman-client SDK. At start-up it registers a new address, listens to any incoming messages, and returns them as they are to the sender.

use libratman::{
    api::{default_api_bind, RatmanIpc, RatmanIpcExtV1, RatmanStreamExtV1},
    tokio,
    types::Recipient,
    Result,
};

#[tokio::main]
async fn main() -> Result<()> {
    let ipc = RatmanIpc::start(default_api_bind()).await?;

    // Create a regular address
    // The second parameter can be used to create a specific namespace key
    let (addr, auth) = ipc.addr_create(Some(&"my-name".to_owned()), None).await?;

    // Start advertising the address on the network
    ipc.addr_up(auth, addr).await?;

    // Wait for incoming messages
    let (letterhead, mut reader) = ipc.recv_one(auth, addr, Recipient::Address(addr)).await?;

    println!("Receiving stream {letterhead:?}");
    let mut stdout = tokio::io::stdout();

    tokio::io::copy(&mut reader.as_reader(), &mut stdout).await?;
    Ok(())
}

Hacking on Irdest

Hey, it's cool that you want to hack on Irdest :) We recommend you install nix to handle dependencies. Depending on the directory you are in you can fetch development dependencies:

$ cd irdest/
$ nix-shell # install the base dependencies
...
$ cd docs/
$ nix-shell # install documentation dependencies

With lorri and direnv installed transitioning from one directory to another will automatically load additional dependencies!

Alternatively, make sure you have the following dependencies installed:

• rustc
• cargo
• rustfmt
• rust-analyzer
• clangStdenv
• pkg-config
• protobuf
• cargo-watch
• binutils
• yarn
• reuse
• jq

Building Ratman

Ratman provides several binaries in the ratman package. The name of the binary is ratmand. You can build the entire package with cargo. By default, the ratman-dashboard will be included, which requires you to build the sources with yarn first.

$ cd ratman/dashboard
$ yarn && yarn build
$ cd ../..
$ cargo build -p ratmand --all-features

Alternatively you can disable the dashboard feature. Unfortunately cargo doesn't allow selective disabling of features, so you will need to disable all default features, then select a new set of features as follows:

$ cargo build -p ratmand --release --disable-default-features \
                        --features "cli datalink inet lan lora upnp"
...
[cargo goes brrr]

Building irdest-echo

irdest-echo is a demo application built specifically to work with Ratman as a networking backend. Build it via the irdest-echo package with cargo.

$ cargo build -p irdest-echo --release
...

Building irdest-mblog

irdest-mblog is probably the most complete user-facing application that is native to the Irdest network. You can build it with Cargo, as long as you have gtk4 installed on your system (or using the Nix environment).

$ cd client/irdest-mblog
$ cargo build --release --bin irdest-mblog-gtk --features "mblog-gtk"
...

What now?

Check the issue tracker for "good first issues" if you are completely new to Irdest, and additionally "help wanted" issues if you already have some experience with the code-base.

Please also don't hesitate to ask us any questions! We're very happy to help :)

This section outlines some internal concepts that are in use by the Ratman routing daemon.

Gossip announcements

Ratman operates on the gossip protocol approach, where each address on the network repeatedly announces itself to other network participants. Based on these announcements the routing tables of passing devices and peers will be updated as needed. This means that no single device will ever have a full view of the network state, but will always know the "direction" a packet needs to be sent in order to make progress towards its destination. This is similar to how existing routing protocols such as BATMAN and BGP work.

A short example

To illustrate this capability, let's look at this simple network graph:

  +--------- [ D ] ----------+
  |                          |
[ A ] ------ [ B ] ------- [ C ]

Node A sends announcements to B an D, which will both proxy it to C. The router at C will use various metrics to decide which link is more stable, and declare it the "primary" for peer A.

When C wants to route a packet to A, it looks up the local interface over which it thinks it can reach A the best (for example D). It then dispatches the packet to D, knowing that this node must be closer to the destination to deliver the packet.

Announcement metadata

As part of the announcement protocol nodes may include metadata in the announcement. According to the current specification draft, it looks as follows:

Announcement {
  origin: {
    timestamp: "2022-09-19 23:40:27+02:00",
  },
  origin_sign: [binary data],
  
  peer: {
    ...
  },
  peer_sign: [binary data],
  
  route: {
    mtu: 1211,
  },
}

For the following section the announcement.route.mtu parameter is especially important!

Message slicing & streaming

An incoming message in Ratman is sliced twice: once into cryptographic blocks via the ERIS encoding, and then again for transport according to the path MTU outlined in the previous metadata section.

When slicing ERIS blocks, frames should be filled completely, with non-overlapping block boundries. This makes sure to not send frames that contain a lot of zero-padding.

This should be implemented as an iterator/ stream, which consumes at iterator/ stream of eris blocks.

ERIS block size = 4 bytes.
Frame size = 5 bytes.

Eris blocks: [1 2 3 4][5 6 7 8][9 10 11]
Frames:      [1 2 3 4 5][6 7 8 9 10][11]

ERIS block size = 12.
Frame size = 5.

Eris blocks: [1 2 3 4 5 6 7 8 9 A B C][D E F 10 11 12 13 14 15 16 17 18]
Frames:      [1 2 3 4 5][6 7 8 9 A][B C D E F][10 11 12 13 14][15 16 17 18]

Selecting frame sizes

The two block sizes supported by ERIS by default are 1kB and 32kB. For small messages these wil create a significant amount of overhead, especially on low-MTU connections.

For these cases we should have small-message optimisations, based on the size of the message, and the path MTU to the recipient.

Messages larger than 32 kB/ 2 kB on a path MTU of <1 kB/ <256 bytes respectively should be rejected by the sending router. We may want to add another small message optimisation between 2kB and 32kB max size messages.

Delay tolerance

This section will be expanded when the implementation of delay tolerance becomes more stable and usable. But in short: messages can be buffered by various nodes across the network when the destination is not reachable. This means that different networks can communicate with each other even when no stable connections between them exist (for example via a sneaker net). This does however require applications to be aware of long delays and handle them gracefully!

Roaming

Because of the distinction between network channels and routing it is easy to roam across network boundries. As long as a channel can be established (even just one-way), packets can be sent through the Irdest network with no knowledge of these network bounds.

It also means that a network can easily be composed of different routing channels. Local UDP discovery, TCP links across the existing internet, Wireless antenna communities, and even phones.

WIP specification

There is a work-in-progress specification available in the wiki!

Ratman client protocol

External applications can connect to the Ratman routing daemon via a Tcp socket connection (by default localhost:5852, but this can be changed via the user configuration). This page outlines the protocol in use for this connection.

If libratman SDK bindings exist for your language (currently only Rust) we highly recommend you use those instead of implementing the protocol from scratch.

Microframe

Messages in the Ratman client protocol use a very simple framing mechanism specific to Irdest called Microframe. Every Microframe message is split into a header and a body.

• modes: u16: encode the message command and payload types
• auth: Option<ClientAuth>: provide a previously registered auth token; field may be blank for initial connections
• payload_size: u32: encode the length of the main payload, up to the configured server maximum

The mode field is split into two parts: the namespace and the method. A namespace specifies the internal API in use for a given command. Not all namespace-command combinations are valid. In libratman the mode is thus constructed as follows:

#![allow(unused)]
fn main() {
pub const fn make(ns: u8, op: u8) -> u16 {
    ((ns as u16) << 8) as u16 | op as u16
}
}

The following namespaces are available:

The following methods are availble:

Since the microframe header can have various sizes it is length-prepended with a 4-byte integer. A full API protocol transmission thus looks as follows:

[ 4 byte header size ]
[ 2 byte mode indicator ]
[ 32 bytes auth token ][ 1 byte placeholder (0) ]
[ 4 byte payload length ]
[ N byte payload ]

Command payload encoding

The available commands are described in libratman/src/api/types. More documentation to be added. Please feel free to ask if you run into any issues or have general questions.

Network drivers

In Irdest a network driver is called a netmod (short for network module). It is responsible for linking different instances of Ratman together through some network channel.

The decoupling of router and network channel means that Ratman can run on many more devices without explicit support in the Kernel for some kind of networking.

Because interfacing with different networking channels comes with a lot of logical overhead these network modules can become quite complex and require their own framing, addressing, and discovery mechanisms. This section in the manual aims to document the internal structure of each network module to allow future contributors to more easily understand and extend the code in question.

Backplane types

There are three types of network backplanes that Irdest can interact with:

• Broadcast :: only allowing messages to all participants
• Unicast :: only alloing messages to a single participant
• Full range :: allowing both broadcast and unicast message sending

Netmod API

The netmod Endpoint API looks as follows:

#![allow(unused)]
fn main() {
#[async_trait]
trait Endpoint {
    fn msg_size_hint(&self) -> usize;

    async fn peer_mtu(&self, target: Option<Target>) -> Result<usize>;

    async fn send(&self, frame: Frame, target: Target, exclude: Option<u16>) -> Result<()>;

    async fn next(&self) -> Result<(Frame, Target)>;
}
}

(This API is still in flux and needs to be extended in various ways in the future. Please note that this documentation may be out of date. If you notice this being the case, please get in touch with us so we can fix it!)

• msg_size_hint is used to communicate a maximum size per message transfer and will be used to populate the announcement.route.size_hint parameter (on endpoints that support this!)

• peer_mtu is used to determine the immediate hop MTU to a target and will be used to populate the announcement.route.mtu parameter

• send is used to send messages.

  The exclude parameter is important on certain unicast & boardcast backplanes to prevent endless replication of flood messages.

• next is polled by the router in an asynchronous task to receive the next segment from the incoming frame queue.

This API is auto-implemented for all Arc<T> where T: Endpoint.

Internet overlay netmod (inet)

The main way to use Ratman with other people at the moment is via the internet overlay network module inet. It creates peering sessions over the internet and TCP. With that comes a significant amount of connection state logic and routing outside of Ratman, because each instance of inet can be connected with many other instances of inet.

Structure diagram

Following is a class structure diagram for the three main components of the inet driver. Note that Server is a dispatch-type, meaning that after allocation it copies itself to a private task-stack and remains running until the containing application is shut down.

TODO: figure out why the mermaid graph is borked

classDiagram-v2

class InetEndpoint {
    +Arc[Routes] routes
    +ChannelPair channel
    +start( bind )
    +port()
    +add_peers( peers )
    +send( target, frame )
    +send_all( frame )
    +next()
}

class Routes {
    +AtomicU16 latest
    +BTreeMap[Target, Peer] inner
    +next_target()
    +add_peer( target, peer )
    +remove_peer( target )
    +exists( target )
    +get_peer_by_id( target )
    +get_all_valid()
}

class Server {
    +Option[TcpListener] ipv4_listen
    +TcpListener ipv6_listen
    +port()
    +run()
}

Flowchart

While the inet driver doesn't have a lot of type components, their interactions can get quite complex. Furthermore there are some stateless function components that can't be expressed in a traditional class diagram.

digraph G {
  size="8,30!"
  graph [fontname = "Handlee"]
  node [fontname = "Handlee"]
  edge [fontname = "Handlee"]
  splines="polyline"
  bgcolor=transparent
  nodesep=0.5
  

  subgraph cluster_0 {
    color=orange
    node [color=orange]
    
    A [label="InetEndpoint::new"]
    B [label="add_peers()"]
    S [label="Listen for\nincoming connections"]
    C [label="for each peer", shape=Mdiamond]
    D [label="Resolve address"]
    E [label="start_connection()"]
    E1 [label="connect()"]
    E2 [label="handshake()"]
    E3 [label="Add peer\nto Arc<Routes>"]
    E4 [label ="SPAWN peer.run()"]
    E5 [label="setup_cleanuptask()"]
    Z [label="SPAWN restart.recv()"]

    A -> B
    A -> S
    B -> C
    C -> D
    D -> E
    E -> E1
    E1 -> E1 [label="retry"]
    E1 -> E2
    E2 -> E3
    E2 -> E4
    E3 -> E5
    E5 -> Z
    
    label = "Initialisation"
    fontsize = 20
  }
  
  subgraph cluster_1 {
    color=cyan
    label = "Peer Frame Receiver"
    node [color=cyan]
    
    P1 [label="run() loop", shape=Mdiamond]
    P2 [label="Read Frame"]
    P22 [label="Release Mutex"]
    P222 [label="break", style=filled]
    P3 [label="receiver.send()"]
    
    P1 -> P2 [label="Lock Mutex"]
    P2 -> P22 [label="No Data"]
    P22 -> P1 [label="yield_now()"]
    P2 -> P222 [label="Read failed"]
    P2 -> P3 [label="Valid Frame"]
  }
  
  subgraph cluster_2 {
    color=green
    label="Server loop"
    node[color=green]
    
    S1 [label="for each\nincoming", shape=Mdiamond]
    S2 [label="SPAWN\nhandle_stream()"]
    S22 [label="Drop stream"]
    S3 [label="accept_connection()"]
    
    S -> S1
    S1 -> S2 [label="valid"]
    S1 -> S22 [label="invalid"]
    S2 -> S3
    S3 -> E4 [label="valid"]
    S3 -> S22 [label="invalid"]
  }
}

LoRa broadcast driver

The netmod-lora endpoint uses the lora-modem embedded firmware in the background to communicate with the actual radio hardware. Check the user manual on how to set this up.

Irdest LoRa implements its own framing mechanism, incompatible to LoRaWAN. We do set the magic number equivalent bit from LoRaWAN to a different identifier though, which means that LoRaWAN devices should ignore Irdest frames.

LoRa is a broadcast backplane, which means that unicasts can only be implemented via filtering.

More documentation to follow. For more in-progress notes check out this wiki page!

Mesh Router Exchange Protocol (MREP)

Status: pre-draft First revision: 2022-04-05 Latest revision: 2024-09-18

The Irdest router Ratman exchanges user packets as well as routing metadata with neighbouring routers. This communication is facilitated through the "mesh router exchange protocol". It has three scopes.

1. Exchange user data packets
2. Collect connection metrics and transmission metadata
3. Perform routing decisions according to these metrics

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC2119

Basics

The Mesh Router Exchange Protocol specifies different mechanisms for multiple routers to communicate with each other in order to facilitate the flow of messages across a shared network.

Routers are connected with each other via different communication channels (or backplanes), meaning that the routing logic is decoupled from the connection logic. This specification does not make assumptions on the API between a router and the connection logic, but does place requirements and limitations on the information exchange that a valid implementation MUST provide between the two component layers.

Simultaneously an Irdest router MUST allow connections from client applications, which can register addresses on the network. A client application can register as many addresses as it needs or wants. From a network perspective the relationship between an address and a device can't necessarily be proven.

The terms "Address" and "Id" may be used interchangeably in this specification draft.

A word on encoding

This specification uses Rust-like pseudo-code to express datastructures. For over-the-network communication MREP uses a low-overhead encoding format called "encoding frames".

The "encoding frames" format supports the following data types. Specific message types are documented in Appendix A:

• u8, u16, u32, and u64, encoded as 1, 2, 4, or 8 big-endian bytes
• bool, encoded as 1 byte which is either 0, or 1
• Option<T>, encoded as a zero-byte
• [u8], encoded as a 2-byte big-endian length indicator, followed by the raw data buffer
• CString, encoded as a c-style string and zero-byte terminator
• Ident32, is a short-hand for a fixed-size 32-byte buffer which can contain either an address or content ID, and is not length-prepended

Encoding frames are not self-documenting and instead message types MUST use a versioning byte which is then used to switch between older and newer type implementations.

The anatomy of a message

When sending a piece of data this is called a "stream", based on the fact that the data is streamed from a connected client to its local router, which encodes the content into eris blocks, which are streamed to the local journal.

From that point routes are selected and the blocks to send are sliced into "frames". A frame is analogous to a single network packet. A frame contains metadata such as the sender and recipients, signatures, and ordering information that allows each block to be re-assembled on the receiving end.

Intermediary routers can see which frames are associated with a block, but not which blocks make up a full stream. Because the eris encoding creates a tree of blocks, only the root reference (+ additional metadata) are required to re-assemble the stream. This message type is called a "manifest", which MUST be additionally encrypted to avoid eavesdropping by intermediary parties.

The transport encryption secret is calculated via a diffie-hellman key exchange between the private sender key and the public address key of the recipient (either a single address, or a namespace).

Address announcements

An address in an Irdest network is a 32-byte ed25519 public key, backed by a corresponding private key, which is not shared outside of the router the address belongs to. The private key MUST be encrypted with some kind of user-facing secret.

For new addresses to spread across the network Irdest uses a gossip announcement approach. Announcements are re-broadcast periodically (currently every 2 seconds) and have a cryptographically signed timestamp, which MUST be verified by any receiving router. This way the authenticity of an announced route can be guaranteed. Announcements with a timestamp outside a particular window of validity MAY be dropped, although this specification does not currently indicate when this should be the case.

Announcements MUST NOT be broadcast to the sender channel of the announcement to avoid infinite replication and an announcement ID that has been seen before by a router MUST be dropped.

Router announcements

Every router on the Irdest network also has a unique address which is not shared with client applications. Messages sent to this address MUST conform to the MREP Message specification (Appendix A).

Routers announce their address to other routers they are immediately connected to. However unlike regular Irdest addresses, these announcements SHOULD NOT be propagated, unless explicitly instructed to do so by the sending router. However this specification does not currently indicate when this would be the case.

Router announcements are re-broadcast periodically (currently every 30 seconds) to all immediately connected routers, via every connection channel. The receiving router of such an announcement MUST keep track of the connection channel and specific "target ID" of the connection in memory, but SHOULD NOT persist any of the announcement data, unless explicitly indicated by the protocol. Currently this specification does not indicate when this would be the case.

Namespaces

A special kind of address exists called a "namespace". While a regular address uses an internal private key, a namespace uses a private key provided by a client application. This allows multiple applications to share the same encryption and verification key for a given namespace to share information amongst different instances of itself across the network.

Route selection / scoring

Because Irdest is a mesh network the selection of a route for any given frame is done by every router that handles it along the way. This is also due to the fact that no one network participant can have a full picture of the network topology and is thus dependent on peers to forward frames to whichever of their neighbours is best suited to deliver a particular frame.

A neighbour in Irdest is a device that is connected to the current router via some communication channel (netmod). The terms "netmod", "connection", and "neighbour" are used interchangably.

Irdest uses two different route selection (or "route scoring") mechanisms.

Live route scoring

When a live connection exists this scorer is used. A connection is considered "live" when the router has received an address announcement from the recipient address from a given neighbour in the last 30 seconds.

Because announcements are re-broadcast every 2 seconds this gives lenience to "network wobbles" and temporary connection drop-outs. When announcements stop being received from a neighbour it is marked as Idle and skipped when doing route selection via the live scorer.

The live scorer uses both ping latency (calculated based on the signed timestamp in an announcement) and a measured available bandwidth approximation of a given connection.

When a router only has a single link available to reach a peer, this link MUST be used. When there are multiple routes to a target the links are sorted by their ping times and the lowest ping link MUST be used.

When two ping times are within 10% of each other (i.e. 10ms vs 11ms) the available bandwidth of a link is used as a tie-breaker. When the bandwidth for each link is also within a 2% window of each other, or one of the links has failed to measure a bandwidth (the announcement didn't contain it, or it was set to 0 for other reasons) only the ping time SHOULD be used to determine the route.

Because ping times are measured end-to-end the overall ping time to a target decreases as a frame gets closer to its destination. Routing loops can be avoided, because even when a secondary route exists that may provide more bandwidth it is not advantageous for a router to "send back" a frame as it would increase the ping time.

Links that a frame was received on MUST be excluded from route selection!

Store & Forward scoring

When no live link to a target address exists (i.e. not announcement has been received within the last 10 seconds) the routing behaviour of the live scorer is inverted:

If only one link towards a target address exists it MUST be used. If multiple links exists, they are sorted by available bandwidth and the link with the highest bandwidth MUST be used. When two bandwidth values are within a 10% window of each other the ping time to a target MAY be used.

When doing a store & forward routing strategy an implementation MAY rely on the fact that trusted addresses may spend more time in closer proximity to each other, meaning that a trust score can be used to rank them (see Bubble Rap routing in the bibliography). This mechanism is currently not implemented.

Security

(I am not a cryptographer and this section will have to be expanded/ reviewed in the future)

All user messages sent through Irdest are encrypted via the ChaCha20 stream cipher, provided by the ERIS block slicing specification which is used to encode user payloads.

An address in Irdest is an public key (also called "address key"), backed by a corresponding secret key. Keys are generated on the ed25519 twisted edwards curve.

Because ChaCha20 is a stream cipher it requires a symmetric secret to work. For this purpose we convert both secret and public keys from the edwards curve representation to the montgomery curve representation, and use this for the x25519 diffie-hellman handshake between the sending address secret key and the recipient address key.

There are two layers of signatures in Irdest. The base layer Frame (defined in Appendix A) contains space for an ed25519 signature. All message payloads are signed by the sending address edward's curve key (currently using ed25519-dalek) and can be verified by any node a message traverses (since the sending address is visible to any network participant).

For user payloads, ERIS guarantees message integrity by verifying block content hashes against the recorded versions in the manifest. This manifest message is also signed via the basic Frame delivery mechanism. Thus user message integrity can be guaranteed.

Message encoding & delivery

User payloads are encoded via ERIS, sliced into carrier frames, and sent towards their destination (see Appendix A on details).

This uses two basic message types: DataFrame and ManifestFrame. Messages in Irdest are sessionless streams, meaning that data is streamed between different Irdest routers, but buffered into complete messages before being exposed to the recipient application.

ERIS specifies a "Read Capability" which for the purposes of Irdest and this spec we are calling the "Manifest".

For a DataFrame the payload of the underlying carrier frame is entirely filled with content from a re-aligned block stream. Frames MUST NOT be padded with trailing zeros to fill the target MTU.

A ManifestFrame contains a binary encoded version of the "Read Capability". If this manifest is too large for the containing carrier frame, it is split into multiple frames (see Appendix A: Manifest Frame)

Journal sync

Irdest allows devices to connect to each other via short-lived (or "ephemeral") connections. One such application is Android phones, where p2p WiFi connections can only be established with a single other party at a time. Bluetooth mesh groups are possible, but are also significantly limited in the number of active connections.

For this purpose we introduce the "journal sync" mechanism.

An Irdest router MUST contain a journal of content-addressed blocks of data (see Appendix B). Messages are indexed via their content hashes, as well as the recipient information. A journal sync is a uni-directional operation, which should be applied in both directions of the link. What that means is that journals are not so much synced, but propagated.

Let's look at an example to demonstrate the process.

Routers A and B are connected to each other via an ephemeral connection (req_ephemeral_connection is called by a netmod driver which has established the connection).

First both routers exchange a list of known addresses. Future versions of this specification MAY implement some kind of compression or optimisation for this transfer, since routing tables may get quite large.

#![allow(unused)]
fn main() {
SyncScopeRequest {
  addrs: BTreeSet<Address>,
}
}

Outline:

• Exchange list of known addresses (with an optimisation for "last recently used")
• Forward blocks addressed to any of the known addresses
• How to avoid re-transmit loops in a group of phones?
• How to avoid having to send too much data?
• Loops between people who both infrequently see the same peer address? Who gets the frames? Both? (probably)

AGPL compliance

Ratman is licensed under the AGPL-3.0 license and as such needs to be able to provide its own source code.

It is not possible to query the source of a node more than one router edge away from your own since router address announcements do not propagate across the network.

A router MAY at any time send a source request to a connected router. The request is time-stamped to avoid repeated and duplicate requests.

#![allow(unused)]
fn main() {
SourceRequest {
  date: "2022-09-22 03:18:32"
}
}

As a response, the recipient router MUST send a SourceResponse reply. The response doesn't contain the source code. Instead it describes the source that is running. A SourceResponse MUST contain the source_urn field. Every other field is optional, but a router SHOULD still provide them. The note field SHOULD contain a list of patch-names that have been applied to the node, if the number_of_patches is not zero. Otherwise this field SHOULD remain empty.

#![allow(unused)]
fn main() {
SourceResponse {
  version: "0.5",
  number_of_patches: 0,
  source_url: "https://git.irde.st/we/irdest",
  source_urn: "b5bb9d8014a0f9b1d61e21e796d78dccdf1352f23cd32812f4850b878ae4944c",
  note: "Hier könnte Ihre Werbung stehen",
}
}

A recipient of this SourceResponse can now check whether the source code their node is running is the same as the router that responded, by checking the source_urn against their own source version (TODO: specify how this URN is generated).

In case the recipient doesn't already have this source code they can now send a PullRequest to the sending node:

#![allow(unused)]
fn main() {
PullRequest {
  urn: "b5bb9d8014a0f9b1d61e21e796d78dccdf1352f23cd32812f4850b878ae4944c",
}
}

Low bandwidth modes

Some links in an Irdest network may be extremely low bandwidth, for example when using netmod-lora for long range communication. This severely constricts the maximum transfer size (< 255 bytes), on a < 1% duty cycle. This means that the maximum incoming message size MUST be constricted as well.

In these cases the "Small Message Optimisation" (SMO) MUST be used. Following is a table that outlines the selection of encoding block sizes based on the determined path MTU and size-hint (via announcement.route.mtu and announcement.route.size-hint)

For these cases we should have small-message optimisations, based on the size of the message, and the path MTU to the recipient.

Messages larger than 32 kB/ 2 kB on a path MTU of <1 kB/ <256 bytes respectively should be rejected by the sending router. We may want to add another small message optimisation between 2kB and 32kB max size messages.

MTU leap-frogging

A frame may encounter a netmod link that doesn't allow for a sufficiently sized MTU

In some cases, the path MTU information on the sending node was incorrect, and a set of frames will encounter a link that is too low-bandwidth to support their size. In this case the "leap-frogging" protocol should be used.

The first frame in a series that is too large to transmit over a connection will be prepended with this metadata section:

#![allow(unused)]
fn main() {
LinkLeapRequest {
  seq_id: "1D90-C2AB-E50D-A4EC-F88C-BD9E-818B-7006-7D32-BED0-4EEC-83F0-756E-D856-40AA-B611",
  inc_mtu: 1222,
  look_ahead: false,
}
}

• seq_id :: the sequence ID for the incoming set of frames. This identifier is used to determine which frames need to be re-sliced
• inc_mtu is the size of incoming frames

MTU leap-frogging performs a single step of look-ahead. This means that a router receiving a LinkLeapRequest MUST perform an MTU look-ahead if request.look_ahead is set to true (and subsequently set it to false). This means that up to two link MTU limitations can be "skipped over" before having to re-collect into the original frame size and re-slicing.

For an incoming LinkLeapRequest a router MUST spawn a LeapFrameCollector

Appendix A: MREP Message specification

This section of the specification outlines the way that MREP container messages are encoded. As per the "encoding frames" rules any optional field that is not present MUST be replaced with a zero byte.

The basic container type of any message in an Irdest network is a carrier frame, which consists of a header and optional payload. The header has the following structure:

#![allow(unused)]
fn main() {
CarrierFrameHeader {
  version: u8,
  modes: u16,
  recipient: [u8; 32] (optional),
  sender: [u8; 32],
  seq_id: [u8; 34] (optional),
  signature: [u8; 32] (optional),
  payload_size: u16,
}
}

This message structure is byte aligned.

• version :: indicate which version of the carrier frame format should be parsed. Currently only the value 0x1 is supported
• modes :: a bitfield that specifies what type of content is encoded into the payload
• recipient :: (Optional) recipient address key. May be replaced with a single zero byte if the frame is not addressed (see below).
• sender :: mandatory sender address key
• seq_id :: (Optional) sequence ID for push messaging payloads, mtu-leap protocol, etc
• signature :: (Optional) payload signature, generated by the sending key. May be replaced with a single zero byte if the frame has a payload-internal signature (see below).
• payload_size :: 16 bit unsigned integer indicating the size of the data section. Frame payloads larger than 32kiB are not supported!

Importantly, the CarrierFrame does not include a transmission checksum to detect transport errors. This is because some transport channels have a built-in checksum mechanism, and thus the effort would be duplicated. It is up to any netmod to decide whether a transmission checksum is required.

Following is a (work in progress!) overview of valid bitfields. If a field is not listed it is invalid! Routers that encounter an invalid message MUST discard it.

Announcement

Announcement frames are special in that they MUST set the recipient and signature field to a single zero byte. This is because announcements are not addressed, and contain a payload-internal signature system. All other message types handled by this specification MUST include both a recipient and signature!

The announcement payload consists of multiple parts:

#![allow(unused)]
fn main() {
OriginData {
  timestamp: CString
}
}

The origin data is signed by the announcement sender and MUST not be modified. Any announcement with an invalid origin data signature MUST be discarded.

#![allow(unused)]
fn main() {
PeerData { }
}

Peer data is refreshed at every hop and signed with the corresponding router address key. This field is currently left blank for future expansion.

#![allow(unused)]
fn main() {
RouteData {
  available_bw: u32,
  available_mtu: u32,
}
}

The route data is conditionally modified on every hop and corresponds to the fully traced route from an announced address to an arbitrary recipient on the network. Both the "bandwidth" and "maximum transfer unit" fields MUST ONLY be updated when the measured value from a particular receiving link is lower than the value that is already included in the route data.

For example, if the given announcement was received over a link that has a measured bandwidth of 32768 B/s (32KB/s) a router MUST update the field before re-broadcasting it to other peers. For this reason the route data section SHOULD NOT be signed.

#![allow(unused)]
fn main() {
RouteData {
  available_bw: 65536
  available_mtu: 1300,
}
}

Data frame

A data frame is already explicitly sliced to fit into a carrier frame (see "MTU leap-frogging" for how to handle exceptions to this). Therefore the payload content can simply be encoded as a set of bytes.

The carrier frame knows the size of the payload. Thus no special encoding for data frames is required.

Manifest frame

Message manifests SHOULD generally fit into a single carrier frame. This may not be the case on low-bandwidth connections. Because the manifest has no well-defined representation in the ERIS spec, we need to wrap it in our own encoding schema.

Additionally the manifest contains some important metadata about the stream that is about to be received, including the from and to address, the full stream size, and any auxiliary metadata, which can be filled in by the sending application.

The manifest SHOULD be encrypted with a shared symmetric secret based on the sending and receiving address keys. When receiving a manifest for an address that is not active, it SHOULD be stored until the address key for decryption becomes available.

#![allow(unused)]
fn main() {
struct Manifest {
    letterhead: Letterhead {
        from: Address,
        to: Recipient,
        stream_size: u64,
        auxiliary_data: Vec<(CString, CString)>,
    },
    block_size: u8,
    block_level: u8,
    root_reference: Ident32,
    root_key: Ident32,
}
}

Netmod peering range

When establishing a peering relationship between two routers their respective netmods MAY have to negotiate some state between them. In most cases this protocol should be simple. In any case, the bitflag range 0000 0000 001x xxxx is reserved for such purposes (in decimal numbers 32 to 63). CarrierFrame's in this modes range MUST not be passed to the router. Instead their payloads SHOULD be parsed by the receiving netmod to influence peering decisions.

It is left up to the netmod implementation to specify how this range is used. Netmods that wish to interact with each other SHOULD coordinate usage of the same frame type flags.

Router peering range

Similar to the netmod peering protocol range, routers have the ability to exchange data with their immediate peers about who they are, where they can route, and any other information that may impact neighbour routing decisions. The bitblag range 0000 0000 01xx xxxx is reserved for such purposes (in decimal numbers 64 to 127). CarrierFrame's in this range MUST NOT be cached in the routing journal, or forwarded to any other peer.

Currently the only specified message structure is the RouterMeta which is used for routers to exchange basic information about their state with their immediate neighbours. Received router protocol messages MUST NOT be forwarded to other peers.

#![allow(unused)]
fn main() {
struct RouterMeta {
    key_id: Ident32,
    available_buffer: Option<u64>,
    known_peers: u32
}
}

SyncScopeRequest

SourceRequest

SourceResponse

PushNotice

DenyNotice

PullRequest

LinkLeapNotice

Appendix B: message routing

Ratman has two message sending capabilities: Push and Pull.

• Push routing is used by default when an active connection to a peer is present.
• Pull routing is used whenever there is no active connection, or for particularly large or static payloads.

Push routing

This is the default routing mode. It is used whenever an active connection is present, or if the sending application didn't provide any additional instructions.

A message stream is encoded into ERIS blocks which are encoded, encrypted, and content addressed. Each block is saved in the Router journal. Lastly a message manifest is generated and signed by the sending key. Blocks are sent to the recipient as they are generated, avoiding having to save the entire message in memory.

Lastly a message manifest is generated which contains the content hashes of each previous block. This manifest is signed by the sending key and also sent to the recipient.

On the receiving side blocks are kept in the journal until the manifest is received, then the message can be verified and decoded for the receiving application.

For messages larger than N MB (tbd), a sending router MUST generate a PushNotice before the final message manifest has been generated.

#![allow(unused)]
fn main() {
struct PushNotice {
  sender: <Address>,
  recipient: <Address>,
  estimate_size: usize, // size in bytes
}
}

A receiving router MAY accept this notice by simply not responding, or MAY reject the incoming message (for example via an automatic filtering rule). The sequence ID can be obtained from the containing carrier frame.

#![allow(unused)]
fn main() {
struct DenyNotice {
  id: <Sequence Id>
}
}

When receiving a DenyNotice a sending router MUST immediately terminate encoding and transmission. Any intermediary router that encounters a DenyNotice, which holds frames in its journal associated with a stream ID MUST remove these frames from their journals.

Pull routing

An incoming message stream is still turned into ERIS blocks which are encoded, encrypted, and content addressed. Each block is saved in the journal as it is generated, but not dispatched. Once the manifest has been created it will be sent towards the recipient peer.

This message routing mode will be used either:

1. When a sending client marks a message as a "Pull Payload"
2. When an active sending stream is interrupted by a broken connection

When the recipient router receives the signed message manifest it MAY generate a set of pull request messages for the sender.

#![allow(unused)]
fn main() {
PullRequest {
  urn: "25660fc21c9b25b7fde985b8ae61b36dedcb8b0192e691eda60dff7c0e5ff00a"
}
}

Appendix C: route scoring API

Currently this API is not exposed outside of the router and there is no mechanism to load external route scoring modules at runtime. Nonetheless, additional route scorers can be added to the ratmand source tree, which comply to the following API:

#![allow(unused)]
fn main() {
#[async_trait]
trait RouteScorer {
    async fn configure(&self, 
        _ctx: &Arc<RatmanContext>, 
        _cfg: &mut ScorerConfiguration
    ) -> Result<()> {
        Ok(())
    }

    async fn irq_live_announcement(&self, 
        _a: &AnnounceFrame,
        _cfg: &mut ScorerConfiguration
    ) -> Result<()> {
        Ok(())
    }

    async fn compute(&self, 
        _stream_size: usize, 
        _cfg: &ScorerConfiguration, 
        _meta: &[&RouteData]
    ) -> Result<EpNeighbourPair>;
}

}

The implementation of configure and irq_live_announcement are optional, since the two default scoring mechanisms can run stateless. Nonetheless configuration and "announcement capture" is supported via this API.

For any given route scoring module a ScorerConfiguration type is constructed and kept in memory along-side the actual scorer. It's defined as follows:

#![allow(unused)]
fn main() {
struct ScorerConfiguration {
    trust_scores: BTreeMap<Address, u32>,
}
}

It contains a mapping of user-provided trust scores for a set of addresses Trusted addresses indicate that the user of the device has manually verified the authenticity and validity of a given peer address (for example, that an address does really belong to a given user). Addresses that have no associated trust score MUST NOT be included in the configuration set. Currently trust scores are defined per-device. It is left as an implementation detail to a scoring module how to compute trust scores based on the available router state.

When calling compute on the route scoring API a module SHOULD return a valid neighbour link and target ID (the link is a connection, the target is a selector for a specific peer on that sub-network). In case no route could be selected this function MUST return NonfatalError::NoAvailableRoute which either fails-over into the next selection strategy or pause routing until a live link could be established again.

Bibliography

Irdest is not built from scratch and relies on a lot of existing work done over the last few decades of networking research. This section is very work in progress while we re-compile papers, articles, and design documentation from other networking projects.

• Serval mesh datagram protocol
• Serval rhyzome bundle specification
• Serval rhyzome manifest
• peer to peer networking application
• A simple pragmatic approach to mesh routing using BATMAN
• Social bubble routing approach
• MTU determinations via QUIC connections

Design Guide

This document outlines the basic design guide for irdest components.

Code design

• Use async only if required
• Use named generics instead of impl T in public functions
• Avoid shared state, if a channel will do the same thing

Visual design

To be figured out when we have visual design :')

Style Guide

A unified styleguide is meant to make the Irdest codebase more consistent and easier for external contributors to pick up. There are three separate style guides contained in this document.

• Source code
• Commit messages
• Commit history

When sending contributions, please make sure to check that your submission adheres to these styles!

Source code checklist

• CI enforces formatting via rustfmt so please run it before sending an MR!
• Add the SDX header to any file that are changed significantly/ created! (what is SDX?)
• If a code block (for example a complex match) becomes less readable by letting rustfmt format it you MAY annotate it with #[rustfmt(ignore)]!
• Use named generic parameters in public facing APIs (So A: Into<A> instead of impl Into<A>) as this improves inter-operability of function API types.
• Structuring imports via nested { } blocks is encouraged. You however MUST NOT condense all imports into a single use statement!
  • Generally: try to make the imports look "pretty" and easy to parse for another contributor (this is vague I know - sorry)!

Commit messages

Prefix your commit message with the component name that the commit touches. For example: android: perform some minor housekeeping.

If a commit touches multiple components then you may ommit the component name, HOWEVER consider breaking the commit up into multiple parts if this makes sense.

Commit messages SHOULD be written in present-tense, passive voice. For example: irdest-core: add authentication module or ratman: improve frame collection algorithm complexity.

Valid components

We haven't been the most consistent about this in the past but please try to format your commit messages along one of these component identifiers:

• client/android-vpn
• client/echo
• client/mblog
• docs/developer
• docs/user
• docs/website
• docs
• netmod/datalink
• netmod/inet
• netmod/lan
• netmod/lora
• ratcat
• ratctl
• ratman/client
• ratman/netmod
• ratman/types
• ratman
• util/<crate name>
• ...

Commit history

You MUST NOT use merge commits, either while merging or in your branch history. Rebase your changes on top of the latest develop HEAD regularly to avoid conflicts that can no longer be merged.

This results in the smallest possible commit-change size. Avoid using squash commits whenever possible!

irde.st website

The irdest website is built via the static site generator hugo. Its contents and sources are part of the irdest mono repo.

Building the website

You need to have hugo installed on your system to build the website.

$ hugo build # build the website for deployment
$ hugo serve # serve the website for development

Build with Nix

Alternatively you can use the Nix package manager to build the website. The build process will create a result symlink to the generated site data.

$ nix build -f nix/ irdest-website
...

Website structure

The website structure is somewhat non-linear and uses a lot of hugo template features to support easy text translations. Following is a breakdown of the structure.

• Template
  • irdest-theme folder contains base HTML and CSS templates. The only page not generated via these files is the root page.
  • The root page template can be found in layouts/index.html
• Content
  • Markdown section content can be found in the content directory
  • The root page content is content/indemd and the content/home directory (to allow multi-language versions).