About 71% of Earth’s surface is covered with water, and about 97% of the water is in our oceans. Although the ocean plays a critical role in everything from the air we breathe to daily weather and climate patterns, we know very little about it. To really understand our oceans, we need a way to sense and observe the numerous complex processes that drive the ocean environment, and to report the data collected back to our data centers. While cabled ocean observatories have been established in a few locations, they are too expensive to setup and maintain for large scale data collection across the vast oceans.

Over the past few decades, wireless communication technology has percolated into every aspect of our lives, and we have come to take it for granted. This technology forms the bedrock of wireless sensor networks, allowing us to gather data with ease. Most of the wireless communication technology we use relies on electromagnetic waves (e.g. radio waves, visible light) that get rapidly absorbed by water. Hence the technology is ineffective for underwater communication, except at very short distances or extremely low data rates. Most underwater communication systems today use acoustic waves, which can travel long distances in the right conditions. At short distances in clear waters, optical communication systems are sometimes used for high speed communications. Although these communication technologies can be leveraged to establish point-to-point communication links, these links do not integrate well with networking technology available today.

The Unet project strives to develop technologies that allow us to build communication networks that extend underwater, be it via acoustic, optical, or even wired links. Some nodes in such networks may be above water, while others are underwater. In this handbook, we explore how to build such networks using UnetStack3 , an agent-based network technology that was developed in the Unet project.

This book is intended for readers interested in deploying networks that extend underwater, or developing technology or protocols for use in underwater networks. Part I of the book provides an overview, and is recommended for all readers. Part II is aimed at readers who wish to deploy and maintain networks that extend underwater. Part III is aimed at application developers and software engineers who wish to integrate with UnetStack-based networks. Parts IV and V dive deeper into UnetStack, and are intended for researchers and engineers who wish to develop, simulate and test novel underwater networking protocols.

The book assumes that readers have a basic understanding of traditional networking technology. While expert software development skills are not required to benefit from this book, familiarity with scripting or programming is essential. Readers with knowledge of Java, Groovy and/or Python will find it easy to follow the examples in the text, but even readers without prior knowledge of these languages should be able to pick up necessary skills along the way.

The Internet has changed our lives beyond anyone’s wildest expectations, fundamentally changing the way we interact, the way we learn, and the way we work. More recently, devices have started connecting to the Internet, and communicating with other devices. This Internet of Things (IoT) has the potential to have a huge impact on the way we understand our environment, and interact with it. Given that most of our planet’s surface is covered with water, would it then not make sense that at least some of these devices might be in water? Some devices might measure ocean temperature and acidification to give us a handle on climate change, while other devices might monitor fresh water quality to ensure safe drinking water for us. Autonomous underwater vehicles (AUVs) may patrol our coastal waters looking for intruders, or tracking down sources of pollution or nutients that encourage harmful algal blooms. Be it static sensors or mobile AUVs, we need a way to connect them into a network that we can communicate and interact with. The Unet project strives to develop technologies that allow us to do precisely this. In this handbook, we explore how to use UnetStack3 , a technology developed as part of the Unet project, to build communication networks that extend underwater.

Most wireless technologies today rely on electromagnetic waves that don’t propagate well underwater. Therefore, to extend IoT underwater, we typically need a mix of technologies — cabled links where possible, otherwise radio frequency (RF) wireless links above water, and mid-to-long range wireless acoustic or short-range wireless optical links underwater. A "Unet" network (which we will simply call Unet henceforth) consists of several nodes (underwater, on the surface of water, or above water) that communicate over various types of links, as shown in Figure 1 .

A Unet consists of many Unet nodes (e.g. underwater sensor nodes, Autonomous Underwater Vehicles (AUVs), gateway buoys, ground stations, boats/ships) that generate, consume or relay data over a variety of links:

A link is simply a logical connection between two nodes, often provided by equipping both the nodes with modems. We summarize various types of links in Table 1 .

Unet nodes are equipped with one or more network interfaces that allow communication over some of these links. For example, to communicate over an underwater acoustic link, we need an underwater acoustic modem . For an underwater optical link, we use an underwater optical modem . Most RF, GSM, satellite or wired links would be accessed over a standard TCP/IP network interface. In all cases, each Unet node would run the UnetStack software that allows us to effectively communicate over all of these types of links using a common Application Programming Interface (API). UnetStack API bindings are available for several languages including Java, Groovy, Python, Julia, C, Javascript, etc.

UnetStack has a several components, as depicted in Figure 2 :

The components are packaged into various editions . The community edition is downloadable free of charge for educational and research purposes. It has all the components required to develop, simulate, test and deploy Unets. The commercial and OEM editions package offer advanced functionality, better performance and tighter integration with vendor-specific hardware.

In the next few chapters, we will learn how to use UnetStack and how to customize it to meet our networking needs. In some cases, it may be necessary to prototype and simulate a Unet before it is actually implemented. We will also learn how to do that using the Unet simulator.

Download UnetStack community edition for your OS and untar/unzip it. Open a terminal window in the simulator’s root folder and start the simulator:

Open two web browser windows and key in the two http URLs shown above in each browser. This should give you a command shell for node A and node B in the two browser windows.

On the command shell for node A, type:

Address 0 is a broadcast address, so you did not need to explicitly know the address of node B to transmit a message to it. After a short delay, you should see the message on the command shell for node B:

Congratulations!!! You have successfully transmitted your first message over the Unet.

The [232] that you see on node B is the from address (of node A). The simulator automatically allocates addresses to each node. You can easily find out the addresses of both nodes (on either node):

You can try sending a message back from node B:

and you should see the message [31]: hi! on node A after just a short delay.

In the simulation, nodes A and B are placed 1 km apart. Since the speed of sound in water is about 1500 m/s (exact sound speed depends on temperature, salinity and depth), the signals take about 0.7 s to travel between the simulated nodes. This explains the short delays you see between sending the message from one node and receiving it on the other. You can also make use of this time delay to measure the distance between the nodes!

On node A, type:

You got an estimate of 1000 m for the range between the two nodes.

In real applications, you may want to send complex datagrams (messages) programmatically between nodes. The simplest way to do this is via the UnetSocket API ( Chapter 9 ). Let’s try it!

On node B, type:

On node A, type:

Node B will receive the bytes as a RxFrameNtf message. You can check the data in the received datagram on the command shell for node B, and close the socket:

UnetStack provides API bindings for many languages (Java, Groovy, Python, Julia, C, Javascript, etc). We demonstrate the use of the Python API here, but the usage is quite similar in other languages too.

We’ll assume you have Python 3.x already installed. Let us start by installing the UnetStack Python API bindings:

We will now write tx.py and rx.py scripts to transmit and receive a datagram respectively. We assume that you have the two-node network setup from the previous section with node A and B available on localhost API port 1101 and 1102 respectively.

First run python rx.py to start reception. Then, on a separate terminal window, run python tx.py to initiate transmission. You should see the received datagram printed by the rx.py script:

So far, we have worked with a simulator. While the experience is similar, it is not exactly the same. There is no real substitute for working with real modems. If you happen to have two UnetStack-compatible acoustic modems, you can use them to set up a simple 2-node network. Put them in a water body (tank, bucket, lake, sea, …​), power them on, and connect each to a computer over Ethernet. The setup would look something like this:

On each computer, open a web browser and key in the IP address of the respective modem. This should give us a command shell for node A and node B on the two computers.

When working with modems, you may need to adjust the transmit power level to a suitable level for use in the water body that you have the modems in. Too high or too low a power level will not allow the modems to communicate well. The modem transmit power can be adjusted using the plvl command. Type help plvl on the command shell for node A to see examples of how the command is used:

Assuming you have the modems in a bucket, you’ll need a fairly low transmit power. On node A, let us set the transmit power to -50 dB and try a transmission:

If all goes well, you should see the message on node B:

Of course you’ll see a different "from" address than the one shown in the example here. It will be the actual address of your modem A. In case you don’t see the message on node B after a few seconds, you may want to adjust the power level up or down and try again.

If you have UnetStack-compatible acoustic modems that support the BASEBAND service, you can use them to transmit and record arbitrary acoustic signals. Even without access to modems, you can try this out using the Unet audio SDOAM — a fully functional modem that uses your computer’s soundcard for transmission and reception. To start Unet audio, open a terminal window in the simulator’s root folder and type:

This should start up the SDOAM and open a browser with a command shell accessing the modem. If the browser does not automatically open, just enter the modem web URL shown above in your browser. At the command shell, you can try transmitting a message:

You should hear the transmission from your computer speaker! If you don’t, check your speaker volume and try again.

Next, try sending a simple 10 kHz tonal signal:

You should hear a 0.5 second 10 kHz tone from your computer speaker. The bbtx command requests transmission of a baseband signal. The function cw() generates such a signal based on the specified frequency and duration.

To generate the baseband representation of the signal you wish to transmit, you will need to know the carrier frequency and the baseband sampling rate of the modem:

For the Unet audio SDOAM, the carrier frequency is 12 kHz and the baseband sampling rate is 12 kSa/s.

You can equally easily ask the SDOAM to make an acoustic recording for you:

The recording is sent to you as a RxBasebandSignalNtf message with 12000 baseband samples in the signal field. You can check the first 32 samples:

The values you’d see would natually be different, since the SDOAM would have recorded whatever sounds it heard using your computer’s microphone.

The simplest way to interact with UnetStack is via the command shell (or simply shell ). The shell may be accessed on the console, a TCP/IP port or via the web interface. In Chapter 2 , we have already seen how to set up a 2-node network and access the command shell for each of the nodes using a web browser. For the rest of this section, we assume that you have shells open on nodes A and B.

On node A, we can ask for a list of agents running:

We can further ask for more details of a specific agent:

We asked for details of the agent phy , and we got a list of parameters supported by the agent. We can get or set individual parameters of the agent:

To find out more about a specific parameter, we can ask for help on the parameter:

We can also ask for help on an agent:

From this help, we see that phy agent also supports channel parameters (also known as indexed parameters). It supports two logical channels, CONTROL (1) and DATA (2). The CONTROL channel is meant for low-rate robust data transmission, whereas the DATA channel is typically configured for higher rate data transmission. Channel parameters work in the same way as normal parameters, but with an index:

Most agents also support some commands. For example, the phy agent supports the plvl command:

The plvl command simply displays or sets the powerLevel parameter of all channels. The same can be manually accomplished by setting or getting individual parameters, if desired:

While you can access a lot of functionality via parameters and commands, to fully harness the power of UnetStack, we require an understanding of the underlying messaging system between the agents. All agents support messages that expose their functionality. In fact, all parameters and commands are implemented by exchanging messages between the shell agent and other agents. In this section, we’ll take a brief look at how messaging between agents works.

Typically, we would want to send a request to an agent and get a response message back. This can be accomplished with the request call (or the equivalent alias << ) on the agent:

Here we made a request to the phy agent to transmit some data. The agent responded with an AGREE response, shortly followed by a TxFrameNtf notification from phy telling us that the transmission was successful.

We can also use the return value in a condition, but we need to remember that the return value from the request is a message:

Unsolicited notification messages can be received by subscribing to the topic of interest. For example, on node B, we can subscribe to physical layer events on node B:

Now, if we broadcast a frame from node A using phy << new TxFrameReq() , we will see the relevant reception events on node B:

The first event RxFrameStartNtf is triggered as soon as the frame is detected at node B. The second event RxFrameNtf is triggered when the frame is fully received, demodulated and successfully decoded at the receiver.

If all of this seems somewhat confusing to you, don’t worry about it. Most of the basic functionality of the stack can be accessed without having to deal with messages directly. As we need functionality that requires an understanding of messaging, we’ll gradually introduce them in later chapters.

The default UnetStack shell accepts any Groovy code, and so is very flexible:

You can also define closures (if you’re not familiar with closures, you can think of them as functions for now):

and call them later:

This only scratches the surface of what the command shell is capable of. However, it should provide you a basic understanding of how the shell works, and illustrate its power. To understand more, we suggest that you explore the online help . As you further understand the UnetStack and fjåge API, you’ll develop expertise on using the shell.

Unet nodes are identified by unique addresses within the Unet. Small Unets might use 8-bit addresses, supporting up to 255 different nodes. Larger Unets might use 16-bit addresses, supporting up to 65535 different nodes in the network. The address space is controlled by the parameter node.addressSize , and must be set to the same value (either 8 or 16) on all nodes in a Unet.

To check the current address size on your node:

Some node parameters have not been shown in the above listing for brevity.

Address 0 is a broadcast address. All other addresses may be assigned to nodes in a Unet. Each Unet node is also associated with a node name ( node.nodeName ). If a node name is not explicity set, it defaults to the string representation of the node address. Descriptive node names may be used, if desired:

It is recommended that, if descriptive node names are used, the corresponding node addresses be set using the ADDRESS_RESOLUTION service. This ensures that name-to-address resolution leads to the correct address for the node. The ADDRESS_RESOLUTION service can be accessed via the host() shell command:

The default ADDRESS_RESOLUTION agent in the UnetStack maps node names to node addresses using a hash function. The method reduces network traffic for host name resolution, but can lead to address conflicts between nodes if two names happen to map to the same address. It is the responsibility of the network engineer to resolve address conflicts manually during the setup of the network, if the default ADDRESS_RESOLUTION agent is used. For small networks, this is simply a matter of checking that all chosen node names in the network lead to unique node addresses:

Datagrams represent packets of data sent between nodes. Each node may have multiple agents and applications running on it, and so we need a way to specify which application the datagram is meant for. To aid with this, each datagram is associated with a protocol number that identifies the consumer on the destination node that the datagram is intended for. The consumer may be an agent or an end-user application. Protocol numbers can be thought of as port numbers in TCP/IP or UDP/IP.

The consumer may be an agent or an end-user application. Protocol number 0 ( Protocol.DATA ) is used for generic application data. Protocol numbers from 1 to Protocol.USER-1 (31) are reserved for use by default stack agents. Protocol numbers from Protocol.USER (32) to Protocol.MAX (63) are available for end-user applications to use.

On node B, type:

to wait for a reception with Protocol.USER .

On node A, type:

Node B will receive only the second message, since it is listening for datagrams with Protocol.USER only. We can confirm this by checking the data in the received datagram on the command shell for node B, and close the socket:

The Netiquette testbed in Singapore is a 3-node network that is deployed at sea (see Figure 4 ), and accessible over the Internet. Nodes A and B are cabled seabed mounted nodes, while node C is a solar-powered buoy. We use a simulated version of the Netiquette testbed to learn how to set up and operate small networks.

To start the simulated network, we simply run the netq-network.groovy simulation script:

We start off by checking the configuration of each node:

All nodes are configured to use 8-bit addresses. Node A has is address 232, node B is 31, and node C is 74. The origin is set to GPS location 1.216° N, 103.851° E. Locations are measured in meters relative to this origin, with x axis pointing east, and y axis pointing north. The mobility of the nodes is set to false to indicate that the nodes are static (for mobile nodes, mobility should be set to true ).

Let us first check the connectivity between the nodes:

The connectivity from node A to nodes B and C looks good. What about the connectivity from node B to node C?

Looks good too!

We can also check cross-check that the routes from node A to nodes B and C are direct:

The first trace shows that the datagram originated at node A (address 232), reached node B (address 31), and was sent back to node A. The second trace similarly went from node A to node C (address 74) and back. No hops in between, since our network is fully connected.

We can also make range measurements (in meters) between the nodes:

Once we have connectivity, we can of course send text messages from the shell:

and we see the text message on node B:

We have already seen in Chapter 2 and Section 4.2 on how to send text messages using the UnetSocket API from the shell, as well as from external applications. Hence we won’t dwell on it here.

Data is often stored in files. Transferring files between nodes is a common requirement. File transfers and remote access is disabled by default. Let us enable this on node B:

Now we can send & receive files, and run remote commands on node B. Let’s try it from node A:

On the shell for node B, we see the notification that the file abc.txt was successfully received:

You can also use fget to receive a file from a remote node, but you have to remember to set remote.enable = true on the receiving node:

The last command failed to get file def.txt , as it does not exist on node B.

When we send commands to execute on a remote node, they are usually silently executed and the output is not sent back. If we want the output to be shown to us, we need to explicity ask for it using tell . Since this is often required, we have a simple Groovy extensions shortcut ? to do this for us:

Sometimes we are not interested in the output, but simply want an acknowledgement that the command was successfully executed. For example, if we set the transmission power on a remote node, we want to know that it was set. That can be requested using the ack function.

As seen in Section 5.2 , some network nodes may know their own locations. This is useful for location-based routing and other applications. Depending on the application needs, we may wish to use different coordinate systems when setting up a network. There are 4 basic options to choose from:

When node locations are not accurately known, we can opt not to define any coordinate system. Local coordinate systems are preferred in applications where such a coordinate system can be agreed upon for the entire network. Range computation and localization is easier to do in local coordinates. GPS coordinates are used when node location is important, but a local coordinate system cannot be easily defined (e.g. ad hoc network with no prior knowledge of area of operation).

UnetStack supports all 4 options through a set of simple conventions:

In Section 5.3 , we measured the acoustic range between nodes A and B to about about 371 m. We can check this against distance computed from the location of nodes A and B. We first get the location of node A:

and then compute the distance to it on node B:

We see that it agrees well with the acoustic range!

It is often necessary to convert between the GPS coordinate system and the local coordinate system. To aid in this, UnetStack provides a set of utility functions:

The GpsLocalFrame class has additional constructors and utility methods to work with GPS coordinates in degrees, minutes and seconds, if desired.

The MISSION 2013 experiment in Singapore featured a 7-node network that was deployed at sea (see Figure 5 ) for several weeks. The network operated in a challenging area with complex 3D bathymetry, several reefs and heavy shipping. During the experiment, we transmitted more than 40000 frames of data and collected statistics on communication performance across various links in the network. These performance statistics are embedded in the Mission2013a channel model in UnetStack. We use a simulated version of the MISSION 2013 network to learn how to set up and operate larger networks that require routing.

To start the simulated network, we simply run the mission2013-network.groovy simulation script:

While the MISSION 2013 network is not physically very large (only about 1.5 km across), the challenging environment kept the network from being fully connected, i.e., not all nodes could directly communicate with all others. The average frame delivery ratio (number of successfully delivered frames / number of transmitted frames) on each link is shown in Table 2 . The link quality is also shown on the map in Figure 5 , with dark blue links being the good ones, dark green ones being the weak ones, and brownish one being the very poor link.

During the MISSION 2013 experiment, node 21 was a gateway node with surface expression and connectivity to the Internet (via a 3G cellular network). All other nodes were on the seabed and not directly accessible. So let us start by exploring the connectivity from node 21 to other nodes:

We see that the connectivity to nodes 22 and 28 is good, that to nodes 27, 29 and 34 is poorer, and to node 31 is non-existent. Since the simulation is probabilistic, your exact results may differ.

From Figure 5 and Table 2 , we see that node 28 has good connectivity to nodes 31 and 34, so perhaps we could relay datagrams via node 28. Although the link between 22 and 27 seems to be better than the rest, the connectivity to that node is generally poor. Let us set up the following routes:

On node 21, we add routes to nodes 31 and 34:

On nodes 31 and 34, we add routes to node 21 via node 28, and enable remote access:

Now, we can check out connectivity from node 21 to nodes 31 and 34 again:

Much better!

The pings to nodes 31 and 34 show rthops (round trip hops) to be 4, which makes sense, since we have 2-hop routes in each direction. We can ask the routing agent for a trace to check what route the datagram took:

This shows that the datagram originated at node 21, passed through node 28 before reaching node 31. Then on the way back, it passed through node 28 again, and reached us back at node 21.

Let us next try to do something using the routes we created. We can get node 21 to ask node 31 to measure the range to node 34 and report it to us. This request will be relayed via node 28, since our routing tables are set up to do so. Remember to set remote.enable = true on node 31 before making the request from node 21:

As you can see from Table 2 , the connectivity between nodes 31 and 34 is poor in this simulated network. You may need to try this command several times before you get a range estimate. When the ranging fails, you should see the message "ERROR: No response from remote node" back from node 31, which by itself demonstrates successful routing.

In the previous section, we learned how to set up static routes manually. But what if we are too lazy to determine the routes manually? Or if we don’t have access to the nodes on the seabed to set up routes? We can use the route discovery agent to populate the routing tables.

To see how to do this, let us restart our MISSION 2013 simulation so that the routing tables are empty (alternatively we can remove the routes we created earlier by typing delroutes on nodes 21, 31 and 34). We can verify that the routing table is indeed empty:

Now, start a route discovery to node 31:

Patiently wait for a minute or two before checking the routing table on node 21:

Your routing table may differ, as the route discovery process is probabilistic. We see that we now have a route to node 31 via node 29. Let us check the routing table on node 31 as well, to see if it has a corresponding entry for a route to node 21:

Indeed it does! In fact, it has 3 routes back to node 21, one via node 29, and two more via nodes 28 and 34. Of these routes, the route via node 28 has the largest metric, and so will be the route that is used. We can verify that by issuing a trace from node 21:

The UdpLink agent offers the LINK service ( Chapter 21 ) over an IP network.

To see how this works, let us revisit the MISSION 2013 network from Figure 5 . Recall that node 21 was a gateway node with surface expression, and was connected to the Internet via a 3G cellular IP connection. During the experiment, we had no direct acoustic connectivity between nodes 21 and 31, and hence we routed all communication to node 31 via node 28.

Let us consider a scenario where node 31 also has a surface expression and 3G cellular IP connectivity. In this case, it would be nice to have a direct link from node 21 to node 31 via UDP/IP. Let’s see how to set that up.

Fire up the mission2013-network.groovy network simulation (or if you already have it running from the last chapter, terminate and restart it, so that we have no routes in our routing tables). Connect to node 21’s shell and add the UdpLink agent, and setup a route to node 31 via the UDP link:

Similarly, connect to node 31’s shell and add the UdpLink agent as well as a route to node 21 via the UDP link:

Go back to node 21’s shell and see if you can communicate to node 31 via the UDP link:

and on node 31, you’ll see:

You’ll also notice that the communication is much faster, since the UDP/IP latency is low and data rate is much higher.

When we added the UdpLink agent in the last section, we set up static routes manually on both nodes. Let’s delete these routes on both nodes:

Now, let’s see what the route discovery agent does when we ask it to discover routes for us:

Note that the route discovery resulted in 3 routes to node 31 in this case. The first one is a single-hop UDP ( udplink ) route. The second one is an acoustic route (using uwlink ) via node 28, and the third one is a 3-hop acoustic route via node 22. We can see that the metric for the 2-hop and 3-hop acoustic routes is lower than that of the UDP route, and so the UDP route is used for data transfer. The metric is computed based on a combination of number of hops and the packet loss on a route.

You can check the routing table on node 31:

We see 3 routes (direct/ udplink , via node 28/ uwlink and via node 34/ uwlink ), and the route with the largest metric is still the udplink direct route.

If you recall from Section 2.4 , you opened a socket connection to UnetStack on the command shell with:

Since the command shell was running on the node you wanted to connect to, the meaning of this was clear. However, in general, you’ll probably be running your application in a different process, or even on a different computer. You’ll therefore need to provide details on how to connect to the node when opening a socket.

For example, to connect to UnetStack from an application over TCP/IP, we need to know the IP address and port of the API connector on UnetStack. Simply type iface on the command shell of node A to find this information:

The first entry starting with tcp:// is the API connector available over TCP/IP. The IP address and port number in this case are 192.168.1.9 and 1101 respectively. The IP address on your setup might differ, so remember to replace it in the example code below when you try it.

To connect to UnetStack from a Groovy application, typical code might look something like this:

The code in other languages looks similar. For example, in Python:

A simple example application in Python using the UnetSocket API was illustrated previously in Section 2.5 .

To send datagrams using a socket, we first specify the destination address and protocol number using the connect() method, and then use the send() method to send data (byte array). In Groovy:

If only a single send() is desired, the connect() call may be omitted and the destination and protocol number can be provided as parameters to send() :

On the receiving end, we specify the protocol number to listen to using bind() , and then receive a datagram using the receive() method:

The receive() method above is blocking by default. The blocking behavior can be controlled using the setTimeout() method, where the blocking timeout can be specified in milliseconds. A timeout of 0 makes the call non-blocking. If no message is available at timeout, a null value is returned. When the receive() call is blocked, a call to cancel() can unblock and cause the receive() call to return immediately.

You have already been introduced to agent parameters in Chapter 3 . Applications can obtain information about an agent by reading its parameters, and can control the behavior of the agent by modifying its parameters.

To access agent parameters, you first have to look up the relevant agent based on its name or a service that it provides. For example:

This will print the value of parameter MTU (maximum transfer unit) of the physical layer, and the physical layer dataRate of the CONTROL (1) channel. You could also change some of the parameters:

As we have already seen in Section 3.2 , the full functionality of UnetStack can be harnessed by sending/receiving messages to/from various agents in the stack. We earlier saw how to do that from the shell. We now look at how to use the UnetSocket API to send/receive messages to/from agents.

To request broadcast of a CONTROL frame, like we did before from the shell, we need to lookup the agent providing the PHYSICAL service and send a TxFrameReq to it:

For lower level transactions, we obtain a fjåge Gateway instance from the UnetSocket API, and use it directly. For example, we can subscribe to event notifications from the physical layer and print them:

In Groovy and Java, services, parameters and messages are defined using enums and classes. These are made available to the client application by putting the relevant jars in the classpath. In other languages (e.g. Python, Julia, Javascript), services and parameters are simply referred to as strings with fully qualified names (e.g. 'org.arl.unet.Services.PHYSICAL' ). Messages are represented by dictionaries, but have to be declared before use.

For example, in Python:

Since UDP is a datagram-oriented protocol, it is easy to map UDP datagrams to Unet datagrams. This is exactly what a UDP portal does, as shown in Figure 6 .

In this example, application X sends a UDP datagram to node A, where a UdpPortal agent listens on UDP port 7000 (arbitrarily chosen port number). The UdpPortal agent converts the UDP datagram into a Unet datagram and sends it to node B. The UdpPortal agent on node B receives this datagram, converts is back to a UDP datagram and sends it to application Y listening on UDP port 7778 (also an arbitrarily chosen port number).

Let us emulate this example with our favorite 2-node network. Fire up the 2-node network simulation, as before:

Open browser windows for shell access to each of the nodes. On node A, create a UdpPortal listening on port 7000 (since application X will send UDP datagrams to this port) and sending Unet datagrams (protocol 0 in this example, but that can be configured using the protocol parameter below) to node B:

The default port and clientPort for the UdpPortal are arbitrarily chosen to be 7777 and 7778 respectively. You can easily change them during creation of the UdpPortal, as we did above, or later, by setting the relevant parameter.

On node B, create the other end-point of the UDP portal to send the UDP datagrams to localhost UDP port 7778, where you will run application Y:

That’s it, your UDP portal is set up! Time to test it out!!

Open a terminal window on your machine and set up a simple UDP server listening on port 7778 (application Y):

Open another terminal window and set up a simple UDP client to send text datagrams to port 7000 (application X). Assuming your IP address is 192.168.1.9 , you can do this using the command shown below. Type a text message and press ENTER.

In a few seconds, you should see that text message appearing on application Y terminal:

The text message went through the Unet to get there!

You can do some cool things once you have set up the UDP portal. Here’s one real-life example:

Say, you wanted to stream video through the Unet. If you have ffmpeg installed, you can set up a UDP video client listening on port 7778:

and you can stream a video ( movie.m4v ) over UDP to port 7000:

The various flags control the quality, frame rate, and encoding of the video, and the pkt_size option controls the size of the datagrams sent.

A TCP portal is set up using the Portal agent. The Portal agent is quite similar to the UdpPortal agent, but provides more flexibility through the fjåge connectors framework. We can use a TCP connector for our TCP portal.

Restart your 2-node network, and on node A set up a TCP portal listening on port 7000:

On node B, create the other end-point of the TCP portal listening on port 7001:

That’s it, your TCP portal is set up! Time to test it out!!

Open a terminal window on your machine and connect over TCP/IP to node A:

Open another terminal window and connect over TCP/IP to node B. Type a text message and press ENTER.

In a few seconds, you should see that text message appearing on the TCP/IP connection to node A:

The text message went through the Unet to get there!

Since the Portal agent uses the fjåge connectors framework, it can easily work with any type of connector. Since fjåge provides a serial port connecor, we can easily set up a serial portal on each of your nodes:

Once you have the serial portal set up on all nodes, you can connect to the node’s serial port using a serial terminal application (e.g. minicom ) and type text messages just like you did with nc during the TCP portal test.

Imagine a network with gateway node A (a standalone buoy), surface node B (deployed from a boat), and an underwater node C (diver). Nodes A and B have underwater acoustic modems and in-air WiFi connectivity. Node C is fully submerged, and only has acoustic connectivity to nodes A and B. An diver tracking script on node B wishes to track the location of diver node C. This requires the script on node B to make range measurements btween node B and node C (easy to do!), but also between node A and node C (not so easy!). The traditional approach to this problem would be to deploy a helper agent on node A to make the range measurement, and have the script communicate with the agent using a custom protocol over a UDP link over WiFi. Another approach would be to enable remote access ( Chapter 24 ) on node A, and use rsh to execute the ranging commands on node A, and send back the results using tell . While this would work, it would be quite fragile. Wormholes provide a much simpler and robust way to do this.

We’ll use the Netiquette 3-node network to demonstrate how to do this:

Nodes A and B are assumed to be connected over an in-air WiFi network, and so we can enable UdpLink on both nodes and establish connectivity over it:

Next, we enable the Wormhole agent on both nodes, and set it up to use the udplink for connectivity:

That’s it for the setup! We’re now ready to prototype out our application over the wormhole.

The functionality we needed was to make range measurements between nodes B and C, and nodes A and C, from a script on node B. Let’s start off measuring range to node C from node B and call it r1 :

In a script, you’d probably want to use ntf = receive(RangeNtf, 5000) to access the RangeNtf that comes back, but we’ll stick to doing this just in the shell for the demonstration. Now comes the magic . From the shell on node B, we want to ask the ranging agent on node A (address 232) to make a range measurement to node C for us:

Now that we have r1 and r2 , we can use it to compute the location of the diver in our script.

In the diver tracking application above, we performed two-way travel-time (TWTT) ranging from nodes A and B to node C. While this works well, each diver location measurement required transmission of 4 frames. With one-way travel-time (OWTT) ranging (see Chapter 17 ), we could make each measurement in just one transmission from diver node C, as long as we had accurate clocks at all nodes.

First, let us enable OWTT ranging by synchronizing clocks between nodes A and C, and nodes B and C (see Section 17.3.2 for details):

OWTT is now enabled. To send out a beacon on node C, we use the beacon command (or equivalently send a BeaconReq message):

On nodes A and B, you’ll see the RangeNtf notifications:

Diver node C can now send out a beacon transmission regularly, and ranging agents on node A and node B will publish RangeNtf notifications on their respective agent topics. On node B, we would then want to subscribe to the ranging agent topics on both nodes. We are already subscribed to node B’s ranging agent, and connected over the wormhole (established in the previous section) to node A. Since we can refer to the ranging agent on node A as agent('ranging@232') , we should be able to subscribe to it:

Now, try sending a beacon again from node C, and see what we get at node B:

We got the notification from node B’s agent, but nothing from node A! 😕

By default, wormholes do not publish messages on topics. However, it is easy to enable certain topics to be published. To enable this, we set the wh.publish parameter on node A:

Now, try sending a beacon again from node C, and see what we get at node B:

Cool! We got both ranges on node B now. 🙂

We can now write our application script to call receive(RangeNtf) to get the ranges, and computes the position of the diver everytime the diver node transmits a beacon.

The AT command interpreter is available as a script engine that can be loaded using a shell agent:

Once started, you can connect to port 5001 (assuming you started the interpreter on TCP port 5001 as shown above) using a terminal window:

Now you can interact with the AT command interpreter.

A small set of basic AT commands are honored by the interpreter:

The interpreter can be customized using shell extensions:

Fields and methods exposed by a shell extension are made available in a shell using this command. Example:

The parameters to methods are specified as a comma-separated list after the = symbol in the command (e.g. -3 in AT~PLVL=-3 ). The parameters may be numeric ( int , long , float or double ), boolean represented by 0 or 1 as false and true respectively, or double-quoted strings (e.g. "this is a string" ).

Agent parameters may be listed, read and written to:

The command interpreter may make requests and receive message notification by defining the messages of interest and subscribing to appropriate topics:

Message formats defined using this command are available for requests and also used for notifications. If a message is not defined, notifications of that message type are silently ignored. The following command defines a message DRQ of class org.arl.unet.DatagramReq with 3 parameters: to , protocol and data in that order:

We also define other messages similarly:

Once we have defined the messages above, we can make a request to PHY to send a datagram to node 2 with protocol 0 and 3 bytes of data: [1,2,3] :

The notification for the datagram transmission completion will be displayed as an unsolicited notification:

The general notifications format as: ~agent>msg=parameter[,parameter]…​ . If any of the parameters are byte[] or float[] , they are not included in the parameter list. Instead a colon ( : ) is added at the end of the line, and the data in hex follows on subsequent lines. Once the data ends, a period ( . ) is sent on a single line. If multiple parameters are arrays, the number of array parameters is given by the number of colons at the end of the line, and each array is terminated by a period, followed by the next array. An example is shown below:

Without subscribing to a topic, we see that the user is not notified about the reception of a frame, although the message type is already defined:

After subscribing to PHY , the received message is reported:

Here we see that the data from the RXNTF is included after the notification message as a data block . This is the case for all byte[] or float[] parameters. Each data block may span several lines, and is terminated by a period ( . ) on a line by itself. The number of data blocks to follow a notification is denoted by the number of colons ( : ) at the end of a notification.

While data may be directly included in a request message, sometimes it is useful to load data into a data buffer first, and then use it multiple times for requests. This is managed using the following commands:

Data is represented as a series of hexadecimal bytes, and may span many lines. Data entry is terminated by a period ( . ) on a line by itself:

The above representation is convenient for byte[] parameters. However, the same representation is used for other data arrays, including float[] , where the IEEE floating point representation is used for the floating point number to be converted to a series of bytes.

An alternative data representation is useful for float[] , where the floating point numbers are directly specified:

For this representation, it is necessary to have a decimal place ( . ) in each number, and each line to contain only one floating point number.

To use the data buffer, we simply use "DATA" instead of the hexadecimal data in a message. For example:

To fully understand services, we need to formally define a few terms, many of which you are already somewhat familiar with:

In Section 9.5 , we have already come across the agentForService() function that helps us find an agent that provides a specific service. In this section, we’ll explore this a bit more.

Fire up your trusty 2-node network and connect to node A’s shell:

We asked for an agent that provides the org.arl.unet.Services.PHYSICAL service, and UnetStack responded back with an agent ID of an agent that can provide you that service. The name of that agent on node A is phy .

But what if there were more than one agents capable of providing the same service? We can ask for the list of all agents that provide a service:

Well, there was only one agent providing the PHYSICAL service. Are there other services that multiple agents provide? Indeed, there are:

The DATAGRAM service is provided by several agents. If you were interested in all incoming datagrams, you’d need to subscribe to the topics of all these agents!

What would have happened if you only asked for agentForService() instead of agentsForService() for the DATAGRAM service? Try it:

Any one of the agents in the list is returned!

You can also ask an agent for the list of services it provides:

or get a list of all services provided by all agents in the stack:

So let’s say you looked up the list of agents that provide the DATAGRAM service:

If you wanted to send a datagram, how do you pick which one you’d rather use? Different agents may provide different optional capabilities. If you were specifically interested in a particular capability (e.g. reliability), you could ask the agent if it supported that:

Here, we asked phy if it can do reliable datagram delivery, and it said "no". Then we asked uwlink , and it confirmed that it can. If you needed reliable delivery of our datagram, you should choose the latter.

You can also ask an agent to list all its optional capabilities:

The transport agent says it can do reliable datagram delivery, fragment & reassemble large datagrams (if necessary), report on the progress of large datagram transfers, and cancel datagram delivery half way through the process (if the user wishes to).

Another way you may choose a service provider is by checking its parameters. For example, the MTU parameter (defined in the DATAGRAM service) tells you what is the largest datagram the agent can deliver:

If you had a small datagram (56 bytes or less) to deliver, and you did not care about reliability, you could ask phy to deliver it for you. But, if your datagram was larger, even if you did not need reliability, you’d have to ask uwlink to deliver it for you.

The following services are currently defined in UnetStack:

You can enjoy reading more about these services in the next few chapters.

Fire up your 2-node network again, and open two browser windows — one connecting to node A’s shell, and the other to node B’s shell.

On node B:

We obtained a list of agents which provide the DATAGRAM service. We checked the MTU of the phy and uwlink agents, and found them to be 56 bytes and 848 bytes respectively. We decided to use the uwlink agent to communicate over a single-hop link between node A and B, and checked its capabilities. We found that it supports reliability and fragmentation. We then decided to listen to all datagrams received by node B’s uwlink agent by subscribing to its topic.

On node A:

If we look at the shell for node B, we should see the 3 successfully delivered datagrams:

We were able to successfully deliver datagrams from node A to node B in the examples in the previous section. We not only saw the DatagramNtf messages on node B, but also got DatagramDeliveryNtf on node A if reliability was enabled.

Let’s try it again, but with a small difference. On node A:

We transmitted a smaller datagram, and node A happily accepted it for delivery. However, if we look at the shell for node B, we don’t see a DatagramNtf message corresponding to the datagram, even though you had already subscribed to uwlink ! What’s going on? Let’s try it again, but this time enable reliability:

We see that the datagram was indeed delivered! And now, if we look at node B’s shell, we’ll see the delivery notification:

It seems that enabling reliability successfully delivered the datagram, but otherwise the DatagramNtf message did not appear on node B’s shell! You can try this many times, and the result will be the same. So it can’t be random packet loss in the network either. What’s going on?

To try and troubleshoot this, let’s subscribe to notifications from the phy agent to see if the data is arriving at the physical layer. On node B:

On node A, transmit the unreliable small datagram again:

On node B, we now see a couple of notifications:

The first notification says that the physical layer detected the start of a data frame. The second notification is for a received frame with 32 bytes from node 232 to node 31. That’s our datagram!!! But why is it delivered by phy and not uwlink , when it was sent by uwlink on node A? And why is it a RxFrameNtf instead of a DatagramNtf ?

Let’s solve the second mystery first. An RxFrameNtf is a subclass of DatagramNtf , so it is indeed a DatagramNtf message. We can easily verify this on node B:

Variable ntf contains the last notification received. It is the RxFrameNtf , and it is indeed an instance of DatagramNtf . So, we indeed got the datagram on node B, and it was delivered as a DatagramNtf with the correct metadata.

But why was it sent on phy agent’s topic and not uwlink agent’s topic, like all other datagrams we transmitted?

This is due to an optimization known as short-circuit delivery (introduced in UnetStack 3), depicted in Figure 8 . The uwlink agent on node A looked at the unreliable DatagramReq for 32 bytes and realized that it is within the phy agent’s capability (no reliability needed, and the datagram size is less than phy.MTU ) to deliver this without the help of the uwlink agent. It delegated the task to the phy agent, which in turn send the datagram to its peer on node B, and therefore it was delivered to us by the phy agent on node B. This delegation not only reduces computation, but more importantly reduces the overhead of link headers in the frame, and therefore save valuable bandwidth in a resource-constrained underwater network.

Short-circuit delivery is not only done by uwlink , but by all agents supporting the DATAGRAM service. If a downstream agent is capable of delivering the datagram, the delivery is delegated automatically.

On node B, we should have done this in the first place:

This single-liner in Groovy iterates over the list of agents providing the DATAGRAM service, and subscribes to the topic of each agent in that list.

The UnetSocket API also supports delivery of datagrams. Let’s try it. On node A:

On node B, we will see the datagram delivery:

Note that we did not have to specify an agent or service when making the datagram request via the UnetSocket API. An appropriate agent was automatically selected by the API for us. In this case, the uwlink agent was used by the API to deliver the datagram.

All agents supporting the PHYSICAL service must also support the DATAGRAM service ( Chapter 14 ).

The physical layer in UnetStack typically supports 2 logical channels (3 if JANUS is supported). The CONTROL channel provides low-rate, robust communication that allows exchange of small amounts of control information in the network. The DATA channel is a usually a higher rate communication link, but may require tuning to operate well in various environmental conditions.

Fire up the 2-node network simulation and connect to node A’s shell. If you simply type phy , you can explore the physical layer parameters for the node:

The phy.MTU and phy.RTU parameters tells us the maximum and recommended amount of user data that can be transmitted in a single frame (56 bytes in this case) respectively. This is based on the DATA channel, as we will see shortly, since DatagramReq are fulfilled using the DATA channel. The PhysicalParam parameters provide us information on whether the channel is busy, transmission power levels supported, receiver sensitivity, and propagation speed of the signal (e.g. speed of sound for underwater modems). The phy.time parameter is a microsecond resolution clock that is used to timestamp all physical layer events such as frame transmission, reception, etc.

We can dig deeper into the parameters for the CONTROL and DATA channel separately:

Here are a few important parameters to take note of:

In the previous section, we explored several parameters from a simplified simulated physical layer. Next let’s look at a real modem. If you are lucky enough to own one with UnetStack on it, you can connect to it’s shell now. Otherwise, we can use Unet audio SDOAM as our test modem:

On the web shell for the modem:

For brevity, we have omitted the baseband service and scheduler service parameters in the listing above. Even then, there are many parameters that allow you to configure the SDOAM. We cannot cover each parameter in detail here, but we encourage you to explore the help pages for the parameters by simply typing help phy. followed by the parameter name.

Further, let’s look at the indexed parameters for the CONTROL channel:

Again, we cannot cover all the parameters in detail here, but will draw your attention to a few important ones. You see that the modulation for the CONTROL channel is set to 'fhbfsk' (frequency-hopping binary frequency shift keying). Depending on your modem, different modulations may be supported. Once a modulation scheme is chosen, you see additional modulation-dependent parameters. In this case, these are the org.arl.yoda.FhbfskParam parameters such as fmin , fstep , hops , chiplen , tukey , etc. These parameters allow you to control the modulation’s frequency band, number of hops, chip duration, windowing, etc.

The preamble parameter determines a detection preamble that is transmitted before each frame. This is used by the receiving modem to determine the start of a frame. The threshold parameter controls the detection probability and false alarm rate for frame detection. A lower threshold will improve detection probability, but increase false alarm rate.

If the test flag is set on the transmission and reception modems, each transmit frame is filled with known test data. This allows the receiving modem to compute the bit error rate (BER), even when the frame has too many errors for FEC to be able to correct.

If you have two computers with speakers and microphones, you could run Unet audio on both, and communicate between the two. If you happen to have only one computer handy, do not worry — we can get one Unet audio instance to transmit and receive at the same time. This is full-duplex communication!

On Unet audio shell, enable full-duplex operation and try a transmission (you should be able to hear it from your computer speaker!). Your output might not look exactly the same, but let’s go over all the notifications we got and see if we can understand all of them:

To get rid of the false alarm on the DATA channel, we could either increase the detection threshold or turn off the detector completely ( phy[DATA].threshold = 0 ). For now, we’ll do the latter. Let’s also turn on the phy[CONTROL].test flag so that we can measure communication performance in terms of BER. To measure BER before error correction, we also need to turn off phy[CONTROL].fec :

Now we can make 10 transmissions, 2 seconds apart, and watch the BER of the received frames:

For brevity, we have omitted the TxFrameStartNtf and RxFrameStartNtf messages. We see that no bits were in error, out of 144 transmitted bits. We had perfect communication, even without FEC! This is not surprising since the speaker and microphone are very close (and hence good signal-to-noise ratio), but real channels are rarely so forgiving. You can try this between 2 computers, and things may not be as rosy.

Feel free to play around with the parameters of the modulation scheme and try transmissions to get a feel for how the parameters affect communication performance. Since your transmission and reception modems are the same, you only need to set the parameters once! In real life, you’ll need to set the same parameters on all modems in your network.

To explore timed and timestamped transmissions, let’s go back to our 2-node network simulation. On the shell for node A:

We see that the phy agent supports the TIMESTAMPED_TX and TIMED_TX optional capabilities. Let us try them out. On node B:

Going back to node A, send a timestamped frame:

We see that the frame was transmitted at time 2489196375 (when you try this, the time will of course be different). You should see the RxFrameNtf for this frame on node B:

Note that the RxFrameNtf now has an additional txTime field that’s populated, and the timestamp in there is the same as the txTime on node A’s TxFrameNtf . The frame was timestamped before transmission, and transmitted at exactly the intended time.

Sometimes you may not need to transmit a timestamped frame, but you do want the frame to be transmitted at a specified time. On node A:

If you check node B’s shell, you’ll find the corresponding RxFrameNtf , but it will not have a txTime field, as the frame transmitted was not timestamped.

If you’re familiar with Ethernet network interface cards, you may have come across promiscuous mode . In this mode, the network card receives all packets that it hears, not just the ones that are addressed to the node. Agents providing the PHYSICAL service essentially do this continuously, but they send the notifications for frames intended for other nodes on a special sub-topic called SNOOP.

With the 2-node network simulation, let’s first only subscribe to the phy agent’s topic on node B:

From node A, transmit a frame to node B and to node C (node C does not exist in this network):

On node B, you’ll find that it receives the RxFrameStartNtf for both transmissions, but only the RxFrameNtf for the transmission addressed to node B:

The RxFrameStartNtf is sent when a frame is detected. At that point in time, the agent has no idea whom the frame is intended for, because the frame contents have not yet arrived. Only when the frame is received and decoded does the agent know the destination address. Seeing that the second frame was intended for node C, node B does not report a RxFrameNtf for it.

If you were interested in snooping conversations between other nodes, you could subscribe to the SNOOP topic on node B:

Now try transmitting another frame from node A to node C. On node A:

Now you’ll see on node B that the corresponding RxFrameNtf is received:

The to address of 74 corresponds to host('C') , but the frame is available for agents on node B through the SNOOP topic.

Agents offering the baseband service are most commonly modem drivers (and simulators). They support a set of messages and parameters that are explained below. Baseband service providers may also provide optional capabilities to send or record signals at a specified time, or based on premable detection.

Communication systems usually represent signals in a complex baseband representation, as this allows them to be sampled at a lower sampling rate than real passband signals. Passband signals have to be sampled at more than twice the highest frequency (Nyquist criterion). Baseband signals need to be sampled at more than twice the bandwidth. Since the bandwidth is typically much lesser than the carrier frequency, the baseband representation is usually more economical than the passband representation.

While the baseband service is aimed at signals represented in the complex baseband representation, it also supports signals represented as real passband samples. Such signals are identified by setting the fc (carrier frequency) field of the TxBasebandSignalReq or RxBasebandSignalNtf to 0. Modems offering the baseband service may optionally support passband signal transmission and recording.

Since Java and Groovy do not support complex numbers natively, the complex baseband signals are represented by an array of floats with alternate samples from the in-phase (real part) and quadrature (imaginary part) channels. In languages that support complex numbers (e.g. Python and Julia), the signals are represented as arrays (or lists) of complex numbers.

If all this seems to be confusing, don’t worry, it’ll become clear as we go through examples shortly.

In Section 2.7 , you already saw how to transmit and record arbitrary signals. We then used the convenience functions bbtx and bbrec for simplicity. In this chapter, we will do the same thing by sending the TxBasebandSignalReq and RecordBasebandSignalReq messages instead. As you’ll see, these messages provide you greater control, and can also be sent via the UnetSocket API or a fjåge gateway from external applications.

Fire up Unet audio to try out the examples here:

On the shell for the Unet audio SDOAM, check which agents provide the baseband service:

We can now direct all our requests to the phy agent. Let’s check the baseband parameters of the phy agent (we omit other parameters here for brevity):

We see that the Unet audio SDOAM operates at a carrier frequency of 12 kHz and a bandband sampling rate of 12 kSa/s. Let’s create a DC signal of length 12000 complex baseband samples (24000 floats) and transmit it. A DC signal of length 12000 complex samples with a carrier frequency of 12 kHz is a sinusoidal 12 kHz signal for 1 second.

You should hear the sound from your computer speaker.

Unet audio supports transmission of passband signals as well. Let us create a half second 2000 kHz passband signal and transmit it:

Next, let’s request a recording of 12000 samples (1 second duration):

You can also ask for recordings to begin at a specified time:

You’d have noticed the 6 second delay (5 seconds to begin recording, 1 more second to finish the recording) before the recording notification.

Interestingly, you can also request recordings in the past! Many modems have a short buffer, allowing recording in the recent past. Go too far in the past and the modem will refuse your request!

Specifying a negative recTime is understood by the baseband service provider as a relative time. So we can simpify our request to record 5 seconds in the past:

Each logical channel (CONTROL, DATA, etc.) is associated with a detection preamble. Detectors in a modem monitor incoming signals, and trigger when the preamble is detected.

We can transmit a preamble easily:

Here, we did not specify a signal to transmit, so only the preamble was transmitted. You should have heard the preamble as a short chirp from your computer speaker. If we had specified a signal, the preamble would have been followed by the signal.

If you had another Unet audio SDOAM running nearby, it would have heard the preamble and detected a CONTROL frame. It would have tried to decode the frame, but failed, as we didn’t actually transmit anything after the preamble. So you’d have seen a RxFrameStartNtf followed by a BadFrameNtf if you had subscribed to phy topic on that modem.

If you don’t have access to another computer to run Unet audio on, we can easily demonstrate the above behavior with a single Unet audio SDOAM by simply enabling the full-duplex mode (and hence using the same computer for transmission and reception simultaneously), as we did in Chapter 15 :

Preambles 1 and 2 are used by the CONTROL and DATA channel respectively. It’s better not to mess with these, but instead use preamble 3, which is left for the user to configure in Unet audio. By default, detection of preamble 3 is disabled. You can enable it by setting the detection threshold phy[3].threshold parameter. Let’s try it:

We see the RxFrameStartNtf of type #3 indicating that preamble 3 was detected. Since phy[3] is not associated with any modulation scheme (we set the modulation parameter to none ), the modem did not generate a BadFrameNtf as it did with preamble 1.

In applications such as sonar or ranging, we may only be interested in the detecting the timing of a known signal. In that case, the RxFrameStartNtf is sufficient for us. But in some applications, we may wish to capture the signal once detected. That can be easily achieved in Unet audio by setting the basebandRx parameter.

If you enable basebandRx , a recording will be triggered every time the preamble is detected:

The RxBasebandSignalNtf notified us of the recorded signal (containing just the detected preamble). If we wanted a longer recording after the preamble, we can ask for that using the basebandExtra parameter, specifying the length of the recording (in samples) beyond the preamble:

You can see that the recording is much longer now.

During the development of signal processing algorithms, one often wants to simply record received signals in the modem for postprocessing. For this, you could write a script to listen for RxBasebandSignalNtf messages from phy agent’s topic, and store them in a file. Since this requirement is common, UnetStack already provides an agent which does exactly this. The agent is called BasebandSignalMonitor or bbmon for short.

The bbmon agent is already loaded when you run Unet audio. However, by default, it is disabled. It’s easy to enable it:

Now, every RxBasebandSignalNtf that is sent to phy agent’s topic will be recorded in a signal-0.txt file in the logs folder.

As you record more signals, they are appended to the same file (with delineating metadata for each signal). If you restart Unet audio, this file will be renumbered to signals-1.txt , as the logs are rotated.

The signals file stores the signals in a base64 encoded format. The Python package arlpy.unet allows you to read this file and work with the signals in it:

In the previous section, we showed you how to record signals for postprocessing. This is great if you postprocessing is what you desire, but sometimes it is important to access the functionality in real time from Python. This is very useful while debugging new signal processing algorithms, since tools such as Jupyter notebooks and libraries such as numpy , scipy , pandas , arlpy , and many others have made Python the preferred platform for a lot of scientific computation.

Let’s try it!

You had already installed the Python package unetpy in Section 2.5 . We’ll be using it now, so in case you don’t have it installed, now is a good time to install it. Start a Jupyter new notebook with Python 3 and connect to your Unet audio instance:

Of course you could do the same thing with Julia or other languages, if you wish, with obvious minor changes to the syntax!

The RANGING service provides messages and parameters to support OWTT and TWTT ranging to other Unet nodes or COTS transponders, and to manage synchronization information between the nodes.

In order to understand how the RANGING service provides OWTT and TWTT ranging, it is instructive to try a few examples using the Netiquette 3-node network simulation ( bin/unet samples/netq-network.groovy ). Start the simulation, and connect to node A:

An agent implementing the NODE_INFO service not only exposes a set of parameters, as described in this section, but also provides some special handling for specific parameters.

If you start the mission2013 network simulation ( bin/unet samples/mission2013-network.groovy ), connect to node 21’s shell and type node , you’ll see the NODE_INFO parameters for the node in this network:

An ADDRESS_RESOLUTION service provider is responsible for address allocation and resolution. The size of the address space is detemined by the addressSize parameter of the NODE_INFO service ( Chapter 18 ).

Shell usage of this service via the host command is described in Section 4.1 . In this section, we show examples of how address allocation and address resolution can be implemented directly by sending messages.

Address allocation is typically required at startup, and usually initiated by the agent providing the NODE_INFO service to populate the address parameter of that service. To ask the ADDRESS_RESOLUTION service provider to allocate an address, an agent sends it a AddressAllocReq . The allocation may depend on the node name, and so the request must contain the node name.

We can start the 2-node network, and manually test this on the shell of node A:

Address resolution is performed via the AddressResolutionReq message. The host command sends this message on your behalf, and shows you the response:

While the address allocation and resolution processes may seem very similar, there is a conceptual difference between the two. Address allocation is performed for a new node without an address. The address allocation process associates it with an address. Address resolution is performed to find the address that was assigned to a node.

Agents offering the medium access control (MAC) service advise other agents on when they may be permitted to make transmissions, in an effort to reduce collisions and improve network throughput.

MAC protocols that use PDUs for channel reservation may support piggybacking of client data in the PDU. If such support is available, it is advertised using a non-zero reservationPayloadSize parameter. A ReservationReq should provide the payload data to be sent to a peer node (as part of RTS or equivalent PDU) to whom the reservation is made. If that node wishes to send payload data back (as part of CTS or equivalent PDU), it may send a ReservationAcceptReq in response to a ReservationStatusNtf to provide its payload data.

MAC agents advise other agents that wish to transmit (we shall call them clients ) on when they may do so. MAC agents make channel reservations on behalf of their clients, as necessary. Some MAC protocols such as Aloha and TDMA may not require explicit handshake for reservation, while others such as MACA and FAMA may involve control packet exchanges between peer MAC agents on various nodes. In either case, a typical client with data to transmit starts by asking the prevailing MAC agent for a channel reservation:

In the above sample code, error handling has been omitted for simplicity. In reality, you would want to have else clauses to handle reservation failures. The MAC agent not only sends a ReservationStatusNtf[status: START] notification, but also a ReservationStatusNtf[status: END] notification at the end of the reservation duration. The sample code above ignores this notification, but a well-behaved client should ensure that the transmission does not exceed the requested duration.

Messages such as ReservationReq and ReservationStatusNtf may carry payloads, when the MAC protocol supports them. When payloads are supported, additional messages such as ReservationAcceptReq , TxAckReq and TxAckNtf are available for clients to provide payloads to the MAC service provider to piggyback on the MAC PDUs. A typical exchange is illustrated in Figure 9 .

For a MAC reservation initiated by node A with node B, we elaborate on the steps for a full reservation lifecycle with payloads:

Sample MAC implementations are illustrated in Chapter 29 .

Agents offering the LINK service provide single-hop communication.

Single-hop here refers to a single hop in the Unet sense. For example, a link may be provided over wireless RF network that has multiple physical hops (e.g. using UDP/IP). However, as long as the link does not pass through multiple Unet nodes, it is considered a logically single-hop link.

All agents supporting the LINK service must also support the DATAGRAM service ( Chapter 14 ).

It is recommended that agents offering the LINK service provide reliability, when requested. Agents that are able to provide reliability, do so by advertising the DATAGRAM service capability RELIABILITY.

LINK service providers using the PHYSICAL service should also consult the MAC service to determine when they should transmit.

In a typical Unet, gateway nodes may have several LINK service providers, with the ROUTING service ( Section 22.2 ) provider forwarding datagrams across links.

Start the 2-node network and connect to node A:

We see that in the 2-node network simulation, the only agent that provides the link service is the uwlink agent of type ReliableLink . This agent fragments large datagrams, and transmits a batch of frames at a time, before waiting for an acknowledgement from the peer node. Unacknowledged frames are retransmitted until all frames are delivered, or there are too many retries. Once all frames are received, the peer node’s uwlink agent reassembles the datagram and delivers it. The agent also provides link status notifications.

With the default PHYSICAL service settings, the nominal data rate provided by this link is 731 bps, and that only 848 or less bytes may be transferred per datagram. The actual data rate may differ, depending on the size of the datagram, reliability settings, and the channel conditions.

We also see that ReliableLink provides a set of configurable parameters:

ECLink uses an erasure correction code (type of error correction code that deals with lost frames) to reduce the protocol overhead required for retransmissions in a lossy channel. This usuaully results in significantly better performance than ReliableLink in poor channel conditions, and when transferring large datagrams.

If you have a modem with the commercial version of UnetStack3, it’ll have ECLink loaded as the default LINK service provider:

We see that the MTU for ECLink is quite large (as compared to ReliableLink ), as ECLink can efficiently transfer large amounts of data. While the dataRate parameter advertises a similar nominal rate as with ReliableLink , you’ll find that ECLink yields better practical performance when transferring large files, and in poor channel conditions. ECLink also supports data compression, and link status notifications.

The phy , controlChannel , dataChannel , mac , maxRetries , and maxPropagationDelay parameters of ECLink are similar to the ones in ReliableLink . However, ECLink has several additional parameters that control performance:

The ROUTING and ROUTE_MAINTENANCE services work closely to provide multi-hop communications. The ROUTING serice provider is responsible for maintaining a routing table, and for routing datagrams according to the table. The ROUTE_MAINTENANCE service provider, on the other hand, is responsible for discovering new routes and providing updated routing information to the ROUTING service provider.

If a network only requires static routes, the ROUTING service provider is sufficient to provide the functionality. In case of networks with dynamic route discovery, a ROUTE_MAINTENANCE service provider discovers routes (or route changes) and publishes RouteDiscoveryNtf messages. The ROUTING service provider subscribes to these messages and updates its routing tables.

The ROUTING service provider also typically listens for LinkStatusNtf messages from the LINK service agents. These messages allow it to keep track of link availability and link quality.

Both ROUTING and ROUTE_MAINTENANCE services are described below.

org.arl.unet.Services.ROUTING

Agents offering the ROUTING service provide multi-hop communication.

All agents supporting the ROUTING service must also support the DATAGRAM service ( Chapter 14 ).

It is recommended that agents offering the ROUTING service provide reliability, when requested. Agents that are able to provide reliability, do so by advertising the DATAGRAM service capability RELIABILITY.

org.arl.unet.Services.ROUTE_MAINTENANCE

Agents offering the ROUTE_MAINTENANCE service generate RouteDiscoveryNtf messages to allow ROUTING service providers to maintain routing tables.

The Router class (agent name router ) provides the ROUTING service in the standard stack. Apart from supporting the routes , addroute , delroute and delroutesto commands, this agent exposes two parameters:

Without auto1hop enabled, every route must be explicitly added to the routing table (even when the node is accessible over a single hop). By enabling auto1hop , we tell the router than any node that isn’t explicitly added to the routing table is assumed to be accessible over a single hop. This is the default setting:

The RouteDiscoveryProtocol class (agent name rdp ) provides the ROUTE_MAINTENANCE service in the standard stack. This agent has no configurable parameters.

In Chapter 6 , we explored several examples of how to set up networks with static and dynamic routes. To find out more about routing, type help router in the shell:

You can also type help followed by any of the commands above to get more information on the usage of that command.

Agents offering the TRANSPORT service provide end-to-end reliability and fragmentation/reassembly for large datagrams. They may also support connection-oriented services for data streaming. Agents providing this service typically use the ROUTING service for multi-hop delivery of data.

All agents supporting the TRANSPORT service must also support the DATAGRAM service ( Chapter 14 ), along with the RELIABILITY and FRAGMENTATION capabilities. It is also recommended that they support the CANCELLATION and PROGRESS capabilities, since datagrams at this level are likely to be large.

There are no special messages or parameters defined by the TRANSPORT service, but agents providing this service may expose additional parameters to configure the transport protocol in use.

The default implementation of the TRANSPORT service is the SWTransport class. If you start the 2-node network simulation and connect to node A, you can explore the configurable parameters that it advertises:

We briefly explain each parameter below:

Next, let’s try a 2048-byte datagram transfer from node A to node B with progress reports:

The entire transfer will take a few minutes. During that time, we get periodic DatagramProgressNtf reports showing how much of the data transfer was completed.

Agents offering the REMOTE service provide text messaging, file transfer, and remote command execution services across a network.

Start the 2-node network and connect to node A:

We see that the REMOTE service is provided by the remote agent of type RemoteControl . The agent’s behavior is controlled by several parameters:

All remote commands ( tell , fget , fput , rsh , and ack ) encountered in Section 5.4 and Section 5.5 are implemented by the shell using the above messages. For example, the same effect as the tell command can be achieved by directly sending the RemoteTextReq message to the remote agent on node A:

We should see the text message delivered on node B:

We encourage you to re-read Section 5.5 and explore the command’s help documentation ( help remote ) to fully appreciate the use of this service.

An agent offering the STATE_MANAGER service provides a way to save the state (parameter values) of specified (or all) agents.

Such an agent typically subscribes to the PARAMCHANGE topic and monitor ParamChangeNtf for all parameter changes for all agents. It then helps persist selected agents' parameter values between reboots.

Fire up Unet audio ( bin/unet audio ) to test out how state persistence works:

Now start Unet audio again, and you’ll find that the plvl state was not retained through reboots:

We can ask it to retain the state:

Start Unet audio again, and you’ll find that the state is retained:

The saved-state.groovy is human-readable, and you’ll see that it simply contains the Groovy code to set the parameters required to restore the state:

Delete this file and start Unet audio again:

The help shows you that the savestate command can be used to save the state of individual agents, if you wish, to a filename of your choice. If you save the state to a different filename, it is not automatically restored on startup. But you can restore it easily with a single command (name of the file):

Agents offering the SCHEDULER service provide a way to schedule sleep/wake tasks in the future.

The Unet simulator currently does not support sleep/wake scheduling. While Unet audio supports the SCHEDULER service, it does not actually put your computer to sleep. Therefore you can experiment with creating and managing schedules on Unet audio, without worrying about your computer going to sleep. Start up Unet audio ( bin/unet audio ) and connect to its shell:

We see that the phy agent provides the SCHEDULER service in Unet audio. We add a sleep schedule, check that it shows up, remove it, and check that it is deleted. We then add another schedule.

The addsleep , showsleep , and rmsleep commands use the AddScheduledSleepReq , GetSleepScheduleReq , and RemoveScheduledSleepReq messages to achieve their functionality. We can manually send this messages to confirm this, if we like:

We can also use the web interface to manage the sleep schedule, if we like:

A big advantage of working with the web interface for sleep scheduling is that the user interface displays date/time in a human readable format. On the other hand, programmatic access with messages requires times to be specified as Unix epoch time.

The Unix epoch is the number of seconds that have elapsed since January 1, 1970 (midnight UTC), not counting leap seconds. While computers find it easy to work with epoch time, we find it hard to interpret. So UnetStack introduces syntactic sugar such as "1.hour.later" that computes the Unix epoch time 1 hour from now:

There are online calculators that’ll help you convert between Unix epoch time and human readable date/time. This works well, if you need to manually convert a few date/times, but what if you needed to do this programmatically? Java provides simple APIs to deal with date/times:

If you want even nicer looking date/time strings, you should check out Java’s SimpleDateFormat class.

Agents providing the SHELL service support execution of commands, and access to files on the node. While shell agents typically provide interactivity using a terminal/console, they also support messages for other agents to request execution of commands or access to files.

The language in which the commands are written is not defined by the service, but depends on the shell agent. fjåge supports a pluggable mechanism for an ShellAgent to use any ScriptEngine . Various script engines are available in fjåge and UnetStack, including the GroovyScriptEngine , EchoScriptEngine , and ATScriptEngine .

Start the 2-node network and connect to node A:

Next, let’s try the GetFileReq and PutFileReq messages to read, write and delete this file:

We interacted with the SHELL service provider using a shell! That’s not very useful in practice, but served to show you how these messages work. Typically, these messages are sent by other agents that wish to get the shell to run commands and access files for them (e.g. RemoteControl agent in Section 24.2 ). The agents may be running remotely in a fjåge slave container or on a gateway (via the UnetSocket API), where they may not have direct access to the filesystem of the node.

Agents are the basic building blocks of the UnetStack. They exchange messages, provide services and implement protocols. While what is expected from a well-behaved agent is quite demanding, most of the necessary core behaviors are already implemented for you by the UnetAgent base class. All you need to do is to extend it, and add in a little code to teach the agent what you want it to do.

The basic skeleton of a Groovy agent looks like this:

While you don’t strictly need the @Override annotations, it is a good practice to use them whenever you are overriding a method from a superclass. The annotation tells the compiler that this is what you intend, and so if you make a typographical mistake and type in a wrong method name (one that doesn’t exist in the superclass), the compiler will warn you.

If you do not need any of these methods, you can skip the definition as the base class provides default implementations. There are a several other methods that you can override to customize your agent, but these are less commonly needed and so we’ll skip them for now. You’ll come across them later.

It’s best to illustrate with a simple example.

Let’s develop an echo daemon that will respond to each incoming echo request datagram with an echo response datagram containing the same data as the echo-request. We need a way to identify which datagram is an echo request, as we don’t want to be echoing datagrams intended for other agents or for the user. We do this by defining an echo request datagram as any datagram with protocol USER (recall that protocol numbers from USER onwards are available for your own applications to use). We do not want the response to use the same protocol, otherwise our daemon (running on the source node) could get confused and echo the response, which would in turn be echoed again by the destination node’s daemon, ad infinitum. So we use protocol DATA for the echo response datagram, as this protocol is intended for generic application data.

Here’s our daemon:

Let’s walk through the above code:

That’s it!

It’s time for us to test this agent. Create a file called EchoDaemon.groovy in the classes folder and copy the above daemon code into it.

Now start the 2-node network simulation that we have been using as a testbed, and on node B, load the agent:

Our daemon is up and running!

Once the daemon is successfully loaded on node B, we can test it from node A:

We have written our first agent! Was easy, wasn’t it?

If you’re a Java programmer and find the Groovy syntax daunting, you might prefer to write your agents in pure Java (at the expense of verbosity and more steps for testing). This is the equivalent Java code below for the Groovy agent we developed in the last section:

In Java, you’ll first need to compile the Java code. Create a EchoDaemon.java file with the above contents. To compile it, you’ll need to have fjåge and unet-framework jar files on the classpath:

You should now have a EchoDaemon.class file which you copy to the classes folder. To avoid duplicate classes, remember to first delete the EchoDaemon.groovy file!

Finally, you can run the 2-node network simulator and test the agent, just as you did in the previous section.

Agents implement most of their functionality with behaviors.

We have been implicitly using two behaviors so far. The startup() method is called by the UnetAgent base class using a OneShotBehavior , and the processMessage() method is called from a MessageBehavior . While you could have manually added these behaviors, the UnetAgent base class does this for you, because almost all Unet agents require this.

Let’s next look at a use case for explicitly adding other behaviors. Say we wanted our echo daemon to not respond immediately, but after 7 seconds. How would we do that?

We could of course add a delay(7000) in the processMessage() method, but that would be a bad idea. If we did that, the agent would sleep for 7 seconds on receiving a request and not process any request from any other nodes! We want the agent to be responsive while waiting, and so do not want to block execution. Instead, we want a behavior that will occur 7 seconds later — this is precisely what a WakerBehavior does. Here’s our new processMessage() method: