DiffQ Scripts Documentation

This page provides documentation on various scripts needed to run diffq and other wireless congestion control (CC) algorithms on WiSeNet.

WiSeNet Management Scripts

Introduction

WiSeNet comprises of 60 nodes, spread across 3 floors of the CSC building. The DNS name for each node is in the format "snXXXX.csc.ncsu.edu", where the first two XX denote the floor and wing of the building and the last two XX denote the node id within that floor/wing. E.g. sn2e01 denotes a node on the (2)nd floor, (e)ast wing and with id 01. The node distribution is as follows:

• floor 3, east wing: sn3e01 to sn3e19
• floor 2, east wing: sn2e01 to sn2e19
• floor 1, east wing: sn1e01 to sn1e08
• floor 1, west wing: sn1w01 to sn1w14

Nodes are mapped to IP addresses as follows: 10.0.[floor index].[node id]:

• floor 3, east wing: 10.0.3.1 to 10.0.3.19
• floor 2, east wing: 10.0.2.1 to 10.0.2.19
• floor 1, east wing: 10.0.1.1 to 10.0.1.8
• floor 1, west wing: 10.0.4.1 to 10.0.4.14

This mapping, as well as several other utility functions are defined in /wisenet/devel/videos-scripts/utilFunc.py.

Running scripts on WiSeNet

The exec2 script in /wisenet/devel/videos-scripts executes commands on remote WiSeNet nodes through an ssh connection:

exec2 -q(quiet) -b(block) [command in quotes] [listOfNodes]

• command: Command to be executed on remote node.
• listOfNodes: list of nodes in snXXXX format. As a shorthand, nodes can be grouped by floor and wing, e.g. (floor2, floor3, floor1e, floor1w, all). Shorthand and node names can be mixed (e.g. floor2e sn1e01 ...).
• -q: output of commands is ignored.
• -b: block for the exit of each command.

By default, exec2 shows output of the remotely executed commands. It shows both STDOUT and STDERR output. This output can be switched off with -q. By default exec2 will run the command in parallel for all nodes, printing output in the order that the command exits among all nodes. To enable serialized execution of the command, use the -b option.

Setting up MadWifi?

The setupMadWifi.py script in /wisenet/devel/videos-scripts sets up the MAC in all nodes and assigns an IP as described above:

setupMadWifi.py [stock(0|1)] [cmd] [listOfNodes]

• stock: 1 will load the stock MadWifi? module, 0 loads the new MadWifi? module with 8 priority levels required for DiffQ.
• cmd: start/stop.
• listOfNodes: as described earlier.

Recording Kernel Logs

The startLog.py in /wisenet/devel/videos-scripts starts logging kernel status messages and puts them into local files:

startLog.py [portNum] [listOfNodes]

• portNum: starting port number on which the log data is listened to. UDP port numbers portNum, portNum+1, ... [number of nodes] will be used, one port number for each node.
• listOfNodes: same as before.

The logs will be stored in the current directory as "snXXXX.log" files. Logging may be stopped with the stopLog.py command:

stopLog.py [listOfNodes]

Misc. Management Scripts

• checkNodes: If no ouptut is generated, it indicates that all nodes are up.
• installKernel: installs the kernel in /wisenet/devel/usr/src/linux-2.6.18.6-wisenet/arch/i386/boot/bzImage onto all nodes, and runs lilo.

Automated Rebooting of Nodes

Nodes may be rebooted via the web interface. However if all nodes on the testbed need rebooting, there is an automated script to do so. The alldown.bat and allup.bat in /wisenet/devel/scripts will bring down and bring up all WiSeNet? nodes respectively. To use the scripts, one needs to first login at http://scripts.comtech.ncsu.edu/switch_admin/ and then get the cookies for values WRAP_REFERER and WRAP16 and replace them in /wisenet/devel/scripts/reboot.php. Once this is done:

alldown.bat; allup.bat

will automatically reboot all nodes. Note that access rights for switches is given by Marhn Fullmer (mhfullme@unity.ncsu.edu).

Routing on WiSeNet

OLSR Routing

Routing on WiSeNet? is primarily performed by OLSR. The OLSR version currently used is located in /wisenet/devel/usr/src/olsrd-0.5.3. To execute olsr on all nodes run:

exec2 "/mnt/devel/usr/src/olsrd-0.5.3/olsrd  -f /mnt/devel/scripts_gw/olsr/olsrd.conf" [listOfNodes]

Above, the -f option gives the location of the olsr configuration file. The default options in the config file should be OK, but the most important option for debugging and logging would be:

DebugLevel 0

By default, the DebugLevel is 0, which disables the routing log. If you need to log routing information, you need to set DebugLevel? to a non-zero number. Higher the number, more information is logged. If logging is enabled, the output needs to be piped to an output file as follows:

exec2 "/mnt/devel/usr/src/olsrd-0.5.3/olsrd  -f /mnt/devel/scripts_gw/olsr/olsrd.conf &> /mnt/storage/route.log" [listOfNodes]

Capturing Routing Information

In some cases, OLSR routing information may need to be captured for offline statistics or flow generation (e.g. genBEFlows5.py). The script getTopology.py in /wisenet/devel/videos-scripts will parse the OLSR log file for any one node. Before running this, the OLSR log files should be generated by setting DebugLevel? to 1 and then running OLSR as described above. Assuming each node's log is stored in /mnt/storage/route.log, the invocation to get routing information from node sn2e01 would be:

getTopology /wisenet/sn2e01/route.log

Note that sn2e01 is interchangable with any other node on the testbed, since OLSR is a link state protocol, so all nodes are assumed to be synchronized w.r.t. link state.

The output is stored in two files:

• nodes: contains list of all nodes seen:
• links: contains link quality entries:

Once these two files have been generated, we can get the routing metric between two nodes on the testbed using getRoutingHops.py in /wisenet/devel/videos-scripts:

getRoutingHops.py [linksFile] [nodeFile] [src] [dst] [metric]

• linksFile, nodeFile are the links and nodes files described above.
• src, dst: source and destination nodes in snXXXX format.
• metric: "hops" gives distance between src and dst in hops, "etx" gives distance in ETX metric.

Static Routing

Once OLSR information is obtained, it is possible to use static routing and completely switch off OLSR using the script genStaticRouting.py in /wisenet/devel/videos-scripts:

genStaticRouting.py [linksFile] [nodeFile] [flowFile]

The flowFile is the list of flows as described in DiffQScripts2. This will generate bidirectional routes for all flows listed in the flowFile.

A quick script to generate routes for single flows is fixedrouting in /wisenet/devel/scripts:

fixedrouting [src] [hop1] [hop2] ... [dst]

will generate bi-directional routes between src and dst via hop1, hop2, ...

OLSR without route updates

The code for OLSR has been patched to enable a version which would allow OLSR to run without routing updates - OLSR will transmit and receive routing messages, but the updates to routes inside the kernel is blocked. This, when combined with static routing (see above) is helpful to check for the amount of overhead (CPU and channel-time) generated by OLSR. To execute this version:

exec2 "/mnt/devel/usr/src/olsrd-0.5.3/olsrd_noUpdates  -f /mnt/devel/scripts_gw/olsr/olsrd.conf" [listOfNodes]

The change is in /wisenet/devel/usr/src/olsrd-0.5.3/src/linux/kernel_routes.c. If ALLOW_ROUTE_UPDATES is defined in this file, route updates are allowed as usual. If undefined, route updates are blocked.

Experiments on WiSeNet:

Protocols/Algorithms used on WiSeNet:

Below we give the aliases used in WiSeNet? scripts for the different CC protocols:

• protocol: could be one of:
  • DCCP: the DCCP CC algorithm.
  • TCP-N: TCP-Reno.
  • TCP-F: TCP-FeW.
  • TCP-S: TCP with SWAN.
  • TCP-V: TCP-Veno.
  • TCP-We: TCP-Westwood.
  • TCP-W: TCP-WiSeNet for use with DiffQ.
  • DF-T: DiffQ with TCP.
  • DF-U: DiffQ with UDP.

The aliases above are used by scripts (e.g. generateload2, runBETest.py) to decide what CC algorithms to use for an experiment. However, before each experiment, the kernel modules for each protocol need to be loaded (and unloaded after the experiment). Below we give the aliases used in WiSeNet? scripts to load modules:

• algorithm: could be one of:
  • diffq: The default diffq scheduling algorithm.
  • diffq_fixedPrio: DiffQ scheduling without MAC prioritization - all packets mapped to same default priority.
  • diffq_noSched: DiffQ source rate control, with FIFO scheduling and fixed MAC priority.
  • swan: Swan scheduling module.
  • more1: Kernel module for DiffQ+MORE.
  • veno: Module for TCP-Veno.
  • westwood: Module for TCP-Westwood.
  • tfrc: Module for TFRC.
  • default: Module for setting default MAC priorities to WiSeNet? nodes.

Loading and unloading Modules

The algoCtl script in /wisenet/devel/videos-scripts loads and unloads diffq (and other modules) on a set of nodes.

algoCtl [algorithm] [start/stop] [set of nodes]

• algorithm: can be one of the algorithms detailed above.
• start/stop: loads and unloads modules respectively.
• set of nodes: as described above.

algoCtl automatically performs background parameter settings for each protocol, e.g. for DiffQ, it invokes the script /wisenet/devel/videos-scripts/setDFParam which will set MAC priorities, and TCP-WiSeNet parameters.

Running a single flow on the testbed

The script /wisnet/devel/videos-scripts/generateload2 runs a single flow for a given CC algorithm/

generateload2 [protocol] [src] [dst] [port] [duration]

• protocol: one of the CC algorithms described above.
• src: source node on WiSeNet? in the format snXXXX (e.g. sn2e01).
• dst: destination node on WiSeNet?.
• port: port number used for the connection.
• duration: duration of the experiment in seconds.

To run an experiment with multiple concurrent flows, use runBETest.py as explained in the next section.

Running a flow scenario on the testbed

The runBETest.py in /wisenet/devel/videos-scripts starts a set of flows on the testbed.

runBETest.py [number of runs] [protocol] [listOfNodes]

• number of runs: number of experiment runs to be executed.
• protocol: could be one of:
  • DCCP: the DCCP CC algorithm.
  • TCP-N: TCP-Reno.
  • TCP-F: TCP-FeW.
  • TCP-S: TCP with SWAN.
  • DF-T: DiffQ with TCP.
  • DF-U: DiffQ with UDP.
• listOfNodes: list of nodes e.g. sn2e01 sn2e02 ....

On running the script, all output from transmitter and receiver of each flow will be stored in [protocol]_[run]_[src]_[dst] files.

The script looks for files beFileRun0, beFileRun1, ... beFileRun[number of runs - 1] in the current directory. These files contain the information about the flows. Sample beFileRunXX file contents:

sn1e04 sn2e01 22346 120

sn1e02 sn3e17 22347 120

sn2e07 sn3e18 22348 120

...

Each line represents the information for one flow - [src] [dst] [port] [duration]:

• src, dst: source and destination node of the flow.
• port: port number to be used for this flow.
• duration: duration in seconds for this flow.

Such flow files may be generated by hand, but it is more convenient to use the genBEFlows.py script described below.

Generating a set of flows

Flow files for an experiment can be generated automatically with randomly selected sources and destinations using the script genBEFlows2.py in /wisenet/devel/videos-scripts:

genBEFlows2.py [number of flows] [number of runs] [duration]

• number of flows: number of flows in each run.
• number of runs: total number of experiments.
• duration: duration in seconds.

genBEFlows2.py generates flows as follows. If run with parameters (3, 1, 60), it will select 3 (source,dest) pairs randomly from the testbed. Note that there may be some overlap between the 3 sources and destinations, e.g. sn2e01 may be a source of one flow and the destination for another flow. This does not cause a problem since port numbers for all flows are different. However, it will cause a problem if the number of flows is high (> ~50). At this point there may be (src,dst) pairs which are repeated. There are other variants of genBEFlows which perform the flow selection in a different manner:

• genBEFlows3.py: selects sources uniquely, destinations may overlap. So a node will not have more than one flow originating on it, but may have more than flow terminating on it. This is useful for MORE since it cannot handle more than one source on a single node. Parameters are same as genBEFlows2.py.
• genBEFlows4.py: generates flows between 1st and 3rd floors to create "long" flows. Some number of flows exclusively on 2nd floor may be generated as background traffic.

genBEFlows4.py [num1-3Flows] [num2Flows] [numRuns] [duration]

• genBEFlows5.py: same as genBEFlows3.py, however one-hop flows are pruned. To do so, routing information needs to be generated and submitted to the script as shown below. The linksFile and nodesFile can be generated using the getTopology.py script (described in [WisenetRouting]).

genBEFlows5.py [numFlows] [numRuns] [duration] [linksFile] [nodesFile]

Getting statistcs for an experiment

After the experiment is completed, statistics about throughput of each flow can be obtained by running the script getBEStats.py in /wisenet/devel/videos-scripts:

getBEStats.py [number of runs] [duration] [protocol]

All the parameters are same as above. The script generates cdf_allRates_[protocol] files which contain the CDF of throughputs of all flows in the experiment across all runs.

Getting realtime view of CDF statistics for an experiment

The script realTimeViewCDF.py in /wisenet/devel/videos-scripts will produce a realtime view of the CDF statistics generated above:

realTimeViewCDF.py [sleep time] | gnuplot -persist -noraise

• sleep time: time to update CDF in seconds

This script uses the getBEStats.py script above to generate statistics for all experiment output files in the current directory, and this is done every [sleep time] seconds. The output of the script is a set of gnuplot commands, which when piped to gnuplot generate the CDF graph.

The script assumes that another script is running one run of a BE experiment for three protocols TCP-N, TCP-F and DF-T for a duration of 120s. The script beTestScript.sh does this automatically. The scenario for this experiment may be changed by modifying the beFileRun0.txt file in the current directory.

beTestScript.sh

Getting realtime view of each flow an experiment

Instead of CDF if the actual realtime throughput of each flow needs to be viewed, the script plotThrRealTime3.py in /wisenet/devel/videos-scripts can be used:

plotThrRealTime3.py [sleep time] [protocol] | gnuplot -persist -noraise

All the parameters are the same as above. The script will print out the realtime throughput of all flows in one BE experiment over time. Internally it stores the realtime throughput of each flow in files [protocol]_[src]_[dst]_tmp. It assumes that another script will run the actual BE experiment.

-- Main.acwarrie - 03 Jul 2008