NACK-Oriented Reliable Multicast (NORM) implementation & tools (RFCs 5740, 5401)
 
 
 
 
 
 
Go to file
mullerj 26944463e2
.NET Extension Design Implementation (#10)
* changes

* Removed GetsNextEvent test

* Implemented GetNextEvent

* Fixed NormStream design

* Fixed HasNextEvent design

* Added NormGetNextEvent overload

* Implemented HasNextEvent

* Updated design

* Added Kernel32 to design

* Modified to use .NET EventWaitHandle instead of kernel32 P/Invoke

* Modified to only copy norm.dll on Windows

* Added copy for non Windows

* add start and stop receiver

* add receiver

* Modified non-Windows to use Socket for HasNextEvent

* Deleted NormFileTests since they break the other tests

* Fixed ReceiverReceives test

* Updated ReceiverReceives test name

* Modified ReceivesFile to get all events instead of adding to array

* Cleaned up NormInstance formatting

* Refactored tests

* Updated NormFile

* Implemented send stream

* Implemented receive stream

* Implemented NormFile Name

* Updated ReceivesFile to use events

* data enqueue

* Fixed merge

* Fixed ReceivesData test

* Modified data test to compare collections without order

* stop instance working

* enqueues data

* Reverted upgrade to .NET 7

* Restartinstance is working

* Modified NormData to read bytes instead of as string

* Removed using statements in NormData

* Updated design

* Updated design

* Switched to nint

* Switched design to nint

* unfinished test for suspendinstance

* working suspend instance

* added resume function to instance

* Updated session send and receive data tests

* Removed launch settings

* session configuration functions

* Modified to use random port due to intermittent failed tests due to session conflict

* Modified to randomize text content

* Implemented NormNode Address

* Updated design for NormNode

* Implemented NormNode Grtt

* Implemented NormNode Id

* implement session configuration functions

* instance

* missing return types

* add missing handle param

* normNode

* Implemented RequeueObject

* Implemented SetWatermark and AckingNode

* Implemented CancelWatermark

* Implemented ResetWatermark

* Implemented SendCommand

* Implemented CancelCommand

* Implemented DebugLevel. Removed DebugPipe

* Updated SetTxPort

* Updated SetRxPortReuse

* Implemented SetEcnSupport

* Added test for RxLoss

* Implemented ReportInterval

* Refactored session calculated properties

* Updated enqueue methods

* Updated StreamOpen design

* Fixed SetWatermark overload

* Implemented GetAckingStatus

* Removed SendCommand overload

* Removed SetSilentReceiver overload

* Implemented NormEvent ToString

* Implemented NormFile Rename

* Implemented SetUnicastNack

* Added ReceivesCommand test

* Implemented SetNackingMode

* Implemented SetRepairBoundary

* Added Fact to SetsRepairBoundary test

* Implemented SetRxRobustFactor

* Implemented NormNode Command

* Implemented FreeBuffers

* Implemented Retain and Release

* Fix using namespaces

* test get object type

* NormObject get size

* NormObject setNackingMode

* NormObject get bytes pending

* NormObjet cancel

* NormObject retain

* NormObject release

* NormObject get sender

* NormObject get hashcode

* normObject equals

* Implemented NormObject Equals

* Removed redundant cast

* Removed unused using directive

* Removed static using

* Added NormApi to global Usings

* Added Enums to global usings

* Cleaned up NormObject Equals

* Cleaned up NormObject GetHashCode() test

* Cleaned up warning

* normStream has vacancy

* normStream read offset

* normStream seek msg start

* NormEventListener interface

* normStream set push enable

* normStream set auto flush

* Simplified NormApi usage

* Updated design of NormStream

* public interface

* Fixed INormEventListener

* fixing name space

* commit initializing  inputstream

* Added StreamBreakException

* Fixed formatting

* Fixed formatting

* Fixed formatting

* Initial implementation of NormOutputStream

* Cleaned up NormObject Equals

* NormInputStream class

Co-authored-by: mullerj <mullerj@users.noreply.github.com>

* normStream data offset

* move offset length arithmetic into write method

* building normInputstream

* all method implemented

Co-authored-by: mullerj <mullerj@users.noreply.github.com>

* NormStreamRead

* Fixed build error

* Completed ReceivesStreamWithOffset test

* Added retries to GetsCommand test

* Updated type for NormStreamRead

* Simplified NormStream Read

* Simplified NormStream Write

* Added dataOffset to DataEnqueue

* Fixed DataEnqueue overload

* Added infoOffset to FileEnqueue

* Added infoOffset to StreamOpen

* Updated design

* Removed extra overload of FileEnqueue

* Removed DataEnqueue overloads

* Removed extra overload from StreamOpen

* Added HasNextEvent overload

* Implemented OpenDebugPipe

* add thread locking

* locking for properties

* Not supported exceptions

* Implementing sychronize

* fixing inputstream

* adding sychronize message

* Reorganized NormApi and updated design

* Modified to skip GetsCommand test on IO exception

* WIP: 537b706 adding sychronize message

* adding message to expections

* Fixed test file cleanup

* Added locking to NormSession methods that use static field

* Changed StartsSender to skippablefact

* Changed to SetsNackingMode to skippable fact

* Converted tests to skippable

* Converted more tests to skippable

* Converted all tests that start sender to skippable

* Updated README about skipped tests

* documentation

* 2 doc

* Cleaned up input and output stream classes

* Cleaned up NormApi

* Updated NormOutputStream Write

* Added document comments to NormApi

* Added comments to NormApi

* Removed unnecessary usings

* Added comments to NormApi

* set up the rest of doc

* api documentation

* fix typo

* fix spacing

* Doc for instance need clean up

* clean up

* document norm session

* NormStream documentation

* commit for node

* NormObject docs

* Updated NormApi comments

* Cleaned up NormInstance comments

* Cleaned up NormNode comments

* Cleaned up NormObject comments

* Cleaned up NormSession comments

* Cleaned up NormStream comments

* Cleaned up NormObject comments

* Adding commits

* Update get commit

* clean up on data and event

* Norm Data change

* clean up normfile docs

* clean up normstream

* clean up api docs

* Fixed data

* space approved

* Cleaned up NormData comment

* documentation clean up

* Update the ReadMe

* Cleaned up comments

* Added comments to NormAckingStatus

* Added comments to NormEventType

* Added comments to NormFecType

* Added comments to NormNackingMode

* Added comments to NormObjectType

* Added comments to NormProbingMode

* Added comments to NormRepairBoundary

* Added comments to NormFlushMode

* Added comments to NormSyncPolicy

* Cleaned up comments.  Fixed default for StartSender

* Added comments to constants

* Moved constants to relevant classes

* Updated design

* Removed NormDataDetachData since it is not referenced

* Updated design

* Updated input and output stream design

* Updated packaging README

* Removed language version since redundant

* Cleaned up project file

* Updated to package from the dotnet directory (test project is set to not pack)

* Added version to project

---------

Co-authored-by: syermak <syerm001@odu.edu>
Co-authored-by: Sylvester Freeman III <Slyfreeman3@gmail.com>
Co-authored-by: Sergiy <47069243+SergiyYermak@users.noreply.github.com>
Co-authored-by: mullerj <mullerj@users.noreply.github.com>
2023-03-01 17:05:29 -05:00
.github/workflows Update cmake.yml 2022-09-11 21:41:50 -04:00
android pynorm fixes 2022-08-20 22:36:29 -04:00
cmake pynorm fixes 2022-08-20 22:36:29 -04:00
doc pynorm fixes 2022-08-20 22:36:29 -04:00
docker feat: adding dockerfile for docker image building (#56) 2022-04-28 22:06:43 -07:00
examples Fixed NormStreamSend (#1) (#77) 2023-01-16 16:44:29 -05:00
include pynorm improvement to keep cache sender enqueued objects in pynorm.instance._objects cache 2022-10-04 19:21:41 -04:00
makefiles cleaned up Visual Studio build configurations 2022-09-11 17:20:15 -04:00
norp pynorm fixes 2022-08-20 22:36:29 -04:00
protolib@c1907cee5e pulled updated submodule code 2022-10-04 19:23:39 -04:00
src .NET Extension Design Implementation (#10) 2023-03-01 17:05:29 -05:00
.gitignore .NET Extension Design Implementation (#10) 2023-03-01 17:05:29 -05:00
.gitmodules pynorm fixes 2022-08-20 22:36:29 -04:00
BUILD.TXT pynorm fixes 2022-08-20 22:36:29 -04:00
CMakeLists.txt pynorm improvement to keep cache sender enqueued objects in pynorm.instance._objects cache 2022-10-04 19:21:41 -04:00
Dockerfile pynorm fixes 2022-08-20 22:36:29 -04:00
LICENSE pynorm fixes 2022-08-20 22:36:29 -04:00
NormSocketBindingNotes.txt pynorm fixes 2022-08-20 22:36:29 -04:00
README-DotNet.md .NET Extension Design Implementation (#10) 2023-03-01 17:05:29 -05:00
README-Java.txt pynorm fixes 2022-08-20 22:36:29 -04:00
README-PyNorm.txt pynorm fixes 2022-08-20 22:36:29 -04:00
README.md pynorm fixes 2022-08-20 22:36:29 -04:00
TODO.TXT pynorm fixes 2022-08-20 22:36:29 -04:00
VERSION.TXT pynorm fixes 2022-08-20 22:36:29 -04:00
norm.pc.in pynorm fixes 2022-08-20 22:36:29 -04:00
setup.py pynorm fixes 2022-08-20 22:36:29 -04:00
waf wscript tweak 2020-02-29 15:10:33 -05:00
wscript pynorm fixes 2022-08-20 22:36:29 -04:00

README.md

norm

                    NORM SOURCE CODE RELEASE

/*********************************************************************
 *
 * AUTHORIZATION TO USE AND DISTRIBUTE
 * 
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that: 
 *
 * (1) source code distributions retain this paragraph in its entirety, 
 *  
 * (2) distributions including binary code include this paragraph in
 *     its entirety in the documentation or other materials provided 
 *     with the distribution.
 * 
 *      "This product includes software written and developed 
 *       by Code 5520 of the Naval Research Laboratory (NRL)." 
 *         
 *  The name of NRL, the name(s) of NRL  employee(s), or any entity
 *  of the United States Government may not be used to endorse or
 *  promote  products derived from this software, nor does the 
 *  inclusion of the NRL written and developed software  directly or
 *  indirectly suggest NRL or United States  Government endorsement
 *  of this product.
 * 
 * THIS SOFTWARE IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR
 * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED
 * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.
 *
 ********************************************************************/

This is the NRL NORM source code repository.

The Github "Discussions" feature is available here for comment and question in addition to the regular code issue reporting:

https://github.com/USNavalResearchLaboratory/norm/discussions

SOURCE CODE

The "norm" source Git repository includes "protolib" as a git submodule. You can add the "--recurse-submodules" to your git clone command to automate inclusion:

git clone --recurse-submodules https://github.com/USNavalResearchLaboratory/norm.git

The following items can be built from this source code release:

  1. libnorm.a - static NORM library for Unix platforms, or

    Norm.lib - static NORM protocol library for Win32 platforms

(These libraries can be used by applications using the NORM API. The file "normApi.h" provides function prototypes for NORM API calls and the included "Norm Developer's Guide" provides a reference manual for NORM application development using this API. Additional tutorial material and API usage examples will be provided as additional documentation in the future)

  1. norm (or norm.exe (WIN32) - command-line "demo" application built from "common/normApp.cpp"

(The included "Norm User's Guide" provides a rough overview of how to use this demo app. This demo application is useful for file transfers (and streaming on Unix))

  1. normTest (or normTest.exe (WIN32)) - very simple test application

(The "normTest" application (see "common/normTest.cpp") is really just a simple playground for preliminary testing of the NORM API and does not do anything useful. But it does provide a very simple example of NORM API usage. More sophisticated (and better-documented) examples of NORM API usage will be provided in the future.

  1. There is also an "examples" directory in the source code tree that contains some simplified examples of NORM API usage. The example set will be expanded as time progresses.

NRL also has started some apps built around NORM including:

A) A reliable multicast "chat" application (users can share files and images, too) with a GUI. Like IRC with no server needed.

B) An application for reliably "tunneling" real-time UDP packet streams (like RTP video or audio) using NORM's streaming capability.

NACK-Oriented Proxy (NORP)

NORM Proxy code is also included in the 'norp' directory. More info is available in the norp/doc directory, as well as the NORP website: https://www.nrl.navy.mil/itd/ncs/products/norp

Docker

A Dockerfile is provided to facilitate containerized NORM environment testing. Notable build arguments passed to docker build will influence how the image is created.

  • platformName represents the base image name (defaults to ubuntu)
  • platformVersion represents the tagged version for the base image (defaults to latest)
  • version is a 1:1 mapping to this repo's tag (defaults to v1.5.9)
  • configure is piped to ./waf configure (defaults to "")
  • build is piped to ./waf (defaults to --targets=*)

Example usage:

CentOS7 build

docker build --build-arg platformName=centos \
  --build-arg platformVersion=7 \
  --build-arg="--build-java" \
  -t norm:centos-1.5.9 .

Running the data send and recv examples:

# Sender in one terminal
docker run -it --rm --name norm-send <norm:image> normDataSend
# Receiver in another terminal
docker run -it --rm --name norm-recv <norm:image> normDataRecv

OTHER FILES:

NormDeveloperGuide.pdf - PDF version of NORM Developer's Guide (nice for printing)

NormDeveloperGuide.html - HTML version of NORM Developer's Guide with Hyperlinked content (nice for browsing)

NormUserGuide.pdf - Guide to "norm" demo app usage

VERSION.TXT - NORM version history notes

README.md - this file

docker - directory containing docker container support

NOTES:

The NORM code depends upon the current "Protolib" release:

https://github.com/USNavalResearchLaboratory/protolib

It has been addded as a git submodule to the NORM git repository. So, to to build you will need to do the following steps to download the protolib code:

git clone --recurse-submodules https://github.com/USNavalResearchLaboratory/norm.git

Alternatively after a basic "git clone" you can do the folowing to pull in the protolib source:

cd norm git submodule update --init

To keep the 'protolib' submodule up to date, you will need to do the following:

cd norm/protolib git checkout master

This will enable you to issue 'git pull', etc to treat the 'protolib' sub-directory as its own (sub-) repository.