BodgeMaster
/
NORM-mirror
mirror of https://github.com/USNavalResearchLaboratory/norm.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Documentation updates and pynorm example bug fixes (#82) 

* Added notes about waf execution in BUILD.TXT

* Updated instructions for cloning and running make

* Added more detailed instructions for win32 builds

* Fixed typo on line 20 (capitalized 'f') that caused the command to fail

* Generic editing and replaced dead URLs

* Substituted setuptools package for the deprecated distutils in setup.py

* Fixed typo in 'norm.exe' file path in Win32 build instructions

* Fixed a bugs in normCast.py and normFileSend.py

---------

Co-authored-by: Lucas <lucas.vancina@nps.edu>
pull/86/head
LVancina 2024-01-24 10:15:06 -08:00 committed by GitHub
parent af9a238a70
commit a992176873
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
7 changed files with 60 additions and 33 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

20
BUILD.TXT
View File

@ -6,11 +6,26 @@ To see a full list of options, run:
./waf -h ./waf -h
Note: if attempting to run waf as an executable as shown above results in a
message like this:
/usr/bin/env: ‘python’: No such file or directory
then try calling python directly:
python waf <args>
or
python3 waf <args>
For more background information about waf, visit the gitlab page at
https://gitlab.com/ita1024/./waf/
Configuring Configuring
----------- -----------
To perform the configure checks, run: To perform the configuration checks, run:
./waf configure ./waf configure
@ -51,6 +66,9 @@ To install, run:
This will install the compiled library and headers to wherever your prefix was This will install the compiled library and headers to wherever your prefix was
specified. (See configure flags) specified. (See configure flags)
This command may have to be run with root priviledges for certain prefix
directories depending on your system setup.
Uninstalling Uninstalling
------------ ------------

2
README-PyNorm.txt
View File

@ -28,7 +28,7 @@ in similar places as your system's compiler. On UNIX systems, this is usually
PATH environment variable. PATH environment variable.
On Windows, PyNORM requires the "PyWin32" module, available at: On Windows, PyNORM requires the "PyWin32" module, available at:
http://sf.net/projects/pywin32 https://sourceforge.net/projects/pywin32
------------- -------------
Installation Installation

39
doc/NormDeveloperGuide.xml
View File

@ -13,19 +13,19 @@
<abstract> <abstract>
<para>This document describes an application programming interface (API) <para>This document describes an application programming interface (API)
for the <ulink url="http://norm.pf.itd.nrl.navy.mil/">Nack-Oriented for the <ulink url="https://www.nrl.navy.mil/Our-Work/Areas-of-Research/Information-Technology/NCS/NORM/">Nack-Oriented
Reliable Multicast (NORM)</ulink> protocol implementation developed by Reliable Multicast (NORM)</ulink> protocol implementation developed by
the Protocol Engineering and Advance Networking (<ulink the Protocol Engineering and Advance Networking (<ulink
url="http://cs.itd.nrl.navy.mil/">PROTEAN</ulink>) Research Group of the url="https://www.nrl.navy.mil/itd/ncs/">PROTEAN</ulink>) Research Group of the
United States <ulink url="http://www.nrl.navy.mil/">Naval Research United States <ulink url="https://www.nrl.navy.mil/">Naval Research
Laboratory</ulink> (NRL). The NORM protocol provides general purpose Laboratory</ulink> (NRL). The NORM protocol provides general purpose
reliable data transport for applications wishing to use Internet reliable data transport for applications wishing to use Internet
Protocol (IP) Multicast services for group data delivery. NORM can also Protocol (IP) Multicast services for group data delivery. NORM can also
support unicast (point-to-point) data communication and may be used for support unicast (point-to-point) data communication and may be used for
such when deemed appropriate. The current NORM protocol specification is such when deemed appropriate. The current NORM protocol specification is
given in the <ulink url="http://www.ietf.org/">Internet Engineering Task given in the <ulink url="https://www.ietf.org/">Internet Engineering Task
Force</ulink> (IETF) <ulink Force</ulink> (IETF) <ulink
url="http://norm.pf.itd.nrl.navy.mil/rfc3940.pdf">RFC 3940</ulink>. This url="https://datatracker.ietf.org/doc/html/rfc5740">RFC 5740</ulink>. This
document is currently a reference guide to the NORM API of the NRL document is currently a reference guide to the NORM API of the NRL
reference implementation. More tutorial material may be include in a reference implementation. More tutorial material may be include in a
future version of this document or a separate developer's tutorial may future version of this document or a separate developer's tutorial may
@ -37,19 +37,19 @@
<title>Background</title> <title>Background</title>
<para>This document describes an application programming interface (API) <para>This document describes an application programming interface (API)
for the <ulink url="http://norm.pf.itd.nrl.navy.mil/">Nack-Oriented for the <ulink url="https://www.nrl.navy.mil/Our-Work/Areas-of-Research/Information-Technology/NCS/NORM/">Nack-Oriented
Reliable Multicast (NORM)</ulink> protocol implementation developed by the Reliable Multicast (NORM)</ulink> protocol implementation developed by the
Protocol Engineering and Advance Networking (<ulink Protocol Engineering and Advance Networking (<ulink
url="http://cs.itd.nrl.navy.mil/">PROTEAN</ulink>) Research Group of the url="https://www.nrl.navy.mil/itd/ncs/">PROTEAN</ulink>) Research Group of the
United States <ulink url="http://www.nrl.navy.mil/">Naval Research United States <ulink url="https://www.nrl.navy.mil/">Naval Research
Laboratory</ulink> (NRL). The NORM protocol provides general purpose Laboratory</ulink> (NRL). The NORM protocol provides general purpose
reliable data transport for applications wishing to use Internet Protocol reliable data transport for applications wishing to use Internet Protocol
(IP) Multicast services for group data delivery. NORM can also support (IP) Multicast services for group data delivery. NORM can also support
unicast (point-to-point) data communication and may be used for such when unicast (point-to-point) data communication and may be used for such when
deemed appropriate. The current NORM protocol specification is given in deemed appropriate. The current NORM protocol specification is given in
the <ulink url="http://www.ietf.org/">Internet Engineering Task the <ulink url="https://www.ietf.org/">Internet Engineering Task
Force</ulink> (IETF) <ulink Force</ulink> (IETF) <ulink
url="http://norm.pf.itd.nrl.navy.mil/rfc5740.pdf">RFC 5740</ulink>.</para> url="https://datatracker.ietf.org/doc/html/rfc5740">RFC 5740</ulink>.</para>
<para>The NORM protocol is designed to provide end-to-end reliable <para>The NORM protocol is designed to provide end-to-end reliable
transport of bulk data objects or streams over generic IP multicast transport of bulk data objects or streams over generic IP multicast
@ -134,7 +134,7 @@
may participate as senders and/or receivers within a NORM session. NORM may participate as senders and/or receivers within a NORM session. NORM
senders transmit data to the session destination address (usually an IP senders transmit data to the session destination address (usually an IP
multicast group) while receivers are notified of incoming data. The NORM multicast group) while receivers are notified of incoming data. The NORM
API provides and event notification scheme to notify the application of API provides an event notification scheme to notify the application of
significant sender and receiver events. There are also a number support significant sender and receiver events. There are also a number support
functions provided for the application to control and monitor its functions provided for the application to control and monitor its
participation within a NORM transport session.</para> participation within a NORM transport session.</para>
@ -159,7 +159,7 @@
linkend="NormCreateInstance"><literal>NormCreateInstance()</literal></link>) linkend="NormCreateInstance"><literal>NormCreateInstance()</literal></link>)
as needed for applications with specific requirements for accessing and as needed for applications with specific requirements for accessing and
controlling participation in multiple <emphasis>NormSessions</emphasis> controlling participation in multiple <emphasis>NormSessions</emphasis>
from separate operating system multiple threads. Or, alternatively, a from separate operating system threads. Or, alternatively, a
single <emphasis>NormInstance</emphasis> could be used, with a "master single <emphasis>NormInstance</emphasis> could be used, with a "master
thread" serving as an intermediary between the <link thread" serving as an intermediary between the <link
linkend="NormGetNextEvent"><literal>NormGetNextEvent()</literal></link> linkend="NormGetNextEvent"><literal>NormGetNextEvent()</literal></link>
@ -194,7 +194,7 @@
number of concurrently active senders, this may translate to significant number of concurrently active senders, this may translate to significant
memory allocation on receiver nodes. Currently, the API allows the memory allocation on receiver nodes. Currently, the API allows the
application to control how much buffer space is allocated for each application to control how much buffer space is allocated for each
active sender (NOTE: In the future, API functions may be provided limit active sender (NOTE: In the future, API functions may be provided to limit
the number of active senders monitored and/or provide the application the number of active senders monitored and/or provide the application
with finer control over receive buffer allocation, perhaps on a per with finer control over receive buffer allocation, perhaps on a per
sender basis).</para> sender basis).</para>
@ -227,7 +227,7 @@
<title>Data Transmission</title> <title>Data Transmission</title>
<para>The behavior of data transport operation is largely placed in <para>The behavior of data transport operation is largely placed in
the control of the NORM sender(s). NORM senders controls their data the control of the NORM sender(s). NORM senders control their data
transmission rate, forward error correction (FEC) encoding settings, transmission rate, forward error correction (FEC) encoding settings,
and parameters controlling feedback from the receiver group. Multiple and parameters controlling feedback from the receiver group. Multiple
senders may operate in a session, each with independent transmission senders may operate in a session, each with independent transmission
@ -253,7 +253,7 @@
be called before any calls to <link be called before any calls to <link
linkend="NormStartReceiver"><literal>NormStartReceiver()</literal></link> linkend="NormStartReceiver"><literal>NormStartReceiver()</literal></link>
are made. However, note that the cache directory may be changed even are made. However, note that the cache directory may be changed even
during active NORM reception. In this case, the new specified during active NORM reception. In this case, the newly specified
directory path will be used for subsequently-received files. Any files directory path will be used for subsequently-received files. Any files
received before a directory path change will remain in the previous received before a directory path change will remain in the previous
cache location. Note that the <link cache location. Note that the <link
@ -477,15 +477,14 @@
Protokit. These are described below.</para> Protokit. These are described below.</para>
<para>The "makefiles" directory contains Unix Makefiles for various <para>The "makefiles" directory contains Unix Makefiles for various
platforms the "win32" and "wince" sub-directories there contain Microsoft platforms. The "win32" and "wince" sub-directories there contain Microsoft
Visual C++ (VC++) and Embedded VC++ project files for building the NORM Visual C++ (VC++) and Embedded VC++ project files for building the NORM
implementation. Additionally, a "waf" (Python-based build tool) build implementation. Additionally, a "waf" (Python-based build tool) build
option is supported that can be used to build and install the NORM library option is supported that can be used to build and install the NORM library
code on the supported platforms. Finally, Python and Java bindings to the code on the supported platforms. Finally, Python and Java bindings to the
NORM API are included and "src/python" and "src/java" directories contain NORM API are included in the "src/python" and "src/java" directories and the
the code for these and the "makefiles/java" directory contains Makefiles "makefiles/java" directory contains Makefiles to build the NORM Java JNI bindings.
to build the NORM Java JNI bindings. Note the "waf" tool can also be used Note the "waf" tool can also be used to build the Java and Python bindings.</para>
to build the Java and Python bindings.</para>
<sect2> <sect2>
<title>Unix Platforms</title> <title>Unix Platforms</title>

20
doc/NormUserGuide.xml
View File

@ -96,11 +96,13 @@
<orderedlist> <orderedlist>
<listitem> <listitem>
<para>Download and unpack the NORM source code tarball.</para> <para>Download and unpack the NORM source code tarball or clone the github
repository using
<literal>git clone --recurse-submodules https://github.com/USNavalResearchLaboratory/norm.git</literal></para>
</listitem> </listitem>
<listitem> <listitem>
<para><literal>cd norm/unix</literal></para> <para><literal>cd norm/makefiles</literal></para>
</listitem> </listitem>
<listitem> <listitem>
@ -118,7 +120,8 @@
<orderedlist> <orderedlist>
<listitem> <listitem>
<para>Download and unpack the NORM source code tarball</para> <para>Download and unpack the NORM source code tarball or clone the
repository as described in the section above.</para>
</listitem> </listitem>
<listitem> <listitem>
@ -128,7 +131,7 @@
</listitem> </listitem>
<listitem> <listitem>
<para>Open the "<filename>norm.sln</filename>" (VC++ .Net), <para>Inside the "<filename>norm/makefiles/win32</filename>" directory, open the "<filename>norm.sln</filename>" (VC++ .Net),
"<filename>norm.dsw</filename>" (VC++ 6.0), or "<filename>norm.dsw</filename>" (VC++ 6.0), or
"<filename>norm.vcw</filename>" (Embedded VC++) workspace "<filename>norm.vcw</filename>" (Embedded VC++) workspace
file.</para> file.</para>
@ -137,9 +140,14 @@
<listitem> <listitem>
<para>The "<emphasis>norm</emphasis>" project can be selected and <para>The "<emphasis>norm</emphasis>" project can be selected and
built.  The "<filename>norm.exe</filename>" file will be found in built.  The "<filename>norm.exe</filename>" file will be found in
the "<filename>norm/win32/norm/Release</filename>" directory.</para> the "<filename>norm/Win32/norm/Release</filename>" directory.</para>
</listitem> </listitem>
</orderedlist> </orderedlist>
<para>The easiest way to run these builds is with Visual Studio. If you do not already have Visual Studio installed, then it can
be downloaded for free <ulink url="https://visualstudio.microsoft.com/downloads/?cid=learn-onpage-download-cta">here</ulink>.
When you launch Visual Studio, make sure to select "Visual C++" and open the "<filename>norm.sln</filename>" file.
This should load all of the project files and allow you to right-click on any of them in the Solution Explorer on the
left side and select "build".</para>
</sect2> </sect2>
</sect1> </sect1>
@ -380,7 +388,7 @@
<para>A minimal example <emphasis>norm</emphasis> sender command-line <para>A minimal example <emphasis>norm</emphasis> sender command-line
syntax is:</para> syntax is:</para>
<para><literal>norm addr &lt;addr/port&gt; sendFile <para><literal>norm addr &lt;addr/port&gt; sendfile
&lt;fileName&gt;</literal></para> &lt;fileName&gt;</literal></para>
<para>The corresponding minimal <emphasis>norm</emphasis> receiver <para>The corresponding minimal <emphasis>norm</emphasis> receiver

6
examples/python/normCast.py
View File

@ -301,14 +301,16 @@ class NormCaster():
self.is_running = False self.is_running = False
return return
obj:Optional[pynorm.Object] = session.fileEnqueue(self.pendingSendFilePath, info= self.pendingSendFilePath.replace("\\","/").encode() ) lastSlash = self.pendingSendFilePath.rfind('/')
filename = self.pendingSendFilePath[lastSlash+1:]
obj:Optional[pynorm.Object] = session.fileEnqueue(self.pendingSendFilePath, info= filename.encode() )
if obj: if obj:
if self.opts.ack: if self.opts.ack:
session.setWatermark(obj,True) session.setWatermark(obj,True)
logging.info(f"add file:{self.pendingSendFilePath} {obj._object}") logging.info(f"add file:{self.pendingSendFilePath} {obj._object}")
self.pendingSendFilePath = None # succeed enqued self.pendingSendFilePath = None # succeed enqued
else: else:
logging.warning(f"fileEnqueue: {sendFilePath} failure!") logging.warning(f"fileEnqueue: {self.pendingSendFilePath} failure!")
def handle_norm_event(self, event:pynorm.Event): def handle_norm_event(self, event:pynorm.Event):
session:pynorm.Session = event.session session:pynorm.Session = event.session

2
examples/python/normFileSend.py
View File

@ -44,7 +44,7 @@ def main(argv):
session.startSender(randint(0, 1000), 1024**2, 1400, 64, 16) session.startSender(randint(0, 1000), 1024**2, 1400, 64, 16)
print(('Sending file %s' % filename)) print(('Sending file %s' % filename))
session.fileEnqueue(filepath, filename) session.fileEnqueue(filepath, info=filename.encode())
try: try:
for event in instance: for event in instance:

2
setup.py
View File

@ -1,6 +1,6 @@
# This Python script can be used to install the Python 'pynorm' package # This Python script can be used to install the Python 'pynorm' package
import platform import platform
from distutils.core import setup, Extension from setuptools import setup, Extension
# Note to use 'pynorm", you will need to have the libnorm shared library # Note to use 'pynorm", you will need to have the libnorm shared library
# (libnorm.so or libnorm.dylib, etc) installed where your Python installation # (libnorm.so or libnorm.dylib, etc) installed where your Python installation