BodgeMaster
/
NORM-mirror
mirror of https://github.com/USNavalResearchLaboratory/norm.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

v1.5.7

pull/2/head v1.5.7
Jeff Weston 2019-09-11 12:21:00 -04:00
parent da5e8f6692
commit a8ab40d749
53 changed files with 7910 additions and 2114 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

47
README-Java.txt
View File

@ -4,28 +4,42 @@ Java JNI bindings for NORM
By: By:
Jason Rush <jason.rush@gd-ais.com> Jason Rush <jason.rush@gd-ais.com>
Peter Griffin <peter.griffin@gd-ais.com> Peter Griffin <peter.griffin@gd-ais.com>
updates covering waf build system by Duc Nguyen <dnguyen@aypeks.com>
Updated: 2018-03-14
The Java JNI bindings for NORM provide Java bindings for the NORM C API. The Java JNI bindings for NORM provide Java bindings for the NORM C API.
For documentation about the main NORM API calls, refer to the NORM Developers For documentation about the main NORM API calls, refer to the NORM Developers
guide in the regular NORM distribution. guide in the regular NORM distribution.
The JNI bindings for NORM can be built using two methods. The original
method uses Make and an Ant build.xml script in makefiles/java. The
newer method uses the waf build system and builds for all platforms with
various configurations via configuration switches.
------------ ------------
Requirements Requirements
------------ ------------
Java JNI bindings for NORM requires at least Java 1.5; however, it has also Java JNI bindings for NORM requires at least Java 1.5; however, it has also
been tested with Java 1.6. been tested with Java 1.6 and Java 1.8
The NORM library should be built prior to building the Java JNI bindings since The NORM library should be built prior to building the Java JNI bindings since
they link against it. they link against it.
### Using Ant and Make ###
Apache Ant is required for building the class files and jar file. Apache Ant is required for building the class files and jar file.
### Python waf ###
Tested with Python 2 but Python 3 should be ok.
------------ ------------
Building Building
------------ ------------
### Using Ant and Make ###
The build files for Java JNI bindings for NORM are located in the The build files for Java JNI bindings for NORM are located in the
makefiles/java directory. makefiles/java directory.
@ -42,6 +56,23 @@ makefiles/java directory with the correct make file for your system:
This will produce a libmil_navy_nrl_norm.so file in the lib/ directory. This will produce a libmil_navy_nrl_norm.so file in the lib/ directory.
### Python waf ###
Reconfigure waf to use java and choose the target on a windows system:
> cd <norm root dir>
> waf configure --build-java --msvc_target=x64
The msvc_target flag allows you to choose the target Windows
architecture. The default is "x86" (i.e. 32bit arch)
Build the NORM libraries:
> waf build
The output will be in the build/ directory. Copy both
mil_navy_nrl_norm.* and norm-1.* to the location the JRE libraries.
------------ ------------
Installation Installation
------------ ------------
@ -75,9 +106,23 @@ If you observe an "java.lang.UnsatisfiedLinkError" exception while running
your application, you do not have the libmil_navy_nrl_norm.so installed in your application, you do not have the libmil_navy_nrl_norm.so installed in
the correct location. the correct location.
If you're running on Windows, you also need to have the PATH environment
variable set with along with the -Djava.library.path=<dir>
------------ ------------
Examples Examples
------------ ------------
Examples using the Java JNI bindings for NORM are provided in the examples/java Examples using the Java JNI bindings for NORM are provided in the examples/java
directory. directory.
Running on Windows (could be put in a .bat script):
set PATH=%PATH%;<path to mil_navy_nrl_norm.dll>
java -Djava.library.path=<path to mil_navy_nrl_norm.dll> ...
Running on Linux or MacOS:
java -Djava.library.path=<path to mil_navy_nrl_norm.dll> ...

40
VERSION.TXT
View File

@ -1,5 +1,45 @@
NORM Version History NORM Version History
Version 1.5.7
=============
- Fixed issue with NormSession::SenderRemoveAckingNode() method where
the removed NormAckingNode instance was not being properly deleted.
(Thanks to Chad Gioia)
- Fixed bug in normPrecoder (npc) when building on 64-bit architectures
(Thanks to Leon Meyer)
- Fixed bug in JNI NormNode.getAddress() method where an "unsigned short"
type was used instead of "UINT16" to receive the port portion of the
address. (Thanks to Jame Nguyen)
- Reduced warnings, added gnu and other build options to 'waf' wscripts
(Thanks to Luca Buccassi)
- Greatly improved high speed performance via enhancements to Protolib
- Fixed bugs related to coding block id wrapping for very long-lived
(or very high speed) NORM sessions.
- Added NormSocket API extension in "norm/examples" that provides a
sockets-like API and client-server connection-oriented paradigm
for using NORM. This currently supports a byte-stream service
for both multicast and unicast using NORM_OBJECT_STREAM. This API
extension is still work in progress and will be expanded to include
message-stream service support as well as NORM's file/data object
modes. The "norm/examples/normClient.cpp" (and "normServer.cpp)
illustrates use of the the API extension defined in
"norm/examples/normSocket.h"
- Added "normStreamer" example that provides the same function as the
"normMsgr" example except using NORM_OBJECT_STREAM.
- The bug fixes and performance enhancements make this a key update.
The NORM Developer's Guide is not yet updated to described some of
the new API features added.
- Added APIs to iterate through sender "Acking Node List" to determine
who has and who has not acknowledged a watermark acknowledgement
request.
- Added NormSetAutoAckingNodes() call with NormTrackingStatus to
automate detection of new group members. The Acking Node List
iterator function NormGetNextAckingNode() can be used to learn
who has joined and the NORM_ACKING_NODE_NEW provides notification
to the application of new members.
- The 'waf' build scripts assign the current version number to
the NORM shared library built for better version control/tracking.
Version 1.5r6 Version 1.5r6
============= =============
- Fixed Protolib ProtoBitmask bug for NDEBUG (release) builds. - Fixed Protolib ProtoBitmask bug for NDEBUG (release) builds.

6
doc/NormDeveloperGuide.xml
View File

@ -1181,7 +1181,7 @@ bool <link linkend="NormSetCacheDirectory"><literal>NormSetCacheDirectory</liter
bool <link linkend="NormGetNextEvent"><literal>NormGetNextEvent</literal></link>(<link bool <link linkend="NormGetNextEvent"><literal>NormGetNextEvent</literal></link>(<link
linkend="NormInstanceHandle"><literal>NormInstanceHandle</literal></link> instanceHandle, linkend="NormInstanceHandle"><literal>NormInstanceHandle</literal></link> instanceHandle,
<link linkend="NormEvent"><literal>NormEvent</literal></link>* theEvent);</programlisting> <link linkend="NormEvent"><literal>NormEvent*</literal></link> theEvent);</programlisting>
</sect4> </sect4>
<sect4> <sect4>
@ -3763,7 +3763,7 @@ void <link linkend="NormSetTxRobustFactor"><literal>NormSetTxRobustFactor</liter
<programlisting>#include &lt;normApi.h&gt; <programlisting>#include &lt;normApi.h&gt;
boolNormRequeueObject(<link linkend="NormSessionHandle"><literal>NormSessionHandle</literal></link> sessionHandle, bool NormRequeueObject(<link linkend="NormSessionHandle"><literal>NormSessionHandle</literal></link> sessionHandle,
<link linkend="NormObjectHandle"><literal>NormObjectHandle</literal></link> objectHandle);</programlisting> <link linkend="NormObjectHandle"><literal>NormObjectHandle</literal></link> objectHandle);</programlisting>
</sect4> </sect4>
@ -6596,7 +6596,7 @@ bool <link linkend="NormNodeGetAddress"><literal>NormNodeGetAddress</literal></l
<programlisting>#include &lt;normApi.h&gt; <programlisting>#include &lt;normApi.h&gt;
double <link linkend="NormNodeGetId"><literal>NormNodeGetId</literal></link>(<link double <link linkend="NormNodeGetId"><literal>NormNodeGetGrtt</literal></link>(<link
linkend="NormNodeHandle"><literal>NormNodeHandle</literal></link> nodeHandle);</programlisting> linkend="NormNodeHandle"><literal>NormNodeHandle</literal></link> nodeHandle);</programlisting>
</sect4> </sect4>

2
examples/java/NormMsgr.java
View File

@ -239,7 +239,7 @@ public class NormMsgr
normTxLock.release(); normTxLock.release();
return false; return false;
} }
//System.err.println("caching msg for object " + obj + "\n"); // System.err.println("caching msg for object " + obj + "\n");
// Cache the msg associated with the resultant tx object. We use the // Cache the msg associated with the resultant tx object. We use the
// input_msg_list HashMap so we can remove the msg upon NORM_TX_OBJECT_PURGED // input_msg_list HashMap so we can remove the msg upon NORM_TX_OBJECT_PURGED
input_msg_list.put(obj, msg); input_msg_list.put(obj, msg);

163
examples/normClient.cpp
View File

@ -1,9 +1,15 @@
#include "normSocket.h" #include "normSocket.h"
#include <stdio.h> #include <stdio.h>
#include <string.h> #include <string.h>
#ifdef WIN32
#include "win32InputHandler.cpp" // brings in the entire implementation
#else
#include <sys/select.h> #include <sys/select.h>
#include <fcntl.h> // for, well, fnctl() #include <fcntl.h> // for, well, fnctl()
#include <errno.h> // obvious child #include <errno.h> // obvious child
#endif // if/else WIN32/UNIX
// BUILD: (assumes "normApi.h" in "include" ... // BUILD: (assumes "normApi.h" in "include" ...
// g++ -I../include -o normClient normClient.cpp normSocket.cpp ../lib/libnorm.a ../protolib/lib/libprotokit.a -lresolv // g++ -I../include -o normClient normClient.cpp normSocket.cpp ../lib/libnorm.a ../protolib/lib/libprotokit.a -lresolv
@ -19,6 +25,11 @@
// To deny a connection, the "server" can send a stream close??? Which means the server will have // To deny a connection, the "server" can send a stream close??? Which means the server will have
// to establish state just to deny a client a connection (yuck!) // to establish state just to deny a client a connection (yuck!)
void Usage()
{
fprintf(stderr, "Usage: normClient [connect <serverAddr>[/<port>][,<groupAddr>]][debug <level>][trace]\n");
}
const unsigned int MSG_LENGTH_MAX = 64; const unsigned int MSG_LENGTH_MAX = 64;
int main(int argc, char* argv[]) int main(int argc, char* argv[])
@ -33,18 +44,19 @@ int main(int argc, char* argv[])
bool trace = false; bool trace = false;
unsigned int debugLevel = 0; unsigned int debugLevel = 0;
for (int i = 1; i < argc; i++) for (int i = 1; i < argc; i++)
{ {
const char* cmd = argv[i]; const char* cmd = argv[i];
unsigned int len = strlen(cmd); unsigned int len = strlen(cmd);
if (0 == strncmp(cmd, "connect", len)) if (0 == strncmp(cmd, "connect", len))
{ {
// connect <serverAddr>/<port>[,<groupAddr> // connect <serverAddr>/<port>[,<groupAddr>]
const char* val = argv[++i]; const char* val = argv[++i];
const char* portPtr = strchr(val, '/'); const char* portPtr = strchr(val, '/');
if (NULL == portPtr) if (NULL == portPtr)
{ {
fprintf(stderr, "normClient error: missing <port> number\n"); fprintf(stderr, "normClient error: missing <port> number\n");
Usage();
return -1; return -1;
} }
portPtr++; portPtr++;
@ -58,6 +70,7 @@ int main(int argc, char* argv[])
else else
{ {
fprintf(stderr, "normClient error: missing <serverAddr>\n"); fprintf(stderr, "normClient error: missing <serverAddr>\n");
Usage();
return -1; return -1;
} }
char portText[32]; char portText[32];
@ -76,6 +89,7 @@ int main(int argc, char* argv[])
if (1 != sscanf(portText, "%hu", &serverPort)) if (1 != sscanf(portText, "%hu", &serverPort))
{ {
fprintf(stderr, "normClient error: invalid <port>\n"); fprintf(stderr, "normClient error: invalid <port>\n");
Usage();
return -1; return -1;
} }
if (NULL != groupPtr) if (NULL != groupPtr)
@ -96,7 +110,8 @@ int main(int argc, char* argv[])
{ {
if (1 != sscanf(argv[++i], "%u", &debugLevel)) if (1 != sscanf(argv[++i], "%u", &debugLevel))
{ {
fprintf(stderr, "normServer error: invalid debug level\n"); fprintf(stderr, "normClient error: invalid debug level\n");
Usage();
return -1; return -1;
} }
} }
@ -107,13 +122,15 @@ int main(int argc, char* argv[])
if (1 != sscanf(val, "%lu", &id)) if (1 != sscanf(val, "%lu", &id))
{ {
fprintf(stderr, "normClient error: invalid 'id' value\n"); fprintf(stderr, "normClient error: invalid 'id' value\n");
Usage();
return -1; return -1;
} }
clientId = id; clientId = id;
} }
else else
{ {
fprintf(stderr, "normServer error: invalid command \"%s\"\n", cmd); fprintf(stderr, "normClient error: invalid command \"%s\"\n", cmd);
Usage();
return -1; return -1;
} }
} }
@ -122,23 +139,39 @@ int main(int argc, char* argv[])
// Initate connection to server ... // Initate connection to server ...
fprintf(stderr, "normClient: connecting to %s/%hu ...\n", serverAddr, serverPort); fprintf(stderr, "normClient: connecting to %s/%hu ...\n", serverAddr, serverPort);
NormSocketHandle normSocket = NormConnect(instance, serverAddr, serverPort, groupAddrPtr, clientId); NormSocketHandle normSocket = NormOpen(instance);
// setting 'localPort' param here to zero lets an ephemeral port be picked
NormConnect(normSocket, serverAddr, serverPort, 0, groupAddrPtr, clientId);
/* // Optional code to test NormWrite() immediately after NormConnect() call
// (Note this is _not_ compatible with newer stale connection reject code)
const char* helloStr = "Hello\n";
unsigned int helloLen = strlen(helloStr) + 1;
NormWrite(normSocket, helloStr, helloLen);
NormFlush(normSocket);*/
if (trace) if (trace)
{ {
NormSetMessageTrace(NormGetSession(normSocket), true); NormSetMessageTrace(NormGetSocketSession(normSocket), true);
if (NULL != groupAddrPtr) if (NULL != groupAddrPtr)
NormSetMessageTrace(NormGetMulticastSession(normSocket), true); NormSetMessageTrace(NormGetSocketMulticastSession(normSocket), true);
} }
if (0 != debugLevel) NormSetDebugLevel(debugLevel); if (0 != debugLevel) NormSetDebugLevel(debugLevel);
//NormSetDebugLevel(3); //NormSetDebugLevel(3);
//NormSetMessageTrace(NormGetSession(normSocket), true); //NormSetMessageTrace(NormGetSocketSession(normSocket), true);
// We use a select() call to multiplex input reading and NormSocket handling #ifdef WIN32
HANDLE hStdout = GetStdHandle(STD_OUTPUT_HANDLE);
Win32InputHandler inputHandler;
inputHandler.Open();
HANDLE handleArray[2];
handleArray[0] = NormGetDescriptor(instance);
handleArray[1] = inputHandler.GetEventHandle();
#else
// On Unix, use a select() call to multiplex input reading and NormSocket handling
fd_set fdset; fd_set fdset;
FD_ZERO(&fdset); FD_ZERO(&fdset);
// Get our input (STDIN) descriptor and set non-blocking // Get our input (STDIN) descriptor and set non-blocking
FILE* inputFile = stdin; FILE* inputFile = stdin;
int inputfd = fileno(inputFile); int inputfd = fileno(inputFile);
@ -146,15 +179,45 @@ int main(int argc, char* argv[])
perror("normClient: fcntl(inputfd, O_NONBLOCK) error"); perror("normClient: fcntl(inputfd, O_NONBLOCK) error");
// Get our NormInstance descriptor // Get our NormInstance descriptor
int normfd = NormGetDescriptor(instance); int normfd = NormGetDescriptor(instance);
# endif // if/else WIN32
bool keepGoing = true; bool keepGoing = true;
bool writeReady = false; bool writeReady = false;
unsigned int inputLength = 0; int inputLength = 0;
unsigned int bytesWritten = 0; unsigned int bytesWritten = 0;
const unsigned int BUFFER_LENGTH = 2048; const unsigned int BUFFER_LENGTH = 2048;
char inputBuffer[BUFFER_LENGTH]; char inputBuffer[BUFFER_LENGTH];
bool inputNeeded = false; // will be set to "true" upon CONNECT bool inputNeeded = false; // will be set to "true" upon CONNECT
while (keepGoing) while (keepGoing)
{ {
bool normEventPending = false;
bool inputEventPending = false;
#ifdef WIN32
DWORD handleCount = inputNeeded ? 2 : 1;
DWORD waitStatus =
MsgWaitForMultipleObjectsEx(handleCount, // number of handles in array
handleArray, // object-handle array
INFINITE, // time-out interval
QS_ALLINPUT, // input-event type
0);
if ((WAIT_OBJECT_0 <= waitStatus) && (waitStatus < (WAIT_OBJECT_0 + handleCount)))
{
if (0 == (waitStatus - WAIT_OBJECT_0))
normEventPending = true;
else
inputEventPending = true;
}
else if (-1 == waitStatus)
{
perror("normClient: MsgWaitForMultipleObjectsEx() error");
break;
}
else
{
// TBD - any other status we should handle?
// (e.g. WAIT_TIMEOUT, WAIT_ABANDONED or WAIT_IO_COMPLETION)
continue; // ignore for now
}
#else
FD_SET(normfd, &fdset); FD_SET(normfd, &fdset);
int maxfd = normfd; int maxfd = normfd;
if (inputNeeded) if (inputNeeded)
@ -173,8 +236,36 @@ int main(int argc, char* argv[])
break; break;
} }
if (FD_ISSET(inputfd, &fdset)) if (FD_ISSET(inputfd, &fdset))
inputEventPending = true;
if (FD_ISSET(normfd, &fdset))
normEventPending = true;
#endif // if/else WIN32/UNIX
if (inputEventPending)
{ {
// Read input into our txBuffer // Read input into our txBuffer
#ifdef WIN32
inputLength = inputHandler.ReadData(inputBuffer, BUFFER_LENGTH);
if (inputLength > 0)
{
// We got our input
bytesWritten = 0;
inputNeeded = false;
}
else if (inputLength < 0)
{
// Input stream has likely closed, initiate client shutown
// TBD - initiate client shutdown
if (NULL == groupAddrPtr)
{
fprintf(stderr, "normClient: CLOSING connection to server ...\n");
NormShutdown(normSocket);
}
inputNeeded = false; // TBD -should we also fclose(inputFile)???
inputHandler.Close();
}
// else zero bytes read, still need input
#else
inputLength = fread(inputBuffer, 1, BUFFER_LENGTH, inputFile); inputLength = fread(inputBuffer, 1, BUFFER_LENGTH, inputFile);
if (inputLength > 0) if (inputLength > 0)
{ {
@ -191,7 +282,11 @@ int main(int argc, char* argv[])
NormShutdown(normSocket); NormShutdown(normSocket);
} }
inputNeeded = false; // TBD -should we also fclose(inputFile)??? inputNeeded = false; // TBD -should we also fclose(inputFile)???
if (stdin != inputFile)
{
fclose(inputFile);
inputFile = NULL;
}
// else stick around to receive stuff from the server // else stick around to receive stuff from the server
} }
else if (ferror(inputFile)) else if (ferror(inputFile))
@ -205,14 +300,15 @@ int main(int argc, char* argv[])
// input starved, wait for next notification // input starved, wait for next notification
break; break;
default: default:
perror("normClient: error reading input"); perror("normClient: error reading input?!");
break; break;
} }
} }
} #endif // if/else WIN32/UNIX
if (FD_ISSET(normfd, &fdset)) } // end if inputEventPending
{
if (normEventPending)
{
// There's a NORM event pending // There's a NORM event pending
NormSocketEvent event; NormSocketEvent event;
if (NormGetSocketEvent(instance, &event)) if (NormGetSocketEvent(instance, &event))
@ -225,7 +321,11 @@ int main(int argc, char* argv[])
case NORM_SOCKET_CONNECT: case NORM_SOCKET_CONNECT:
{ {
fprintf(stderr, "normClient: CONNECTED to server ...\n"); char remoteAddr[16];
unsigned int addrLen = 16;
UINT16 remotePort;
NormGetPeerName(normSocket, remoteAddr, &addrLen, &remotePort);
fprintf(stderr, "normClient: CONNECTED to server %s/%hu\n", serverAddr, remotePort);
inputNeeded = true; inputNeeded = true;
writeReady = true; writeReady = true;
break; break;
@ -246,7 +346,17 @@ int main(int argc, char* argv[])
continue; continue;
} }
if (bytesRead > 0) if (bytesRead > 0)
{
#ifdef WIN32
// Use Win32 WriteFile() so there is no buffer delay
DWORD dwWritten;
WriteFile(hStdout, buffer, bytesRead, &dwWritten, NULL);
#else
fwrite(buffer, sizeof(char), bytesRead, stdout); fwrite(buffer, sizeof(char), bytesRead, stdout);
#endif // if/else WIN32
}
// If less bytes read than request then need
// to wait for next NORM_SOCKET_READ event
if (bytesRead < 1024) rxReady = false; if (bytesRead < 1024) rxReady = false;
} }
break; break;
@ -256,13 +366,16 @@ int main(int argc, char* argv[])
break; break;
case NORM_SOCKET_CLOSING: case NORM_SOCKET_CLOSING:
fprintf(stderr, "normClient: server CLOSING connection ...\n"); fprintf(stderr, "normClient: server CLOSING connection ...\n");
writeReady = false;
inputNeeded = false; inputNeeded = false;
break; break;
case NORM_SOCKET_CLOSED: case NORM_SOCKET_CLOSE:
{ {
fprintf(stderr, "normClient: connection to server CLOSED.\n"); fprintf(stderr, "normClient: connection to server CLOSED.\n");
writeReady = false;
inputNeeded = false; inputNeeded = false;
keepGoing = false; keepGoing = false;
break; break;
} }
case NORM_SOCKET_NONE: case NORM_SOCKET_NONE:
@ -273,7 +386,7 @@ int main(int argc, char* argv[])
{ {
fprintf(stderr, "normClient: NormGetSocketEvent() returned false\n"); fprintf(stderr, "normClient: NormGetSocketEvent() returned false\n");
} }
} // end if FD_ISSET(normfd) } // end if normEventPending
// If we have data in our inputBuffer and the NormSocket is "writeReady", then send it // If we have data in our inputBuffer and the NormSocket is "writeReady", then send it
if (writeReady && (inputLength > 0)) if (writeReady && (inputLength > 0))
@ -293,4 +406,16 @@ int main(int argc, char* argv[])
} }
} }
} // end while(keepGoing) } // end while(keepGoing)
NormClose(normSocket);
#ifdef WIN32
inputHandler.Close();
#else
if ((stdin != inputFile) && (NULL != inputFile))
{
fclose(inputFile);
inputFile = NULL;
}
#endif // if/else WIN32
fprintf(stderr, "normClient: Done.\n");
return 0;
} }

3
examples/normFileRecv.cpp
View File

@ -58,6 +58,9 @@ int main(int argc, char* argv[])
6003, 6003,
NORM_NODE_ANY); NORM_NODE_ANY);
NormSetRxPortReuse(session, true);
NormSetMulticastLoopback(session, true);
// NOTE: These are debugging routines available // NOTE: These are debugging routines available
// (not necessary for normal app use) // (not necessary for normal app use)
// (Need to include "protolib/common/protoDebug.h" for this // (Need to include "protolib/common/protoDebug.h" for this

5
examples/normFileSend.cpp
View File

@ -64,7 +64,10 @@ int main(int argc, char* argv[])
NormSessionHandle session = NormCreateSession(instance, NormSessionHandle session = NormCreateSession(instance,
"224.1.2.3", "224.1.2.3",
6003, 6003,
NORM_NODE_ANY); 1);
NormSetRxPortReuse(session, true);
NormSetMulticastLoopback(session, true);
// NOTE: These are some debugging routines available // NOTE: These are some debugging routines available
// (not necessary for normal app use) // (not necessary for normal app use)

538
examples/normMsgr.cpp
View File

@ -1,8 +1,6 @@
#include "normApi.h" #include "normApi.h"
#include <unistd.h> // for read() and write()
#include "protoCheck.h"
#include <stdio.h> // for printf(), etc #include <stdio.h> // for printf(), etc
#include <stdlib.h> // for srand() #include <stdlib.h> // for srand()
#include <string.h> // for strrchr(), memset(), etc #include <string.h> // for strrchr(), memset(), etc
@ -12,6 +10,10 @@
#include <errno.h> // obvious child #include <errno.h> // obvious child
#include <assert.h> // embarrassingly obvious #include <assert.h> // embarrassingly obvious
#define SHOOT_NOW 1
#include "protoCheck.h"
class NormMsgr class NormMsgr
{ {
public: public:
@ -87,12 +89,18 @@ class NormMsgr
void Destroy(); void Destroy();
unsigned int GetCount() const
{return msg_count;}
private: private:
Message* head; Message* head;
Message* tail; Message* tail;
unsigned int msg_count;
}; // end class NormMsgr::MessageQueue }; // end class NormMsgr::MessageQueue
Message* NewMessage(unsigned int size); Message* NewMessage(unsigned int size);
void Destroy();
bool OpenNormSession(NormInstanceHandle instance, bool OpenNormSession(NormInstanceHandle instance,
const char* addr, const char* addr,
unsigned short port, unsigned short port,
@ -110,6 +118,12 @@ class NormMsgr
assert(NORM_SESSION_INVALID != norm_session); assert(NORM_SESSION_INVALID != norm_session);
NormSetMulticastInterface(norm_session, ifaceName); NormSetMulticastInterface(norm_session, ifaceName);
} }
void SetLoopback(bool state)
{
loopback = state;
if (NORM_SESSION_INVALID != norm_session)
NormSetMulticastLoopback(norm_session, state);
}
void SetNormMessageTrace(bool state) void SetNormMessageTrace(bool state)
{ {
assert(NORM_SESSION_INVALID != norm_session); assert(NORM_SESSION_INVALID != norm_session);
@ -121,36 +135,57 @@ class NormMsgr
NormAddAckingNode(norm_session, ackId); NormAddAckingNode(norm_session, ackId);
norm_acking = true; // invoke ack-based flow control norm_acking = true; // invoke ack-based flow control
} }
void SetFlushing(bool state)
{norm_flushing = state;}
bool Start(bool sender, bool receiver); bool Start(bool sender, bool receiver);
void Stop() void Stop()
{is_running = false;} {is_running = false;}
bool IsRunning() const bool IsRunning() const
{return is_running;} {return is_running;}
unsigned long GetSentCount()
{return sent_count;}
void HandleNormEvent(const NormEvent& event); void HandleNormEvent(const NormEvent& event);
// Sender methods // Sender methods
FILE* GetInputFile() const FILE* GetInputFile() const
{return input_file;} {return input_file;}
int GetInputDescriptor() const
{return input_fd;}
bool InputReady() const
{return input_ready;}
void SetInputReady()
{input_ready = true;}
bool InputNeeded() const bool InputNeeded() const
{return input_needed;} {return input_needed;}
bool InputMessageReady() const bool InputMessageReady() const
{return ((NULL != input_msg) && !input_needed);} {return ((NULL != input_msg) && !input_needed);}
bool ReadInput(); bool ReadInput();
bool NormTxReady() const; bool TxReady() const;
bool SendMessage(); bool SendMessage();
bool EnqueueMessageObject(); bool EnqueueMessageObject();
// Receiver methods // Receiver methods
void SetOutputFile(FILE* filePtr)
{
output_file = filePtr;
output_fd = fileno(filePtr);
}
FILE* GetOutputFile() const FILE* GetOutputFile() const
{return output_file;} {return output_file;}
int GetOutputDescriptor() const
{return output_fd;}
void SetOutputReady() void SetOutputReady()
{output_ready = true;} {output_ready = true;}
bool OutputReady() const bool OutputReady() const
{return output_ready;} {return output_ready;}
bool OutputPending() const bool OutputPending() const
{return output_pending;} {return (NULL != output_msg);}
bool RxNeeded() const
{return rx_needed;}
bool WriteOutput(); bool WriteOutput();
void OmitHeader(bool state) void OmitHeader(bool state)
@ -165,23 +200,33 @@ class NormMsgr
private: private:
bool is_running; bool is_running;
FILE* input_file; // stdin by default FILE* input_file; // stdin by default
int input_fd;
bool input_ready;
bool input_needed; // bool input_needed; //
bool input_finished;
Message* input_msg; // current input message being read/sent* Message* input_msg; // current input message being read/sent*
MessageQueue input_msg_list; // list of enqueued messages (in norm sender cache) MessageQueue input_msg_list; // list of enqueued messages (in norm sender cache)
NormSessionHandle norm_session; NormSessionHandle norm_session;
bool is_multicast; bool is_multicast;
bool loopback;
unsigned int norm_tx_queue_max; // max number of objects that can be enqueued at once unsigned int norm_tx_queue_max; // max number of objects that can be enqueued at once
unsigned int norm_tx_queue_count; // count of unacknowledged enqueued objects (TBD - optionally track size too) unsigned int norm_tx_queue_count; // count of unacknowledged enqueued objects (TBD - optionally track size too)
bool norm_tx_watermark_pending; bool norm_flow_control_pending;
bool norm_tx_vacancy; bool norm_tx_vacancy;
bool norm_acking; bool norm_acking;
bool norm_flushing;
NormObjectHandle norm_flush_object;
NormObjectHandle norm_last_object;
unsigned long sent_count;
bool rx_needed;
FILE* output_file; FILE* output_file;
int output_fd;
bool output_ready; bool output_ready;
bool output_pending;
Message* output_msg; Message* output_msg;
MessageQueue output_msg_queue; MessageQueue output_msg_queue;
MessageQueue temp_msg_queue;
// These are some options mainly for testing purposes // These are some options mainly for testing purposes
bool omit_header; // if "true", receive message length header is _not_ written to output bool omit_header; // if "true", receive message length header is _not_ written to output
bool rx_silent; bool rx_silent;
@ -190,22 +235,39 @@ class NormMsgr
}; // end class NormMsgr }; // end class NormMsgr
NormMsgr::NormMsgr() NormMsgr::NormMsgr()
: input_file(stdin), input_needed(false), input_msg(NULL), : input_file(stdin), input_fd(fileno(stdin)), input_ready(false), input_needed(false), input_finished(false),
norm_session(NORM_SESSION_INVALID), is_multicast(false), norm_tx_queue_max(2048), norm_tx_queue_count(0), input_msg(NULL), norm_session(NORM_SESSION_INVALID), is_multicast(false), loopback(false),
norm_tx_watermark_pending(false), norm_tx_vacancy(true), norm_acking(false), norm_tx_queue_max(8192), norm_tx_queue_count(0),
output_file(stdout), output_ready(true), output_pending(false), output_msg(NULL), norm_flow_control_pending(false), norm_tx_vacancy(true), norm_acking(false),
norm_flushing(true), norm_flush_object(NORM_OBJECT_INVALID), norm_last_object(NORM_OBJECT_INVALID),
sent_count(0), rx_needed(false), output_file(stdout), output_fd(fileno(stdout)),
output_ready(true), output_msg(NULL),
omit_header(false), rx_silent(false), tx_loss(0.0) omit_header(false), rx_silent(false), tx_loss(0.0)
{ {
} }
NormMsgr::~NormMsgr() NormMsgr::~NormMsgr()
{ {
if (NULL != input_msg) delete input_msg; Destroy();
}
void NormMsgr::Destroy()
{
if (NULL != input_msg)
{
delete input_msg;
input_msg = NULL;
}
input_msg_list.Destroy(); input_msg_list.Destroy();
if (NULL != output_msg) delete output_msg; if (NULL != output_msg)
{
delete output_msg;
delete output_msg;
}
output_msg_queue.Destroy(); output_msg_queue.Destroy();
} }
bool NormMsgr::OpenNormSession(NormInstanceHandle instance, const char* addr, unsigned short port, NormNodeId nodeId) bool NormMsgr::OpenNormSession(NormInstanceHandle instance, const char* addr, unsigned short port, NormNodeId nodeId)
{ {
if (NormIsUnicastAddress(addr)) if (NormIsUnicastAddress(addr))
@ -222,21 +284,24 @@ bool NormMsgr::OpenNormSession(NormInstanceHandle instance, const char* addr, un
if (is_multicast) if (is_multicast)
{ {
NormSetRxPortReuse(norm_session, true); NormSetRxPortReuse(norm_session, true);
// TBD - make full loopback a command line option? if (loopback)
NormSetMulticastLoopback(norm_session, true); NormSetMulticastLoopback(norm_session, true);
} }
// Set some default parameters (maybe we should put parameter setting in Start()) // Set some default parameters (maybe we should put parameter setting in Start())
fprintf(stderr, "setting rx cache limit to %u\n", norm_tx_queue_max);
if (norm_tx_queue_max > 65535/2) norm_tx_queue_max = 65535/2;
NormSetRxCacheLimit(norm_session, norm_tx_queue_max); NormSetRxCacheLimit(norm_session, norm_tx_queue_max);
NormSetDefaultSyncPolicy(norm_session, NORM_SYNC_ALL); NormSetDefaultSyncPolicy(norm_session, NORM_SYNC_ALL);
NormSetDefaultUnicastNack(norm_session, true); if (!is_multicast)
NormSetDefaultUnicastNack(norm_session, true);
NormSetTxCacheBounds(norm_session, 10*1024*1024, norm_tx_queue_max, norm_tx_queue_max); NormSetTxCacheBounds(norm_session, 10*1024*1024, norm_tx_queue_max, norm_tx_queue_max);
//NormSetMessageTrace(norm_session, true); //NormSetMessageTrace(norm_session, true);
NormSetTxRobustFactor(norm_session, 2); //NormSetTxRobustFactor(norm_session, 20);
return true; return true;
} // end NormMsgr::OpenNormSession() } // end NormMsgr::OpenNormSession()
@ -274,13 +339,27 @@ void NormMsgr::SetNormCongestionControl(CCMode ccMode)
bool NormMsgr::Start(bool sender, bool receiver) bool NormMsgr::Start(bool sender, bool receiver)
{ {
fprintf(stderr, "enter NormMsgr::Start() ...\n");
// TBD - make these command-line accessible
unsigned int bufferSize = 64*1024*1024;
unsigned int segmentSize = 1400;
unsigned int blockSize = 64;
unsigned int numParity = 0;
unsigned int txSockBufferSize = 4*1024*1024;
unsigned int rxSockBufferSize = 6*1024*1024;
if (receiver) if (receiver)
{ {
if (!NormStartReceiver(norm_session, 10*1024*1024)) if (!NormStartReceiver(norm_session, bufferSize))
{ {
fprintf(stderr, "normMsgr error: unable to start NORM receiver\n"); fprintf(stderr, "normMsgr error: unable to start NORM receiver\n");
return false; return false;
} }
// Note: NormPreallocateRemoteSender() MUST be called AFTER NormStartReceiver()
NormPreallocateRemoteSender(norm_session, bufferSize, segmentSize, blockSize, numParity, bufferSize);
NormSetRxSocketBuffer(norm_session, rxSockBufferSize);
rx_needed = true;
fprintf(stderr, "normMsgr: receiver ready.\n");
} }
if (sender) if (sender)
{ {
@ -290,19 +369,26 @@ bool NormMsgr::Start(bool sender, bool receiver)
// so disable timer-based flow control // so disable timer-based flow control
NormSetFlowControl(norm_session, 0.0); NormSetFlowControl(norm_session, 0.0);
} }
NormSetGrttEstimate(norm_session, 0.001);
//NormSetGrttMax(norm_session, 0.100);
NormSetBackoffFactor(norm_session, 0);
// Pick a random instance id for now // Pick a random instance id for now
struct timeval currentTime; struct timeval currentTime;
gettimeofday(&currentTime, NULL); gettimeofday(&currentTime, NULL);
srand(currentTime.tv_usec); // seed random number generator srand(currentTime.tv_usec); // seed random number generator
NormSessionId instanceId = (NormSessionId)rand(); NormSessionId instanceId = (NormSessionId)rand();
if (!NormStartSender(norm_session, instanceId, 10*1024*1024, 1400, 16, 4)) if (!NormStartSender(norm_session, instanceId, bufferSize, segmentSize, blockSize, numParity))
{ {
fprintf(stderr, "normMsgr error: unable to start NORM sender\n"); fprintf(stderr, "normMsgr error: unable to start NORM sender\n");
if (receiver) NormStopReceiver(norm_session); if (receiver) NormStopReceiver(norm_session);
return false; return false;
} }
//NormSetAutoParity(norm_session, 2);
NormSetTxSocketBuffer(norm_session, txSockBufferSize);
input_needed = true; input_needed = true;
} }
//ProtoCheckResetLogging();
is_running = true; is_running = true;
return true; return true;
} // end NormMsgr::Start(); } // end NormMsgr::Start();
@ -337,11 +423,11 @@ bool NormMsgr::ReadInput()
readLength = input_msg->GetSize() - offset; readLength = input_msg->GetSize() - offset;
bufferPtr = input_msg->AccessBuffer() + offset; bufferPtr = input_msg->AccessBuffer() + offset;
} }
size_t result = fread(bufferPtr, 1, readLength, input_file); ssize_t result = read(input_fd, bufferPtr, readLength);
if (result > 0) if (result > 0)
{ {
input_msg->IncrementIndex(result); input_msg->IncrementIndex(result);
if (result < readLength) if ((size_t)result < readLength)
{ {
// Still need more input // Still need more input
// (wait for next input notification to read more) // (wait for next input notification to read more)
@ -368,27 +454,37 @@ bool NormMsgr::ReadInput()
input_needed = false; input_needed = false;
} }
} }
else else if (0 == result)
{ {
if (feof(input_file)) // end-of-file reached, TBD - trigger final flushing and wrap-up
fprintf(stderr, "normMsgr: input end-of-file detected (last:%p)...\n", norm_last_object);
delete input_msg;
input_msg = NULL;
input_ready = false;
input_needed = false;
input_finished = true;
if (norm_acking)
{ {
// end-of-file reached, TBD - trigger final flushing and wrap-up if (NORM_OBJECT_INVALID == norm_last_object)
fprintf(stderr, "normMsgr: input end-of-file detected ...\n"); is_running = false; // everything sent and acked
else if (!norm_flushing)
NormSetWatermark(norm_session, norm_last_object, true);
} }
else if (ferror(input_file)) }
else // result < 0
{
switch (errno)
{ {
switch (errno) case EINTR:
{ continue; // interupted, try again
case EINTR: case EAGAIN:
continue; // interupted, try again // input starved, wait for next notification
case EAGAIN: break;
// input starved, wait for next notification default:
break; perror("normMsgr error reading input");
default: break;
perror("normMsgr error reading input");
break;
}
} }
input_ready = false;
return false; return false;
} }
} }
@ -397,7 +493,7 @@ bool NormMsgr::ReadInput()
bool NormMsgr::WriteOutput() bool NormMsgr::WriteOutput()
{ {
while (output_pending) while (NULL != output_msg)
{ {
size_t writeLength; size_t writeLength;
const char* bufferPtr; const char* bufferPtr;
@ -412,38 +508,40 @@ bool NormMsgr::WriteOutput()
writeLength = output_msg->GetSize() - offset; writeLength = output_msg->GetSize() - offset;
bufferPtr = output_msg->GetBuffer() + offset; bufferPtr = output_msg->GetBuffer() + offset;
} }
size_t result = fwrite(bufferPtr, 1, writeLength, output_file); ssize_t result = write(output_fd, bufferPtr, writeLength);
if (result >= 0)
output_msg->IncrementIndex(result);
if (result < writeLength)
{ {
if (feof(output_file)) output_msg->IncrementIndex(result);
if ((size_t)result < writeLength)
output_ready = false; // blocked, wait for output notification
}
else
{
switch (errno)
{ {
// end-of-file reached, TBD - stop acting as receiver, signal sender we're done? case EINTR:
fprintf(stderr, "normMsgr: output end-of-file detected ...\n"); continue; // interupted, try again
} case EAGAIN:
else if (ferror(input_file)) // input starved, wait for next notification
{ output_ready = false;
fprintf(stderr, "normMsgr: output error detected ...\n"); break;
switch (errno) default:
{ perror("normMsgr error writing output");
case EINTR: break;
continue; // interupted, try again
case EAGAIN:
// input starved, wait for next notification
output_ready = false;
break;
default:
perror("normMsgr error writing output");
break;
}
} }
return false; return false;
} }
if (output_msg->IsComplete()) if (output_msg->IsComplete())
{ {
fflush(output_file); fflush(output_file);
delete output_msg; //delete output_msg;
temp_msg_queue.Append(*output_msg); // cache for debugging purposes
if (temp_msg_queue.GetCount() > 32)
{
//fprintf(stderr, "deleting cached recv'd message ...\n");
delete temp_msg_queue.RemoveHead();
}
output_msg = output_msg_queue.RemoveHead(); output_msg = output_msg_queue.RemoveHead();
if (NULL != output_msg) if (NULL != output_msg)
{ {
@ -454,7 +552,7 @@ bool NormMsgr::WriteOutput()
} }
else else
{ {
output_pending = false; rx_needed = true;
} }
break; break;
} }
@ -462,7 +560,7 @@ bool NormMsgr::WriteOutput()
return true; return true;
} // end NormMsgr::WriteOutput() } // end NormMsgr::WriteOutput()
bool NormMsgr::NormTxReady() const bool NormMsgr::TxReady() const
{ {
// This returns true if new tx data can be enqueued to NORM // This returns true if new tx data can be enqueued to NORM
// This is based on the state with respect to prior successful data // This is based on the state with respect to prior successful data
@ -484,8 +582,6 @@ bool NormMsgr::NormTxReady() const
bool NormMsgr::SendMessage() bool NormMsgr::SendMessage()
{ {
// TBD - call EnqueueMessageObject() or WriteMessageStream()
// depending upon on configured mode
if (EnqueueMessageObject()) if (EnqueueMessageObject())
{ {
// Our buffered message was sent, so reset input indices // Our buffered message was sent, so reset input indices
@ -493,6 +589,7 @@ bool NormMsgr::SendMessage()
input_msg_list.Append(*input_msg); input_msg_list.Append(*input_msg);
input_msg = NULL; input_msg = NULL;
input_needed = true; input_needed = true;
sent_count++;
return true; return true;
} }
// else will be prompted to retry by NORM event (queue vacancy, watermark completion) // else will be prompted to retry by NORM event (queue vacancy, watermark completion)
@ -515,23 +612,42 @@ bool NormMsgr::EnqueueMessageObject()
// This might happen if a non-acking receiver is present and // This might happen if a non-acking receiver is present and
// has nacked for the oldest object in the queue even if all // has nacked for the oldest object in the queue even if all
// of our acking receivers have acknowledged it. // of our acking receivers have acknowledged it.
fprintf(stderr, "NO VACANCY count:%u max:%u\n", norm_tx_queue_count, norm_tx_queue_max);
norm_tx_vacancy = false; norm_tx_vacancy = false;
return false; return false;
} }
//NormObjectRetain(object);
NormObjectSetUserData(object, input_msg); // so we can remove/delete upon purge NormObjectSetUserData(object, input_msg); // so we can remove/delete upon purge
if (norm_acking) if (norm_acking)
{ {
// ack-based flow control has been enabled // ack-based flow control has been enabled
norm_tx_queue_count++; norm_tx_queue_count++;
if (!norm_tx_watermark_pending && (norm_tx_queue_count >= (norm_tx_queue_max / 2))) if (!norm_flow_control_pending && (norm_tx_queue_count >= (norm_tx_queue_max / 2)))
{ {
NormSetWatermark(norm_session, object); NormSetWatermark(norm_session, object, true); // overrideFlush == true
norm_tx_watermark_pending = true; norm_last_object = object;
norm_flow_control_pending = true;
}
else if (norm_flushing) // per-message acking
{
#ifdef SHOOT_FIRST
NormSetWatermark(norm_session, object, true);
norm_last_object = object;
#else // ACK_LATER
if (norm_flow_control_pending)
{
norm_flush_object = object; // will be used as watermark upon flow control ack
}
else
{
NormSetWatermark(norm_session, object, true);
norm_last_object = object;
}
#endif // SHOOT_FIRST/ACK_LATER
} }
else else
{ {
// TBD - make non-flow control acking separable option? //norm_last_object = object;
NormSetWatermark(norm_session, object);
} }
} }
return true; return true;
@ -539,7 +655,6 @@ bool NormMsgr::EnqueueMessageObject()
void NormMsgr::HandleNormEvent(const NormEvent& event) void NormMsgr::HandleNormEvent(const NormEvent& event)
{ {
bool logAllocs = false;
switch (event.type) switch (event.type)
{ {
case NORM_TX_QUEUE_EMPTY: case NORM_TX_QUEUE_EMPTY:
@ -555,18 +670,37 @@ void NormMsgr::HandleNormEvent(const NormEvent& event)
if (NORM_ACK_SUCCESS == NormGetAckingStatus(norm_session)) if (NORM_ACK_SUCCESS == NormGetAckingStatus(norm_session))
{ {
//fprintf(stderr, "WATERMARK COMPLETED\n"); //fprintf(stderr, "WATERMARK COMPLETED\n");
if (norm_tx_watermark_pending) norm_last_object = NORM_OBJECT_INVALID;
if (norm_flow_control_pending)
{ {
norm_tx_queue_count -= (norm_tx_queue_max / 2); norm_tx_queue_count -= (norm_tx_queue_max / 2);
norm_tx_watermark_pending = false; norm_flow_control_pending = false;
if (NORM_OBJECT_INVALID != norm_flush_object)
{
NormSetWatermark(norm_session, norm_flush_object, true);
norm_last_object = norm_flush_object;
norm_flush_object = NORM_OBJECT_INVALID;
}
} }
if (input_finished && (NORM_OBJECT_INVALID == norm_last_object))
is_running = false;
} }
else else
{ {
// TBD - we could see who didn't ACK and possibly remove them // TBD - we could see who did and how didn't ACK and possibly remove them
// from our acking list. For now, we are infinitely // from our acking "membership". For now, we are infinitely
// persistent by resetting watermark ack request // persistent by resetting watermark ack request without clearing
NormResetWatermark(norm_session); // flow control
if (NORM_OBJECT_INVALID == norm_flush_object)
{
NormResetWatermark(norm_session);
}
else // might as request ack for most recent enqueued object
{
NormSetWatermark(norm_session, norm_flush_object, true);
norm_last_object = norm_flush_object;
norm_flush_object = NORM_OBJECT_INVALID;
}
} }
break; break;
@ -574,11 +708,16 @@ void NormMsgr::HandleNormEvent(const NormEvent& event)
{ {
NormDataDetachData(event.object); NormDataDetachData(event.object);
Message* msg = (Message*)NormObjectGetUserData(event.object); Message* msg = (Message*)NormObjectGetUserData(event.object);
if(event.object == norm_flush_object)
norm_flush_object = NORM_OBJECT_INVALID;
if (NULL != msg) if (NULL != msg)
{ {
input_msg_list.Remove(*msg); input_msg_list.Remove(*msg);
delete msg; delete msg;
} }
//NormObjectRelease(event.object);
//fprintf(stderr, "normMsgr LOGGING ALLOCATIONS\n");
//ProtoCheckLogAllocations(stderr);
break; break;
} }
@ -590,11 +729,11 @@ void NormMsgr::HandleNormEvent(const NormEvent& event)
case NORM_RX_OBJECT_ABORTED: case NORM_RX_OBJECT_ABORTED:
//fprintf(stderr, "NORM_RX_OBJECT_ABORTED\n");// %hu\n", NormObjectGetTransportId(event.object)); //fprintf(stderr, "NORM_RX_OBJECT_ABORTED\n");// %hu\n", NormObjectGetTransportId(event.object));
logAllocs = true;
break; break;
case NORM_RX_OBJECT_COMPLETED: case NORM_RX_OBJECT_COMPLETED:
{ {
sent_count++;
char* data = NormDataDetachData(event.object); char* data = NormDataDetachData(event.object);
if (NULL != data) if (NULL != data)
{ {
@ -613,12 +752,12 @@ void NormMsgr::HandleNormEvent(const NormEvent& event)
output_msg->ResetIndex(MSG_HEADER_SIZE); output_msg->ResetIndex(MSG_HEADER_SIZE);
else else
output_msg->ResetIndex(); output_msg->ResetIndex();
output_pending = true;
} }
else else
{ {
output_msg_queue.Append(*msg); output_msg_queue.Append(*msg);
} }
rx_needed = false;
} }
// else TBD - "termination" info-only object? // else TBD - "termination" info-only object?
break; break;
@ -629,13 +768,6 @@ void NormMsgr::HandleNormEvent(const NormEvent& event)
} }
//NormReleasePreviousEvent(NormGetInstance(norm_session)); //NormReleasePreviousEvent(NormGetInstance(norm_session));
if (logAllocs)
{
#ifdef USE_PROTO_CHECK
ProtoCheckLogAllocations(stderr);
#endif // USE_PROTO_CHECK
}
} // end NormMsgr::HandleNormEvent() } // end NormMsgr::HandleNormEvent()
@ -643,7 +775,7 @@ void Usage()
{ {
fprintf(stderr, "Usage: normMsgr id <nodeId> {send &| recv} [addr <addr>[/<port>]][ack <node1>[,<node2>,...]\n" fprintf(stderr, "Usage: normMsgr id <nodeId> {send &| recv} [addr <addr>[/<port>]][ack <node1>[,<node2>,...]\n"
" [cc|cce|ccl|rate <bitsPerSecond>][interface <name>][debug <level>][trace]\n" " [cc|cce|ccl|rate <bitsPerSecond>][interface <name>][debug <level>][trace]\n"
" [omit][silent][txloss <lossFraction>]\n"); " [flush {none|active}][omit][silent][txloss <lossFraction>]\n");
} }
int main(int argc, char* argv[]) int main(int argc, char* argv[])
{ {
@ -658,16 +790,21 @@ int main(int argc, char* argv[])
NormNodeId ackingNodeList[256]; NormNodeId ackingNodeList[256];
unsigned int ackingNodeCount = 0; unsigned int ackingNodeCount = 0;
bool flushing = false;
double txRate = 0.0; // used for non-default NORM_FIXED ccMode double txRate = 0.0; // used for non-default NORM_FIXED ccMode
NormMsgr::CCMode ccMode = NormMsgr::NORM_CC; NormMsgr::CCMode ccMode = NormMsgr::NORM_CC;
const char* mcastIface = NULL; const char* mcastIface = NULL;
int debugLevel = 0; int debugLevel = 0;
const char* debugLog = NULL; // stderr by default
bool trace = false; bool trace = false;
bool omitHeaderOnOutput = false; bool omitHeaderOnOutput = false;
bool silentReceiver = false; bool silentReceiver = false;
double txloss = 0.0; double txloss = 0.0;
bool loopback = false;
NormMsgr normMsgr;
// Parse command-line // Parse command-line
int i = 1; int i = 1;
@ -683,6 +820,10 @@ int main(int argc, char* argv[])
{ {
recv = true; recv = true;
} }
else if (0 == strncmp(cmd, "loopback", len))
{
loopback = true;
}
else if (0 == strncmp(cmd, "addr", len)) else if (0 == strncmp(cmd, "addr", len))
{ {
if (i >= argc) if (i >= argc)
@ -708,6 +849,23 @@ int main(int argc, char* argv[])
sessionPort = atoi(portPtr); sessionPort = atoi(portPtr);
} }
} }
else if (0 == strncmp(cmd, "output", len))
{
if (i >= argc)
{
fprintf(stderr, "normMsgr error: missing output 'device' name!\n");
Usage();
return -1;
}
FILE* outfile = fopen(argv[i++], "w+");
if (NULL == outfile)
{
perror("normMsgr output device fopen() error");
Usage();
return -1;
}
normMsgr.SetOutputFile(outfile);
}
else if (0 == strncmp(cmd, "id", len)) else if (0 == strncmp(cmd, "id", len))
{ {
if (i >= argc) if (i >= argc)
@ -731,17 +889,47 @@ int main(int argc, char* argv[])
while ((NULL != alist) && (*alist != '\0')) while ((NULL != alist) && (*alist != '\0'))
{ {
// TBD - Do we need to skip leading white space? // TBD - Do we need to skip leading white space?
if (1 != sscanf(alist, "%d", ackingNodeList + ackingNodeCount)) int id;
if (1 != sscanf(alist, "%d", &id))
{ {
fprintf(stderr, "nodeMsgr error: invalid acking node list!\n"); fprintf(stderr, "nodeMsgr error: invalid acking node list!\n");
Usage(); Usage();
return -1; return -1;
} }
ackingNodeList[ackingNodeCount] = NormNodeId(id);
ackingNodeCount++; ackingNodeCount++;
alist = strchr(alist, ','); alist = strchr(alist, ',');
if (NULL != alist) alist++; // point past comma if (NULL != alist) alist++; // point past comma
} }
} }
else if (0 == strncmp(cmd, "flush", len))
{
// "none", "passive", or "active"
if (i >= argc)
{
fprintf(stderr, "nodeMsgr error: missing 'flush' <mode>!\n");
Usage();
return -1;
}
const char* mode = argv[i++];
if (0 == strcmp(mode, "none"))
{
flushing = false;
}
else if (0 == strcmp(mode, "passive"))
{
flushing = false;
}
else if (0 == strcmp(mode, "active"))
{
flushing = true;
}
else
{
fprintf(stderr, "normMsgr error: invalid 'flush' mode \"%s\"\n", mode);
return -1;
}
}
else if (0 == strncmp(cmd, "rate", len)) else if (0 == strncmp(cmd, "rate", len))
{ {
if (i >= argc) if (i >= argc)
@ -802,12 +990,22 @@ int main(int argc, char* argv[])
{ {
if (i >= argc) if (i >= argc)
{ {
fprintf(stderr, "nodeMsgr error: missing 'interface' <name>!\n"); fprintf(stderr, "nodeMsgr error: missing 'debug' <level>!\n");
Usage(); Usage();
return -1; return -1;
} }
debugLevel = atoi(argv[i++]); debugLevel = atoi(argv[i++]);
} }
else if (0 == strncmp(cmd, "log", len))
{
if (i >= argc)
{
fprintf(stderr, "nodeMsgr error: missing 'log' <fileName>!\n");
Usage();
return -1;
}
debugLog = argv[i++];
}
else if (0 == strncmp(cmd, "trace", len)) else if (0 == strncmp(cmd, "trace", len))
{ {
trace = true; trace = true;
@ -841,8 +1039,11 @@ int main(int argc, char* argv[])
// TBD - should provide more error checking of calls // TBD - should provide more error checking of calls
NormInstanceHandle normInstance = NormCreateInstance(); NormInstanceHandle normInstance = NormCreateInstance();
NormSetDebugLevel(debugLevel); NormSetDebugLevel(debugLevel);
if (NULL != debugLog)
NormOpenDebugLog(normInstance, debugLog);
NormMsgr normMsgr; normMsgr.SetLoopback(loopback);
normMsgr.SetFlushing(flushing);
if (omitHeaderOnOutput) normMsgr.OmitHeader(true); if (omitHeaderOnOutput) normMsgr.OmitHeader(true);
@ -873,89 +1074,134 @@ int main(int argc, char* argv[])
int normfd = NormGetDescriptor(normInstance); int normfd = NormGetDescriptor(normInstance);
// Get input/output descriptors and set to non-blocking i/o // Get input/output descriptors and set to non-blocking i/o
int inputfd = fileno(normMsgr.GetInputFile()); int inputfd = normMsgr.GetInputDescriptor();
if (-1 == fcntl(inputfd, F_SETFL, fcntl(inputfd, F_GETFL, 0) | O_NONBLOCK)) if (-1 == fcntl(inputfd, F_SETFL, fcntl(inputfd, F_GETFL, 0) | O_NONBLOCK))
perror("normMsgr: fcntl(inputfd, O_NONBLOCK) error"); perror("normMsgr: fcntl(inputfd, O_NONBLOCK) error");
int outputfd = fileno(normMsgr.GetOutputFile()); int outputfd = normMsgr.GetOutputDescriptor();
if (-1 == fcntl(outputfd, F_SETFL, fcntl(outputfd, F_GETFL, 0) | O_NONBLOCK)) if (-1 == fcntl(outputfd, F_SETFL, fcntl(outputfd, F_GETFL, 0) | O_NONBLOCK))
perror("normMsgr: fcntl(outputfd, O_NONBLOCK) error"); perror("normMsgr: fcntl(outputfd, O_NONBLOCK) error");
fd_set fdsetInput, fdsetOutput; fd_set fdsetInput, fdsetOutput;
FD_ZERO(&fdsetInput); FD_ZERO(&fdsetInput);
FD_ZERO(&fdsetOutput); FD_ZERO(&fdsetOutput);
struct timeval lastTime;
gettimeofday(&lastTime, NULL);
while (normMsgr.IsRunning()) while (normMsgr.IsRunning())
{ {
int maxfd = normfd; int maxfd = -1;
FD_SET(normfd, &fdsetInput); // Only wait on NORM if needed for tx readiness
if (normMsgr.InputNeeded()) bool waitOnNorm = true;
if (!(normMsgr.RxNeeded() || normMsgr.InputMessageReady()))
waitOnNorm = false; // no need to wait
else if (normMsgr.InputMessageReady() && normMsgr.TxReady())
waitOnNorm = false; // no need to wait if already tx ready
if (waitOnNorm)
{
maxfd = normfd;
FD_SET(normfd, &fdsetInput);
}
else
{
FD_CLR(normfd, &fdsetInput);
}
if (normMsgr.InputNeeded() && !normMsgr.InputReady())
{ {
//fprintf(stderr, "NEED INPUT ...\n");
FD_SET(inputfd, &fdsetInput); FD_SET(inputfd, &fdsetInput);
if (inputfd > maxfd) if (inputfd > maxfd) maxfd = inputfd;
maxfd = inputfd;
} }
else else
{ {
FD_CLR(inputfd, &fdsetInput); FD_CLR(inputfd, &fdsetInput);
} }
int result;
if (normMsgr.OutputPending() && !normMsgr.OutputReady()) if (normMsgr.OutputPending() && !normMsgr.OutputReady())
{ {
FD_SET(outputfd, &fdsetOutput); FD_SET(outputfd, &fdsetOutput);
if (outputfd > maxfd) maxfd = outputfd; if (outputfd > maxfd) maxfd = outputfd;
result = select(maxfd+1, &fdsetInput, &fdsetOutput, NULL, NULL);
} }
else else
{ {
FD_CLR(outputfd, &fdsetOutput); FD_CLR(outputfd, &fdsetOutput);
result = select(maxfd+1, &fdsetInput, NULL, NULL, NULL);
} }
switch (result) if (maxfd >= 0)
{ {
case -1: struct timeval timeout;
switch (errno) timeout.tv_sec = 1;
{ timeout.tv_usec = 0;
case EINTR: int result = select(maxfd+1, &fdsetInput, &fdsetOutput, NULL, &timeout);
case EAGAIN: switch (result)
continue; {
default: case -1:
perror("normMsgr select() error"); switch (errno)
// TBD - stop NormMsgr {
break; case EINTR:
} case EAGAIN:
break; continue;
case 0: default:
// shouldn't occur for now (no timeout) perror("normMsgr select() error");
continue; // TBD - stop NormMsgr
default: break;
if (FD_ISSET(inputfd, &fdsetInput)) }
{ break;
normMsgr.ReadInput(); case 0:
} // shouldn't occur for now (no timeout)
if (FD_ISSET(normfd, &fdsetInput)) //fprintf(stderr, "normMsgr timeout ...\n");
{ continue;
NormEvent event; default:
if (NormGetNextEvent(normInstance, &event)) if (FD_ISSET(inputfd, &fdsetInput))
normMsgr.HandleNormEvent(event); {
} normMsgr.SetInputReady();
if (FD_ISSET(outputfd, &fdsetOutput)) }
{ /*if (FD_ISSET(normfd, &fdsetInput))
normMsgr.SetOutputReady(); {
} NormEvent event;
break; while (NormGetNextEvent(normInstance, &event, false))
normMsgr.HandleNormEvent(event);
}*/
if (FD_ISSET(outputfd, &fdsetOutput))
{
normMsgr.SetOutputReady();
}
break;
}
} }
NormSuspendInstance(normInstance);
NormEvent event;
while (NormGetNextEvent(normInstance, &event, false))
normMsgr.HandleNormEvent(event);
// As a result of reading input or NORM notification events, // As a result of reading input or NORM notification events,
// we may be ready to send a message if it's been read // we may be ready to read input and/or send a message if it's been read
if (normMsgr.InputMessageReady() && normMsgr.NormTxReady()) const int LOOP_MAX = 100;
normMsgr.SendMessage(); int loopCount = 0;
// and/or output a received message if we need while (loopCount < LOOP_MAX)
if (normMsgr.OutputPending() && normMsgr.OutputReady()) {
normMsgr.WriteOutput(); // TBD - implement output async i/o notification as needed loopCount++;
if (normMsgr.InputNeeded() && normMsgr.InputReady())
normMsgr.ReadInput();
if (normMsgr.InputMessageReady() && normMsgr.TxReady())
normMsgr.SendMessage();
// and/or output a received message if we need
if (normMsgr.OutputPending() && normMsgr.OutputReady())
normMsgr.WriteOutput();
}
NormResumeInstance(normInstance);
} // end while(normMsgr.IsRunning() } // end while(normMsgr.IsRunning()
fprintf(stderr, "normMsgr exiting ...\n");
NormCloseDebugLog(normInstance);
fprintf(stderr, "destroying session ...\n");
normMsgr.CloseNormSession();
fprintf(stderr, "destroying instance ...\n");
NormDestroyInstance(normInstance); NormDestroyInstance(normInstance);
normMsgr.Destroy();
fprintf(stderr, "normMsgr exiting ...\n");
} // end main() } // end main()
NormMsgr::Message::Message() NormMsgr::Message::Message()
@ -980,6 +1226,7 @@ NormMsgr::Message::~Message()
// to delete msg_buffer if it came from NORM // to delete msg_buffer if it came from NORM
if (NULL != msg_buffer) if (NULL != msg_buffer)
{ {
//fprintf(stderr, "deleting msg_buffer ...\n");
delete[] msg_buffer; delete[] msg_buffer;
msg_buffer = NULL; msg_buffer = NULL;
} }
@ -1012,7 +1259,7 @@ bool NormMsgr::Message::Init(unsigned int size)
} // end NormMsgr::Message::Init() } // end NormMsgr::Message::Init()
NormMsgr::MessageQueue::MessageQueue() NormMsgr::MessageQueue::MessageQueue()
: head(NULL), tail(NULL) : head(NULL), tail(NULL), msg_count(0)
{ {
} }
@ -1038,6 +1285,7 @@ void NormMsgr::MessageQueue::Prepend(Message& msg)
tail = &msg; tail = &msg;
msg.next = head; msg.next = head;
head = &msg; head = &msg;
msg_count++;
} // end NormMsgr::MessageQueue::Prepend() } // end NormMsgr::MessageQueue::Prepend()
void NormMsgr::MessageQueue::Append(Message& msg) void NormMsgr::MessageQueue::Append(Message& msg)
@ -1049,6 +1297,7 @@ void NormMsgr::MessageQueue::Append(Message& msg)
head = &msg; head = &msg;
msg.prev = tail; msg.prev = tail;
tail = &msg; tail = &msg;
msg_count++;
} // end NormMsgr::MessageQueue::Append() } // end NormMsgr::MessageQueue::Append()
void NormMsgr::MessageQueue::Remove(Message& msg) void NormMsgr::MessageQueue::Remove(Message& msg)
@ -1062,6 +1311,7 @@ void NormMsgr::MessageQueue::Remove(Message& msg)
else else
msg.next->prev = msg.prev; msg.next->prev = msg.prev;
msg.prev = msg.next = NULL; msg.prev = msg.next = NULL;
msg_count--;
} // end NormMsgr::MessageQueue::Remove() } // end NormMsgr::MessageQueue::Remove()
NormMsgr::Message* NormMsgr::MessageQueue::RemoveHead() NormMsgr::Message* NormMsgr::MessageQueue::RemoveHead()

420
examples/normServer.cpp
View File

@ -36,14 +36,27 @@
// socket" approach could be supported for unicast and SSM streams without too much // socket" approach could be supported for unicast and SSM streams without too much
// difficulty. // difficulty.
#include "normSocket.h"
#include <arpa/inet.h> // for inet_ntoa
#include <stdio.h> // for fprintf() #include <stdio.h> // for fprintf()
#include <string.h> // for memcmp() #include <string.h> // for memcmp()
#include <map> // for std::map<> #include <map> // for std::map<>
#include <assert.h> // more obvious
#include "normSocket.h"
#ifdef WIN32
#include "win32InputHandler.cpp" // to include full implementation
#include <Winsock2.h> // for inet_ntoa()
#include <Ws2tcpip.h> // for inet_ntop()
#else
#include <arpa/inet.h> // for inet_ntoa
#include <sys/select.h> #include <sys/select.h>
#include <fcntl.h> // for, well, fnctl() #include <fcntl.h> // for, well, fnctl()
#include <errno.h> // obvious child #include <errno.h> // obvious child
#endif // if/else WIN32/UNIX
void Usage()
{
fprintf(stderr, "Usage: normServer [listen [<groupAddr>/]<port>][debug <level>][trace]\n");
}
// Our "server" indexes clients by their source addr/port // Our "server" indexes clients by their source addr/port
class ClientInfo class ClientInfo
@ -58,11 +71,11 @@ class ClientInfo
UINT16 GetPort() const UINT16 GetPort() const
{return client_port;} {return client_port;}
const char* GetAddressString() const; const char* GetAddressString();
void Print(FILE* filePtr) const; void Print(FILE* filePtr);
private: private:
UINT8 addr_version; // 4 or 6 UINT8 addr_version; // 4 or 6
char client_addr[16]; // big enough for IPv6 char client_addr[16]; // big enough for IPv6
UINT16 client_port; UINT16 client_port;
@ -110,7 +123,7 @@ int ClientInfo::GetAddressFamily() const
} // end ClientInfo::GetAddressFamily() } // end ClientInfo::GetAddressFamily()
const char* ClientInfo::GetAddressString() const const char* ClientInfo::GetAddressString()
{ {
static char text[64]; static char text[64];
text[63] = '\0'; text[63] = '\0';
@ -123,7 +136,7 @@ const char* ClientInfo::GetAddressString() const
return text; return text;
} // end ClientInfo::GetAddressString() } // end ClientInfo::GetAddressString()
void ClientInfo::Print(FILE* filePtr) const void ClientInfo::Print(FILE* filePtr)
{ {
char text[64]; char text[64];
text[63] = '\0'; text[63] = '\0';
@ -136,9 +149,45 @@ void ClientInfo::Print(FILE* filePtr) const
fprintf(filePtr, "%s/%hu", text, client_port); fprintf(filePtr, "%s/%hu", text, client_port);
} // end ClientInfo::Print() } // end ClientInfo::Print()
class Client
{
public:
Client(NormSocketHandle clientSocket);
~Client();
NormSocketHandle GetSocket() const
{return client_socket;}
bool GetWriteReady() const
{return write_ready;}
void SetWriteReady(bool state)
{write_ready = state;}
unsigned int GetBytesWritten() const
{return bytes_written;}
void SetBytesWritten(unsigned long numBytes)
{bytes_written = numBytes;}
private:
NormSocketHandle client_socket;
// These are state variables for unicast server -> client communication
bool write_ready;
unsigned int bytes_written;
}; // end class Client
Client::Client(NormSocketHandle clientSocket)
: client_socket(clientSocket),
write_ready(true), bytes_written(0)
{
}
Client::~Client()
{
}
// C++ map used to index client sessions by the client source addr/port // C++ map used to index client sessions by the client source addr/port
typedef std::map<ClientInfo, NormSocketHandle> ClientMap; typedef std::map<ClientInfo, Client*> ClientMap;
ClientInfo NormGetClientInfo(NormNodeHandle client) ClientInfo NormGetClientInfo(NormNodeHandle client)
{ {
@ -146,32 +195,46 @@ ClientInfo NormGetClientInfo(NormNodeHandle client)
unsigned int addrLen = 16; unsigned int addrLen = 16;
UINT16 port; UINT16 port;
NormNodeGetAddress(client, addr, &addrLen, &port); NormNodeGetAddress(client, addr, &addrLen, &port);
int addrFamily;
UINT8 version; UINT8 version;
if (4 == addrLen) if (4 == addrLen)
{
addrFamily = AF_INET;
version = 4; version = 4;
}
else else
{
addrFamily = AF_INET6;
version = 6; version = 6;
}
return ClientInfo(version, addr, port); return ClientInfo(version, addr, port);
} // end NormGetClientInfo() } // end NormGetClientInfo(NormNodeHandle)
NormSocketHandle FindClientSocket(ClientMap& clientMap, const ClientInfo& clientInfo) ClientInfo NormGetSocketInfo(NormSocketHandle socket)
{
char addr[16]; // big enough for IPv6
unsigned int addrLen = 16;
UINT16 port;
NormGetPeerName(socket, addr, &addrLen, &port);
UINT8 version;
if (4 == addrLen)
version = 4;
else
version = 6;
return ClientInfo(version, addr, port);
} // end NormGetSocketInfo(NormSocketHandle)
Client* FindClient(ClientMap& clientMap, const ClientInfo& clientInfo)
{ {
ClientMap::iterator it = clientMap.find(clientInfo); ClientMap::iterator it = clientMap.find(clientInfo);
if (clientMap.end() != it) if (clientMap.end() != it)
return &(it->second); return it->second;
else else
return NULL;
} // end FindClient()
NormSocketHandle FindClientSocket(ClientMap& clientMap, const ClientInfo& clientInfo)
{
Client* client = FindClient(clientMap, clientInfo);
if (NULL == client)
return NORM_SOCKET_INVALID; return NORM_SOCKET_INVALID;
else
return client->GetSocket();
} // end FindClientSocket() } // end FindClientSocket()
int main(int argc, char* argv[]) int main(int argc, char* argv[])
{ {
ClientMap clientMap; ClientMap clientMap;
@ -185,7 +248,6 @@ int main(int argc, char* argv[])
bool trace = false; bool trace = false;
unsigned int debugLevel = 0; unsigned int debugLevel = 0;
for (int i = 1; i < argc; i++) for (int i = 1; i < argc; i++)
{ {
const char* cmd = argv[i]; const char* cmd = argv[i];
@ -210,11 +272,13 @@ int main(int argc, char* argv[])
if (1 != sscanf(portPtr, "%hu", &serverPort)) if (1 != sscanf(portPtr, "%hu", &serverPort))
{ {
fprintf(stderr, "normServer error: invalid <port> \"%s\"\n", portPtr); fprintf(stderr, "normServer error: invalid <port> \"%s\"\n", portPtr);
Usage();
return -1; return -1;
} }
} }
else if (0 == strncmp(cmd, "interface", len)) else if (0 == strncmp(cmd, "interface", len))
{ {
// Note the NormSocket code does not yet expose mcast interface
mcastInterface = argv[++i]; mcastInterface = argv[++i];
} }
else if (0 == strncmp(cmd, "trace", len)) else if (0 == strncmp(cmd, "trace", len))
@ -226,37 +290,42 @@ int main(int argc, char* argv[])
if (1 != sscanf(argv[++i], "%u", &debugLevel)) if (1 != sscanf(argv[++i], "%u", &debugLevel))
{ {
fprintf(stderr, "normServer error: invalid debug level\n"); fprintf(stderr, "normServer error: invalid debug level\n");
Usage();
return -1; return -1;
} }
} }
else else
{ {
fprintf(stderr, "normServer error: invalid command \"%s\"\n", cmd); fprintf(stderr, "normServer error: invalid command \"%s\"\n", cmd);
Usage();
return -1; return -1;
} }
} }
// For unicast operation in this demo app, the server only "talks back"
// to one client on a first come, first serve basis. (Multiple clients
// can connect and send data _to_ the server, but in this simple example,
// the server only sends to one at a time. For the multicast server case,
// the server multicasts to the entire group.
NormSocketHandle firstClientSocket = NORM_SOCKET_INVALID;
NormInstanceHandle instance = NormCreateInstance(); NormInstanceHandle instance = NormCreateInstance();
NormSocketHandle serverSocket = NormListen(instance, serverPort, groupAddrPtr); NormSocketHandle serverSocket = NormOpen(instance);
if (trace) NormSetMessageTrace(NormGetSession(serverSocket), true); NormListen(serverSocket, serverPort, groupAddrPtr);
if (trace) NormSetMessageTrace(NormGetSocketSession(serverSocket), true);
if (0 != debugLevel) NormSetDebugLevel(debugLevel); if (0 != debugLevel) NormSetDebugLevel(debugLevel);
//NormSetDebugLevel(8); //NormSetDebugLevel(8);
//NormSetMessageTrace(NormGetSession(serverSocket), true); //NormSetMessageTrace(NormGetSocketSession(serverSocket), true);
#ifdef WIN32
HANDLE hStdout = GetStdHandle(STD_OUTPUT_HANDLE);
Win32InputHandler inputHandler;
inputHandler.Open();
HANDLE handleArray[2];
handleArray[0] = NormGetDescriptor(instance);
handleArray[1] = inputHandler.GetEventHandle();
#else
// We use a select() call to multiplex input reading and NormSocket handling // We use a select() call to multiplex input reading and NormSocket handling
fd_set fdset; fd_set fdset;
FD_ZERO(&fdset); FD_ZERO(&fdset);
// Get our input (STDIN) descriptor and set non-blocking // Get our input (STDIN) descriptor and set non-blocking
FILE* inputFile = stdin; FILE* inputFile = stdin;
int inputfd = fileno(inputFile); int inputfd = fileno(inputFile);
@ -264,15 +333,48 @@ int main(int argc, char* argv[])
perror("normClient: fcntl(inputfd, O_NONBLOCK) error"); perror("normClient: fcntl(inputfd, O_NONBLOCK) error");
// Get our NormInstance descriptor // Get our NormInstance descriptor
int normfd = NormGetDescriptor(instance); int normfd = NormGetDescriptor(instance);
#endif // if/else WIN32/UNIX
bool keepGoing = true; bool keepGoing = true;
bool writeReady = false; bool writeReady = false;
unsigned int inputLength = 0; int inputLength = 0;
unsigned int bytesWritten = 0; unsigned int bytesWritten = 0;
const unsigned int BUFFER_LENGTH = 2048; const unsigned int BUFFER_LENGTH = 2048;
char inputBuffer[BUFFER_LENGTH]; char inputBuffer[BUFFER_LENGTH];
bool inputNeeded = false; // will be set to "true" upon CONNECT bool inputNeeded = false; // will be set to "true" upon CONNECT
bool inputClosed = false;
unsigned int clientCount = 0;
while (keepGoing) while (keepGoing)
{ {
bool normEventPending = false;
bool inputEventPending = false;
#ifdef WIN32
DWORD handleCount = inputNeeded ? 2 : 1;
DWORD waitStatus =
MsgWaitForMultipleObjectsEx(handleCount, // number of handles in array
handleArray, // object-handle array
INFINITE, // time-out interval
QS_ALLINPUT, // input-event type
0);
if ((WAIT_OBJECT_0 <= waitStatus) && (waitStatus < (WAIT_OBJECT_0 + handleCount)))
{
if (0 == (waitStatus - WAIT_OBJECT_0))
normEventPending = true;
else
inputEventPending = true;
}
else if (-1 == waitStatus)
{
perror("normServer: MsgWaitForMultipleObjectsEx() error");
break;
}
else
{
// TBD - any other status we should handle?
// (e.g. WAIT_TIMEOUT, WAIT_ABANDONED or WAIT_IO_COMPLETION)
continue; // ignore for now
}
#else
FD_SET(normfd, &fdset); FD_SET(normfd, &fdset);
int maxfd = normfd; int maxfd = normfd;
if (inputNeeded) if (inputNeeded)
@ -287,12 +389,33 @@ int main(int argc, char* argv[])
int result = select(maxfd+1, &fdset, NULL, NULL, NULL); int result = select(maxfd+1, &fdset, NULL, NULL, NULL);
if (result <= 0) if (result <= 0)
{ {
perror("normClient: select() error"); perror("normServer: select() error");
break; break;
} }
if (FD_ISSET(inputfd, &fdset)) if (FD_ISSET(inputfd, &fdset))
inputEventPending = true;
if (FD_ISSET(normfd, &fdset))
normEventPending = true;
#endif // if/else WIN32/UNIX
if (inputEventPending)
{ {
// Read input into our txBuffer // Read input into our txBuffer
#ifdef WIN32
inputLength = inputHandler.ReadData(inputBuffer, BUFFER_LENGTH);
if (inputLength > 0)
{
// We got our input
bytesWritten = 0;
inputNeeded = false;
}
else if (inputLength < 0)
{
inputHandler.Close();
inputClosed = true;
}
// else zero bytes read, still need input
#else
inputLength = fread(inputBuffer, 1, BUFFER_LENGTH, inputFile); inputLength = fread(inputBuffer, 1, BUFFER_LENGTH, inputFile);
if (inputLength > 0) if (inputLength > 0)
{ {
@ -302,10 +425,13 @@ int main(int argc, char* argv[])
} }
else if (feof(inputFile)) else if (feof(inputFile))
{ {
// TBD - initiate server shutdown if it's job is if (stdin != inputFile)
// only until the input is closed {
// Meanwhile, we just stick around to receive stuff from the clients fclose(inputFile);
inputNeeded = false; // TBD - should also fclose(inputFile)?? inputFile = NULL;
}
inputClosed = true;
} }
else if (ferror(inputFile)) else if (ferror(inputFile))
{ {
@ -318,12 +444,35 @@ int main(int argc, char* argv[])
// input starved, wait for next notification // input starved, wait for next notification
break; break;
default: default:
perror("normClient: error reading input"); perror("normServer: error reading input?!");
break; break;
} }
} }
} #endif // if/else WIN32/UNIX
if (FD_ISSET(normfd, &fdset)) if (inputClosed)
{
inputNeeded = false;
// Gracefully shutdown any connected clients
// TBD - set state variable to indicate exit upon all closed?
if (clientMap.empty())
{
keepGoing = false;
continue;
}
else
{
ClientMap::iterator it;
for (it = clientMap.begin(); it != clientMap.end(); it++)
{
Client* client = it->second;
NormSocketHandle clientSocket = client->GetSocket();
NormShutdown(clientSocket);
}
}
}
} // end if inputEventPending
if (normEventPending)
{ {
// There's a NORM event pending // There's a NORM event pending
@ -333,6 +482,8 @@ int main(int argc, char* argv[])
ClientInfo clientInfo; ClientInfo clientInfo;
if (NORM_NODE_INVALID != event.sender) if (NORM_NODE_INVALID != event.sender)
clientInfo = NormGetClientInfo(event.sender); clientInfo = NormGetClientInfo(event.sender);
else
clientInfo = NormGetSocketInfo(event.socket);
switch (event.type) switch (event.type)
{ {
case NORM_SOCKET_ACCEPT: case NORM_SOCKET_ACCEPT:
@ -344,26 +495,44 @@ int main(int argc, char* argv[])
if (NORM_SOCKET_INVALID != FindClientSocket(clientMap, clientInfo)) if (NORM_SOCKET_INVALID != FindClientSocket(clientMap, clientInfo))
{ {
// We think we're already connected to this client // We think we're already connected to this client
fprintf(stderr, "normServer: duplicative %s client ...\n", fprintf(stderr, "normServer: duplicative %s from client %s/%hu...\n",
(NORM_REMOTE_SENDER_NEW == event.event.type) ? "new" : "reset"); (NORM_REMOTE_SENDER_NEW == event.event.type) ? "new" : "reset",
clientInfo.GetAddressString(), clientInfo.GetPort());
continue; continue;
} }
NormSocketHandle clientSocket = NormAccept(serverSocket, event.sender); NormSocketHandle clientSocket = NormAccept(serverSocket, event.sender);
// TBD - For multicast, if we are sending a tx_stream, we could flush it here to Client* client = new Client(clientSocket);
if (NULL == client)
{
perror("normServer: new Client() error");
NormClose(clientSocket);
continue;
}
// TBD - For multicast, if we are sending a tx_stream, we could flush it here to
// bring the new receiver "up to date" ... probably would be best to // bring the new receiver "up to date" ... probably would be best to
// do this on some sort of timer-basis in the case of a bunch of receivers // do this on some sort of timer-basis in the case of a bunch of receivers
// joining in a short window of time ... // joining in a short window of time ...
if (trace) // note we're already tracing the mcast session if (trace) // note we're already tracing the mcast session
NormSetMessageTrace(NormGetSession(clientSocket), true); NormSetMessageTrace(NormGetSocketSession(clientSocket), true);
//NormSetMessageTrace(NormGetSession(clientSocket), true); //NormSetMessageTrace(NormGetSocketSession(clientSocket), true);
clientMap[clientInfo] = clientSocket; clientMap[clientInfo] = client;
// ACCEPTED is good as CONNECTED, so enable writing right away
client->SetWriteReady(true);
if (0 == clientCount)
{
// We have at least one client, so lets serve up some juicy input
inputNeeded = true;
writeReady = true;
}
clientCount++;
fprintf(stderr, "normServer: ACCEPTED connection from %s/%hu\n", fprintf(stderr, "normServer: ACCEPTED connection from %s/%hu\n",
clientInfo.GetAddressString(), clientInfo.GetPort()); clientInfo.GetAddressString(), clientInfo.GetPort());
// We have at least one client, so lets serve up some juicy input // Note that an ACCEPTED socket is essentially CONNECTED, so we could
inputNeeded = true; // go ahead and set writeReady to true, etc here
writeReady = true;
} }
else else
{ {
@ -375,8 +544,8 @@ int main(int argc, char* argv[])
{ {
fprintf(stderr, "normServer: CONNECTED to %s/%hu ...\n", fprintf(stderr, "normServer: CONNECTED to %s/%hu ...\n",
clientInfo.GetAddressString(), clientInfo.GetPort()); clientInfo.GetAddressString(), clientInfo.GetPort());
if (NORM_SOCKET_INVALID == firstClientSocket) Client* client = FindClient(clientMap, clientInfo);
firstClientSocket = event.socket; assert(NULL != client);
break; break;
} }
case NORM_SOCKET_READ: case NORM_SOCKET_READ:
@ -395,36 +564,60 @@ int main(int argc, char* argv[])
continue; continue;
} }
if (bytesRead > 0) if (bytesRead > 0)
{
#ifdef WIN32
// Use WriteFile() so there is no buffer delay
DWORD dwWritten;
WriteFile(hStdout, buffer, bytesRead, &dwWritten, NULL);
#else
fwrite(buffer, sizeof(char), bytesRead, stdout); fwrite(buffer, sizeof(char), bytesRead, stdout);
#endif // if/else WIN32
}
if (bytesRead < 1024) rxReady = false; if (bytesRead < 1024) rxReady = false;
} }
// Following lines are test code (to immediately close connection after first reaad)
// (tests server stale connection "reject" command)
//clientMap.erase(clientInfo);
//NormClose(event.socket);
break; break;
} }
case NORM_SOCKET_WRITE: case NORM_SOCKET_WRITE:
// We only demo server data transmission for the multicast server case {
// (see comment below) if (NULL != groupAddrPtr)
if ((NULL != groupAddrPtr) || (event.socket == firstClientSocket)) {
// We are a multicast server
writeReady = true; writeReady = true;
break; }
else
{
Client* client = FindClient(clientMap, clientInfo);
assert(NULL != client);
client->SetWriteReady(true);
}
break;
}
case NORM_SOCKET_CLOSING: case NORM_SOCKET_CLOSING:
{
fprintf(stderr, "normServer: client %s/%hu CLOSING connection ...\n", fprintf(stderr, "normServer: client %s/%hu CLOSING connection ...\n",
clientInfo.GetAddressString(), clientInfo.GetPort()); clientInfo.GetAddressString(), clientInfo.GetPort());
if (event.socket == firstClientSocket) Client* client = FindClient(clientMap, clientInfo);
firstClientSocket = NORM_SOCKET_INVALID; assert(NULL != client);
client->SetWriteReady(false);
break; break;
case NORM_SOCKET_CLOSED: }
case NORM_SOCKET_CLOSE:
{ {
fprintf(stderr, "normServer: connection to client %s/%hu CLOSED ...\n", fprintf(stderr, "normServer: connection to client %s/%hu CLOSED ...\n",
clientInfo.GetAddressString(), clientInfo.GetPort()); clientInfo.GetAddressString(), clientInfo.GetPort());
if (event.socket == firstClientSocket)
firstClientSocket = NORM_SOCKET_INVALID;
clientMap.erase(clientInfo); clientMap.erase(clientInfo);
break; NormClose(event.socket);
if (inputClosed && clientMap.empty())
keepGoing = false;
break;
} }
case NORM_SOCKET_NONE: case NORM_SOCKET_NONE:
break; break;
break;
} // end switch(event.type) } // end switch(event.type)
} }
else else
@ -433,31 +626,96 @@ int main(int argc, char* argv[])
} }
} // end if FD_ISSET(normfd) } // end if FD_ISSET(normfd)
// For our _multicast_ "normServer" example, the server can send to the group // If the normServer app has unsent data from STDIN, send it to the clients.
// (For a unicast "normServer", we would need to do something more complex // Note that a _multicast_ server multicasts to all clients at once while a
// to manage sending data to each individual client that connects to us. // unicast server sends to each connected client individually
// So, for the moment, the unicast "normServer" only sends to the "firstClientSocket" if ((inputLength > 0) && !inputNeeded)
NormSocketHandle sendSocket = (NULL != groupAddrPtr) ? serverSocket : firstClientSocket;
if (NORM_SOCKET_INVALID != sendSocket)
{ {
if (writeReady && (inputLength > 0)) // There is inputBuffer data for the server to send to the client(s)
if (NULL == groupAddrPtr)
{ {
// We have data in our inputBuffer and the NormSocket is "writeReady", so send it // Unicast the data to each connected client individually by iterating
bytesWritten += NormWrite(sendSocket, inputBuffer + bytesWritten, inputLength - bytesWritten); // over the clientMap and sending data out to each pending client socket
if (bytesWritten < inputLength) // (inputNeeded is reset to "true" when _all_ clients are non-pending)
bool clientPending = false;
ClientMap::iterator it;
for (it = clientMap.begin(); it != clientMap.end(); it++)
{ {
// Couldn't write whole inputBuffer, need to wait for NORM_SOCKET_WRITE event Client* client = it->second;
writeReady = false; if (!client->GetWriteReady())
{
clientPending = true;
continue;
}
unsigned int numBytes = client->GetBytesWritten();
if (numBytes < inputLength)
{
NormSocketHandle clientSocket = client->GetSocket();
bytesWritten += NormWrite(clientSocket, inputBuffer + numBytes, inputLength - numBytes);
client->SetBytesWritten(numBytes);
if (bytesWritten < inputLength)
{
// Couldn't write whole inputBuffer, need to wait for NORM_SOCKET_WRITE event
// for this client socket
client->SetWriteReady(false);
clientPending = true;
}
else
{
// inputBuffer has been completely written to this client
NormFlush(clientSocket);
}
}
} }
else if (!clientPending)
{ {
// inputBuffer has been completely written // inputBuffer was sent to _all_ clients, so reset
inputLength = 0; inputLength = 0;
inputNeeded = true; inputNeeded = true;
NormFlush(sendSocket); // Reset all client "bytes_written" to zero for next chunk of data
for (it = clientMap.begin(); it != clientMap.end(); it++)
it->second->SetBytesWritten(0);
} }
} }
} else
{
// Multicast the data out the "serverSocket" to all clients
// (Use the 'writeReady' and 'bytesWritten' state variables)
NormSocketHandle sendSocket = serverSocket;
if (NORM_SOCKET_INVALID != sendSocket)
{
if (writeReady && (inputLength > 0))
{
// We have data in our inputBuffer and the NormSocket is "writeReady", so send it
bytesWritten += NormWrite(sendSocket, inputBuffer + bytesWritten, inputLength - bytesWritten);
if (bytesWritten < inputLength)
{
// Couldn't write whole inputBuffer, need to wait for NORM_SOCKET_WRITE event
writeReady = false;
}
else
{
// inputBuffer has been completely written
inputLength = 0;
inputNeeded = true;
NormFlush(sendSocket);
}
}
}
}
} // end if ((inputLength > 0) && !inputNeeded)
} // end while (keepGoing) } // end while (keepGoing)
#ifdef WIN32
inputHandler.Close();
#else
if ((stdin != inputFile) && (NULL != inputFile))
{
fclose(inputFile);
inputFile = NULL;
}
#endif // if/else WIN32
NormClose(serverSocket);
serverSocket = NORM_SOCKET_INVALID;
} // end main() } // end main()

775
examples/normSocket.cpp
View File

File diff suppressed because it is too large Load Diff

52
examples/normSocket.h
View File

@ -19,7 +19,7 @@
// //
// 0) Make "NormInstanceHandle" optional for Listen/Connect/Accept // 0) Make "NormInstanceHandle" optional for Listen/Connect/Accept
// (for eventual blocking usage, it wouldn't be needed) // (for eventual blocking usage, it wouldn't be needed)
// 1) Support asymmetric server->multicast, client->unicast model // 1) Support asymmetric server->multicast, client->unicast model (DONE)
// 2) Provide non-blocking option for read/write calls // 2) Provide non-blocking option for read/write calls
// 3) leverage ICMP (e.g., port unreachable) feedback to detect connect() failure (DONE) // 3) leverage ICMP (e.g., port unreachable) feedback to detect connect() failure (DONE)
// 4) provide means for connect() to time out??? (DONE) // 4) provide means for connect() to time out??? (DONE)
@ -41,6 +41,12 @@
// f) Buffer size (sender, stream, receiver) // f) Buffer size (sender, stream, receiver)
// g) cache bounds / limits // g) cache bounds / limits
#ifdef WIN32
#include <BaseTsd.h> // for SSIZE_T
typedef SSIZE_T ssize_t;
#endif // WIN32
#include "normApi.h" #include "normApi.h"
typedef const void* NormSocketHandle; typedef const void* NormSocketHandle;
@ -55,42 +61,55 @@ extern const double NORM_DEFAULT_CONNECT_TIMEOUT;
// Main NormSocket API Functions // Main NormSocket API Functions
NormSocketHandle NormOpen(NormInstanceHandle instance = NORM_INSTANCE_INVALID); NormSocketHandle NormOpen(NormInstanceHandle instance);
NormSocketHandle NormListen(NormInstanceHandle instance, UINT16 serverPort, const char* groupAddr = NULL); bool NormListen(NormSocketHandle normSocket,
UINT16 serverPort,
const char* groupAddr = NULL,
const char* serverAddr = NULL);
NormSocketHandle NormConnect(NormInstanceHandle instance, bool NormConnect(NormSocketHandle normSocket,
const char* serverAddr, const char* serverAddr,
UINT16 serverPort, UINT16 serverPort,
const char* groupAddr = NULL, UINT16 localPort = 0,
NormNodeId clientId = NORM_NODE_NONE); const char* groupAddr = NULL,
NormNodeId clientId = NORM_NODE_ANY);
NormSocketHandle NormAccept(NormSocketHandle serverSocket, NormNodeHandle clientNode, NormInstanceHandle instance = NORM_INSTANCE_INVALID); NormSocketHandle NormAccept(NormSocketHandle serverSocket,
NormNodeHandle clientNode,
NormInstanceHandle instance = NORM_INSTANCE_INVALID);
void NormShutdown(NormSocketHandle normSocket); void NormShutdown(NormSocketHandle normSocket);
void NormClose(NormSocketHandle normSocket); void NormClose(NormSocketHandle normSocket);
ssize_t NormRead(NormSocketHandle normSocket, void *buf, size_t nbyte); ssize_t NormRead(NormSocketHandle normSocket, void* buf, size_t nbyte);
ssize_t NormWrite(NormSocketHandle normSocket, const void *buf, size_t nbyte); ssize_t NormWrite(NormSocketHandle normSocket, const void* buf, size_t nbyte);
int NormFlush(NormSocketHandle normSocket); int NormFlush(NormSocketHandle normSocket);
// Helper API functions // NormSocket helper functions
NormSessionHandle NormGetSession(NormSocketHandle normSocket); void NormSetSocketUserData(NormSocketHandle normSocket, const void* userData);
NormSessionHandle NormGetMulticastSession(NormSocketHandle normSocket); const void* NormGetSocketUserData(NormSocketHandle normSocket);
NormInstanceHandle NormGetSocketInstance(NormSocketHandle normSocket);
NormSessionHandle NormGetSocketSession(NormSocketHandle normSocket);
NormSessionHandle NormGetSocketMulticastSession(NormSocketHandle normSocket);
void NormGetPeerName(NormSocketHandle normSocket, char* addr, unsigned int* addrLen, UINT16* port);
void NormSetSocketTrace(NormSocketHandle normSocket, bool enable);
typedef enum NormSocketEventType typedef enum NormSocketEventType
{ {
NORM_SOCKET_NONE = 0, NORM_SOCKET_NONE = 0, // applications should generally ignore these
NORM_SOCKET_ACCEPT, // only issued for listening server sockets NORM_SOCKET_ACCEPT, // only issued for listening server sockets
NORM_SOCKET_CONNECT, // notification confirming connection to server NORM_SOCKET_CONNECT, // notification confirming connection to server
NORM_SOCKET_READ, // edge-triggered notification that socket is ready for reading NORM_SOCKET_READ, // edge-triggered notification that socket is ready for reading
NORM_SOCKET_WRITE, // edge-triggered notification that socket is ready for writing NORM_SOCKET_WRITE, // edge-triggered notification that socket is ready for writing
NORM_SOCKET_CLOSING, // indicates remote endpoint is closing socket (only read data at this point) NORM_SOCKET_CLOSING, // indicates remote endpoint is closing socket (only read data at this point)
NORM_SOCKET_CLOSED // indicates socket is now closed (invalid for further operations) NORM_SOCKET_CLOSE // indicates socket is now closed (invalid for further operations)
} NormSocketEventType; } NormSocketEventType;
typedef struct typedef struct
@ -116,5 +135,4 @@ typedef struct
bool NormGetSocketEvent(NormInstanceHandle normInstance, NormSocketEvent* event, bool waitForEvent = true); bool NormGetSocketEvent(NormInstanceHandle normInstance, NormSocketEvent* event, bool waitForEvent = true);
#endif // !_NORM_SOCKET #endif // !_NORM_SOCKET

155
examples/normSocketNotes.txt Normal file
View File

@ -0,0 +1,155 @@
NormSocket API Extension Notes
The NormSocket API (currently defined in examples/normSocket.h) is an in-development extension to the
base (low-level) NORM API (defined in include/normApi.h) that provides a more familiar (socket-like),
easier-to-use API for some specific NORM use patterns. The use patterns supported here are those of a
client-server paradigm where "clients" have individual, reliable connections to the server and the
"server" can have either a one-to-many multicast connection or individual unicast connections to the set
of clients. The paradigm follows a connection-oriented socket model where the server "listens" for
incoming connections initiated by clients. The low-level NORM API is used to establish multiple NORM
sessions as needed to support the necessary data exhange relationships. Since this is a
connection-oriented model, explicit ACK-based flow control used for both client-to-server and
server-to-client(s) data transfers. This is a more controlled model of NORM use as compared to the
looser "connection-free" model of operation NORM use has typically observed.
The NormSocket API code makes careful use of operating system socket-binding to enable port reuse and
simpler session establishment where clients can "connect" to the server listening on a well-known port
as compared to typical base NORM API use where participants can exchange data on a a priori known common
port number (and group address if multicast). However, the clients do need to know the unicast address
of the server and the paradigm is somewhat that of Single-Source Multicast (SSM). Client application
can, of course, using multiple NormSocket instances connect to multiple multicast (or unicast) servers..
THere
A server application can use the NormListen() call to establish a session that listens for incoming
connection requests, Client applications can use the NormConnect() call to initiate a connection to a
listening server. In response, the server application can use NormAccept() to establish connections with
the clients.
At the moment a NormGetSocketEvent() (analogous to the NormGetNextEvent() call in the base NORM API) can
be used to provide cues to changes in NormSocket state (notification of incoming connections, connection
establishment, read/write readiness, etc.). A descriptor is provided for asynchronous I/O notification
of pending events. At the moment, the default behavior is a single NormInstance, and hence a single
descriptor, can be used for multiple NormSockets, but it is possible to call NormCreateInstance() and
associate a separate NormInstance (and hence separate descriptor) with each NormSocket that is
listending, connected or accepted if desired.
As the NormSocket API extension matures, more examples and documentation will be provided as well as
incorporating it (possibly directly) into the "normApi.h" header file and NORM library. At the moment,
"examples/normClient.cpp" and "examples/normServer.cpp" illustrate (and test) basic usage of the API.
The "normServer" can be run as a unicast server using the syntax:
./normServer listen <serverPort>
or a multicast server using the syntax:
./normServer listen <groupAddr>/<serverPort>
Corresponding "normClient" instances can connect to unicast server using the syntax:
./normClient connect <serverAddr>/<serverPort>
or to a multicast server using the syntax:
./normClient connect <serverAddr>/<serverPort>,<groupAddr>
There are some additional command-line options shared by these examples such as "trace" to enable NORM's
message trace output and "debug <debugLevel>" to invoke different levels of debug output. Note that an
option to specify the multicast interface is planned but not yet implemented in these examples.
Similarly, an option to set a specific unicast server binding will also be provided.
TODO (not in priority order):
1) When client initiates connection and server-listener receives NORM_CMD(CC), add code
to directly "inject" the NORM_CMD(CC) into the newly created server-side client
session such that NORM_ACK is immediately sent. This will give client a more immediate
RTT measurement, etc.
2) Add options to set transmit rate, congestion control, etc options via NormSocketHandle
(current code works at default tx rate of 64 kbps)
3) The server-listener session should be modified so NORM_DATA messages are ignored. Also,
NormSetServerListener() could also set the session a silent receiver, too?
4) Add support for additional transport modes in addition to current byte stream:
` 1) Message stream
2) NORM_OBJECT_DATA (w/ data copy option?)
3) NORM_OBJECT_FILE
5) Add option to specify multicast "interface" and use in examples.
6) Enable setting a specific unicast server binding address.
===========================================================================
MISC. DEVELOPMENT NOTES
// Notional program outline
// (optional mcastAddr iff asymm multicast/unicast (i.e. SSM))
normSocket = NormListen(port, [mcastAddr])
_or_
normSocket = NormConnect(addr, port, [mcastAddr])
while NormGetSocketEvent(normSocket)
{
switch (event.GetType())
NORM_ACCEPT:
clientSocket = NormAccept(normSocket, client, [instance]); // rx-only for asymm mcast case
NORM_CONNECTED:
// in response to NormConnect when "server" is detected (NEW_REMOTE_SENDER)
NORM_TX_READY:
upon QUEUE_VACANCY, etc ... socket _might_ be write() ready
NORM_RX_READY:
upon RX_OBJECT_NEW, RX_OBJECT_UPDATE ... socket _might_ be read() ready
NORM_CLOSING: ???
upon stream end received from remote sender
NORM_CLOSED: ???
upon stream end _and_ ack to our stream end
}
*** Server listen on a port (optional mcast address):
- Create NormSession that receive-only on given port (if it's asymm mcast, then sender enabled, too)
- For each client (detected via NEW_REMOTE_SENDER yielding a NormSocket::NORM_ACCEPT event)
Create tightly binded session for rx (and tx if unicast) ... this will fail if binding already exists
*** Client connect
- Create single socket (common ephemeral tx/rx port) NormSession for server destination addr/port.
- Call NormStartSender() to initiate probing of server
- NORM_CONNECTED upon NEW_REMOTE_SENDER (from server) ???
QUESTIONS:
1) How does a server reject a connection request? (for now, just ignore)
2) How is a "connection" gracefully terminated? (NormShutdown() vs. NormClose()?)
NormClose(graceful = false) ...
a) stop sender/receiver, destroy everything
NormShutdown(graceful = true) ...
a) close stream (gracefully, w/ ack request)
b) upon ack receipt (or timeout) _and_ rx stream end, destroy everything
Upon receipt of rx stream end, close tx stream gracefully ...
initiator -> stream end; upon ack set tx_stream = NULL and wait for rx_stream end
receptor -> upon rx_stream end, initiate tx_stream shutdown and wait for ack

2178
examples/normStreamer.cpp Normal file
View File

File diff suppressed because it is too large Load Diff

243
examples/win32InputHandler.cpp Normal file
View File

@ -0,0 +1,243 @@
#include <windows.h>
#include <stdio.h> // for perror()
#include <string.h> // for memset()
#include <process.h> // for _beginthreadex(), _endthreadex()
// This class creates a child thread to read STDIN into a buffer.
// An event is set that can be monitored by the parent to know
// when input is available. Mutexes are used to regulate the
// thread's reading (i.e., flow control).
class Win32InputHandler
{
public:
Win32InputHandler();
~Win32InputHandler();
bool Open(int buflen = 4096);
void Close();
// The event handle returned here can be used in
// a call like WaitForSignalObject() to get notified
// when there is data to be read.
HANDLE GetEventHandle() const
{return input_event;}
// This does not block and the input_event
// handle is a cue for when to call it
// Returns number of bytes copied into "buffer"
// (-1 is returned upon input error (input closed))
int ReadData(char* buffer, int numBytes);
private:
static unsigned int __stdcall DoThreadOpen(void* param);
unsigned int Run();
HANDLE input_handle;
HANDLE input_event;
HANDLE thread_handle;
bool input_ready;
char* input_buffer;
int input_buflen;
int input_index;
int input_outdex;
CONDITION_VARIABLE buffer_vacancy;
CRITICAL_SECTION buffer_lock;
bool is_running;
int input_length;
bool input_error;
}; // end class Win32InputHandler
Win32InputHandler::Win32InputHandler()
: input_handle(NULL), input_event(NULL), thread_handle(NULL),
input_ready(false), input_buffer(NULL), input_buflen(0),
input_index(0), input_outdex(0), is_running(false),
input_length(0), input_error(false)
{
}
Win32InputHandler::~Win32InputHandler()
{
if (is_running) Close();
}
bool Win32InputHandler::Open(int buflen)
{
if (is_running) Close();
if (NULL != input_buffer)
{
input_buffer = NULL;
delete[] input_buffer;
}
input_buflen = input_index = input_outdex = input_length = 0;
input_error = false;
if (NULL != input_event) CloseHandle(input_event);
// Get the STDIN handle whatever it may be
input_handle = GetStdHandle(STD_INPUT_HANDLE);
/*if (FILE_TYPE_CHAR == GetFileType(input_handle))
{
// The STDIN is a console
DWORD consoleMode;
bool result = GetConsoleMode(input_handle, &consoleMode);
consoleMode &= ~(ENABLE_LINE_INPUT | ENABLE_MOUSE_INPUT | ENABLE_WINDOW_INPUT);
SetConsoleMode(input_handle, consoleMode);
}*/
// Create an event handle, non-signaled initially
input_event = CreateEvent(NULL, TRUE, FALSE, NULL);
if (NULL == input_event)
{
perror("Win32InputHandler::Open() CreateEvent() error");
return false;
}
if (NULL == (input_buffer = new char[buflen]))
{
perror("Win32InputHandler::Open() new input_buffer error");
CloseHandle(input_event);
input_event = NULL;
return false;
}
input_buflen = buflen;
InitializeConditionVariable(&buffer_vacancy);
InitializeCriticalSection(&buffer_lock);
is_running = true;
if (0 == (thread_handle = (HANDLE)_beginthreadex(NULL, 0, DoThreadOpen, this, 0, NULL)))
{
perror("Win32InputHandler::Open() _beginthreadex() error");
DeleteCriticalSection(&buffer_lock);
delete[] input_buffer;
input_buffer = NULL;
CloseHandle(input_event);
input_event = NULL;
return false;
}
return true;
} // end Win32InputHandler::Open()
void Win32InputHandler::Close()
{
EnterCriticalSection(&buffer_lock);
is_running = false;
WakeConditionVariable(&buffer_vacancy); // in case it was full
LeaveCriticalSection(&buffer_lock);
WaitForSingleObject(thread_handle, INFINITE);
DeleteCriticalSection(&buffer_lock);
delete[] input_buffer;
input_buffer = NULL;
CloseHandle(input_event);
input_event = NULL;
} // end Win32InputHandler::Close()
unsigned int __stdcall Win32InputHandler::DoThreadOpen(void* param)
{
Win32InputHandler* inputHandler = reinterpret_cast<Win32InputHandler*>(param);
unsigned int exitStatus = inputHandler->Run();
_endthreadex(exitStatus);
return exitStatus;
} // end Win32InputHandler::DoThreadOpen()
unsigned int Win32InputHandler::Run()
{
// This loop is for the child thread to read input one byte at a time
// (Since consoles/anonymous pipes don't support async i/o)
// The input bytes are copied to a circularly-managed "input_buffer"
// that can be accessed via the "ReadData()" method by the parent
while (is_running)
{
// Read input one byte at a time and buffer
DWORD dwRead = 0;
BOOL result = ReadFile(input_handle, input_buffer+input_index, 1, &dwRead, NULL);
EnterCriticalSection(&buffer_lock);
if (result)
{
if (0 != dwRead)
{
input_length++;
input_index++;
if (input_index >= input_buflen)
input_index = 0;
}
else
{
LeaveCriticalSection(&buffer_lock);
continue;
}
}
else
{
// What do we do on error?!
input_error = true;
is_running = false;
}
// Signals the parent of data ready (or error)
if (!input_ready)
{
SetEvent(input_event);
input_ready = true;
}
// If the buffer has been filled, this blocks this child
// loop until buffer space is available or it is stopped.
while ((input_index == input_outdex) && (0 != input_length) && is_running)
SleepConditionVariableCS(&buffer_vacancy, &buffer_lock, INFINITE);
LeaveCriticalSection(&buffer_lock);
} // end while(is_running)
return (input_error ? 1 : 0);
} // end Win32InputHandler::Run()
// Called by parent to gobble up data (non-blocking read)
// Returns number of bytes read or -1 on error (e.g., STDIN pipe closed)
int Win32InputHandler::ReadData(char* buffer, int numBytes)
{
EnterCriticalSection(&buffer_lock);
if ((0 != input_length) && (0 != numBytes))
{
if (numBytes > input_length)
numBytes = input_length;
input_length -= numBytes;
if (0 == input_length)
{
// On error condition, the event is left triggered
// after the input_buffer is emptied so parent gets
// all data buffered before the error (e.g., eof)
// occurred.
if (input_ready && !input_error)
{
input_ready = false;
ResetEvent(input_event);
}
}
// Copy "numBytes" of available data to caller's "buffer"
// (two-step process when input_buffer wraps)
unsigned int offset = input_outdex;
input_outdex += numBytes;
if (input_outdex >= input_buflen)
{
input_outdex -= input_buflen;
unsigned int count = numBytes - input_outdex;
memcpy(buffer, input_buffer+offset, count);
if (0 != input_outdex)
memcpy(buffer+count, input_buffer, input_outdex);
}
else
{
memcpy(buffer, input_buffer + offset, numBytes);
}
}
else
{
// Doing the error check
if (input_error && (0 != numBytes))
numBytes = -1; // to indicate an error has occurred and child is dead
else
numBytes = 0;
}
// This wakes up the child if it is sleeping because the buffer was full
if (numBytes > 0)
WakeConditionVariable(&buffer_vacancy);
LeaveCriticalSection(&buffer_lock);
return numBytes;
} // end Win32InputHandler::ReadData()

57
examples/wintest.cpp Normal file
View File

@ -0,0 +1,57 @@
#include "win32InputHandler.cpp" // for class Win32InputHandler
#include <windows.h>
#include <stdio.h>
#include <stdlib.h>
#define BUFSIZE 4096
int main(int argc, char** argv)
{
HANDLE hStdout = GetStdHandle(STD_OUTPUT_HANDLE);
if (argc > 1)
{
// Generate random output
const char* text = "This is some text to output\n";
int textlen = strlen(text);
while (true)
{
int count = rand() % 100;
while (count > 0)
{
printf(text);
DWORD dwWritten;
BOOL fSuccess = WriteFile(hStdout, text, textlen, &dwWritten, NULL);
count -= textlen;
}
int delay = (rand() % 1000);
Sleep(1000);
}
return 0;
}
Win32InputHandler inputHandler;
inputHandler.Start();
while (true)
{
char buffer[BUFSIZE];
WaitForSingleObject(inputHandler.GetEventHandle(), INFINITE);
int numBytes = inputHandler.ReadData(buffer, 1024);
if (numBytes < 0) break;
DWORD dwWritten;
BOOL fSuccess = WriteFile(hStdout, buffer, numBytes, &dwWritten, NULL);
}
inputHandler.Stop();
return 0;
} // end main()

51
include/normApi.h
View File

@ -59,8 +59,8 @@ extern "C" {
#endif /* __cplusplus */ #endif /* __cplusplus */
#define NORM_VERSION_MAJOR 1 #define NORM_VERSION_MAJOR 1
#define NORM_VERSION_MINOR 0 #define NORM_VERSION_MINOR 5
#define NORM_VERSION_PATCH 0 #define NORM_VERSION_PATCH 7
/** NORM API Types */ /** NORM API Types */
typedef const void* NormInstanceHandle; typedef const void* NormInstanceHandle;
@ -187,7 +187,7 @@ typedef enum NormEventType
NORM_GRTT_UPDATED, NORM_GRTT_UPDATED,
NORM_CC_ACTIVE, NORM_CC_ACTIVE,
NORM_CC_INACTIVE, NORM_CC_INACTIVE,
NORM_ACKING_NODE_NEW, // whe NormSetAutoAcking xxx NORM_ACKING_NODE_NEW, // whe NormSetAutoAcking
NORM_SEND_ERROR, // ICMP error (e.g. destination unreachable) NORM_SEND_ERROR, // ICMP error (e.g. destination unreachable)
NORM_USER_TIMEOUT // issues when timeout set by NormSetUserTimer() expires NORM_USER_TIMEOUT // issues when timeout set by NormSetUserTimer() expires
} NormEventType; } NormEventType;
@ -201,6 +201,11 @@ typedef struct
} NormEvent; } NormEvent;
// For setting custom NORM_OBJECT_DATA alloc/free functions
typedef char* (*NormAllocFunctionHandle)(size_t);
typedef void (*NormFreeFunctionHandle)(char*);
/** NORM API General Initialization and Operation Functions */ /** NORM API General Initialization and Operation Functions */
NORM_API_LINKAGE NORM_API_LINKAGE
@ -208,6 +213,9 @@ int NormGetVersion(int* major DEFAULT((int*)0),
int* minor DEFAULT((int*)0), int* minor DEFAULT((int*)0),
int* patch DEFAULT((int*)0)); int* patch DEFAULT((int*)0));
NORM_API_LINKAGE NORM_API_LINKAGE
NormInstanceHandle NormCreateInstance(bool priorityBoost DEFAULT(false)); NormInstanceHandle NormCreateInstance(bool priorityBoost DEFAULT(false));
@ -256,6 +264,11 @@ const NormDescriptor NORM_DESCRIPTOR_INVALID;
NORM_API_LINKAGE NORM_API_LINKAGE
NormDescriptor NormGetDescriptor(NormInstanceHandle instanceHandle); NormDescriptor NormGetDescriptor(NormInstanceHandle instanceHandle);
NORM_API_LINKAGE
void NormSetAllocationFunctions(NormInstanceHandle instance,
NormAllocFunctionHandle allocFunc,
NormFreeFunctionHandle freeFunc);
/** NORM Session Creation and Control Functions */ /** NORM Session Creation and Control Functions */
NORM_API_LINKAGE NORM_API_LINKAGE
@ -288,6 +301,11 @@ void NormCancelUserTimer(NormSessionHandle sessionHandle);
NORM_API_LINKAGE NORM_API_LINKAGE
NormNodeId NormGetLocalNodeId(NormSessionHandle sessionHandle); NormNodeId NormGetLocalNodeId(NormSessionHandle sessionHandle);
NORM_API_LINKAGE
bool NormGetAddress(NormSessionHandle sessionHandle,
char* addrBuffer,
unsigned int* bufferLen,
UINT16* port DEFAULT((UINT16*)0));
NORM_API_LINKAGE NORM_API_LINKAGE
UINT16 NormGetRxPort(NormSessionHandle sessionHandle); UINT16 NormGetRxPort(NormSessionHandle sessionHandle);
@ -315,6 +333,13 @@ bool NormChangeDestination(NormSessionHandle sessionHandle,
UINT16 sessionPort, UINT16 sessionPort,
bool connectToSessionAddress DEFAULT(false)); bool connectToSessionAddress DEFAULT(false));
NORM_API_LINKAGE
void NormSetServerListener(NormSessionHandle sessionHandle, bool state);
NORM_API_LINKAGE
bool NormTransferSender(NormSessionHandle sessionHandle, NormNodeHandle sender);
NORM_API_LINKAGE NORM_API_LINKAGE
void NormSetRxPortReuse(NormSessionHandle sessionHandle, void NormSetRxPortReuse(NormSessionHandle sessionHandle,
bool enableReuse, bool enableReuse,
@ -410,7 +435,8 @@ bool NormStartSender(NormSessionHandle sessionHandle,
UINT32 bufferSpace, UINT32 bufferSpace,
UINT16 segmentSize, UINT16 segmentSize,
UINT16 numData, UINT16 numData,
UINT16 numParity); UINT16 numParity,
UINT8 fecId DEFAULT(0));
NORM_API_LINKAGE NORM_API_LINKAGE
void NormStopSender(NormSessionHandle sessionHandle); void NormStopSender(NormSessionHandle sessionHandle);
@ -584,6 +610,9 @@ bool NormSendCommand(NormSessionHandle sessionHandle,
NORM_API_LINKAGE NORM_API_LINKAGE
void NormCancelCommand(NormSessionHandle sessionHandle); void NormCancelCommand(NormSessionHandle sessionHandle);
NORM_API_LINKAGE
void NormSetSynStatus(NormSessionHandle sessionHandle, bool state);
/* NORM Receiver Functions */ /* NORM Receiver Functions */
NORM_API_LINKAGE NORM_API_LINKAGE
@ -646,6 +675,14 @@ NORM_API_LINKAGE
void NormNodeSetRxRobustFactor(NormNodeHandle remoteSender, void NormNodeSetRxRobustFactor(NormNodeHandle remoteSender,
int robustFactor); int robustFactor);
NORM_API_LINKAGE
bool NormPreallocateRemoteSender(NormSessionHandle sessionHandle,
unsigned long bufferSize,
UINT16 segmentSize,
UINT16 numData,
UINT16 numParity,
unsigned int streamBufferSize DEFAULT(0));
NORM_API_LINKAGE NORM_API_LINKAGE
bool NormStreamRead(NormObjectHandle streamHandle, bool NormStreamRead(NormObjectHandle streamHandle,
char* buffer, char* buffer,
@ -706,6 +743,12 @@ const char* NormDataAccessData(NormObjectHandle objectHandle);
NORM_API_LINKAGE NORM_API_LINKAGE
char* NormDataDetachData(NormObjectHandle objectHandle); char* NormDataDetachData(NormObjectHandle objectHandle);
NORM_API_LINKAGE
char* NormAlloc(size_t numBytes);
NORM_API_LINKAGE
void NormFree(char* dataPtr);
NORM_API_LINKAGE NORM_API_LINKAGE
NormNodeHandle NormObjectGetSender(NormObjectHandle objectHandle); NormNodeHandle NormObjectGetSender(NormObjectHandle objectHandle);

202
include/normMessage.h
View File

@ -134,6 +134,8 @@ class NormObjectSize
// Operators // Operators
bool operator==(const NormObjectSize& b) const bool operator==(const NormObjectSize& b) const
{return (b.size == size);} {return (b.size == size);}
bool operator!=(const NormObjectSize& b) const
{return (b.size != size);}
NormObjectSize operator+(const NormObjectSize& b) const NormObjectSize operator+(const NormObjectSize& b) const
{ {
NormObjectSize result(size); NormObjectSize result(size);
@ -201,9 +203,9 @@ class NormObjectId
} }
bool operator<=(const NormObjectId& id) const bool operator<=(const NormObjectId& id) const
{return ((value == id.value) || (*this<id));} {return ((value == id.value) || (*this < id));}
bool operator>=(const NormObjectId& id) const bool operator>=(const NormObjectId& id) const
{return ((value == id.value) || (*this>id));} {return ((value == id.value) || (*this > id));}
bool operator==(const NormObjectId& id) const bool operator==(const NormObjectId& id) const
{return (value == id.value);} {return (value == id.value);}
bool operator!=(const NormObjectId& id) const bool operator!=(const NormObjectId& id) const
@ -212,6 +214,8 @@ class NormObjectId
void operator-=(UINT16 delta) void operator-=(UINT16 delta)
{value -= delta;} {value -= delta;}
const char* GetValuePtr() const
{return (const char*)(&value);}
NormObjectId& operator++(int) {value++; return *this;} NormObjectId& operator++(int) {value++; return *this;}
NormObjectId& operator--(int) {value--; return *this;} NormObjectId& operator--(int) {value--; return *this;}
@ -224,33 +228,93 @@ class NormBlockId
{ {
public: public:
NormBlockId() {}; NormBlockId() {};
NormBlockId(UINT32 id) {value = id;} NormBlockId(UINT32 id) : value(id) {}
NormBlockId(const NormObjectId& id) {value = (UINT32)id;}
operator UINT32() const {return value;} UINT32 GetValue() const
{return value;}
const char* GetValuePtr() const
{return ((const char*)&value);}
bool operator==(const NormBlockId& id) const bool operator==(const NormBlockId& id) const
{return (value == (UINT32)id);} {return (value == id.value);}
bool operator!=(const NormBlockId& id) const bool operator!=(const NormBlockId& id) const
{return (value != (UINT32)id);} {return (value != id.value);}
// These static helper methods provide a "mask" parameter to allow
// for different bit-length NormBlockId values (depends upon FEC encoding scheme).
// The "mask" is in the eye-of-the-beholder, i.e., the given NormObject
// or NormBlockBuffer that is manipulating block id values for protocol purposes
// provide the FEC block "mask" to these methods.
// We may reconsider adding the "mask" as a NormBlockId member variable, but since
// NormBlockId usage is so ubiquitous, it could be added overhead consider within
// the context of sender, object, etc a common fec_block_mask is used.
//INT32 operator-(const NormBlockId& id) const // Compute difference of (a - b). If a non-zero bit "mask" is given, the difference
// {return ((INT32)value - id.value);} // is for the masked word size (e.g., mask = 0x00ffffff is a 24-bit integer).
static INT32 Difference(const NormBlockId& a, const NormBlockId& b, UINT32 mask)
bool operator<(const NormBlockId& id) const
{ {
UINT32 diff = value - id.value; if (mask)
return ((diff > 0x80000000) || ((0x80000000 == diff) && (value > id.value))); {
UINT32 sign = (mask ^ (mask >> 1));
UINT32 result = a.value - b.value;
if (0 == (result & sign))
return (INT32)(result & mask);
else if ((result != sign) || (a.value < b.value))
return (INT32)(result | ~mask);
else
return (INT32)(result & mask);
}
else
{
return ((INT32)(a.value - b.value));
}
} }
bool operator>(const NormBlockId& id) const // Compare two block ids. If a non-zero bit "mask" is given, the comparison
// is a "sliding window" (signed) over the bit space. Otherwise, it is
// simply an unsigned value comparison.
// Returns -1, 0, +1 for (a < b), (a == b), and (a > b), respectively
static int Compare(const NormBlockId& a, const NormBlockId& b, UINT32 mask)
{ {
UINT32 diff = id.value - value; if (mask)
return ((diff > 0x80000000) || ((0x80000000 == diff) && (id.value > value))); {
// "Sliding window" comparison
INT32 delta = Difference(a, b, mask);
if (delta < 0)
return -1;
else if (0 == delta)
return 0;
else // if delta > 0
return 1;
}
else if (a.value < b.value)
{
return -1;
}
else if (a.value == b.value)
{
return 0;
}
else // if (a > b)
{
return 1;
}
} }
void Increment(UINT32 i, UINT32 mask)
{
value = value + i;
if (mask) value &= mask;
}
NormBlockId& operator++(int ) {value++; return *this;} void Decrement(UINT32 i, UINT32 mask)
{
if (mask && (value < i))
value = (mask - (i - value) + 1);
else
value -= i;
}
private: private:
UINT32 value; UINT32 value;
}; // end class NormBlockId }; // end class NormBlockId
@ -308,7 +372,7 @@ class NormHeaderExtension
}; // end class NormHeaderExtension }; // end class NormHeaderExtension
// This class is some we use to set/get // This class is what we use to set/get
// FEC Payload Id content. The FEC Payload // FEC Payload Id content. The FEC Payload
// Id format is dependent upon the "fec_id" (FEC Type) // Id format is dependent upon the "fec_id" (FEC Type)
// and, in some cases, its field size ("m") parameter // and, in some cases, its field size ("m") parameter
@ -523,7 +587,10 @@ class NormMsg
{ {
buffer[SOURCE_ID_OFFSET] = htonl(sourceId); buffer[SOURCE_ID_OFFSET] = htonl(sourceId);
} }
// For messages to be sent, "addr" is destination
void SetDestination(const ProtoAddress& dst) {addr = dst;} void SetDestination(const ProtoAddress& dst) {addr = dst;}
// For message received, "addr" is source
void SetSource(const ProtoAddress& src) {addr = src;}
void AttachExtension(NormHeaderExtension& extension) void AttachExtension(NormHeaderExtension& extension)
{ {
@ -545,6 +612,8 @@ class NormMsg
{return (Type)(((UINT8*)buffer)[TYPE_OFFSET] & 0x0f);} {return (Type)(((UINT8*)buffer)[TYPE_OFFSET] & 0x0f);}
UINT16 GetHeaderLength() UINT16 GetHeaderLength()
{return ((UINT8*)buffer)[HDR_LEN_OFFSET] << 2;} {return ((UINT8*)buffer)[HDR_LEN_OFFSET] << 2;}
UINT16 GetBaseHeaderLength()
{return header_length_base;}
UINT16 GetSequence() const UINT16 GetSequence() const
{ {
return (ntohs((((UINT16*)buffer)[SEQUENCE_OFFSET]))); return (ntohs((((UINT16*)buffer)[SEQUENCE_OFFSET])));
@ -555,9 +624,11 @@ class NormMsg
} }
const ProtoAddress& GetDestination() const {return addr;} const ProtoAddress& GetDestination() const {return addr;}
const ProtoAddress& GetSource() const {return addr;} const ProtoAddress& GetSource() const {return addr;}
const char* GetBuffer() {return ((char*)buffer);} const char* GetBuffer() const {return ((char*)buffer);}
UINT16 GetLength() const {return length;} UINT16 GetLength() const {return length;}
void Display() const; // hex output to log
// To retrieve any attached header extensions // To retrieve any attached header extensions
bool HasExtensions() const {return (header_length > header_length_base);} bool HasExtensions() const {return (header_length > header_length_base);}
bool GetNextExtension(NormHeaderExtension& ext) const bool GetNextExtension(NormHeaderExtension& ext) const
@ -566,7 +637,7 @@ class NormMsg
UINT16 nextOffset = UINT16 nextOffset =
(UINT16)(currentBuffer ? (currentBuffer - buffer + (ext.GetLength()/4)) : (UINT16)(currentBuffer ? (currentBuffer - buffer + (ext.GetLength()/4)) :
(header_length_base/4)); (header_length_base/4));
bool result = (nextOffset < (header_length/4)); bool result = HasExtensions() ? (nextOffset < (header_length/4)) : false;
ext.AttachBuffer(result ? (buffer+nextOffset) : (UINT32*)NULL); ext.AttachBuffer(result ? (buffer+nextOffset) : (UINT32*)NULL);
return result; return result;
} }
@ -627,7 +698,8 @@ class NormObjectMsg : public NormMsg
FLAG_INFO = 0x04, FLAG_INFO = 0x04,
FLAG_UNRELIABLE = 0x08, FLAG_UNRELIABLE = 0x08,
FLAG_FILE = 0x10, FLAG_FILE = 0x10,
FLAG_STREAM = 0x20 FLAG_STREAM = 0x20,
FLAG_SYN = 0x40
//FLAG_MSG_START = 0x40 deprecated //FLAG_MSG_START = 0x40 deprecated
}; };
UINT16 GetInstanceId() const UINT16 GetInstanceId() const
@ -733,6 +805,51 @@ class NormFtiExtension2 : public NormHeaderExtension
}; };
}; // end class NormFtiExtension2 }; // end class NormFtiExtension2
// Helper class for containing key FTI params
class NormFtiData
{
public:
NormFtiData()
: object_size(NormObjectSize(0)), segment_size(0),
num_data(0), num_parity(0), fec_m(0), instance_id(0) {}
~NormFtiData() {}
void SetObjectSize(const NormObjectSize& objectSize)
{object_size = objectSize;}
void SetSegmentSize(UINT16 segmentSize)
{segment_size = segmentSize;}
void SetFecMaxBlockLen(UINT16 numData)
{num_data = numData;}
void SetFecNumParity(UINT16 numParity)
{num_parity = numParity;}
void SetFecFieldSize(UINT8 fecM)
{fec_m = fecM;}
void SetFecInstanceId(UINT16 instanceId)
{instance_id = instanceId;}
const NormObjectSize& GetObjectSize() const
{return object_size;}
UINT16 GetSegmentSize() const
{return segment_size;}
UINT16 GetFecMaxBlockLen() const
{return num_data;}
UINT16 GetFecNumParity() const
{return num_parity;}
UINT8 GetFecFieldSize() const
{return fec_m;}
UINT16 GetFecInstanceId() const
{return instance_id;}
private:
NormObjectSize object_size;
UINT16 segment_size;
UINT16 num_data;
UINT16 num_parity;
UINT8 fec_m;
UINT16 instance_id;
}; // end class NormFtiData
// This FEC Object Transmission Information assumes "fec_id" == 5 (RFC 5510) // This FEC Object Transmission Information assumes "fec_id" == 5 (RFC 5510)
// (this is the fully-defined 8-bit Reed-Solomon codec) // (this is the fully-defined 8-bit Reed-Solomon codec)
@ -1282,6 +1399,13 @@ class NormCmdCCMsg : public NormCmdMsg
} }
bool GetCCNode(NormNodeId nodeId, UINT8& flags, UINT8& rtt, UINT16& rate) const; bool GetCCNode(NormNodeId nodeId, UINT8& flags, UINT8& rtt, UINT16& rate) const;
// This function uses the "reserved" field of the NORM_CMD(CC) message
// and is not strictly compliant with RFC 5740 when invoked.
enum {FLAG_SYN = 0x01};
bool SynIsSet() const
{return (0 != (FLAG_SYN & ((UINT8*)buffer)[RESERVED_OFFSET]));}
void SetSyn()
{((UINT8*)buffer)[RESERVED_OFFSET] = FLAG_SYN;}
class Iterator; class Iterator;
friend class Iterator; friend class Iterator;
@ -1390,6 +1514,8 @@ class NormRepairRequest
{flags |= theFlag;} {flags |= theFlag;}
void ClearFlag(NormRepairRequest::Flag theFlag) void ClearFlag(NormRepairRequest::Flag theFlag)
{flags &= ~theFlag;} {flags &= ~theFlag;}
void SetFlags(int theFlags)
{flags = theFlags;}
// Returns length (each repair item requires 8 bytes of space) // Returns length (each repair item requires 8 bytes of space)
bool AppendRepairItem(UINT8 fecId, bool AppendRepairItem(UINT8 fecId,
@ -1425,7 +1551,13 @@ class NormRepairRequest
{return form;} {return form;}
bool FlagIsSet(NormRepairRequest::Flag theFlag) const bool FlagIsSet(NormRepairRequest::Flag theFlag) const
{return (0 != (theFlag & flags));} {return (0 != (theFlag & flags));}
UINT16 GetLength() const {return length;} int GetFlags() const
{return flags;}
UINT16 GetLength() const
{return (ITEM_LIST_OFFSET + length);}
UINT32* GetBuffer() const
{return buffer;}
// Outputs textual representation of RepairRequest content // Outputs textual representation of RepairRequest content
void Log(UINT8 fecId, UINT8 fecM) const; void Log(UINT8 fecId, UINT8 fecM) const;
@ -1436,10 +1568,10 @@ class NormRepairRequest
// Checks for matching fecId and assumes constant 'm' ?!?! // Checks for matching fecId and assumes constant 'm' ?!?!
Iterator(const NormRepairRequest& theRequest, UINT8 fecId, UINT8 fecM); Iterator(const NormRepairRequest& theRequest, UINT8 fecId, UINT8 fecM);
void Reset() {offset = 0;} void Reset() {offset = 0;}
bool NextRepairItem(NormObjectId* objectId, UINT16 NextRepairItem(NormObjectId* objectId,
NormBlockId* blockId, NormBlockId* blockId,
UINT16* blockLen, UINT16* blockLen,
UINT16* symbolId); UINT16* symbolId);
private: private:
const NormRepairRequest& request; const NormRepairRequest& request;
UINT8 fec_id; UINT8 fec_id;
@ -1732,6 +1864,22 @@ class NormNackMsg : public NormMsg
return requestLength; return requestLength;
} }
// TBD - add some safety checks to these methods
void InitFrom(NormNackMsg nack)
{
// Copy header from "nack"
memcpy(buffer, nack.buffer, nack.GetHeaderLength());
header_length_base = nack.header_length_base;
length = header_length = nack.GetHeaderLength();
}
void AppendRepairRequest(const NormRepairRequest request)
{
memcpy(buffer+length/4, request.GetBuffer(), request.GetLength());
length += request.GetLength();
}
void ResetPayload()
{length = GetHeaderLength();}
// Message processing // Message processing
NormNodeId GetSenderId() const NormNodeId GetSenderId() const
{return (ntohl(buffer[SENDER_ID_OFFSET]));} {return (ntohl(buffer[SENDER_ID_OFFSET]));}

108
include/normNode.h
View File

@ -141,6 +141,10 @@ class NormLossEstimator2
void SetTolerateLoss(bool state) void SetTolerateLoss(bool state)
{tolerate_loss = state;} {tolerate_loss = state;}
unsigned short GetLagIndex() const
{return lag_index;}
private: private:
enum {DEPTH = 8}; enum {DEPTH = 8};
@ -174,7 +178,6 @@ class NormLossEstimator2
{init = true; Sync(theSequence);} {init = true; Sync(theSequence);}
void Sync(unsigned short theSequence) void Sync(unsigned short theSequence)
{lag_index = theSequence;} {lag_index = theSequence;}
void ChangeLagDepth(unsigned int theDepth) void ChangeLagDepth(unsigned int theDepth)
{ {
theDepth = (theDepth > 20) ? 20 : theDepth; theDepth = (theDepth > 20) ? 20 : theDepth;
@ -270,14 +273,13 @@ class NormCCNode : public NormNode
UINT16 cc_sequence; UINT16 cc_sequence;
}; // end class NormCCNode }; // end class NormCCNode
class NormSenderNode : public NormNode class NormSenderNode : public NormNode, public ProtoTree::Item
{ {
public: public:
enum ObjectStatus {OBJ_INVALID, OBJ_NEW, OBJ_PENDING, OBJ_COMPLETE}; enum ObjectStatus {OBJ_INVALID, OBJ_NEW, OBJ_PENDING, OBJ_COMPLETE};
enum RepairBoundary {BLOCK_BOUNDARY, OBJECT_BOUNDARY}; enum RepairBoundary {BLOCK_BOUNDARY, OBJECT_BOUNDARY};
enum SyncPolicy enum SyncPolicy
{ {
SYNC_CURRENT, // sync to detect transmit point, iff NORM_DATA from first FEC block unless stream SYNC_CURRENT, // sync to detect transmit point, iff NORM_DATA from first FEC block unless stream
@ -291,6 +293,12 @@ class NormSenderNode : public NormNode
void SetInstanceId(UINT16 instanceId) void SetInstanceId(UINT16 instanceId)
{instance_id = instanceId;} {instance_id = instanceId;}
bool PreallocateRxStream(unsigned int bufferSize,
UINT16 segmentSize ,
UINT16 numData,
UINT16 numParity);
bool GetFtiData(const NormObjectMsg& msg, NormFtiData& ftiData);
// Parameters // Parameters
NormObject::NackingMode GetDefaultNackingMode() const NormObject::NackingMode GetDefaultNackingMode() const
{return default_nacking_mode;} {return default_nacking_mode;}
@ -317,11 +325,15 @@ class NormSenderNode : public NormNode
void UpdateGrttEstimate(UINT8 grttQuantized); void UpdateGrttEstimate(UINT8 grttQuantized);
double GetGrttEstimate() const {return grtt_estimate;} double GetGrttEstimate() const {return grtt_estimate;}
void ResetGrttNotification() {notify_on_grtt_update = true;}
bool UpdateLossEstimate(const struct timeval& currentTime, bool UpdateLossEstimate(const struct timeval& currentTime,
unsigned short theSequence, unsigned short theSequence,
bool ecnStatus = false); bool ecnStatus = false);
double LossEstimate() {return loss_estimator.LossFraction();} double LossEstimate() {return loss_estimator.LossFraction();}
unsigned short GetCurrentSequence() const
{return loss_estimator.GetLagIndex();}
double GetRttEstimate() const {return rtt_estimate;}
void CheckCCFeedback(); void CheckCCFeedback();
@ -340,7 +352,13 @@ class NormSenderNode : public NormNode
UINT16 GetInstanceId() {return instance_id;} UINT16 GetInstanceId() {return instance_id;}
bool IsOpen() const {return is_open;} bool IsOpen() const {return is_open;}
void Close(); void Close();
bool AllocateBuffers(UINT8 fecId, UINT16 fecInstanceId, UINT8 fecM, UINT16 segmentSize, UINT16 numData, UINT16 numParity); bool AllocateBuffers(unsigned int bufferSpace,
UINT8 fecId,
UINT16 fecInstanceId,
UINT8 fecM,
UINT16 segmentSize,
UINT16 numData,
UINT16 numParity);
bool BuffersAllocated() {return (0 != segment_size);} bool BuffersAllocated() {return (0 != segment_size);}
void FreeBuffers(); void FreeBuffers();
void Activate(bool isObjectMsg); void Activate(bool isObjectMsg);
@ -381,6 +399,15 @@ class NormSenderNode : public NormNode
void DeleteObject(NormObject* obj); void DeleteObject(NormObject* obj);
NormObject* GetNextPendingObject()
{
NormObjectId objid;
if (GetNextPending(objid))
return rx_table.Find(objid);
else
return NULL;
}
UINT16 SegmentSize() {return segment_size;} UINT16 SegmentSize() {return segment_size;}
UINT16 BlockSize() {return ndata;} UINT16 BlockSize() {return ndata;}
UINT16 NumParity() {return nparity;} UINT16 NumParity() {return nparity;}
@ -437,9 +464,9 @@ class NormSenderNode : public NormNode
unsigned long BufferOverunCount() const unsigned long BufferOverunCount() const
{return segment_pool.OverunCount() + block_pool.OverrunCount();} {return segment_pool.OverunCount() + block_pool.OverrunCount();}
unsigned long CurrentStreamBufferUsage() const; unsigned long CurrentStreamBufferUsage();
unsigned long PeakStreamBufferUsage() const; unsigned long PeakStreamBufferUsage();
unsigned long StreamBufferOverunCount() const; unsigned long StreamBufferOverunCount();
//unsigned long RecvTotal() const {return recv_total;} //unsigned long RecvTotal() const {return recv_total;}
@ -461,6 +488,7 @@ class NormSenderNode : public NormNode
recv_goodput.Reset(); recv_goodput.Reset();
} }
void IncrementResyncCount() {resync_count++;} void IncrementResyncCount() {resync_count++;}
void DecrementResyncCount() {resync_count--;}
unsigned long ResyncCount() const {return resync_count;} unsigned long ResyncCount() const {return resync_count;}
unsigned long NackCount() const {return nack_count;} unsigned long NackCount() const {return nack_count;}
unsigned long SuppressCount() const {return suppress_count;} unsigned long SuppressCount() const {return suppress_count;}
@ -502,9 +530,34 @@ class NormSenderNode : public NormNode
bool ReadNextCmd(char* buffer, unsigned int* buflen); bool ReadNextCmd(char* buffer, unsigned int* buflen);
void SetAddress(const ProtoAddress& address)
{
unsigned int len = address.GetLength();
memcpy(key_buffer, address.GetRawHostAddress(), len);
UINT16 port = htons(address.GetPort());
memcpy(key_buffer+len, &port, 2);
key_size = (len+2) << 3;
NormNode::SetAddress(address);
}
UINT8 GetGrttQuantized() const
{return grtt_quantized;}
UINT8 GetBackoffFactor() const
{return backoff_factor;}
UINT8 GetGroupSizeQuantized() const
{return gsize_quantized;}
UINT16 GetCCSequence() const
{return cc_sequence;}
double GetSendRate() const
{return send_rate;}
private: private:
const char* GetKey() const
{return key_buffer;}
unsigned int GetKeysize() const
{return key_size;}
static const double DEFAULT_NOMINAL_INTERVAL; static const double DEFAULT_NOMINAL_INTERVAL;
static const double ACTIVITY_INTERVAL_MIN; static const double ACTIVITY_INTERVAL_MIN;
@ -523,6 +576,9 @@ class NormSenderNode : public NormNode
void AttachCCFeedback(NormAckMsg& ack); void AttachCCFeedback(NormAckMsg& ack);
void HandleRepairContent(const UINT32* buffer, UINT16 bufferLen); void HandleRepairContent(const UINT32* buffer, UINT16 bufferLen);
void FragmentNack(NormNackMsg& superNack);
UINT16 instance_id; UINT16 instance_id;
int robust_factor; int robust_factor;
@ -540,6 +596,7 @@ class NormSenderNode : public NormNode
UINT8 fec_m; UINT8 fec_m;
unsigned int ndata; unsigned int ndata;
unsigned int nparity; unsigned int nparity;
NormStreamObject* preset_stream;
NormObjectTable rx_table; NormObjectTable rx_table;
ProtoSlidingMask rx_pending_mask; ProtoSlidingMask rx_pending_mask;
@ -561,7 +618,7 @@ class NormSenderNode : public NormNode
// Watermark acknowledgement // Watermark acknowledgement
ProtoTimer ack_timer; ProtoTimer ack_timer;
bool ack_pending; bool ack_pending;
NormObjectId watermark_object_id; NormObjectId watermark_object_id;
NormBlockId watermark_block_id; NormBlockId watermark_block_id;
NormSegmentId watermark_segment_id; NormSegmentId watermark_segment_id;
@ -574,6 +631,7 @@ class NormSenderNode : public NormNode
double gsize_estimate; double gsize_estimate;
UINT8 gsize_quantized; UINT8 gsize_quantized;
double backoff_factor; double backoff_factor;
bool notify_on_grtt_update; // for API
// Remote sender congestion control state // Remote sender congestion control state
NormLossEstimator2 loss_estimator; NormLossEstimator2 loss_estimator;
@ -600,6 +658,10 @@ class NormSenderNode : public NormNode
CmdBuffer* cmd_buffer_tail; // newly-received commands appended here CmdBuffer* cmd_buffer_tail; // newly-received commands appended here
CmdBuffer* cmd_buffer_pool; // we "pool" allocated buffers for possible reuse here CmdBuffer* cmd_buffer_pool; // we "pool" allocated buffers for possible reuse here
// Used for NormSocket API extension to index by addr/port
char key_buffer[16+2]; // big enough for IPv6 plus port
unsigned int key_size;
// For statistics tracking // For statistics tracking
Accumulator recv_total; // total recvd accumulator Accumulator recv_total; // total recvd accumulator
Accumulator recv_goodput; // goodput recvd accumulator Accumulator recv_goodput; // goodput recvd accumulator
@ -612,7 +674,8 @@ class NormSenderNode : public NormNode
}; // end class NormSenderNode }; // end class NormSenderNode
// Used for binary trees of NormNodes // Used for binary trees of NormNodes sorted by NormNodeId
// (TBD - update to use ProtoTree instead?
class NormNodeTree class NormNodeTree
{ {
friend class NormNodeTreeIterator; friend class NormNodeTreeIterator;
@ -639,7 +702,6 @@ class NormNodeTreeIterator
void Reset(NormNode* prevNode = NULL); void Reset(NormNode* prevNode = NULL);
NormNode* GetNextNode(); NormNode* GetNextNode();
private: private:
const NormNodeTree& tree; const NormNodeTree& tree;
NormNode* next; NormNode* next;
@ -689,4 +751,30 @@ class NormNodeListIterator
const NormNodeList& list; const NormNodeList& list;
NormNode* next; NormNode* next;
}; // end class NormNodeListIterator }; // end class NormNodeListIterator
// Used to track remote client sender nodes for server/listener sessions
class NormClientTree : public ProtoTreeTemplate<NormSenderNode>
{
public:
void InsertNode(NormSenderNode& sender)
{
sender.Retain();
Insert(sender);
}
void RemoveNode(NormSenderNode& sender)
{
Remove(sender);
sender.Release();
}
NormSenderNode* FindNodeByAddress(const ProtoAddress& addr)
{
char key[16+2];
unsigned int len = addr.GetLength();
memcpy(key, addr.GetRawHostAddress(), len);
UINT16 port = htons(addr.GetPort());
memcpy(key+len, &port, 2);
return Find(key, (len+2) << 3);
}
}; // end class NormClientTree
#endif // NORM_NODE #endif // NORM_NODE

168
include/normObject.h
View File

@ -7,7 +7,15 @@
#include <stdio.h> #include <stdio.h>
#define USE_PROTO_TREE 1 // for more better performing NormObjectTable?
#ifdef USE_PROTO_TREE
#include "protoTree.h"
class NormObject : public ProtoSortedTree::Item
#else
class NormObject class NormObject
#endif // if/else USE_PROTO_TREE
{ {
friend class NormObjectTable; friend class NormObjectTable;
@ -44,16 +52,43 @@ class NormObject
// This must be reset after each update // This must be reset after each update
void SetNotifyOnUpdate(bool state) void SetNotifyOnUpdate(bool state)
{ {notify_on_update = state;}
notify_on_update = state;
}
// Object information // Object information
NormObject::Type GetType() const {return type;} NormObject::Type GetType() const {return type;}
const NormObjectId& GetId() const {return transport_id;} const NormObjectId& GetId() const {return transport_id;}
const NormObjectSize& GetSize() const {return object_size;} const NormObjectSize& GetSize() const {return object_size;}
void SetId(const NormObjectId transportId)
{transport_id = transportId;}
UINT16 GetSegmentSize() const {return segment_size;}
UINT8 GetFecId() const {return fec_id;}
UINT16 GetFecMaxBlockLen() const {return ndata;}
UINT16 GetFecNumParity() const {return nparity;}
UINT8 GetFecFieldSize() const {return fec_m;}
// returns difference (a - b)
INT32 Difference(NormBlockId a, NormBlockId b) const
{return NormBlockId::Difference(a, b, fec_block_mask);}
// returns -1, 0, or 1 for (a < b), (a == b), or (a > b), respectively
int Compare(NormBlockId a, NormBlockId b) const
{return NormBlockId::Compare(a, b, fec_block_mask);}
void Increment(NormBlockId& b, UINT32 i = 1) const
{b.Increment(i, fec_block_mask);}
void Decrement(NormBlockId& b, UINT32 i = 1) const
{b.Decrement(i, fec_block_mask);}
bool HaveInfo() const {return (info_len > 0);} bool HaveInfo() const {return (info_len > 0);}
bool HasInfo() const {return (NULL != info_ptr);} bool HasInfo() const {return (NULL != info_ptr);}
void ClearInfo()
{
if (NULL != info_ptr) delete[] info_ptr;
info_ptr = NULL;
info_len = 0;
pending_info = false;
}
const char* GetInfo() const {return info_ptr;} const char* GetInfo() const {return info_ptr;}
UINT16 GetInfoLength() const {return info_len;} UINT16 GetInfoLength() const {return info_len;}
@ -94,8 +129,6 @@ class NormObject
} }
void Close(); void Close();
virtual bool WriteSegment(NormBlockId blockId, virtual bool WriteSegment(NormBlockId blockId,
NormSegmentId segmentId, NormSegmentId segmentId,
const char* buffer) = 0; const char* buffer) = 0;
@ -119,17 +152,19 @@ class NormObject
NormBlockId GetFinalBlockId() const {return final_block_id;} NormBlockId GetFinalBlockId() const {return final_block_id;}
UINT32 GetBlockSize(NormBlockId blockId) const UINT32 GetBlockSize(NormBlockId blockId) const
{ {
return (((UINT32)blockId < large_block_count) ? large_block_size : return ((blockId.GetValue() < large_block_count) ? large_block_size :
small_block_size); small_block_size);
} }
NormObjectSize GetBytesPending() const; NormObjectSize GetBytesPending() const;
bool IsPending(bool flush = true) const; bool IsPending(bool flush = true) const;
bool IsRepairPending() const; bool IsRepairPending();
bool IsPendingInfo() {return pending_info;} bool IsPendingInfo() {return pending_info;}
bool PendingMaskIsSet() const bool PendingMaskIsSet() const
{return pending_mask.IsSet();} {return pending_mask.IsSet();}
unsigned int GetPendingMaskSize() const
{return pending_mask.GetSize();}
bool GetFirstPending(NormBlockId& blockId) const bool GetFirstPending(NormBlockId& blockId) const
{ {
UINT32 index = 0; UINT32 index = 0;
@ -139,7 +174,7 @@ class NormObject
} }
bool GetNextPending(NormBlockId& blockId) const bool GetNextPending(NormBlockId& blockId) const
{ {
UINT32 index = (UINT32)blockId; UINT32 index = blockId.GetValue();
bool result = pending_mask.GetNextSet(index); bool result = pending_mask.GetNextSet(index);
blockId = NormBlockId(index); blockId = NormBlockId(index);
return result; return result;
@ -161,7 +196,7 @@ class NormObject
bool GetNextRepair(NormBlockId& blockId) const bool GetNextRepair(NormBlockId& blockId) const
{ {
UINT32 index = (UINT32)blockId; UINT32 index = blockId.GetValue();
bool result = repair_mask.GetNextSet(index); bool result = repair_mask.GetNextSet(index);
blockId = NormBlockId(index); blockId = NormBlockId(index);
return result; return result;
@ -174,7 +209,7 @@ class NormObject
return result; return result;
} }
bool FindRepairIndex(NormBlockId& blockId, NormSegmentId& segmentId) const; bool FindRepairIndex(NormBlockId& blockId, NormSegmentId& segmentId);
// Methods available to sender for transmission // Methods available to sender for transmission
bool NextSenderMsg(NormObjectMsg* msg); bool NextSenderMsg(NormObjectMsg* msg);
@ -185,26 +220,17 @@ class NormObject
void ClearFirstPass() {first_pass = false};*/ void ClearFirstPass() {first_pass = false};*/
bool TxReset(NormBlockId firstBlock = NormBlockId(0), bool requeue = false); bool TxReset(NormBlockId firstBlock = NormBlockId(0), bool requeue = false);
bool TxResetBlocks(NormBlockId nextId, NormBlockId lastId); bool TxResetBlocks(const NormBlockId& nextId, const NormBlockId& lastId);
bool TxUpdateBlock(NormBlock* theBlock, bool TxUpdateBlock(NormBlock* theBlock,
NormSegmentId firstSegmentId, NormSegmentId firstSegmentId,
NormSegmentId lastSegmentId, NormSegmentId lastSegmentId,
UINT16 numErasures) UINT16 numErasures);
{
NormBlockId blockId = theBlock->GetId();
bool result = theBlock->TxUpdate(firstSegmentId, lastSegmentId,
GetBlockSize(blockId), nparity,
numErasures);
ASSERT(result ? pending_mask.Set(blockId) : true);
result = result ? pending_mask.Set(blockId) : false;
return result;
}
bool HandleInfoRequest(bool holdoff); bool HandleInfoRequest(bool holdoff);
bool HandleBlockRequest(NormBlockId nextId, NormBlockId lastId); bool HandleBlockRequest(const NormBlockId& nextId, const NormBlockId& lastId);
NormBlock* FindBlock(NormBlockId blockId) {return block_buffer.Find(blockId);} NormBlock* FindBlock(NormBlockId blockId) {return block_buffer.Find(blockId);}
bool ActivateRepairs(); bool ActivateRepairs();
bool IsRepairSet(NormBlockId blockId) {return repair_mask.Test(blockId);} bool IsRepairSet(NormBlockId blockId) {return repair_mask.Test(blockId.GetValue());}
bool IsPendingSet(NormBlockId blockId) {return pending_mask.Test(blockId);} bool IsPendingSet(NormBlockId blockId) {return pending_mask.Test(blockId.GetValue());}
bool AppendRepairAdv(NormCmdRepairAdvMsg& cmd); bool AppendRepairAdv(NormCmdRepairAdvMsg& cmd);
NormBlockId GetMaxPendingBlockId() const {return max_pending_block;} NormBlockId GetMaxPendingBlockId() const {return max_pending_block;}
@ -236,12 +262,12 @@ class NormObject
bool ReceiverRewindCheck(NormBlockId blockId, bool ReceiverRewindCheck(NormBlockId blockId,
NormSegmentId segmentId); NormSegmentId segmentId);
bool IsRepairPending(bool flush); bool IsRepairPending(bool flush);
bool AppendRepairRequest(NormNackMsg& nack, bool flush); bool AppendRepairRequest(NormNackMsg& nack, bool flush, UINT16 payloadMax);
void SetRepairInfo() {repair_info = true;} void SetRepairInfo() {repair_info = true;}
bool SetRepairs(NormBlockId first, NormBlockId last) bool SetRepairs(NormBlockId first, NormBlockId last)
{ {
return (first == last) ? repair_mask.Set(first) : return ((first == last) ? repair_mask.Set(first.GetValue()) :
repair_mask.SetBits(first, repair_mask.Delta(last,first)+1); repair_mask.SetBits(first.GetValue(), repair_mask.Difference(last.GetValue(),first.GetValue())+1));
} }
void SetLastNackTime(const ProtoTime& theTime) void SetLastNackTime(const ProtoTime& theTime)
{last_nack_time = theTime;} {last_nack_time = theTime;}
@ -259,6 +285,16 @@ class NormObject
void Accept() {accepted = true;} void Accept() {accepted = true;}
#ifdef USE_PROTO_TREE
// Proto::Tree item required overrides
const char* GetKey() const
{return transport_id.GetValuePtr();}
unsigned int GetKeysize() const
{return (8*sizeof(UINT16));}
ProtoTree::Endian GetEndian() const
{return ProtoTree::GetNativeEndian();}
#endif // USE PROTO_TREE
NormObject::Type type; NormObject::Type type;
class NormSession& session; class NormSession& session;
class NormSenderNode* sender; // NULL value indicates local (tx) object class NormSenderNode* sender; // NULL value indicates local (tx) object
@ -269,6 +305,7 @@ class NormObject
UINT16 segment_size; UINT16 segment_size;
UINT8 fec_id; UINT8 fec_id;
UINT8 fec_m; UINT8 fec_m;
UINT32 fec_block_mask;
UINT16 ndata; UINT16 ndata;
UINT16 nparity; UINT16 nparity;
NormBlockBuffer block_buffer; NormBlockBuffer block_buffer;
@ -298,8 +335,9 @@ class NormObject
bool notify_on_update; bool notify_on_update;
const void* user_data; // for NORM API usage only const void* user_data; // for NORM API usage only
#ifndef USE_PROTO_TREE
NormObject* next; NormObject* next;
#endif
}; // end class NormObject }; // end class NormObject
@ -325,7 +363,7 @@ class NormFileObject : public NormObject
return result; return result;
} }
bool PadToSize() bool PadToSize()
{return file.Pad(NormObject::GetSize().GetOffset());} {return file.IsOpen() ? file.Pad(NormObject::GetSize().GetOffset()) : false;}
virtual bool WriteSegment(NormBlockId blockId, virtual bool WriteSegment(NormBlockId blockId,
NormSegmentId segmentId, NormSegmentId segmentId,
@ -349,9 +387,12 @@ class NormDataObject : public NormObject
{ {
// (TBD) allow support of greater than 4GB size data objects // (TBD) allow support of greater than 4GB size data objects
public: public:
typedef void (*DataFreeFunctionHandle)(char*);
NormDataObject(class NormSession& theSession, NormDataObject(class NormSession& theSession,
class NormSenderNode* theSender, class NormSenderNode* theSender,
const NormObjectId& objectId); const NormObjectId& objectId,
DataFreeFunctionHandle dataFreeFunc);
~NormDataObject(); ~NormDataObject();
bool Open(char* dataPtr, bool Open(char* dataPtr,
@ -381,12 +422,16 @@ class NormDataObject : public NormObject
virtual char* RetrieveSegment(NormBlockId blockId, virtual char* RetrieveSegment(NormBlockId blockId,
NormSegmentId segmentId); NormSegmentId segmentId);
private: private:
NormObjectSize large_block_length; NormObjectSize large_block_length;
NormObjectSize small_block_length; NormObjectSize small_block_length;
char* data_ptr; char* data_ptr;
UINT32 data_max; UINT32 data_max;
bool data_released; // when true, data_ptr is deleted bool data_released; // when true, data_ptr is deleted
DataFreeFunctionHandle data_free_func;
// on NormDataObject destruction // on NormDataObject destruction
}; // end class NormDataObject }; // end class NormDataObject
@ -426,7 +471,7 @@ class NormStreamObject : public NormObject
bool GetPushMode() const {return push_mode;} bool GetPushMode() const {return push_mode;}
bool IsOldBlock(NormBlockId blockId) const bool IsOldBlock(NormBlockId blockId) const
{return (!stream_buffer.IsEmpty() && (blockId < stream_buffer.RangeLo()));} {return (!stream_buffer.IsEmpty() && (Compare(blockId, stream_buffer.RangeLo()) < 0));}
bool IsClosing() {return stream_closing;} bool IsClosing() {return stream_closing;}
bool HasVacancy() bool HasVacancy()
@ -459,6 +504,11 @@ class NormStreamObject : public NormObject
} }
bool StreamAdvance(); bool StreamAdvance();
NormBlockId GetSyncId() const
{return stream_sync_id;}
NormBlockId GetNextId() const
{return stream_next_id;}
virtual bool WriteSegment(NormBlockId blockId, virtual bool WriteSegment(NormBlockId blockId,
NormSegmentId segmentId, NormSegmentId segmentId,
const char* buffer); const char* buffer);
@ -479,15 +529,15 @@ class NormStreamObject : public NormObject
bool LockSegments(NormBlockId blockId, NormSegmentId firstId, NormSegmentId lastId); bool LockSegments(NormBlockId blockId, NormSegmentId firstId, NormSegmentId lastId);
NormBlockId StreamBufferLo() const {return stream_buffer.RangeLo();} NormBlockId StreamBufferLo() const {return stream_buffer.RangeLo();}
NormBlockId RepairWindowLo() const;
void Prune(NormBlockId blockId, bool updateStatus); void Prune(NormBlockId blockId, bool updateStatus);
bool IsFlushPending() {return flush_pending;} bool IsFlushPending() {return flush_pending;}
NormBlockId FlushBlockId()
{return (write_index.segment ? write_index.block : NormBlockId FlushBlockId() const;
(NormBlockId((UINT32)write_index.block-1)));}
NormSegmentId FlushSegmentId() NormSegmentId FlushSegmentId() const
{return (write_index.segment ? (write_index.segment-1) : {return (write_index.segment ? (write_index.segment-1) : (ndata-1));}
(ndata-1));}
NormBlockId GetNextBlockId() const NormBlockId GetNextBlockId() const
{return (sender ? read_index.block : write_index.block);} {return (sender ? read_index.block : write_index.block);}
@ -529,6 +579,7 @@ class NormStreamObject : public NormObject
public: public:
NormBlockId block; NormBlockId block;
NormSegmentId segment; NormSegmentId segment;
UINT16 offset;
}; };
// Extra state for STREAM objects // Extra state for STREAM objects
bool stream_sync; bool stream_sync;
@ -551,9 +602,10 @@ class NormStreamObject : public NormObject
bool msg_start; bool msg_start;
FlushMode flush_mode; FlushMode flush_mode;
bool push_mode; bool push_mode;
bool stream_broken;
bool stream_closing; bool stream_closing;
// For threaded API purposes // For threaded API purposes
UINT32 block_pool_threshold; UINT32 block_pool_threshold;
}; // end class NormStreamObject }; // end class NormStreamObject
@ -588,6 +640,10 @@ class NormSimObject : public NormObject
}; // end class NormSimObject }; // end class NormSimObject
#endif // SIMULATE #endif // SIMULATE
#ifdef USE_PROTO_TREE
class NormObjectTree : public ProtoSortedTreeTemplate<NormObject> {};
#endif // USE_PROTO_TREE
class NormObjectTable class NormObjectTable
{ {
public: public:
@ -600,7 +656,6 @@ class NormObjectTable
void SetRangeMax(UINT16 rangeMax); void SetRangeMax(UINT16 rangeMax);
void Destroy(); void Destroy();
bool IsInited() const {return (NULL != table);}
UINT16 GetRangeMax() const {return range_max;} UINT16 GetRangeMax() const {return range_max;}
bool CanInsert(NormObjectId objectId) const; bool CanInsert(NormObjectId objectId) const;
bool Insert(NormObject* theObject); bool Insert(NormObject* theObject);
@ -613,6 +668,22 @@ class NormObjectTable
UINT32 GetCount() const {return count;} UINT32 GetCount() const {return count;}
const NormObjectSize& GetSize() const {return size;} const NormObjectSize& GetSize() const {return size;}
#ifdef USE_PROTO_TREE
class Iterator
{
public:
Iterator(NormObjectTable& objectTable);
NormObject* GetNextObject();
NormObject* GetPrevObject();
void Reset();
private:
const NormObjectTable& table;
NormObjectTree::Iterator iterator;
NormObject* next_object;
};
#else
class Iterator class Iterator
{ {
public: public:
@ -626,12 +697,19 @@ class NormObjectTable
bool reset; bool reset;
NormObjectId index; NormObjectId index;
}; };
#endif // if/else USE_PROTO_TREE
private: private:
#ifndef USE_PROTO_TREE
NormObject* Next(NormObject* o) const {return o->next;} NormObject* Next(NormObject* o) const {return o->next;}
#endif
#ifdef USE_PROTO_TREE
NormObjectTree tree;
#else
NormObject** table; NormObject** table;
UINT16 hash_mask; UINT16 hash_mask;
#endif // if/else USE_PROTO_TREE
UINT16 range_max; // max range of objects that can be kept UINT16 range_max; // max range of objects that can be kept
UINT16 range; // zero if "object table" is empty UINT16 range; // zero if "object table" is empty
NormObjectId range_lo; NormObjectId range_lo;

66
include/normSegment.h
View File

@ -4,6 +4,9 @@
#include "normMessage.h" #include "normMessage.h"
#include "protoBitmask.h" #include "protoBitmask.h"
#define USE_PROTO_TREE 1 // for more better performing NormBlockBuffer?
// Norm uses preallocated (or dynamically allocated) pools of // Norm uses preallocated (or dynamically allocated) pools of
// segments (vectors) for different buffering purposes // segments (vectors) for different buffering purposes
@ -43,7 +46,11 @@ class NormSegmentPool
bool overrun_flag; bool overrun_flag;
}; // end class NormSegmentPool }; // end class NormSegmentPool
#ifdef USE_PROTO_TREE
class NormBlock : public ProtoSortedTree::Item
#else
class NormBlock class NormBlock
#endif // if/else USE_PROTO_TREE
{ {
friend class NormBlockPool; friend class NormBlockPool;
friend class NormBlockBuffer; friend class NormBlockBuffer;
@ -144,7 +151,7 @@ class NormBlock
UINT8 fecId, UINT8 fecId,
UINT8 fecM, UINT8 fecM,
UINT16 numData, UINT16 numData,
UINT16 segmentSize); UINT16 payloadMax);
// Receiver routines // Receiver routines
void RxInit(NormBlockId& blockId, UINT16 ndata, UINT16 nparity) void RxInit(NormBlockId& blockId, UINT16 ndata, UINT16 nparity)
@ -238,7 +245,7 @@ class NormBlock
UINT16 numParity, UINT16 numParity,
NormObjectId objectId, NormObjectId objectId,
bool pendingInfo, bool pendingInfo,
UINT16 segmentSize); UINT16 payloadMax);
void SetLastNackTime(const ProtoTime& theTime) void SetLastNackTime(const ProtoTime& theTime)
@ -254,6 +261,15 @@ class NormBlock
void EmptyToPool(NormSegmentPool& segmentPool); void EmptyToPool(NormSegmentPool& segmentPool);
private: private:
#ifdef USE_PROTO_TREE
const char* GetKey() const
{return blk_id.GetValuePtr();}
unsigned int GetKeysize() const
{return (8*sizeof(UINT32));}
ProtoTree::Endian GetEndian() const
{return ProtoTree::GetNativeEndian();}
#endif // USE_PROTO_TREE
NormBlockId blk_id; NormBlockId blk_id;
UINT16 size; UINT16 size;
char** segment_table; char** segment_table;
@ -267,8 +283,7 @@ class NormBlock
ProtoBitmask pending_mask; ProtoBitmask pending_mask;
ProtoBitmask repair_mask; ProtoBitmask repair_mask;
ProtoTime last_nack_time; // for stream flow control ProtoTime last_nack_time; // for stream flow control
NormBlock* next; // used for NormBlockPool
NormBlock* next;
}; // end class NormBlock }; // end class NormBlock
class NormBlockPool class NormBlockPool
@ -314,6 +329,10 @@ class NormBlockPool
bool overrun_flag; bool overrun_flag;
}; // end class NormBlockPool }; // end class NormBlockPool
#ifdef USE_PROTO_TREE
class NormBlockTree : public ProtoSortedTreeTemplate<NormBlock> {};
#endif // USE_PROTO_TREE
class NormBlockBuffer class NormBlockBuffer
{ {
public: public:
@ -322,18 +341,34 @@ class NormBlockBuffer
NormBlockBuffer(); NormBlockBuffer();
~NormBlockBuffer(); ~NormBlockBuffer();
bool Init(unsigned long rangeMax, unsigned long tableSize = 256); bool Init(unsigned long rangeMax, unsigned long tableSize, UINT32 fecBlockMask);
void Destroy(); void Destroy();
bool Insert(NormBlock* theBlock); bool Insert(NormBlock* theBlock);
bool Remove(const NormBlock* theBlock); bool Remove(NormBlock* theBlock);
NormBlock* Find(const NormBlockId& blockId) const; NormBlock* Find(const NormBlockId& blockId) const;
NormBlockId RangeLo() const {return range_lo;} NormBlockId RangeLo() const {return range_lo;}
NormBlockId RangeHi() const {return range_hi;} NormBlockId RangeHi() const {return range_hi;}
NormBlockId RangeMin() const;
bool IsEmpty() const {return (0 == range);} bool IsEmpty() const {return (0 == range);}
bool CanInsert(NormBlockId blockId) const; bool CanInsert(NormBlockId blockId) const;
#ifdef USE_PROTO_TREE
class Iterator
{
public:
Iterator(NormBlockBuffer& blockBuffer);
NormBlock* GetNextBlock();
void Reset();
private:
NormBlockBuffer& buffer;
NormBlockTree::Iterator iterator;
NormBlock* next_block;
};
#else
class Iterator class Iterator
{ {
public: public:
@ -346,18 +381,29 @@ class NormBlockBuffer
bool reset; bool reset;
NormBlockId index; NormBlockId index;
}; };
#endif // if/else USE_PROTO_TREE
private: private:
static NormBlock* Next(NormBlock* b) {return b->next;} int Compare(NormBlockId a, NormBlockId b) const
{return NormBlockId::Compare(a, b, fec_block_mask);}
INT32 Difference(NormBlockId a, NormBlockId b) const
{return NormBlockId::Difference(a, b, fec_block_mask);}
void Increment(NormBlockId& b, UINT32 i = 1) const
{b.Increment(i, fec_block_mask);}
void Decrement(NormBlockId& b, UINT32 i = 1) const
{b.Decrement(i, fec_block_mask);}
#ifdef USE_PROTO_TREE
NormBlockTree tree;
#else
static NormBlock* Next(NormBlock* b) {return b->next;}
NormBlock** table; NormBlock** table;
unsigned long hash_mask; unsigned long hash_mask;
#endif // if/else USE_PROTO_TREE
unsigned long range_max; // max range of blocks that can be buffered unsigned long range_max; // max range of blocks that can be buffered
unsigned long range; // zero if "block buffer" is empty unsigned long range; // zero if "block buffer" is empty
UINT32 fec_block_mask;
NormBlockId range_lo; NormBlockId range_lo;
NormBlockId range_hi; NormBlockId range_hi;
}; // end class NormBlockBuffer }; // end class NormBlockBuffer
#endif // _NORM_SEGMENT #endif // _NORM_SEGMENT

84
include/normSession.h
View File

@ -103,11 +103,18 @@ class NormSessionMgr
NormController* GetController() const {return controller;} NormController* GetController() const {return controller;}
void SetDataFreeFunction(NormDataObject::DataFreeFunctionHandle freeFunc)
{data_free_func = freeFunc;}
NormDataObject::DataFreeFunctionHandle GetDataFreeFunction() const
{return data_free_func;}
private: private:
ProtoTimerMgr& timer_mgr; ProtoTimerMgr& timer_mgr;
ProtoSocket::Notifier& socket_notifier; ProtoSocket::Notifier& socket_notifier;
ProtoChannel::Notifier* channel_notifier; ProtoChannel::Notifier* channel_notifier;
NormController* controller; NormController* controller;
NormDataObject::DataFreeFunctionHandle data_free_func;
class NormSession* top_session; // top of NormSession list class NormSession* top_session; // top of NormSession list
@ -161,6 +168,8 @@ class NormSession
}; };
// General methods // General methods
void SetNodeId(NormNodeId nodeId)
{local_node_id = nodeId;}
const NormNodeId& LocalNodeId() const {return local_node_id;} const NormNodeId& LocalNodeId() const {return local_node_id;}
bool Open(); bool Open();
void Close(); void Close();
@ -210,7 +219,7 @@ class NormSession
UINT16 GetTxPort() const; UINT16 GetTxPort() const;
bool SetRxPortReuse(bool enableReuse, bool SetRxPortReuse(bool enableReuse,
const char* rxAddress, // bind() to <rxAddress>/<sessionPort> const char* rxAddress = NULL, // bind() to <rxAddress>/<sessionPort>
const char* senderAddress = (const char*)0, // connect() to <senderAddress>/<senderPort> const char* senderAddress = (const char*)0, // connect() to <senderAddress>/<senderPort>
UINT16 senderPort = 0); UINT16 senderPort = 0);
@ -289,6 +298,7 @@ class NormSession
// flow control is imposed // flow control is imposed
double GetFlowControlDelay() const double GetFlowControlDelay() const
{ {
if (0.0 == flow_control_factor) return 0.0;
double fdelay = (flow_control_factor * (SenderGrtt() * (backoff_factor + 1))); double fdelay = (flow_control_factor * (SenderGrtt() * (backoff_factor + 1)));
return ((fdelay > 0.020) ? fdelay : 0.020); // minimum 20 msec flow control return ((fdelay > 0.020) ? fdelay : 0.020); // minimum 20 msec flow control
} }
@ -301,6 +311,13 @@ class NormSession
bool SetTxCacheBounds(NormObjectSize sizeMax, bool SetTxCacheBounds(NormObjectSize sizeMax,
unsigned long countMin, unsigned long countMin,
unsigned long countMax); unsigned long countMax);
// For NormSocket API extension support only
void SetServerListener(bool state)
{is_server_listener = state;}
bool IsServerListener() const
{return is_server_listener;}
void Notify(NormController::Event event, void Notify(NormController::Event event,
class NormNode* node, class NormNode* node,
class NormObject* object) class NormObject* object)
@ -313,7 +330,14 @@ class NormSession
NormMsg* GetMessageFromPool() {return message_pool.RemoveHead();} NormMsg* GetMessageFromPool() {return message_pool.RemoveHead();}
void ReturnMessageToPool(NormMsg* msg) {message_pool.Append(msg);} void ReturnMessageToPool(NormMsg* msg) {message_pool.Append(msg);}
void QueueMessage(NormMsg* msg); void QueueMessage(NormMsg* msg);
bool SendMessage(NormMsg& msg); enum MessageStatus
{
MSG_SEND_FAILED,
MSG_SEND_BLOCKED,
MSG_SEND_OK
};
MessageStatus SendMessage(NormMsg& msg);
void ActivateTimer(ProtoTimer& timer) {session_mgr.ActivateTimer(timer);} void ActivateTimer(ProtoTimer& timer) {session_mgr.ActivateTimer(timer);}
void SetUserData(const void* userData) void SetUserData(const void* userData)
@ -334,7 +358,8 @@ class NormSession
UINT32 bufferSpace, UINT32 bufferSpace,
UINT16 segmentSize, UINT16 segmentSize,
UINT16 numData, UINT16 numData,
UINT16 numParity); UINT16 numParity,
UINT8 fecId = 0);
void StopSender(); void StopSender();
void SetTxOnly(bool txOnly, bool connectToSessionAddress = false); void SetTxOnly(bool txOnly, bool connectToSessionAddress = false);
bool GetTxOnly() const bool GetTxOnly() const
@ -385,6 +410,9 @@ class NormSession
bool SenderSendCmd(const char* cmdBuffer, unsigned int cmdLength, bool robust); bool SenderSendCmd(const char* cmdBuffer, unsigned int cmdLength, bool robust);
void SenderCancelCmd(); void SenderCancelCmd();
void SenderSetSynStatus(bool state)
{syn_status = state;}
// robust factor // robust factor
void SetTxRobustFactor(int value) void SetTxRobustFactor(int value)
{tx_robust_factor = value;} {tx_robust_factor = value;}
@ -407,6 +435,13 @@ class NormSession
void SenderSetExtraParity(UINT16 extraParity) void SenderSetExtraParity(UINT16 extraParity)
{extra_parity = extraParity;} {extra_parity = extraParity;}
INT32 Difference(NormBlockId a, NormBlockId b) const
{return NormBlockId::Difference(a, b, fec_block_mask);}
int Compare(NormBlockId a, NormBlockId b) const
{return NormBlockId::Compare(a, b, fec_block_mask);}
void Increment(NormBlockId& b, UINT32 i = 1) const
{b.Increment(i, fec_block_mask);}
// EMCON Sender (useful when there are silent receivers) // EMCON Sender (useful when there are silent receivers)
// (NORM_INFO is redundantly sent) // (NORM_INFO is redundantly sent)
@ -431,6 +466,8 @@ class NormSession
} }
double SenderGrtt() const {return grtt_advertised;} double SenderGrtt() const {return grtt_advertised;}
void ResetGrttNotification()
{notify_on_grtt_update = true;}
void SenderSetGrtt(double grttValue) void SenderSetGrtt(double grttValue)
{ {
if (IsSender()) if (IsSender())
@ -482,13 +519,21 @@ class NormSession
unsigned long RemoteSenderBufferSize() const unsigned long RemoteSenderBufferSize() const
{return remote_sender_buffer_size;} {return remote_sender_buffer_size;}
bool InsertRemoteSender(NormSenderNode& sender);
void DeleteRemoteSender(NormSenderNode& senderNode); void DeleteRemoteSender(NormSenderNode& senderNode);
// Call this to do remote sender memory allocations ahead of time // Call this to do remote sender memory allocations ahead of time
bool PreallocateRemoteSender(UINT16 segmentSize, UINT16 numData, UINT16 numParity); bool PreallocateRemoteSender(unsigned int bufferSize,
UINT16 segmentSize,
UINT16 numData,
UINT16 numParity,
unsigned int streamBufferSize = 0);
void ReceiverSetUnicastNacks(bool state) {unicast_nacks = state;} void ReceiverSetUnicastNacks(bool state)
bool ReceiverGetUnicastNacks() const {return unicast_nacks;} {unicast_nacks = state;}
bool ReceiverGetUnicastNacks() const
{return unicast_nacks;}
void ReceiverSetSilent(bool state) void ReceiverSetSilent(bool state)
{receiver_silent = state;} {receiver_silent = state;}
@ -500,7 +545,7 @@ class NormSession
{return rcvr_ignore_info;} {return rcvr_ignore_info;}
// The default "rcvr_max_delay = -1" corresponds to typical // The default "rcvr_max_delay = -1" corresponds to typical
// operation where source data forpartially received FEC blocks // operation where source data for partially received FEC blocks
// are only provided to the app when buffer constraints require it. // are only provided to the app when buffer constraints require it.
// Otherwise, the "maxDelay" corresponds to the max number // Otherwise, the "maxDelay" corresponds to the max number
// of FEC blocks the receiver waits before passing partially // of FEC blocks the receiver waits before passing partially
@ -541,7 +586,7 @@ class NormSession
// Set default "max_pending_range" of NormObjects for reception // Set default "max_pending_range" of NormObjects for reception
void SetRxCacheMax(UINT16 maxCount) void SetRxCacheMax(UINT16 maxCount)
{rx_cache_count_max = maxCount;} {rx_cache_count_max = (maxCount > 0x7fff) ? 0x7fff : maxCount;}
UINT16 GetRxCacheMax() const UINT16 GetRxCacheMax() const
{return rx_cache_count_max;} {return rx_cache_count_max;}
@ -560,6 +605,7 @@ class NormSession
#endif // SIMULATE #endif // SIMULATE
void SetProbeCount(unsigned probeCount) {probe_count = probeCount;} void SetProbeCount(unsigned probeCount) {probe_count = probeCount;}
bool SenderQueueSquelch(NormObjectId objectId);
private: private:
// Only NormSessionMgr can create/delete sessions // Only NormSessionMgr can create/delete sessions
@ -607,12 +653,14 @@ class NormSession
UINT16 ccSequence); UINT16 ccSequence);
void AdjustRate(bool onResponse); void AdjustRate(bool onResponse);
void SetTxRateInternal(double txRate); // here, txRate is bytes/sec void SetTxRateInternal(double txRate); // here, txRate is bytes/sec
bool SenderQueueSquelch(NormObjectId objectId); //bool SenderQueueSquelch(NormObjectId objectId);
void SenderQueueFlush(); void SenderQueueFlush();
bool SenderQueueWatermarkFlush(); bool SenderQueueWatermarkFlush();
bool SenderBuildRepairAdv(NormCmdRepairAdvMsg& cmd); bool SenderBuildRepairAdv(NormCmdRepairAdvMsg& cmd);
void SenderUpdateGroupSize(); void SenderUpdateGroupSize();
bool SenderQueueAppCmd(); bool SenderQueueAppCmd();
// The following method is only used for NormSocket purposes
bool SenderSendAppCmd(const char* buffer, unsigned int length, const ProtoAddress& dst);
// Receiver message handling routines // Receiver message handling routines
void ReceiverHandleObjectMessage(const struct timeval& currentTime, void ReceiverHandleObjectMessage(const struct timeval& currentTime,
@ -660,6 +708,7 @@ class NormSession
double tx_rate; // bytes per second double tx_rate; // bytes per second
double tx_rate_min; double tx_rate_min;
double tx_rate_max; double tx_rate_max;
unsigned int tx_residual; // for NORM_CMD(CC)/NORM_DATA "packet pairing"
// Sender parameters and state // Sender parameters and state
@ -685,6 +734,7 @@ class NormSession
NormEncoder* encoder; NormEncoder* encoder;
UINT8 fec_id; UINT8 fec_id;
UINT8 fec_m; UINT8 fec_m;
INT32 fec_block_mask;
NormObjectId next_tx_object_id; NormObjectId next_tx_object_id;
unsigned int tx_cache_count_min; unsigned int tx_cache_count_min;
@ -761,6 +811,7 @@ class NormSession
char* cmd_buffer; char* cmd_buffer;
unsigned int cmd_length; unsigned int cmd_length;
ProtoTimer cmd_timer; ProtoTimer cmd_timer;
bool syn_status;
// Receiver parameters // Receiver parameters
bool is_receiver; bool is_receiver;
@ -778,6 +829,13 @@ class NormSession
NormSenderNode::SyncPolicy default_sync_policy; NormSenderNode::SyncPolicy default_sync_policy;
UINT16 rx_cache_count_max; UINT16 rx_cache_count_max;
// For NormSocket server-listener support
bool is_server_listener;
NormClientTree client_tree;
// API-specific state variables
bool notify_on_grtt_update;
// State for some experimental congestion control // State for some experimental congestion control
bool ecn_ignore_loss; bool ecn_ignore_loss;
bool cc_tolerate_loss; bool cc_tolerate_loss;

2
include/normVersion.h
View File

@ -36,6 +36,6 @@
#ifndef _NORM_VERSION #ifndef _NORM_VERSION
#define _NORM_VERSION #define _NORM_VERSION
#define VERSION "1.5r6" #define VERSION "1.5.7"
#endif // _NORM_VERSION #endif // _NORM_VERSION

9
makefiles/Makefile.common
View File

@ -178,6 +178,15 @@ normMsgr: $(MSGR_OBJ) libnorm.a $(LIBPROTO)
mkdir -p ../bin mkdir -p ../bin
cp $@ ../bin/$@ cp $@ ../bin/$@
# (normStreamer) message sender/receiver
STREAMER_SRC = $(EXAMPLE)/normStreamer.cpp
STREAMER_OBJ = $(STREAMER_SRC:.cpp=.o)
normStreamer: $(STREAMER_OBJ) libnorm.a $(LIBPROTO)
$(CC) $(CFLAGS) -o $@ $(STREAMER_OBJ) $(LDFLAGS) libnorm.a $(LIBPROTO) $(LIBS)
mkdir -p ../bin
cp $@ ../bin/$@
# (pcap2norm) - parses pcap (e.g. tcpdump) file and prints NORM trace # (pcap2norm) - parses pcap (e.g. tcpdump) file and prints NORM trace
PCAP_SRC = $(COMMON)/pcap2norm.cpp PCAP_SRC = $(COMMON)/pcap2norm.cpp
PCAP_OBJ = $(PCAP_SRC:.cpp=.o) PCAP_OBJ = $(PCAP_SRC:.cpp=.o)

9
makefiles/Makefile.linux
View File

@ -35,15 +35,18 @@ SYSTEM_LIBS = -ldl
# (We export these for other Makefiles as needed) # (We export these for other Makefiles as needed)
# #
SYSTEM_HAVES = -DLINUX -DECN_SUPPORT -DHAVE_IPV6 -DHAVE_GETLOGIN -D_FILE_OFFSET_BITS=64 -DHAVE_LOCKF \ SYSTEM_HAVES = -DLINUX -DECN_SUPPORT -DHAVE_IPV6 -DHAVE_GETLOGIN -D_FILE_OFFSET_BITS=64 -DHAVE_LOCKF \
-DHAVE_OLD_SIGNALHANDLER -DHAVE_DIRFD -DHAVE_ASSERT -DHAVE_OLD_SIGNALHANDLER -DHAVE_DIRFD -DHAVE_ASSERT -DNO_SCM_RIGHTS -DHAVE_SCHED -DUNIX \
-DUSE_SELECT -DUSE_TIMERFD -DUSE_EVENTFD -DHAVE_PSELECT
SYSTEM_SRC = ../protolib/src/linux/linuxCap.cpp SYSTEM_SRC = ../protolib/src/linux/linuxCap.cpp
SYSTEM = linux SYSTEM = linux
export CC = g++ export CC = g++
export SYSTEM_CFLAGS = -Wall -Wcast-align -pedantic -fPIC export SYSTEM_CFLAGS = -Wall -Wcast-align -pedantic -fPIC -Wno-variadic-macros
export SYSTEM_SOFLAGS = -shared -Wl,-soname,libnorm.so.1 export SYSTEM_SOFLAGS = -shared -Wl,-soname,libnorm.so.1
SYSTEM_SOEXT = so SYSTEM_SOEXT = so
export RANLIB = ranlib export RANLIB = ranlib

4
makefiles/android/AndroidManifest.xml
View File

@ -3,6 +3,6 @@
package="mil.navy.nrl.norm" package="mil.navy.nrl.norm"
android:versionCode="1" android:versionCode="1"
android:versionName="1.0"> android:versionName="1.0">
<uses-sdk android:minSdkVersion="9" <uses-sdk android:minSdkVersion="14"
android:targetSdkVersion="17"/> android:targetSdkVersion="23"/>
</manifest> </manifest>

2
makefiles/android/jni/Application.mk
View File

@ -1,3 +1,3 @@
APP_ABI := all APP_ABI := all
APP_PLATFORM := android-9 APP_PLATFORM := android-14
NDK_MODULE_PATH := ../..:../../protolib/ NDK_MODULE_PATH := ../..:../../protolib/

2
makefiles/android/project.properties
View File

@ -9,4 +9,4 @@
android.library=true android.library=true
# Project target. # Project target.
target=android-14 target=android-18

7
makefiles/win32/NormDll.vcxproj
View File

@ -1,5 +1,5 @@
<?xml version="1.0" encoding="utf-8"?> <?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003"> <Project DefaultTargets="Build" ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup Label="ProjectConfigurations"> <ItemGroup Label="ProjectConfigurations">
<ProjectConfiguration Include="Debug|Win32"> <ProjectConfiguration Include="Debug|Win32">
<Configuration>Debug</Configuration> <Configuration>Debug</Configuration>
@ -13,16 +13,17 @@
<PropertyGroup Label="Globals"> <PropertyGroup Label="Globals">
<ProjectGuid>{182006F3-188F-466E-89FE-8421C0478691}</ProjectGuid> <ProjectGuid>{182006F3-188F-466E-89FE-8421C0478691}</ProjectGuid>
<Keyword>Win32Proj</Keyword> <Keyword>Win32Proj</Keyword>
<WindowsTargetPlatformVersion>10.0.16299.0</WindowsTargetPlatformVersion>
</PropertyGroup> </PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" /> <Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration"> <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration">
<ConfigurationType>DynamicLibrary</ConfigurationType> <ConfigurationType>DynamicLibrary</ConfigurationType>
<PlatformToolset>v120</PlatformToolset> <PlatformToolset>v141</PlatformToolset>
<CharacterSet>MultiByte</CharacterSet> <CharacterSet>MultiByte</CharacterSet>
</PropertyGroup> </PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration"> <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
<ConfigurationType>DynamicLibrary</ConfigurationType> <ConfigurationType>DynamicLibrary</ConfigurationType>
<PlatformToolset>v120</PlatformToolset> <PlatformToolset>v141</PlatformToolset>
<CharacterSet>MultiByte</CharacterSet> <CharacterSet>MultiByte</CharacterSet>
</PropertyGroup> </PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" /> <Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />

7
makefiles/win32/NormLib.vcxproj
View File

@ -12,19 +12,22 @@
</ItemGroup> </ItemGroup>
<PropertyGroup Label="Globals"> <PropertyGroup Label="Globals">
<ProjectGuid>{D7B0023C-8798-4918-8DA0-05C9054D70B9}</ProjectGuid> <ProjectGuid>{D7B0023C-8798-4918-8DA0-05C9054D70B9}</ProjectGuid>
<WindowsTargetPlatformVersion>10.0.16299.0</WindowsTargetPlatformVersion>
</PropertyGroup> </PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" /> <Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration"> <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration">
<ConfigurationType>StaticLibrary</ConfigurationType> <ConfigurationType>StaticLibrary</ConfigurationType>
<PlatformToolset>v120_xp</PlatformToolset> <PlatformToolset>v141</PlatformToolset>
<UseOfMfc>false</UseOfMfc> <UseOfMfc>false</UseOfMfc>
<CharacterSet>MultiByte</CharacterSet> <CharacterSet>MultiByte</CharacterSet>
<CLRSupport>false</CLRSupport>
</PropertyGroup> </PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration"> <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
<ConfigurationType>StaticLibrary</ConfigurationType> <ConfigurationType>StaticLibrary</ConfigurationType>
<PlatformToolset>v120_xp</PlatformToolset> <PlatformToolset>v141</PlatformToolset>
<UseOfMfc>false</UseOfMfc> <UseOfMfc>false</UseOfMfc>
<CharacterSet>MultiByte</CharacterSet> <CharacterSet>MultiByte</CharacterSet>
<CLRSupport>false</CLRSupport>
</PropertyGroup> </PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" /> <Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
<ImportGroup Label="ExtensionSettings"> <ImportGroup Label="ExtensionSettings">

13
makefiles/win32/norm/norm.vcxproj
View File

@ -1,5 +1,5 @@
<?xml version="1.0" encoding="utf-8"?> <?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003"> <Project DefaultTargets="Build" ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup Label="ProjectConfigurations"> <ItemGroup Label="ProjectConfigurations">
<ProjectConfiguration Include="Debug|Win32"> <ProjectConfiguration Include="Debug|Win32">
<Configuration>Debug</Configuration> <Configuration>Debug</Configuration>
@ -12,17 +12,18 @@
</ItemGroup> </ItemGroup>
<PropertyGroup Label="Globals"> <PropertyGroup Label="Globals">
<ProjectGuid>{6E1308A6-D40F-489E-A4F1-40D859380D64}</ProjectGuid> <ProjectGuid>{6E1308A6-D40F-489E-A4F1-40D859380D64}</ProjectGuid>
<WindowsTargetPlatformVersion>10.0.16299.0</WindowsTargetPlatformVersion>
</PropertyGroup> </PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" /> <Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration"> <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType> <ConfigurationType>Application</ConfigurationType>
<PlatformToolset>v120</PlatformToolset> <PlatformToolset>v141</PlatformToolset>
<UseOfMfc>false</UseOfMfc> <UseOfMfc>false</UseOfMfc>
<CharacterSet>MultiByte</CharacterSet> <CharacterSet>MultiByte</CharacterSet>
</PropertyGroup> </PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration"> <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType> <ConfigurationType>Application</ConfigurationType>
<PlatformToolset>v120</PlatformToolset> <PlatformToolset>v141</PlatformToolset>
<UseOfMfc>false</UseOfMfc> <UseOfMfc>false</UseOfMfc>
<CharacterSet>MultiByte</CharacterSet> <CharacterSet>MultiByte</CharacterSet>
</PropertyGroup> </PropertyGroup>
@ -64,7 +65,7 @@
<AdditionalOptions>/vmg %(AdditionalOptions)</AdditionalOptions> <AdditionalOptions>/vmg %(AdditionalOptions)</AdditionalOptions>
<Optimization>Disabled</Optimization> <Optimization>Disabled</Optimization>
<AdditionalIncludeDirectories>..\..\..\include;..\..\..\protolib\include;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories> <AdditionalIncludeDirectories>..\..\..\include;..\..\..\protolib\include;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
<PreprocessorDefinitions>_DEBUG;PROTO_DEBUG;HAVE_IPV6;HAVE_ASSERT;WIN32;_CRT_SECURE_NO_WARNINGS;_WINDOWS;%(PreprocessorDefinitions)</PreprocessorDefinitions> <PreprocessorDefinitions>_CONSOLE,_DEBUG;PROTO_DEBUG;HAVE_IPV6;HAVE_ASSERT;WIN32;_CRT_SECURE_NO_WARNINGS;_WINDOWS;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<BasicRuntimeChecks>EnableFastChecks</BasicRuntimeChecks> <BasicRuntimeChecks>EnableFastChecks</BasicRuntimeChecks>
<RuntimeLibrary>MultiThreadedDebugDLL</RuntimeLibrary> <RuntimeLibrary>MultiThreadedDebugDLL</RuntimeLibrary>
<PrecompiledHeader /> <PrecompiledHeader />
@ -107,7 +108,7 @@
<Optimization>MaxSpeed</Optimization> <Optimization>MaxSpeed</Optimization>
<InlineFunctionExpansion>OnlyExplicitInline</InlineFunctionExpansion> <InlineFunctionExpansion>OnlyExplicitInline</InlineFunctionExpansion>
<AdditionalIncludeDirectories>..\..\..\include;..\..\..\protolib\include;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories> <AdditionalIncludeDirectories>..\..\..\include;..\..\..\protolib\include;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
<PreprocessorDefinitions>NDEBUG;PROTO_DEBUG;HAVE_IPV6;HAVE_ASSERT;WIN32;_WINDOWS;_CRT_SECURE_NO_WARNINGS;%(PreprocessorDefinitions)</PreprocessorDefinitions> <PreprocessorDefinitions>_CONSOLE,NDEBUG;PROTO_DEBUG;HAVE_IPV6;HAVE_ASSERT;WIN32;_WINDOWS;_CRT_SECURE_NO_WARNINGS;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<StringPooling>true</StringPooling> <StringPooling>true</StringPooling>
<RuntimeLibrary>MultiThreadedDLL</RuntimeLibrary> <RuntimeLibrary>MultiThreadedDLL</RuntimeLibrary>
<FunctionLevelLinking>true</FunctionLevelLinking> <FunctionLevelLinking>true</FunctionLevelLinking>
@ -139,7 +140,7 @@
<ItemGroup> <ItemGroup>
<ClCompile Include="..\..\..\src\common\normApp.cpp" /> <ClCompile Include="..\..\..\src\common\normApp.cpp" />
<ClCompile Include="..\..\..\src\common\normPostProcess.cpp" /> <ClCompile Include="..\..\..\src\common\normPostProcess.cpp" />
<ClCompile Include="..\win32PostProcess.cpp" /> <ClCompile Include="..\..\..\src\win32\win32PostProcess.cpp" />
</ItemGroup> </ItemGroup>
<ItemGroup> <ItemGroup>
<ProjectReference Include="..\..\..\protolib\makefiles\win32\Protokit.vcxproj"> <ProjectReference Include="..\..\..\protolib\makefiles\win32\Protokit.vcxproj">

7
makefiles/win32/normTest.vcxproj
View File

@ -1,5 +1,5 @@
<?xml version="1.0" encoding="utf-8"?> <?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003"> <Project DefaultTargets="Build" ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup Label="ProjectConfigurations"> <ItemGroup Label="ProjectConfigurations">
<ProjectConfiguration Include="Debug|Win32"> <ProjectConfiguration Include="Debug|Win32">
<Configuration>Debug</Configuration> <Configuration>Debug</Configuration>
@ -13,17 +13,18 @@
<PropertyGroup Label="Globals"> <PropertyGroup Label="Globals">
<ProjectGuid>{32CF83E0-D2E0-4E43-8F7B-72BD0AB64647}</ProjectGuid> <ProjectGuid>{32CF83E0-D2E0-4E43-8F7B-72BD0AB64647}</ProjectGuid>
<RootNamespace>normTest</RootNamespace> <RootNamespace>normTest</RootNamespace>
<WindowsTargetPlatformVersion>10.0.16299.0</WindowsTargetPlatformVersion>
</PropertyGroup> </PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" /> <Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration"> <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType> <ConfigurationType>Application</ConfigurationType>
<PlatformToolset>v120</PlatformToolset> <PlatformToolset>v141</PlatformToolset>
<UseOfMfc>false</UseOfMfc> <UseOfMfc>false</UseOfMfc>
<CharacterSet>MultiByte</CharacterSet> <CharacterSet>MultiByte</CharacterSet>
</PropertyGroup> </PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration"> <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType> <ConfigurationType>Application</ConfigurationType>
<PlatformToolset>v120</PlatformToolset> <PlatformToolset>v141</PlatformToolset>
<UseOfMfc>false</UseOfMfc> <UseOfMfc>false</UseOfMfc>
<CharacterSet>MultiByte</CharacterSet> <CharacterSet>MultiByte</CharacterSet>
</PropertyGroup> </PropertyGroup>

7
makefiles/win32/npc.vcxproj
View File

@ -1,5 +1,5 @@
<?xml version="1.0" encoding="utf-8"?> <?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003"> <Project DefaultTargets="Build" ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup Label="ProjectConfigurations"> <ItemGroup Label="ProjectConfigurations">
<ProjectConfiguration Include="Debug|Win32"> <ProjectConfiguration Include="Debug|Win32">
<Configuration>Debug</Configuration> <Configuration>Debug</Configuration>
@ -13,17 +13,18 @@
<PropertyGroup Label="Globals"> <PropertyGroup Label="Globals">
<ProjectGuid>{7BF4525B-9185-4296-A2BF-FFE77BB66EAF}</ProjectGuid> <ProjectGuid>{7BF4525B-9185-4296-A2BF-FFE77BB66EAF}</ProjectGuid>
<RootNamespace>normTest</RootNamespace> <RootNamespace>normTest</RootNamespace>
<WindowsTargetPlatformVersion>10.0.16299.0</WindowsTargetPlatformVersion>
</PropertyGroup> </PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" /> <Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration"> <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType> <ConfigurationType>Application</ConfigurationType>
<PlatformToolset>v120</PlatformToolset> <PlatformToolset>v141</PlatformToolset>
<UseOfMfc>false</UseOfMfc> <UseOfMfc>false</UseOfMfc>
<CharacterSet>MultiByte</CharacterSet> <CharacterSet>MultiByte</CharacterSet>
</PropertyGroup> </PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration"> <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType> <ConfigurationType>Application</ConfigurationType>
<PlatformToolset>v120</PlatformToolset> <PlatformToolset>v141</PlatformToolset>
<UseOfMfc>false</UseOfMfc> <UseOfMfc>false</UseOfMfc>
<CharacterSet>MultiByte</CharacterSet> <CharacterSet>MultiByte</CharacterSet>
</PropertyGroup> </PropertyGroup>

252
src/common/normApi.cpp
View File

@ -68,6 +68,13 @@ class NormInstance : public NormController
bool GetNextEvent(NormEvent* theEvent); bool GetNextEvent(NormEvent* theEvent);
bool SetCacheDirectory(const char* cachePath); bool SetCacheDirectory(const char* cachePath);
void SetAllocationFunctions(NormAllocFunctionHandle allocFunc,
NormFreeFunctionHandle freeFunc)
{
data_alloc_func = allocFunc;
session_mgr.SetDataFreeFunction(freeFunc);
}
void ReleasePreviousEvent(); void ReleasePreviousEvent();
bool NotifyQueueIsEmpty() const bool NotifyQueueIsEmpty() const
@ -159,13 +166,14 @@ class NormInstance : public NormController
ProtoDispatcher dispatcher; ProtoDispatcher dispatcher;
bool priority_boost; bool priority_boost;
NormSessionMgr session_mgr; NormSessionMgr session_mgr;
NormAllocFunctionHandle data_alloc_func;
private: private:
void ResetNotificationEvent() void ResetNotificationEvent()
{ {
#ifdef WIN32 #ifdef WIN32
if (0 == ResetEvent(notify_event)) if (0 == ResetEvent(notify_event))
PLOG(PL_ERROR, "NormInstance::GetNextEvent() ResetEvent error: %s\n", GetErrorString()); PLOG(PL_ERROR, "NormInstance::ResetNotificationEvent() ResetEvent error: %s\n", GetErrorString());
#else #else
char byte[32]; char byte[32];
while (read(notify_fd[0], byte, 32) > 0); // TBD - error check while (read(notify_fd[0], byte, 32) > 0); // TBD - error check
@ -212,13 +220,14 @@ NormInstance::NormInstance()
session_mgr(static_cast<ProtoTimerMgr&>(dispatcher), session_mgr(static_cast<ProtoTimerMgr&>(dispatcher),
static_cast<ProtoSocket::Notifier&>(dispatcher), static_cast<ProtoSocket::Notifier&>(dispatcher),
static_cast<ProtoChannel::Notifier*>(&dispatcher)), static_cast<ProtoChannel::Notifier*>(&dispatcher)),
previous_notification(NULL), rx_cache_path(NULL) data_alloc_func(NULL), previous_notification(NULL), rx_cache_path(NULL)
{ {
#ifdef WIN32 #ifdef WIN32
notify_event = NULL; notify_event = NULL;
#else #else
notify_fd[0] = notify_fd[1] = -1; notify_fd[0] = notify_fd[1] = -1;
#endif // if/else WIN32/UNIX #endif // if/else WIN32/UNIX
dispatcher.SetUserData(&session_mgr); // for debugging
session_mgr.SetController(static_cast<NormController*>(this)); session_mgr.SetController(static_cast<NormController*>(this));
} }
@ -374,7 +383,7 @@ void NormInstance::Notify(NormController::Event event,
{ {
NormDataObject* dataObj = static_cast<NormDataObject*>(object); NormDataObject* dataObj = static_cast<NormDataObject*>(object);
unsigned int dataLen = (unsigned int)(object->GetSize().GetOffset()); unsigned int dataLen = (unsigned int)(object->GetSize().GetOffset());
char* dataPtr = new char[dataLen]; char* dataPtr = (NULL != data_alloc_func) ? data_alloc_func(dataLen) : new char[dataLen];
if (NULL == dataPtr) if (NULL == dataPtr)
{ {
PLOG(PL_FATAL, "NormInstance::Notify(RX_OBJECT_NEW) new dataPtr error: %s\n", PLOG(PL_FATAL, "NormInstance::Notify(RX_OBJECT_NEW) new dataPtr error: %s\n",
@ -474,7 +483,7 @@ void NormInstance::PurgeObjectNotifications(NormObjectHandle objectHandle)
// TBD - check if event queue is emptied and reset event/fd // TBD - check if event queue is emptied and reset event/fd
} // end NormInstance::PurgeObjectNotifications() } // end NormInstance::PurgeObjectNotifications()
// Purge any notifications associated with a specific object // Purge any notifications associated with a specific remote sender node
void NormInstance::PurgeNodeNotifications(NormNodeHandle nodeHandle) void NormInstance::PurgeNodeNotifications(NormNodeHandle nodeHandle)
{ {
if (NORM_NODE_INVALID == nodeHandle) return; if (NORM_NODE_INVALID == nodeHandle) return;
@ -507,7 +516,7 @@ void NormInstance::PurgeNodeNotifications(NormNodeHandle nodeHandle)
// "Release" any previously-retained object or node handle // "Release" any previously-retained object or node handle
if (NORM_OBJECT_INVALID != previous_notification->event.object) if (NORM_OBJECT_INVALID != previous_notification->event.object)
((NormObject*)(previous_notification->event.object))->Release(); ((NormObject*)(previous_notification->event.object))->Release();
else if (NORM_NODE_INVALID != previous_notification->event.sender) else
((NormNode*)(previous_notification->event.sender))->Release(); ((NormNode*)(previous_notification->event.sender))->Release();
notify_pool.Append(previous_notification); notify_pool.Append(previous_notification);
previous_notification = NULL; previous_notification = NULL;
@ -649,8 +658,6 @@ bool NormInstance::GetNextEvent(NormEvent* theEvent)
return (NULL != n); return (NULL != n);
} // end NormInstance::GetNextEvent() } // end NormInstance::GetNextEvent()
bool NormInstance::WaitForEvent() bool NormInstance::WaitForEvent()
{ {
if (!dispatcher.IsThreaded()) if (!dispatcher.IsThreaded())
@ -924,6 +931,16 @@ NormDescriptor NormGetDescriptor(NormInstanceHandle instanceHandle)
} // end NormGetDescriptor() } // end NormGetDescriptor()
NORM_API_LINKAGE
void NormSetAllocationFunctions(NormInstanceHandle instanceHandle,
NormAllocFunctionHandle allocFunc,
NormFreeFunctionHandle freeFunc)
{
NormInstance* instance = (NormInstance*)instanceHandle;
instance->SetAllocationFunctions(allocFunc, freeFunc);
} // end NormSetAllocationFunctions()
// if "waitForEvent" is false, this is a non-blocking call // if "waitForEvent" is false, this is a non-blocking call
// (TBD) add a timeout option to this? // (TBD) add a timeout option to this?
@ -986,7 +1003,7 @@ NormSessionHandle NormCreateSession(NormInstanceHandle instanceHandle,
NormSession* session = NormSession* session =
instance->session_mgr.NewSession(sessionAddr, sessionPort, localNodeId); instance->session_mgr.NewSession(sessionAddr, sessionPort, localNodeId);
instance->dispatcher.ResumeThread(); instance->dispatcher.ResumeThread();
if (session) if (NULL != session)
return ((NormSessionHandle)session); return ((NormSessionHandle)session);
} }
return NORM_SESSION_INVALID; return NORM_SESSION_INVALID;
@ -1067,7 +1084,7 @@ NORM_API_LINKAGE
void NormCancelUserTimer(NormSessionHandle sessionHandle) void NormCancelUserTimer(NormSessionHandle sessionHandle)
{ {
NormInstance* instance = NormInstance::GetInstanceFromSession(sessionHandle); NormInstance* instance = NormInstance::GetInstanceFromSession(sessionHandle);
if (instance) if (NULL != instance)
{ {
if (instance->dispatcher.SuspendThread()) if (instance->dispatcher.SuspendThread())
{ {
@ -1090,6 +1107,38 @@ NormNodeId NormGetLocalNodeId(NormSessionHandle sessionHandle)
return NORM_NODE_NONE; return NORM_NODE_NONE;
} // end NormGetLocalNodeId() } // end NormGetLocalNodeId()
NORM_API_LINKAGE
bool NormGetAddress(NormSessionHandle sessionHandle,
char* addrBuffer,
unsigned int* bufferLen,
UINT16* port)
{
bool result = false;
NormInstance* instance = NormInstance::GetInstanceFromSession(sessionHandle);
if (NULL != instance)
{
if (instance->dispatcher.SuspendThread())
{
NormSession* session = (NormSession*)sessionHandle;
const ProtoAddress& sessionAddr = session->Address();
unsigned int addrLen = sessionAddr.GetLength();
if (addrBuffer && bufferLen && (addrLen <= *bufferLen))
{
memcpy(addrBuffer, sessionAddr.GetRawHostAddress(), addrLen);
result = true;
}
else if (NULL == addrBuffer)
{
result = true; // just a query for addrLen and/or port
}
if (bufferLen) *bufferLen = addrLen;
if (port) *port = sessionAddr.GetPort();
instance->dispatcher.ResumeThread();
}
}
return result;
} // end NormGetAddress()
NORM_API_LINKAGE NORM_API_LINKAGE
UINT16 NormGetRxPort(NormSessionHandle sessionHandle) UINT16 NormGetRxPort(NormSessionHandle sessionHandle)
{ {
@ -1146,22 +1195,44 @@ void NormSetTxOnly(NormSessionHandle sessionHandle,
} }
} // end NormSetTxOnly() } // end NormSetTxOnly()
// These functions are used internally by the NormSocket API extensiont
NORM_API_LINKAGE NORM_API_LINKAGE
bool NormChangeDestination(NormSessionHandle sessionHandle, void NormSetId(NormSessionHandle sessionHandle, NormNodeId normId)
const char* sessionAddress,
UINT16 sessionPort,
bool connectToSessionAddress)
{ {
// First, see if we can make a valid ProtoAddress
ProtoAddress dest;
if (!dest.ResolveFromString(sessionAddress))
return false;
dest.SetPort(sessionPort);
NormInstance* instance = NormInstance::GetInstanceFromSession(sessionHandle); NormInstance* instance = NormInstance::GetInstanceFromSession(sessionHandle);
if (instance && instance->dispatcher.SuspendThread()) if (instance && instance->dispatcher.SuspendThread())
{ {
NormSession* session = (NormSession*)sessionHandle; NormSession* session = (NormSession*)sessionHandle;
if (session) if (NULL != session) session->SetNodeId(normId);
instance->dispatcher.ResumeThread();
}
} // end NormSetId()
NORM_API_LINKAGE
bool NormChangeDestination(NormSessionHandle sessionHandle,
const char* sessionAddress,
UINT16 sessionPort,
bool connectToSessionAddress)
{
NormSession* session = (NormSession*)sessionHandle;
if (NULL == session) return false;
// First, see if we can make a valid ProtoAddress
ProtoAddress dest;
if (NULL != sessionAddress)
{
if (!dest.ResolveFromString(sessionAddress))
return false;
}
else
{
dest = session->Address();
}
dest.SetPort(sessionPort);
NormInstance* instance = NormInstance::GetInstanceFromSession(sessionHandle);
if (instance && instance->dispatcher.SuspendThread())
{
if (NULL != session)
{ {
session->SetAddress(dest); session->SetAddress(dest);
if (connectToSessionAddress) if (connectToSessionAddress)
@ -1172,6 +1243,46 @@ bool NormChangeDestination(NormSessionHandle sessionHandle,
return true; return true;
} // end NormChangeDestination() } // end NormChangeDestination()
NORM_API_LINKAGE
void NormSetServerListener(NormSessionHandle sessionHandle, bool state)
{
NormInstance* instance = NormInstance::GetInstanceFromSession(sessionHandle);
if (NULL != instance)
{
if (instance->dispatcher.SuspendThread())
{
NormSession* session = (NormSession*)sessionHandle;
if (session) session->SetServerListener(state);
instance->dispatcher.ResumeThread();
}
}
} // end NormSetServerListener()
NORM_API_LINKAGE
bool NormTransferSender(NormSessionHandle sessionHandle, NormNodeHandle senderHandle)
{
bool result = false;
NormInstance* dstInstance = NormInstance::GetInstanceFromSession(sessionHandle);
if (NULL != dstInstance)
{
if (dstInstance->dispatcher.SuspendThread())
{
NormInstance* srcInstance = NormInstance::GetInstanceFromNode(senderHandle);
if (srcInstance->dispatcher.SuspendThread())
{
NormSession* session = (NormSession*)sessionHandle;
NormSenderNode* sender = (NormSenderNode*)senderHandle;
if ((NULL != session) && (NULL != sender))
result = session->InsertRemoteSender(*sender);
srcInstance->dispatcher.ResumeThread();
}
dstInstance->dispatcher.ResumeThread();
}
}
return result;
} // end NormTransferSender()
NORM_API_LINKAGE NORM_API_LINKAGE
void NormSetRxPortReuse(NormSessionHandle sessionHandle, void NormSetRxPortReuse(NormSessionHandle sessionHandle,
bool enableReuse, bool enableReuse,
@ -1447,7 +1558,8 @@ bool NormStartSender(NormSessionHandle sessionHandle,
UINT32 bufferSpace, UINT32 bufferSpace,
UINT16 segmentSize, UINT16 segmentSize,
UINT16 numData, UINT16 numData,
UINT16 numParity) UINT16 numParity,
UINT8 fecId)
{ {
bool result = false; bool result = false;
NormInstance* instance = NormInstance::GetInstanceFromSession(sessionHandle); NormInstance* instance = NormInstance::GetInstanceFromSession(sessionHandle);
@ -1455,7 +1567,7 @@ bool NormStartSender(NormSessionHandle sessionHandle,
{ {
NormSession* session = (NormSession*)sessionHandle; NormSession* session = (NormSession*)sessionHandle;
if (session) if (session)
result = session->StartSender(sessionId, bufferSpace, segmentSize, numData, numParity); result = session->StartSender(sessionId, bufferSpace, segmentSize, numData, numParity, fecId);
else else
result = false; result = false;
instance->dispatcher.ResumeThread(); instance->dispatcher.ResumeThread();
@ -1596,18 +1708,16 @@ void NormSetGrttEstimate(NormSessionHandle sessionHandle,
NORM_API_LINKAGE NORM_API_LINKAGE
double NormGetGrttEstimate(NormSessionHandle sessionHandle) double NormGetGrttEstimate(NormSessionHandle sessionHandle)
{ {
NormInstance* instance = NormInstance::GetInstanceFromSession(sessionHandle); if (NORM_SESSION_INVALID != sessionHandle)
if (instance && instance->dispatcher.SuspendThread())
{ {
NormSession* session = (NormSession*)sessionHandle; NormSession* session = (NormSession*)sessionHandle;
if (session) session->SenderGrtt(); session->ResetGrttNotification();
instance->dispatcher.ResumeThread(); return (session->SenderGrtt());
} }
if (NORM_SESSION_INVALID != sessionHandle)
return (((NormSession*)sessionHandle)->SenderGrtt());
else else
{
return -1.0; return -1.0;
}
} // end NormGetGrttEstimate() } // end NormGetGrttEstimate()
NORM_API_LINKAGE NORM_API_LINKAGE
@ -1728,7 +1838,7 @@ NormObjectHandle NormDataEnqueue(NormSessionHandle sessionHandle,
{ {
NormObject* obj = NormObject* obj =
static_cast<NormObject*>(session->QueueTxData(dataPtr, dataLen, infoPtr, infoLen)); static_cast<NormObject*>(session->QueueTxData(dataPtr, dataLen, infoPtr, infoLen));
if (obj) objectHandle = (NormObjectHandle)obj; if (NULL != obj) objectHandle = (NormObjectHandle)obj;
} }
instance->dispatcher.ResumeThread(); instance->dispatcher.ResumeThread();
} }
@ -1803,6 +1913,8 @@ NORM_API_LINKAGE
unsigned int NormGetStreamBufferSegmentCount(unsigned int bufferBytes, UINT16 segmentSize, UINT16 blockSize) unsigned int NormGetStreamBufferSegmentCount(unsigned int bufferBytes, UINT16 segmentSize, UINT16 blockSize)
{ {
// This same computation is performed in NormStreamObject::Open() in "normObject.cpp" // This same computation is performed in NormStreamObject::Open() in "normObject.cpp"
// (Note the number of stream buffer segments is always smaller than the
// blockSize*pending_mask.GetSize())
unsigned int numBlocks = bufferBytes / (blockSize * segmentSize); unsigned int numBlocks = bufferBytes / (blockSize * segmentSize);
if (numBlocks < 2) numBlocks = 2; // NORM enforces a 2-block minimum buffer size if (numBlocks < 2) numBlocks = 2; // NORM enforces a 2-block minimum buffer size
return (numBlocks * blockSize); return (numBlocks * blockSize);
@ -1813,14 +1925,13 @@ unsigned int NormStreamWrite(NormObjectHandle streamHandle,
const char* buffer, const char* buffer,
unsigned int numBytes) unsigned int numBytes)
{ {
// TBD - the thread suspend/resume here may _not_ be sufficient. // Note: Since an underlying issue with ProtoDispatcher::SignalThread() had been resolved,
// we may need to do something to make sure the NORM thread // using ProtoDispatcher::SuspendThread() should be sufficient since the underlying
// wakes up right away (i.e. SignalThread() // protolib time scheduling, etc. code actually invokes SignalThread() on an
// Also so the same for flush, send command, and data/file enqueue calls! // as-needed basis. Thus, using SuspendThread() (lighter weight) should suffice
// (i.e. in any case where immediate data send action may be invoked)
unsigned int result = 0; unsigned int result = 0;
NormInstance* instance = NormInstance::GetInstanceFromObject(streamHandle); NormInstance* instance = NormInstance::GetInstanceFromObject(streamHandle);
if (instance && instance->dispatcher.SuspendThread()) if ((NULL != instance) && instance->dispatcher.SuspendThread())
{ {
NormStreamObject* stream = NormStreamObject* stream =
static_cast<NormStreamObject*>((NormObject*)streamHandle); static_cast<NormStreamObject*>((NormObject*)streamHandle);
@ -1836,7 +1947,7 @@ void NormStreamFlush(NormObjectHandle streamHandle,
NormFlushMode flushMode) NormFlushMode flushMode)
{ {
NormInstance* instance = NormInstance::GetInstanceFromObject(streamHandle); NormInstance* instance = NormInstance::GetInstanceFromObject(streamHandle);
if (instance && instance->dispatcher.SuspendThread()) if ((NULL != instance) && instance->dispatcher.SuspendThread())
{ {
NormStreamObject* stream = NormStreamObject* stream =
static_cast<NormStreamObject*>((NormObject*)streamHandle); static_cast<NormStreamObject*>((NormObject*)streamHandle);
@ -1908,7 +2019,7 @@ bool NormSetWatermark(NormSessionHandle sessionHandle,
NormObject* obj = (NormObject*)objectHandle; NormObject* obj = (NormObject*)objectHandle;
if (session && obj) if (session && obj)
{ {
// Purge any existing to NORM_TX_WATERMARK_COMPLETED notification to be safe // Purge any existing NORM_TX_WATERMARK_COMPLETED notifications to be safe
instance->PurgeNotifications(sessionHandle, NORM_TX_WATERMARK_COMPLETED); instance->PurgeNotifications(sessionHandle, NORM_TX_WATERMARK_COMPLETED);
// (segmentId doesn't matter for non-stream) // (segmentId doesn't matter for non-stream)
if (obj->IsStream()) if (obj->IsStream())
@ -1941,6 +2052,8 @@ bool NormResetWatermark(NormSessionHandle sessionHandle)
NormInstance* instance = NormInstance::GetInstanceFromSession(sessionHandle); NormInstance* instance = NormInstance::GetInstanceFromSession(sessionHandle);
if (instance && instance->dispatcher.SuspendThread()) if (instance && instance->dispatcher.SuspendThread())
{ {
// Purge any existing NORM_TX_WATERMARK_COMPLETED notifications to be safe
instance->PurgeNotifications(sessionHandle, NORM_TX_WATERMARK_COMPLETED);
NormSession* session = (NormSession*)sessionHandle; NormSession* session = (NormSession*)sessionHandle;
session->SenderResetWatermark(); session->SenderResetWatermark();
instance->dispatcher.ResumeThread(); instance->dispatcher.ResumeThread();
@ -1997,12 +2110,17 @@ NORM_API_LINKAGE
NormNodeHandle NormGetAckingNodeHandle(NormSessionHandle sessionHandle, NormNodeHandle NormGetAckingNodeHandle(NormSessionHandle sessionHandle,
NormNodeId nodeId) NormNodeId nodeId)
{ {
if (NORM_SESSION_INVALID != sessionHandle) if (NORM_SESSION_INVALID != sessionHandle)
{ {
NormSession* session = (NormSession*)sessionHandle; NormInstance* instance = NormInstance::GetInstanceFromSession(sessionHandle);
NormAckingNode* acker = session->SenderFindAckingNode(nodeId); if (instance && instance->dispatcher.SuspendThread())
if (NULL != acker) {
return ((NormNodeHandle)static_cast<NormNode*>(acker)); NormSession* session = (NormSession*)sessionHandle;
NormAckingNode* acker = session->SenderFindAckingNode(nodeId);
instance->dispatcher.ResumeThread();
if (NULL != acker)
return ((NormNodeHandle)static_cast<NormNode*>(acker));
}
} }
return NORM_NODE_INVALID; return NORM_NODE_INVALID;
} // end NormGetAckingNodeHandle() } // end NormGetAckingNodeHandle()
@ -2104,6 +2222,19 @@ void NormCancelCommand(NormSessionHandle sessionHandle)
} }
} // end NormCancelCommand() } // end NormCancelCommand()
NORM_API_LINKAGE
void NormSetSynStatus(NormSessionHandle sessionHandle, bool state)
{
NormInstance* instance = NormInstance::GetInstanceFromSession(sessionHandle);
if (instance && instance->dispatcher.SuspendThread())
{
NormSession* session = (NormSession*)sessionHandle;
session->SenderSetSynStatus(state);
instance->dispatcher.ResumeThread();
}
} // end NormSetSynStatus()
/** NORM Receiver Functions */ /** NORM Receiver Functions */
NORM_API_LINKAGE NORM_API_LINKAGE
@ -2138,8 +2269,6 @@ NORM_API_LINKAGE
void NormSetRxCacheLimit(NormSessionHandle sessionHandle, void NormSetRxCacheLimit(NormSessionHandle sessionHandle,
unsigned short countMax) unsigned short countMax)
{ {
// We limit this to 1/4 of the 16-bit NormObjectId space
if (countMax > 16384) countMax = 16384;
NormInstance* instance = NormInstance::GetInstanceFromSession(sessionHandle); NormInstance* instance = NormInstance::GetInstanceFromSession(sessionHandle);
if (instance && instance->dispatcher.SuspendThread()) if (instance && instance->dispatcher.SuspendThread())
{ {
@ -2290,6 +2419,25 @@ void NormNodeSetRxRobustFactor(NormNodeHandle nodeHandle,
} }
} // end NormNodeSetRxRobustFactor() } // end NormNodeSetRxRobustFactor()
NORM_API_LINKAGE
bool NormPreallocateRemoteSender(NormSessionHandle sessionHandle,
unsigned long bufferSize,
UINT16 segmentSize,
UINT16 numData,
UINT16 numParity,
unsigned int streamBufferSize)
{
bool result = false;
NormInstance* instance = NormInstance::GetInstanceFromSession(sessionHandle);
if (instance && instance->dispatcher.SuspendThread())
{
NormSession* session = (NormSession*)sessionHandle;
result = session->PreallocateRemoteSender(bufferSize, segmentSize, numData, numParity, streamBufferSize);
instance->dispatcher.ResumeThread();
}
return result;
} // end NormPreallocateRemoteSender()
NORM_API_LINKAGE NORM_API_LINKAGE
bool NormStreamRead(NormObjectHandle streamHandle, bool NormStreamRead(NormObjectHandle streamHandle,
char* buffer, char* buffer,
@ -2545,6 +2693,19 @@ char* NormDataDetachData(NormObjectHandle dataHandle)
return ptr; return ptr;
} // end NormDataDetachData() } // end NormDataDetachData()
NORM_API_LINKAGE
char* NormAlloc(size_t numBytes)
{
return new char[numBytes];
} // end NormAlloc()
NORM_API_LINKAGE
void NormFree(char* dataPtr)
{
if (NULL != dataPtr)
delete[] dataPtr;
} // end NormFree()
NORM_API_LINKAGE NORM_API_LINKAGE
NormNodeHandle NormObjectGetSender(NormObjectHandle objectHandle) NormNodeHandle NormObjectGetSender(NormObjectHandle objectHandle)
{ {
@ -2576,7 +2737,7 @@ bool NormNodeGetAddress(NormNodeHandle nodeHandle,
//NormInstance* instance = NormInstance::GetInstanceFromNode(nodeHandle); //NormInstance* instance = NormInstance::GetInstanceFromNode(nodeHandle);
//if (instance && instance->dispatcher.SuspendThread()) //if (instance && instance->dispatcher.SuspendThread())
{ {
NormNode* node = (NormSenderNode*)nodeHandle; NormNode* node = (NormNode*)nodeHandle;
const ProtoAddress& nodeAddr = node->GetAddress(); const ProtoAddress& nodeAddr = node->GetAddress();
unsigned int addrLen = nodeAddr.GetLength(); unsigned int addrLen = nodeAddr.GetLength();
if (addrBuffer && bufferLen && (addrLen <= *bufferLen)) if (addrBuffer && bufferLen && (addrLen <= *bufferLen))
@ -2623,6 +2784,7 @@ double NormNodeGetGrtt(NormNodeHandle nodeHandle)
if ((NULL != node) && (NormNode::SENDER == node->GetType())) if ((NULL != node) && (NormNode::SENDER == node->GetType()))
{ {
NormSenderNode* sender = static_cast<NormSenderNode*>(node); NormSenderNode* sender = static_cast<NormSenderNode*>(node);
sender->ResetGrttNotification();
return sender->GetGrttEstimate(); return sender->GetGrttEstimate();
} }
else else

183
src/common/normApp.cpp
View File

@ -1,5 +1,10 @@
// normApp.cpp - Command-line NORM application // normApp.cpp - Command-line NORM "demo" application
// Note this code does _not_ use the NORM API. It is kept as a demonstrator
// and test tool and serves as a performance comparison reference for testing
// the threaded NORM API code. So this should _not_ be used as a NORM
// programming example!!!!
#include "protokit.h" #include "protokit.h"
#include "normSession.h" #include "normSession.h"
@ -52,7 +57,6 @@ class NormApp : public NormController, public ProtoApp
NormObjectId GetObjectId() const NormObjectId GetObjectId() const
{return object_id;} {return object_id;}
private: private:
// ProtoTree::Item overrides // ProtoTree::Item overrides
const char* GetKey() const const char* GetKey() const
@ -62,9 +66,7 @@ class NormApp : public NormController, public ProtoApp
char file_path[PATH_MAX]; char file_path[PATH_MAX];
NormObjectId object_id; NormObjectId object_id;
}; }; // end class NormApp::FileCacheItem
void ShowHelp(); void ShowHelp();
void OnInputReady(); void OnInputReady();
@ -145,14 +147,17 @@ class NormApp : public NormController, public ProtoApp
UINT16 extra_parity; UINT16 extra_parity;
double backoff_factor; double backoff_factor;
double grtt_estimate; // initial grtt estimate double grtt_estimate; // initial grtt estimate
NormSession::ProbingMode grtt_probing_mode;
double group_size; double group_size;
unsigned long tx_buffer_size; // bytes unsigned long tx_buffer_size; // bytes
unsigned int tx_sock_buffer_size; unsigned int tx_sock_buffer_size;
unsigned long tx_cache_min; unsigned long tx_cache_min;
unsigned long tx_cache_max; unsigned long tx_cache_max;
NormObjectSize tx_cache_size; NormObjectSize tx_cache_size;
NormFileList tx_file_list; NormFileList tx_file_list;
char tx_file_name[PATH_MAX];
bool tx_file_info;
ProtoTree tx_file_cache; ProtoTree tx_file_cache;
bool tx_one_shot; bool tx_one_shot;
bool tx_ack_shot; bool tx_ack_shot;
@ -188,6 +193,7 @@ class NormApp : public NormController, public ProtoApp
bool rx_persistent; bool rx_persistent;
bool process_aborted_files; bool process_aborted_files;
bool preallocate_sender; bool preallocate_sender;
NormSenderNode::RepairBoundary repair_boundary;
// Debug parameters // Debug parameters
bool tracing; bool tracing;
@ -222,17 +228,16 @@ NormApp::NormApp()
cc_enable(false), ecn_mode(ECN_OFF), tolerate_loss(false), cc_enable(false), ecn_mode(ECN_OFF), tolerate_loss(false),
node_id(NORM_NODE_ANY), segment_size(1024), ndata(32), nparity(16), auto_parity(0), extra_parity(0), node_id(NORM_NODE_ANY), segment_size(1024), ndata(32), nparity(16), auto_parity(0), extra_parity(0),
backoff_factor(NormSession::DEFAULT_BACKOFF_FACTOR), grtt_estimate(NormSession::DEFAULT_GRTT_ESTIMATE), backoff_factor(NormSession::DEFAULT_BACKOFF_FACTOR), grtt_estimate(NormSession::DEFAULT_GRTT_ESTIMATE),
group_size(NormSession::DEFAULT_GSIZE_ESTIMATE), grtt_probing_mode(NormSession::PROBE_ACTIVE), group_size(NormSession::DEFAULT_GSIZE_ESTIMATE),
tx_buffer_size(1024*1024), tx_sock_buffer_size(0), tx_cache_min(8), tx_cache_max(256), tx_cache_size((UINT32)20*1024*1024), tx_buffer_size(1024*1024), tx_sock_buffer_size(0), tx_cache_min(8), tx_cache_max(256), tx_cache_size((UINT32)20*1024*1024),
tx_one_shot(false), tx_ack_shot(false), tx_file_queued(false), tx_file_info(true), tx_one_shot(false), tx_ack_shot(false), tx_file_queued(false),
tx_robust_factor(NormSession::DEFAULT_ROBUST_FACTOR), tx_object_interval(0.0), tx_repeat_count(0), tx_robust_factor(NormSession::DEFAULT_ROBUST_FACTOR), tx_object_interval(0.0), tx_repeat_count(0),
tx_repeat_interval(2.0), tx_repeat_clear(true), tx_requeue(0), tx_requeue_count(0), acking_node_list(NULL), tx_repeat_interval(2.0), tx_repeat_clear(true), tx_requeue(0), tx_requeue_count(0), acking_node_list(NULL),
acking_flushes(false), watermark_pending(false), rx_buffer_size(1024*1024), rx_sock_buffer_size(0), acking_flushes(false), watermark_pending(false), rx_buffer_size(1024*1024), rx_sock_buffer_size(0),
rx_cache_path(NULL), post_processor(NULL), unicast_nacks(false), silent_receiver(false), rx_cache_path(NULL), post_processor(NULL), unicast_nacks(false), silent_receiver(false),
low_delay(false), realtime(false), rx_robust_factor(NormSession::DEFAULT_ROBUST_FACTOR), rx_persistent(true), process_aborted_files(false), low_delay(false), realtime(false), rx_robust_factor(NormSession::DEFAULT_ROBUST_FACTOR), rx_persistent(true), process_aborted_files(false),
preallocate_sender(false), tracing(false), tx_loss(0.0), rx_loss(0.0) preallocate_sender(false), repair_boundary(NormSenderNode::BLOCK_BOUNDARY), tracing(false), tx_loss(0.0), rx_loss(0.0)
{ {
control_pipe.SetListener(this, &NormApp::OnControlEvent); control_pipe.SetListener(this, &NormApp::OnControlEvent);
control_pipe.SetNotifier(&GetSocketNotifier()); control_pipe.SetNotifier(&GetSocketNotifier());
@ -243,6 +248,8 @@ NormApp::NormApp()
interval_timer.SetInterval(0.0); interval_timer.SetInterval(0.0);
interval_timer.SetRepeat(0); interval_timer.SetRepeat(0);
tx_file_name[0] = '\0';
struct timeval currentTime; struct timeval currentTime;
ProtoSystemTime(currentTime); ProtoSystemTime(currentTime);
srand(currentTime.tv_usec); srand(currentTime.tv_usec);
@ -307,10 +314,12 @@ const char* const NormApp::cmd_list[] =
"+minput", // send message stream input "+minput", // send message stream input
"+moutput", // recv message stream output "+moutput", // recv message stream output
"+sendfile", // file/directory list to transmit "+sendfile", // file/directory list to transmit
"+info", // 'on' | 'off' to enable/disable file info (name) transmission (default = 'on')
"+interval", // delay time (sec) between files (0.0 sec default) "+interval", // delay time (sec) between files (0.0 sec default)
"+repeatcount", // How many times to repeat the file/directory list tx "+repeatcount", // How many times to repeat the file/directory list tx
"+rinterval", // Interval (sec) between file/directory list repeats "+rinterval", // Interval (sec) between file/directory list repeats
"+requeue", // <count> how many times files are retransmitted w/ same objId "+requeue", // <count> how many times files are retransmitted w/ same objId
"+boundary", // 'block' or 'file' to set NORM_REPAIR_BOUNDARY (default is 'block')
"-oneshot", // Transmit file(s), exiting upon TX_FLUSH_COMPLETED "-oneshot", // Transmit file(s), exiting upon TX_FLUSH_COMPLETED
"-ackshot", // Transmit file(s), exiting upon TX_WATERMARK_COMPLETED "-ackshot", // Transmit file(s), exiting upon TX_WATERMARK_COMPLETED
"-updatesOnly", // only send updated files on repeat transmission "-updatesOnly", // only send updated files on repeat transmission
@ -325,6 +334,7 @@ const char* const NormApp::cmd_list[] =
"+extra", // Number of extra FEC packets sent in response to repair requests "+extra", // Number of extra FEC packets sent in response to repair requests
"+backoff", // Backoff factor to use "+backoff", // Backoff factor to use
"+grtt", // Set sender's initial GRTT estimate "+grtt", // Set sender's initial GRTT estimate
"+probe", // {'active', 'passive' | 'none'} Set sender;s GRTT probing mode 'active' is default)
"+gsize", // Set sender's group size estimate "+gsize", // Set sender's group size estimate
"+txbuffer", // Size of sender's buffer "+txbuffer", // Size of sender's buffer
"+txsockbuffer", // tx socket buffer size "+txsockbuffer", // tx socket buffer size
@ -377,10 +387,12 @@ void NormApp::ShowHelp()
" +minput, // sender message stream input\n" " +minput, // sender message stream input\n"
" +moutput, // receiver message stream output\n" " +moutput, // receiver message stream output\n"
" +sendfile, // file/directory list to transmit\n" " +sendfile, // file/directory list to transmit\n"
" +info, // 'on' | 'off' to enable/disable file info (name) transmission (default = 'on')\n"
" +interval, // delay time (sec) between files (0.0 sec default)\n" " +interval, // delay time (sec) between files (0.0 sec default)\n"
" +repeatcount, // How many times to repeat the file/directory list tx\n" " +repeatcount, // How many times to repeat the file/directory list tx\n"
" +rinterval, // Interval (sec) between file/directory list repeats\n" " +rinterval, // Interval (sec) between file/directory list repeats\n"
" +requeue, // <count> how many times files are retransmitted w/ same objId\n" " +requeue, // <count> how many times files are retransmitted w/ same objId\n"
" +boundary // 'block' or 'file' to set NORM_REPAIR_BOUNDARY (default is 'block')\n"
" -oneshot, // Exit upon sender TX_FLUSH_COMPLETED event (sender exits after transmission)\n" " -oneshot, // Exit upon sender TX_FLUSH_COMPLETED event (sender exits after transmission)\n"
" -ackshot, // Exit upon sender TX_WATERMARK_COMPLETED event (sender exits after transmission)\n" " -ackshot, // Exit upon sender TX_WATERMARK_COMPLETED event (sender exits after transmission)\n"
" -updatesOnly, // only send updated files on repeat transmission\n" " -updatesOnly, // only send updated files on repeat transmission\n"
@ -395,6 +407,7 @@ void NormApp::ShowHelp()
" +extra, // Number of extra FEC packets sent in response to repair requests\n" " +extra, // Number of extra FEC packets sent in response to repair requests\n"
" +backoff, // Backoff factor to use\n" " +backoff, // Backoff factor to use\n"
" +grtt, // Set sender's initial GRTT estimate\n" " +grtt, // Set sender's initial GRTT estimate\n"
" +probe, // {'active', 'passive' | 'none'} Set sender;s GRTT probing mode 'active' is default)\n"
" +gsize, // Set sender's group size estimate\n" " +gsize, // Set sender's group size estimate\n"
" +txbuffer, // Size of sender's buffer\n" " +txbuffer, // Size of sender's buffer\n"
" +txcachebounds,// <countMin:countMax:sizeMax> limits on sender tx object caching\n" " +txcachebounds,// <countMin:countMax:sizeMax> limits on sender tx object caching\n"
@ -428,7 +441,7 @@ void NormApp::OnControlEvent(ProtoSocket& /*theSocket*/, ProtoSocket::Event theE
if (control_pipe.Recv(buffer, len)) if (control_pipe.Recv(buffer, len))
{ {
buffer[len] = '\0'; buffer[len] = '\0';
PLOG(PL_ERROR, "norm: received command \"%s\"\n", buffer); PLOG(PL_DEBUG, "norm: received command \"%s\"\n", buffer);
char* cmd = buffer; char* cmd = buffer;
char* val = NULL; char* val = NULL;
for (unsigned int i = 0; i < len; i++) for (unsigned int i = 0; i < len; i++)
@ -631,6 +644,8 @@ bool NormApp::OnCommand(const char* cmd, const char* val)
} }
bool result = session ? session->SetLoopback(loopTemp) : true; bool result = session ? session->SetLoopback(loopTemp) : true;
loopback = result ? loopTemp : loopback; loopback = result ? loopTemp : loopback;
if (loopback && (NULL != session) && session->Address().IsMulticast())
session->SetRxPortReuse(true);
if (!result) if (!result)
{ {
PLOG(PL_FATAL, "NormApp::OnCommand(loopback) error setting socket loopback!\n"); PLOG(PL_FATAL, "NormApp::OnCommand(loopback) error setting socket loopback!\n");
@ -828,15 +843,23 @@ bool NormApp::OnCommand(const char* cmd, const char* val)
// (This will allow the "send" command to be invoked at run-time via the ProtoPipe // (This will allow the "send" command to be invoked at run-time via the ProtoPipe
// remote control interface). // remote control interface).
} }
else if (!strncmp("info", cmd, len))
{
if (!strcmp("on", val))
tx_file_info = true;
else if (!strcmp("off", val))
tx_file_info = false;
else
{
PLOG(PL_FATAL, "NormApp::OnCommand(info) invalid argument!\n");
return false;
}
}
else if (!strncmp("interval", cmd, len)) else if (!strncmp("interval", cmd, len))
{ {
if (1 != sscanf(val, "%lf", &tx_object_interval)) if (1 != sscanf(val, "%lf", &tx_object_interval))
tx_object_interval = -1.0;
if (tx_object_interval < 0.0)
{ {
PLOG(PL_FATAL, "NormApp::OnCommand(interval) Invalid tx object interval: %s\n", PLOG(PL_FATAL, "NormApp::OnCommand(interval) Invalid tx object interval: %s\n", val);
val);
tx_object_interval = 0.0;
return false; return false;
} }
} }
@ -860,11 +883,27 @@ bool NormApp::OnCommand(const char* cmd, const char* val)
{ {
tx_requeue = tx_requeue_count = atoi(val); tx_requeue = tx_requeue_count = atoi(val);
} }
else if (!strncmp("boundary", cmd, len))
{
if (0 == strcmp("block", val))
{
repair_boundary = NormSenderNode::BLOCK_BOUNDARY;
}
else if (0 == strcmp("file", val))
{
repair_boundary = NormSenderNode::OBJECT_BOUNDARY;
}
else
{
PLOG(PL_FATAL, "NormApp::OnCommand(boundary) error: invalid repair boundary \"%s\"\n", val);
return false;
}
if (session) session->ReceiverSetDefaultRepairBoundary(repair_boundary);
}
else if (!strncmp("oneshot", cmd, len)) else if (!strncmp("oneshot", cmd, len))
{ {
tx_one_shot = true; tx_one_shot = true;
} }
else if (!strncmp("ackshot", cmd, len)) else if (!strncmp("ackshot", cmd, len))
{ {
tx_ack_shot = true; tx_ack_shot = true;
@ -998,6 +1037,22 @@ bool NormApp::OnCommand(const char* cmd, const char* val)
grtt_estimate = grttEstimate; grtt_estimate = grttEstimate;
if (session) session->SenderSetGrtt(grttEstimate); if (session) session->SenderSetGrtt(grttEstimate);
} }
else if (!strncmp("probe", cmd, len))
{
size_t valLen = strlen(val);
if (!strncmp("none", val, valLen))
grtt_probing_mode = NormSession::PROBE_NONE;
else if (!strncmp("passive", val, valLen))
grtt_probing_mode = NormSession::PROBE_PASSIVE;
else if (!strncmp("active", val, valLen))
grtt_probing_mode = NormSession::PROBE_ACTIVE;
else
{
PLOG(PL_FATAL, "NormApp::OnCommand(probe) invalid grtt probing mode!\n");
return false;
}
if (NULL != session) session->SetGrttProbingMode(grtt_probing_mode);
}
else if (!strncmp("gsize", cmd, len)) else if (!strncmp("gsize", cmd, len))
{ {
double groupSize = atof(val); double groupSize = atof(val);
@ -1028,7 +1083,11 @@ bool NormApp::OnCommand(const char* cmd, const char* val)
tx_cache_min = countMin; tx_cache_min = countMin;
tx_cache_max = countMax; tx_cache_max = countMax;
tx_cache_size = sizeMax; tx_cache_size = sizeMax;
if (session) session->SetTxCacheBounds(tx_cache_size, tx_cache_min, tx_cache_max); if (session)
{
session->SetTxCacheBounds(tx_cache_size, tx_cache_min, tx_cache_max);
session->SetRxCacheMax(tx_cache_max);
}
} }
else if (!strncmp("txrobustfactor", cmd, len)) else if (!strncmp("txrobustfactor", cmd, len))
{ {
@ -1393,7 +1452,7 @@ void NormApp::OnInputReady()
{ {
if (feof(input)) if (feof(input))
{ {
PLOG(PL_FATAL, "norm: input end-of-file.\n"); PLOG(PL_ALWAYS, "norm: input end-of-file.\n");
if (input_active) if (input_active)
{ {
#ifdef WIN32 #ifdef WIN32
@ -1461,9 +1520,9 @@ void NormApp::OnInputReady()
if (input_active) if (input_active)
{ {
#ifdef WIN32 #ifdef WIN32
ASSERT(0); // no Win32 support for stream i/o yet ASSERT(0); // no Win32 support for stream i/o yet
#else #else
dispatcher.RemoveGenericInput(fileno(input)); dispatcher.RemoveGenericInput(fileno(input));
#endif // if/else WIN32/UNIX #endif // if/else WIN32/UNIX
input_active = false; input_active = false;
} }
@ -1516,7 +1575,7 @@ void NormApp::Notify(NormController::Event event,
if ((msg_test || input) && (object == tx_stream)) if ((msg_test || input) && (object == tx_stream))
OnInputReady(); OnInputReady();
} }
else else if (tx_object_interval >= 0.0)
{ {
// Can queue a new object for transmission // Can queue a new object for transmission
if (interval_timer.IsActive()) interval_timer.Deactivate(); if (interval_timer.IsActive()) interval_timer.Deactivate();
@ -1535,9 +1594,18 @@ void NormApp::Notify(NormController::Event event,
PLOG(PL_DEBUG, "NormApp::Notify(TX_FLUSH_COMPLETED) ...\n"); PLOG(PL_DEBUG, "NormApp::Notify(TX_FLUSH_COMPLETED) ...\n");
if (tx_one_shot) if (tx_one_shot)
{ {
PLOG(PL_FATAL, "norm: transmit flushing completed, exiting.\n"); PLOG(PL_ALWAYS, "norm: transmit flushing completed, exiting.\n");
Stop(); Stop();
} }
else if (tx_object_interval < 0.0)
{
// Can queue a new object for transmission
if (interval_timer.IsActive()) interval_timer.Deactivate();
if (interval_timer.GetInterval() > 0.0)
ActivateTimer(interval_timer);
else
OnIntervalTimeout(interval_timer);
}
break; break;
case TX_WATERMARK_COMPLETED: case TX_WATERMARK_COMPLETED:
@ -1547,7 +1615,7 @@ void NormApp::Notify(NormController::Event event,
watermark_pending = false; // enable new watermark to be set watermark_pending = false; // enable new watermark to be set
if (tx_ack_shot) if (tx_ack_shot)
{ {
PLOG(PL_ERROR, "norm: transmit flushing completed, exiting.\n"); PLOG(PL_ALWAYS, "norm: transmit flushing completed, exiting.\n");
Stop(); Stop();
} }
break; break;
@ -1954,12 +2022,20 @@ void NormApp::Notify(NormController::Event event,
} // end NormApp::Notify() } // end NormApp::Notify()
bool NormApp::OnIntervalTimeout(ProtoTimer& /*theTimer*/) bool NormApp::OnIntervalTimeout(ProtoTimer& theTimer)
{ {
char fileName[PATH_MAX]; char fileName[PATH_MAX];
if (tx_file_list.GetNextFile(fileName)) if (('\0' != tx_file_name[0]) || tx_file_list.GetNextFile(fileName))
{ {
tx_repeat_clear = true; tx_repeat_clear = true;
// The "tx_file_name" is used to cache the path of the current file for enqueue
// re-attempt upon possible flow control NormSession::QueueTxFile() failure
if ('\0' == tx_file_name[0])
strcpy(tx_file_name, fileName);
else
strcpy(fileName, tx_file_name); // used cached tx_file_name for enqueue re-attempt
// 1) Build up the full path name of the file // 1) Build up the full path name of the file
char pathName[PATH_MAX]; char pathName[PATH_MAX];
tx_file_list.GetCurrentBasePath(pathName); tx_file_list.GetCurrentBasePath(pathName);
@ -1978,9 +2054,6 @@ bool NormApp::OnIntervalTimeout(ProtoTimer& /*theTimer*/)
if (PROTO_PATH_DELIMITER == fileNameInfo[i]) if (PROTO_PATH_DELIMITER == fileNameInfo[i])
fileNameInfo[i] = '/'; fileNameInfo[i] = '/';
} }
char temp[PATH_MAX];
strncpy(temp, fileNameInfo, len);
temp[len] = '\0';
FileCacheItem* fileCacheItem = NULL; FileCacheItem* fileCacheItem = NULL;
NormFileObject* obj = NULL; NormFileObject* obj = NULL;
@ -1994,22 +2067,44 @@ bool NormApp::OnIntervalTimeout(ProtoTimer& /*theTimer*/)
bool requeueSuccess = false; bool requeueSuccess = false;
if (NULL != obj) requeueSuccess = session->RequeueTxObject(obj); if (NULL != obj) requeueSuccess = session->RequeueTxObject(obj);
if (!requeueSuccess) if (!requeueSuccess)
{
PLOG(PL_ERROR, "norm warning: requeue attempt exceeded configured tx cache bounds!\n"); PLOG(PL_ERROR, "norm warning: requeue attempt exceeded configured tx cache bounds!\n");
// TBD - remove file from tx_file_cache???
}
// RequeueTxObject() is not subject to flow control since the object is already buffered
tx_file_name[0] = '\0'; // reset so next file will be fetched
} }
} }
if (NULL == obj) if (NULL == obj)
{ {
// It's a new object (or requeue failure) // This makes sure it's not an empty file
if (NULL == (obj = session->QueueTxFile(fileName, fileNameInfo, (UINT16)len))) // (TBD - provide for empty files to be sent)
if (0 == NormFile::GetSize(fileName))
{ {
PLOG(PL_ERROR, "NormApp::OnIntervalTimeout() Error queuing tx file: %s\n", PLOG(PL_WARN, "norm warning: empty file \"%s\" will not be transmitted\n");
fileName); tx_file_name[0] = '\0'; // reset so next file will be fetched
// Wait a second, then try the next file in the list return OnIntervalTimeout(theTimer);
}
// It's a new object
const char* infoPtr = tx_file_info ? fileNameInfo : NULL;
UINT16 infoLen = tx_file_info ? (UINT16)len : 0;
if (NULL == (obj = session->QueueTxFile(fileName, infoPtr, infoLen)))
{
PLOG(PL_DEBUG, "NormApp::OnIntervalTimeout() error queuing tx file: %s\n", fileName);
// The code currently assumes that QueueTxFile() failed because of
// flow control and so we simply return and wait for a TX_QUEUE_EMPTY notification
// to trigger a re-attempt to enqueue the file name in "tx_file_name"
if (interval_timer.IsActive()) interval_timer.Deactivate(); if (interval_timer.IsActive()) interval_timer.Deactivate();
interval_timer.SetInterval(1.0); // Old behavior was to wait a second and go to next file
ActivateTimer(interval_timer); // (commented out here)
//interval_timer.SetInterval(1.0);
//ActivateTimer(interval_timer);
return false; return false;
} }
else
{
tx_file_name[0] = '\0'; // reset so next file will be fetched
}
if (0 != tx_requeue) if (0 != tx_requeue)
{ {
if (NULL == fileCacheItem) if (NULL == fileCacheItem)
@ -2094,7 +2189,7 @@ bool NormApp::OnIntervalTimeout(ProtoTimer& /*theTimer*/)
else else
{ {
tx_repeat_clear = true; tx_repeat_clear = true;
PLOG(PL_FATAL, "norm: End of tx file list reached.\n"); PLOG(PL_ALWAYS, "norm: End of tx file list reached.\n");
if (tx_ack_shot && !watermark_pending && tx_file_queued) if (tx_ack_shot && !watermark_pending && tx_file_queued)
{ {
session->SenderSetWatermark(tx_last_object_id, session->SenderSetWatermark(tx_last_object_id,
@ -2169,18 +2264,26 @@ bool NormApp::OnStartup(int argc, const char*const* argv)
session->SetRxLoss(rx_loss); session->SetRxLoss(rx_loss);
session->SetTTL(ttl); session->SetTTL(ttl);
session->SetLoopback(loopback); session->SetLoopback(loopback);
if (loopback && session->Address().IsMulticast())
session->SetRxPortReuse(true);
if (NULL != interface_name) if (NULL != interface_name)
session->SetMulticastInterface(interface_name); session->SetMulticastInterface(interface_name);
session->SetEcnSupport(ECN_OFF != ecn_mode, ECN_ONLY == ecn_mode, tolerate_loss); session->SetEcnSupport(ECN_OFF != ecn_mode, ECN_ONLY == ecn_mode, tolerate_loss);
// We turn off flow control when no NACKing is happening
// (TBD - make flow control a command-line option)
if (silent_receiver)
session->SetFlowControl(0.0);
if (msg_test || input || !tx_file_list.IsEmpty()) if (msg_test || input || !tx_file_list.IsEmpty())
{ {
NormObjectId baseId = (unsigned short)(rand() * (65535.0/ (double)RAND_MAX)); NormObjectId baseId = (unsigned short)(rand() * (65535.0/ (double)RAND_MAX));
session->SenderSetBaseObjectId(baseId); session->SenderSetBaseObjectId(baseId);
session->SetCongestionControl(cc_enable); session->SetCongestionControl(cc_enable);
session->SetBackoffFactor(backoff_factor); session->SetBackoffFactor(backoff_factor);
//session->SenderSetGrtt(grtt_estimate); session->SenderSetGrtt(grtt_estimate);
session->SetGrttProbingMode(grtt_probing_mode);
session->SenderSetGroupSize(group_size); session->SenderSetGroupSize(group_size);
session->SetTxRobustFactor(tx_robust_factor); session->SetTxRobustFactor(tx_robust_factor);
session->SetTxCacheBounds(tx_cache_size, tx_cache_min, tx_cache_max); session->SetTxCacheBounds(tx_cache_size, tx_cache_min, tx_cache_max);
@ -2226,11 +2329,12 @@ bool NormApp::OnStartup(int argc, const char*const* argv)
if (output || rx_cache_path) if (output || rx_cache_path)
{ {
// StartReceiver(bufferMax (per-sender)) // StartReceiver(bufferMax (per-sender))
session->SetRxCacheMax(tx_cache_max);
session->ReceiverSetUnicastNacks(unicast_nacks); session->ReceiverSetUnicastNacks(unicast_nacks);
session->ReceiverSetSilent(silent_receiver); session->ReceiverSetSilent(silent_receiver);
if (preallocate_sender) if (preallocate_sender)
{ {
if (!session->PreallocateRemoteSender(segment_size, ndata, nparity)) if (!session->PreallocateRemoteSender(rx_buffer_size, segment_size, ndata, nparity))
{ {
PLOG(PL_FATAL, "NormApp::OnStartup() remote sender preallocation error!\n"); PLOG(PL_FATAL, "NormApp::OnStartup() remote sender preallocation error!\n");
session_mgr.Destroy(); session_mgr.Destroy();
@ -2244,6 +2348,7 @@ bool NormApp::OnStartup(int argc, const char*const* argv)
if (realtime) if (realtime)
session->RcvrSetRealtime(true); session->RcvrSetRealtime(true);
session->SetRxRobustFactor(rx_robust_factor); session->SetRxRobustFactor(rx_robust_factor);
session->ReceiverSetDefaultRepairBoundary(repair_boundary);
if (!session->StartReceiver(rx_buffer_size)) if (!session->StartReceiver(rx_buffer_size))
{ {
PLOG(PL_FATAL, "NormApp::OnStartup() start receiver error!\n"); PLOG(PL_FATAL, "NormApp::OnStartup() start receiver error!\n");

3
src/common/normFile.cpp
View File

@ -188,8 +188,7 @@ void NormFile::Unlock()
flock(fd, LOCK_UN); flock(fd, LOCK_UN);
#else #else
#ifdef HAVE_LOCKF #ifdef HAVE_LOCKF
int ret = lockf(fd, F_ULOCK, 0); lockf(fd, F_ULOCK, 0);
ret = 0; // done to avoid compiler warning (should optimize out)
#endif // HAVE_LOCKF #endif // HAVE_LOCKF
#endif // if/elseHAVE_FLOCK #endif // if/elseHAVE_FLOCK
fchmod(fd, 0640); fchmod(fd, 0640);

46
src/common/normMessage.cpp
View File

@ -7,7 +7,6 @@ NormHeaderExtension::NormHeaderExtension()
} }
NormMsg::NormMsg() NormMsg::NormMsg()
: length(8), header_length(8), header_length_base(8) : length(8), header_length(8), header_length_base(8)
{ {
@ -92,6 +91,13 @@ bool NormMsg::InitFromBuffer(UINT16 msgLength)
} }
} // end NormMsg::InitFromBuffer() } // end NormMsg::InitFromBuffer()
void NormMsg::Display() const
{
const unsigned char* ptr = (const unsigned char*)buffer;
for (unsigned int i = 0; i < length; i++)
PLOG(PL_ALWAYS, "%02x", *ptr++);
} // end NormMsg::Display()
bool NormCmdCCMsg::GetCCNode(NormNodeId nodeId, bool NormCmdCCMsg::GetCCNode(NormNodeId nodeId,
UINT8& flags, UINT8& flags,
UINT8& rtt, UINT8& rtt,
@ -150,10 +156,10 @@ bool NormRepairRequest::AppendRepairItem(UINT8 fecId,
{ {
if (RANGES == form) if (RANGES == form)
PLOG(PL_TRACE, "NormRepairRequest::AppendRepairItem-Range(fecId>%d obj>%hu blk>%lu seg>%hu) ...\n", PLOG(PL_TRACE, "NormRepairRequest::AppendRepairItem-Range(fecId>%d obj>%hu blk>%lu seg>%hu) ...\n",
fecId, (UINT16)objectId, (UINT32)blockId, (UINT16)symbolId); fecId, (UINT16)objectId, (unsigned long)blockId.GetValue(), (UINT16)symbolId);
else else
PLOG(PL_TRACE, "NormRepairRequest::AppendRepairItem(fecId>%d obj>%hu blk>%lu seg>%hu) ...\n", PLOG(PL_TRACE, "NormRepairRequest::AppendRepairItem(fecId>%d obj>%hu blk>%lu seg>%hu) ...\n",
fecId, (UINT16)objectId, (UINT32)blockId, (UINT16)symbolId); fecId, (UINT16)objectId, (unsigned long)blockId.GetValue(), (UINT16)symbolId);
ASSERT(NormPayloadId::IsValid(fecId)); ASSERT(NormPayloadId::IsValid(fecId));
UINT16 itemLength = RepairItemLength(fecId); UINT16 itemLength = RepairItemLength(fecId);
@ -164,7 +170,7 @@ bool NormRepairRequest::AppendRepairItem(UINT8 fecId,
((UINT8*)ptr)[RESERVED_OFFSET] = 0; // 1 byte ((UINT8*)ptr)[RESERVED_OFFSET] = 0; // 1 byte
((UINT16*)ptr)[OBJ_ID_OFFSET] = htons((UINT16)objectId); // 2 bytes ((UINT16*)ptr)[OBJ_ID_OFFSET] = htons((UINT16)objectId); // 2 bytes
NormPayloadId payloadId(fecId, fecM, ptr + 1); NormPayloadId payloadId(fecId, fecM, ptr + 1);
payloadId.SetFecPayloadId(blockId, symbolId, blockLen); payloadId.SetFecPayloadId(blockId.GetValue(), symbolId, blockLen);
length += itemLength; length += itemLength;
return true; return true;
} }
@ -186,8 +192,8 @@ bool NormRepairRequest::AppendRepairRange(UINT8 fecId,
UINT16 endSymbolId) UINT16 endSymbolId)
{ {
PLOG(PL_TRACE, "NormRepairRequest::AppendRepairRange(%hu:%lu:%hu->%hu:%lu:%hu) ...\n", PLOG(PL_TRACE, "NormRepairRequest::AppendRepairRange(%hu:%lu:%hu->%hu:%lu:%hu) ...\n",
(UINT16)startObjectId, (UINT32)startBlockId, (UINT16)startSymbolId, (UINT16)startObjectId, (unsigned long)startBlockId.GetValue(), (UINT16)startSymbolId,
(UINT16)endObjectId, (UINT32)endBlockId, (UINT16)endSymbolId); (UINT16)endObjectId, (unsigned long)endBlockId.GetValue(), (UINT16)endSymbolId);
// Note a "range" is two repair items // Note a "range" is two repair items
UINT16 rangeLength = RepairRangeLength(fecId); UINT16 rangeLength = RepairRangeLength(fecId);
if (buffer_len >= (ITEM_LIST_OFFSET+length+rangeLength)) if (buffer_len >= (ITEM_LIST_OFFSET+length+rangeLength))
@ -198,14 +204,14 @@ bool NormRepairRequest::AppendRepairRange(UINT8 fecId,
((UINT8*)ptr)[RESERVED_OFFSET] = 0; ((UINT8*)ptr)[RESERVED_OFFSET] = 0;
((UINT16*)ptr)[OBJ_ID_OFFSET] = htons((UINT16)startObjectId); ((UINT16*)ptr)[OBJ_ID_OFFSET] = htons((UINT16)startObjectId);
NormPayloadId startId(fecId, fecM, ptr + 1); NormPayloadId startId(fecId, fecM, ptr + 1);
startId.SetFecPayloadId(startBlockId, startSymbolId, startBlockLen); startId.SetFecPayloadId(startBlockId.GetValue(), startSymbolId, startBlockLen);
// range end // range end
ptr += (rangeLength/8); // advance ptr to second part of repair range ptr += (rangeLength/8); // advance ptr to second part of repair range
((UINT8*)ptr)[FEC_ID_OFFSET] = fecId; ((UINT8*)ptr)[FEC_ID_OFFSET] = fecId;
((UINT8*)ptr)[RESERVED_OFFSET] = 0; ((UINT8*)ptr)[RESERVED_OFFSET] = 0;
((UINT16*)ptr)[OBJ_ID_OFFSET] = htons((UINT16)endObjectId); ((UINT16*)ptr)[OBJ_ID_OFFSET] = htons((UINT16)endObjectId);
NormPayloadId endId(fecId, fecM, ptr + 1); NormPayloadId endId(fecId, fecM, ptr + 1);
endId.SetFecPayloadId(endBlockId, endSymbolId, endBlockLen); endId.SetFecPayloadId(endBlockId.GetValue(), endSymbolId, endBlockLen);
length += rangeLength; length += rangeLength;
return true; return true;
} }
@ -230,7 +236,7 @@ bool NormRepairRequest::AppendErasureCount(UINT8 fecId,
((UINT8*)ptr)[RESERVED_OFFSET] = 0; ((UINT8*)ptr)[RESERVED_OFFSET] = 0;
((UINT16*)ptr)[OBJ_ID_OFFSET] = htons((UINT16)objectId); ((UINT16*)ptr)[OBJ_ID_OFFSET] = htons((UINT16)objectId);
NormPayloadId erasureId(fecId, fecM, ptr + 1); NormPayloadId erasureId(fecId, fecM, ptr + 1);
erasureId.SetFecPayloadId(blockId, erasureCount, blockLen); erasureId.SetFecPayloadId(blockId.GetValue(), erasureCount, blockLen);
length += itemLength; length += itemLength;
return true; return true;
} }
@ -319,10 +325,10 @@ NormRepairRequest::Iterator::Iterator(const NormRepairRequest& theRequest, UINT8
} }
// For erasure requests, symbolId is loaded with erasureCount // For erasure requests, symbolId is loaded with erasureCount
bool NormRepairRequest::Iterator::NextRepairItem(NormObjectId* objectId, UINT16 NormRepairRequest::Iterator::NextRepairItem(NormObjectId* objectId,
NormBlockId* blockId, NormBlockId* blockId,
UINT16* blockLen, UINT16* blockLen,
UINT16* symbolId) UINT16* symbolId)
{ {
UINT8 fecId; // for check UINT8 fecId; // for check
UINT16 itemLength = request.RetrieveRepairItem(fec_m, offset, &fecId, objectId, blockId, blockLen, symbolId); UINT16 itemLength = request.RetrieveRepairItem(fec_m, offset, &fecId, objectId, blockId, blockLen, symbolId);
@ -334,12 +340,8 @@ bool NormRepairRequest::Iterator::NextRepairItem(NormObjectId* objectId,
return false; return false;
} }
offset += itemLength; offset += itemLength;
return true;
}
else
{
return false;
} }
return itemLength;
} // end NormRepairRequest::Iterator::NextRepairItem() } // end NormRepairRequest::Iterator::NextRepairItem()
@ -366,18 +368,18 @@ void NormRepairRequest::Log(UINT8 fecId, UINT8 fecM) const
while (iterator.NextRepairItem(&objId, &blkId, &blkLen, &segId)) while (iterator.NextRepairItem(&objId, &blkId, &blkLen, &segId))
{ {
if (FlagIsSet(SEGMENT)) if (FlagIsSet(SEGMENT))
PLOG(PL_ALWAYS, "RepairItem> %hu:%lu:%hu", (UINT16)objId, (UINT32)blkId, (UINT16)segId); PLOG(PL_ALWAYS, "RepairItem> %hu:%lu:%hu", (UINT16)objId, (unsigned long)blkId.GetValue(), (UINT16)segId);
else if (FlagIsSet(BLOCK)) else if (FlagIsSet(BLOCK))
PLOG(PL_ALWAYS, "RepairItem> %hu:%lu", (UINT16)objId, (UINT32)blkId, (UINT16)segId); PLOG(PL_ALWAYS, "RepairItem> %hu:%lu", (UINT16)objId, (unsigned long)blkId.GetValue(), (UINT16)segId);
else else
PLOG(PL_ALWAYS, "RepairItem> %hu", (UINT16)objId); PLOG(PL_ALWAYS, "RepairItem> %hu", (UINT16)objId);
if (form == RANGES) if (form == RANGES)
{ {
iterator.NextRepairItem(&objId, &blkId, &blkLen, &segId); iterator.NextRepairItem(&objId, &blkId, &blkLen, &segId);
if (FlagIsSet(SEGMENT)) if (FlagIsSet(SEGMENT))
PLOG(PL_ALWAYS, " -> %hu:%lu:%hu", (UINT16)objId, (UINT32)blkId, (UINT16)segId); PLOG(PL_ALWAYS, " -> %hu:%lu:%hu", (UINT16)objId, (unsigned long)blkId.GetValue(), (UINT16)segId);
else if (FlagIsSet(BLOCK)) else if (FlagIsSet(BLOCK))
PLOG(PL_ALWAYS, " -> %hu:%lu", (UINT16)objId, (UINT32)blkId, (UINT16)segId); PLOG(PL_ALWAYS, " -> %hu:%lu", (UINT16)objId, (unsigned long)blkId.GetValue(), (UINT16)segId);
else else
PLOG(PL_ALWAYS, " -> %hu", (UINT16)objId); PLOG(PL_ALWAYS, " -> %hu", (UINT16)objId);
} }

774
src/common/normNode.cpp
View File

File diff suppressed because it is too large Load Diff

1586
src/common/normObject.cpp
View File

File diff suppressed because it is too large Load Diff

121
src/common/normPrecode.cpp
View File

@ -1,7 +1,16 @@
#include "protoApp.h" #include "protoApp.h"
#include "normFile.h" #include "normFile.h"
// Commment this #define out to use new, faster RS8 codec instead
// (TBD - provide option use 16-bit Reed Solomon for large block sizes?)
//#define USE_MDP_FEC
#ifdef USE_MDP_FEC
#include "normEncoderMDP.h"
#else
#include "normEncoderRS8.h" #include "normEncoderRS8.h"
#endif // if/else USE_MDP_FEC
#include <sys/types.h> // for BYTE_ORDER macro #include <sys/types.h> // for BYTE_ORDER macro
#include <stdlib.h> // for atoi() #include <stdlib.h> // for atoi()
@ -48,8 +57,8 @@ class NormPrecodeApp : public ProtoApp
{ {
case 8: case 8:
{ {
long* outPtr = (long*)&result; UINT32* outPtr = (UINT32*)&result;
long* inPtr = (long*)&offset; UINT32* inPtr = (UINT32*)&offset;
outPtr[0] = ntohl(inPtr[1]); outPtr[0] = ntohl(inPtr[1]);
outPtr[1] = ntohl(inPtr[0]); outPtr[1] = ntohl(inPtr[0]);
break; break;
@ -559,7 +568,11 @@ bool NormPrecodeApp::Encode()
InitInterleaver(numOutputSegments); InitInterleaver(numOutputSegments);
// 1) Init our FEC encoder // 1) Init our FEC encoder
#ifdef USE_MDP_FEC
NormEncoderMDP encoder;
#else
NormEncoderRS8 encoder; NormEncoderRS8 encoder;
#endif // if/else USE_MDP_FEC
if (!encoder.Init(num_data, num_parity, dataSegmentSize)) // 4 CRC bytes are _not_ encoded if (!encoder.Init(num_data, num_parity, dataSegmentSize)) // 4 CRC bytes are _not_ encoded
{ {
PLOG(PL_FATAL, "npc: error initializing FEC encoder\n"); PLOG(PL_FATAL, "npc: error initializing FEC encoder\n");
@ -638,7 +651,7 @@ bool NormPrecodeApp::Encode()
strncpy(metaData+8, ptr, segment_size - 12); strncpy(metaData+8, ptr, segment_size - 12);
// 2) Read "in_file" segments, encode, and output to "out_file" // 2) Read "in_file" segments, encode, and output to "out_file"
PLOG(PL_FATAL, "npc: encoding file ... (progress: 0%%)"); PLOG(PL_ALWAYS, "npc: encoding file ... (progress: 0%%)");
// State to track/display encoding progress // State to track/display encoding progress
NormFile::Offset progressThreshold = numOutputSegments / 100; NormFile::Offset progressThreshold = numOutputSegments / 100;
double progressIncrement = 100.0; double progressIncrement = 100.0;
@ -659,59 +672,55 @@ bool NormPrecodeApp::Encode()
{ {
NormFile::Offset interleaverOffset = ComputeInterleaverOffset(outputSegmentId, numOutputSegments); NormFile::Offset interleaverOffset = ComputeInterleaverOffset(outputSegmentId, numOutputSegments);
char* segment = useBuffering ? (iBuffer + (interleaverOffset % interleaverBytes)) : iBuffer; char* segment = useBuffering ? (iBuffer + (interleaverOffset % interleaverBytes)) : iBuffer;
if (parityReady)
switch (parityReady)
{ {
case false: // D) Output parity segment for this block
char* pvec = parityVec[num_parity - parityCount];
memcpy(segment, pvec, dataSegmentSize);
if (0 == --parityCount)
{ {
// Read and encode a segment memset(parityVec[0], 0, num_parity * segment_size);
inputSegmentId++; parityReady = false;
if (1 == inputSegmentId) blockId++;
}
}
else
{
// Read and encode a segment
inputSegmentId++;
if (1 == inputSegmentId)
{
// A) Segment '0' is the meta-data segment
memcpy(segment, metaData, dataSegmentSize);
}
else
{
// B) Read in data portion of next "segment"
unsigned int bytesToRead;
if (inputSegmentId != numInputSegments)
{ {
// A) Segment '0' is the meta-data segment bytesToRead = dataSegmentSize;
memcpy(segment, metaData, dataSegmentSize);
} }
else else
{ {
// B) Read in data portion of next "segment" memset(segment, 0, dataSegmentSize);
unsigned int bytesToRead; bytesToRead = lastFecSegSize;
if (inputSegmentId != numInputSegments)
{
bytesToRead = dataSegmentSize;
}
else
{
memset(segment, 0, dataSegmentSize);
bytesToRead = lastFecSegSize;
}
if (in_file.Read(segment, bytesToRead) != bytesToRead)
{
PLOG(PL_FATAL, "\nnpc: unexpected error reading input file: %s\n", GetErrorString());
return false;
}
} }
// C) Encode and check for parity readiness if (in_file.Read(segment, bytesToRead) != bytesToRead)
encoder.Encode(outputSegmentId, segment, parityVec);
unsigned int numData = (blockId != lastBlockId) ? fecBlockSize : lastBlockSize;
if (numData == ++parityCount)
{ {
parityCount = num_parity; PLOG(PL_FATAL, "\nnpc: unexpected error reading input file: %s\n", GetErrorString());
parityReady = true; return false;
} }
break;
} }
case true: // C) Encode and check for parity readiness
//TRACE("outputSegmentId:%lu\n", outputSegmentId);
encoder.Encode(outputSegmentId % fecBlockSize, segment, parityVec);
unsigned int numData = (blockId != lastBlockId) ? fecBlockSize : lastBlockSize;
if (numData == ++parityCount)
{ {
// D) Output parity segment for this block parityCount = num_parity;
char* pvec = parityVec[num_parity - parityCount]; parityReady = true;
memcpy(segment, pvec, dataSegmentSize);
if (0 == --parityCount)
{
memset(parityVec[0], 0, num_parity * segment_size);
parityReady = false;
blockId++;
}
break;
} }
} }
// E) Calculate and add CRC32 checksum to each "segment" // E) Calculate and add CRC32 checksum to each "segment"
@ -760,17 +769,17 @@ bool NormPrecodeApp::Encode()
if (++progressCounter >= progressThreshold) if (++progressCounter >= progressThreshold)
{ {
if (progressPercent < 9) if (progressPercent < 9)
PLOG(PL_FATAL, "\b\b\b%d%%)", progressPercent + 1); PLOG(PL_ALWAYS, "\b\b\b%d%%)", progressPercent + 1);
else if (progressPercent < 99) else if (progressPercent < 99)
PLOG(PL_FATAL, "\b\b\b\b%d%%)", progressPercent + 1); PLOG(PL_ALWAYS, "\b\b\b\b%d%%)", progressPercent + 1);
if (progressPercent < 99) progressPercent++; if (progressPercent < 99) progressPercent++;
progressCounter = 0; progressCounter = 0;
} }
} }
if (progressPercent < 10) if (progressPercent < 10)
PLOG(PL_FATAL, "\b\b\b100%%)\n"); PLOG(PL_ALWAYS, "\b\b\b100%%)\n");
else else
PLOG(PL_FATAL, "\b\b\b\b100%%)\n"); PLOG(PL_ALWAYS, "\b\b\b\b100%%)\n");
in_file.Close(); in_file.Close();
out_file.Close(); out_file.Close();
@ -815,7 +824,11 @@ bool NormPrecodeApp::Decode()
InitInterleaver(numInputSegments); InitInterleaver(numInputSegments);
// 2) init FEC decoder // 2) init FEC decoder
#ifdef USE_MDP_FEC
NormDecoderMDP decoder;
#else
NormDecoderRS8 decoder; NormDecoderRS8 decoder;
#endif // if/else USE_MDP_FEC
unsigned int dataSegmentSize = segment_size - 4; // leaves space for our CRC unsigned int dataSegmentSize = segment_size - 4; // leaves space for our CRC
if (!decoder.Init(num_data, num_parity, dataSegmentSize)) if (!decoder.Init(num_data, num_parity, dataSegmentSize))
{ {
@ -894,7 +907,7 @@ bool NormPrecodeApp::Decode()
// Read and decode each block in "in_file" // Read and decode each block in "in_file"
NormFile::Offset fecBlockId = 0; NormFile::Offset fecBlockId = 0;
NormFile::Offset outFileSize; NormFile::Offset outFileSize = 0;
NormFile::Offset lastInterleaverBlockId = numInputSegments / interleaver_size; NormFile::Offset lastInterleaverBlockId = numInputSegments / interleaver_size;
NormFile::Offset lastInterleaverBytes = (numInputSegments % interleaver_size) * segment_size; NormFile::Offset lastInterleaverBytes = (numInputSegments % interleaver_size) * segment_size;
@ -1084,9 +1097,9 @@ bool NormPrecodeApp::Decode()
if (++progressCounter >= progressThreshold) if (++progressCounter >= progressThreshold)
{ {
if (progressPercent < 9) if (progressPercent < 9)
PLOG(PL_FATAL, "\b\b\b%d%%)", progressPercent + 1); PLOG(PL_ALWAYS, "\b\b\b%d%%)", progressPercent + 1);
else if (progressPercent < 99) else if (progressPercent < 99)
PLOG(PL_FATAL, "\b\b\b\b%d%%)", progressPercent + 1); PLOG(PL_ALWAYS, "\b\b\b\b%d%%)", progressPercent + 1);
if (progressPercent < 99) progressPercent++; if (progressPercent < 99) progressPercent++;
progressCounter = 0; progressCounter = 0;
} }
@ -1094,9 +1107,9 @@ bool NormPrecodeApp::Decode()
} // end while (inputSegmentId < numInputSegments) } // end while (inputSegmentId < numInputSegments)
if (progressPercent < 10) if (progressPercent < 10)
PLOG(PL_FATAL, "\b\b\b100%%)\n"); PLOG(PL_ALWAYS, "\b\b\b100%%)\n");
else else
PLOG(PL_FATAL, "\b\b\b\b100%%)\n"); PLOG(PL_ALWAYS, "\b\b\b\b100%%)\n");
// Cleanup, cleanup // Cleanup, cleanup
delete[] iBuffer; delete[] iBuffer;

303
src/common/normSegment.cpp
View File

@ -6,7 +6,6 @@ NormSegmentPool::NormSegmentPool()
{ {
} }
NormSegmentPool::~NormSegmentPool() NormSegmentPool::~NormSegmentPool()
{ {
Destroy(); Destroy();
@ -282,7 +281,6 @@ bool NormBlock::TxUpdate(NormSegmentId nextId, NormSegmentId lastId,
UINT16 numData, UINT16 numParity, UINT16 erasureCount) UINT16 numData, UINT16 numParity, UINT16 erasureCount)
{ {
bool increasedRepair = false; bool increasedRepair = false;
if (nextId < numData) if (nextId < numData)
{ {
// Explicit data repair request // Explicit data repair request
@ -314,6 +312,8 @@ bool NormBlock::TxUpdate(NormSegmentId nextId, NormSegmentId lastId,
} }
else else
{ {
// TBD - double-check this ... not sure this is exactly right
// (may need to always do explicit repair here?)
// Use any remaining fresh parity ... // Use any remaining fresh parity ...
if (parity_count < parityAvailable) if (parity_count < parityAvailable)
{ {
@ -342,7 +342,7 @@ bool NormBlock::HandleSegmentRequest(NormSegmentId nextId, NormSegmentId lastId,
UINT16 numData, UINT16 numParity, UINT16 erasureCount) UINT16 numData, UINT16 numParity, UINT16 erasureCount)
{ {
PLOG(PL_TRACE, "NormBlock::HandleSegmentRequest() blk>%lu seg>%hu:%hu erasures:%hu\n", PLOG(PL_TRACE, "NormBlock::HandleSegmentRequest() blk>%lu seg>%hu:%hu erasures:%hu\n",
(UINT32)blk_id, (UINT16)nextId, (UINT16)lastId, erasureCount); (unsigned long)blk_id.GetValue(), (UINT16)nextId, (UINT16)lastId, erasureCount);
bool increasedRepair = false; bool increasedRepair = false;
if (nextId < numData) if (nextId < numData)
{ {
@ -375,6 +375,8 @@ bool NormBlock::HandleSegmentRequest(NormSegmentId nextId, NormSegmentId lastId,
} }
else else
{ {
// TBD - double-check this. It may not be exactly right
// (may need to alwayds do explicitr repair here)
// Use any remaining fresh parity ... // Use any remaining fresh parity ...
if (parity_count < parityAvailable) if (parity_count < parityAvailable)
{ {
@ -399,15 +401,16 @@ bool NormBlock::HandleSegmentRequest(NormSegmentId nextId, NormSegmentId lastId,
return increasedRepair; return increasedRepair;
} // end NormBlock::HandleSegmentRequest() } // end NormBlock::HandleSegmentRequest()
// (TBD) this should return true is something is appending, false otherwise // (TBD) this should return true if something is appended, false otherwise
bool NormBlock::AppendRepairAdv(NormCmdRepairAdvMsg& cmd, bool NormBlock::AppendRepairAdv(NormCmdRepairAdvMsg& cmd,
NormObjectId objectId, NormObjectId objectId,
bool repairInfo, bool repairInfo,
UINT8 fecId, UINT8 fecId,
UINT8 fecM, UINT8 fecM,
UINT16 numData, UINT16 numData,
UINT16 segmentSize) UINT16 payloadMax)
{ {
bool requestAppended = false;
NormRepairRequest req; NormRepairRequest req;
req.SetFlag(NormRepairRequest::SEGMENT); req.SetFlag(NormRepairRequest::SEGMENT);
if (repairInfo) req.SetFlag(NormRepairRequest::INFO); if (repairInfo) req.SetFlag(NormRepairRequest::INFO);
@ -447,12 +450,14 @@ bool NormBlock::AppendRepairAdv(NormCmdRepairAdvMsg& cmd,
{ {
if (0 == cmd.PackRepairRequest(req)) if (0 == cmd.PackRepairRequest(req))
{ {
prevForm = NormRepairRequest::INVALID;
PLOG(PL_WARN, "NormBlock::AppendRepairAdv() warning: full msg\n"); PLOG(PL_WARN, "NormBlock::AppendRepairAdv() warning: full msg\n");
break; break;
} }
requestAppended = true;
} }
req.SetForm(form); req.SetForm(form);
cmd.AttachRepairRequest(req, segmentSize); // (TBD) error check cmd.AttachRepairRequest(req, payloadMax); // (TBD) error check
prevForm = form; prevForm = form;
} }
switch(form) switch(form)
@ -480,9 +485,11 @@ bool NormBlock::AppendRepairAdv(NormCmdRepairAdvMsg& cmd,
{ {
if (0 == cmd.PackRepairRequest(req)) if (0 == cmd.PackRepairRequest(req))
PLOG(PL_WARN, "NormBlock::AppendRepairAdv() warning: full msg\n"); PLOG(PL_WARN, "NormBlock::AppendRepairAdv() warning: full msg\n");
else
requestAppended = true;
} }
} }
return true; return requestAppended;
} // end NormBlock::AppendRepairAdv() } // end NormBlock::AppendRepairAdv()
NormObjectSize NormBlock::GetBytesPending(UINT16 numData, NormObjectSize NormBlock::GetBytesPending(UINT16 numData,
@ -521,8 +528,9 @@ bool NormBlock::AppendRepairRequest(NormNackMsg& nack,
UINT16 numParity, UINT16 numParity,
NormObjectId objectId, NormObjectId objectId,
bool pendingInfo, bool pendingInfo,
UINT16 segmentSize) UINT16 payloadMax)
{ {
bool requestAppended = false;
NormSegmentId nextId = 0; NormSegmentId nextId = 0;
NormSegmentId endId; NormSegmentId endId;
if (erasure_count > numParity) if (erasure_count > numParity)
@ -580,11 +588,13 @@ bool NormBlock::AppendRepairRequest(NormNackMsg& nack,
{ {
if (0 == nack.PackRepairRequest(req)) if (0 == nack.PackRepairRequest(req))
{ {
prevForm = NormRepairRequest::INVALID; // so we don't re-attempt pack
PLOG(PL_WARN, "NormBlock::AppendRepairRequest() warning: full NACK msg\n"); PLOG(PL_WARN, "NormBlock::AppendRepairRequest() warning: full NACK msg\n");
break; break;
} }
requestAppended = true;
} }
nack.AttachRepairRequest(req, segmentSize); // (TBD) error check nack.AttachRepairRequest(req, payloadMax); // (TBD) error check
req.SetForm(form); req.SetForm(form);
prevForm = form; prevForm = form;
} }
@ -614,8 +624,10 @@ bool NormBlock::AppendRepairRequest(NormNackMsg& nack,
{ {
if (0 == nack.PackRepairRequest(req)) if (0 == nack.PackRepairRequest(req))
PLOG(PL_WARN, "NormBlock::AppendRepairRequest() warning: full NACK msg\n"); PLOG(PL_WARN, "NormBlock::AppendRepairRequest() warning: full NACK msg\n");
else
requestAppended = true;
} }
return true; return requestAppended;
} // end NormBlock::AppendRepairRequest() } // end NormBlock::AppendRepairRequest()
NormBlockPool::NormBlockPool() NormBlockPool::NormBlockPool()
@ -671,7 +683,12 @@ void NormBlockPool::Destroy()
} // end NormBlockPool::Destroy() } // end NormBlockPool::Destroy()
NormBlockBuffer::NormBlockBuffer() NormBlockBuffer::NormBlockBuffer()
: table((NormBlock**)NULL), range_max(0), range(0) #ifdef USE_PROTO_TREE
:
#else
: table((NormBlock**)NULL),
#endif // if/else USE_PROTO_TREE
range_max(0), range(0), fec_block_mask(0)
{ {
} }
@ -680,16 +697,17 @@ NormBlockBuffer::~NormBlockBuffer()
Destroy(); Destroy();
} }
bool NormBlockBuffer::Init(unsigned long rangeMax, unsigned long tableSize) bool NormBlockBuffer::Init(unsigned long rangeMax, unsigned long tableSize, UINT32 fecBlockMask)
{ {
if (table) Destroy(); Destroy();
// Make sure tableSize is greater than 0 and 2^n // Make sure tableSize is greater than 0 and 2^n
if (!rangeMax || !tableSize) if (!rangeMax || !tableSize)
{ {
PLOG(PL_FATAL, "NormBlockBuffer::Init() bad range(%lu) or tableSize(%lu)\n", PLOG(PL_FATAL, "NormBlockBuffer::Init() bad range(%lu) or tableSize(%lu)\n",
rangeMax, tableSize); rangeMax, tableSize);
return false; return false;
} }
#ifndef USE_PROTO_TREE
if (0 != (tableSize & 0x07)) tableSize = (tableSize >> 3) + 1; if (0 != (tableSize & 0x07)) tableSize = (tableSize >> 3) + 1;
if (!(table = new NormBlock*[tableSize])) if (!(table = new NormBlock*[tableSize]))
{ {
@ -698,14 +716,38 @@ bool NormBlockBuffer::Init(unsigned long rangeMax, unsigned long tableSize)
} }
memset(table, 0, tableSize*sizeof(char*)); memset(table, 0, tableSize*sizeof(char*));
hash_mask = tableSize - 1; hash_mask = tableSize - 1;
#endif // !USE_PROTO_TREE
range_max = rangeMax; range_max = rangeMax;
range = 0; range = 0;
fec_block_mask = fecBlockMask;
return true; return true;
} // end NormBlockBuffer::Init() } // end NormBlockBuffer::Init()
#ifdef USE_PROTO_TREE
void NormBlockBuffer::Destroy() void NormBlockBuffer::Destroy()
{ {
NormBlock* block;
while((block = Find(range_lo)))
{
PLOG(PL_ERROR, "NormBlockBuffer::Destroy() buffer not empty!?\n");
Remove(block);
delete block;
}
range_max = range = 0; range_max = range = 0;
} // end NormBlockBuffer::Destroy()
NormBlock* NormBlockBuffer::Find(const NormBlockId& blockId) const
{
if ((0 == range) || (Compare(blockId, range_lo) < 0) || (Compare(blockId, range_hi) > 0))
return NULL;
else
return tree.Find(blockId.GetValuePtr(), 8*sizeof(UINT32));
} // end NormBlockBuffer::Find()
#else
void NormBlockBuffer::Destroy()
{
if (table) if (table)
{ {
NormBlock* block; NormBlock* block;
@ -717,18 +759,19 @@ void NormBlockBuffer::Destroy()
} }
delete []table; delete []table;
table = (NormBlock**)NULL; table = (NormBlock**)NULL;
range_max = 0;
} }
range_max = range = 0;
} // end NormBlockBuffer::Destroy() } // end NormBlockBuffer::Destroy()
NormBlock* NormBlockBuffer::Find(const NormBlockId& blockId) const NormBlock* NormBlockBuffer::Find(const NormBlockId& blockId) const
{ {
if (range) if (range)
{ {
if ((blockId < range_lo) || (blockId > range_hi)) //if ((blockId < range_lo) || (blockId > range_hi))
if ((Compare(blockId, range_lo) < 0) || (Compare(blockId, range_hi) > 0))
return (NormBlock*)NULL; return (NormBlock*)NULL;
NormBlock* theBlock = table[((UINT32)blockId) & hash_mask]; NormBlock* theBlock = table[(blockId.GetValue()) & hash_mask];
while (theBlock && (blockId != theBlock->GetId())) while ((NULL != theBlock) && (blockId != theBlock->GetId()))
theBlock = theBlock->next; theBlock = theBlock->next;
return theBlock; return theBlock;
} }
@ -738,21 +781,38 @@ NormBlock* NormBlockBuffer::Find(const NormBlockId& blockId) const
} }
} // end NormBlockBuffer::Find() } // end NormBlockBuffer::Find()
#endif // if/else USE_PROTO_TREE
NormBlockId NormBlockBuffer::RangeMin() const
{
if (range_max > 1)
{
NormBlockId rangeMin = range_hi;
Decrement(rangeMin, range_max - 1);
return rangeMin;
}
else
{
return range_lo;
}
} // end NormBlockBuffer::RangeMin()
bool NormBlockBuffer::CanInsert(NormBlockId blockId) const bool NormBlockBuffer::CanInsert(NormBlockId blockId) const
{ {
if (0 != range) if (0 != range)
{ {
if (blockId < range_lo) // if (blockId < range_lo)
if (Compare(blockId, range_lo) < 0)
{ {
if ((range_lo - blockId + range) > range_max) if (((UINT32)Difference(range_lo, blockId) + range) > range_max)
return false; return false;
else else
return true; return true;
} }
else if (blockId > range_hi) // else if (blockId > range_hi)
else if (Compare(blockId, range_hi) > 0)
{ {
if ((blockId - range_hi + range) > range_max) if (((UINT32)Difference(blockId, range_hi) + range) > range_max)
return false; return false;
else else
return true; return true;
@ -772,29 +832,38 @@ bool NormBlockBuffer::CanInsert(NormBlockId blockId) const
bool NormBlockBuffer::Insert(NormBlock* theBlock) bool NormBlockBuffer::Insert(NormBlock* theBlock)
{ {
const NormBlockId& blockId = theBlock->GetId(); const NormBlockId& blockId = theBlock->GetId();
if (!range) if (0 == range)
{ {
range_lo = range_hi = blockId; range_lo = range_hi = blockId;
range = 1; range = 1;
} }
if (blockId < range_lo) // else if (blockId < range_lo)
else if (Compare(blockId, range_lo) < 0)
{ {
UINT32 newRange = range_lo - blockId + range; UINT32 newRange = (UINT32)Difference(range_lo, blockId) + range;
if (newRange > range_max) return false; if (newRange > range_max) return false;
range_lo = blockId; range_lo = blockId;
range = newRange; range = newRange;
} }
else if (blockId > range_hi) // else if (blockId > range_hi)
else if (Compare(blockId, range_hi) > 0)
{ {
UINT32 newRange = blockId - range_hi + range; UINT32 newRange = (UINT32)Difference(blockId, range_hi) + range;
if (newRange > range_max) return false; if (newRange > range_max) return false;
range_hi = blockId; range_hi = blockId;
range = newRange; range = newRange;
} }
UINT32 index = ((UINT32)blockId) & hash_mask; ASSERT(Compare(range_hi, range_lo) >= 0);
// else unchanged range
#ifdef USE_PROTO_TREE
ASSERT(NULL == Find(theBlock->GetId()));
tree.Insert(*theBlock);
#else
UINT32 index = blockId.GetValue() & hash_mask;
NormBlock* prev = NULL; NormBlock* prev = NULL;
NormBlock* entry = table[index]; NormBlock* entry = table[index];
while (entry && (entry->GetId() < blockId)) // while (entry && (entry->GetId() < blockId))
while ((NULL != entry) && (Compare(entry->GetId(), blockId) < 0))
{ {
prev = entry; prev = entry;
entry = entry->next; entry = entry->next;
@ -803,21 +872,66 @@ bool NormBlockBuffer::Insert(NormBlock* theBlock)
prev->next = theBlock; prev->next = theBlock;
else else
table[index] = theBlock; table[index] = theBlock;
ASSERT((entry ? (blockId != entry->GetId()) : true)); ASSERT((entry ? (blockId != entry->GetId()) : true));
theBlock->next = entry; theBlock->next = entry;
#endif // if/else USE_PROTO_TREE
return true; return true;
} // end NormBlockBuffer::Insert() } // end NormBlockBuffer::Insert()
bool NormBlockBuffer::Remove(const NormBlock* theBlock) #ifdef USE_PROTO_TREE
bool NormBlockBuffer::Remove(NormBlock* theBlock)
{
ASSERT(NULL != theBlock);
const NormBlockId& blockId = theBlock->GetId();
switch (range)
{
case 0:
return false; // empty NormBlockBuffer
case 1:
if (blockId != range_lo)
return false; // out-of-range
range = 0;
break;
default:
if ((Compare(blockId, range_lo) < 0) || (Compare(blockId, range_hi) > 0))
return false; // out-of-range
if (blockId == range_lo)
{
const NormBlock* next = static_cast<const NormBlock*>(theBlock->GetNext());
if (NULL == next) next = static_cast<const NormBlock*>(tree.GetHead());
ASSERT(NULL != next);
range_lo = next->GetId();
range = Difference(range_hi, range_lo) + 1;
}
else if (blockId == range_hi)
{
const NormBlock* prev = static_cast<const NormBlock*>(theBlock->GetPrev());
if (NULL == prev) prev = static_cast<const NormBlock*>(tree.GetTail());
ASSERT(NULL != prev);
range_hi = prev->GetId();
range = Difference(range_hi, range_lo) + 1;
}
// else range unchanged
break;
}
ASSERT(NULL != tree.Find(theBlock->GetId().GetValuePtr(), 8*sizeof(UINT32)));
tree.Remove(*theBlock);
return true;
} // end NormBlockBuffer::Remove()
#else
bool NormBlockBuffer::Remove(NormBlock* theBlock)
{ {
ASSERT(NULL != theBlock); ASSERT(NULL != theBlock);
if (range) if (range)
{ {
const NormBlockId& blockId = theBlock->GetId(); const NormBlockId& blockId = theBlock->GetId();
if ((blockId < range_lo) || (blockId > range_hi)) return false; // if ((blockId < range_lo) || (blockId > range_hi))
UINT32 index = ((UINT32)blockId) & hash_mask; if ((Compare(blockId, range_lo) < 0) || (Compare(blockId, range_hi) > 0))
return false;
UINT32 index = blockId.GetValue() & hash_mask;
NormBlock* prev = NULL; NormBlock* prev = NULL;
NormBlock* entry = table[index]; NormBlock* entry = table[index];
while (entry && (entry->GetId() != blockId)) while (entry && (entry->GetId() != blockId))
@ -825,8 +939,8 @@ bool NormBlockBuffer::Remove(const NormBlock* theBlock)
prev = entry; prev = entry;
entry = entry->next; entry = entry->next;
} }
if (!entry) return false; if (NULL == entry) return false;
if (prev) if (NULL != prev)
prev->next = entry->next; prev->next = entry->next;
else else
table[index] = entry->next; table[index] = entry->next;
@ -835,7 +949,7 @@ bool NormBlockBuffer::Remove(const NormBlock* theBlock)
{ {
if (blockId == range_lo) if (blockId == range_lo)
{ {
// Find next entry for range_lo // Find next entry for new range_lo
UINT32 i = index; UINT32 i = index;
UINT32 endex; UINT32 endex;
if (range <= hash_mask) if (range <= hash_mask)
@ -849,29 +963,33 @@ bool NormBlockBuffer::Remove(const NormBlock* theBlock)
{ {
++i &= hash_mask; ++i &= hash_mask;
offset++; offset++;
if ((entry = table[i])) if (NULL != (entry = table[i]))
{ {
//NormBlockId id = (UINT32)index + offset; // NormBlockId id = blockId + offset;
NormBlockId id = (UINT32)blockId + offset; NormBlockId id = blockId;
Increment(id, offset);
while(entry && (entry->GetId() != id)) while(entry && (entry->GetId() != id))
{ {
if ((entry->GetId() > blockId) && // if ((entry->GetId() > blockId) && (entry->GetId() < nextId)
(entry->GetId() < nextId)) nextId = entry->GetId(); if ((Compare(entry->GetId(), blockId) > 0) &&
(Compare(entry->GetId(), nextId) < 0))
{
nextId = entry->GetId();
}
entry = entry->next; entry = entry->next;
} }
if (entry) break; if (NULL != entry) break;
} }
} while (i != endex); } while (i != endex);
if (entry) if (NULL != entry)
range_lo = entry->GetId(); range_lo = entry->GetId();
else else
range_lo = nextId; range_lo = nextId;
range = range_hi - range_lo + 1; range = (UINT32)Difference(range_hi, range_lo) + 1;
} }
else if (blockId == range_hi) else if (blockId == range_hi)
{ {
// Find prev entry for range_hi // Find prev entry for new range_hi
UINT32 i = index; UINT32 i = index;
UINT32 endex; UINT32 endex;
if (range <= hash_mask) if (range <= hash_mask)
@ -880,7 +998,6 @@ bool NormBlockBuffer::Remove(const NormBlock* theBlock)
endex = index; endex = index;
entry = NULL; entry = NULL;
UINT32 offset = 0; UINT32 offset = 0;
//printf("preving i:%lu endex:%lu lo:%lu hi:%lu\n", i, endex, (UINT32)range_lo, (UINT32) range_hi);
NormBlockId prevId = range_lo; NormBlockId prevId = range_lo;
do do
{ {
@ -888,23 +1005,27 @@ bool NormBlockBuffer::Remove(const NormBlock* theBlock)
offset++; offset++;
if ((entry = table[i])) if ((entry = table[i]))
{ {
//NormBlockId id = (UINT32)index - offset; // NormBlockId id = blockId - offset;
NormBlockId id = (UINT32)blockId - offset; NormBlockId id = blockId;
//printf("Looking for id:%lu at index:%lu\n", (UINT32)id, i); Decrement(id, offset);
while(entry && (entry->GetId() != id)) while(entry && (entry->GetId() != id))
{ {
if ((entry->GetId() < blockId) && // if ((entry->GetId() < blockId) && (entry->GetId() > prevId))
(entry->GetId() > prevId)) prevId = entry->GetId(); if ((Compare(entry->GetId(), blockId) < 0) &&
(Compare(entry->GetId(), prevId) > 0))
{
prevId = entry->GetId();
}
entry = entry->next; entry = entry->next;
} }
if (entry) break; if (NULL != entry) break;
} }
} while (i != endex); } while (i != endex);
if (entry) if (NULL != entry)
range_hi = entry->GetId(); range_hi = entry->GetId();
else else
range_hi = prevId; range_hi = prevId;
range = range_hi - range_lo + 1; range = (UINT32)Difference(range_hi, range_lo) + 1;
} }
} }
else else
@ -919,6 +1040,48 @@ bool NormBlockBuffer::Remove(const NormBlock* theBlock)
} }
} // end NormBlockBuffer::Remove() } // end NormBlockBuffer::Remove()
#endif // if/else USE_PROTO_TREE
#ifdef USE_PROTO_TREE
NormBlockBuffer::Iterator::Iterator(NormBlockBuffer& blockBuffer)
: buffer(blockBuffer), iterator(blockBuffer.tree, false, blockBuffer.range_lo.GetValuePtr(), 8*sizeof(UINT32))
{
next_block = iterator.GetNextItem();
ASSERT((NULL == next_block) || (blockBuffer.range_lo == next_block->GetId()))
}
void NormBlockBuffer::Iterator::Reset()
{
iterator.Reset(false, buffer.range_lo.GetValuePtr(), 8*sizeof(UINT32));
next_block = iterator.GetNextItem();
ASSERT(buffer.IsEmpty() || (NULL != next_block));
ASSERT((NULL == next_block) || (buffer.range_lo == next_block->GetId()))
} // end NormBlockBuffer::Iterator::Reset()
NormBlock* NormBlockBuffer::Iterator::GetNextBlock()
{
NormBlock* nextBlock = next_block;
if (NULL != nextBlock)
{
next_block = iterator.GetNextItem();
if (NULL == next_block)
{
iterator.Reset();
next_block = iterator.GetNextItem();
if (buffer.Compare(next_block->GetId(), nextBlock->GetId()) <= 0)
next_block = NULL;
}
else if (buffer.Compare(next_block->GetId(), nextBlock->GetId()) <= 0)
{
next_block = NULL;
}
}
return nextBlock;
} // end NormBlockBuffer::Iterator::GetNextBlock()
#else
NormBlockBuffer::Iterator::Iterator(const NormBlockBuffer& blockBuffer) NormBlockBuffer::Iterator::Iterator(const NormBlockBuffer& blockBuffer)
: buffer(blockBuffer), reset(true) : buffer(blockBuffer), reset(true)
{ {
@ -941,30 +1104,38 @@ NormBlock* NormBlockBuffer::Iterator::GetNextBlock()
} }
else else
{ {
if (buffer.range && // if (buffer.range && (index < buffer.range_hi) && (index >= buffer.range_lo))
(index < buffer.range_hi) && if ((0 != buffer.range) &&
(index >= buffer.range_lo)) (buffer.Compare(index, buffer.range_hi) < 0) &&
(buffer.Compare(index, buffer.range_lo) >= 0))
{ {
// Find next entry _after_ current "index" // Find next entry _after_ current "index"
UINT32 i = index; UINT32 i = index.GetValue();;
UINT32 endex; UINT32 endex;
if ((UINT32)(buffer.range_hi - index) <= buffer.hash_mask) // if ((UINT32)(buffer.range_hi - index) <= buffer.hash_mask)
endex = buffer.range_hi & buffer.hash_mask; if ((UINT32)buffer.Difference(buffer.range_hi, index) <= buffer.hash_mask)
endex = buffer.range_hi.GetValue() & buffer.hash_mask;
else else
endex = index; endex = index.GetValue();
UINT32 offset = 0; UINT32 offset = 0;
NormBlockId nextId = buffer.range_hi; NormBlockId nextId = buffer.range_hi;
do do
{ {
++i &= buffer.hash_mask; ++i &= buffer.hash_mask;
offset++; offset++;
NormBlockId id = (UINT32)index + offset; // NormBlockId id = (UINT32)index + offset;
NormBlockId id = index;
buffer.Increment(id, offset);
ASSERT(i < 256); ASSERT(i < 256);
NormBlock* entry = buffer.table[i]; NormBlock* entry = buffer.table[i];
while ((NULL != entry ) && (entry->GetId() != id)) while ((NULL != entry ) && (entry->GetId() != id))
{ {
if ((entry->GetId() > index) && (entry->GetId() < nextId)) // if ((entry->GetId() > index) && (entry->GetId() < nextId))
if ((buffer.Compare(entry->GetId(), index) > 0) &&
(buffer.Compare(entry->GetId(), nextId) < 0))
{
nextId = entry->GetId(); nextId = entry->GetId();
}
entry = NormBlockBuffer::Next(entry); entry = NormBlockBuffer::Next(entry);
} }
if (entry) if (entry)
@ -983,3 +1154,5 @@ NormBlock* NormBlockBuffer::Iterator::GetNextBlock()
} }
} }
} // end NormBlockBuffer::Iterator::GetNextBlock() } // end NormBlockBuffer::Iterator::GetNextBlock()
#endif // if/else USE_PROTO_TREE

975
src/common/normSession.cpp
View File

File diff suppressed because it is too large Load Diff

54
src/common/pcap2norm.cpp
View File

@ -5,6 +5,7 @@
#include <stdio.h> #include <stdio.h>
#include <pcap.h> #include <pcap.h>
#include <sys/socket.h> // for PF_ types (protocol family)
#include "protoPktETH.h" // for Ethernet frame parsing #include "protoPktETH.h" // for Ethernet frame parsing
#include "protoPktIP.h" // for IP packet parsing #include "protoPktIP.h" // for IP packet parsing
#include "protoPktARP.h" #include "protoPktARP.h"
@ -68,8 +69,9 @@ int main(int argc, char* argv[])
return -1; return -1;
} }
int deviceType = pcap_datalink(pcapDevice);
UINT32 alignedBuffer[4096/4]; // 128 buffer for packet parsing UINT32 alignedBuffer[4096/4]; // 4096 byte buffer for packet parsing
UINT16* ethBuffer = ((UINT16*)alignedBuffer) + 1; UINT16* ethBuffer = ((UINT16*)alignedBuffer) + 1;
unsigned int maxBytes = 4096 - 2; // due to offset, can only use 4094 bytes of buffer unsigned int maxBytes = 4096 - 2; // due to offset, can only use 4094 bytes of buffer
@ -79,21 +81,47 @@ int main(int argc, char* argv[])
{ {
unsigned int numBytes = maxBytes; unsigned int numBytes = maxBytes;
if (hdr.caplen < numBytes) numBytes = hdr.caplen; if (hdr.caplen < numBytes) numBytes = hdr.caplen;
memcpy(ethBuffer, pktData, numBytes); ProtoPktETH::Type ethType;
ProtoPktETH ethPkt((UINT32*)ethBuffer, maxBytes); unsigned int payloadLength;
if (!ethPkt.InitFromBuffer(hdr.len)) UINT32* payloadPtr;
if (DLT_NULL == deviceType)
{ {
fprintf(stderr, "pcap2norm error: invalid Ether frame in pcap file\n"); // pcap was captured from "loopback" device
continue; memcpy(alignedBuffer, pktData, numBytes);
switch (alignedBuffer[0])
{
case PF_INET:
ethType = ProtoPktETH::IP;
break;
case PF_INET6:
ethType = ProtoPktETH::IPv6;
break;
default:
continue; // not an IP packet
}
payloadLength = numBytes - 4;
payloadPtr = alignedBuffer + 1;
} }
else
{
memcpy(ethBuffer, pktData, numBytes);
ProtoPktETH ethPkt((UINT32*)ethBuffer, maxBytes);
if (!ethPkt.InitFromBuffer(hdr.len))
{
fprintf(stderr, "pcap2norm error: invalid Ether frame in pcap file\n");
continue;
}
ethType = ethPkt.GetType();
payloadLength = ethPkt.GetPayloadLength();
payloadPtr = (UINT32*)ethPkt.AccessPayload();
}
ProtoPktIP ipPkt; ProtoPktIP ipPkt;
ProtoAddress srcAddr, dstAddr; ProtoAddress srcAddr, dstAddr;
ProtoPktETH::Type ethType = ethPkt.GetType();
if ((ProtoPktETH::IP == ethType) || if ((ProtoPktETH::IP == ethType) ||
(ProtoPktETH::IPv6 == ethType)) (ProtoPktETH::IPv6 == ethType))
{ {
unsigned int payloadLength = ethPkt.GetPayloadLength(); if (!ipPkt.InitFromBuffer(payloadLength, payloadPtr, payloadLength))
if (!ipPkt.InitFromBuffer(payloadLength, (UINT32*)ethPkt.AccessPayload(), payloadLength))
{ {
fprintf(stderr, "pcap2norm error: bad IP packet\n"); fprintf(stderr, "pcap2norm error: bad IP packet\n");
continue; continue;
@ -123,6 +151,10 @@ int main(int argc, char* argv[])
//PLOG(PL_ALWAYS, "pcap2norm IP packet dst>%s ", dstAddr.GetHostString()); //PLOG(PL_ALWAYS, "pcap2norm IP packet dst>%s ", dstAddr.GetHostString());
//PLOG(PL_ALWAYS," src>%s length>%d\n", srcAddr.GetHostString(), ipPkt.GetLength()); //PLOG(PL_ALWAYS," src>%s length>%d\n", srcAddr.GetHostString(), ipPkt.GetLength());
} }
else
{
fprintf(stderr, "eth type = %d\n", ethType);
}
if (!srcAddr.IsValid()) continue; // wasn't an IP packet if (!srcAddr.IsValid()) continue; // wasn't an IP packet
ProtoPktUDP udpPkt; ProtoPktUDP udpPkt;
@ -153,7 +185,7 @@ void NormTrace2(const struct timeval& currentTime,
const ProtoAddress& dstAddr) const ProtoAddress& dstAddr)
{ {
UINT8 fecM = 16; // NOTE - this assumes 16-bit RS code for fec_id == 2 UINT8 fecM = 8; // NOTE - this assumes 16-bit RS code for fec_id == 2
static const char* MSG_NAME[] = static const char* MSG_NAME[] =
{ {
@ -227,7 +259,7 @@ void NormTrace2(const struct timeval& currentTime,
{ {
const NormDataMsg& data = (const NormDataMsg&)msg; const NormDataMsg& data = (const NormDataMsg&)msg;
lastFecId = data.GetFecId(); lastFecId = data.GetFecId();
PLOG(PL_ALWAYS, "inst>%hu seq>%hu DATA obj>%hu blk>%lu seg>%hu ", PLOG(PL_ALWAYS, "inst>%hu seq>%hu DATA obj>%hu blk>%u seg>%04hu ",
data.GetInstanceId(), data.GetInstanceId(),
seq, seq,
//data.IsData() ? "DATA" : "PRTY", //data.IsData() ? "DATA" : "PRTY",

10
src/common/raft.cpp
View File

@ -362,9 +362,15 @@ void RaftApp::OnInputReady()
} }
else if (tx_msg_index == tx_msg_length) else if (tx_msg_index == tx_msg_length)
{ {
if (!tx_socket.SendTo(tx_msg_buffer+2, tx_msg_length-2, tx_address)) unsigned int bytesSent = tx_msg_length-2;
if (!tx_socket.SendTo(tx_msg_buffer+2, bytesSent, tx_address))
{ {
PLOG(PL_ERROR, "raft: tx_socket.SendTo() error\n"); PLOG(PL_ERROR, "raft: tx_socket.SendTo() error: %s\n", GetErrorString());
return;
}
else if (0 == bytesSent)
{
PLOG(PL_WARN, "raft: tx_socket.SendTo() error: %s\n", GetErrorString());
return; return;
} }
tx_msg_index = tx_msg_length = 0; tx_msg_index = tx_msg_length = 0;

6
src/java/jni/normInstanceJni.cpp
View File

@ -179,6 +179,12 @@ JNIEXPORT jobject JNICALL PKGNAME(NormInstance_getNextEvent)
// Get the event type // Get the event type
jobjectArray array = (jobjectArray)env->CallStaticObjectMethod( jobjectArray array = (jobjectArray)env->CallStaticObjectMethod(
(jclass)env->NewLocalRef(jw_NormEventType), mid_NormEventType_values); (jclass)env->NewLocalRef(jw_NormEventType), mid_NormEventType_values);
if (env->GetArrayLength(array) <= event.type) {
env->ThrowNew((jclass)env->NewLocalRef(jw_IOException), "Invalid NORM event type (NormEventType.java out of sync with NORM API event header?)");
return false;
}
jobject type = env->GetObjectArrayElement(array, event.type); jobject type = env->GetObjectArrayElement(array, event.type);
// Create the event // Create the event

2
src/java/jni/normNodeJni.cpp
View File

@ -57,7 +57,7 @@ JNIEXPORT jobject JNICALL PKGNAME(NormNode_getAddress)
NormNodeHandle handle; NormNodeHandle handle;
char buffer[256]; char buffer[256];
unsigned int bufferLen; unsigned int bufferLen;
unsigned short port; UINT16 port;
jobject address; jobject address;
jbyteArray array; jbyteArray array;
jbyte *ptr; jbyte *ptr;

5
src/java/src/mil/navy/nrl/norm/enums/NormEventType.java
View File

@ -32,5 +32,8 @@ public enum NormEventType {
NORM_RX_OBJECT_ABORTED, NORM_RX_OBJECT_ABORTED,
NORM_GRTT_UPDATED, NORM_GRTT_UPDATED,
NORM_CC_ACTIVE, NORM_CC_ACTIVE,
NORM_CC_INACTIVE; NORM_CC_INACTIVE,
NORM_ACKING_NODE_NEW,
NORM_SEND_ERROR,
NORM_USER_TIMEOUT
} }

2
src/pynorm/core.py
View File

@ -506,5 +506,5 @@ def get_libnorm():
return libnorm return libnorm
# Global libnorm refrence that everyone should use # Global libnorm reference that everyone should use
libnorm = get_libnorm() libnorm = get_libnorm()

6
src/pynorm/event.py
View File

@ -91,5 +91,11 @@ class Event(object):
return "NORM_CC_ACTIVE" return "NORM_CC_ACTIVE"
elif self.type == c.NORM_CC_INACTIVE: elif self.type == c.NORM_CC_INACTIVE:
return "NORM_CC_INACTIVE" return "NORM_CC_INACTIVE"
elif self.type == c.NORM_ACKING_NODE_NEW:
return "NORM_ACKING_NODE_NEW"
elif self.type == c.NORM_SEND_ERROR:
return "NORM_SEND_ERROR"
elif self.type == c.NORM_USER_TIMEOUT:
return "NORM_USER_TIMEOUT"
else: else:
return "Unknown event type" return "Unknown event type"

9
src/pynorm/node.py
View File

@ -29,14 +29,17 @@ class Node(object):
return self._address return self._address
except AttributeError: except AttributeError:
port = ctypes.c_uint16() port = ctypes.c_uint16()
buffer = ctypes.create_string_buffer(50) buf = ctypes.create_string_buffer(50)
size = ctypes.c_uint(50) size = ctypes.c_uint(50)
if not libnorm.NormNodeGetAddress(self, buffer, ctypes.byref(size), if not libnorm.NormNodeGetAddress(self, buf, ctypes.byref(size),
ctypes.byref(port)): ctypes.byref(port)):
raise NormError("Node getAddress failed") raise NormError("Node getAddress failed")
self._address = (buffer.value, port.value) self._address = (buf.value, port.value)
return self._address return self._address
def getCommand(self, buf):
return libnorm.NormNodeGetCommand(self, buf, len(buf))
def getGrtt(self): def getGrtt(self):
grtt = libnorm.NormNodeGetGrtt(self) grtt = libnorm.NormNodeGetGrtt(self)
if grtt == -1.0: if grtt == -1.0:

6
src/pynorm/session.py
View File

@ -133,6 +133,12 @@ class Session(object):
def streamOpen(self, bufferSize, info=""): def streamOpen(self, bufferSize, info=""):
return Object(libnorm.NormStreamOpen(self, bufferSize, info, len(info))) return Object(libnorm.NormStreamOpen(self, bufferSize, info, len(info)))
def sendCommand(self, cmdBuffer, robust=False):
return libnorm.NormSendCommand(self, cmdBuffer, len(cmdBuffer), robust)
def cancelCommand(self):
libnorm.NormCancelCommand(self)
def setWatermark(self, normObject, overrideFlush=False): def setWatermark(self, normObject, overrideFlush=False):
libnorm.NormSetWatermark(self, normObject, overrideFlush) libnorm.NormSetWatermark(self, normObject, overrideFlush)

0
makefiles/win32/win32PostProcess.cpp → src/win32/win32PostProcess.cpp
View File

21
waf vendored
View File

File diff suppressed because one or more lines are too long

96
wscript
View File

@ -12,12 +12,28 @@ build), use the -o (--out) flag when configuring. For example:
./waf configure -o build-release ./waf configure -o build-release
./waf ./waf
To build examples, use the --target directive. For example:
./waf build --target=normClient,normServer
''' '''
import platform import platform
import waflib import waflib
# Fetch VERSION from include/normVersion.h file
VERSION = None
vfile = open('include/normVersion.h', 'r')
for line in vfile.readlines():
line = line.split()
if len(line) != 3:
continue
if "#define" == line[0] and "VERSION" == line[1]:
VERSION = line[2].strip('"')
if VERSION is None:
print "Warning: NORM VERSION not found!?"
# So you don't need to do ./waf configure if you are just using the defaults # So you don't need to do ./waf configure if you are just using the defaults
waflib.Configure.autoconfig = True waflib.Configure.autoconfig = True
@ -31,6 +47,10 @@ system = platform.system().lower()
def options(ctx): def options(ctx):
ctx.recurse('protolib') ctx.recurse('protolib')
build_opts = ctx.parser.add_option_group('Compile/install Options', 'Use during build/install step.')
build_opts.add_option('--enable-static-library', action='store_true',
help='Enable building and installing static library. [default:false]')
def configure(ctx): def configure(ctx):
ctx.recurse('protolib') ctx.recurse('protolib')
@ -38,7 +58,7 @@ def configure(ctx):
# Use this USE variable to add flags to NORM's compilation # Use this USE variable to add flags to NORM's compilation
ctx.env.USE_BUILD_NORM += ['BUILD_NORM', 'protolib'] ctx.env.USE_BUILD_NORM += ['BUILD_NORM', 'protolib']
if system in ('linux', 'darwin', 'freebsd'): if system in ('linux', 'darwin', 'freebsd', 'gnu', 'gnu/kfreebsd'):
ctx.env.DEFINES_BUILD_NORM += ['ECN_SUPPORT'] ctx.env.DEFINES_BUILD_NORM += ['ECN_SUPPORT']
if system == 'windows': if system == 'windows':
@ -47,12 +67,15 @@ def configure(ctx):
def build(ctx): def build(ctx):
ctx.recurse('protolib') ctx.recurse('protolib')
ctx.shlib( # Setup to install NORM header file
target = 'norm', ctx.install_files("${PREFIX}/include/", "include/normApi.h")
ctx.objects(
target = 'objs',
includes = ['include'], includes = ['include'],
export_includes = ['include'], export_includes = ['include'],
vnum = '1.0.0',
use = ctx.env.USE_BUILD_NORM, use = ctx.env.USE_BUILD_NORM,
stlib = ["protolib"],
source = ['src/common/{0}.cpp'.format(x) for x in [ source = ['src/common/{0}.cpp'.format(x) for x in [
'galois', 'galois',
'normApi', 'normApi',
@ -69,6 +92,32 @@ def build(ctx):
]], ]],
) )
# Use static lib for Unix examples for convenience
# (so we don't have to worry about LD_LIBRARY_PATH)
ctx.shlib(
target = 'norm',
includes = ['include'],
export_includes = ['include'],
vnum = VERSION,
stlib = ["protolib"],
use = ['objs'] + ctx.env.USE_BUILD_NORM,
source = [],
features = 'cxx cxxshlib',
)
if ctx.options.enable_static_library:
ctx.stlib(
target = 'norm',
includes = ['include'],
export_includes = ['include'],
vnum = VERSION,
stlib = ["protolib"],
use = ['objs'] + ctx.env.USE_BUILD_NORM,
source = [],
features = 'cxx cxxstlib',
install_path = '${LIBDIR}',
)
if ctx.env.BUILD_PYTHON: if ctx.env.BUILD_PYTHON:
ctx( ctx(
features='py', features='py',
@ -79,7 +128,8 @@ def build(ctx):
if ctx.env.BUILD_JAVA: if ctx.env.BUILD_JAVA:
ctx.shlib( ctx.shlib(
target = 'mil_navy_nrl_norm', target = 'mil_navy_nrl_norm',
use = ['norm', 'JAVA'], use = ['norm_shlib', 'JAVA'],
vnum = VERSION,
source = ['src/java/jni/{0}.cpp'.format(x) for x in [ source = ['src/java/jni/{0}.cpp'.format(x) for x in [
'normJni', 'normJni',
'normInstanceJni', 'normInstanceJni',
@ -101,11 +151,12 @@ def build(ctx):
) )
normapp = ctx.program( normapp = ctx.program(
target = 'norm', # Need to explicitly set a different name, because
# Need to explicitly set a different name, because the library is also # the library is also named "norm"
# named "norm"
name = 'normapp', name = 'normapp',
use = ['protolib', 'norm'], target = 'normapp',
use = ['protolib', 'norm_stlib'],
defines = [],
source = ['src/common/{0}'.format(x) for x in [ source = ['src/common/{0}'.format(x) for x in [
'normPostProcess.cpp', 'normPostProcess.cpp',
'normApp.cpp', 'normApp.cpp',
@ -114,11 +165,13 @@ def build(ctx):
posted = True, posted = True,
) )
if system in ('linux', 'darwin', 'freebsd'): if system in ('linux', 'darwin', 'freebsd', 'gnu', 'gnu/kfreebsd'):
normapp.source.append('src/unix/unixPostProcess.cpp') normapp.source.append('src/unix/unixPostProcess.cpp')
if system == 'windows': if system == 'windows':
normapp.source.append('src/win32/win32PostProcess.cpp') normapp.source.append('src/win32/win32PostProcess.cpp')
normapp.defines.append('_CONSOLE')
normapp.stlib = (["Shell32"]);
for example in ( for example in (
'normDataExample', 'normDataExample',
@ -128,6 +181,11 @@ def build(ctx):
'normFileSend', 'normFileSend',
'normStreamRecv', 'normStreamRecv',
'normStreamSend', 'normStreamSend',
'normMsgr',
'normStreamer',
'normClient',
'normServer',
'wintest'
): ):
_make_simple_example(ctx, example) _make_simple_example(ctx, example)
@ -146,14 +204,24 @@ def build(ctx):
def _make_simple_example(ctx, name, path='examples'): def _make_simple_example(ctx, name, path='examples'):
'''Makes a task from a single source file in the examples directory. '''Makes a task from a single source file in the examples directory.
These tasks are not built by default. Use the --targets flag. Note these tasks are not built by default. Use the --targets flag.
''' '''
ctx.program( source = ['{0}/{1}.cpp'.format(path, name)]
if 'normClient' == name or 'normServer' == name:
source.append('%s/normSocket.cpp' % path)
example = ctx.program(
target = name, target = name,
use = ['protolib', 'norm'], use = ['protolib'],
source = ['{0}/{1}.cpp'.format(path, name)], defines = [],
source = source,
# Don't build examples by default # Don't build examples by default
posted = True, posted = True,
# Don't install examples # Don't install examples
install_path = False, install_path = False,
) )
if 'windows' == system:
example.use.append('norm_shlib')
example.defines.append('_CONSOLE')
else:
example.use.append('norm_stlib')