BodgeMaster
/
NORM-mirror
mirror of https://github.com/USNavalResearchLaboratory/norm.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

v1.5.7

pull/2/head v1.5.7
Jeff Weston 2019-09-11 12:21:00 -04:00
parent da5e8f6692
commit a8ab40d749
53 changed files with 7910 additions and 2114 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

47
README-Java.txt
View File

@ -4,28 +4,42 @@ Java JNI bindings for NORM
By:
Jason Rush <jason.rush@gd-ais.com>
Peter Griffin <peter.griffin@gd-ais.com>
updates covering waf build system by Duc Nguyen <dnguyen@aypeks.com>
Updated: 2018-03-14
The Java JNI bindings for NORM provide Java bindings for the NORM C API.
For documentation about the main NORM API calls, refer to the NORM Developers
guide in the regular NORM distribution.
The JNI bindings for NORM can be built using two methods. The original
method uses Make and an Ant build.xml script in makefiles/java. The
newer method uses the waf build system and builds for all platforms with
various configurations via configuration switches.
------------
Requirements
------------
Java JNI bindings for NORM requires at least Java 1.5; however, it has also
been tested with Java 1.6.
been tested with Java 1.6 and Java 1.8
The NORM library should be built prior to building the Java JNI bindings since
they link against it.
### Using Ant and Make ###
Apache Ant is required for building the class files and jar file.
### Python waf ###
Tested with Python 2 but Python 3 should be ok.
------------
Building
------------
### Using Ant and Make ###
The build files for Java JNI bindings for NORM are located in the
makefiles/java directory.
@ -42,6 +56,23 @@ makefiles/java directory with the correct make file for your system:
This will produce a libmil_navy_nrl_norm.so file in the lib/ directory.
### Python waf ###
Reconfigure waf to use java and choose the target on a windows system:
> cd <norm root dir>
> waf configure --build-java --msvc_target=x64
The msvc_target flag allows you to choose the target Windows
architecture. The default is "x86" (i.e. 32bit arch)
Build the NORM libraries:
> waf build
The output will be in the build/ directory. Copy both
mil_navy_nrl_norm.* and norm-1.* to the location the JRE libraries.
------------
Installation
------------
@ -75,9 +106,23 @@ If you observe an "java.lang.UnsatisfiedLinkError" exception while running
your application, you do not have the libmil_navy_nrl_norm.so installed in
the correct location.
If you're running on Windows, you also need to have the PATH environment
variable set with along with the -Djava.library.path=<dir>
------------
Examples
------------
Examples using the Java JNI bindings for NORM are provided in the examples/java
directory.
Running on Windows (could be put in a .bat script):
set PATH=%PATH%;<path to mil_navy_nrl_norm.dll>
java -Djava.library.path=<path to mil_navy_nrl_norm.dll> ...
Running on Linux or MacOS:
java -Djava.library.path=<path to mil_navy_nrl_norm.dll> ...

40
VERSION.TXT
View File

@ -1,5 +1,45 @@
NORM Version History
Version 1.5.7
=============
- Fixed issue with NormSession::SenderRemoveAckingNode() method where
the removed NormAckingNode instance was not being properly deleted.
(Thanks to Chad Gioia)
- Fixed bug in normPrecoder (npc) when building on 64-bit architectures
(Thanks to Leon Meyer)
- Fixed bug in JNI NormNode.getAddress() method where an "unsigned short"
type was used instead of "UINT16" to receive the port portion of the
address. (Thanks to Jame Nguyen)
- Reduced warnings, added gnu and other build options to 'waf' wscripts
(Thanks to Luca Buccassi)
- Greatly improved high speed performance via enhancements to Protolib
- Fixed bugs related to coding block id wrapping for very long-lived
(or very high speed) NORM sessions.
- Added NormSocket API extension in "norm/examples" that provides a
sockets-like API and client-server connection-oriented paradigm
for using NORM. This currently supports a byte-stream service
for both multicast and unicast using NORM_OBJECT_STREAM. This API
extension is still work in progress and will be expanded to include
message-stream service support as well as NORM's file/data object
modes. The "norm/examples/normClient.cpp" (and "normServer.cpp)
illustrates use of the the API extension defined in
"norm/examples/normSocket.h"
- Added "normStreamer" example that provides the same function as the
"normMsgr" example except using NORM_OBJECT_STREAM.
- The bug fixes and performance enhancements make this a key update.
The NORM Developer's Guide is not yet updated to described some of
the new API features added.
- Added APIs to iterate through sender "Acking Node List" to determine
who has and who has not acknowledged a watermark acknowledgement
request.
- Added NormSetAutoAckingNodes() call with NormTrackingStatus to
automate detection of new group members. The Acking Node List
iterator function NormGetNextAckingNode() can be used to learn
who has joined and the NORM_ACKING_NODE_NEW provides notification
to the application of new members.
- The 'waf' build scripts assign the current version number to
the NORM shared library built for better version control/tracking.
Version 1.5r6
=============
- Fixed Protolib ProtoBitmask bug for NDEBUG (release) builds.

6
doc/NormDeveloperGuide.xml
View File

@ -1181,7 +1181,7 @@ bool <link linkend="NormSetCacheDirectory"><literal>NormSetCacheDirectory</liter
bool <link linkend="NormGetNextEvent"><literal>NormGetNextEvent</literal></link>(<link
linkend="NormInstanceHandle"><literal>NormInstanceHandle</literal></link> instanceHandle,
<link linkend="NormEvent"><literal>NormEvent</literal></link>* theEvent);</programlisting>
<link linkend="NormEvent"><literal>NormEvent*</literal></link> theEvent);</programlisting>
</sect4>
<sect4>
@ -3763,7 +3763,7 @@ void <link linkend="NormSetTxRobustFactor"><literal>NormSetTxRobustFactor</liter
<programlisting>#include &lt;normApi.h&gt;
boolNormRequeueObject(<link linkend="NormSessionHandle"><literal>NormSessionHandle</literal></link> sessionHandle,
bool NormRequeueObject(<link linkend="NormSessionHandle"><literal>NormSessionHandle</literal></link> sessionHandle,
<link linkend="NormObjectHandle"><literal>NormObjectHandle</literal></link> objectHandle);</programlisting>
</sect4>
@ -6596,7 +6596,7 @@ bool <link linkend="NormNodeGetAddress"><literal>NormNodeGetAddress</literal></l
<programlisting>#include &lt;normApi.h&gt;
double <link linkend="NormNodeGetId"><literal>NormNodeGetId</literal></link>(<link
double <link linkend="NormNodeGetId"><literal>NormNodeGetGrtt</literal></link>(<link
linkend="NormNodeHandle"><literal>NormNodeHandle</literal></link> nodeHandle);</programlisting>
</sect4>

2
examples/java/NormMsgr.java
View File

@ -239,7 +239,7 @@ public class NormMsgr
normTxLock.release();
return false;
}
//System.err.println("caching msg for object " + obj + "\n");
// System.err.println("caching msg for object " + obj + "\n");
// Cache the msg associated with the resultant tx object. We use the
// input_msg_list HashMap so we can remove the msg upon NORM_TX_OBJECT_PURGED
input_msg_list.put(obj, msg);

163
examples/normClient.cpp
View File

@ -1,9 +1,15 @@
#include "normSocket.h"
#include <stdio.h>
#include <string.h>
#ifdef WIN32
#include "win32InputHandler.cpp" // brings in the entire implementation
#else
#include <sys/select.h>
#include <fcntl.h> // for, well, fnctl()
#include <errno.h> // obvious child
#endif // if/else WIN32/UNIX
// BUILD: (assumes "normApi.h" in "include" ...
// g++ -I../include -o normClient normClient.cpp normSocket.cpp ../lib/libnorm.a ../protolib/lib/libprotokit.a -lresolv
@ -19,6 +25,11 @@
// To deny a connection, the "server" can send a stream close??? Which means the server will have
// to establish state just to deny a client a connection (yuck!)
void Usage()
{
fprintf(stderr, "Usage: normClient [connect <serverAddr>[/<port>][,<groupAddr>]][debug <level>][trace]\n");
}
const unsigned int MSG_LENGTH_MAX = 64;
int main(int argc, char* argv[])
@ -33,18 +44,19 @@ int main(int argc, char* argv[])
bool trace = false;
unsigned int debugLevel = 0;
for (int i = 1; i < argc; i++)
for (int i = 1; i < argc; i++)
{
const char* cmd = argv[i];
unsigned int len = strlen(cmd);
if (0 == strncmp(cmd, "connect", len))
{
// connect <serverAddr>/<port>[,<groupAddr>
// connect <serverAddr>/<port>[,<groupAddr>]
const char* val = argv[++i];
const char* portPtr = strchr(val, '/');
if (NULL == portPtr)
{
fprintf(stderr, "normClient error: missing <port> number\n");
Usage();
return -1;
}
portPtr++;
@ -58,6 +70,7 @@ int main(int argc, char* argv[])
else
{
fprintf(stderr, "normClient error: missing <serverAddr>\n");
Usage();
return -1;
}
char portText[32];
@ -76,6 +89,7 @@ int main(int argc, char* argv[])
if (1 != sscanf(portText, "%hu", &serverPort))
{
fprintf(stderr, "normClient error: invalid <port>\n");
Usage();
return -1;
}
if (NULL != groupPtr)
@ -96,7 +110,8 @@ int main(int argc, char* argv[])
{
if (1 != sscanf(argv[++i], "%u", &debugLevel))
{
fprintf(stderr, "normServer error: invalid debug level\n");
fprintf(stderr, "normClient error: invalid debug level\n");
Usage();
return -1;
}
}
@ -107,13 +122,15 @@ int main(int argc, char* argv[])
if (1 != sscanf(val, "%lu", &id))
{
fprintf(stderr, "normClient error: invalid 'id' value\n");
Usage();
return -1;
}
clientId = id;
}
else
{
fprintf(stderr, "normServer error: invalid command \"%s\"\n", cmd);
fprintf(stderr, "normClient error: invalid command \"%s\"\n", cmd);
Usage();
return -1;
}
}
@ -122,23 +139,39 @@ int main(int argc, char* argv[])
// Initate connection to server ...
fprintf(stderr, "normClient: connecting to %s/%hu ...\n", serverAddr, serverPort);
NormSocketHandle normSocket = NormConnect(instance, serverAddr, serverPort, groupAddrPtr, clientId);
NormSocketHandle normSocket = NormOpen(instance);
// setting 'localPort' param here to zero lets an ephemeral port be picked
NormConnect(normSocket, serverAddr, serverPort, 0, groupAddrPtr, clientId);
/* // Optional code to test NormWrite() immediately after NormConnect() call
// (Note this is _not_ compatible with newer stale connection reject code)
const char* helloStr = "Hello\n";
unsigned int helloLen = strlen(helloStr) + 1;
NormWrite(normSocket, helloStr, helloLen);
NormFlush(normSocket);*/
if (trace)
{
NormSetMessageTrace(NormGetSession(normSocket), true);
NormSetMessageTrace(NormGetSocketSession(normSocket), true);
if (NULL != groupAddrPtr)
NormSetMessageTrace(NormGetMulticastSession(normSocket), true);
NormSetMessageTrace(NormGetSocketMulticastSession(normSocket), true);
}
if (0 != debugLevel) NormSetDebugLevel(debugLevel);
//NormSetDebugLevel(3);
//NormSetMessageTrace(NormGetSession(normSocket), true);
//NormSetMessageTrace(NormGetSocketSession(normSocket), true);
// We use a select() call to multiplex input reading and NormSocket handling
#ifdef WIN32
HANDLE hStdout = GetStdHandle(STD_OUTPUT_HANDLE);
Win32InputHandler inputHandler;
inputHandler.Open();
HANDLE handleArray[2];
handleArray[0] = NormGetDescriptor(instance);
handleArray[1] = inputHandler.GetEventHandle();
#else
// On Unix, use a select() call to multiplex input reading and NormSocket handling
fd_set fdset;
FD_ZERO(&fdset);
// Get our input (STDIN) descriptor and set non-blocking
FILE* inputFile = stdin;
int inputfd = fileno(inputFile);
@ -146,15 +179,45 @@ int main(int argc, char* argv[])
perror("normClient: fcntl(inputfd, O_NONBLOCK) error");
// Get our NormInstance descriptor
int normfd = NormGetDescriptor(instance);
# endif // if/else WIN32
bool keepGoing = true;
bool writeReady = false;
unsigned int inputLength = 0;
int inputLength = 0;
unsigned int bytesWritten = 0;
const unsigned int BUFFER_LENGTH = 2048;
char inputBuffer[BUFFER_LENGTH];
bool inputNeeded = false; // will be set to "true" upon CONNECT
while (keepGoing)
{
bool normEventPending = false;
bool inputEventPending = false;
#ifdef WIN32
DWORD handleCount = inputNeeded ? 2 : 1;
DWORD waitStatus =
MsgWaitForMultipleObjectsEx(handleCount, // number of handles in array
handleArray, // object-handle array
INFINITE, // time-out interval
QS_ALLINPUT, // input-event type
0);
if ((WAIT_OBJECT_0 <= waitStatus) && (waitStatus < (WAIT_OBJECT_0 + handleCount)))
{
if (0 == (waitStatus - WAIT_OBJECT_0))
normEventPending = true;
else
inputEventPending = true;
}
else if (-1 == waitStatus)
{
perror("normClient: MsgWaitForMultipleObjectsEx() error");
break;
}
else
{
// TBD - any other status we should handle?
// (e.g. WAIT_TIMEOUT, WAIT_ABANDONED or WAIT_IO_COMPLETION)
continue; // ignore for now
}
#else
FD_SET(normfd, &fdset);
int maxfd = normfd;
if (inputNeeded)
@ -173,8 +236,36 @@ int main(int argc, char* argv[])
break;
}
if (FD_ISSET(inputfd, &fdset))
inputEventPending = true;
if (FD_ISSET(normfd, &fdset))
normEventPending = true;
#endif // if/else WIN32/UNIX
if (inputEventPending)
{
// Read input into our txBuffer
#ifdef WIN32
inputLength = inputHandler.ReadData(inputBuffer, BUFFER_LENGTH);
if (inputLength > 0)
{
// We got our input
bytesWritten = 0;
inputNeeded = false;
}
else if (inputLength < 0)
{
// Input stream has likely closed, initiate client shutown
// TBD - initiate client shutdown
if (NULL == groupAddrPtr)
{
fprintf(stderr, "normClient: CLOSING connection to server ...\n");
NormShutdown(normSocket);
}
inputNeeded = false; // TBD -should we also fclose(inputFile)???
inputHandler.Close();
}
// else zero bytes read, still need input
#else
inputLength = fread(inputBuffer, 1, BUFFER_LENGTH, inputFile);
if (inputLength > 0)
{
@ -191,7 +282,11 @@ int main(int argc, char* argv[])
NormShutdown(normSocket);
}
inputNeeded = false; // TBD -should we also fclose(inputFile)???
if (stdin != inputFile)
{
fclose(inputFile);
inputFile = NULL;
}
// else stick around to receive stuff from the server
}
else if (ferror(inputFile))
@ -205,14 +300,15 @@ int main(int argc, char* argv[])
// input starved, wait for next notification
break;
default:
perror("normClient: error reading input");
perror("normClient: error reading input?!");
break;
}
}
}
if (FD_ISSET(normfd, &fdset))
#endif // if/else WIN32/UNIX
} // end if inputEventPending
if (normEventPending)
{
// There's a NORM event pending
NormSocketEvent event;
if (NormGetSocketEvent(instance, &event))
@ -225,7 +321,11 @@ int main(int argc, char* argv[])
case NORM_SOCKET_CONNECT:
{
fprintf(stderr, "normClient: CONNECTED to server ...\n");
char remoteAddr[16];
unsigned int addrLen = 16;
UINT16 remotePort;
NormGetPeerName(normSocket, remoteAddr, &addrLen, &remotePort);
fprintf(stderr, "normClient: CONNECTED to server %s/%hu\n", serverAddr, remotePort);
inputNeeded = true;
writeReady = true;
break;
@ -246,7 +346,17 @@ int main(int argc, char* argv[])
continue;
}
if (bytesRead > 0)
{
#ifdef WIN32
// Use Win32 WriteFile() so there is no buffer delay
DWORD dwWritten;
WriteFile(hStdout, buffer, bytesRead, &dwWritten, NULL);
#else
fwrite(buffer, sizeof(char), bytesRead, stdout);
#endif // if/else WIN32
}
// If less bytes read than request then need
// to wait for next NORM_SOCKET_READ event
if (bytesRead < 1024) rxReady = false;
}
break;
@ -256,13 +366,16 @@ int main(int argc, char* argv[])
break;
case NORM_SOCKET_CLOSING:
fprintf(stderr, "normClient: server CLOSING connection ...\n");
writeReady = false;
inputNeeded = false;
break;
case NORM_SOCKET_CLOSED:
case NORM_SOCKET_CLOSE:
{
fprintf(stderr, "normClient: connection to server CLOSED.\n");
writeReady = false;
inputNeeded = false;
keepGoing = false;
break;
}
case NORM_SOCKET_NONE:
@ -273,7 +386,7 @@ int main(int argc, char* argv[])
{
fprintf(stderr, "normClient: NormGetSocketEvent() returned false\n");
}
} // end if FD_ISSET(normfd)
} // end if normEventPending
// If we have data in our inputBuffer and the NormSocket is "writeReady", then send it
if (writeReady && (inputLength > 0))
@ -293,4 +406,16 @@ int main(int argc, char* argv[])
}
}
} // end while(keepGoing)
NormClose(normSocket);
#ifdef WIN32
inputHandler.Close();
#else
if ((stdin != inputFile) && (NULL != inputFile))
{
fclose(inputFile);
inputFile = NULL;
}
#endif // if/else WIN32
fprintf(stderr, "normClient: Done.\n");
return 0;
}

3
examples/normFileRecv.cpp
View File

@ -58,6 +58,9 @@ int main(int argc, char* argv[])
6003,
NORM_NODE_ANY);
NormSetRxPortReuse(session, true);
NormSetMulticastLoopback(session, true);
// NOTE: These are debugging routines available
// (not necessary for normal app use)
// (Need to include "protolib/common/protoDebug.h" for this

5
examples/normFileSend.cpp
View File

@ -64,7 +64,10 @@ int main(int argc, char* argv[])
NormSessionHandle session = NormCreateSession(instance,
"224.1.2.3",
6003,
NORM_NODE_ANY);
1);
NormSetRxPortReuse(session, true);
NormSetMulticastLoopback(session, true);
// NOTE: These are some debugging routines available
// (not necessary for normal app use)

542
examples/normMsgr.cpp
View File

@ -1,8 +1,6 @@
#include "normApi.h"
#include "protoCheck.h"
#include <unistd.h> // for read() and write()
#include <stdio.h> // for printf(), etc
#include <stdlib.h> // for srand()
#include <string.h> // for strrchr(), memset(), etc
@ -12,6 +10,10 @@
#include <errno.h> // obvious child
#include <assert.h> // embarrassingly obvious
#define SHOOT_NOW 1
#include "protoCheck.h"
class NormMsgr
{
public:
@ -87,11 +89,17 @@ class NormMsgr
void Destroy();
unsigned int GetCount() const
{return msg_count;}
private:
Message* head;
Message* tail;
Message* head;
Message* tail;
unsigned int msg_count;
}; // end class NormMsgr::MessageQueue
Message* NewMessage(unsigned int size);
void Destroy();
bool OpenNormSession(NormInstanceHandle instance,
const char* addr,
@ -110,6 +118,12 @@ class NormMsgr
assert(NORM_SESSION_INVALID != norm_session);
NormSetMulticastInterface(norm_session, ifaceName);
}
void SetLoopback(bool state)
{
loopback = state;
if (NORM_SESSION_INVALID != norm_session)
NormSetMulticastLoopback(norm_session, state);
}
void SetNormMessageTrace(bool state)
{
assert(NORM_SESSION_INVALID != norm_session);
@ -121,36 +135,57 @@ class NormMsgr
NormAddAckingNode(norm_session, ackId);
norm_acking = true; // invoke ack-based flow control
}
void SetFlushing(bool state)
{norm_flushing = state;}
bool Start(bool sender, bool receiver);
void Stop()
{is_running = false;}
bool IsRunning() const
{return is_running;}
unsigned long GetSentCount()
{return sent_count;}
void HandleNormEvent(const NormEvent& event);
// Sender methods
FILE* GetInputFile() const
{return input_file;}
int GetInputDescriptor() const
{return input_fd;}
bool InputReady() const
{return input_ready;}
void SetInputReady()
{input_ready = true;}
bool InputNeeded() const
{return input_needed;}
bool InputMessageReady() const
{return ((NULL != input_msg) && !input_needed);}
bool ReadInput();
bool NormTxReady() const;
bool TxReady() const;
bool SendMessage();
bool EnqueueMessageObject();
// Receiver methods
void SetOutputFile(FILE* filePtr)
{
output_file = filePtr;
output_fd = fileno(filePtr);
}
FILE* GetOutputFile() const
{return output_file;}
int GetOutputDescriptor() const
{return output_fd;}
void SetOutputReady()
{output_ready = true;}
bool OutputReady() const
{return output_ready;}
bool OutputPending() const
{return output_pending;}
{return (NULL != output_msg);}
bool RxNeeded() const
{return rx_needed;}
bool WriteOutput();
void OmitHeader(bool state)
@ -164,24 +199,34 @@ class NormMsgr
private:
bool is_running;
FILE* input_file; // stdin by default
bool input_needed; //
FILE* input_file; // stdin by default
int input_fd;
bool input_ready;
bool input_needed; //
bool input_finished;
Message* input_msg; // current input message being read/sent*
MessageQueue input_msg_list; // list of enqueued messages (in norm sender cache)
NormSessionHandle norm_session;
bool is_multicast;
bool loopback;
unsigned int norm_tx_queue_max; // max number of objects that can be enqueued at once
unsigned int norm_tx_queue_count; // count of unacknowledged enqueued objects (TBD - optionally track size too)
bool norm_tx_watermark_pending;
bool norm_flow_control_pending;
bool norm_tx_vacancy;
bool norm_acking;
bool norm_flushing;
NormObjectHandle norm_flush_object;
NormObjectHandle norm_last_object;
unsigned long sent_count;
bool rx_needed;
FILE* output_file;
int output_fd;
bool output_ready;
bool output_pending;
Message* output_msg;
MessageQueue output_msg_queue;
MessageQueue temp_msg_queue;
// These are some options mainly for testing purposes
bool omit_header; // if "true", receive message length header is _not_ written to output
bool rx_silent;
@ -190,22 +235,39 @@ class NormMsgr
}; // end class NormMsgr
NormMsgr::NormMsgr()
: input_file(stdin), input_needed(false), input_msg(NULL),
norm_session(NORM_SESSION_INVALID), is_multicast(false), norm_tx_queue_max(2048), norm_tx_queue_count(0),
norm_tx_watermark_pending(false), norm_tx_vacancy(true), norm_acking(false),
output_file(stdout), output_ready(true), output_pending(false), output_msg(NULL),
: input_file(stdin), input_fd(fileno(stdin)), input_ready(false), input_needed(false), input_finished(false),
input_msg(NULL), norm_session(NORM_SESSION_INVALID), is_multicast(false), loopback(false),
norm_tx_queue_max(8192), norm_tx_queue_count(0),
norm_flow_control_pending(false), norm_tx_vacancy(true), norm_acking(false),
norm_flushing(true), norm_flush_object(NORM_OBJECT_INVALID), norm_last_object(NORM_OBJECT_INVALID),
sent_count(0), rx_needed(false), output_file(stdout), output_fd(fileno(stdout)),
output_ready(true), output_msg(NULL),
omit_header(false), rx_silent(false), tx_loss(0.0)
{
}
NormMsgr::~NormMsgr()
{
if (NULL != input_msg) delete input_msg;
Destroy();
}
void NormMsgr::Destroy()
{
if (NULL != input_msg)
{
delete input_msg;
input_msg = NULL;
}
input_msg_list.Destroy();
if (NULL != output_msg) delete output_msg;
if (NULL != output_msg)
{
delete output_msg;
delete output_msg;
}
output_msg_queue.Destroy();
}
bool NormMsgr::OpenNormSession(NormInstanceHandle instance, const char* addr, unsigned short port, NormNodeId nodeId)
{
if (NormIsUnicastAddress(addr))
@ -222,21 +284,24 @@ bool NormMsgr::OpenNormSession(NormInstanceHandle instance, const char* addr, un
if (is_multicast)
{
NormSetRxPortReuse(norm_session, true);
// TBD - make full loopback a command line option?
NormSetMulticastLoopback(norm_session, true);
if (loopback)
NormSetMulticastLoopback(norm_session, true);
}
// Set some default parameters (maybe we should put parameter setting in Start())
fprintf(stderr, "setting rx cache limit to %u\n", norm_tx_queue_max);
if (norm_tx_queue_max > 65535/2) norm_tx_queue_max = 65535/2;
NormSetRxCacheLimit(norm_session, norm_tx_queue_max);
NormSetDefaultSyncPolicy(norm_session, NORM_SYNC_ALL);
NormSetDefaultUnicastNack(norm_session, true);
if (!is_multicast)
NormSetDefaultUnicastNack(norm_session, true);
NormSetTxCacheBounds(norm_session, 10*1024*1024, norm_tx_queue_max, norm_tx_queue_max);
//NormSetMessageTrace(norm_session, true);
NormSetTxRobustFactor(norm_session, 2);
//NormSetTxRobustFactor(norm_session, 20);
return true;
} // end NormMsgr::OpenNormSession()
@ -274,13 +339,27 @@ void NormMsgr::SetNormCongestionControl(CCMode ccMode)
bool NormMsgr::Start(bool sender, bool receiver)
{
fprintf(stderr, "enter NormMsgr::Start() ...\n");
// TBD - make these command-line accessible
unsigned int bufferSize = 64*1024*1024;
unsigned int segmentSize = 1400;
unsigned int blockSize = 64;
unsigned int numParity = 0;
unsigned int txSockBufferSize = 4*1024*1024;
unsigned int rxSockBufferSize = 6*1024*1024;
if (receiver)
{
if (!NormStartReceiver(norm_session, 10*1024*1024))
if (!NormStartReceiver(norm_session, bufferSize))
{
fprintf(stderr, "normMsgr error: unable to start NORM receiver\n");
return false;
}
// Note: NormPreallocateRemoteSender() MUST be called AFTER NormStartReceiver()
NormPreallocateRemoteSender(norm_session, bufferSize, segmentSize, blockSize, numParity, bufferSize);
NormSetRxSocketBuffer(norm_session, rxSockBufferSize);
rx_needed = true;
fprintf(stderr, "normMsgr: receiver ready.\n");
}
if (sender)
{
@ -290,19 +369,26 @@ bool NormMsgr::Start(bool sender, bool receiver)
// so disable timer-based flow control
NormSetFlowControl(norm_session, 0.0);
}
NormSetGrttEstimate(norm_session, 0.001);
//NormSetGrttMax(norm_session, 0.100);
NormSetBackoffFactor(norm_session, 0);
// Pick a random instance id for now
struct timeval currentTime;
gettimeofday(&currentTime, NULL);
srand(currentTime.tv_usec); // seed random number generator
NormSessionId instanceId = (NormSessionId)rand();
if (!NormStartSender(norm_session, instanceId, 10*1024*1024, 1400, 16, 4))
if (!NormStartSender(norm_session, instanceId, bufferSize, segmentSize, blockSize, numParity))
{
fprintf(stderr, "normMsgr error: unable to start NORM sender\n");
if (receiver) NormStopReceiver(norm_session);
return false;
}
//NormSetAutoParity(norm_session, 2);
NormSetTxSocketBuffer(norm_session, txSockBufferSize);
input_needed = true;
}
//ProtoCheckResetLogging();
is_running = true;
return true;
} // end NormMsgr::Start();
@ -337,11 +423,11 @@ bool NormMsgr::ReadInput()
readLength = input_msg->GetSize() - offset;
bufferPtr = input_msg->AccessBuffer() + offset;
}
size_t result = fread(bufferPtr, 1, readLength, input_file);
ssize_t result = read(input_fd, bufferPtr, readLength);
if (result > 0)
{
input_msg->IncrementIndex(result);
if (result < readLength)
if ((size_t)result < readLength)
{
// Still need more input
// (wait for next input notification to read more)
@ -368,27 +454,37 @@ bool NormMsgr::ReadInput()
input_needed = false;
}
}
else
else if (0 == result)
{
if (feof(input_file))
// end-of-file reached, TBD - trigger final flushing and wrap-up
fprintf(stderr, "normMsgr: input end-of-file detected (last:%p)...\n", norm_last_object);
delete input_msg;
input_msg = NULL;
input_ready = false;
input_needed = false;
input_finished = true;
if (norm_acking)
{
// end-of-file reached, TBD - trigger final flushing and wrap-up
fprintf(stderr, "normMsgr: input end-of-file detected ...\n");
if (NORM_OBJECT_INVALID == norm_last_object)
is_running = false; // everything sent and acked
else if (!norm_flushing)
NormSetWatermark(norm_session, norm_last_object, true);
}
else if (ferror(input_file))
}
else // result < 0
{
switch (errno)
{
switch (errno)
{
case EINTR:
continue; // interupted, try again
case EAGAIN:
// input starved, wait for next notification
break;
default:
perror("normMsgr error reading input");
break;
}
case EINTR:
continue; // interupted, try again
case EAGAIN:
// input starved, wait for next notification
break;
default:
perror("normMsgr error reading input");
break;
}
input_ready = false;
return false;
}
}
@ -397,7 +493,7 @@ bool NormMsgr::ReadInput()
bool NormMsgr::WriteOutput()
{
while (output_pending)
while (NULL != output_msg)
{
size_t writeLength;
const char* bufferPtr;
@ -412,38 +508,40 @@ bool NormMsgr::WriteOutput()
writeLength = output_msg->GetSize() - offset;
bufferPtr = output_msg->GetBuffer() + offset;
}
size_t result = fwrite(bufferPtr, 1, writeLength, output_file);
output_msg->IncrementIndex(result);
if (result < writeLength)
ssize_t result = write(output_fd, bufferPtr, writeLength);
if (result >= 0)
{
if (feof(output_file))
output_msg->IncrementIndex(result);
if ((size_t)result < writeLength)
output_ready = false; // blocked, wait for output notification
}
else
{
switch (errno)
{
// end-of-file reached, TBD - stop acting as receiver, signal sender we're done?
fprintf(stderr, "normMsgr: output end-of-file detected ...\n");
}
else if (ferror(input_file))
{
fprintf(stderr, "normMsgr: output error detected ...\n");
switch (errno)
{
case EINTR:
continue; // interupted, try again
case EAGAIN:
// input starved, wait for next notification
output_ready = false;
break;
default:
perror("normMsgr error writing output");
break;
}
case EINTR:
continue; // interupted, try again
case EAGAIN:
// input starved, wait for next notification
output_ready = false;
break;
default:
perror("normMsgr error writing output");
break;
}
return false;
}
if (output_msg->IsComplete())
{
fflush(output_file);
delete output_msg;
//delete output_msg;
temp_msg_queue.Append(*output_msg); // cache for debugging purposes
if (temp_msg_queue.GetCount() > 32)
{
//fprintf(stderr, "deleting cached recv'd message ...\n");
delete temp_msg_queue.RemoveHead();
}
output_msg = output_msg_queue.RemoveHead();
if (NULL != output_msg)
{
@ -454,7 +552,7 @@ bool NormMsgr::WriteOutput()
}
else
{
output_pending = false;
rx_needed = true;
}
break;
}
@ -462,7 +560,7 @@ bool NormMsgr::WriteOutput()
return true;
} // end NormMsgr::WriteOutput()
bool NormMsgr::NormTxReady() const
bool NormMsgr::TxReady() const
{
// This returns true if new tx data can be enqueued to NORM
// This is based on the state with respect to prior successful data
@ -484,8 +582,6 @@ bool NormMsgr::NormTxReady() const
bool NormMsgr::SendMessage()
{
// TBD - call EnqueueMessageObject() or WriteMessageStream()
// depending upon on configured mode
if (EnqueueMessageObject())
{
// Our buffered message was sent, so reset input indices
@ -493,6 +589,7 @@ bool NormMsgr::SendMessage()
input_msg_list.Append(*input_msg);
input_msg = NULL;
input_needed = true;
sent_count++;
return true;
}
// else will be prompted to retry by NORM event (queue vacancy, watermark completion)
@ -515,23 +612,42 @@ bool NormMsgr::EnqueueMessageObject()
// This might happen if a non-acking receiver is present and
// has nacked for the oldest object in the queue even if all
// of our acking receivers have acknowledged it.
fprintf(stderr, "NO VACANCY count:%u max:%u\n", norm_tx_queue_count, norm_tx_queue_max);
norm_tx_vacancy = false;
return false;
}
//NormObjectRetain(object);
NormObjectSetUserData(object, input_msg); // so we can remove/delete upon purge
if (norm_acking)
{
// ack-based flow control has been enabled
norm_tx_queue_count++;
if (!norm_tx_watermark_pending && (norm_tx_queue_count >= (norm_tx_queue_max / 2)))
if (!norm_flow_control_pending && (norm_tx_queue_count >= (norm_tx_queue_max / 2)))
{
NormSetWatermark(norm_session, object);
norm_tx_watermark_pending = true;
NormSetWatermark(norm_session, object, true); // overrideFlush == true
norm_last_object = object;
norm_flow_control_pending = true;
}
else if (norm_flushing) // per-message acking
{
#ifdef SHOOT_FIRST
NormSetWatermark(norm_session, object, true);
norm_last_object = object;
#else // ACK_LATER
if (norm_flow_control_pending)
{
norm_flush_object = object; // will be used as watermark upon flow control ack
}
else
{
NormSetWatermark(norm_session, object, true);
norm_last_object = object;
}
#endif // SHOOT_FIRST/ACK_LATER
}
else
{
// TBD - make non-flow control acking separable option?
NormSetWatermark(norm_session, object);
//norm_last_object = object;
}
}
return true;
@ -539,7 +655,6 @@ bool NormMsgr::EnqueueMessageObject()
void NormMsgr::HandleNormEvent(const NormEvent& event)
{
bool logAllocs = false;
switch (event.type)
{
case NORM_TX_QUEUE_EMPTY:
@ -555,18 +670,37 @@ void NormMsgr::HandleNormEvent(const NormEvent& event)
if (NORM_ACK_SUCCESS == NormGetAckingStatus(norm_session))
{
//fprintf(stderr, "WATERMARK COMPLETED\n");
if (norm_tx_watermark_pending)
norm_last_object = NORM_OBJECT_INVALID;
if (norm_flow_control_pending)
{
norm_tx_queue_count -= (norm_tx_queue_max / 2);
norm_tx_watermark_pending = false;
norm_flow_control_pending = false;
if (NORM_OBJECT_INVALID != norm_flush_object)
{
NormSetWatermark(norm_session, norm_flush_object, true);
norm_last_object = norm_flush_object;
norm_flush_object = NORM_OBJECT_INVALID;
}
}
if (input_finished && (NORM_OBJECT_INVALID == norm_last_object))
is_running = false;
}
else
{
// TBD - we could see who didn't ACK and possibly remove them
// from our acking list. For now, we are infinitely
// persistent by resetting watermark ack request
NormResetWatermark(norm_session);
// TBD - we could see who did and how didn't ACK and possibly remove them
// from our acking "membership". For now, we are infinitely
// persistent by resetting watermark ack request without clearing
// flow control
if (NORM_OBJECT_INVALID == norm_flush_object)
{
NormResetWatermark(norm_session);
}
else // might as request ack for most recent enqueued object
{
NormSetWatermark(norm_session, norm_flush_object, true);
norm_last_object = norm_flush_object;
norm_flush_object = NORM_OBJECT_INVALID;
}
}
break;
@ -574,11 +708,16 @@ void NormMsgr::HandleNormEvent(const NormEvent& event)
{
NormDataDetachData(event.object);
Message* msg = (Message*)NormObjectGetUserData(event.object);
if(event.object == norm_flush_object)
norm_flush_object = NORM_OBJECT_INVALID;
if (NULL != msg)
{
input_msg_list.Remove(*msg);
delete msg;
}
//NormObjectRelease(event.object);
//fprintf(stderr, "normMsgr LOGGING ALLOCATIONS\n");
//ProtoCheckLogAllocations(stderr);
break;
}
@ -590,11 +729,11 @@ void NormMsgr::HandleNormEvent(const NormEvent& event)
case NORM_RX_OBJECT_ABORTED:
//fprintf(stderr, "NORM_RX_OBJECT_ABORTED\n");// %hu\n", NormObjectGetTransportId(event.object));
logAllocs = true;
break;
case NORM_RX_OBJECT_COMPLETED:
{
sent_count++;
char* data = NormDataDetachData(event.object);
if (NULL != data)
{
@ -613,12 +752,12 @@ void NormMsgr::HandleNormEvent(const NormEvent& event)
output_msg->ResetIndex(MSG_HEADER_SIZE);
else
output_msg->ResetIndex();
output_pending = true;
}
else
{
output_msg_queue.Append(*msg);
}
rx_needed = false;
}
// else TBD - "termination" info-only object?
break;
@ -628,13 +767,6 @@ void NormMsgr::HandleNormEvent(const NormEvent& event)
break;
}
//NormReleasePreviousEvent(NormGetInstance(norm_session));
if (logAllocs)
{
#ifdef USE_PROTO_CHECK
ProtoCheckLogAllocations(stderr);
#endif // USE_PROTO_CHECK
}
} // end NormMsgr::HandleNormEvent()
@ -643,7 +775,7 @@ void Usage()
{
fprintf(stderr, "Usage: normMsgr id <nodeId> {send &| recv} [addr <addr>[/<port>]][ack <node1>[,<node2>,...]\n"
" [cc|cce|ccl|rate <bitsPerSecond>][interface <name>][debug <level>][trace]\n"
" [omit][silent][txloss <lossFraction>]\n");
" [flush {none|active}][omit][silent][txloss <lossFraction>]\n");
}
int main(int argc, char* argv[])
{
@ -658,16 +790,21 @@ int main(int argc, char* argv[])
NormNodeId ackingNodeList[256];
unsigned int ackingNodeCount = 0;
bool flushing = false;
double txRate = 0.0; // used for non-default NORM_FIXED ccMode
NormMsgr::CCMode ccMode = NormMsgr::NORM_CC;
const char* mcastIface = NULL;
int debugLevel = 0;
const char* debugLog = NULL; // stderr by default
bool trace = false;
bool omitHeaderOnOutput = false;
bool silentReceiver = false;
double txloss = 0.0;
bool loopback = false;
NormMsgr normMsgr;
// Parse command-line
int i = 1;
@ -683,6 +820,10 @@ int main(int argc, char* argv[])
{
recv = true;
}
else if (0 == strncmp(cmd, "loopback", len))
{
loopback = true;
}
else if (0 == strncmp(cmd, "addr", len))
{
if (i >= argc)
@ -708,6 +849,23 @@ int main(int argc, char* argv[])
sessionPort = atoi(portPtr);
}
}
else if (0 == strncmp(cmd, "output", len))
{
if (i >= argc)
{
fprintf(stderr, "normMsgr error: missing output 'device' name!\n");
Usage();
return -1;
}
FILE* outfile = fopen(argv[i++], "w+");
if (NULL == outfile)
{
perror("normMsgr output device fopen() error");
Usage();
return -1;
}
normMsgr.SetOutputFile(outfile);
}
else if (0 == strncmp(cmd, "id", len))
{
if (i >= argc)
@ -731,17 +889,47 @@ int main(int argc, char* argv[])
while ((NULL != alist) && (*alist != '\0'))
{
// TBD - Do we need to skip leading white space?
if (1 != sscanf(alist, "%d", ackingNodeList + ackingNodeCount))
int id;
if (1 != sscanf(alist, "%d", &id))
{
fprintf(stderr, "nodeMsgr error: invalid acking node list!\n");
Usage();
return -1;
}
ackingNodeList[ackingNodeCount] = NormNodeId(id);
ackingNodeCount++;
alist = strchr(alist, ',');
if (NULL != alist) alist++; // point past comma
}
}
else if (0 == strncmp(cmd, "flush", len))
{
// "none", "passive", or "active"
if (i >= argc)
{
fprintf(stderr, "nodeMsgr error: missing 'flush' <mode>!\n");
Usage();
return -1;
}
const char* mode = argv[i++];
if (0 == strcmp(mode, "none"))
{
flushing = false;
}
else if (0 == strcmp(mode, "passive"))
{
flushing = false;
}
else if (0 == strcmp(mode, "active"))
{
flushing = true;
}
else
{
fprintf(stderr, "normMsgr error: invalid 'flush' mode \"%s\"\n", mode);
return -1;
}
}
else if (0 == strncmp(cmd, "rate", len))
{
if (i >= argc)
@ -802,12 +990,22 @@ int main(int argc, char* argv[])
{
if (i >= argc)
{
fprintf(stderr, "nodeMsgr error: missing 'interface' <name>!\n");
fprintf(stderr, "nodeMsgr error: missing 'debug' <level>!\n");
Usage();
return -1;
}
debugLevel = atoi(argv[i++]);
}
else if (0 == strncmp(cmd, "log", len))
{
if (i >= argc)
{
fprintf(stderr, "nodeMsgr error: missing 'log' <fileName>!\n");
Usage();
return -1;
}
debugLog = argv[i++];
}
else if (0 == strncmp(cmd, "trace", len))
{
trace = true;
@ -841,8 +1039,11 @@ int main(int argc, char* argv[])
// TBD - should provide more error checking of calls
NormInstanceHandle normInstance = NormCreateInstance();
NormSetDebugLevel(debugLevel);
if (NULL != debugLog)
NormOpenDebugLog(normInstance, debugLog);
NormMsgr normMsgr;
normMsgr.SetLoopback(loopback);
normMsgr.SetFlushing(flushing);
if (omitHeaderOnOutput) normMsgr.OmitHeader(true);
@ -873,89 +1074,134 @@ int main(int argc, char* argv[])
int normfd = NormGetDescriptor(normInstance);
// Get input/output descriptors and set to non-blocking i/o
int inputfd = fileno(normMsgr.GetInputFile());
int inputfd = normMsgr.GetInputDescriptor();
if (-1 == fcntl(inputfd, F_SETFL, fcntl(inputfd, F_GETFL, 0) | O_NONBLOCK))
perror("normMsgr: fcntl(inputfd, O_NONBLOCK) error");
int outputfd = fileno(normMsgr.GetOutputFile());
int outputfd = normMsgr.GetOutputDescriptor();
if (-1 == fcntl(outputfd, F_SETFL, fcntl(outputfd, F_GETFL, 0) | O_NONBLOCK))
perror("normMsgr: fcntl(outputfd, O_NONBLOCK) error");
fd_set fdsetInput, fdsetOutput;
FD_ZERO(&fdsetInput);
FD_ZERO(&fdsetOutput);
struct timeval lastTime;
gettimeofday(&lastTime, NULL);
while (normMsgr.IsRunning())
{
int maxfd = normfd;
FD_SET(normfd, &fdsetInput);
if (normMsgr.InputNeeded())
int maxfd = -1;
// Only wait on NORM if needed for tx readiness
bool waitOnNorm = true;
if (!(normMsgr.RxNeeded() || normMsgr.InputMessageReady()))
waitOnNorm = false; // no need to wait
else if (normMsgr.InputMessageReady() && normMsgr.TxReady())
waitOnNorm = false; // no need to wait if already tx ready
if (waitOnNorm)
{
maxfd = normfd;
FD_SET(normfd, &fdsetInput);
}
else
{
FD_CLR(normfd, &fdsetInput);
}
if (normMsgr.InputNeeded() && !normMsgr.InputReady())
{
//fprintf(stderr, "NEED INPUT ...\n");
FD_SET(inputfd, &fdsetInput);
if (inputfd > maxfd)
maxfd = inputfd;
if (inputfd > maxfd) maxfd = inputfd;
}
else
{
FD_CLR(inputfd, &fdsetInput);
}
int result;
if (normMsgr.OutputPending() && !normMsgr.OutputReady())
{
FD_SET(outputfd, &fdsetOutput);
if (outputfd > maxfd) maxfd = outputfd;
result = select(maxfd+1, &fdsetInput, &fdsetOutput, NULL, NULL);
}
else
{
FD_CLR(outputfd, &fdsetOutput);
result = select(maxfd+1, &fdsetInput, NULL, NULL, NULL);
}
switch (result)
if (maxfd >= 0)
{
case -1:
switch (errno)
{
case EINTR:
case EAGAIN:
continue;
default:
perror("normMsgr select() error");
// TBD - stop NormMsgr
break;
}
break;
case 0:
// shouldn't occur for now (no timeout)
continue;
default:
if (FD_ISSET(inputfd, &fdsetInput))
{
normMsgr.ReadInput();
}
if (FD_ISSET(normfd, &fdsetInput))
{
NormEvent event;
if (NormGetNextEvent(normInstance, &event))
normMsgr.HandleNormEvent(event);
}
if (FD_ISSET(outputfd, &fdsetOutput))
{
normMsgr.SetOutputReady();
}
break;
struct timeval timeout;
timeout.tv_sec = 1;
timeout.tv_usec = 0;
int result = select(maxfd+1, &fdsetInput, &fdsetOutput, NULL, &timeout);
switch (result)
{
case -1:
switch (errno)
{
case EINTR:
case EAGAIN:
continue;
default:
perror("normMsgr select() error");
// TBD - stop NormMsgr
break;
}
break;
case 0:
// shouldn't occur for now (no timeout)
//fprintf(stderr, "normMsgr timeout ...\n");
continue;
default:
if (FD_ISSET(inputfd, &fdsetInput))
{
normMsgr.SetInputReady();
}
/*if (FD_ISSET(normfd, &fdsetInput))
{
NormEvent event;
while (NormGetNextEvent(normInstance, &event, false))
normMsgr.HandleNormEvent(event);
}*/
if (FD_ISSET(outputfd, &fdsetOutput))
{
normMsgr.SetOutputReady();
}
break;
}
}
NormSuspendInstance(normInstance);
NormEvent event;
while (NormGetNextEvent(normInstance, &event, false))
normMsgr.HandleNormEvent(event);
// As a result of reading input or NORM notification events,
// we may be ready to send a message if it's been read
if (normMsgr.InputMessageReady() && normMsgr.NormTxReady())
normMsgr.SendMessage();
// and/or output a received message if we need
if (normMsgr.OutputPending() && normMsgr.OutputReady())
normMsgr.WriteOutput(); // TBD - implement output async i/o notification as needed
// we may be ready to read input and/or send a message if it's been read
const int LOOP_MAX = 100;
int loopCount = 0;
while (loopCount < LOOP_MAX)
{
loopCount++;
if (normMsgr.InputNeeded() && normMsgr.InputReady())
normMsgr.ReadInput();
if (normMsgr.InputMessageReady() && normMsgr.TxReady())
normMsgr.SendMessage();
// and/or output a received message if we need
if (normMsgr.OutputPending() && normMsgr.OutputReady())
normMsgr.WriteOutput();
}
NormResumeInstance(normInstance);
} // end while(normMsgr.IsRunning()
fprintf(stderr, "normMsgr exiting ...\n");
NormCloseDebugLog(normInstance);
fprintf(stderr, "destroying session ...\n");
normMsgr.CloseNormSession();
fprintf(stderr, "destroying instance ...\n");
NormDestroyInstance(normInstance);
normMsgr.Destroy();
fprintf(stderr, "normMsgr exiting ...\n");
} // end main()
NormMsgr::Message::Message()
@ -980,6 +1226,7 @@ NormMsgr::Message::~Message()
// to delete msg_buffer if it came from NORM
if (NULL != msg_buffer)
{
//fprintf(stderr, "deleting msg_buffer ...\n");
delete[] msg_buffer;
msg_buffer = NULL;
}
@ -1012,7 +1259,7 @@ bool NormMsgr::Message::Init(unsigned int size)
} // end NormMsgr::Message::Init()
NormMsgr::MessageQueue::MessageQueue()
: head(NULL), tail(NULL)
: head(NULL), tail(NULL), msg_count(0)
{
}
@ -1038,6 +1285,7 @@ void NormMsgr::MessageQueue::Prepend(Message& msg)
tail = &msg;
msg.next = head;
head = &msg;
msg_count++;
} // end NormMsgr::MessageQueue::Prepend()
void NormMsgr::MessageQueue::Append(Message& msg)
@ -1049,6 +1297,7 @@ void NormMsgr::MessageQueue::Append(Message& msg)
head = &msg;
msg.prev = tail;
tail = &msg;
msg_count++;
} // end NormMsgr::MessageQueue::Append()
void NormMsgr::MessageQueue::Remove(Message& msg)
@ -1062,6 +1311,7 @@ void NormMsgr::MessageQueue::Remove(Message& msg)
else
msg.next->prev = msg.prev;
msg.prev = msg.next = NULL;
msg_count--;
} // end NormMsgr::MessageQueue::Remove()
NormMsgr::Message* NormMsgr::MessageQueue::RemoveHead()

422
examples/normServer.cpp
View File

@ -36,14 +36,27 @@
// socket" approach could be supported for unicast and SSM streams without too much
// difficulty.
#include "normSocket.h"
#include <arpa/inet.h> // for inet_ntoa
#include <stdio.h> // for fprintf()
#include <string.h> // for memcmp()
#include <map> // for std::map<>
#include <assert.h> // more obvious
#include "normSocket.h"
#ifdef WIN32
#include "win32InputHandler.cpp" // to include full implementation
#include <Winsock2.h> // for inet_ntoa()
#include <Ws2tcpip.h> // for inet_ntop()
#else
#include <arpa/inet.h> // for inet_ntoa
#include <sys/select.h>
#include <fcntl.h> // for, well, fnctl()
#include <errno.h> // obvious child
#endif // if/else WIN32/UNIX
void Usage()
{
fprintf(stderr, "Usage: normServer [listen [<groupAddr>/]<port>][debug <level>][trace]\n");
}
// Our "server" indexes clients by their source addr/port
class ClientInfo
@ -58,11 +71,11 @@ class ClientInfo
UINT16 GetPort() const
{return client_port;}
const char* GetAddressString() const;
void Print(FILE* filePtr) const;
const char* GetAddressString();
void Print(FILE* filePtr);
private:
UINT8 addr_version; // 4 or 6
UINT8 addr_version; // 4 or 6
char client_addr[16]; // big enough for IPv6
UINT16 client_port;
@ -110,7 +123,7 @@ int ClientInfo::GetAddressFamily() const
} // end ClientInfo::GetAddressFamily()
const char* ClientInfo::GetAddressString() const
const char* ClientInfo::GetAddressString()
{
static char text[64];
text[63] = '\0';
@ -123,7 +136,7 @@ const char* ClientInfo::GetAddressString() const
return text;
} // end ClientInfo::GetAddressString()
void ClientInfo::Print(FILE* filePtr) const
void ClientInfo::Print(FILE* filePtr)
{
char text[64];
text[63] = '\0';
@ -136,9 +149,45 @@ void ClientInfo::Print(FILE* filePtr) const
fprintf(filePtr, "%s/%hu", text, client_port);
} // end ClientInfo::Print()
class Client
{
public:
Client(NormSocketHandle clientSocket);
~Client();
NormSocketHandle GetSocket() const
{return client_socket;}
bool GetWriteReady() const
{return write_ready;}
void SetWriteReady(bool state)
{write_ready = state;}
unsigned int GetBytesWritten() const
{return bytes_written;}
void SetBytesWritten(unsigned long numBytes)
{bytes_written = numBytes;}
private:
NormSocketHandle client_socket;
// These are state variables for unicast server -> client communication
bool write_ready;
unsigned int bytes_written;
}; // end class Client
Client::Client(NormSocketHandle clientSocket)
: client_socket(clientSocket),
write_ready(true), bytes_written(0)
{
}
Client::~Client()
{
}
// C++ map used to index client sessions by the client source addr/port
typedef std::map<ClientInfo, NormSocketHandle> ClientMap;
typedef std::map<ClientInfo, Client*> ClientMap;
ClientInfo NormGetClientInfo(NormNodeHandle client)
{
@ -146,32 +195,46 @@ ClientInfo NormGetClientInfo(NormNodeHandle client)
unsigned int addrLen = 16;
UINT16 port;
NormNodeGetAddress(client, addr, &addrLen, &port);
int addrFamily;
UINT8 version;
if (4 == addrLen)
{
addrFamily = AF_INET;
version = 4;
}
else
{
addrFamily = AF_INET6;
version = 6;
}
return ClientInfo(version, addr, port);
} // end NormGetClientInfo()
} // end NormGetClientInfo(NormNodeHandle)
NormSocketHandle FindClientSocket(ClientMap& clientMap, const ClientInfo& clientInfo)
ClientInfo NormGetSocketInfo(NormSocketHandle socket)
{
char addr[16]; // big enough for IPv6
unsigned int addrLen = 16;
UINT16 port;
NormGetPeerName(socket, addr, &addrLen, &port);
UINT8 version;
if (4 == addrLen)
version = 4;
else
version = 6;
return ClientInfo(version, addr, port);
} // end NormGetSocketInfo(NormSocketHandle)
Client* FindClient(ClientMap& clientMap, const ClientInfo& clientInfo)
{
ClientMap::iterator it = clientMap.find(clientInfo);
if (clientMap.end() != it)
return &(it->second);
return it->second;
else
return NULL;
} // end FindClient()
NormSocketHandle FindClientSocket(ClientMap& clientMap, const ClientInfo& clientInfo)
{
Client* client = FindClient(clientMap, clientInfo);
if (NULL == client)
return NORM_SOCKET_INVALID;
else
return client->GetSocket();
} // end FindClientSocket()
int main(int argc, char* argv[])
{
ClientMap clientMap;
@ -185,7 +248,6 @@ int main(int argc, char* argv[])
bool trace = false;
unsigned int debugLevel = 0;
for (int i = 1; i < argc; i++)
{
const char* cmd = argv[i];
@ -210,11 +272,13 @@ int main(int argc, char* argv[])
if (1 != sscanf(portPtr, "%hu", &serverPort))
{
fprintf(stderr, "normServer error: invalid <port> \"%s\"\n", portPtr);
Usage();
return -1;
}
}
else if (0 == strncmp(cmd, "interface", len))
{
// Note the NormSocket code does not yet expose mcast interface
mcastInterface = argv[++i];
}
else if (0 == strncmp(cmd, "trace", len))
@ -226,37 +290,42 @@ int main(int argc, char* argv[])
if (1 != sscanf(argv[++i], "%u", &debugLevel))
{
fprintf(stderr, "normServer error: invalid debug level\n");
Usage();
return -1;
}
}
else
{
fprintf(stderr, "normServer error: invalid command \"%s\"\n", cmd);
Usage();
return -1;
}
}
// For unicast operation in this demo app, the server only "talks back"
// to one client on a first come, first serve basis. (Multiple clients
// can connect and send data _to_ the server, but in this simple example,
// the server only sends to one at a time. For the multicast server case,
// the server multicasts to the entire group.
NormSocketHandle firstClientSocket = NORM_SOCKET_INVALID;
NormInstanceHandle instance = NormCreateInstance();
NormSocketHandle serverSocket = NormListen(instance, serverPort, groupAddrPtr);
NormSocketHandle serverSocket = NormOpen(instance);
if (trace) NormSetMessageTrace(NormGetSession(serverSocket), true);
NormListen(serverSocket, serverPort, groupAddrPtr);
if (trace) NormSetMessageTrace(NormGetSocketSession(serverSocket), true);
if (0 != debugLevel) NormSetDebugLevel(debugLevel);
//NormSetDebugLevel(8);
//NormSetMessageTrace(NormGetSession(serverSocket), true);
//NormSetMessageTrace(NormGetSocketSession(serverSocket), true);
#ifdef WIN32
HANDLE hStdout = GetStdHandle(STD_OUTPUT_HANDLE);
Win32InputHandler inputHandler;
inputHandler.Open();
HANDLE handleArray[2];
handleArray[0] = NormGetDescriptor(instance);
handleArray[1] = inputHandler.GetEventHandle();
#else
// We use a select() call to multiplex input reading and NormSocket handling
fd_set fdset;
FD_ZERO(&fdset);
// Get our input (STDIN) descriptor and set non-blocking
FILE* inputFile = stdin;
int inputfd = fileno(inputFile);
@ -264,15 +333,48 @@ int main(int argc, char* argv[])
perror("normClient: fcntl(inputfd, O_NONBLOCK) error");
// Get our NormInstance descriptor
int normfd = NormGetDescriptor(instance);
#endif // if/else WIN32/UNIX
bool keepGoing = true;
bool writeReady = false;
unsigned int inputLength = 0;
int inputLength = 0;
unsigned int bytesWritten = 0;
const unsigned int BUFFER_LENGTH = 2048;
char inputBuffer[BUFFER_LENGTH];
bool inputNeeded = false; // will be set to "true" upon CONNECT
bool inputClosed = false;
unsigned int clientCount = 0;
while (keepGoing)
{
bool normEventPending = false;
bool inputEventPending = false;
#ifdef WIN32
DWORD handleCount = inputNeeded ? 2 : 1;
DWORD waitStatus =
MsgWaitForMultipleObjectsEx(handleCount, // number of handles in array
handleArray, // object-handle array
INFINITE, // time-out interval
QS_ALLINPUT, // input-event type
0);
if ((WAIT_OBJECT_0 <= waitStatus) && (waitStatus < (WAIT_OBJECT_0 + handleCount)))
{
if (0 == (waitStatus - WAIT_OBJECT_0))
normEventPending = true;
else
inputEventPending = true;
}
else if (-1 == waitStatus)
{
perror("normServer: MsgWaitForMultipleObjectsEx() error");
break;
}
else
{
// TBD - any other status we should handle?
// (e.g. WAIT_TIMEOUT, WAIT_ABANDONED or WAIT_IO_COMPLETION)
continue; // ignore for now
}
#else
FD_SET(normfd, &fdset);
int maxfd = normfd;
if (inputNeeded)
@ -287,12 +389,33 @@ int main(int argc, char* argv[])
int result = select(maxfd+1, &fdset, NULL, NULL, NULL);
if (result <= 0)
{
perror("normClient: select() error");
perror("normServer: select() error");
break;
}
if (FD_ISSET(inputfd, &fdset))
inputEventPending = true;
if (FD_ISSET(normfd, &fdset))
normEventPending = true;
#endif // if/else WIN32/UNIX
if (inputEventPending)
{
// Read input into our txBuffer
#ifdef WIN32
inputLength = inputHandler.ReadData(inputBuffer, BUFFER_LENGTH);
if (inputLength > 0)
{
// We got our input
bytesWritten = 0;
inputNeeded = false;
}
else if (inputLength < 0)
{
inputHandler.Close();
inputClosed = true;
}
// else zero bytes read, still need input
#else
inputLength = fread(inputBuffer, 1, BUFFER_LENGTH, inputFile);
if (inputLength > 0)
{
@ -302,10 +425,13 @@ int main(int argc, char* argv[])
}
else if (feof(inputFile))
{
// TBD - initiate server shutdown if it's job is
// only until the input is closed
// Meanwhile, we just stick around to receive stuff from the clients
inputNeeded = false; // TBD - should also fclose(inputFile)??
if (stdin != inputFile)
{
fclose(inputFile);
inputFile = NULL;
}
inputClosed = true;
}
else if (ferror(inputFile))
{
@ -318,12 +444,35 @@ int main(int argc, char* argv[])
// input starved, wait for next notification
break;
default:
perror("normClient: error reading input");
perror("normServer: error reading input?!");
break;
}
}
}
if (FD_ISSET(normfd, &fdset))
#endif // if/else WIN32/UNIX
if (inputClosed)
{
inputNeeded = false;
// Gracefully shutdown any connected clients
// TBD - set state variable to indicate exit upon all closed?
if (clientMap.empty())
{
keepGoing = false;
continue;
}
else
{
ClientMap::iterator it;
for (it = clientMap.begin(); it != clientMap.end(); it++)
{
Client* client = it->second;
NormSocketHandle clientSocket = client->GetSocket();
NormShutdown(clientSocket);
}
}
}
} // end if inputEventPending
if (normEventPending)
{
// There's a NORM event pending
@ -333,6 +482,8 @@ int main(int argc, char* argv[])
ClientInfo clientInfo;
if (NORM_NODE_INVALID != event.sender)
clientInfo = NormGetClientInfo(event.sender);
else
clientInfo = NormGetSocketInfo(event.socket);
switch (event.type)
{
case NORM_SOCKET_ACCEPT:
@ -344,26 +495,44 @@ int main(int argc, char* argv[])
if (NORM_SOCKET_INVALID != FindClientSocket(clientMap, clientInfo))
{
// We think we're already connected to this client
fprintf(stderr, "normServer: duplicative %s client ...\n",
(NORM_REMOTE_SENDER_NEW == event.event.type) ? "new" : "reset");
fprintf(stderr, "normServer: duplicative %s from client %s/%hu...\n",
(NORM_REMOTE_SENDER_NEW == event.event.type) ? "new" : "reset",
clientInfo.GetAddressString(), clientInfo.GetPort());
continue;
}
NormSocketHandle clientSocket = NormAccept(serverSocket, event.sender);
// TBD - For multicast, if we are sending a tx_stream, we could flush it here to
Client* client = new Client(clientSocket);
if (NULL == client)
{
perror("normServer: new Client() error");
NormClose(clientSocket);
continue;
}
// TBD - For multicast, if we are sending a tx_stream, we could flush it here to
// bring the new receiver "up to date" ... probably would be best to
// do this on some sort of timer-basis in the case of a bunch of receivers
// joining in a short window of time ...
if (trace) // note we're already tracing the mcast session
NormSetMessageTrace(NormGetSession(clientSocket), true);
//NormSetMessageTrace(NormGetSession(clientSocket), true);
clientMap[clientInfo] = clientSocket;
NormSetMessageTrace(NormGetSocketSession(clientSocket), true);
//NormSetMessageTrace(NormGetSocketSession(clientSocket), true);
clientMap[clientInfo] = client;
// ACCEPTED is good as CONNECTED, so enable writing right away
client->SetWriteReady(true);
if (0 == clientCount)
{
// We have at least one client, so lets serve up some juicy input
inputNeeded = true;
writeReady = true;
}
clientCount++;
fprintf(stderr, "normServer: ACCEPTED connection from %s/%hu\n",
clientInfo.GetAddressString(), clientInfo.GetPort());
// We have at least one client, so lets serve up some juicy input
inputNeeded = true;
writeReady = true;
// Note that an ACCEPTED socket is essentially CONNECTED, so we could
// go ahead and set writeReady to true, etc here
}
else
{
@ -375,8 +544,8 @@ int main(int argc, char* argv[])
{
fprintf(stderr, "normServer: CONNECTED to %s/%hu ...\n",
clientInfo.GetAddressString(), clientInfo.GetPort());
if (NORM_SOCKET_INVALID == firstClientSocket)
firstClientSocket = event.socket;
Client* client = FindClient(clientMap, clientInfo);
assert(NULL != client);
break;
}
case NORM_SOCKET_READ:
@ -395,36 +564,60 @@ int main(int argc, char* argv[])
continue;
}
if (bytesRead > 0)
{
#ifdef WIN32
// Use WriteFile() so there is no buffer delay
DWORD dwWritten;
WriteFile(hStdout, buffer, bytesRead, &dwWritten, NULL);
#else
fwrite(buffer, sizeof(char), bytesRead, stdout);
#endif // if/else WIN32
}
if (bytesRead < 1024) rxReady = false;
}
// Following lines are test code (to immediately close connection after first reaad)
// (tests server stale connection "reject" command)
//clientMap.erase(clientInfo);
//NormClose(event.socket);
break;
}
case NORM_SOCKET_WRITE:
// We only demo server data transmission for the multicast server case
// (see comment below)
if ((NULL != groupAddrPtr) || (event.socket == firstClientSocket))
{
if (NULL != groupAddrPtr)
{
// We are a multicast server
writeReady = true;
break;
}
else
{
Client* client = FindClient(clientMap, clientInfo);
assert(NULL != client);
client->SetWriteReady(true);
}
break;
}
case NORM_SOCKET_CLOSING:
{
fprintf(stderr, "normServer: client %s/%hu CLOSING connection ...\n",
clientInfo.GetAddressString(), clientInfo.GetPort());
if (event.socket == firstClientSocket)
firstClientSocket = NORM_SOCKET_INVALID;
break;
case NORM_SOCKET_CLOSED:
Client* client = FindClient(clientMap, clientInfo);
assert(NULL != client);
client->SetWriteReady(false);
break;
}
case NORM_SOCKET_CLOSE:
{
fprintf(stderr, "normServer: connection to client %s/%hu CLOSED ...\n",
clientInfo.GetAddressString(), clientInfo.GetPort());
if (event.socket == firstClientSocket)
firstClientSocket = NORM_SOCKET_INVALID;
clientMap.erase(clientInfo);
break;
NormClose(event.socket);
if (inputClosed && clientMap.empty())
keepGoing = false;
break;
}
case NORM_SOCKET_NONE:
break;
break;
} // end switch(event.type)
}
else
@ -433,31 +626,96 @@ int main(int argc, char* argv[])
}
} // end if FD_ISSET(normfd)
// For our _multicast_ "normServer" example, the server can send to the group
// (For a unicast "normServer", we would need to do something more complex
// to manage sending data to each individual client that connects to us.
// So, for the moment, the unicast "normServer" only sends to the "firstClientSocket"
NormSocketHandle sendSocket = (NULL != groupAddrPtr) ? serverSocket : firstClientSocket;
if (NORM_SOCKET_INVALID != sendSocket)
// If the normServer app has unsent data from STDIN, send it to the clients.
// Note that a _multicast_ server multicasts to all clients at once while a
// unicast server sends to each connected client individually
if ((inputLength > 0) && !inputNeeded)
{
if (writeReady && (inputLength > 0))
// There is inputBuffer data for the server to send to the client(s)
if (NULL == groupAddrPtr)
{
// We have data in our inputBuffer and the NormSocket is "writeReady", so send it
bytesWritten += NormWrite(sendSocket, inputBuffer + bytesWritten, inputLength - bytesWritten);
if (bytesWritten < inputLength)
// Unicast the data to each connected client individually by iterating
// over the clientMap and sending data out to each pending client socket
// (inputNeeded is reset to "true" when _all_ clients are non-pending)
bool clientPending = false;
ClientMap::iterator it;
for (it = clientMap.begin(); it != clientMap.end(); it++)
{
// Couldn't write whole inputBuffer, need to wait for NORM_SOCKET_WRITE event
writeReady = false;
}
else
Client* client = it->second;
if (!client->GetWriteReady())
{
clientPending = true;
continue;
}
unsigned int numBytes = client->GetBytesWritten();
if (numBytes < inputLength)
{
NormSocketHandle clientSocket = client->GetSocket();
bytesWritten += NormWrite(clientSocket, inputBuffer + numBytes, inputLength - numBytes);
client->SetBytesWritten(numBytes);
if (bytesWritten < inputLength)
{
// Couldn't write whole inputBuffer, need to wait for NORM_SOCKET_WRITE event
// for this client socket
client->SetWriteReady(false);
clientPending = true;
}
else
{
// inputBuffer has been completely written to this client
NormFlush(clientSocket);
}
}
}
if (!clientPending)
{
// inputBuffer has been completely written
// inputBuffer was sent to _all_ clients, so reset
inputLength = 0;
inputNeeded = true;
NormFlush(sendSocket);
// Reset all client "bytes_written" to zero for next chunk of data
for (it = clientMap.begin(); it != clientMap.end(); it++)
it->second->SetBytesWritten(0);
}
}
}
else
{
// Multicast the data out the "serverSocket" to all clients
// (Use the 'writeReady' and 'bytesWritten' state variables)
NormSocketHandle sendSocket = serverSocket;
if (NORM_SOCKET_INVALID != sendSocket)
{
if (writeReady && (inputLength > 0))
{
// We have data in our inputBuffer and the NormSocket is "writeReady", so send it
bytesWritten += NormWrite(sendSocket, inputBuffer + bytesWritten, inputLength - bytesWritten);
if (bytesWritten < inputLength)
{
// Couldn't write whole inputBuffer, need to wait for NORM_SOCKET_WRITE event
writeReady = false;
}
else
{
// inputBuffer has been completely written
inputLength = 0;
inputNeeded = true;
NormFlush(sendSocket);
}
}
}
}
} // end if ((inputLength > 0) && !inputNeeded)
} // end while (keepGoing)
#ifdef WIN32
inputHandler.Close();
#else
if ((stdin != inputFile) && (NULL != inputFile))
{
fclose(inputFile);
inputFile = NULL;
}
#endif // if/else WIN32
NormClose(serverSocket);
serverSocket = NORM_SOCKET_INVALID;
} // end main()

793
examples/normSocket.cpp
View File

File diff suppressed because it is too large Load Diff

52
examples/normSocket.h
View File

@ -19,7 +19,7 @@
//
// 0) Make "NormInstanceHandle" optional for Listen/Connect/Accept
// (for eventual blocking usage, it wouldn't be needed)
// 1) Support asymmetric server->multicast, client->unicast model
// 1) Support asymmetric server->multicast, client->unicast model (DONE)
// 2) Provide non-blocking option for read/write calls
// 3) leverage ICMP (e.g., port unreachable) feedback to detect connect() failure (DONE)
// 4) provide means for connect() to time out??? (DONE)
@ -41,6 +41,12 @@
// f) Buffer size (sender, stream, receiver)
// g) cache bounds / limits
#ifdef WIN32
#include <BaseTsd.h> // for SSIZE_T
typedef SSIZE_T ssize_t;
#endif // WIN32
#include "normApi.h"
typedef const void* NormSocketHandle;
@ -55,42 +61,55 @@ extern const double NORM_DEFAULT_CONNECT_TIMEOUT;
// Main NormSocket API Functions
NormSocketHandle NormOpen(NormInstanceHandle instance = NORM_INSTANCE_INVALID);
NormSocketHandle NormOpen(NormInstanceHandle instance);
NormSocketHandle NormListen(NormInstanceHandle instance, UINT16 serverPort, const char* groupAddr = NULL);
bool NormListen(NormSocketHandle normSocket,
UINT16 serverPort,
const char* groupAddr = NULL,
const char* serverAddr = NULL);
NormSocketHandle NormConnect(NormInstanceHandle instance,
const char* serverAddr,
UINT16 serverPort,
const char* groupAddr = NULL,
NormNodeId clientId = NORM_NODE_NONE);
bool NormConnect(NormSocketHandle normSocket,
const char* serverAddr,
UINT16 serverPort,
UINT16 localPort = 0,
const char* groupAddr = NULL,
NormNodeId clientId = NORM_NODE_ANY);
NormSocketHandle NormAccept(NormSocketHandle serverSocket, NormNodeHandle clientNode, NormInstanceHandle instance = NORM_INSTANCE_INVALID);
NormSocketHandle NormAccept(NormSocketHandle serverSocket,
NormNodeHandle clientNode,
NormInstanceHandle instance = NORM_INSTANCE_INVALID);
void NormShutdown(NormSocketHandle normSocket);
void NormClose(NormSocketHandle normSocket);
ssize_t NormRead(NormSocketHandle normSocket, void *buf, size_t nbyte);
ssize_t NormRead(NormSocketHandle normSocket, void* buf, size_t nbyte);
ssize_t NormWrite(NormSocketHandle normSocket, const void *buf, size_t nbyte);
ssize_t NormWrite(NormSocketHandle normSocket, const void* buf, size_t nbyte);
int NormFlush(NormSocketHandle normSocket);
// Helper API functions
// NormSocket helper functions
NormSessionHandle NormGetSession(NormSocketHandle normSocket);
NormSessionHandle NormGetMulticastSession(NormSocketHandle normSocket);
void NormSetSocketUserData(NormSocketHandle normSocket, const void* userData);
const void* NormGetSocketUserData(NormSocketHandle normSocket);
NormInstanceHandle NormGetSocketInstance(NormSocketHandle normSocket);
NormSessionHandle NormGetSocketSession(NormSocketHandle normSocket);
NormSessionHandle NormGetSocketMulticastSession(NormSocketHandle normSocket);
void NormGetPeerName(NormSocketHandle normSocket, char* addr, unsigned int* addrLen, UINT16* port);
void NormSetSocketTrace(NormSocketHandle normSocket, bool enable);
typedef enum NormSocketEventType
{
NORM_SOCKET_NONE = 0,
NORM_SOCKET_NONE = 0, // applications should generally ignore these
NORM_SOCKET_ACCEPT, // only issued for listening server sockets
NORM_SOCKET_CONNECT, // notification confirming connection to server
NORM_SOCKET_READ, // edge-triggered notification that socket is ready for reading
NORM_SOCKET_WRITE, // edge-triggered notification that socket is ready for writing
NORM_SOCKET_CLOSING, // indicates remote endpoint is closing socket (only read data at this point)
NORM_SOCKET_CLOSED // indicates socket is now closed (invalid for further operations)
NORM_SOCKET_CLOSE // indicates socket is now closed (invalid for further operations)
} NormSocketEventType;
typedef struct
@ -116,5 +135,4 @@ typedef struct
bool NormGetSocketEvent(NormInstanceHandle normInstance, NormSocketEvent* event, bool waitForEvent = true);
#endif // !_NORM_SOCKET

155
examples/normSocketNotes.txt Normal file
View File

@ -0,0 +1,155 @@
NormSocket API Extension Notes
The NormSocket API (currently defined in examples/normSocket.h) is an in-development extension to the
base (low-level) NORM API (defined in include/normApi.h) that provides a more familiar (socket-like),
easier-to-use API for some specific NORM use patterns. The use patterns supported here are those of a
client-server paradigm where "clients" have individual, reliable connections to the server and the
"server" can have either a one-to-many multicast connection or individual unicast connections to the set
of clients. The paradigm follows a connection-oriented socket model where the server "listens" for
incoming connections initiated by clients. The low-level NORM API is used to establish multiple NORM
sessions as needed to support the necessary data exhange relationships. Since this is a
connection-oriented model, explicit ACK-based flow control used for both client-to-server and
server-to-client(s) data transfers. This is a more controlled model of NORM use as compared to the
looser "connection-free" model of operation NORM use has typically observed.
The NormSocket API code makes careful use of operating system socket-binding to enable port reuse and
simpler session establishment where clients can "connect" to the server listening on a well-known port
as compared to typical base NORM API use where participants can exchange data on a a priori known common
port number (and group address if multicast). However, the clients do need to know the unicast address
of the server and the paradigm is somewhat that of Single-Source Multicast (SSM). Client application
can, of course, using multiple NormSocket instances connect to multiple multicast (or unicast) servers..
THere
A server application can use the NormListen() call to establish a session that listens for incoming
connection requests, Client applications can use the NormConnect() call to initiate a connection to a
listening server. In response, the server application can use NormAccept() to establish connections with
the clients.
At the moment a NormGetSocketEvent() (analogous to the NormGetNextEvent() call in the base NORM API) can
be used to provide cues to changes in NormSocket state (notification of incoming connections, connection
establishment, read/write readiness, etc.). A descriptor is provided for asynchronous I/O notification
of pending events. At the moment, the default behavior is a single NormInstance, and hence a single
descriptor, can be used for multiple NormSockets, but it is possible to call NormCreateInstance() and
associate a separate NormInstance (and hence separate descriptor) with each NormSocket that is
listending, connected or accepted if desired.
As the NormSocket API extension matures, more examples and documentation will be provided as well as
incorporating it (possibly directly) into the "normApi.h" header file and NORM library. At the moment,
"examples/normClient.cpp" and "examples/normServer.cpp" illustrate (and test) basic usage of the API.
The "normServer" can be run as a unicast server using the syntax:
./normServer listen <serverPort>
or a multicast server using the syntax:
./normServer listen <groupAddr>/<serverPort>
Corresponding "normClient" instances can connect to unicast server using the syntax:
./normClient connect <serverAddr>/<serverPort>
or to a multicast server using the syntax:
./normClient connect <serverAddr>/<serverPort>,<groupAddr>
There are some additional command-line options shared by these examples such as "trace" to enable NORM's
message trace output and "debug <debugLevel>" to invoke different levels of debug output. Note that an
option to specify the multicast interface is planned but not yet implemented in these examples.
Similarly, an option to set a specific unicast server binding will also be provided.
TODO (not in priority order):
1) When client initiates connection and server-listener receives NORM_CMD(CC), add code
to directly "inject" the NORM_CMD(CC) into the newly created server-side client
session such that NORM_ACK is immediately sent. This will give client a more immediate
RTT measurement, etc.
2) Add options to set transmit rate, congestion control, etc options via NormSocketHandle
(current code works at default tx rate of 64 kbps)
3) The server-listener session should be modified so NORM_DATA messages are ignored. Also,
NormSetServerListener() could also set the session a silent receiver, too?
4) Add support for additional transport modes in addition to current byte stream:
` 1) Message stream
2) NORM_OBJECT_DATA (w/ data copy option?)
3) NORM_OBJECT_FILE
5) Add option to specify multicast "interface" and use in examples.
6) Enable setting a specific unicast server binding address.
===========================================================================
MISC. DEVELOPMENT NOTES
// Notional program outline
// (optional mcastAddr iff asymm multicast/unicast (i.e. SSM))
normSocket = NormListen(port, [mcastAddr])
_or_
normSocket = NormConnect(addr, port, [mcastAddr])
while NormGetSocketEvent(normSocket)
{
switch (event.GetType())
NORM_ACCEPT:
clientSocket = NormAccept(normSocket, client, [instance]); // rx-only for asymm mcast case
NORM_CONNECTED:
// in response to NormConnect when "server" is detected (NEW_REMOTE_SENDER)
NORM_TX_READY:
upon QUEUE_VACANCY, etc ... socket _might_ be write() ready
NORM_RX_READY:
upon RX_OBJECT_NEW, RX_OBJECT_UPDATE ... socket _might_ be read() ready
NORM_CLOSING: ???
upon stream end received from remote sender
NORM_CLOSED: ???
upon stream end _and_ ack to our stream end
}
*** Server listen on a port (optional mcast address):
- Create NormSession that receive-only on given port (if it's asymm mcast, then sender enabled, too)
- For each client (detected via NEW_REMOTE_SENDER yielding a NormSocket::NORM_ACCEPT event)
Create tightly binded session for rx (and tx if unicast) ... this will fail if binding already exists
*** Client connect
- Create single socket (common ephemeral tx/rx port) NormSession for server destination addr/port.
- Call NormStartSender() to initiate probing of server
- NORM_CONNECTED upon NEW_REMOTE_SENDER (from server) ???
QUESTIONS:
1) How does a server reject a connection request? (for now, just ignore)
2) How is a "connection" gracefully terminated? (NormShutdown() vs. NormClose()?)
NormClose(graceful = false) ...
a) stop sender/receiver, destroy everything
NormShutdown(graceful = true) ...
a) close stream (gracefully, w/ ack request)
b) upon ack receipt (or timeout) _and_ rx stream end, destroy everything
Upon receipt of rx stream end, close tx stream gracefully ...
initiator -> stream end; upon ack set tx_stream = NULL and wait for rx_stream end
receptor -> upon rx_stream end, initiate tx_stream shutdown and wait for ack

2178
examples/normStreamer.cpp Normal file
View File

File diff suppressed because it is too large Load Diff

243
examples/win32InputHandler.cpp Normal file
View File

@ -0,0 +1,243 @@
#include <windows.h>
#include <stdio.h> // for perror()
#include <string.h> // for memset()
#include <process.h> // for _beginthreadex(), _endthreadex()
// This class creates a child thread to read STDIN into a buffer.
// An event is set that can be monitored by the parent to know
// when input is available. Mutexes are used to regulate the
// thread's reading (i.e., flow control).
class Win32InputHandler
{
public:
Win32InputHandler();
~Win32InputHandler();
bool Open(int buflen = 4096);
void Close();
// The event handle returned here can be used in
// a call like WaitForSignalObject() to get notified
// when there is data to be read.
HANDLE GetEventHandle() const
{return input_event;}
// This does not block and the input_event
// handle is a cue for when to call it
// Returns number of bytes copied into "buffer"
// (-1 is returned upon input error (input closed))
int ReadData(char* buffer, int numBytes);
private:
static unsigned int __stdcall DoThreadOpen(void* param);
unsigned int Run();
HANDLE input_handle;
HANDLE input_event;
HANDLE thread_handle;
bool input_ready;
char* input_buffer;
int input_buflen;
int input_index;
int input_outdex;
CONDITION_VARIABLE buffer_vacancy;
CRITICAL_SECTION buffer_lock;
bool is_running;
int input_length;
bool input_error;
}; // end class Win32InputHandler
Win32InputHandler::Win32InputHandler()
: input_handle(NULL), input_event(NULL), thread_handle(NULL),
input_ready(false), input_buffer(NULL), input_buflen(0),
input_index(0), input_outdex(0), is_running(false),
input_length(0), input_error(false)
{
}
Win32InputHandler::~Win32InputHandler()
{
if (is_running) Close();
}
bool Win32InputHandler::Open(int buflen)
{
if (is_running) Close();
if (NULL != input_buffer)
{
input_buffer = NULL;
delete[] input_buffer;
}
input_buflen = input_index = input_outdex = input_length = 0;
input_error = false;
if (NULL != input_event) CloseHandle(input_event);
// Get the STDIN handle whatever it may be
input_handle = GetStdHandle(STD_INPUT_HANDLE);
/*if (FILE_TYPE_CHAR == GetFileType(input_handle))
{
// The STDIN is a console
DWORD consoleMode;
bool result = GetConsoleMode(input_handle, &consoleMode);
consoleMode &= ~(ENABLE_LINE_INPUT | ENABLE_MOUSE_INPUT | ENABLE_WINDOW_INPUT);
SetConsoleMode(input_handle, consoleMode);
}*/
// Create an event handle, non-signaled initially
input_event = CreateEvent(NULL, TRUE, FALSE, NULL);
if (NULL == input_event)
{
perror("Win32InputHandler::Open() CreateEvent() error");
return false;
}
if (NULL == (input_buffer = new char[buflen]))
{
perror("Win32InputHandler::Open() new input_buffer error");
CloseHandle(input_event);
input_event = NULL;
return false;
}
input_buflen = buflen;
InitializeConditionVariable(&buffer_vacancy);
InitializeCriticalSection(&buffer_lock);
is_running = true;
if (0 == (thread_handle = (HANDLE)_beginthreadex(NULL, 0, DoThreadOpen, this, 0, NULL)))
{
perror("Win32InputHandler::Open() _beginthreadex() error");
DeleteCriticalSection(&buffer_lock);
delete[] input_buffer;
input_buffer = NULL;
CloseHandle(input_event);
input_event = NULL;
return false;
}
return true;
} // end Win32InputHandler::Open()
void Win32InputHandler::Close()
{
EnterCriticalSection(&buffer_lock);
is_running = false;
WakeConditionVariable(&buffer_vacancy); // in case it was full
LeaveCriticalSection(&buffer_lock);
WaitForSingleObject(thread_handle, INFINITE);
DeleteCriticalSection(&buffer_lock);
delete[] input_buffer;
input_buffer = NULL;
CloseHandle(input_event);
input_event = NULL;
} // end Win32InputHandler::Close()
unsigned int __stdcall Win32InputHandler::DoThreadOpen(void* param)
{
Win32InputHandler* inputHandler = reinterpret_cast<Win32InputHandler*>(param);
unsigned int exitStatus = inputHandler->Run();
_endthreadex(exitStatus);
return exitStatus;
} // end Win32InputHandler::DoThreadOpen()
unsigned int Win32InputHandler::Run()
{
// This loop is for the child thread to read input one byte at a time
// (Since consoles/anonymous pipes don't support async i/o)
// The input bytes are copied to a circularly-managed "input_buffer"
// that can be accessed via the "ReadData()" method by the parent
while (is_running)
{
// Read input one byte at a time and buffer
DWORD dwRead = 0;
BOOL result = ReadFile(input_handle, input_buffer+input_index, 1, &dwRead, NULL);
EnterCriticalSection(&buffer_lock);
if (result)
{
if (0 != dwRead)
{
input_length++;
input_index++;
if (input_index >= input_buflen)
input_index = 0;
}
else
{
LeaveCriticalSection(&buffer_lock);
continue;
}
}
else
{
// What do we do on error?!
input_error = true;
is_running = false;
}
// Signals the parent of data ready (or error)
if (!input_ready)
{
SetEvent(input_event);
input_ready = true;
}
// If the buffer has been filled, this blocks this child
// loop until buffer space is available or it is stopped.
while ((input_index == input_outdex) && (0 != input_length) && is_running)
SleepConditionVariableCS(&buffer_vacancy, &buffer_lock, INFINITE);
LeaveCriticalSection(&buffer_lock);
} // end while(is_running)
return (input_error ? 1 : 0);
} // end Win32InputHandler::Run()
// Called by parent to gobble up data (non-blocking read)
// Returns number of bytes read or -1 on error (e.g., STDIN pipe closed)
int Win32InputHandler::ReadData(char* buffer, int numBytes)
{
EnterCriticalSection(&buffer_lock);
if ((0 != input_length) && (0 != numBytes))
{
if (numBytes > input_length)
numBytes = input_length;
input_length -= numBytes;
if (0 == input_length)
{
// On error condition, the event is left triggered
// after the input_buffer is emptied so parent gets
// all data buffered before the error (e.g., eof)
// occurred.
if (input_ready && !input_error)
{
input_ready = false;
ResetEvent(input_event);
}
}
// Copy "numBytes" of available data to caller's "buffer"
// (two-step process when input_buffer wraps)
unsigned int offset = input_outdex;
input_outdex += numBytes;
if (input_outdex >= input_buflen)
{
input_outdex -= input_buflen;
unsigned int count = numBytes - input_outdex;
memcpy(buffer, input_buffer+offset, count);
if (0 != input_outdex)
memcpy(buffer+count, input_buffer, input_outdex);
}
else
{
memcpy(buffer, input_buffer + offset, numBytes);
}
}
else
{
// Doing the error check
if (input_error && (0 != numBytes))
numBytes = -1; // to indicate an error has occurred and child is dead
else
numBytes = 0;
}
// This wakes up the child if it is sleeping because the buffer was full
if (numBytes > 0)
WakeConditionVariable(&buffer_vacancy);
LeaveCriticalSection(&buffer_lock);
return numBytes;
} // end Win32InputHandler::ReadData()

57
examples/wintest.cpp Normal file
View File

@ -0,0 +1,57 @@
#include "win32InputHandler.cpp" // for class Win32InputHandler
#include <windows.h>
#include <stdio.h>
#include <stdlib.h>
#define BUFSIZE 4096
int main(int argc, char** argv)
{
HANDLE hStdout = GetStdHandle(STD_OUTPUT_HANDLE);
if (argc > 1)
{
// Generate random output
const char* text = "This is some text to output\n";
int textlen = strlen(text);
while (true)
{
int count = rand() % 100;
while (count > 0)
{
printf(text);
DWORD dwWritten;
BOOL fSuccess = WriteFile(hStdout, text, textlen, &dwWritten, NULL);
count -= textlen;
}
int delay = (rand() % 1000);
Sleep(1000);
}
return 0;
}
Win32InputHandler inputHandler;
inputHandler.Start();
while (true)
{
char buffer[BUFSIZE];
WaitForSingleObject(inputHandler.GetEventHandle(), INFINITE);
int numBytes = inputHandler.ReadData(buffer, 1024);
if (numBytes < 0) break;
DWORD dwWritten;
BOOL fSuccess = WriteFile(hStdout, buffer, numBytes, &dwWritten, NULL);
}
inputHandler.Stop();
return 0;
} // end main()

51
include/normApi.h
View File

@ -59,8 +59,8 @@ extern "C" {
#endif /* __cplusplus */
#define NORM_VERSION_MAJOR 1
#define NORM_VERSION_MINOR 0
#define NORM_VERSION_PATCH 0
#define NORM_VERSION_MINOR 5
#define NORM_VERSION_PATCH 7
/** NORM API Types */
typedef const void* NormInstanceHandle;
@ -187,7 +187,7 @@ typedef enum NormEventType
NORM_GRTT_UPDATED,
NORM_CC_ACTIVE,
NORM_CC_INACTIVE,
NORM_ACKING_NODE_NEW, // whe NormSetAutoAcking xxx
NORM_ACKING_NODE_NEW, // whe NormSetAutoAcking
NORM_SEND_ERROR, // ICMP error (e.g. destination unreachable)
NORM_USER_TIMEOUT // issues when timeout set by NormSetUserTimer() expires
} NormEventType;
@ -201,6 +201,11 @@ typedef struct
} NormEvent;
// For setting custom NORM_OBJECT_DATA alloc/free functions
typedef char* (*NormAllocFunctionHandle)(size_t);
typedef void (*NormFreeFunctionHandle)(char*);
/** NORM API General Initialization and Operation Functions */
NORM_API_LINKAGE
@ -208,6 +213,9 @@ int NormGetVersion(int* major DEFAULT((int*)0),
int* minor DEFAULT((int*)0),
int* patch DEFAULT((int*)0));
NORM_API_LINKAGE
NormInstanceHandle NormCreateInstance(bool priorityBoost DEFAULT(false));
@ -256,6 +264,11 @@ const NormDescriptor NORM_DESCRIPTOR_INVALID;
NORM_API_LINKAGE
NormDescriptor NormGetDescriptor(NormInstanceHandle instanceHandle);
NORM_API_LINKAGE
void NormSetAllocationFunctions(NormInstanceHandle instance,
NormAllocFunctionHandle allocFunc,
NormFreeFunctionHandle freeFunc);
/** NORM Session Creation and Control Functions */
NORM_API_LINKAGE
@ -288,6 +301,11 @@ void NormCancelUserTimer(NormSessionHandle sessionHandle);
NORM_API_LINKAGE
NormNodeId NormGetLocalNodeId(NormSessionHandle sessionHandle);
NORM_API_LINKAGE
bool NormGetAddress(NormSessionHandle sessionHandle,
char* addrBuffer,
unsigned int* bufferLen,
UINT16* port DEFAULT((UINT16*)0));
NORM_API_LINKAGE
UINT16 NormGetRxPort(NormSessionHandle sessionHandle);
@ -315,6 +333,13 @@ bool NormChangeDestination(NormSessionHandle sessionHandle,
UINT16 sessionPort,
bool connectToSessionAddress DEFAULT(false));
NORM_API_LINKAGE
void NormSetServerListener(NormSessionHandle sessionHandle, bool state);
NORM_API_LINKAGE
bool NormTransferSender(NormSessionHandle sessionHandle, NormNodeHandle sender);
NORM_API_LINKAGE
void NormSetRxPortReuse(NormSessionHandle sessionHandle,
bool enableReuse,
@ -410,7 +435,8 @@ bool NormStartSender(NormSessionHandle sessionHandle,
UINT32 bufferSpace,
UINT16 segmentSize,
UINT16 numData,
UINT16 numParity);
UINT16 numParity,
UINT8 fecId DEFAULT(0));
NORM_API_LINKAGE
void NormStopSender(NormSessionHandle sessionHandle);
@ -583,6 +609,9 @@ bool NormSendCommand(NormSessionHandle sessionHandle,
NORM_API_LINKAGE
void NormCancelCommand(NormSessionHandle sessionHandle);
NORM_API_LINKAGE
void NormSetSynStatus(NormSessionHandle sessionHandle, bool state);
/* NORM Receiver Functions */
@ -646,6 +675,14 @@ NORM_API_LINKAGE
void NormNodeSetRxRobustFactor(NormNodeHandle remoteSender,
int robustFactor);
NORM_API_LINKAGE
bool NormPreallocateRemoteSender(NormSessionHandle sessionHandle,
unsigned long bufferSize,
UINT16 segmentSize,
UINT16 numData,
UINT16 numParity,
unsigned int streamBufferSize DEFAULT(0));
NORM_API_LINKAGE
bool NormStreamRead(NormObjectHandle streamHandle,
char* buffer,
@ -706,6 +743,12 @@ const char* NormDataAccessData(NormObjectHandle objectHandle);
NORM_API_LINKAGE
char* NormDataDetachData(NormObjectHandle objectHandle);
NORM_API_LINKAGE
char* NormAlloc(size_t numBytes);
NORM_API_LINKAGE
void NormFree(char* dataPtr);
NORM_API_LINKAGE
NormNodeHandle NormObjectGetSender(NormObjectHandle objectHandle);

204
include/normMessage.h
View File

@ -134,6 +134,8 @@ class NormObjectSize
// Operators
bool operator==(const NormObjectSize& b) const
{return (b.size == size);}
bool operator!=(const NormObjectSize& b) const
{return (b.size != size);}
NormObjectSize operator+(const NormObjectSize& b) const
{
NormObjectSize result(size);
@ -201,9 +203,9 @@ class NormObjectId
}
bool operator<=(const NormObjectId& id) const
{return ((value == id.value) || (*this<id));}
{return ((value == id.value) || (*this < id));}
bool operator>=(const NormObjectId& id) const
{return ((value == id.value) || (*this>id));}
{return ((value == id.value) || (*this > id));}
bool operator==(const NormObjectId& id) const
{return (value == id.value);}
bool operator!=(const NormObjectId& id) const
@ -212,6 +214,8 @@ class NormObjectId
void operator-=(UINT16 delta)
{value -= delta;}
const char* GetValuePtr() const
{return (const char*)(&value);}
NormObjectId& operator++(int) {value++; return *this;}
NormObjectId& operator--(int) {value--; return *this;}
@ -224,33 +228,93 @@ class NormBlockId
{
public:
NormBlockId() {};
NormBlockId(UINT32 id) {value = id;}
NormBlockId(const NormObjectId& id) {value = (UINT32)id;}
operator UINT32() const {return value;}
NormBlockId(UINT32 id) : value(id) {}
UINT32 GetValue() const
{return value;}
const char* GetValuePtr() const
{return ((const char*)&value);}
bool operator==(const NormBlockId& id) const
{return (value == (UINT32)id);}
{return (value == id.value);}
bool operator!=(const NormBlockId& id) const
{return (value != (UINT32)id);}
{return (value != id.value);}
// These static helper methods provide a "mask" parameter to allow
// for different bit-length NormBlockId values (depends upon FEC encoding scheme).
// The "mask" is in the eye-of-the-beholder, i.e., the given NormObject
// or NormBlockBuffer that is manipulating block id values for protocol purposes
// provide the FEC block "mask" to these methods.
// We may reconsider adding the "mask" as a NormBlockId member variable, but since
// NormBlockId usage is so ubiquitous, it could be added overhead consider within
// the context of sender, object, etc a common fec_block_mask is used.
//INT32 operator-(const NormBlockId& id) const
// {return ((INT32)value - id.value);}
bool operator<(const NormBlockId& id) const
// Compute difference of (a - b). If a non-zero bit "mask" is given, the difference
// is for the masked word size (e.g., mask = 0x00ffffff is a 24-bit integer).
static INT32 Difference(const NormBlockId& a, const NormBlockId& b, UINT32 mask)
{
UINT32 diff = value - id.value;
return ((diff > 0x80000000) || ((0x80000000 == diff) && (value > id.value)));
if (mask)
{
UINT32 sign = (mask ^ (mask >> 1));
UINT32 result = a.value - b.value;
if (0 == (result & sign))
return (INT32)(result & mask);
else if ((result != sign) || (a.value < b.value))
return (INT32)(result | ~mask);
else
return (INT32)(result & mask);
}
else
{
return ((INT32)(a.value - b.value));
}
}
bool operator>(const NormBlockId& id) const
// Compare two block ids. If a non-zero bit "mask" is given, the comparison
// is a "sliding window" (signed) over the bit space. Otherwise, it is
// simply an unsigned value comparison.
// Returns -1, 0, +1 for (a < b), (a == b), and (a > b), respectively
static int Compare(const NormBlockId& a, const NormBlockId& b, UINT32 mask)
{
UINT32 diff = id.value - value;
return ((diff > 0x80000000) || ((0x80000000 == diff) && (id.value > value)));
if (mask)
{
// "Sliding window" comparison
INT32 delta = Difference(a, b, mask);
if (delta < 0)
return -1;
else if (0 == delta)
return 0;
else // if delta > 0
return 1;
}
else if (a.value < b.value)
{
return -1;
}
else if (a.value == b.value)
{
return 0;
}
else // if (a > b)
{
return 1;
}
}
void Increment(UINT32 i, UINT32 mask)
{
value = value + i;
if (mask) value &= mask;
}
NormBlockId& operator++(int ) {value++; return *this;}
void Decrement(UINT32 i, UINT32 mask)
{
if (mask && (value < i))
value = (mask - (i - value) + 1);
else
value -= i;
}
private:
UINT32 value;
}; // end class NormBlockId
@ -308,7 +372,7 @@ class NormHeaderExtension
}; // end class NormHeaderExtension
// This class is some we use to set/get
// This class is what we use to set/get
// FEC Payload Id content. The FEC Payload
// Id format is dependent upon the "fec_id" (FEC Type)
// and, in some cases, its field size ("m") parameter
@ -523,7 +587,10 @@ class NormMsg
{
buffer[SOURCE_ID_OFFSET] = htonl(sourceId);
}
// For messages to be sent, "addr" is destination
void SetDestination(const ProtoAddress& dst) {addr = dst;}
// For message received, "addr" is source
void SetSource(const ProtoAddress& src) {addr = src;}
void AttachExtension(NormHeaderExtension& extension)
{
@ -545,6 +612,8 @@ class NormMsg
{return (Type)(((UINT8*)buffer)[TYPE_OFFSET] & 0x0f);}
UINT16 GetHeaderLength()
{return ((UINT8*)buffer)[HDR_LEN_OFFSET] << 2;}
UINT16 GetBaseHeaderLength()
{return header_length_base;}
UINT16 GetSequence() const
{
return (ntohs((((UINT16*)buffer)[SEQUENCE_OFFSET])));
@ -555,9 +624,11 @@ class NormMsg
}
const ProtoAddress& GetDestination() const {return addr;}
const ProtoAddress& GetSource() const {return addr;}
const char* GetBuffer() {return ((char*)buffer);}
const char* GetBuffer() const {return ((char*)buffer);}
UINT16 GetLength() const {return length;}
void Display() const; // hex output to log
// To retrieve any attached header extensions
bool HasExtensions() const {return (header_length > header_length_base);}
bool GetNextExtension(NormHeaderExtension& ext) const
@ -566,7 +637,7 @@ class NormMsg
UINT16 nextOffset =
(UINT16)(currentBuffer ? (currentBuffer - buffer + (ext.GetLength()/4)) :
(header_length_base/4));
bool result = (nextOffset < (header_length/4));
bool result = HasExtensions() ? (nextOffset < (header_length/4)) : false;
ext.AttachBuffer(result ? (buffer+nextOffset) : (UINT32*)NULL);
return result;
}
@ -627,7 +698,8 @@ class NormObjectMsg : public NormMsg
FLAG_INFO = 0x04,
FLAG_UNRELIABLE = 0x08,
FLAG_FILE = 0x10,
FLAG_STREAM = 0x20
FLAG_STREAM = 0x20,
FLAG_SYN = 0x40
//FLAG_MSG_START = 0x40 deprecated
};
UINT16 GetInstanceId() const
@ -733,6 +805,51 @@ class NormFtiExtension2 : public NormHeaderExtension
};
}; // end class NormFtiExtension2
// Helper class for containing key FTI params
class NormFtiData
{
public:
NormFtiData()
: object_size(NormObjectSize(0)), segment_size(0),
num_data(0), num_parity(0), fec_m(0), instance_id(0) {}
~NormFtiData() {}
void SetObjectSize(const NormObjectSize& objectSize)
{object_size = objectSize;}
void SetSegmentSize(UINT16 segmentSize)
{segment_size = segmentSize;}
void SetFecMaxBlockLen(UINT16 numData)
{num_data = numData;}
void SetFecNumParity(UINT16 numParity)
{num_parity = numParity;}
void SetFecFieldSize(UINT8 fecM)
{fec_m = fecM;}
void SetFecInstanceId(UINT16 instanceId)
{instance_id = instanceId;}
const NormObjectSize& GetObjectSize() const
{return object_size;}
UINT16 GetSegmentSize() const
{return segment_size;}
UINT16 GetFecMaxBlockLen() const
{return num_data;}
UINT16 GetFecNumParity() const
{return num_parity;}
UINT8 GetFecFieldSize() const
{return fec_m;}
UINT16 GetFecInstanceId() const
{return instance_id;}
private:
NormObjectSize object_size;
UINT16 segment_size;
UINT16 num_data;
UINT16 num_parity;
UINT8 fec_m;
UINT16 instance_id;
}; // end class NormFtiData
// This FEC Object Transmission Information assumes "fec_id" == 5 (RFC 5510)
// (this is the fully-defined 8-bit Reed-Solomon codec)
@ -1282,6 +1399,13 @@ class NormCmdCCMsg : public NormCmdMsg
}
bool GetCCNode(NormNodeId nodeId, UINT8& flags, UINT8& rtt, UINT16& rate) const;
// This function uses the "reserved" field of the NORM_CMD(CC) message
// and is not strictly compliant with RFC 5740 when invoked.
enum {FLAG_SYN = 0x01};
bool SynIsSet() const
{return (0 != (FLAG_SYN & ((UINT8*)buffer)[RESERVED_OFFSET]));}
void SetSyn()
{((UINT8*)buffer)[RESERVED_OFFSET] = FLAG_SYN;}
class Iterator;
friend class Iterator;
@ -1389,7 +1513,9 @@ class NormRepairRequest
void SetFlag(NormRepairRequest::Flag theFlag)
{flags |= theFlag;}
void ClearFlag(NormRepairRequest::Flag theFlag)
{flags &= ~theFlag;}
{flags &= ~theFlag;}
void SetFlags(int theFlags)
{flags = theFlags;}
// Returns length (each repair item requires 8 bytes of space)
bool AppendRepairItem(UINT8 fecId,
@ -1425,7 +1551,13 @@ class NormRepairRequest
{return form;}
bool FlagIsSet(NormRepairRequest::Flag theFlag) const
{return (0 != (theFlag & flags));}
UINT16 GetLength() const {return length;}
int GetFlags() const
{return flags;}
UINT16 GetLength() const
{return (ITEM_LIST_OFFSET + length);}
UINT32* GetBuffer() const
{return buffer;}
// Outputs textual representation of RepairRequest content
void Log(UINT8 fecId, UINT8 fecM) const;
@ -1436,10 +1568,10 @@ class NormRepairRequest
// Checks for matching fecId and assumes constant 'm' ?!?!
Iterator(const NormRepairRequest& theRequest, UINT8 fecId, UINT8 fecM);
void Reset() {offset = 0;}
bool NextRepairItem(NormObjectId* objectId,
NormBlockId* blockId,
UINT16* blockLen,
UINT16* symbolId);
UINT16 NextRepairItem(NormObjectId* objectId,
NormBlockId* blockId,
UINT16* blockLen,
UINT16* symbolId);
private:
const NormRepairRequest& request;
UINT8 fec_id;
@ -1731,6 +1863,22 @@ class NormNackMsg : public NormMsg
length += requestLength;
return requestLength;
}
// TBD - add some safety checks to these methods
void InitFrom(NormNackMsg nack)
{
// Copy header from "nack"
memcpy(buffer, nack.buffer, nack.GetHeaderLength());
header_length_base = nack.header_length_base;
length = header_length = nack.GetHeaderLength();
}
void AppendRepairRequest(const NormRepairRequest request)
{
memcpy(buffer+length/4, request.GetBuffer(), request.GetLength());
length += request.GetLength();
}
void ResetPayload()
{length = GetHeaderLength();}
// Message processing
NormNodeId GetSenderId() const

110
include/normNode.h
View File

@ -140,6 +140,10 @@ class NormLossEstimator2
{ignore_loss = state;}
void SetTolerateLoss(bool state)
{tolerate_loss = state;}
unsigned short GetLagIndex() const
{return lag_index;}
private:
enum {DEPTH = 8};
@ -174,7 +178,6 @@ class NormLossEstimator2
{init = true; Sync(theSequence);}
void Sync(unsigned short theSequence)
{lag_index = theSequence;}
void ChangeLagDepth(unsigned int theDepth)
{
theDepth = (theDepth > 20) ? 20 : theDepth;
@ -270,14 +273,13 @@ class NormCCNode : public NormNode
UINT16 cc_sequence;
}; // end class NormCCNode
class NormSenderNode : public NormNode
class NormSenderNode : public NormNode, public ProtoTree::Item
{
public:
enum ObjectStatus {OBJ_INVALID, OBJ_NEW, OBJ_PENDING, OBJ_COMPLETE};
enum RepairBoundary {BLOCK_BOUNDARY, OBJECT_BOUNDARY};
enum SyncPolicy
{
SYNC_CURRENT, // sync to detect transmit point, iff NORM_DATA from first FEC block unless stream
@ -291,6 +293,12 @@ class NormSenderNode : public NormNode
void SetInstanceId(UINT16 instanceId)
{instance_id = instanceId;}
bool PreallocateRxStream(unsigned int bufferSize,
UINT16 segmentSize ,
UINT16 numData,
UINT16 numParity);
bool GetFtiData(const NormObjectMsg& msg, NormFtiData& ftiData);
// Parameters
NormObject::NackingMode GetDefaultNackingMode() const
{return default_nacking_mode;}
@ -317,11 +325,15 @@ class NormSenderNode : public NormNode
void UpdateGrttEstimate(UINT8 grttQuantized);
double GetGrttEstimate() const {return grtt_estimate;}
void ResetGrttNotification() {notify_on_grtt_update = true;}
bool UpdateLossEstimate(const struct timeval& currentTime,
unsigned short theSequence,
bool ecnStatus = false);
double LossEstimate() {return loss_estimator.LossFraction();}
unsigned short GetCurrentSequence() const
{return loss_estimator.GetLagIndex();}
double GetRttEstimate() const {return rtt_estimate;}
void CheckCCFeedback();
@ -340,7 +352,13 @@ class NormSenderNode : public NormNode
UINT16 GetInstanceId() {return instance_id;}
bool IsOpen() const {return is_open;}
void Close();
bool AllocateBuffers(UINT8 fecId, UINT16 fecInstanceId, UINT8 fecM, UINT16 segmentSize, UINT16 numData, UINT16 numParity);
bool AllocateBuffers(unsigned int bufferSpace,
UINT8 fecId,
UINT16 fecInstanceId,
UINT8 fecM,
UINT16 segmentSize,
UINT16 numData,
UINT16 numParity);
bool BuffersAllocated() {return (0 != segment_size);}
void FreeBuffers();
void Activate(bool isObjectMsg);
@ -381,6 +399,15 @@ class NormSenderNode : public NormNode
void DeleteObject(NormObject* obj);
NormObject* GetNextPendingObject()
{
NormObjectId objid;
if (GetNextPending(objid))
return rx_table.Find(objid);
else
return NULL;
}
UINT16 SegmentSize() {return segment_size;}
UINT16 BlockSize() {return ndata;}
UINT16 NumParity() {return nparity;}
@ -437,9 +464,9 @@ class NormSenderNode : public NormNode
unsigned long BufferOverunCount() const
{return segment_pool.OverunCount() + block_pool.OverrunCount();}
unsigned long CurrentStreamBufferUsage() const;
unsigned long PeakStreamBufferUsage() const;
unsigned long StreamBufferOverunCount() const;
unsigned long CurrentStreamBufferUsage();
unsigned long PeakStreamBufferUsage();
unsigned long StreamBufferOverunCount();
//unsigned long RecvTotal() const {return recv_total;}
@ -461,6 +488,7 @@ class NormSenderNode : public NormNode
recv_goodput.Reset();
}
void IncrementResyncCount() {resync_count++;}
void DecrementResyncCount() {resync_count--;}
unsigned long ResyncCount() const {return resync_count;}
unsigned long NackCount() const {return nack_count;}
unsigned long SuppressCount() const {return suppress_count;}
@ -502,9 +530,34 @@ class NormSenderNode : public NormNode
bool ReadNextCmd(char* buffer, unsigned int* buflen);
void SetAddress(const ProtoAddress& address)
{
unsigned int len = address.GetLength();
memcpy(key_buffer, address.GetRawHostAddress(), len);
UINT16 port = htons(address.GetPort());
memcpy(key_buffer+len, &port, 2);
key_size = (len+2) << 3;
NormNode::SetAddress(address);
}
UINT8 GetGrttQuantized() const
{return grtt_quantized;}
UINT8 GetBackoffFactor() const
{return backoff_factor;}
UINT8 GetGroupSizeQuantized() const
{return gsize_quantized;}
UINT16 GetCCSequence() const
{return cc_sequence;}
double GetSendRate() const
{return send_rate;}
private:
const char* GetKey() const
{return key_buffer;}
unsigned int GetKeysize() const
{return key_size;}
static const double DEFAULT_NOMINAL_INTERVAL;
static const double ACTIVITY_INTERVAL_MIN;
@ -523,7 +576,10 @@ class NormSenderNode : public NormNode
void AttachCCFeedback(NormAckMsg& ack);
void HandleRepairContent(const UINT32* buffer, UINT16 bufferLen);
void FragmentNack(NormNackMsg& superNack);
UINT16 instance_id;
int robust_factor;
SyncPolicy sync_policy;
@ -540,6 +596,7 @@ class NormSenderNode : public NormNode
UINT8 fec_m;
unsigned int ndata;
unsigned int nparity;
NormStreamObject* preset_stream;
NormObjectTable rx_table;
ProtoSlidingMask rx_pending_mask;
@ -561,7 +618,7 @@ class NormSenderNode : public NormNode
// Watermark acknowledgement
ProtoTimer ack_timer;
bool ack_pending;
bool ack_pending;
NormObjectId watermark_object_id;
NormBlockId watermark_block_id;
NormSegmentId watermark_segment_id;
@ -574,6 +631,7 @@ class NormSenderNode : public NormNode
double gsize_estimate;
UINT8 gsize_quantized;
double backoff_factor;
bool notify_on_grtt_update; // for API
// Remote sender congestion control state
NormLossEstimator2 loss_estimator;
@ -600,6 +658,10 @@ class NormSenderNode : public NormNode
CmdBuffer* cmd_buffer_tail; // newly-received commands appended here
CmdBuffer* cmd_buffer_pool; // we "pool" allocated buffers for possible reuse here
// Used for NormSocket API extension to index by addr/port
char key_buffer[16+2]; // big enough for IPv6 plus port
unsigned int key_size;
// For statistics tracking
Accumulator recv_total; // total recvd accumulator
Accumulator recv_goodput; // goodput recvd accumulator
@ -612,7 +674,8 @@ class NormSenderNode : public NormNode
}; // end class NormSenderNode
// Used for binary trees of NormNodes
// Used for binary trees of NormNodes sorted by NormNodeId
// (TBD - update to use ProtoTree instead?
class NormNodeTree
{
friend class NormNodeTreeIterator;
@ -639,7 +702,6 @@ class NormNodeTreeIterator
void Reset(NormNode* prevNode = NULL);
NormNode* GetNextNode();
private:
const NormNodeTree& tree;
NormNode* next;
@ -689,4 +751,30 @@ class NormNodeListIterator
const NormNodeList& list;
NormNode* next;
}; // end class NormNodeListIterator
// Used to track remote client sender nodes for server/listener sessions
class NormClientTree : public ProtoTreeTemplate<NormSenderNode>
{
public:
void InsertNode(NormSenderNode& sender)
{
sender.Retain();
Insert(sender);
}
void RemoveNode(NormSenderNode& sender)
{
Remove(sender);
sender.Release();
}
NormSenderNode* FindNodeByAddress(const ProtoAddress& addr)
{
char key[16+2];
unsigned int len = addr.GetLength();
memcpy(key, addr.GetRawHostAddress(), len);
UINT16 port = htons(addr.GetPort());
memcpy(key+len, &port, 2);
return Find(key, (len+2) << 3);
}
}; // end class NormClientTree
#endif // NORM_NODE

174
include/normObject.h
View File

@ -7,7 +7,15 @@
#include <stdio.h>
#define USE_PROTO_TREE 1 // for more better performing NormObjectTable?
#ifdef USE_PROTO_TREE
#include "protoTree.h"
class NormObject : public ProtoSortedTree::Item
#else
class NormObject
#endif // if/else USE_PROTO_TREE
{
friend class NormObjectTable;
@ -44,16 +52,43 @@ class NormObject
// This must be reset after each update
void SetNotifyOnUpdate(bool state)
{
notify_on_update = state;
}
{notify_on_update = state;}
// Object information
NormObject::Type GetType() const {return type;}
const NormObjectId& GetId() const {return transport_id;}
const NormObjectSize& GetSize() const {return object_size;}
void SetId(const NormObjectId transportId)
{transport_id = transportId;}
UINT16 GetSegmentSize() const {return segment_size;}
UINT8 GetFecId() const {return fec_id;}
UINT16 GetFecMaxBlockLen() const {return ndata;}
UINT16 GetFecNumParity() const {return nparity;}
UINT8 GetFecFieldSize() const {return fec_m;}
// returns difference (a - b)
INT32 Difference(NormBlockId a, NormBlockId b) const
{return NormBlockId::Difference(a, b, fec_block_mask);}
// returns -1, 0, or 1 for (a < b), (a == b), or (a > b), respectively
int Compare(NormBlockId a, NormBlockId b) const
{return NormBlockId::Compare(a, b, fec_block_mask);}
void Increment(NormBlockId& b, UINT32 i = 1) const
{b.Increment(i, fec_block_mask);}
void Decrement(NormBlockId& b, UINT32 i = 1) const
{b.Decrement(i, fec_block_mask);}
bool HaveInfo() const {return (info_len > 0);}
bool HasInfo() const {return (NULL != info_ptr);}
void ClearInfo()
{
if (NULL != info_ptr) delete[] info_ptr;
info_ptr = NULL;
info_len = 0;
pending_info = false;
}
const char* GetInfo() const {return info_ptr;}
UINT16 GetInfoLength() const {return info_len;}
@ -94,8 +129,6 @@ class NormObject
}
void Close();
virtual bool WriteSegment(NormBlockId blockId,
NormSegmentId segmentId,
const char* buffer) = 0;
@ -119,17 +152,19 @@ class NormObject
NormBlockId GetFinalBlockId() const {return final_block_id;}
UINT32 GetBlockSize(NormBlockId blockId) const
{
return (((UINT32)blockId < large_block_count) ? large_block_size :
small_block_size);
return ((blockId.GetValue() < large_block_count) ? large_block_size :
small_block_size);
}
NormObjectSize GetBytesPending() const;
bool IsPending(bool flush = true) const;
bool IsRepairPending() const;
bool IsRepairPending();
bool IsPendingInfo() {return pending_info;}
bool PendingMaskIsSet() const
{return pending_mask.IsSet();}
unsigned int GetPendingMaskSize() const
{return pending_mask.GetSize();}
bool GetFirstPending(NormBlockId& blockId) const
{
UINT32 index = 0;
@ -139,7 +174,7 @@ class NormObject
}
bool GetNextPending(NormBlockId& blockId) const
{
UINT32 index = (UINT32)blockId;
UINT32 index = blockId.GetValue();
bool result = pending_mask.GetNextSet(index);
blockId = NormBlockId(index);
return result;
@ -161,7 +196,7 @@ class NormObject
bool GetNextRepair(NormBlockId& blockId) const
{
UINT32 index = (UINT32)blockId;
UINT32 index = blockId.GetValue();
bool result = repair_mask.GetNextSet(index);
blockId = NormBlockId(index);
return result;
@ -174,7 +209,7 @@ class NormObject
return result;
}
bool FindRepairIndex(NormBlockId& blockId, NormSegmentId& segmentId) const;
bool FindRepairIndex(NormBlockId& blockId, NormSegmentId& segmentId);
// Methods available to sender for transmission
bool NextSenderMsg(NormObjectMsg* msg);
@ -185,26 +220,17 @@ class NormObject
void ClearFirstPass() {first_pass = false};*/
bool TxReset(NormBlockId firstBlock = NormBlockId(0), bool requeue = false);
bool TxResetBlocks(NormBlockId nextId, NormBlockId lastId);
bool TxResetBlocks(const NormBlockId& nextId, const NormBlockId& lastId);
bool TxUpdateBlock(NormBlock* theBlock,
NormSegmentId firstSegmentId,
NormSegmentId lastSegmentId,
UINT16 numErasures)
{
NormBlockId blockId = theBlock->GetId();
bool result = theBlock->TxUpdate(firstSegmentId, lastSegmentId,
GetBlockSize(blockId), nparity,
numErasures);
ASSERT(result ? pending_mask.Set(blockId) : true);
result = result ? pending_mask.Set(blockId) : false;
return result;
}
UINT16 numErasures);
bool HandleInfoRequest(bool holdoff);
bool HandleBlockRequest(NormBlockId nextId, NormBlockId lastId);
bool HandleBlockRequest(const NormBlockId& nextId, const NormBlockId& lastId);
NormBlock* FindBlock(NormBlockId blockId) {return block_buffer.Find(blockId);}
bool ActivateRepairs();
bool IsRepairSet(NormBlockId blockId) {return repair_mask.Test(blockId);}
bool IsPendingSet(NormBlockId blockId) {return pending_mask.Test(blockId);}
bool IsRepairSet(NormBlockId blockId) {return repair_mask.Test(blockId.GetValue());}
bool IsPendingSet(NormBlockId blockId) {return pending_mask.Test(blockId.GetValue());}
bool AppendRepairAdv(NormCmdRepairAdvMsg& cmd);
NormBlockId GetMaxPendingBlockId() const {return max_pending_block;}
@ -236,12 +262,12 @@ class NormObject
bool ReceiverRewindCheck(NormBlockId blockId,
NormSegmentId segmentId);
bool IsRepairPending(bool flush);
bool AppendRepairRequest(NormNackMsg& nack, bool flush);
bool AppendRepairRequest(NormNackMsg& nack, bool flush, UINT16 payloadMax);
void SetRepairInfo() {repair_info = true;}
bool SetRepairs(NormBlockId first, NormBlockId last)
{
return (first == last) ? repair_mask.Set(first) :
repair_mask.SetBits(first, repair_mask.Delta(last,first)+1);
return ((first == last) ? repair_mask.Set(first.GetValue()) :
repair_mask.SetBits(first.GetValue(), repair_mask.Difference(last.GetValue(),first.GetValue())+1));
}
void SetLastNackTime(const ProtoTime& theTime)
{last_nack_time = theTime;}
@ -258,7 +284,17 @@ class NormObject
const NormObjectId& objectId);
void Accept() {accepted = true;}
#ifdef USE_PROTO_TREE
// Proto::Tree item required overrides
const char* GetKey() const
{return transport_id.GetValuePtr();}
unsigned int GetKeysize() const
{return (8*sizeof(UINT16));}
ProtoTree::Endian GetEndian() const
{return ProtoTree::GetNativeEndian();}
#endif // USE PROTO_TREE
NormObject::Type type;
class NormSession& session;
class NormSenderNode* sender; // NULL value indicates local (tx) object
@ -269,6 +305,7 @@ class NormObject
UINT16 segment_size;
UINT8 fec_id;
UINT8 fec_m;
UINT32 fec_block_mask;
UINT16 ndata;
UINT16 nparity;
NormBlockBuffer block_buffer;
@ -298,8 +335,9 @@ class NormObject
bool notify_on_update;
const void* user_data; // for NORM API usage only
NormObject* next;
#ifndef USE_PROTO_TREE
NormObject* next;
#endif
}; // end class NormObject
@ -325,7 +363,7 @@ class NormFileObject : public NormObject
return result;
}
bool PadToSize()
{return file.Pad(NormObject::GetSize().GetOffset());}
{return file.IsOpen() ? file.Pad(NormObject::GetSize().GetOffset()) : false;}
virtual bool WriteSegment(NormBlockId blockId,
NormSegmentId segmentId,
@ -349,9 +387,12 @@ class NormDataObject : public NormObject
{
// (TBD) allow support of greater than 4GB size data objects
public:
typedef void (*DataFreeFunctionHandle)(char*);
NormDataObject(class NormSession& theSession,
class NormSenderNode* theSender,
const NormObjectId& objectId);
const NormObjectId& objectId,
DataFreeFunctionHandle dataFreeFunc);
~NormDataObject();
bool Open(char* dataPtr,
@ -380,13 +421,17 @@ class NormDataObject : public NormObject
virtual char* RetrieveSegment(NormBlockId blockId,
NormSegmentId segmentId);
private:
NormObjectSize large_block_length;
NormObjectSize small_block_length;
char* data_ptr;
UINT32 data_max;
bool data_released; // when true, data_ptr is deleted
NormObjectSize large_block_length;
NormObjectSize small_block_length;
char* data_ptr;
UINT32 data_max;
bool data_released; // when true, data_ptr is deleted
DataFreeFunctionHandle data_free_func;
// on NormDataObject destruction
}; // end class NormDataObject
@ -426,7 +471,7 @@ class NormStreamObject : public NormObject
bool GetPushMode() const {return push_mode;}
bool IsOldBlock(NormBlockId blockId) const
{return (!stream_buffer.IsEmpty() && (blockId < stream_buffer.RangeLo()));}
{return (!stream_buffer.IsEmpty() && (Compare(blockId, stream_buffer.RangeLo()) < 0));}
bool IsClosing() {return stream_closing;}
bool HasVacancy()
@ -459,6 +504,11 @@ class NormStreamObject : public NormObject
}
bool StreamAdvance();
NormBlockId GetSyncId() const
{return stream_sync_id;}
NormBlockId GetNextId() const
{return stream_next_id;}
virtual bool WriteSegment(NormBlockId blockId,
NormSegmentId segmentId,
const char* buffer);
@ -479,15 +529,15 @@ class NormStreamObject : public NormObject
bool LockSegments(NormBlockId blockId, NormSegmentId firstId, NormSegmentId lastId);
NormBlockId StreamBufferLo() const {return stream_buffer.RangeLo();}
NormBlockId RepairWindowLo() const;
void Prune(NormBlockId blockId, bool updateStatus);
bool IsFlushPending() {return flush_pending;}
NormBlockId FlushBlockId()
{return (write_index.segment ? write_index.block :
(NormBlockId((UINT32)write_index.block-1)));}
NormSegmentId FlushSegmentId()
{return (write_index.segment ? (write_index.segment-1) :
(ndata-1));}
NormBlockId FlushBlockId() const;
NormSegmentId FlushSegmentId() const
{return (write_index.segment ? (write_index.segment-1) : (ndata-1));}
NormBlockId GetNextBlockId() const
{return (sender ? read_index.block : write_index.block);}
@ -529,6 +579,7 @@ class NormStreamObject : public NormObject
public:
NormBlockId block;
NormSegmentId segment;
UINT16 offset;
};
// Extra state for STREAM objects
bool stream_sync;
@ -551,9 +602,10 @@ class NormStreamObject : public NormObject
bool msg_start;
FlushMode flush_mode;
bool push_mode;
bool stream_broken;
bool stream_closing;
// For threaded API purposes
UINT32 block_pool_threshold;
}; // end class NormStreamObject
@ -588,6 +640,10 @@ class NormSimObject : public NormObject
}; // end class NormSimObject
#endif // SIMULATE
#ifdef USE_PROTO_TREE
class NormObjectTree : public ProtoSortedTreeTemplate<NormObject> {};
#endif // USE_PROTO_TREE
class NormObjectTable
{
public:
@ -600,7 +656,6 @@ class NormObjectTable
void SetRangeMax(UINT16 rangeMax);
void Destroy();
bool IsInited() const {return (NULL != table);}
UINT16 GetRangeMax() const {return range_max;}
bool CanInsert(NormObjectId objectId) const;
bool Insert(NormObject* theObject);
@ -612,7 +667,23 @@ class NormObjectTable
bool IsEmpty() const {return (0 == range);}
UINT32 GetCount() const {return count;}
const NormObjectSize& GetSize() const {return size;}
#ifdef USE_PROTO_TREE
class Iterator
{
public:
Iterator(NormObjectTable& objectTable);
NormObject* GetNextObject();
NormObject* GetPrevObject();
void Reset();
private:
const NormObjectTable& table;
NormObjectTree::Iterator iterator;
NormObject* next_object;
};
#else
class Iterator
{
public:
@ -626,12 +697,19 @@ class NormObjectTable
bool reset;
NormObjectId index;
};
#endif // if/else USE_PROTO_TREE
private:
#ifndef USE_PROTO_TREE
NormObject* Next(NormObject* o) const {return o->next;}
#endif
#ifdef USE_PROTO_TREE
NormObjectTree tree;
#else
NormObject** table;
UINT16 hash_mask;
#endif // if/else USE_PROTO_TREE
UINT16 range_max; // max range of objects that can be kept
UINT16 range; // zero if "object table" is empty
NormObjectId range_lo;

68
include/normSegment.h
View File

@ -4,6 +4,9 @@
#include "normMessage.h"
#include "protoBitmask.h"
#define USE_PROTO_TREE 1 // for more better performing NormBlockBuffer?
// Norm uses preallocated (or dynamically allocated) pools of
// segments (vectors) for different buffering purposes
@ -43,7 +46,11 @@ class NormSegmentPool
bool overrun_flag;
}; // end class NormSegmentPool
#ifdef USE_PROTO_TREE
class NormBlock : public ProtoSortedTree::Item
#else
class NormBlock
#endif // if/else USE_PROTO_TREE
{
friend class NormBlockPool;
friend class NormBlockBuffer;
@ -144,7 +151,7 @@ class NormBlock
UINT8 fecId,
UINT8 fecM,
UINT16 numData,
UINT16 segmentSize);
UINT16 payloadMax);
// Receiver routines
void RxInit(NormBlockId& blockId, UINT16 ndata, UINT16 nparity)
@ -238,7 +245,7 @@ class NormBlock
UINT16 numParity,
NormObjectId objectId,
bool pendingInfo,
UINT16 segmentSize);
UINT16 payloadMax);
void SetLastNackTime(const ProtoTime& theTime)
@ -254,6 +261,15 @@ class NormBlock
void EmptyToPool(NormSegmentPool& segmentPool);
private:
#ifdef USE_PROTO_TREE
const char* GetKey() const
{return blk_id.GetValuePtr();}
unsigned int GetKeysize() const
{return (8*sizeof(UINT32));}
ProtoTree::Endian GetEndian() const
{return ProtoTree::GetNativeEndian();}
#endif // USE_PROTO_TREE
NormBlockId blk_id;
UINT16 size;
char** segment_table;
@ -267,8 +283,7 @@ class NormBlock
ProtoBitmask pending_mask;
ProtoBitmask repair_mask;
ProtoTime last_nack_time; // for stream flow control
NormBlock* next;
NormBlock* next; // used for NormBlockPool
}; // end class NormBlock
class NormBlockPool
@ -314,6 +329,10 @@ class NormBlockPool
bool overrun_flag;
}; // end class NormBlockPool
#ifdef USE_PROTO_TREE
class NormBlockTree : public ProtoSortedTreeTemplate<NormBlock> {};
#endif // USE_PROTO_TREE
class NormBlockBuffer
{
public:
@ -322,18 +341,34 @@ class NormBlockBuffer
NormBlockBuffer();
~NormBlockBuffer();
bool Init(unsigned long rangeMax, unsigned long tableSize = 256);
bool Init(unsigned long rangeMax, unsigned long tableSize, UINT32 fecBlockMask);
void Destroy();
bool Insert(NormBlock* theBlock);
bool Remove(const NormBlock* theBlock);
bool Remove(NormBlock* theBlock);
NormBlock* Find(const NormBlockId& blockId) const;
NormBlockId RangeLo() const {return range_lo;}
NormBlockId RangeHi() const {return range_hi;}
NormBlockId RangeMin() const;
bool IsEmpty() const {return (0 == range);}
bool CanInsert(NormBlockId blockId) const;
#ifdef USE_PROTO_TREE
class Iterator
{
public:
Iterator(NormBlockBuffer& blockBuffer);
NormBlock* GetNextBlock();
void Reset();
private:
NormBlockBuffer& buffer;
NormBlockTree::Iterator iterator;
NormBlock* next_block;
};
#else
class Iterator
{
public:
@ -346,18 +381,29 @@ class NormBlockBuffer
bool reset;
NormBlockId index;
};
#endif // if/else USE_PROTO_TREE
private:
int Compare(NormBlockId a, NormBlockId b) const
{return NormBlockId::Compare(a, b, fec_block_mask);}
INT32 Difference(NormBlockId a, NormBlockId b) const
{return NormBlockId::Difference(a, b, fec_block_mask);}
void Increment(NormBlockId& b, UINT32 i = 1) const
{b.Increment(i, fec_block_mask);}
void Decrement(NormBlockId& b, UINT32 i = 1) const
{b.Decrement(i, fec_block_mask);}
#ifdef USE_PROTO_TREE
NormBlockTree tree;
#else
static NormBlock* Next(NormBlock* b) {return b->next;}
NormBlock** table;
unsigned long hash_mask;
unsigned long hash_mask;
#endif // if/else USE_PROTO_TREE
unsigned long range_max; // max range of blocks that can be buffered
unsigned long range; // zero if "block buffer" is empty
UINT32 fec_block_mask;
NormBlockId range_lo;
NormBlockId range_hi;
}; // end class NormBlockBuffer
#endif // _NORM_SEGMENT

88
include/normSession.h
View File

@ -103,11 +103,18 @@ class NormSessionMgr
NormController* GetController() const {return controller;}
void SetDataFreeFunction(NormDataObject::DataFreeFunctionHandle freeFunc)
{data_free_func = freeFunc;}
NormDataObject::DataFreeFunctionHandle GetDataFreeFunction() const
{return data_free_func;}
private:
ProtoTimerMgr& timer_mgr;
ProtoSocket::Notifier& socket_notifier;
ProtoChannel::Notifier* channel_notifier;
NormController* controller;
ProtoTimerMgr& timer_mgr;
ProtoSocket::Notifier& socket_notifier;
ProtoChannel::Notifier* channel_notifier;
NormController* controller;
NormDataObject::DataFreeFunctionHandle data_free_func;
class NormSession* top_session; // top of NormSession list
@ -161,6 +168,8 @@ class NormSession
};
// General methods
void SetNodeId(NormNodeId nodeId)
{local_node_id = nodeId;}
const NormNodeId& LocalNodeId() const {return local_node_id;}
bool Open();
void Close();
@ -210,7 +219,7 @@ class NormSession
UINT16 GetTxPort() const;
bool SetRxPortReuse(bool enableReuse,
const char* rxAddress, // bind() to <rxAddress>/<sessionPort>
const char* rxAddress = NULL, // bind() to <rxAddress>/<sessionPort>
const char* senderAddress = (const char*)0, // connect() to <senderAddress>/<senderPort>
UINT16 senderPort = 0);
@ -289,6 +298,7 @@ class NormSession
// flow control is imposed
double GetFlowControlDelay() const
{
if (0.0 == flow_control_factor) return 0.0;
double fdelay = (flow_control_factor * (SenderGrtt() * (backoff_factor + 1)));
return ((fdelay > 0.020) ? fdelay : 0.020); // minimum 20 msec flow control
}
@ -301,6 +311,13 @@ class NormSession
bool SetTxCacheBounds(NormObjectSize sizeMax,
unsigned long countMin,
unsigned long countMax);
// For NormSocket API extension support only
void SetServerListener(bool state)
{is_server_listener = state;}
bool IsServerListener() const
{return is_server_listener;}
void Notify(NormController::Event event,
class NormNode* node,
class NormObject* object)
@ -313,7 +330,14 @@ class NormSession
NormMsg* GetMessageFromPool() {return message_pool.RemoveHead();}
void ReturnMessageToPool(NormMsg* msg) {message_pool.Append(msg);}
void QueueMessage(NormMsg* msg);
bool SendMessage(NormMsg& msg);
enum MessageStatus
{
MSG_SEND_FAILED,
MSG_SEND_BLOCKED,
MSG_SEND_OK
};
MessageStatus SendMessage(NormMsg& msg);
void ActivateTimer(ProtoTimer& timer) {session_mgr.ActivateTimer(timer);}
void SetUserData(const void* userData)
@ -334,7 +358,8 @@ class NormSession
UINT32 bufferSpace,
UINT16 segmentSize,
UINT16 numData,
UINT16 numParity);
UINT16 numParity,
UINT8 fecId = 0);
void StopSender();
void SetTxOnly(bool txOnly, bool connectToSessionAddress = false);
bool GetTxOnly() const
@ -385,6 +410,9 @@ class NormSession
bool SenderSendCmd(const char* cmdBuffer, unsigned int cmdLength, bool robust);
void SenderCancelCmd();
void SenderSetSynStatus(bool state)
{syn_status = state;}
// robust factor
void SetTxRobustFactor(int value)
{tx_robust_factor = value;}
@ -407,6 +435,13 @@ class NormSession
void SenderSetExtraParity(UINT16 extraParity)
{extra_parity = extraParity;}
INT32 Difference(NormBlockId a, NormBlockId b) const
{return NormBlockId::Difference(a, b, fec_block_mask);}
int Compare(NormBlockId a, NormBlockId b) const
{return NormBlockId::Compare(a, b, fec_block_mask);}
void Increment(NormBlockId& b, UINT32 i = 1) const
{b.Increment(i, fec_block_mask);}
// EMCON Sender (useful when there are silent receivers)
// (NORM_INFO is redundantly sent)
@ -431,6 +466,8 @@ class NormSession
}
double SenderGrtt() const {return grtt_advertised;}
void ResetGrttNotification()
{notify_on_grtt_update = true;}
void SenderSetGrtt(double grttValue)
{
if (IsSender())
@ -439,7 +476,7 @@ class NormSession
grttValue = (grttValue < grttMin) ? grttMin : grttValue;
}
grtt_quantized = NormQuantizeRtt(grttValue);
grtt_measured = grtt_advertised = NormUnquantizeRtt(grtt_quantized);
grtt_measured = grtt_advertised = NormUnquantizeRtt(grtt_quantized);
}
double SenderGroupSize() {return gsize_measured;}
void SenderSetGroupSize(double gsize)
@ -482,13 +519,21 @@ class NormSession
unsigned long RemoteSenderBufferSize() const
{return remote_sender_buffer_size;}
bool InsertRemoteSender(NormSenderNode& sender);
void DeleteRemoteSender(NormSenderNode& senderNode);
// Call this to do remote sender memory allocations ahead of time
bool PreallocateRemoteSender(UINT16 segmentSize, UINT16 numData, UINT16 numParity);
bool PreallocateRemoteSender(unsigned int bufferSize,
UINT16 segmentSize,
UINT16 numData,
UINT16 numParity,
unsigned int streamBufferSize = 0);
void ReceiverSetUnicastNacks(bool state) {unicast_nacks = state;}
bool ReceiverGetUnicastNacks() const {return unicast_nacks;}
void ReceiverSetUnicastNacks(bool state)
{unicast_nacks = state;}
bool ReceiverGetUnicastNacks() const
{return unicast_nacks;}
void ReceiverSetSilent(bool state)
{receiver_silent = state;}
@ -500,7 +545,7 @@ class NormSession
{return rcvr_ignore_info;}
// The default "rcvr_max_delay = -1" corresponds to typical
// operation where source data forpartially received FEC blocks
// operation where source data for partially received FEC blocks
// are only provided to the app when buffer constraints require it.
// Otherwise, the "maxDelay" corresponds to the max number
// of FEC blocks the receiver waits before passing partially
@ -541,7 +586,7 @@ class NormSession
// Set default "max_pending_range" of NormObjects for reception
void SetRxCacheMax(UINT16 maxCount)
{rx_cache_count_max = maxCount;}
{rx_cache_count_max = (maxCount > 0x7fff) ? 0x7fff : maxCount;}
UINT16 GetRxCacheMax() const
{return rx_cache_count_max;}
@ -560,7 +605,8 @@ class NormSession
#endif // SIMULATE
void SetProbeCount(unsigned probeCount) {probe_count = probeCount;}
bool SenderQueueSquelch(NormObjectId objectId);
private:
// Only NormSessionMgr can create/delete sessions
NormSession(NormSessionMgr& sessionMgr, NormNodeId localNodeId);
@ -607,12 +653,14 @@ class NormSession
UINT16 ccSequence);
void AdjustRate(bool onResponse);
void SetTxRateInternal(double txRate); // here, txRate is bytes/sec
bool SenderQueueSquelch(NormObjectId objectId);
//bool SenderQueueSquelch(NormObjectId objectId);
void SenderQueueFlush();
bool SenderQueueWatermarkFlush();
bool SenderBuildRepairAdv(NormCmdRepairAdvMsg& cmd);
void SenderUpdateGroupSize();
bool SenderQueueAppCmd();
// The following method is only used for NormSocket purposes
bool SenderSendAppCmd(const char* buffer, unsigned int length, const ProtoAddress& dst);
// Receiver message handling routines
void ReceiverHandleObjectMessage(const struct timeval& currentTime,
@ -660,6 +708,7 @@ class NormSession
double tx_rate; // bytes per second
double tx_rate_min;
double tx_rate_max;
unsigned int tx_residual; // for NORM_CMD(CC)/NORM_DATA "packet pairing"
// Sender parameters and state
@ -685,6 +734,7 @@ class NormSession
NormEncoder* encoder;
UINT8 fec_id;
UINT8 fec_m;
INT32 fec_block_mask;
NormObjectId next_tx_object_id;
unsigned int tx_cache_count_min;
@ -761,6 +811,7 @@ class NormSession
char* cmd_buffer;
unsigned int cmd_length;
ProtoTimer cmd_timer;
bool syn_status;
// Receiver parameters
bool is_receiver;
@ -778,6 +829,13 @@ class NormSession
NormSenderNode::SyncPolicy default_sync_policy;
UINT16 rx_cache_count_max;
// For NormSocket server-listener support
bool is_server_listener;
NormClientTree client_tree;
// API-specific state variables
bool notify_on_grtt_update;
// State for some experimental congestion control
bool ecn_ignore_loss;
bool cc_tolerate_loss;

2
include/normVersion.h
View File

@ -36,6 +36,6 @@
#ifndef _NORM_VERSION
#define _NORM_VERSION
#define VERSION "1.5r6"
#define VERSION "1.5.7"
#endif // _NORM_VERSION

9
makefiles/Makefile.common
View File

@ -178,6 +178,15 @@ normMsgr: $(MSGR_OBJ) libnorm.a $(LIBPROTO)
mkdir -p ../bin
cp $@ ../bin/$@
# (normStreamer) message sender/receiver
STREAMER_SRC = $(EXAMPLE)/normStreamer.cpp
STREAMER_OBJ = $(STREAMER_SRC:.cpp=.o)
normStreamer: $(STREAMER_OBJ) libnorm.a $(LIBPROTO)
$(CC) $(CFLAGS) -o $@ $(STREAMER_OBJ) $(LDFLAGS) libnorm.a $(LIBPROTO) $(LIBS)
mkdir -p ../bin
cp $@ ../bin/$@
# (pcap2norm) - parses pcap (e.g. tcpdump) file and prints NORM trace
PCAP_SRC = $(COMMON)/pcap2norm.cpp
PCAP_OBJ = $(PCAP_SRC:.cpp=.o)

9
makefiles/Makefile.linux
View File

@ -35,15 +35,18 @@ SYSTEM_LIBS = -ldl
# (We export these for other Makefiles as needed)
#
SYSTEM_HAVES = -DLINUX -DECN_SUPPORT -DHAVE_IPV6 -DHAVE_GETLOGIN -D_FILE_OFFSET_BITS=64 -DHAVE_LOCKF \
-DHAVE_OLD_SIGNALHANDLER -DHAVE_DIRFD -DHAVE_ASSERT
SYSTEM_HAVES = -DLINUX -DECN_SUPPORT -DHAVE_IPV6 -DHAVE_GETLOGIN -D_FILE_OFFSET_BITS=64 -DHAVE_LOCKF \
-DHAVE_OLD_SIGNALHANDLER -DHAVE_DIRFD -DHAVE_ASSERT -DNO_SCM_RIGHTS -DHAVE_SCHED -DUNIX \
-DUSE_SELECT -DUSE_TIMERFD -DUSE_EVENTFD -DHAVE_PSELECT
SYSTEM_SRC = ../protolib/src/linux/linuxCap.cpp
SYSTEM = linux
export CC = g++
export SYSTEM_CFLAGS = -Wall -Wcast-align -pedantic -fPIC
export SYSTEM_CFLAGS = -Wall -Wcast-align -pedantic -fPIC -Wno-variadic-macros
export SYSTEM_SOFLAGS = -shared -Wl,-soname,libnorm.so.1
SYSTEM_SOEXT = so
export RANLIB = ranlib

4
makefiles/android/AndroidManifest.xml
View File

@ -3,6 +3,6 @@
package="mil.navy.nrl.norm"
android:versionCode="1"
android:versionName="1.0">
<uses-sdk android:minSdkVersion="9"
android:targetSdkVersion="17"/>
<uses-sdk android:minSdkVersion="14"
android:targetSdkVersion="23"/>
</manifest>

2
makefiles/android/jni/Application.mk
View File

@ -1,3 +1,3 @@
APP_ABI := all
APP_PLATFORM := android-9
APP_PLATFORM := android-14
NDK_MODULE_PATH := ../..:../../protolib/

2
makefiles/android/project.properties
View File

@ -9,4 +9,4 @@
android.library=true
# Project target.
target=android-14
target=android-18

7
makefiles/win32/NormDll.vcxproj
View File

@ -1,5 +1,5 @@
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Project DefaultTargets="Build" ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup Label="ProjectConfigurations">
<ProjectConfiguration Include="Debug|Win32">
<Configuration>Debug</Configuration>
@ -13,16 +13,17 @@
<PropertyGroup Label="Globals">
<ProjectGuid>{182006F3-188F-466E-89FE-8421C0478691}</ProjectGuid>
<Keyword>Win32Proj</Keyword>
<WindowsTargetPlatformVersion>10.0.16299.0</WindowsTargetPlatformVersion>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration">
<ConfigurationType>DynamicLibrary</ConfigurationType>
<PlatformToolset>v120</PlatformToolset>
<PlatformToolset>v141</PlatformToolset>
<CharacterSet>MultiByte</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
<ConfigurationType>DynamicLibrary</ConfigurationType>
<PlatformToolset>v120</PlatformToolset>
<PlatformToolset>v141</PlatformToolset>
<CharacterSet>MultiByte</CharacterSet>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />

7
makefiles/win32/NormLib.vcxproj
View File

@ -12,19 +12,22 @@
</ItemGroup>
<PropertyGroup Label="Globals">
<ProjectGuid>{D7B0023C-8798-4918-8DA0-05C9054D70B9}</ProjectGuid>
<WindowsTargetPlatformVersion>10.0.16299.0</WindowsTargetPlatformVersion>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration">
<ConfigurationType>StaticLibrary</ConfigurationType>
<PlatformToolset>v120_xp</PlatformToolset>
<PlatformToolset>v141</PlatformToolset>
<UseOfMfc>false</UseOfMfc>
<CharacterSet>MultiByte</CharacterSet>
<CLRSupport>false</CLRSupport>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
<ConfigurationType>StaticLibrary</ConfigurationType>
<PlatformToolset>v120_xp</PlatformToolset>
<PlatformToolset>v141</PlatformToolset>
<UseOfMfc>false</UseOfMfc>
<CharacterSet>MultiByte</CharacterSet>
<CLRSupport>false</CLRSupport>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
<ImportGroup Label="ExtensionSettings">

13
makefiles/win32/norm/norm.vcxproj
View File

@ -1,5 +1,5 @@
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Project DefaultTargets="Build" ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup Label="ProjectConfigurations">
<ProjectConfiguration Include="Debug|Win32">
<Configuration>Debug</Configuration>
@ -12,17 +12,18 @@
</ItemGroup>
<PropertyGroup Label="Globals">
<ProjectGuid>{6E1308A6-D40F-489E-A4F1-40D859380D64}</ProjectGuid>
<WindowsTargetPlatformVersion>10.0.16299.0</WindowsTargetPlatformVersion>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<PlatformToolset>v120</PlatformToolset>
<PlatformToolset>v141</PlatformToolset>
<UseOfMfc>false</UseOfMfc>
<CharacterSet>MultiByte</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<PlatformToolset>v120</PlatformToolset>
<PlatformToolset>v141</PlatformToolset>
<UseOfMfc>false</UseOfMfc>
<CharacterSet>MultiByte</CharacterSet>
</PropertyGroup>
@ -64,7 +65,7 @@
<AdditionalOptions>/vmg %(AdditionalOptions)</AdditionalOptions>
<Optimization>Disabled</Optimization>
<AdditionalIncludeDirectories>..\..\..\include;..\..\..\protolib\include;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
<PreprocessorDefinitions>_DEBUG;PROTO_DEBUG;HAVE_IPV6;HAVE_ASSERT;WIN32;_CRT_SECURE_NO_WARNINGS;_WINDOWS;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<PreprocessorDefinitions>_CONSOLE,_DEBUG;PROTO_DEBUG;HAVE_IPV6;HAVE_ASSERT;WIN32;_CRT_SECURE_NO_WARNINGS;_WINDOWS;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<BasicRuntimeChecks>EnableFastChecks</BasicRuntimeChecks>
<RuntimeLibrary>MultiThreadedDebugDLL</RuntimeLibrary>
<PrecompiledHeader />
@ -107,7 +108,7 @@
<Optimization>MaxSpeed</Optimization>
<InlineFunctionExpansion>OnlyExplicitInline</InlineFunctionExpansion>
<AdditionalIncludeDirectories>..\..\..\include;..\..\..\protolib\include;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
<PreprocessorDefinitions>NDEBUG;PROTO_DEBUG;HAVE_IPV6;HAVE_ASSERT;WIN32;_WINDOWS;_CRT_SECURE_NO_WARNINGS;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<PreprocessorDefinitions>_CONSOLE,NDEBUG;PROTO_DEBUG;HAVE_IPV6;HAVE_ASSERT;WIN32;_WINDOWS;_CRT_SECURE_NO_WARNINGS;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<StringPooling>true</StringPooling>
<RuntimeLibrary>MultiThreadedDLL</RuntimeLibrary>
<FunctionLevelLinking>true</FunctionLevelLinking>
@ -139,7 +140,7 @@
<ItemGroup>
<ClCompile Include="..\..\..\src\common\normApp.cpp" />
<ClCompile Include="..\..\..\src\common\normPostProcess.cpp" />
<ClCompile Include="..\win32PostProcess.cpp" />
<ClCompile Include="..\..\..\src\win32\win32PostProcess.cpp" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\..\..\protolib\makefiles\win32\Protokit.vcxproj">

7
makefiles/win32/normTest.vcxproj
View File

@ -1,5 +1,5 @@
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Project DefaultTargets="Build" ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup Label="ProjectConfigurations">
<ProjectConfiguration Include="Debug|Win32">
<Configuration>Debug</Configuration>
@ -13,17 +13,18 @@
<PropertyGroup Label="Globals">
<ProjectGuid>{32CF83E0-D2E0-4E43-8F7B-72BD0AB64647}</ProjectGuid>
<RootNamespace>normTest</RootNamespace>
<WindowsTargetPlatformVersion>10.0.16299.0</WindowsTargetPlatformVersion>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<PlatformToolset>v120</PlatformToolset>
<PlatformToolset>v141</PlatformToolset>
<UseOfMfc>false</UseOfMfc>
<CharacterSet>MultiByte</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<PlatformToolset>v120</PlatformToolset>
<PlatformToolset>v141</PlatformToolset>
<UseOfMfc>false</UseOfMfc>
<CharacterSet>MultiByte</CharacterSet>
</PropertyGroup>

7
makefiles/win32/npc.vcxproj
View File

@ -1,5 +1,5 @@
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Project DefaultTargets="Build" ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup Label="ProjectConfigurations">
<ProjectConfiguration Include="Debug|Win32">
<Configuration>Debug</Configuration>
@ -13,17 +13,18 @@
<PropertyGroup Label="Globals">
<ProjectGuid>{7BF4525B-9185-4296-A2BF-FFE77BB66EAF}</ProjectGuid>
<RootNamespace>normTest</RootNamespace>
<WindowsTargetPlatformVersion>10.0.16299.0</WindowsTargetPlatformVersion>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<PlatformToolset>v120</PlatformToolset>
<PlatformToolset>v141</PlatformToolset>
<UseOfMfc>false</UseOfMfc>
<CharacterSet>MultiByte</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<PlatformToolset>v120</PlatformToolset>
<PlatformToolset>v141</PlatformToolset>
<UseOfMfc>false</UseOfMfc>
<CharacterSet>MultiByte</CharacterSet>
</PropertyGroup>

256
src/common/normApi.cpp
View File

@ -68,6 +68,13 @@ class NormInstance : public NormController
bool GetNextEvent(NormEvent* theEvent);
bool SetCacheDirectory(const char* cachePath);
void SetAllocationFunctions(NormAllocFunctionHandle allocFunc,
NormFreeFunctionHandle freeFunc)
{
data_alloc_func = allocFunc;
session_mgr.SetDataFreeFunction(freeFunc);
}
void ReleasePreviousEvent();
bool NotifyQueueIsEmpty() const
@ -158,14 +165,15 @@ class NormInstance : public NormController
ProtoDispatcher dispatcher;
bool priority_boost;
NormSessionMgr session_mgr;
NormSessionMgr session_mgr;
NormAllocFunctionHandle data_alloc_func;
private:
void ResetNotificationEvent()
{
#ifdef WIN32
if (0 == ResetEvent(notify_event))
PLOG(PL_ERROR, "NormInstance::GetNextEvent() ResetEvent error: %s\n", GetErrorString());
PLOG(PL_ERROR, "NormInstance::ResetNotificationEvent() ResetEvent error: %s\n", GetErrorString());
#else
char byte[32];
while (read(notify_fd[0], byte, 32) > 0); // TBD - error check
@ -190,7 +198,7 @@ class NormInstance : public NormController
NormInstance::Notification::Queue::Queue()
: head(NULL), tail(NULL)
{
}
@ -212,13 +220,14 @@ NormInstance::NormInstance()
session_mgr(static_cast<ProtoTimerMgr&>(dispatcher),
static_cast<ProtoSocket::Notifier&>(dispatcher),
static_cast<ProtoChannel::Notifier*>(&dispatcher)),
previous_notification(NULL), rx_cache_path(NULL)
data_alloc_func(NULL), previous_notification(NULL), rx_cache_path(NULL)
{
#ifdef WIN32
notify_event = NULL;
#else
notify_fd[0] = notify_fd[1] = -1;
#endif // if/else WIN32/UNIX
dispatcher.SetUserData(&session_mgr); // for debugging
session_mgr.SetController(static_cast<NormController*>(this));
}
@ -374,7 +383,7 @@ void NormInstance::Notify(NormController::Event event,
{
NormDataObject* dataObj = static_cast<NormDataObject*>(object);
unsigned int dataLen = (unsigned int)(object->GetSize().GetOffset());
char* dataPtr = new char[dataLen];
char* dataPtr = (NULL != data_alloc_func) ? data_alloc_func(dataLen) : new char[dataLen];
if (NULL == dataPtr)
{
PLOG(PL_FATAL, "NormInstance::Notify(RX_OBJECT_NEW) new dataPtr error: %s\n",
@ -474,7 +483,7 @@ void NormInstance::PurgeObjectNotifications(NormObjectHandle objectHandle)
// TBD - check if event queue is emptied and reset event/fd
} // end NormInstance::PurgeObjectNotifications()
// Purge any notifications associated with a specific object
// Purge any notifications associated with a specific remote sender node
void NormInstance::PurgeNodeNotifications(NormNodeHandle nodeHandle)
{
if (NORM_NODE_INVALID == nodeHandle) return;
@ -507,7 +516,7 @@ void NormInstance::PurgeNodeNotifications(NormNodeHandle nodeHandle)
// "Release" any previously-retained object or node handle
if (NORM_OBJECT_INVALID != previous_notification->event.object)
((NormObject*)(previous_notification->event.object))->Release();
else if (NORM_NODE_INVALID != previous_notification->event.sender)
else
((NormNode*)(previous_notification->event.sender))->Release();
notify_pool.Append(previous_notification);
previous_notification = NULL;
@ -649,8 +658,6 @@ bool NormInstance::GetNextEvent(NormEvent* theEvent)
return (NULL != n);
} // end NormInstance::GetNextEvent()
bool NormInstance::WaitForEvent()
{
if (!dispatcher.IsThreaded())
@ -924,6 +931,16 @@ NormDescriptor NormGetDescriptor(NormInstanceHandle instanceHandle)
} // end NormGetDescriptor()
NORM_API_LINKAGE
void NormSetAllocationFunctions(NormInstanceHandle instanceHandle,
NormAllocFunctionHandle allocFunc,
NormFreeFunctionHandle freeFunc)
{
NormInstance* instance = (NormInstance*)instanceHandle;
instance->SetAllocationFunctions(allocFunc, freeFunc);
} // end NormSetAllocationFunctions()
// if "waitForEvent" is false, this is a non-blocking call
// (TBD) add a timeout option to this?
@ -986,7 +1003,7 @@ NormSessionHandle NormCreateSession(NormInstanceHandle instanceHandle,
NormSession* session =
instance->session_mgr.NewSession(sessionAddr, sessionPort, localNodeId);
instance->dispatcher.ResumeThread();
if (session)
if (NULL != session)
return ((NormSessionHandle)session);
}
return NORM_SESSION_INVALID;
@ -1067,7 +1084,7 @@ NORM_API_LINKAGE
void NormCancelUserTimer(NormSessionHandle sessionHandle)
{
NormInstance* instance = NormInstance::GetInstanceFromSession(sessionHandle);
if (instance)
if (NULL != instance)
{
if (instance->dispatcher.SuspendThread())
{
@ -1090,6 +1107,38 @@ NormNodeId NormGetLocalNodeId(NormSessionHandle sessionHandle)
return NORM_NODE_NONE;
} // end NormGetLocalNodeId()
NORM_API_LINKAGE
bool NormGetAddress(NormSessionHandle sessionHandle,
char* addrBuffer,
unsigned int* bufferLen,
UINT16* port)
{
bool result = false;
NormInstance* instance = NormInstance::GetInstanceFromSession(sessionHandle);
if (NULL != instance)
{
if (instance->dispatcher.SuspendThread())
{
NormSession* session = (NormSession*)sessionHandle;
const ProtoAddress& sessionAddr = session->Address();
unsigned int addrLen = sessionAddr.GetLength();
if (addrBuffer && bufferLen && (addrLen <= *bufferLen))
{
memcpy(addrBuffer, sessionAddr.GetRawHostAddress(), addrLen);
result = true;
}
else if (NULL == addrBuffer)
{
result = true; // just a query for addrLen and/or port
}
if (bufferLen) *bufferLen = addrLen;
if (port) *port = sessionAddr.GetPort();
instance->dispatcher.ResumeThread();
}
}
return result;
} // end NormGetAddress()
NORM_API_LINKAGE
UINT16 NormGetRxPort(NormSessionHandle sessionHandle)
{
@ -1146,22 +1195,44 @@ void NormSetTxOnly(NormSessionHandle sessionHandle,
}
} // end NormSetTxOnly()
// These functions are used internally by the NormSocket API extensiont
NORM_API_LINKAGE
bool NormChangeDestination(NormSessionHandle sessionHandle,
const char* sessionAddress,
UINT16 sessionPort,
bool connectToSessionAddress)
void NormSetId(NormSessionHandle sessionHandle, NormNodeId normId)
{
// First, see if we can make a valid ProtoAddress
ProtoAddress dest;
if (!dest.ResolveFromString(sessionAddress))
return false;
dest.SetPort(sessionPort);
NormInstance* instance = NormInstance::GetInstanceFromSession(sessionHandle);
if (instance && instance->dispatcher.SuspendThread())
{
NormSession* session = (NormSession*)sessionHandle;
if (session)
if (NULL != session) session->SetNodeId(normId);
instance->dispatcher.ResumeThread();
}
} // end NormSetId()
NORM_API_LINKAGE
bool NormChangeDestination(NormSessionHandle sessionHandle,
const char* sessionAddress,
UINT16 sessionPort,
bool connectToSessionAddress)
{
NormSession* session = (NormSession*)sessionHandle;
if (NULL == session) return false;
// First, see if we can make a valid ProtoAddress
ProtoAddress dest;
if (NULL != sessionAddress)
{
if (!dest.ResolveFromString(sessionAddress))
return false;
}
else
{
dest = session->Address();
}
dest.SetPort(sessionPort);
NormInstance* instance = NormInstance::GetInstanceFromSession(sessionHandle);
if (instance && instance->dispatcher.SuspendThread())
{
if (NULL != session)
{
session->SetAddress(dest);
if (connectToSessionAddress)
@ -1172,6 +1243,46 @@ bool NormChangeDestination(NormSessionHandle sessionHandle,
return true;
} // end NormChangeDestination()
NORM_API_LINKAGE
void NormSetServerListener(NormSessionHandle sessionHandle, bool state)
{
NormInstance* instance = NormInstance::GetInstanceFromSession(sessionHandle);
if (NULL != instance)
{
if (instance->dispatcher.SuspendThread())
{
NormSession* session = (NormSession*)sessionHandle;
if (session) session->SetServerListener(state);
instance->dispatcher.ResumeThread();
}
}
} // end NormSetServerListener()
NORM_API_LINKAGE
bool NormTransferSender(NormSessionHandle sessionHandle, NormNodeHandle senderHandle)
{
bool result = false;
NormInstance* dstInstance = NormInstance::GetInstanceFromSession(sessionHandle);
if (NULL != dstInstance)
{
if (dstInstance->dispatcher.SuspendThread())
{
NormInstance* srcInstance = NormInstance::GetInstanceFromNode(senderHandle);
if (srcInstance->dispatcher.SuspendThread())
{
NormSession* session = (NormSession*)sessionHandle;
NormSenderNode* sender = (NormSenderNode*)senderHandle;
if ((NULL != session) && (NULL != sender))
result = session->InsertRemoteSender(*sender);
srcInstance->dispatcher.ResumeThread();
}
dstInstance->dispatcher.ResumeThread();
}
}
return result;
} // end NormTransferSender()
NORM_API_LINKAGE
void NormSetRxPortReuse(NormSessionHandle sessionHandle,
bool enableReuse,
@ -1447,7 +1558,8 @@ bool NormStartSender(NormSessionHandle sessionHandle,
UINT32 bufferSpace,
UINT16 segmentSize,
UINT16 numData,
UINT16 numParity)
UINT16 numParity,
UINT8 fecId)
{
bool result = false;
NormInstance* instance = NormInstance::GetInstanceFromSession(sessionHandle);
@ -1455,7 +1567,7 @@ bool NormStartSender(NormSessionHandle sessionHandle,
{
NormSession* session = (NormSession*)sessionHandle;
if (session)
result = session->StartSender(sessionId, bufferSpace, segmentSize, numData, numParity);
result = session->StartSender(sessionId, bufferSpace, segmentSize, numData, numParity, fecId);
else
result = false;
instance->dispatcher.ResumeThread();
@ -1596,18 +1708,16 @@ void NormSetGrttEstimate(NormSessionHandle sessionHandle,
NORM_API_LINKAGE
double NormGetGrttEstimate(NormSessionHandle sessionHandle)
{
NormInstance* instance = NormInstance::GetInstanceFromSession(sessionHandle);
if (instance && instance->dispatcher.SuspendThread())
if (NORM_SESSION_INVALID != sessionHandle)
{
NormSession* session = (NormSession*)sessionHandle;
if (session) session->SenderGrtt();
instance->dispatcher.ResumeThread();
session->ResetGrttNotification();
return (session->SenderGrtt());
}
if (NORM_SESSION_INVALID != sessionHandle)
return (((NormSession*)sessionHandle)->SenderGrtt());
else
{
return -1.0;
}
} // end NormGetGrttEstimate()
NORM_API_LINKAGE
@ -1728,7 +1838,7 @@ NormObjectHandle NormDataEnqueue(NormSessionHandle sessionHandle,
{
NormObject* obj =
static_cast<NormObject*>(session->QueueTxData(dataPtr, dataLen, infoPtr, infoLen));
if (obj) objectHandle = (NormObjectHandle)obj;
if (NULL != obj) objectHandle = (NormObjectHandle)obj;
}
instance->dispatcher.ResumeThread();
}
@ -1803,6 +1913,8 @@ NORM_API_LINKAGE
unsigned int NormGetStreamBufferSegmentCount(unsigned int bufferBytes, UINT16 segmentSize, UINT16 blockSize)
{
// This same computation is performed in NormStreamObject::Open() in "normObject.cpp"
// (Note the number of stream buffer segments is always smaller than the
// blockSize*pending_mask.GetSize())
unsigned int numBlocks = bufferBytes / (blockSize * segmentSize);
if (numBlocks < 2) numBlocks = 2; // NORM enforces a 2-block minimum buffer size
return (numBlocks * blockSize);
@ -1813,14 +1925,13 @@ unsigned int NormStreamWrite(NormObjectHandle streamHandle,
const char* buffer,
unsigned int numBytes)
{
// TBD - the thread suspend/resume here may _not_ be sufficient.
// we may need to do something to make sure the NORM thread
// wakes up right away (i.e. SignalThread()
// Also so the same for flush, send command, and data/file enqueue calls!
// (i.e. in any case where immediate data send action may be invoked)
// Note: Since an underlying issue with ProtoDispatcher::SignalThread() had been resolved,
// using ProtoDispatcher::SuspendThread() should be sufficient since the underlying
// protolib time scheduling, etc. code actually invokes SignalThread() on an
// as-needed basis. Thus, using SuspendThread() (lighter weight) should suffice
unsigned int result = 0;
NormInstance* instance = NormInstance::GetInstanceFromObject(streamHandle);
if (instance && instance->dispatcher.SuspendThread())
if ((NULL != instance) && instance->dispatcher.SuspendThread())
{
NormStreamObject* stream =
static_cast<NormStreamObject*>((NormObject*)streamHandle);
@ -1836,7 +1947,7 @@ void NormStreamFlush(NormObjectHandle streamHandle,
NormFlushMode flushMode)
{
NormInstance* instance = NormInstance::GetInstanceFromObject(streamHandle);
if (instance && instance->dispatcher.SuspendThread())
if ((NULL != instance) && instance->dispatcher.SuspendThread())
{
NormStreamObject* stream =
static_cast<NormStreamObject*>((NormObject*)streamHandle);
@ -1908,7 +2019,7 @@ bool NormSetWatermark(NormSessionHandle sessionHandle,
NormObject* obj = (NormObject*)objectHandle;
if (session && obj)
{
// Purge any existing to NORM_TX_WATERMARK_COMPLETED notification to be safe
// Purge any existing NORM_TX_WATERMARK_COMPLETED notifications to be safe
instance->PurgeNotifications(sessionHandle, NORM_TX_WATERMARK_COMPLETED);
// (segmentId doesn't matter for non-stream)
if (obj->IsStream())
@ -1941,6 +2052,8 @@ bool NormResetWatermark(NormSessionHandle sessionHandle)
NormInstance* instance = NormInstance::GetInstanceFromSession(sessionHandle);
if (instance && instance->dispatcher.SuspendThread())
{
// Purge any existing NORM_TX_WATERMARK_COMPLETED notifications to be safe
instance->PurgeNotifications(sessionHandle, NORM_TX_WATERMARK_COMPLETED);
NormSession* session = (NormSession*)sessionHandle;
session->SenderResetWatermark();
instance->dispatcher.ResumeThread();
@ -1997,12 +2110,17 @@ NORM_API_LINKAGE
NormNodeHandle NormGetAckingNodeHandle(NormSessionHandle sessionHandle,
NormNodeId nodeId)
{
if (NORM_SESSION_INVALID != sessionHandle)
if (NORM_SESSION_INVALID != sessionHandle)
{
NormSession* session = (NormSession*)sessionHandle;
NormAckingNode* acker = session->SenderFindAckingNode(nodeId);
if (NULL != acker)
return ((NormNodeHandle)static_cast<NormNode*>(acker));
NormInstance* instance = NormInstance::GetInstanceFromSession(sessionHandle);
if (instance && instance->dispatcher.SuspendThread())
{
NormSession* session = (NormSession*)sessionHandle;
NormAckingNode* acker = session->SenderFindAckingNode(nodeId);
instance->dispatcher.ResumeThread();
if (NULL != acker)
return ((NormNodeHandle)static_cast<NormNode*>(acker));
}
}
return NORM_NODE_INVALID;
} // end NormGetAckingNodeHandle()
@ -2104,6 +2222,19 @@ void NormCancelCommand(NormSessionHandle sessionHandle)
}
} // end NormCancelCommand()
NORM_API_LINKAGE
void NormSetSynStatus(NormSessionHandle sessionHandle, bool state)
{
NormInstance* instance = NormInstance::GetInstanceFromSession(sessionHandle);
if (instance && instance->dispatcher.SuspendThread())
{
NormSession* session = (NormSession*)sessionHandle;
session->SenderSetSynStatus(state);
instance->dispatcher.ResumeThread();
}
} // end NormSetSynStatus()
/** NORM Receiver Functions */
NORM_API_LINKAGE
@ -2138,8 +2269,6 @@ NORM_API_LINKAGE
void NormSetRxCacheLimit(NormSessionHandle sessionHandle,
unsigned short countMax)
{
// We limit this to 1/4 of the 16-bit NormObjectId space
if (countMax > 16384) countMax = 16384;
NormInstance* instance = NormInstance::GetInstanceFromSession(sessionHandle);
if (instance && instance->dispatcher.SuspendThread())
{
@ -2290,6 +2419,25 @@ void NormNodeSetRxRobustFactor(NormNodeHandle nodeHandle,
}
} // end NormNodeSetRxRobustFactor()
NORM_API_LINKAGE
bool NormPreallocateRemoteSender(NormSessionHandle sessionHandle,
unsigned long bufferSize,
UINT16 segmentSize,
UINT16 numData,
UINT16 numParity,
unsigned int streamBufferSize)
{
bool result = false;
NormInstance* instance = NormInstance::GetInstanceFromSession(sessionHandle);
if (instance && instance->dispatcher.SuspendThread())
{
NormSession* session = (NormSession*)sessionHandle;
result = session->PreallocateRemoteSender(bufferSize, segmentSize, numData, numParity, streamBufferSize);
instance->dispatcher.ResumeThread();
}
return result;
} // end NormPreallocateRemoteSender()
NORM_API_LINKAGE
bool NormStreamRead(NormObjectHandle streamHandle,
char* buffer,
@ -2545,6 +2693,19 @@ char* NormDataDetachData(NormObjectHandle dataHandle)
return ptr;
} // end NormDataDetachData()
NORM_API_LINKAGE
char* NormAlloc(size_t numBytes)
{
return new char[numBytes];
} // end NormAlloc()
NORM_API_LINKAGE
void NormFree(char* dataPtr)
{
if (NULL != dataPtr)
delete[] dataPtr;
} // end NormFree()
NORM_API_LINKAGE
NormNodeHandle NormObjectGetSender(NormObjectHandle objectHandle)
{
@ -2576,7 +2737,7 @@ bool NormNodeGetAddress(NormNodeHandle nodeHandle,
//NormInstance* instance = NormInstance::GetInstanceFromNode(nodeHandle);
//if (instance && instance->dispatcher.SuspendThread())
{
NormNode* node = (NormSenderNode*)nodeHandle;
NormNode* node = (NormNode*)nodeHandle;
const ProtoAddress& nodeAddr = node->GetAddress();
unsigned int addrLen = nodeAddr.GetLength();
if (addrBuffer && bufferLen && (addrLen <= *bufferLen))
@ -2623,6 +2784,7 @@ double NormNodeGetGrtt(NormNodeHandle nodeHandle)
if ((NULL != node) && (NormNode::SENDER == node->GetType()))
{
NormSenderNode* sender = static_cast<NormSenderNode*>(node);
sender->ResetGrttNotification();
return sender->GetGrttEstimate();
}
else

189
src/common/normApp.cpp
View File

@ -1,5 +1,10 @@
// normApp.cpp - Command-line NORM application
// normApp.cpp - Command-line NORM "demo" application
// Note this code does _not_ use the NORM API. It is kept as a demonstrator
// and test tool and serves as a performance comparison reference for testing
// the threaded NORM API code. So this should _not_ be used as a NORM
// programming example!!!!
#include "protokit.h"
#include "normSession.h"
@ -51,7 +56,6 @@ class NormApp : public NormController, public ProtoApp
{object_id = objectId;}
NormObjectId GetObjectId() const
{return object_id;}
private:
// ProtoTree::Item overrides
@ -62,9 +66,7 @@ class NormApp : public NormController, public ProtoApp
char file_path[PATH_MAX];
NormObjectId object_id;
};
}; // end class NormApp::FileCacheItem
void ShowHelp();
void OnInputReady();
@ -145,14 +147,17 @@ class NormApp : public NormController, public ProtoApp
UINT16 extra_parity;
double backoff_factor;
double grtt_estimate; // initial grtt estimate
NormSession::ProbingMode grtt_probing_mode;
double group_size;
unsigned long tx_buffer_size; // bytes
unsigned int tx_sock_buffer_size;
unsigned int tx_sock_buffer_size;
unsigned long tx_cache_min;
unsigned long tx_cache_max;
NormObjectSize tx_cache_size;
NormFileList tx_file_list;
char tx_file_name[PATH_MAX];
bool tx_file_info;
ProtoTree tx_file_cache;
bool tx_one_shot;
bool tx_ack_shot;
@ -188,6 +193,7 @@ class NormApp : public NormController, public ProtoApp
bool rx_persistent;
bool process_aborted_files;
bool preallocate_sender;
NormSenderNode::RepairBoundary repair_boundary;
// Debug parameters
bool tracing;
@ -222,17 +228,16 @@ NormApp::NormApp()
cc_enable(false), ecn_mode(ECN_OFF), tolerate_loss(false),
node_id(NORM_NODE_ANY), segment_size(1024), ndata(32), nparity(16), auto_parity(0), extra_parity(0),
backoff_factor(NormSession::DEFAULT_BACKOFF_FACTOR), grtt_estimate(NormSession::DEFAULT_GRTT_ESTIMATE),
group_size(NormSession::DEFAULT_GSIZE_ESTIMATE),
grtt_probing_mode(NormSession::PROBE_ACTIVE), group_size(NormSession::DEFAULT_GSIZE_ESTIMATE),
tx_buffer_size(1024*1024), tx_sock_buffer_size(0), tx_cache_min(8), tx_cache_max(256), tx_cache_size((UINT32)20*1024*1024),
tx_one_shot(false), tx_ack_shot(false), tx_file_queued(false),
tx_file_info(true), tx_one_shot(false), tx_ack_shot(false), tx_file_queued(false),
tx_robust_factor(NormSession::DEFAULT_ROBUST_FACTOR), tx_object_interval(0.0), tx_repeat_count(0),
tx_repeat_interval(2.0), tx_repeat_clear(true), tx_requeue(0), tx_requeue_count(0), acking_node_list(NULL),
acking_flushes(false), watermark_pending(false), rx_buffer_size(1024*1024), rx_sock_buffer_size(0),
rx_cache_path(NULL), post_processor(NULL), unicast_nacks(false), silent_receiver(false),
low_delay(false), realtime(false), rx_robust_factor(NormSession::DEFAULT_ROBUST_FACTOR), rx_persistent(true), process_aborted_files(false),
preallocate_sender(false), tracing(false), tx_loss(0.0), rx_loss(0.0)
preallocate_sender(false), repair_boundary(NormSenderNode::BLOCK_BOUNDARY), tracing(false), tx_loss(0.0), rx_loss(0.0)
{
control_pipe.SetListener(this, &NormApp::OnControlEvent);
control_pipe.SetNotifier(&GetSocketNotifier());
@ -243,6 +248,8 @@ NormApp::NormApp()
interval_timer.SetInterval(0.0);
interval_timer.SetRepeat(0);
tx_file_name[0] = '\0';
struct timeval currentTime;
ProtoSystemTime(currentTime);
srand(currentTime.tv_usec);
@ -307,10 +314,12 @@ const char* const NormApp::cmd_list[] =
"+minput", // send message stream input
"+moutput", // recv message stream output
"+sendfile", // file/directory list to transmit
"+info", // 'on' | 'off' to enable/disable file info (name) transmission (default = 'on')
"+interval", // delay time (sec) between files (0.0 sec default)
"+repeatcount", // How many times to repeat the file/directory list tx
"+rinterval", // Interval (sec) between file/directory list repeats
"+requeue", // <count> how many times files are retransmitted w/ same objId
"+boundary", // 'block' or 'file' to set NORM_REPAIR_BOUNDARY (default is 'block')
"-oneshot", // Transmit file(s), exiting upon TX_FLUSH_COMPLETED
"-ackshot", // Transmit file(s), exiting upon TX_WATERMARK_COMPLETED
"-updatesOnly", // only send updated files on repeat transmission
@ -325,6 +334,7 @@ const char* const NormApp::cmd_list[] =
"+extra", // Number of extra FEC packets sent in response to repair requests
"+backoff", // Backoff factor to use
"+grtt", // Set sender's initial GRTT estimate
"+probe", // {'active', 'passive' | 'none'} Set sender;s GRTT probing mode 'active' is default)
"+gsize", // Set sender's group size estimate
"+txbuffer", // Size of sender's buffer
"+txsockbuffer", // tx socket buffer size
@ -377,10 +387,12 @@ void NormApp::ShowHelp()
" +minput, // sender message stream input\n"
" +moutput, // receiver message stream output\n"
" +sendfile, // file/directory list to transmit\n"
" +info, // 'on' | 'off' to enable/disable file info (name) transmission (default = 'on')\n"
" +interval, // delay time (sec) between files (0.0 sec default)\n"
" +repeatcount, // How many times to repeat the file/directory list tx\n"
" +rinterval, // Interval (sec) between file/directory list repeats\n"
" +requeue, // <count> how many times files are retransmitted w/ same objId\n"
" +boundary // 'block' or 'file' to set NORM_REPAIR_BOUNDARY (default is 'block')\n"
" -oneshot, // Exit upon sender TX_FLUSH_COMPLETED event (sender exits after transmission)\n"
" -ackshot, // Exit upon sender TX_WATERMARK_COMPLETED event (sender exits after transmission)\n"
" -updatesOnly, // only send updated files on repeat transmission\n"
@ -395,6 +407,7 @@ void NormApp::ShowHelp()
" +extra, // Number of extra FEC packets sent in response to repair requests\n"
" +backoff, // Backoff factor to use\n"
" +grtt, // Set sender's initial GRTT estimate\n"
" +probe, // {'active', 'passive' | 'none'} Set sender;s GRTT probing mode 'active' is default)\n"
" +gsize, // Set sender's group size estimate\n"
" +txbuffer, // Size of sender's buffer\n"
" +txcachebounds,// <countMin:countMax:sizeMax> limits on sender tx object caching\n"
@ -428,7 +441,7 @@ void NormApp::OnControlEvent(ProtoSocket& /*theSocket*/, ProtoSocket::Event theE
if (control_pipe.Recv(buffer, len))
{
buffer[len] = '\0';
PLOG(PL_ERROR, "norm: received command \"%s\"\n", buffer);
PLOG(PL_DEBUG, "norm: received command \"%s\"\n", buffer);
char* cmd = buffer;
char* val = NULL;
for (unsigned int i = 0; i < len; i++)
@ -631,6 +644,8 @@ bool NormApp::OnCommand(const char* cmd, const char* val)
}
bool result = session ? session->SetLoopback(loopTemp) : true;
loopback = result ? loopTemp : loopback;
if (loopback && (NULL != session) && session->Address().IsMulticast())
session->SetRxPortReuse(true);
if (!result)
{
PLOG(PL_FATAL, "NormApp::OnCommand(loopback) error setting socket loopback!\n");
@ -828,17 +843,25 @@ bool NormApp::OnCommand(const char* cmd, const char* val)
// (This will allow the "send" command to be invoked at run-time via the ProtoPipe
// remote control interface).
}
else if (!strncmp("interval", cmd, len))
else if (!strncmp("info", cmd, len))
{
if (1 != sscanf(val, "%lf", &tx_object_interval))
tx_object_interval = -1.0;
if (tx_object_interval < 0.0)
if (!strcmp("on", val))
tx_file_info = true;
else if (!strcmp("off", val))
tx_file_info = false;
else
{
PLOG(PL_FATAL, "NormApp::OnCommand(interval) Invalid tx object interval: %s\n",
val);
tx_object_interval = 0.0;
PLOG(PL_FATAL, "NormApp::OnCommand(info) invalid argument!\n");
return false;
}
}
else if (!strncmp("interval", cmd, len))
{
if (1 != sscanf(val, "%lf", &tx_object_interval))
{
PLOG(PL_FATAL, "NormApp::OnCommand(interval) Invalid tx object interval: %s\n", val);
return false;
}
}
else if (!strncmp("repeatcount", cmd, len))
{
@ -860,11 +883,27 @@ bool NormApp::OnCommand(const char* cmd, const char* val)
{
tx_requeue = tx_requeue_count = atoi(val);
}
else if (!strncmp("boundary", cmd, len))
{
if (0 == strcmp("block", val))
{
repair_boundary = NormSenderNode::BLOCK_BOUNDARY;
}
else if (0 == strcmp("file", val))
{
repair_boundary = NormSenderNode::OBJECT_BOUNDARY;
}
else
{
PLOG(PL_FATAL, "NormApp::OnCommand(boundary) error: invalid repair boundary \"%s\"\n", val);
return false;
}
if (session) session->ReceiverSetDefaultRepairBoundary(repair_boundary);
}
else if (!strncmp("oneshot", cmd, len))
{
tx_one_shot = true;
}
}
else if (!strncmp("ackshot", cmd, len))
{
tx_ack_shot = true;
@ -998,6 +1037,22 @@ bool NormApp::OnCommand(const char* cmd, const char* val)
grtt_estimate = grttEstimate;
if (session) session->SenderSetGrtt(grttEstimate);
}
else if (!strncmp("probe", cmd, len))
{
size_t valLen = strlen(val);
if (!strncmp("none", val, valLen))
grtt_probing_mode = NormSession::PROBE_NONE;
else if (!strncmp("passive", val, valLen))
grtt_probing_mode = NormSession::PROBE_PASSIVE;
else if (!strncmp("active", val, valLen))
grtt_probing_mode = NormSession::PROBE_ACTIVE;
else
{
PLOG(PL_FATAL, "NormApp::OnCommand(probe) invalid grtt probing mode!\n");
return false;
}
if (NULL != session) session->SetGrttProbingMode(grtt_probing_mode);
}
else if (!strncmp("gsize", cmd, len))
{
double groupSize = atof(val);
@ -1028,7 +1083,11 @@ bool NormApp::OnCommand(const char* cmd, const char* val)
tx_cache_min = countMin;
tx_cache_max = countMax;
tx_cache_size = sizeMax;
if (session) session->SetTxCacheBounds(tx_cache_size, tx_cache_min, tx_cache_max);
if (session)
{
session->SetTxCacheBounds(tx_cache_size, tx_cache_min, tx_cache_max);
session->SetRxCacheMax(tx_cache_max);
}
}
else if (!strncmp("txrobustfactor", cmd, len))
{
@ -1393,7 +1452,7 @@ void NormApp::OnInputReady()
{
if (feof(input))
{
PLOG(PL_FATAL, "norm: input end-of-file.\n");
PLOG(PL_ALWAYS, "norm: input end-of-file.\n");
if (input_active)
{
#ifdef WIN32
@ -1461,9 +1520,9 @@ void NormApp::OnInputReady()
if (input_active)
{
#ifdef WIN32
ASSERT(0); // no Win32 support for stream i/o yet
ASSERT(0); // no Win32 support for stream i/o yet
#else
dispatcher.RemoveGenericInput(fileno(input));
dispatcher.RemoveGenericInput(fileno(input));
#endif // if/else WIN32/UNIX
input_active = false;
}
@ -1516,7 +1575,7 @@ void NormApp::Notify(NormController::Event event,
if ((msg_test || input) && (object == tx_stream))
OnInputReady();
}
else
else if (tx_object_interval >= 0.0)
{
// Can queue a new object for transmission
if (interval_timer.IsActive()) interval_timer.Deactivate();
@ -1535,9 +1594,18 @@ void NormApp::Notify(NormController::Event event,
PLOG(PL_DEBUG, "NormApp::Notify(TX_FLUSH_COMPLETED) ...\n");
if (tx_one_shot)
{
PLOG(PL_FATAL, "norm: transmit flushing completed, exiting.\n");
PLOG(PL_ALWAYS, "norm: transmit flushing completed, exiting.\n");
Stop();
}
else if (tx_object_interval < 0.0)
{
// Can queue a new object for transmission
if (interval_timer.IsActive()) interval_timer.Deactivate();
if (interval_timer.GetInterval() > 0.0)
ActivateTimer(interval_timer);
else
OnIntervalTimeout(interval_timer);
}
break;
case TX_WATERMARK_COMPLETED:
@ -1547,7 +1615,7 @@ void NormApp::Notify(NormController::Event event,
watermark_pending = false; // enable new watermark to be set
if (tx_ack_shot)
{
PLOG(PL_ERROR, "norm: transmit flushing completed, exiting.\n");
PLOG(PL_ALWAYS, "norm: transmit flushing completed, exiting.\n");
Stop();
}
break;
@ -1954,12 +2022,20 @@ void NormApp::Notify(NormController::Event event,
} // end NormApp::Notify()
bool NormApp::OnIntervalTimeout(ProtoTimer& /*theTimer*/)
bool NormApp::OnIntervalTimeout(ProtoTimer& theTimer)
{
char fileName[PATH_MAX];
if (tx_file_list.GetNextFile(fileName))
if (('\0' != tx_file_name[0]) || tx_file_list.GetNextFile(fileName))
{
tx_repeat_clear = true;
// The "tx_file_name" is used to cache the path of the current file for enqueue
// re-attempt upon possible flow control NormSession::QueueTxFile() failure
if ('\0' == tx_file_name[0])
strcpy(tx_file_name, fileName);
else
strcpy(fileName, tx_file_name); // used cached tx_file_name for enqueue re-attempt
// 1) Build up the full path name of the file
char pathName[PATH_MAX];
tx_file_list.GetCurrentBasePath(pathName);
@ -1978,9 +2054,6 @@ bool NormApp::OnIntervalTimeout(ProtoTimer& /*theTimer*/)
if (PROTO_PATH_DELIMITER == fileNameInfo[i])
fileNameInfo[i] = '/';
}
char temp[PATH_MAX];
strncpy(temp, fileNameInfo, len);
temp[len] = '\0';
FileCacheItem* fileCacheItem = NULL;
NormFileObject* obj = NULL;
@ -1994,22 +2067,44 @@ bool NormApp::OnIntervalTimeout(ProtoTimer& /*theTimer*/)
bool requeueSuccess = false;
if (NULL != obj) requeueSuccess = session->RequeueTxObject(obj);
if (!requeueSuccess)
{
PLOG(PL_ERROR, "norm warning: requeue attempt exceeded configured tx cache bounds!\n");
// TBD - remove file from tx_file_cache???
}
// RequeueTxObject() is not subject to flow control since the object is already buffered
tx_file_name[0] = '\0'; // reset so next file will be fetched
}
}
if (NULL == obj)
{
// It's a new object (or requeue failure)
if (NULL == (obj = session->QueueTxFile(fileName, fileNameInfo, (UINT16)len)))
// This makes sure it's not an empty file
// (TBD - provide for empty files to be sent)
if (0 == NormFile::GetSize(fileName))
{
PLOG(PL_ERROR, "NormApp::OnIntervalTimeout() Error queuing tx file: %s\n",
fileName);
// Wait a second, then try the next file in the list
PLOG(PL_WARN, "norm warning: empty file \"%s\" will not be transmitted\n");
tx_file_name[0] = '\0'; // reset so next file will be fetched
return OnIntervalTimeout(theTimer);
}
// It's a new object
const char* infoPtr = tx_file_info ? fileNameInfo : NULL;
UINT16 infoLen = tx_file_info ? (UINT16)len : 0;
if (NULL == (obj = session->QueueTxFile(fileName, infoPtr, infoLen)))
{
PLOG(PL_DEBUG, "NormApp::OnIntervalTimeout() error queuing tx file: %s\n", fileName);
// The code currently assumes that QueueTxFile() failed because of
// flow control and so we simply return and wait for a TX_QUEUE_EMPTY notification
// to trigger a re-attempt to enqueue the file name in "tx_file_name"
if (interval_timer.IsActive()) interval_timer.Deactivate();
interval_timer.SetInterval(1.0);
ActivateTimer(interval_timer);
// Old behavior was to wait a second and go to next file
// (commented out here)
//interval_timer.SetInterval(1.0);
//ActivateTimer(interval_timer);
return false;
}
else
{
tx_file_name[0] = '\0'; // reset so next file will be fetched
}
if (0 != tx_requeue)
{
if (NULL == fileCacheItem)
@ -2094,7 +2189,7 @@ bool NormApp::OnIntervalTimeout(ProtoTimer& /*theTimer*/)
else
{
tx_repeat_clear = true;
PLOG(PL_FATAL, "norm: End of tx file list reached.\n");
PLOG(PL_ALWAYS, "norm: End of tx file list reached.\n");
if (tx_ack_shot && !watermark_pending && tx_file_queued)
{
session->SenderSetWatermark(tx_last_object_id,
@ -2169,18 +2264,26 @@ bool NormApp::OnStartup(int argc, const char*const* argv)
session->SetRxLoss(rx_loss);
session->SetTTL(ttl);
session->SetLoopback(loopback);
if (loopback && session->Address().IsMulticast())
session->SetRxPortReuse(true);
if (NULL != interface_name)
session->SetMulticastInterface(interface_name);
session->SetEcnSupport(ECN_OFF != ecn_mode, ECN_ONLY == ecn_mode, tolerate_loss);
// We turn off flow control when no NACKing is happening
// (TBD - make flow control a command-line option)
if (silent_receiver)
session->SetFlowControl(0.0);
if (msg_test || input || !tx_file_list.IsEmpty())
{
NormObjectId baseId = (unsigned short)(rand() * (65535.0/ (double)RAND_MAX));
session->SenderSetBaseObjectId(baseId);
session->SetCongestionControl(cc_enable);
session->SetBackoffFactor(backoff_factor);
//session->SenderSetGrtt(grtt_estimate);
session->SenderSetGrtt(grtt_estimate);
session->SetGrttProbingMode(grtt_probing_mode);
session->SenderSetGroupSize(group_size);
session->SetTxRobustFactor(tx_robust_factor);
session->SetTxCacheBounds(tx_cache_size, tx_cache_min, tx_cache_max);
@ -2226,11 +2329,12 @@ bool NormApp::OnStartup(int argc, const char*const* argv)
if (output || rx_cache_path)
{
// StartReceiver(bufferMax (per-sender))
session->SetRxCacheMax(tx_cache_max);
session->ReceiverSetUnicastNacks(unicast_nacks);
session->ReceiverSetSilent(silent_receiver);
if (preallocate_sender)
{
if (!session->PreallocateRemoteSender(segment_size, ndata, nparity))
if (!session->PreallocateRemoteSender(rx_buffer_size, segment_size, ndata, nparity))
{
PLOG(PL_FATAL, "NormApp::OnStartup() remote sender preallocation error!\n");
session_mgr.Destroy();
@ -2244,6 +2348,7 @@ bool NormApp::OnStartup(int argc, const char*const* argv)
if (realtime)
session->RcvrSetRealtime(true);
session->SetRxRobustFactor(rx_robust_factor);
session->ReceiverSetDefaultRepairBoundary(repair_boundary);
if (!session->StartReceiver(rx_buffer_size))
{
PLOG(PL_FATAL, "NormApp::OnStartup() start receiver error!\n");

3
src/common/normFile.cpp
View File

@ -188,8 +188,7 @@ void NormFile::Unlock()
flock(fd, LOCK_UN);
#else
#ifdef HAVE_LOCKF
int ret = lockf(fd, F_ULOCK, 0);
ret = 0; // done to avoid compiler warning (should optimize out)
lockf(fd, F_ULOCK, 0);
#endif // HAVE_LOCKF
#endif // if/elseHAVE_FLOCK
fchmod(fd, 0640);

46
src/common/normMessage.cpp
View File

@ -7,7 +7,6 @@ NormHeaderExtension::NormHeaderExtension()
}
NormMsg::NormMsg()
: length(8), header_length(8), header_length_base(8)
{
@ -92,6 +91,13 @@ bool NormMsg::InitFromBuffer(UINT16 msgLength)
}
} // end NormMsg::InitFromBuffer()
void NormMsg::Display() const
{
const unsigned char* ptr = (const unsigned char*)buffer;
for (unsigned int i = 0; i < length; i++)
PLOG(PL_ALWAYS, "%02x", *ptr++);
} // end NormMsg::Display()
bool NormCmdCCMsg::GetCCNode(NormNodeId nodeId,
UINT8& flags,
UINT8& rtt,
@ -150,10 +156,10 @@ bool NormRepairRequest::AppendRepairItem(UINT8 fecId,
{
if (RANGES == form)
PLOG(PL_TRACE, "NormRepairRequest::AppendRepairItem-Range(fecId>%d obj>%hu blk>%lu seg>%hu) ...\n",
fecId, (UINT16)objectId, (UINT32)blockId, (UINT16)symbolId);
fecId, (UINT16)objectId, (unsigned long)blockId.GetValue(), (UINT16)symbolId);
else
PLOG(PL_TRACE, "NormRepairRequest::AppendRepairItem(fecId>%d obj>%hu blk>%lu seg>%hu) ...\n",
fecId, (UINT16)objectId, (UINT32)blockId, (UINT16)symbolId);
fecId, (UINT16)objectId, (unsigned long)blockId.GetValue(), (UINT16)symbolId);
ASSERT(NormPayloadId::IsValid(fecId));
UINT16 itemLength = RepairItemLength(fecId);
@ -164,7 +170,7 @@ bool NormRepairRequest::AppendRepairItem(UINT8 fecId,
((UINT8*)ptr)[RESERVED_OFFSET] = 0; // 1 byte
((UINT16*)ptr)[OBJ_ID_OFFSET] = htons((UINT16)objectId); // 2 bytes
NormPayloadId payloadId(fecId, fecM, ptr + 1);
payloadId.SetFecPayloadId(blockId, symbolId, blockLen);
payloadId.SetFecPayloadId(blockId.GetValue(), symbolId, blockLen);
length += itemLength;
return true;
}
@ -186,8 +192,8 @@ bool NormRepairRequest::AppendRepairRange(UINT8 fecId,
UINT16 endSymbolId)
{
PLOG(PL_TRACE, "NormRepairRequest::AppendRepairRange(%hu:%lu:%hu->%hu:%lu:%hu) ...\n",
(UINT16)startObjectId, (UINT32)startBlockId, (UINT16)startSymbolId,
(UINT16)endObjectId, (UINT32)endBlockId, (UINT16)endSymbolId);
(UINT16)startObjectId, (unsigned long)startBlockId.GetValue(), (UINT16)startSymbolId,
(UINT16)endObjectId, (unsigned long)endBlockId.GetValue(), (UINT16)endSymbolId);
// Note a "range" is two repair items
UINT16 rangeLength = RepairRangeLength(fecId);
if (buffer_len >= (ITEM_LIST_OFFSET+length+rangeLength))
@ -198,14 +204,14 @@ bool NormRepairRequest::AppendRepairRange(UINT8 fecId,
((UINT8*)ptr)[RESERVED_OFFSET] = 0;
((UINT16*)ptr)[OBJ_ID_OFFSET] = htons((UINT16)startObjectId);
NormPayloadId startId(fecId, fecM, ptr + 1);
startId.SetFecPayloadId(startBlockId, startSymbolId, startBlockLen);
startId.SetFecPayloadId(startBlockId.GetValue(), startSymbolId, startBlockLen);
// range end
ptr += (rangeLength/8); // advance ptr to second part of repair range
((UINT8*)ptr)[FEC_ID_OFFSET] = fecId;
((UINT8*)ptr)[RESERVED_OFFSET] = 0;
((UINT16*)ptr)[OBJ_ID_OFFSET] = htons((UINT16)endObjectId);
NormPayloadId endId(fecId, fecM, ptr + 1);
endId.SetFecPayloadId(endBlockId, endSymbolId, endBlockLen);
endId.SetFecPayloadId(endBlockId.GetValue(), endSymbolId, endBlockLen);
length += rangeLength;
return true;
}
@ -230,7 +236,7 @@ bool NormRepairRequest::AppendErasureCount(UINT8 fecId,
((UINT8*)ptr)[RESERVED_OFFSET] = 0;
((UINT16*)ptr)[OBJ_ID_OFFSET] = htons((UINT16)objectId);
NormPayloadId erasureId(fecId, fecM, ptr + 1);
erasureId.SetFecPayloadId(blockId, erasureCount, blockLen);
erasureId.SetFecPayloadId(blockId.GetValue(), erasureCount, blockLen);
length += itemLength;
return true;
}
@ -319,10 +325,10 @@ NormRepairRequest::Iterator::Iterator(const NormRepairRequest& theRequest, UINT8
}
// For erasure requests, symbolId is loaded with erasureCount
bool NormRepairRequest::Iterator::NextRepairItem(NormObjectId* objectId,
NormBlockId* blockId,
UINT16* blockLen,
UINT16* symbolId)
UINT16 NormRepairRequest::Iterator::NextRepairItem(NormObjectId* objectId,
NormBlockId* blockId,
UINT16* blockLen,
UINT16* symbolId)
{
UINT8 fecId; // for check
UINT16 itemLength = request.RetrieveRepairItem(fec_m, offset, &fecId, objectId, blockId, blockLen, symbolId);
@ -334,12 +340,8 @@ bool NormRepairRequest::Iterator::NextRepairItem(NormObjectId* objectId,
return false;
}
offset += itemLength;
return true;
}
else
{
return false;
}
return itemLength;
} // end NormRepairRequest::Iterator::NextRepairItem()
@ -366,18 +368,18 @@ void NormRepairRequest::Log(UINT8 fecId, UINT8 fecM) const
while (iterator.NextRepairItem(&objId, &blkId, &blkLen, &segId))
{
if (FlagIsSet(SEGMENT))
PLOG(PL_ALWAYS, "RepairItem> %hu:%lu:%hu", (UINT16)objId, (UINT32)blkId, (UINT16)segId);
PLOG(PL_ALWAYS, "RepairItem> %hu:%lu:%hu", (UINT16)objId, (unsigned long)blkId.GetValue(), (UINT16)segId);
else if (FlagIsSet(BLOCK))
PLOG(PL_ALWAYS, "RepairItem> %hu:%lu", (UINT16)objId, (UINT32)blkId, (UINT16)segId);
PLOG(PL_ALWAYS, "RepairItem> %hu:%lu", (UINT16)objId, (unsigned long)blkId.GetValue(), (UINT16)segId);
else
PLOG(PL_ALWAYS, "RepairItem> %hu", (UINT16)objId);
if (form == RANGES)
{
iterator.NextRepairItem(&objId, &blkId, &blkLen, &segId);
if (FlagIsSet(SEGMENT))
PLOG(PL_ALWAYS, " -> %hu:%lu:%hu", (UINT16)objId, (UINT32)blkId, (UINT16)segId);
PLOG(PL_ALWAYS, " -> %hu:%lu:%hu", (UINT16)objId, (unsigned long)blkId.GetValue(), (UINT16)segId);
else if (FlagIsSet(BLOCK))
PLOG(PL_ALWAYS, " -> %hu:%lu", (UINT16)objId, (UINT32)blkId, (UINT16)segId);
PLOG(PL_ALWAYS, " -> %hu:%lu", (UINT16)objId, (unsigned long)blkId.GetValue(), (UINT16)segId);
else
PLOG(PL_ALWAYS, " -> %hu", (UINT16)objId);
}

772
src/common/normNode.cpp
View File

File diff suppressed because it is too large Load Diff

1602
src/common/normObject.cpp
View File

File diff suppressed because it is too large Load Diff

125
src/common/normPrecode.cpp
View File

@ -1,7 +1,16 @@
#include "protoApp.h"
#include "normFile.h"
// Commment this #define out to use new, faster RS8 codec instead
// (TBD - provide option use 16-bit Reed Solomon for large block sizes?)
//#define USE_MDP_FEC
#ifdef USE_MDP_FEC
#include "normEncoderMDP.h"
#else
#include "normEncoderRS8.h"
#endif // if/else USE_MDP_FEC
#include <sys/types.h> // for BYTE_ORDER macro
#include <stdlib.h> // for atoi()
@ -48,8 +57,8 @@ class NormPrecodeApp : public ProtoApp
{
case 8:
{
long* outPtr = (long*)&result;
long* inPtr = (long*)&offset;
UINT32* outPtr = (UINT32*)&result;
UINT32* inPtr = (UINT32*)&offset;
outPtr[0] = ntohl(inPtr[1]);
outPtr[1] = ntohl(inPtr[0]);
break;
@ -559,7 +568,11 @@ bool NormPrecodeApp::Encode()
InitInterleaver(numOutputSegments);
// 1) Init our FEC encoder
#ifdef USE_MDP_FEC
NormEncoderMDP encoder;
#else
NormEncoderRS8 encoder;
#endif // if/else USE_MDP_FEC
if (!encoder.Init(num_data, num_parity, dataSegmentSize)) // 4 CRC bytes are _not_ encoded
{
PLOG(PL_FATAL, "npc: error initializing FEC encoder\n");
@ -607,7 +620,7 @@ bool NormPrecodeApp::Encode()
parityVec[i] = pvec;
pvec += segment_size;
}
// 3) Build "meta_data" segment for the file
// (TBD) This could be built directly into iBuffer segment zero
char metaData[SEGMENT_MAX+4];
@ -638,7 +651,7 @@ bool NormPrecodeApp::Encode()
strncpy(metaData+8, ptr, segment_size - 12);
// 2) Read "in_file" segments, encode, and output to "out_file"
PLOG(PL_FATAL, "npc: encoding file ... (progress: 0%%)");
PLOG(PL_ALWAYS, "npc: encoding file ... (progress: 0%%)");
// State to track/display encoding progress
NormFile::Offset progressThreshold = numOutputSegments / 100;
double progressIncrement = 100.0;
@ -659,61 +672,57 @@ bool NormPrecodeApp::Encode()
{
NormFile::Offset interleaverOffset = ComputeInterleaverOffset(outputSegmentId, numOutputSegments);
char* segment = useBuffering ? (iBuffer + (interleaverOffset % interleaverBytes)) : iBuffer;
switch (parityReady)
if (parityReady)
{
case false:
// D) Output parity segment for this block
char* pvec = parityVec[num_parity - parityCount];
memcpy(segment, pvec, dataSegmentSize);
if (0 == --parityCount)
{
// Read and encode a segment
inputSegmentId++;
if (1 == inputSegmentId)
memset(parityVec[0], 0, num_parity * segment_size);
parityReady = false;
blockId++;
}
}
else
{
// Read and encode a segment
inputSegmentId++;
if (1 == inputSegmentId)
{
// A) Segment '0' is the meta-data segment
memcpy(segment, metaData, dataSegmentSize);
}
else
{
// B) Read in data portion of next "segment"
unsigned int bytesToRead;
if (inputSegmentId != numInputSegments)
{
// A) Segment '0' is the meta-data segment
memcpy(segment, metaData, dataSegmentSize);
bytesToRead = dataSegmentSize;
}
else
{
// B) Read in data portion of next "segment"
unsigned int bytesToRead;
if (inputSegmentId != numInputSegments)
{
bytesToRead = dataSegmentSize;
}
else
{
memset(segment, 0, dataSegmentSize);
bytesToRead = lastFecSegSize;
}
if (in_file.Read(segment, bytesToRead) != bytesToRead)
{
PLOG(PL_FATAL, "\nnpc: unexpected error reading input file: %s\n", GetErrorString());
return false;
}
memset(segment, 0, dataSegmentSize);
bytesToRead = lastFecSegSize;
}
// C) Encode and check for parity readiness
encoder.Encode(outputSegmentId, segment, parityVec);
unsigned int numData = (blockId != lastBlockId) ? fecBlockSize : lastBlockSize;
if (numData == ++parityCount)
if (in_file.Read(segment, bytesToRead) != bytesToRead)
{
parityCount = num_parity;
parityReady = true;
PLOG(PL_FATAL, "\nnpc: unexpected error reading input file: %s\n", GetErrorString());
return false;
}
break;
}
case true:
// C) Encode and check for parity readiness
//TRACE("outputSegmentId:%lu\n", outputSegmentId);
encoder.Encode(outputSegmentId % fecBlockSize, segment, parityVec);
unsigned int numData = (blockId != lastBlockId) ? fecBlockSize : lastBlockSize;
if (numData == ++parityCount)
{
// D) Output parity segment for this block
char* pvec = parityVec[num_parity - parityCount];
memcpy(segment, pvec, dataSegmentSize);
if (0 == --parityCount)
{
memset(parityVec[0], 0, num_parity * segment_size);
parityReady = false;
blockId++;
}
break;
parityCount = num_parity;
parityReady = true;
}
}
}
// E) Calculate and add CRC32 checksum to each "segment"
UINT32 checksum = ComputeCRC32(segment, dataSegmentSize);
checksum = htonl(checksum);
@ -760,17 +769,17 @@ bool NormPrecodeApp::Encode()
if (++progressCounter >= progressThreshold)
{
if (progressPercent < 9)
PLOG(PL_FATAL, "\b\b\b%d%%)", progressPercent + 1);
PLOG(PL_ALWAYS, "\b\b\b%d%%)", progressPercent + 1);
else if (progressPercent < 99)
PLOG(PL_FATAL, "\b\b\b\b%d%%)", progressPercent + 1);
PLOG(PL_ALWAYS, "\b\b\b\b%d%%)", progressPercent + 1);
if (progressPercent < 99) progressPercent++;
progressCounter = 0;
}
}
if (progressPercent < 10)
PLOG(PL_FATAL, "\b\b\b100%%)\n");
PLOG(PL_ALWAYS, "\b\b\b100%%)\n");
else
PLOG(PL_FATAL, "\b\b\b\b100%%)\n");
PLOG(PL_ALWAYS, "\b\b\b\b100%%)\n");
in_file.Close();
out_file.Close();
@ -815,7 +824,11 @@ bool NormPrecodeApp::Decode()
InitInterleaver(numInputSegments);
// 2) init FEC decoder
#ifdef USE_MDP_FEC
NormDecoderMDP decoder;
#else
NormDecoderRS8 decoder;
#endif // if/else USE_MDP_FEC
unsigned int dataSegmentSize = segment_size - 4; // leaves space for our CRC
if (!decoder.Init(num_data, num_parity, dataSegmentSize))
{
@ -894,7 +907,7 @@ bool NormPrecodeApp::Decode()
// Read and decode each block in "in_file"
NormFile::Offset fecBlockId = 0;
NormFile::Offset outFileSize;
NormFile::Offset outFileSize = 0;
NormFile::Offset lastInterleaverBlockId = numInputSegments / interleaver_size;
NormFile::Offset lastInterleaverBytes = (numInputSegments % interleaver_size) * segment_size;
@ -1084,9 +1097,9 @@ bool NormPrecodeApp::Decode()
if (++progressCounter >= progressThreshold)
{
if (progressPercent < 9)
PLOG(PL_FATAL, "\b\b\b%d%%)", progressPercent + 1);
PLOG(PL_ALWAYS, "\b\b\b%d%%)", progressPercent + 1);
else if (progressPercent < 99)
PLOG(PL_FATAL, "\b\b\b\b%d%%)", progressPercent + 1);
PLOG(PL_ALWAYS, "\b\b\b\b%d%%)", progressPercent + 1);
if (progressPercent < 99) progressPercent++;
progressCounter = 0;
}
@ -1094,9 +1107,9 @@ bool NormPrecodeApp::Decode()
} // end while (inputSegmentId < numInputSegments)
if (progressPercent < 10)
PLOG(PL_FATAL, "\b\b\b100%%)\n");
PLOG(PL_ALWAYS, "\b\b\b100%%)\n");
else
PLOG(PL_FATAL, "\b\b\b\b100%%)\n");
PLOG(PL_ALWAYS, "\b\b\b\b100%%)\n");
// Cleanup, cleanup
delete[] iBuffer;

307
src/common/normSegment.cpp
View File

@ -6,7 +6,6 @@ NormSegmentPool::NormSegmentPool()
{
}
NormSegmentPool::~NormSegmentPool()
{
Destroy();
@ -282,7 +281,6 @@ bool NormBlock::TxUpdate(NormSegmentId nextId, NormSegmentId lastId,
UINT16 numData, UINT16 numParity, UINT16 erasureCount)
{
bool increasedRepair = false;
if (nextId < numData)
{
// Explicit data repair request
@ -314,6 +312,8 @@ bool NormBlock::TxUpdate(NormSegmentId nextId, NormSegmentId lastId,
}
else
{
// TBD - double-check this ... not sure this is exactly right
// (may need to always do explicit repair here?)
// Use any remaining fresh parity ...
if (parity_count < parityAvailable)
{
@ -342,7 +342,7 @@ bool NormBlock::HandleSegmentRequest(NormSegmentId nextId, NormSegmentId lastId,
UINT16 numData, UINT16 numParity, UINT16 erasureCount)
{
PLOG(PL_TRACE, "NormBlock::HandleSegmentRequest() blk>%lu seg>%hu:%hu erasures:%hu\n",
(UINT32)blk_id, (UINT16)nextId, (UINT16)lastId, erasureCount);
(unsigned long)blk_id.GetValue(), (UINT16)nextId, (UINT16)lastId, erasureCount);
bool increasedRepair = false;
if (nextId < numData)
{
@ -375,6 +375,8 @@ bool NormBlock::HandleSegmentRequest(NormSegmentId nextId, NormSegmentId lastId,
}
else
{
// TBD - double-check this. It may not be exactly right
// (may need to alwayds do explicitr repair here)
// Use any remaining fresh parity ...
if (parity_count < parityAvailable)
{
@ -399,15 +401,16 @@ bool NormBlock::HandleSegmentRequest(NormSegmentId nextId, NormSegmentId lastId,
return increasedRepair;
} // end NormBlock::HandleSegmentRequest()
// (TBD) this should return true is something is appending, false otherwise
// (TBD) this should return true if something is appended, false otherwise
bool NormBlock::AppendRepairAdv(NormCmdRepairAdvMsg& cmd,
NormObjectId objectId,
bool repairInfo,
UINT8 fecId,
UINT8 fecM,
UINT16 numData,
UINT16 segmentSize)
UINT16 payloadMax)
{
bool requestAppended = false;
NormRepairRequest req;
req.SetFlag(NormRepairRequest::SEGMENT);
if (repairInfo) req.SetFlag(NormRepairRequest::INFO);
@ -447,12 +450,14 @@ bool NormBlock::AppendRepairAdv(NormCmdRepairAdvMsg& cmd,
{
if (0 == cmd.PackRepairRequest(req))
{
prevForm = NormRepairRequest::INVALID;
PLOG(PL_WARN, "NormBlock::AppendRepairAdv() warning: full msg\n");
break;
}
requestAppended = true;
}
req.SetForm(form);
cmd.AttachRepairRequest(req, segmentSize); // (TBD) error check
cmd.AttachRepairRequest(req, payloadMax); // (TBD) error check
prevForm = form;
}
switch(form)
@ -480,9 +485,11 @@ bool NormBlock::AppendRepairAdv(NormCmdRepairAdvMsg& cmd,
{
if (0 == cmd.PackRepairRequest(req))
PLOG(PL_WARN, "NormBlock::AppendRepairAdv() warning: full msg\n");
else
requestAppended = true;
}
}
return true;
return requestAppended;
} // end NormBlock::AppendRepairAdv()
NormObjectSize NormBlock::GetBytesPending(UINT16 numData,
@ -521,8 +528,9 @@ bool NormBlock::AppendRepairRequest(NormNackMsg& nack,
UINT16 numParity,
NormObjectId objectId,
bool pendingInfo,
UINT16 segmentSize)
UINT16 payloadMax)
{
bool requestAppended = false;
NormSegmentId nextId = 0;
NormSegmentId endId;
if (erasure_count > numParity)
@ -580,11 +588,13 @@ bool NormBlock::AppendRepairRequest(NormNackMsg& nack,
{
if (0 == nack.PackRepairRequest(req))
{
prevForm = NormRepairRequest::INVALID; // so we don't re-attempt pack
PLOG(PL_WARN, "NormBlock::AppendRepairRequest() warning: full NACK msg\n");
break;
}
requestAppended = true;
}
nack.AttachRepairRequest(req, segmentSize); // (TBD) error check
nack.AttachRepairRequest(req, payloadMax); // (TBD) error check
req.SetForm(form);
prevForm = form;
}
@ -614,8 +624,10 @@ bool NormBlock::AppendRepairRequest(NormNackMsg& nack,
{
if (0 == nack.PackRepairRequest(req))
PLOG(PL_WARN, "NormBlock::AppendRepairRequest() warning: full NACK msg\n");
else
requestAppended = true;
}
return true;
return requestAppended;
} // end NormBlock::AppendRepairRequest()
NormBlockPool::NormBlockPool()
@ -671,7 +683,12 @@ void NormBlockPool::Destroy()
} // end NormBlockPool::Destroy()
NormBlockBuffer::NormBlockBuffer()
: table((NormBlock**)NULL), range_max(0), range(0)
#ifdef USE_PROTO_TREE
:
#else
: table((NormBlock**)NULL),
#endif // if/else USE_PROTO_TREE
range_max(0), range(0), fec_block_mask(0)
{
}
@ -680,16 +697,17 @@ NormBlockBuffer::~NormBlockBuffer()
Destroy();
}
bool NormBlockBuffer::Init(unsigned long rangeMax, unsigned long tableSize)
bool NormBlockBuffer::Init(unsigned long rangeMax, unsigned long tableSize, UINT32 fecBlockMask)
{
if (table) Destroy();
Destroy();
// Make sure tableSize is greater than 0 and 2^n
if (!rangeMax || !tableSize)
{
PLOG(PL_FATAL, "NormBlockBuffer::Init() bad range(%lu) or tableSize(%lu)\n",
rangeMax, tableSize);
rangeMax, tableSize);
return false;
}
#ifndef USE_PROTO_TREE
if (0 != (tableSize & 0x07)) tableSize = (tableSize >> 3) + 1;
if (!(table = new NormBlock*[tableSize]))
{
@ -698,14 +716,38 @@ bool NormBlockBuffer::Init(unsigned long rangeMax, unsigned long tableSize)
}
memset(table, 0, tableSize*sizeof(char*));
hash_mask = tableSize - 1;
#endif // !USE_PROTO_TREE
range_max = rangeMax;
range = 0;
fec_block_mask = fecBlockMask;
return true;
} // end NormBlockBuffer::Init()
#ifdef USE_PROTO_TREE
void NormBlockBuffer::Destroy()
{
NormBlock* block;
while((block = Find(range_lo)))
{
PLOG(PL_ERROR, "NormBlockBuffer::Destroy() buffer not empty!?\n");
Remove(block);
delete block;
}
range_max = range = 0;
} // end NormBlockBuffer::Destroy()
NormBlock* NormBlockBuffer::Find(const NormBlockId& blockId) const
{
if ((0 == range) || (Compare(blockId, range_lo) < 0) || (Compare(blockId, range_hi) > 0))
return NULL;
else
return tree.Find(blockId.GetValuePtr(), 8*sizeof(UINT32));
} // end NormBlockBuffer::Find()
#else
void NormBlockBuffer::Destroy()
{
range_max = range = 0;
if (table)
{
NormBlock* block;
@ -717,18 +759,19 @@ void NormBlockBuffer::Destroy()
}
delete []table;
table = (NormBlock**)NULL;
range_max = 0;
}
}
range_max = range = 0;
} // end NormBlockBuffer::Destroy()
NormBlock* NormBlockBuffer::Find(const NormBlockId& blockId) const
{
if (range)
{
if ((blockId < range_lo) || (blockId > range_hi))
//if ((blockId < range_lo) || (blockId > range_hi))
if ((Compare(blockId, range_lo) < 0) || (Compare(blockId, range_hi) > 0))
return (NormBlock*)NULL;
NormBlock* theBlock = table[((UINT32)blockId) & hash_mask];
while (theBlock && (blockId != theBlock->GetId()))
NormBlock* theBlock = table[(blockId.GetValue()) & hash_mask];
while ((NULL != theBlock) && (blockId != theBlock->GetId()))
theBlock = theBlock->next;
return theBlock;
}
@ -738,21 +781,38 @@ NormBlock* NormBlockBuffer::Find(const NormBlockId& blockId) const
}
} // end NormBlockBuffer::Find()
#endif // if/else USE_PROTO_TREE
NormBlockId NormBlockBuffer::RangeMin() const
{
if (range_max > 1)
{
NormBlockId rangeMin = range_hi;
Decrement(rangeMin, range_max - 1);
return rangeMin;
}
else
{
return range_lo;
}
} // end NormBlockBuffer::RangeMin()
bool NormBlockBuffer::CanInsert(NormBlockId blockId) const
{
if (0 != range)
{
if (blockId < range_lo)
// if (blockId < range_lo)
if (Compare(blockId, range_lo) < 0)
{
if ((range_lo - blockId + range) > range_max)
if (((UINT32)Difference(range_lo, blockId) + range) > range_max)
return false;
else
return true;
}
else if (blockId > range_hi)
// else if (blockId > range_hi)
else if (Compare(blockId, range_hi) > 0)
{
if ((blockId - range_hi + range) > range_max)
if (((UINT32)Difference(blockId, range_hi) + range) > range_max)
return false;
else
return true;
@ -772,29 +832,38 @@ bool NormBlockBuffer::CanInsert(NormBlockId blockId) const
bool NormBlockBuffer::Insert(NormBlock* theBlock)
{
const NormBlockId& blockId = theBlock->GetId();
if (!range)
if (0 == range)
{
range_lo = range_hi = blockId;
range = 1;
}
if (blockId < range_lo)
// else if (blockId < range_lo)
else if (Compare(blockId, range_lo) < 0)
{
UINT32 newRange = range_lo - blockId + range;
UINT32 newRange = (UINT32)Difference(range_lo, blockId) + range;
if (newRange > range_max) return false;
range_lo = blockId;
range = newRange;
}
else if (blockId > range_hi)
// else if (blockId > range_hi)
else if (Compare(blockId, range_hi) > 0)
{
UINT32 newRange = blockId - range_hi + range;
UINT32 newRange = (UINT32)Difference(blockId, range_hi) + range;
if (newRange > range_max) return false;
range_hi = blockId;
range = newRange;
}
UINT32 index = ((UINT32)blockId) & hash_mask;
ASSERT(Compare(range_hi, range_lo) >= 0);
// else unchanged range
#ifdef USE_PROTO_TREE
ASSERT(NULL == Find(theBlock->GetId()));
tree.Insert(*theBlock);
#else
UINT32 index = blockId.GetValue() & hash_mask;
NormBlock* prev = NULL;
NormBlock* entry = table[index];
while (entry && (entry->GetId() < blockId))
// while (entry && (entry->GetId() < blockId))
while ((NULL != entry) && (Compare(entry->GetId(), blockId) < 0))
{
prev = entry;
entry = entry->next;
@ -803,21 +872,66 @@ bool NormBlockBuffer::Insert(NormBlock* theBlock)
prev->next = theBlock;
else
table[index] = theBlock;
ASSERT((entry ? (blockId != entry->GetId()) : true));
theBlock->next = entry;
#endif // if/else USE_PROTO_TREE
return true;
} // end NormBlockBuffer::Insert()
bool NormBlockBuffer::Remove(const NormBlock* theBlock)
#ifdef USE_PROTO_TREE
bool NormBlockBuffer::Remove(NormBlock* theBlock)
{
ASSERT(NULL != theBlock);
const NormBlockId& blockId = theBlock->GetId();
switch (range)
{
case 0:
return false; // empty NormBlockBuffer
case 1:
if (blockId != range_lo)
return false; // out-of-range
range = 0;
break;
default:
if ((Compare(blockId, range_lo) < 0) || (Compare(blockId, range_hi) > 0))
return false; // out-of-range
if (blockId == range_lo)
{
const NormBlock* next = static_cast<const NormBlock*>(theBlock->GetNext());
if (NULL == next) next = static_cast<const NormBlock*>(tree.GetHead());
ASSERT(NULL != next);
range_lo = next->GetId();
range = Difference(range_hi, range_lo) + 1;
}
else if (blockId == range_hi)
{
const NormBlock* prev = static_cast<const NormBlock*>(theBlock->GetPrev());
if (NULL == prev) prev = static_cast<const NormBlock*>(tree.GetTail());
ASSERT(NULL != prev);
range_hi = prev->GetId();
range = Difference(range_hi, range_lo) + 1;
}
// else range unchanged
break;
}
ASSERT(NULL != tree.Find(theBlock->GetId().GetValuePtr(), 8*sizeof(UINT32)));
tree.Remove(*theBlock);
return true;
} // end NormBlockBuffer::Remove()
#else
bool NormBlockBuffer::Remove(NormBlock* theBlock)
{
ASSERT(NULL != theBlock);
if (range)
{
const NormBlockId& blockId = theBlock->GetId();
if ((blockId < range_lo) || (blockId > range_hi)) return false;
UINT32 index = ((UINT32)blockId) & hash_mask;
// if ((blockId < range_lo) || (blockId > range_hi))
if ((Compare(blockId, range_lo) < 0) || (Compare(blockId, range_hi) > 0))
return false;
UINT32 index = blockId.GetValue() & hash_mask;
NormBlock* prev = NULL;
NormBlock* entry = table[index];
while (entry && (entry->GetId() != blockId))
@ -825,8 +939,8 @@ bool NormBlockBuffer::Remove(const NormBlock* theBlock)
prev = entry;
entry = entry->next;
}
if (!entry) return false;
if (prev)
if (NULL == entry) return false;
if (NULL != prev)
prev->next = entry->next;
else
table[index] = entry->next;
@ -835,7 +949,7 @@ bool NormBlockBuffer::Remove(const NormBlock* theBlock)
{
if (blockId == range_lo)
{
// Find next entry for range_lo
// Find next entry for new range_lo
UINT32 i = index;
UINT32 endex;
if (range <= hash_mask)
@ -849,29 +963,33 @@ bool NormBlockBuffer::Remove(const NormBlock* theBlock)
{
++i &= hash_mask;
offset++;
if ((entry = table[i]))
if (NULL != (entry = table[i]))
{
//NormBlockId id = (UINT32)index + offset;
NormBlockId id = (UINT32)blockId + offset;
// NormBlockId id = blockId + offset;
NormBlockId id = blockId;
Increment(id, offset);
while(entry && (entry->GetId() != id))
{
if ((entry->GetId() > blockId) &&
(entry->GetId() < nextId)) nextId = entry->GetId();
// if ((entry->GetId() > blockId) && (entry->GetId() < nextId)
if ((Compare(entry->GetId(), blockId) > 0) &&
(Compare(entry->GetId(), nextId) < 0))
{
nextId = entry->GetId();
}
entry = entry->next;
}
if (entry) break;
if (NULL != entry) break;
}
} while (i != endex);
if (entry)
if (NULL != entry)
range_lo = entry->GetId();
else
range_lo = nextId;
range = range_hi - range_lo + 1;
range = (UINT32)Difference(range_hi, range_lo) + 1;
}
else if (blockId == range_hi)
{
// Find prev entry for range_hi
// Find prev entry for new range_hi
UINT32 i = index;
UINT32 endex;
if (range <= hash_mask)
@ -880,7 +998,6 @@ bool NormBlockBuffer::Remove(const NormBlock* theBlock)
endex = index;
entry = NULL;
UINT32 offset = 0;
//printf("preving i:%lu endex:%lu lo:%lu hi:%lu\n", i, endex, (UINT32)range_lo, (UINT32) range_hi);
NormBlockId prevId = range_lo;
do
{
@ -888,23 +1005,27 @@ bool NormBlockBuffer::Remove(const NormBlock* theBlock)
offset++;
if ((entry = table[i]))
{
//NormBlockId id = (UINT32)index - offset;
NormBlockId id = (UINT32)blockId - offset;
//printf("Looking for id:%lu at index:%lu\n", (UINT32)id, i);
// NormBlockId id = blockId - offset;
NormBlockId id = blockId;
Decrement(id, offset);
while(entry && (entry->GetId() != id))
{
if ((entry->GetId() < blockId) &&
(entry->GetId() > prevId)) prevId = entry->GetId();
// if ((entry->GetId() < blockId) && (entry->GetId() > prevId))
if ((Compare(entry->GetId(), blockId) < 0) &&
(Compare(entry->GetId(), prevId) > 0))
{
prevId = entry->GetId();
}
entry = entry->next;
}
if (entry) break;
if (NULL != entry) break;
}
} while (i != endex);
if (entry)
if (NULL != entry)
range_hi = entry->GetId();
else
range_hi = prevId;
range = range_hi - range_lo + 1;
range = (UINT32)Difference(range_hi, range_lo) + 1;
}
}
else
@ -919,6 +1040,48 @@ bool NormBlockBuffer::Remove(const NormBlock* theBlock)
}
} // end NormBlockBuffer::Remove()
#endif // if/else USE_PROTO_TREE
#ifdef USE_PROTO_TREE
NormBlockBuffer::Iterator::Iterator(NormBlockBuffer& blockBuffer)
: buffer(blockBuffer), iterator(blockBuffer.tree, false, blockBuffer.range_lo.GetValuePtr(), 8*sizeof(UINT32))
{
next_block = iterator.GetNextItem();
ASSERT((NULL == next_block) || (blockBuffer.range_lo == next_block->GetId()))
}
void NormBlockBuffer::Iterator::Reset()
{
iterator.Reset(false, buffer.range_lo.GetValuePtr(), 8*sizeof(UINT32));
next_block = iterator.GetNextItem();
ASSERT(buffer.IsEmpty() || (NULL != next_block));
ASSERT((NULL == next_block) || (buffer.range_lo == next_block->GetId()))
} // end NormBlockBuffer::Iterator::Reset()
NormBlock* NormBlockBuffer::Iterator::GetNextBlock()
{
NormBlock* nextBlock = next_block;
if (NULL != nextBlock)
{
next_block = iterator.GetNextItem();
if (NULL == next_block)
{
iterator.Reset();
next_block = iterator.GetNextItem();
if (buffer.Compare(next_block->GetId(), nextBlock->GetId()) <= 0)
next_block = NULL;
}
else if (buffer.Compare(next_block->GetId(), nextBlock->GetId()) <= 0)
{
next_block = NULL;
}
}
return nextBlock;
} // end NormBlockBuffer::Iterator::GetNextBlock()
#else
NormBlockBuffer::Iterator::Iterator(const NormBlockBuffer& blockBuffer)
: buffer(blockBuffer), reset(true)
{
@ -941,30 +1104,38 @@ NormBlock* NormBlockBuffer::Iterator::GetNextBlock()
}
else
{
if (buffer.range &&
(index < buffer.range_hi) &&
(index >= buffer.range_lo))
// if (buffer.range && (index < buffer.range_hi) && (index >= buffer.range_lo))
if ((0 != buffer.range) &&
(buffer.Compare(index, buffer.range_hi) < 0) &&
(buffer.Compare(index, buffer.range_lo) >= 0))
{
// Find next entry _after_ current "index"
UINT32 i = index;
UINT32 i = index.GetValue();;
UINT32 endex;
if ((UINT32)(buffer.range_hi - index) <= buffer.hash_mask)
endex = buffer.range_hi & buffer.hash_mask;
// if ((UINT32)(buffer.range_hi - index) <= buffer.hash_mask)
if ((UINT32)buffer.Difference(buffer.range_hi, index) <= buffer.hash_mask)
endex = buffer.range_hi.GetValue() & buffer.hash_mask;
else
endex = index;
endex = index.GetValue();
UINT32 offset = 0;
NormBlockId nextId = buffer.range_hi;
do
{
++i &= buffer.hash_mask;
offset++;
NormBlockId id = (UINT32)index + offset;
// NormBlockId id = (UINT32)index + offset;
NormBlockId id = index;
buffer.Increment(id, offset);
ASSERT(i < 256);
NormBlock* entry = buffer.table[i];
while ((NULL != entry ) && (entry->GetId() != id))
{
if ((entry->GetId() > index) && (entry->GetId() < nextId))
// if ((entry->GetId() > index) && (entry->GetId() < nextId))
if ((buffer.Compare(entry->GetId(), index) > 0) &&
(buffer.Compare(entry->GetId(), nextId) < 0))
{
nextId = entry->GetId();
}
entry = NormBlockBuffer::Next(entry);
}
if (entry)
@ -983,3 +1154,5 @@ NormBlock* NormBlockBuffer::Iterator::GetNextBlock()
}
}
} // end NormBlockBuffer::Iterator::GetNextBlock()
#endif // if/else USE_PROTO_TREE

979
src/common/normSession.cpp
View File

File diff suppressed because it is too large Load Diff

56
src/common/pcap2norm.cpp
View File

@ -5,6 +5,7 @@
#include <stdio.h>
#include <pcap.h>
#include <sys/socket.h> // for PF_ types (protocol family)
#include "protoPktETH.h" // for Ethernet frame parsing
#include "protoPktIP.h" // for IP packet parsing
#include "protoPktARP.h"
@ -68,8 +69,9 @@ int main(int argc, char* argv[])
return -1;
}
int deviceType = pcap_datalink(pcapDevice);
UINT32 alignedBuffer[4096/4]; // 128 buffer for packet parsing
UINT32 alignedBuffer[4096/4]; // 4096 byte buffer for packet parsing
UINT16* ethBuffer = ((UINT16*)alignedBuffer) + 1;
unsigned int maxBytes = 4096 - 2; // due to offset, can only use 4094 bytes of buffer
@ -79,21 +81,47 @@ int main(int argc, char* argv[])
{
unsigned int numBytes = maxBytes;
if (hdr.caplen < numBytes) numBytes = hdr.caplen;
memcpy(ethBuffer, pktData, numBytes);
ProtoPktETH ethPkt((UINT32*)ethBuffer, maxBytes);
if (!ethPkt.InitFromBuffer(hdr.len))
ProtoPktETH::Type ethType;
unsigned int payloadLength;
UINT32* payloadPtr;
if (DLT_NULL == deviceType)
{
fprintf(stderr, "pcap2norm error: invalid Ether frame in pcap file\n");
continue;
}
// pcap was captured from "loopback" device
memcpy(alignedBuffer, pktData, numBytes);
switch (alignedBuffer[0])
{
case PF_INET:
ethType = ProtoPktETH::IP;
break;
case PF_INET6:
ethType = ProtoPktETH::IPv6;
break;
default:
continue; // not an IP packet
}
payloadLength = numBytes - 4;
payloadPtr = alignedBuffer + 1;
}
else
{
memcpy(ethBuffer, pktData, numBytes);
ProtoPktETH ethPkt((UINT32*)ethBuffer, maxBytes);
if (!ethPkt.InitFromBuffer(hdr.len))
{
fprintf(stderr, "pcap2norm error: invalid Ether frame in pcap file\n");
continue;
}
ethType = ethPkt.GetType();
payloadLength = ethPkt.GetPayloadLength();
payloadPtr = (UINT32*)ethPkt.AccessPayload();
}
ProtoPktIP ipPkt;
ProtoAddress srcAddr, dstAddr;
ProtoPktETH::Type ethType = ethPkt.GetType();
if ((ProtoPktETH::IP == ethType) ||
(ProtoPktETH::IPv6 == ethType))
{
unsigned int payloadLength = ethPkt.GetPayloadLength();
if (!ipPkt.InitFromBuffer(payloadLength, (UINT32*)ethPkt.AccessPayload(), payloadLength))
if (!ipPkt.InitFromBuffer(payloadLength, payloadPtr, payloadLength))
{
fprintf(stderr, "pcap2norm error: bad IP packet\n");
continue;
@ -123,6 +151,10 @@ int main(int argc, char* argv[])
//PLOG(PL_ALWAYS, "pcap2norm IP packet dst>%s ", dstAddr.GetHostString());
//PLOG(PL_ALWAYS," src>%s length>%d\n", srcAddr.GetHostString(), ipPkt.GetLength());
}
else
{
fprintf(stderr, "eth type = %d\n", ethType);
}
if (!srcAddr.IsValid()) continue; // wasn't an IP packet
ProtoPktUDP udpPkt;
@ -153,7 +185,7 @@ void NormTrace2(const struct timeval& currentTime,
const ProtoAddress& dstAddr)
{
UINT8 fecM = 16; // NOTE - this assumes 16-bit RS code for fec_id == 2
UINT8 fecM = 8; // NOTE - this assumes 16-bit RS code for fec_id == 2
static const char* MSG_NAME[] =
{
@ -227,7 +259,7 @@ void NormTrace2(const struct timeval& currentTime,
{
const NormDataMsg& data = (const NormDataMsg&)msg;
lastFecId = data.GetFecId();
PLOG(PL_ALWAYS, "inst>%hu seq>%hu DATA obj>%hu blk>%lu seg>%hu ",
PLOG(PL_ALWAYS, "inst>%hu seq>%hu DATA obj>%hu blk>%u seg>%04hu ",
data.GetInstanceId(),
seq,
//data.IsData() ? "DATA" : "PRTY",

10
src/common/raft.cpp
View File

@ -362,11 +362,17 @@ void RaftApp::OnInputReady()
}
else if (tx_msg_index == tx_msg_length)
{
if (!tx_socket.SendTo(tx_msg_buffer+2, tx_msg_length-2, tx_address))
unsigned int bytesSent = tx_msg_length-2;
if (!tx_socket.SendTo(tx_msg_buffer+2, bytesSent, tx_address))
{
PLOG(PL_ERROR, "raft: tx_socket.SendTo() error\n");
PLOG(PL_ERROR, "raft: tx_socket.SendTo() error: %s\n", GetErrorString());
return;
}
else if (0 == bytesSent)
{
PLOG(PL_WARN, "raft: tx_socket.SendTo() error: %s\n", GetErrorString());
return;
}
tx_msg_index = tx_msg_length = 0;
}

6
src/java/jni/normInstanceJni.cpp
View File

@ -179,6 +179,12 @@ JNIEXPORT jobject JNICALL PKGNAME(NormInstance_getNextEvent)
// Get the event type
jobjectArray array = (jobjectArray)env->CallStaticObjectMethod(
(jclass)env->NewLocalRef(jw_NormEventType), mid_NormEventType_values);
if (env->GetArrayLength(array) <= event.type) {
env->ThrowNew((jclass)env->NewLocalRef(jw_IOException), "Invalid NORM event type (NormEventType.java out of sync with NORM API event header?)");
return false;
}
jobject type = env->GetObjectArrayElement(array, event.type);
// Create the event

2
src/java/jni/normNodeJni.cpp
View File

@ -57,7 +57,7 @@ JNIEXPORT jobject JNICALL PKGNAME(NormNode_getAddress)
NormNodeHandle handle;
char buffer[256];
unsigned int bufferLen;
unsigned short port;
UINT16 port;
jobject address;
jbyteArray array;
jbyte *ptr;

55
src/java/src/mil/navy/nrl/norm/enums/NormEventType.java
View File

@ -7,30 +7,33 @@ package mil.navy.nrl.norm.enums;
*
* @author Jason Rush
*/
public enum NormEventType {
NORM_EVENT_INVALID,
NORM_TX_QUEUE_VACANCY,
NORM_TX_QUEUE_EMPTY,
NORM_TX_FLUSH_COMPLETED,
NORM_TX_WATERMARK_COMPLETED,
NORM_TX_CMD_SENT,
NORM_TX_OBJECT_SENT,
NORM_TX_OBJECT_PURGED,
NORM_TX_RATE_CHANGED,
NORM_LOCAL_SENDER_CLOSED,
NORM_REMOTE_SENDER_NEW,
NORM_REMOTE_SENDER_RESET,
NORM_REMOTE_SENDER_ADDRESS,
NORM_REMOTE_SENDER_ACTIVE,
NORM_REMOTE_SENDER_INACTIVE,
NORM_REMOTE_SENDER_PURGED,
NORM_RX_CMD_NEW,
NORM_RX_OBJECT_NEW,
NORM_RX_OBJECT_INFO,
NORM_RX_OBJECT_UPDATED,
NORM_RX_OBJECT_COMPLETED,
NORM_RX_OBJECT_ABORTED,
NORM_GRTT_UPDATED,
NORM_CC_ACTIVE,
NORM_CC_INACTIVE;
public enum NormEventType {
NORM_EVENT_INVALID,
NORM_TX_QUEUE_VACANCY,
NORM_TX_QUEUE_EMPTY,
NORM_TX_FLUSH_COMPLETED,
NORM_TX_WATERMARK_COMPLETED,
NORM_TX_CMD_SENT,
NORM_TX_OBJECT_SENT,
NORM_TX_OBJECT_PURGED,
NORM_TX_RATE_CHANGED,
NORM_LOCAL_SENDER_CLOSED,
NORM_REMOTE_SENDER_NEW,
NORM_REMOTE_SENDER_RESET,
NORM_REMOTE_SENDER_ADDRESS,
NORM_REMOTE_SENDER_ACTIVE,
NORM_REMOTE_SENDER_INACTIVE,
NORM_REMOTE_SENDER_PURGED,
NORM_RX_CMD_NEW,
NORM_RX_OBJECT_NEW,
NORM_RX_OBJECT_INFO,
NORM_RX_OBJECT_UPDATED,
NORM_RX_OBJECT_COMPLETED,
NORM_RX_OBJECT_ABORTED,
NORM_GRTT_UPDATED,
NORM_CC_ACTIVE,
NORM_CC_INACTIVE,
NORM_ACKING_NODE_NEW,
NORM_SEND_ERROR,
NORM_USER_TIMEOUT
}

2
src/pynorm/core.py
View File

@ -506,5 +506,5 @@ def get_libnorm():
return libnorm
# Global libnorm refrence that everyone should use
# Global libnorm reference that everyone should use
libnorm = get_libnorm()

6
src/pynorm/event.py
View File

@ -91,5 +91,11 @@ class Event(object):
return "NORM_CC_ACTIVE"
elif self.type == c.NORM_CC_INACTIVE:
return "NORM_CC_INACTIVE"
elif self.type == c.NORM_ACKING_NODE_NEW:
return "NORM_ACKING_NODE_NEW"
elif self.type == c.NORM_SEND_ERROR:
return "NORM_SEND_ERROR"
elif self.type == c.NORM_USER_TIMEOUT:
return "NORM_USER_TIMEOUT"
else:
return "Unknown event type"

9
src/pynorm/node.py
View File

@ -29,14 +29,17 @@ class Node(object):
return self._address
except AttributeError:
port = ctypes.c_uint16()
buffer = ctypes.create_string_buffer(50)
buf = ctypes.create_string_buffer(50)
size = ctypes.c_uint(50)
if not libnorm.NormNodeGetAddress(self, buffer, ctypes.byref(size),
if not libnorm.NormNodeGetAddress(self, buf, ctypes.byref(size),
ctypes.byref(port)):
raise NormError("Node getAddress failed")
self._address = (buffer.value, port.value)
self._address = (buf.value, port.value)
return self._address
def getCommand(self, buf):
return libnorm.NormNodeGetCommand(self, buf, len(buf))
def getGrtt(self):
grtt = libnorm.NormNodeGetGrtt(self)
if grtt == -1.0:

6
src/pynorm/session.py
View File

@ -133,6 +133,12 @@ class Session(object):
def streamOpen(self, bufferSize, info=""):
return Object(libnorm.NormStreamOpen(self, bufferSize, info, len(info)))
def sendCommand(self, cmdBuffer, robust=False):
return libnorm.NormSendCommand(self, cmdBuffer, len(cmdBuffer), robust)
def cancelCommand(self):
libnorm.NormCancelCommand(self)
def setWatermark(self, normObject, overrideFlush=False):
libnorm.NormSetWatermark(self, normObject, overrideFlush)

0
makefiles/win32/win32PostProcess.cpp → src/win32/win32PostProcess.cpp
View File

21
waf vendored
View File

File diff suppressed because one or more lines are too long

100
wscript
View File

@ -11,6 +11,10 @@ build), use the -o (--out) flag when configuring. For example:
./waf configure -o build-release
./waf
To build examples, use the --target directive. For example:
./waf build --target=normClient,normServer
'''
@ -18,6 +22,18 @@ import platform
import waflib
# Fetch VERSION from include/normVersion.h file
VERSION = None
vfile = open('include/normVersion.h', 'r')
for line in vfile.readlines():
line = line.split()
if len(line) != 3:
continue
if "#define" == line[0] and "VERSION" == line[1]:
VERSION = line[2].strip('"')
if VERSION is None:
print "Warning: NORM VERSION not found!?"
# So you don't need to do ./waf configure if you are just using the defaults
waflib.Configure.autoconfig = True
@ -30,7 +46,11 @@ out = 'build'
system = platform.system().lower()
def options(ctx):
ctx.recurse('protolib')
ctx.recurse('protolib')
build_opts = ctx.parser.add_option_group('Compile/install Options', 'Use during build/install step.')
build_opts.add_option('--enable-static-library', action='store_true',
help='Enable building and installing static library. [default:false]')
def configure(ctx):
ctx.recurse('protolib')
@ -38,7 +58,7 @@ def configure(ctx):
# Use this USE variable to add flags to NORM's compilation
ctx.env.USE_BUILD_NORM += ['BUILD_NORM', 'protolib']
if system in ('linux', 'darwin', 'freebsd'):
if system in ('linux', 'darwin', 'freebsd', 'gnu', 'gnu/kfreebsd'):
ctx.env.DEFINES_BUILD_NORM += ['ECN_SUPPORT']
if system == 'windows':
@ -46,13 +66,16 @@ def configure(ctx):
def build(ctx):
ctx.recurse('protolib')
ctx.shlib(
target = 'norm',
# Setup to install NORM header file
ctx.install_files("${PREFIX}/include/", "include/normApi.h")
ctx.objects(
target = 'objs',
includes = ['include'],
export_includes = ['include'],
vnum = '1.0.0',
use = ctx.env.USE_BUILD_NORM,
stlib = ["protolib"],
source = ['src/common/{0}.cpp'.format(x) for x in [
'galois',
'normApi',
@ -68,6 +91,32 @@ def build(ctx):
'normSession',
]],
)
# Use static lib for Unix examples for convenience
# (so we don't have to worry about LD_LIBRARY_PATH)
ctx.shlib(
target = 'norm',
includes = ['include'],
export_includes = ['include'],
vnum = VERSION,
stlib = ["protolib"],
use = ['objs'] + ctx.env.USE_BUILD_NORM,
source = [],
features = 'cxx cxxshlib',
)
if ctx.options.enable_static_library:
ctx.stlib(
target = 'norm',
includes = ['include'],
export_includes = ['include'],
vnum = VERSION,
stlib = ["protolib"],
use = ['objs'] + ctx.env.USE_BUILD_NORM,
source = [],
features = 'cxx cxxstlib',
install_path = '${LIBDIR}',
)
if ctx.env.BUILD_PYTHON:
ctx(
@ -79,7 +128,8 @@ def build(ctx):
if ctx.env.BUILD_JAVA:
ctx.shlib(
target = 'mil_navy_nrl_norm',
use = ['norm', 'JAVA'],
use = ['norm_shlib', 'JAVA'],
vnum = VERSION,
source = ['src/java/jni/{0}.cpp'.format(x) for x in [
'normJni',
'normInstanceJni',
@ -101,11 +151,12 @@ def build(ctx):
)
normapp = ctx.program(
target = 'norm',
# Need to explicitly set a different name, because the library is also
# named "norm"
# Need to explicitly set a different name, because
# the library is also named "norm"
name = 'normapp',
use = ['protolib', 'norm'],
target = 'normapp',
use = ['protolib', 'norm_stlib'],
defines = [],
source = ['src/common/{0}'.format(x) for x in [
'normPostProcess.cpp',
'normApp.cpp',
@ -114,11 +165,13 @@ def build(ctx):
posted = True,
)
if system in ('linux', 'darwin', 'freebsd'):
if system in ('linux', 'darwin', 'freebsd', 'gnu', 'gnu/kfreebsd'):
normapp.source.append('src/unix/unixPostProcess.cpp')
if system == 'windows':
normapp.source.append('src/win32/win32PostProcess.cpp')
normapp.defines.append('_CONSOLE')
normapp.stlib = (["Shell32"]);
for example in (
'normDataExample',
@ -128,6 +181,11 @@ def build(ctx):
'normFileSend',
'normStreamRecv',
'normStreamSend',
'normMsgr',
'normStreamer',
'normClient',
'normServer',
'wintest'
):
_make_simple_example(ctx, example)
@ -146,14 +204,24 @@ def build(ctx):
def _make_simple_example(ctx, name, path='examples'):
'''Makes a task from a single source file in the examples directory.
These tasks are not built by default. Use the --targets flag.
Note these tasks are not built by default. Use the --targets flag.
'''
ctx.program(
source = ['{0}/{1}.cpp'.format(path, name)]
if 'normClient' == name or 'normServer' == name:
source.append('%s/normSocket.cpp' % path)
example = ctx.program(
target = name,
use = ['protolib', 'norm'],
source = ['{0}/{1}.cpp'.format(path, name)],
use = ['protolib'],
defines = [],
source = source,
# Don't build examples by default
posted = True,
# Don't install examples
install_path = False,
)
if 'windows' == system:
example.use.append('norm_shlib')
example.defines.append('_CONSOLE')
else:
example.use.append('norm_stlib')