Reverse I/O Stream library (RIOS)

Basically there are two ways to use RIOS:

1. compile a library and link you program to it. Together with the library the test program stream-test is also build.
2. just include the generated all-sources file librios.cc or librios-windows.cc for Windows builds into you program.

Invoke make, possibly with setting the MPI_IMPL variable to an adequate value. The Makefile variable MPI_IMPL controls the use of MPI-IO. Possible values are:

• none
• openmpi
• openmpi_naked
• mpich2
• mpich2_naked
• mpich2_compiled
• mpich2_compiled_naked

The default value is mpich2.

The setting none means no MPI-IO support. In this case the macro RIOS_WITH_MPI is not defined, in all other cases it is defined.

Here <name>_naked means that the regular C++ compiler is used and the required libraries added manually. This will likely break sooner or later as the set of libraries changes in one MPI distro ro the other.

Further, the idea is that openmpi and mpich2 use the system libraries when the relevant packages have been installed on a Debian system.

The setting mpich2_compiled is for an MPICH2 library compiled and installed by hand. It's location is determined by the variable MPICH2_HOME (mpich2_compiled) or MPICH2_INC and MPICH2_LIB (in the case of mpich2_compiled_naked). These are variables that are commonly set by the module environment on a cluster.

make MPI_IMPL=mpich2

The two all-sources files librios.cc and librios-windows.cc are already present in the sources. They have a target in the Makefile that depends on Makefile, so when you change the list of library source files in the Makefile, they will be regenerated.

To use RIOS in your project, include the main header file rios.hh. In addition, link your program to you library or also include the all-sources file librios.cc (or librios-windows.cc for Windows).

At the beginning of your program, you should call RIOS::init(). This will take care of initializing the MIP library. If you use RIOS without MPI-IO support, this may be omitted.

Likewise, at the end of your program, call RIOS::finalize(). If you use RIOS without MPI-IO support, this may be omitted.

The function to create a RIOS iostream is called RIOS::create, with the following signature:

RiosStreamWrapper *RIOS::create(std::string const &outfileName, RiosStreamType const streamType, 
				size_t bufferSize, size_t numBuffers, int verbosity);

streamType specifies the type of stream you wish to create:

• RIOS_nullstream
• RIOS_stringstream
• RIOS_fstream
• RIOS_astream
• RIOS_SBB_Files
• RIOS_SBB_AIO
• RIOS_SBB_MPIO
• RIOS_MB_SyncIO
• RIOS_MB_OSHints
• RIOS_MB_AIO
• RIOS_MB_MPIO
• RIOS_CFile
• RIOS_MB_LIBAIO
• RIOS_WinFile
• RIOS_MB_WIN

Some of these are experimental and not all are available on all systems. We recommend that you use RIOS_MB_AIO or RIOS_MB_MPIO. Using RIOS_stringstream or RIOS_fstream will create a plain std::stringstream or std::fstream, resp. RIOS_CFile will use a streambuffer that is a thin wrapper around plain C I/O.

outfileName is the name of the file on disk (ignored in some cases, and a template in the case of RIOS_SBB_Files).

bufferSize specifies the size of the main memory buffer to use.

numBuffers specifies the number of the buffers to use.

verbose specifies the verbosity. The higher the number, the more messages you will get.

The function returns an RiosStreamWrapper object that hold the iostream created. It has two methods:

• getStream returns a pointer to the std::iostream
• deleteFiles is a utility function to delete the files created by the stream. This is useful in the case of RIOS_SBB_Files, which creates several files.

To release the resources associated with a RIOS stream, pass the pointer returned by RIOS::create to the RIOS::release function.

• RIOS_USE_AIO_INIT: whether to call aio_init when AIO is used (0 or 1). Default is 1
• AIO_THREADS: set the number of threads AIO should use
• AIO_NUM: set the expected number of simultaneous requests
• AIO_IDLE_TIME: set the idle timeout for AIO threads
• RIOS_BUFFER_CACHING: Whether to use posix_fadvise to turn OS buffer caching off. Only used when the UNIX file descriptor is known, i.e. with RIOS_MB_AIO, RIOS_MB_SyncIO, and RIOS_MB_OSHints.
• RIOS_ODIRECT: When set to 1, try to turn off system caches and buffers. With UNIX files, this adds the flag O_DIRECT when the file is open()'ed. On Windows, this add the flag FILE_FLAG_NO_BUFFERING to the CreateFile call.
• RIOS_MPI_WAITSOME_VARIANT: whether to use MPI_Waitsome from the MPI library (0) or the custom implementation that is included in RIOS, myMPI_Waitsome (1). Default is 1.