E-MailRelay Internals

Module structure

There are four main C++ libraries in the E-MailRelay code: glib provides low-level classes for file-system abstraction, date and time representation, string utility functions, logging, command line parsing etc., gnet provides network classes using the Berkley socket and Winsock APIs, gsmtp contains SMTP and message-store classes, and gpop contains POP3 classes. All four libraries are portable between POSIX-like systems (eg. Linux) and Windows.

Under Windows there is an additional library for event handling. Windows has historically built network event processing on top of the GUI event system which means that the gnet library has to be able to create GUI windows in order to process network events. The extra GUI and event classes are put into a separate library in the src/win32 directory, using the namespace GGui.

There is also a separate configuration GUI program which uses the glib library together with TrollTech's Qt.

Event model

The E-MailRelay server uses non-blocking socket i/o, with a select() event loop. This event model means that the server can handle multiple clients simultaneously from a single thread and the only significant blocking occurs when external programs are executed (see --filter and --verifier).

See C10K Problem for a discussion of different network event models.

At higher levels the C++ slot/signal design pattern is used to propagate events between objects (not to be confused with operating system signals). The slot/signal implementation has been simplified compared to Qt or boost by not supporting signal multicasting, so each signal connects to no more than one slot. For historical reasons the slot/signal pattern is not used in the lowest layers of the network library.

Event handling and exceptions

The use of non-blocking i/o in the network library means that most processing operates within the context of an i/o event or timeout callback so the top level of the call stack is nearly always the event loop code. This can make using C++ exceptions a bit awkward compared to a multi-threaded approach because it is not possible to put a single catch block around a particular high-level feature.

The event loop delivers all asynchronous events to the abstract EventHandler and AbstractTimer interfaces. If these callbacks throw exceptions then the event loop will catch them and deliver them back to the same interface through the virtual functions onException() and onTimerException() respectively. If exceptions are thrown out of _these_ callbacks then the event loop code lets them propagate back to main(), typically terminating the program.

The two callback interfaces are brought together by having a concrete Timer class that requires an EventHandler object to be associated with each timer. The Timer class routes any exceptions thrown out of the timeout callback to the designated EventHandler interface so that both i/o and timeout exceptions are delivered to the same place.

In common with other event-driven frameworks this leads to a programming model where objects are instantiated on the heap and the objects delete themselves when they receive certain events from the framework. In the gnet library the ServerPeer and HeapClient classes do this lifetime management; instances of these classes delete themselves when the associated network connection goes away or when an exception is thrown out their event-handling code.

Core class structure

The message-store functionality uses three abstract interfaces: MessageStore, NewMessage and StoredMessage. The NewMessage interface is used to create messages within the store, and the StoredMessage interface is used for reading and extracting messages from the store. The concrete implementation classes based on these interfaces are respectively FileStore, NewFile and StoredFile.

Protocol classes such as GSmtp::ServerProtocol receive network and timer events from their container and use an abstract Sender interface to send network data. This means that the protocols can be largely independent of the network and event loop framework.

The interaction between the SMTP server protocol class and the message store is mediated by the ProtocolMessage interface. Two main implementations of this interface are available: one for normal spooling (ProtocolMessageStore), and another for immediate forwarding (ProtocolMessageForward). The Decorator pattern is used whereby the forwarding class uses an instance of the storage class to do the message storing and pre-processing, while adding in an instance of the GSmtp::Client class to do the forwarding.

Message pre-processing (see --filter) is implemented via an abstract Processor interface. Concrete implementations are provided for doing nothing, running an external executable program and talking to an external network server.

The protocol, processor and message-store interfaces are brought together by the high-level GSmtp::Server and GSmtp::Client classes. Dependency injection is used to create the concrete instances of the ProtocolMessage and Processor interfaces.

Simplified class diagrams for the GNet and GSmtp namespaces are available.

Windows service

To get E-MailRelay to run as a Windows service there is a service wrapper program called emailrelay-service.exe. This program registers itself as a service when run with the --install commandline switch. When the service runs the wrapper starts the actual E-MailRelay server by looking for a batch file called emailrelay-start.bat in the same directory as service wrapper executable. It reads the contents of this batch file in order to construct the E-MailRelay command-line, adding --no-daemon and --hidden switches if they are not there already.

The service name and display name can be added to the wrapper's --install command-line, and it is the service name that is used to derive the name of the start batch file. This allows more than one server to be run as services, using different server command-line switches on each one.

Diagrams

Class diagrams:

State transition diagrams:

Sequence diagrams:

Configuration GUI

The optional configuration GUI program emailrelay-gui uses TrollTech Qt v4 for its user interface components. The GUI can run as a stand-alone configuration helper or as part of a self-extracting installation program called emailrelay-setup.

The packing scheme used to assemble a self-extracting archive is a simple concatenation of the stub executable followed by a table of contents for the payload files, followed by the payload files themselves (possibly compressed by zlib), and ending with an twelve-byte ascii representation of the offset of the table of contents.

On Windows there are two levels of packing: the setup program has a stub executable written in C that prints an extracting... message to the standard output, with a payload comprising another packed executable and a small number of C++ runtime library files. The inner packed executable has the emailrelay GUI program as its stub and all the other installable files, including the main emailrelay executable, as its payload.

When the GUI runs it checks whether it has a payload of packed files. If it has then it runs as an installer; if it does not then it runs as a configuration helper. Refer to the comments in src/gui/guimain.cpp for more details.

The code works exactly the same on Windows, Mac OS X and unix-like operating systems. However, on unix-like operating systems make install, possibly run via some package manager, is the preferred way to install files so the setup program is never normally built or distributed. On Mac OS X the default packing scheme works well enough, but there is also provision for having a separate payload file within the Mac bundle rather than appending the payload to the stub executable.

The user interface is structured as a wizard having a dialog box with the forward and back buttons at the bottom and a single Qt layout object for the main area. A stack of Qt widgets representing the various pages of the wizard are installed into the main layout object in turn as the user navigates from one page to the next.

Once the wizard is completed it asks each page to dump its state as a set of key-value pairs into a stringstream (see src/gui/pages.cpp). These key-value pairs are processed by an installer class into a list of action objects (in the Command design pattern) and then the action objects are run in turn. In order to display the progress of the installation each action object is run within a timer callback so that the Qt framework get a chance to update the GUI between each one.

During development the user interface pages and the installer can be tested separately since the interface between them is a simple text stream containing key-value pairs.

Source control

The source code is stored in the SourceForge svn repository. A working copy can be checked out as follows:

$ svn co https://emailrelay.svn.sourceforge.net/svnroot/emailrelay

Directory structure

src
Parent directory for source code.
src/glib
A low-level class library, including classes for file-system abstraction, date and time, string utility functions, logging, command line parsing etc.
src/gnet
A network library using Berkley sockets or Winsock.
src/gsmtp
An SMTP library.
src/gpop
A POP3 library.
src/win32
Additional classes for windows event processing.
src/main
Application-level classes for E-MailRelay.
src/gui
Installation and configuration GUI program using Qt v4.
lib
Parent directory for ISO C++ fixups for various compilers.
test
Test scripts and utilities.

Source file names

Generally the source file names are follow the name of the principal class, (often including the namespace) but all in lowercase. Any underscores in the name indicate a choice of implementation, so class G::Foo might have two implementations in the files gfoo_main.cpp and gfoo_alternate.cpp. The choice is normally made by the makefile.

Portability

The E-MailRelay code is written in ISO C++, although avoiding less-widely supported language features such as mutable, templated methods and export.

The header files gdef.h in src/glib, and gnet.h in src/gnet are intended to be used to fix up compiler portability issues such as missing standard types, non-standard system headers etc. Conditional compilation directives (#if etc.) are confined to these headers as far as possible in order to improve readability.

Deficiencies in the standard headers files provided by older compilers are fixed up by files in the lib directory tree. For example, the msvc6.0 compiler sometimes does not put its names into the std namespace, even though the std-namespace headers are used. This can be worked round by additional using declarations in the lib/msvc6.0 headers. These work-rounds are kept out of the src tree because they are not necessary for more modern compilers.

Windows/unix portability is generally addressed by providing a common class declaration with two implementations. Where necessary a pimple (or Bridge) pattern is used to hide the system-specific parts of the declaration.

A good example is the G::Directory class used for iterating through files in a directory. The header file src/glib/gdirectory.h is common to both systems, but two implementations are provided in gdirectory_unix.cpp and gdirectory_win32.cpp. The unix implementation uses opendir() and glob(), while the windows implementation uses FindFirstFile().

Sometimes only small parts of the implementation are system-specific. In these cases there are three source files per header. For example, gsocket.cpp, gsocket_win32.cpp and gsocket_unix.cpp in the src/gnet directory.

Compile-time features

Compile-time features can be selected with switches passed to the configure script. These include the following:

The --enable-fhs switch alters the compiled-in default directories to conform to the Linux File Hierarchy Standard (FHS). This is recommended for most modern Linux distributions.

Some functionality can be disabled at compile-time in order to reduce the size of the executable, typically when building for embedded systems:

The --enable-small-config switch can be used to change the command-line parsing code to use a configuration file instead, resulting in a smaller executable. This also removes a lot of the configuration checking code, so it is not recommended unless size is critical. (The format of the configuration file is similar to the command-line using the long-form switches without the double-dash and using '=' to separate the switch from the switch value.)

Use ./configure --help to see a complete list of options and refer to acinclude.m4 for more detailed comments.

Patterns

Gang-of-four Design Patterns (ISBN 0-201-63361-2):

Lakos' Large Scale C++ Software Design patterns (ISBN 0-201-63362-0):

Meyer's More Effective C++ patterns (ISBN 0-201-63371-X):

Other patterns:

Idioms

The <<= operator defined in src/glib/gmemory.h is used idiomatically to reassign a std::auto_ptr<> since reset() is not always available.

Copyright (C) 2001-2008 Graeme Walker <graeme_walker@users.sourceforge.net>. All rights reserved.