Perhaps the most awkward thing about the SML module system is that circular dependencies between modules are not allowed. Declarations have to be carefully partitioned between modules to avoid circular dependencies. For example this often means that type declarations have to be factored out to a separate module. This is because you can only have mutually recursive types if they are datatypes in the same group of declarations (i.e. joined with the and keyword) in the same module. So you end up with a tree of modules to manage.
To help manage the modules I divide them into layers with each layer in its own directory. The module dependencies between layers always point from the top down. Table 9-1 shows the layers of the server and the files in each layer. The layers are shown from the highest to the lowest. The files in each layer are listed in alphabetical order. I usually only place one module in each file.
Table 9-1. The Module Layers of the Server.
Layer | File (Module) | Purpose |
---|---|---|
main | main.sml (Main) | This contains the main function for the server. |
startup.sml (Startup) | This contains the code for creating the lock and pid files, setting the uid/gid and reversing this on shutdown. | |
server | connect.sml (Connect) | This implements a type for a connection along with I/O and utility operations on the connection. |
http10.sml (HTTP_1_0) | This implements the HTTP1.0 protocol over a connection and communicates with the resource store. | |
listener.sml (Listener) | This creates the listener socket and accepts new connections. | |
store | builtin_node.sml (SimpleBuiltinHandler) | This implements some built-in node handlers. |
cgibin_node.sml (CgiNodeHandler) | This implements the CGI node handler. | |
dir_node.sml (DirNodeHandler) | This implements nodes that map to files and directories. | |
gen_node.sml (GenericNodeFn) | This implements backbone nodes that route requests to handlers. It provides a functor that is specialise by each kind of handler. | |
node_auth.sml (NodeAuth) | This implements authorisation checking functions. | |
node_factory.sml (NodeFactory) | This implements a function to create resource nodes of different kinds. | |
node_handler_sig.sml (NODE_HANDLER) | This defines the signature that a node handler must have. This is the configuration interface between backbone and handler nodes. | |
node.sml (Node) | This defines common types for resource nodes. This includes the message protocol between backbone nodes and handlers. Some utility functions are included. | |
resp_utils.sml (ResponseUtils) | This implements a few utility functions for creating HTTP responses. | |
store.sml (Store) | This is the entry point for the resource store. It builds the node tree and accepts request from the HTTP protocol manager. | |
ietf | entity.sml (Entity) | This defines a type for a HTTP entity. The entity producer and consumer types and protocol is also included. All of the kinds of producers are implemented in here. |
http_header.sml (HTTPHeader) | This runs the parsing of HTTP headers. A type is defined to represent all of the well-known headers in a fully-parsed format. | |
http_msg.sml (HTTPMsg) | This defines the types for a HTTP request and response. | |
http_status.sml (HTTPStatus) | This defines the type for a HTTP status code. | |
ietf.lex (IETFLex) | This implements a ML-Lex lexer to help parse HTTP headers. It performs all of the regular-expression operations. | |
ietf_line.sml (IETF_Line) | This has more functions for parsing HTTP headers. It uses the lexer to split a header line into tokens according the the HTTP specification. | |
ietf_part.sml (IETF_Part) | This defines the type for the tokens returned by the functions in ietf_line.sml. | |
ietf_utils.sml (IETF_Utils) | This implements a few utility functions. Currently it only has base64 encoding and decoding. | |
config | config.grm | This is a ML-Yacc grammar for the server's configuration file. |
config.lex | This is a ML-Lex lexer to match the grammar. | |
config.sml (Config) | This implements the configuration file parsing. It stores the configuration parameters as global values. Some parameters are pushed down into the common modules to avoid circular dependencies (see init_globals). | |
config_types.sml (ConfigTypes) | This defines types for the parse tree produced by the grammar. | |
common | abort.sml (Abort) | This defines a type to represent abort conditions on connections. |
common.sml (Common) | This defines common types and functions that are used all over the place. | |
file_io.sml (FileIO) | This implements functions that generally operate on the contents of files, especially wrappers around OS.FileSys functions. | |
files.sml (Files) | This implements functions that manage file paths and provide information about files. | |
finalise.sml (FinaliseFn) | This implements a functor to provides finalisation of values after garbage collection. | |
logging.sml (Log) | This implement the Logging Manager. | |
mutex.sml (Mutex) | This implements mutexes for atomic operations on static variables. | |
open_mgr.sml | This implements the modules of the Open Manager. This include the open file counter (OpenCounter) and the managers specialised to each kind of openable object such as files and processes. | |
profile.sml (MyProfile) | This implements some profiling utilities. | |
signals.sml (SignalMgr) | This implements signal catchers. It also catches the pseudo-signal from the garbage collector. The signals are distributed to the rest of the server using multicasted messages. | |
singleton.sml (Singleton) | This implements a functor that provides the common pattern of a singleton concurrent object that is started on demand. | |
text.sml (TextFrag) | This defines a type for a fragment of text. Fragments can be combined cheaply without copying the text. | |
tmpfile.sml (TmpFile) | This implements the management of temporary files including limiting the disk space used and cleaning up when a connection closes. | |
url.sml (URL) | This implements a type for representing URLs. |
The main dependencies between the modules of the top three levels are shown in Figure 9-1. Only the main interactions are shown, enough to place the modules into a hierarchy. The modules on the right hand side are those in the ietf layer. There would be too many connections joining the store and ietf layers to draw in. I've omitted the config layer as it is fairly simple. The common layer is mostly flat.
The following detailed discussions will be organised by layer from the top down. I'm going to describe the modules in a logical order that roughly traces the flow of control for a HTTP request through the server. This will also mesh with the top-down description of the layers since the functions are calling from the top down through the layers.
This ordering of the discussions means there will be many forward references to modules that will be discussed later, especially those in the the config and common layers. You may want to jump ahead and scan the modules of the common layer first. Especially have a look at the TextFrag and Log modules in the section called The Common Layer.
Throughout the code I've used some common abbreviations for module names.
structure Cfg = Config structure Hdr = HTTPHeader structure Req = HTTPMsg structure S = Socket structure SS = Substring structure Status = HTTPStatus structure TF = TextFrag structure U = ResponseUtils |
The Common module is always opened for direct access.
Each directory contains a sources.cm CM file to control the compilation of the server. Here is the CM file for the main directory. (See the section called Assembling the Hello World Program in Chapter 2 for more on CM files).
group is /src/smlnj/current/lib/cml.cm /src/smlnj/current/lib/cml-lib.cm ../common/sources.cm ../config/sources.cm ../server/sources.cm main.sml startup.sml |
This CM file includes all other CM files directly or indirectly so you only need to compile in the main directory to compile all of the program.
Note that all uses of sml-lib (the SML libary) must be replaced with cml-lib in all .cm files in all directories. You cannot mix sml-lib and cml-lib in the same program.
To build the entire program just do the following or use the Makefile in the top directory.
$ cd main $ sml Standard ML of New Jersey, Version 110.0.7, ... - CM.make(); ........ write 5,0: 57 big objects (544 pages) @ 0x1064a0 $ |