胡斌 / srs

转到一个项目
Toggle navigation Toggle navigation pinning
继续操作前请注册或者登录。
winlin
7692373e 1 parent 7f4c113e

remove the unused files. add upp project file

内嵌 并排对比
正在显示 19 个修改的文件 包含 0 行增加 和 4795 行删除

要显示太多修改。
重新载入完整差异 差异文件 邮件补丁

为保证性能只显示 19 of 19+ 个文件。

WELCOME!
The State Threads Library is a small application library which provides
a foundation for writing fast and highly scalable Internet applications
(such as web servers, proxy servers, mail transfer agents, and so on,
really any network-data-driven application) on UNIX-like platforms. It
combines the simplicity of the multithreaded programming paradigm, in
which one thread supports each simultaneous connection, with the
performance and scalability of an event-driven state machine
architecture. In other words, this library offers a threading API for
structuring an Internet application as a state machine. For more
details, please see the library documentation in the "docs" directory or
on-line at
http://state-threads.sourceforge.net/docs/
The State Threads Project is an open source project for maintaining and
enhancing the State Threads Library. For more information about this
project, please see
http://state-threads.sourceforge.net/
BUILDING
To build the library by hand, use the GNU make utility. Run the make
command (e.g., `gmake') with no arguments to display all supported
targets.
To build more or less automatically, first set the CONFIG_GUESS_PATH
variable in either osguess.sh or your environment then run "make
default" which guesses your OS and builds. Requires the "config.guess"
utility from GNU autoconf (not included with ST). You can use one from
a larger "main" software project or just use any config.guess available
on your system. You can also get it directly from GNU:
ftp://ftp.gnu.org/gnu/autoconf/
To build rpms (RedHat Linux 6.2 or later, Linux/Mandrake, Solaris with
gnome, etc.):
download the latest st-x.y.tar.gz
# rpm -ta st-x.y.tar.gz
The .rpms will land in /usr/src/RPMS/<arch>. Install them with:
# rpm -i libst*.rpm
Requires GNU automake and rpm 3.0.3 or later.
Debian users:
If you run potato, please upgrade to woody.
If you run woody, "apt-get install libst-dev" will get you v1.3.
If you run testing/unstable, you will get the newest available version.
If you *must* have the newest libst in woody, you may follow these
not-recommended instructions:
1. Add "deb-src <your-favourite-debian-mirror> unstable main" to your
/etc/apt/sources.list
2. apt-get update
3. apt-get source st
4. cd st-1.4 (or whatever version you got)
5. debuild
6. dpkg -i ../*.deb
If your application uses autoconf to search for dependencies and you
want to search for a given version of libst, you can simply add
PKG_CHECK_MODULES(MYAPP, st >= 1.3 mumble >= 0.2.23)
to your configure.ac/in. This will define @MYAPP_LIBS@ and
@MYAPP_CFLAGS@ which you may then use in your Makefile.am/in files to
link against mumble and st.
LICENSE
The State Threads library is a derivative of the Netscape Portable
Runtime library (NSPR). All source code in this directory is
distributed under the terms of the Mozilla Public License (MPL) version
1.1 or the GNU General Public License (GPL) version 2 or later. For
more information about these licenses please see
http://www.mozilla.org/MPL/ and http://www.gnu.org/copyleft/.
All source code in the "examples" directory is distributed under the BSD
style license.
PLATFORMS
Please see the "docs/notes.html" file for the list of currently
supported platforms.
DEBUGGER SUPPORT
It's almost impossible to print SP and PC in a portable way. The only
way to see thread's stack platform-independently is to actually jump to
the saved context. That's what the _st_iterate_threads() function does.
Do the following to iterate over all threads:
- set the _st_iterate_threads_flag to 1 in debugger
- set breakpoint at the _st_show_thread_stack() function
(which does nothing)
- call the _st_iterate_threads() function which jumps to the
next thread
- at each break you can explore thread's stack
- continue
- when iteration is complete, you return to the original
point (you can see thread id and a message as arguments of
the _st_show_thread_stack() function).
You can call _st_iterate_threads() in three ways:
- Insert it into your source code at the point you want to
go over threads.
- Just run application and this function will be called at
the first context switch.
- Call it directly from the debugger at any point.
This works with gdb and dbx.
Example using gdb:
(gdb) set _st_iterate_threads_flag = 1
(gdb) b _st_show_thread_stack
...
(gdb) call _st_iterate_threads()
...
(gdb) bt
...
(gdb) c
...
(gdb) bt
...
(gdb) c
...
and so on...
_st_iterate_threads_flag will be set to 0 automatically
after iteration is over or you can set it to 0 at any time
to stop iteration.
Sometimes gdb complains about SIGSEGV when you call a function
directly at gdb command-line. It can be ignored -- just call the
same function right away again, it works just fine. For example:
(gdb) set _st_iterate_threads_flag = 1
(gdb) b _st_show_thread_stack
Breakpoint 1 at 0x809bbbb: file sched.c, line 856.
(gdb) call _st_iterate_threads()
Program received signal SIGSEGV, Segmentation fault.
....
(gdb) # just call the function again:
(gdb) call _st_iterate_threads()
Breakpoint 1, _st_show_thread_stack (thread=0x4017aee4, messg=0x80ae7a2
"Iteration started") at sched.c:856
856 }
....
You can use simple gdb command-line scripting to display
all threads and their stack traces at once:
(gdb) while _st_iterate_threads_flag
>bt
>c
>end
....
Another script to stop at the thread with the specific thread id
(e.g., 0x40252ee4):
(gdb) # set the flag again:
(gdb) set _st_iterate_threads_flag = 1
(gdb) call _st_iterate_threads()
Breakpoint 1, _st_show_thread_stack (thread=0x4017aee4, messg=0x80ae7a2
"Iteration started") at sched.c:856
856 }
....
(gdb) while thread != 0x40252ee4
>c
>end
....
....
Breakpoint 1, _st_show_thread_stack (thread=0x40252ee4, messg=0x0) at
sched.c:856
856 }
(gdb) bt
....
(gdb) # don't want to continue iteration, unset the flag:
(gdb) set _st_iterate_threads_flag = 0
(gdb) c
Continuing.
Breakpoint 1, _st_show_thread_stack (thread=0x0, messg=0x80ae78e "Iteration
completed")
at sched.c:856
856 }
(gdb) c
Continuing.
(gdb) return
Make selected stack frame return now? (y or n) y
#0 0x4011254e in __select ()
from /lib/libc.so.6
(gdb) detach
CHANGE LOG
Changes from 1.8 to 1.9.
------------------------
o Support 32-bit and 64-bit Intel Macs.
o Added ST_VERSION string, and ST_VERSION_MAJOR and ST_VERSION_MINOR
[bug 1796801].
o Fixed some compiler warnings, based on a patch from Brian Wellington
[bug 1932741].
Changes from 1.7 to 1.8.
--------------------------
o Added support for kqueue and epoll on platforms that support them.
Added ability to choose the event notification system at program
startup.
o Long-overdue public definitions of ST_UTIME_NO_TIMEOUT (-1ULL) and
ST_UTIME_NO_WAIT (0) [bug 1514436].
o Documentation patch for st_utime() [bug 1514484].
o Documentation patch for st_timecache_set() [bug 1514486].
o Documentation patch for st_netfd_serialize_accept() [bug 1514494].
o Added st_writev_resid() [rfe 1538344].
o Added st_readv_resid() [rfe 1538768] and, for symmetry, st_readv().
Changes from 1.6 to 1.7.
------------------------
o Support glibc 2.4, which breaks programs that manipulate jump buffers.
Replaced Linux IA64 special cases with new md.S that covers all
Linux.
Changes from 1.5.2 to 1.6.
--------------------------
none
Changes from 1.5.1 to 1.5.2.
----------------------------
o Alfred Perlstein's context switch callback feature.
o Claus Assmann's st_recvmsg/st_sendmsg wrappers.
o Extra stack padding for platforms that need it.
o Ron Arts's timeout clarifications in the reference manual.
o Raymond Bero and Anton Berezin's AMD64 FreeBSD port.
o Claus Assmann's AMD64 SunOS 5.10 port.
o Claus Assmann's AMD64 OpenBSD port.
o Michael Abd-El-Malek's Mac OS X port.
o Michael Abd-El-Malek's stack printing patch.
Changes from 1.5.0 to 1.5.1.
----------------------------
o Andreas Gustafsson's USE_POLL fix.
o Gene's st_set_utime_function() enhancement.
Changes from 1.4 to 1.5.0.
--------------------------
o Andreas Gustafsson's performance patch.
o New extensions: Improved DNS resolver, generic LRU cache, in-process
DNS cache, and a program to test the resolver and cache.
o Support for AMD Opteron 64-bit CPUs under Linux.
o Support for SPARC-64 under Solaris.
o Andreas Gustafsson's support for VAX under NetBSD.
o Changed unportable #warning directives in md.h to #error.
Changes from 1.3 to 1.4.
------------------------
o Andreas Gustafsson's NetBSD port.
o Wesley W. Terpstra's Darwin (MacOS X) port.
o Support for many CPU architectures under Linux and *BSD.
o Renamed private typedefs so they don't conflict with public ones any
more.
o common.h now includes public.h for strict prototyping.
o Joshua Levy's recommendation to make st_connect() and st_sendto()
accept const struct sockaddr pointers, as the originals do.
o Clarified the documentation regarding blocking vs. non-blocking I/O.
o Cygwin support.
o Created the extensions directory.
o Fixed warnings from ia64asm.S.
Changes from 1.2 to 1.3.
------------------------
o Added st_read_resid() and st_write_resid() to allow the caller to know
how much data was transferred before an error occurred. Updated
documentation.
o Updated project link, copyrights, and documentation regarding
timeouts. Added comment to st_connect().
o Optimized the _st_add_sleep_q() function in sched.c. Now we walk the
sleep queue *backward* when inserting a thread into it. When you
have lots (hundreds) of threads and several timeout values, it takes
a while to insert a thread at the appropriate point in the sleep
queue. The idea is that often this appropriate point is closer to
the end of the queue rather than the beginning. Measurements show
performance improves with this change. In any case this change
should do no harm.
o Added a hint of when to define USE_POLL and when not to, to the
Makefile.
o Added debugging support (files common.h and sched.c). See above.
o Decreased the number of reallocations of _ST_POLLFDS in sched.c.
Inspired by Lev Walkin.
o Fixed st_usleep(-1) and st_sleep(-1), and added a warning to the
documentation about too-large timeouts.
o Linux/*BSD Alpha port.
o Wesley W. Terpstra modernized the build process:
- properly build relocatable libraries under bsd and linux
- use library versioning
- added rpm spec file
- added debian/ files
See above for build instructions.
Changes from 1.1 to 1.2.
------------------------
o Added st_randomize_stacks().
o Added a patch contributed by Sascha Schumann.
Changes from 1.0 to 1.1.
------------------------
o Relicensed under dual MPL-GPL.
o OpenBSD port.
o Compile-time option to use poll() instead of select() for
event polling (see Makefile).
This is useful if you want to support a large number of open
file descriptors (larger than FD_SETSIZE) within a single
process.
o Linux IA-64 port.
Two issues make IA-64 different from other platforms:
- Besides the traditional call stack in memory, IA-64 uses the
general register stack. Thus each thread needs a backing store
for the register stack in addition to the memory stack.
- Current implementation of setjmp()/longjmp() can not be used
for thread context-switching since it assumes that only one
register stack exists. Using special assembly functions for
context-switching is unavoidable.
o Thread stack capping on IRIX.
This allows some profiling tools (such as SpeedShop) to know when
to stop unwinding the stack. Without this libexc, used by SpeedShop,
traces right off the stack and crashes.
o Miscellaneous documentation additions.
COPYRIGHTS
Portions created by SGI are Copyright (C) 2000 Silicon Graphics, Inc.
All Rights Reserved.
<HTML>
<HEAD>
<TITLE>State Threads Library Programming Notes</TITLE>
</HEAD>
<BODY BGCOLOR=#FFFFFF>
<H2>Programming Notes</H2>
<P>
<B>
<UL>
<LI><A HREF=#porting>Porting</A></LI>
<LI><A HREF=#signals>Signals</A></LI>
<LI><A HREF=#intra>Intra-Process Synchronization</A></LI>
<LI><A HREF=#inter>Inter-Process Synchronization</A></LI>
<LI><A HREF=#nonnet>Non-Network I/O</A></LI>
<LI><A HREF=#timeouts>Timeouts</A></LI>
</UL>
</B>
<P>
<HR>
<P>
<A NAME="porting">
<H3>Porting</H3>
The State Threads library uses OS concepts that are available in some
form on most UNIX platforms, making the library very portable across
many flavors of UNIX. However, there are several parts of the library
that rely on platform-specific features. Here is the list of such parts:
<P>
<UL>
<LI><I>Thread context initialization</I>: Two ingredients of the
<TT>jmp_buf</TT>
data structure (the program counter and the stack pointer) have to be
manually set in the thread creation routine. The <TT>jmp_buf</TT> data
structure is defined in the <TT>setjmp.h</TT> header file and differs from
platform to platform. Usually the program counter is a structure member
with <TT>PC</TT> in the name and the stack pointer is a structure member
with <TT>SP</TT> in the name. One can also look in the
<A HREF="http://www.mozilla.org/source.html">Netscape's NSPR library source</A>
which already has this code for many UNIX-like platforms
(<TT>mozilla/nsprpub/pr/include/md/*.h</TT> files).
<P>
Note that on some BSD-derived platforms <TT>_setjmp(3)/_longjmp(3)</TT>
calls should be used instead of <TT>setjmp(3)/longjmp(3)</TT> (that is
the calls that manipulate only the stack and registers and do <I>not</I>
save and restore the process's signal mask).</LI>
<P>
Starting with glibc 2.4 on Linux the opacity of the <TT>jmp_buf</TT> data
structure is enforced by <TT>setjmp(3)/longjmp(3)</TT> so the
<TT>jmp_buf</TT> ingredients cannot be accessed directly anymore (unless
special environmental variable LD_POINTER_GUARD is set before application
execution). To avoid dependency on custom environment, the State Threads
library provides <TT>setjmp/longjmp</TT> replacement functions for
all Intel CPU architectures. Other CPU architectures can also be easily
supported (the <TT>setjmp/longjmp</TT> source code is widely available for
many CPU architectures).
<P>
<LI><I>High resolution time function</I>: Some platforms (IRIX, Solaris)
provide a high resolution time function based on the free running hardware
counter. This function returns the time counted since some arbitrary
moment in the past (usually machine power up time). It is not correlated in
any way to the time of day, and thus is not subject to resetting,
drifting, etc. This type of time is ideal for tasks where cheap, accurate
interval timing is required. If such a function is not available on a
particular platform, the <TT>gettimeofday(3)</TT> function can be used
(though on some platforms it involves a system call).
<P>
<LI><I>The stack growth direction</I>: The library needs to know whether the
stack grows toward lower (down) or higher (up) memory addresses.
One can write a simple test program that detects the stack growth direction
on a particular platform.</LI>
<P>
<LI><I>Non-blocking attribute inheritance</I>: On some platforms (e.g. IRIX)
the socket created as a result of the <TT>accept(2)</TT> call inherits the
non-blocking attribute of the listening socket. One needs to consult the manual
pages or write a simple test program to see if this applies to a specific
platform.</LI>
<P>
<LI><I>Anonymous memory mapping</I>: The library allocates memory segments
for thread stacks by doing anonymous memory mapping (<TT>mmap(2)</TT>). This
mapping is somewhat different on SVR4 and BSD4.3 derived platforms.
<P>
The memory mapping can be avoided altogether by using <TT>malloc(3)</TT> for
stack allocation. In this case the <TT>MALLOC_STACK</TT> macro should be
defined.</LI>
</UL>
<P>
All machine-dependent feature test macros should be defined in the
<TT>md.h</TT> header file. The assembly code for <TT>setjmp/longjmp</TT>
replacement functions for all CPU architectures should be placed in
the <TT>md.S</TT> file.
<P>
The current version of the library is ported to:
<UL>
<LI>IRIX 6.x (both 32 and 64 bit)</LI>
<LI>Linux (kernel 2.x and glibc 2.x) on x86, Alpha, MIPS and MIPSEL,
SPARC, ARM, PowerPC, 68k, HPPA, S390, IA-64, and Opteron (AMD-64)</LI>
<LI>Solaris 2.x (SunOS 5.x) on x86, AMD64, SPARC, and SPARC-64</LI>
<LI>AIX 4.x</LI>
<LI>HP-UX 11 (both 32 and 64 bit)</LI>
<LI>Tru64/OSF1</LI>
<LI>FreeBSD on x86, AMD64, and Alpha</LI>
<LI>OpenBSD on x86, AMD64, Alpha, and SPARC</LI>
<LI>NetBSD on x86, Alpha, SPARC, and VAX</LI>
<LI>MacOS X (Darwin) on PowerPC (32 bit) and Intel (both 32 and 64 bit) [universal]</LI>
<LI>Cygwin</LI>
</UL>
<P>
<A NAME="signals">
<H3>Signals</H3>
Signal handling in an application using State Threads should be treated the
same way as in a classical UNIX process application. There is no such
thing as per-thread signal mask, all threads share the same signal handlers,
and only asynchronous-safe functions can be used in signal handlers.
However, there is a way to process signals synchronously by converting a
signal event to an I/O event: a signal catching function does a write to
a pipe which will be processed synchronously by a dedicated signal handling
thread. The following code demonstrates this technique (error handling is
omitted for clarity):
<PRE>
/* Per-process pipe which is used as a signal queue. */
/* Up to PIPE_BUF/sizeof(int) signals can be queued up. */
int sig_pipe[2];
/* Signal catching function. */
/* Converts signal event to I/O event. */
void sig_catcher(int signo)
{
int err;
/* Save errno to restore it after the write() */
err = errno;
/* write() is reentrant/async-safe */
write(sig_pipe[1], &signo, sizeof(int));
errno = err;
}
/* Signal processing function. */
/* This is the "main" function of the signal processing thread. */
void *sig_process(void *arg)
{
st_netfd_t nfd;
int signo;
nfd = st_netfd_open(sig_pipe[0]);
for ( ; ; ) {
/* Read the next signal from the pipe */
st_read(nfd, &signo, sizeof(int), ST_UTIME_NO_TIMEOUT);
/* Process signal synchronously */
switch (signo) {
case SIGHUP:
/* do something here - reread config files, etc. */
break;
case SIGTERM:
/* do something here - cleanup, etc. */
break;
/* .
.
Other signals
.
.
*/
}
}
return NULL;
}
int main(int argc, char *argv[])
{
struct sigaction sa;
.
.
.
/* Create signal pipe */
pipe(sig_pipe);
/* Create signal processing thread */
st_thread_create(sig_process, NULL, 0, 0);
/* Install sig_catcher() as a signal handler */
sa.sa_handler = sig_catcher;
sigemptyset(&sa.sa_mask);
sa.sa_flags = 0;
sigaction(SIGHUP, &sa, NULL);
sa.sa_handler = sig_catcher;
sigemptyset(&sa.sa_mask);
sa.sa_flags = 0;
sigaction(SIGTERM, &sa, NULL);
.
.
.
}
</PRE>
<P>
Note that if multiple processes are used (see below), the signal pipe should
be initialized after the <TT>fork(2)</TT> call so that each process has its
own private pipe.
<P>
<A NAME="intra">
<H3>Intra-Process Synchronization</H3>
Due to the event-driven nature of the library scheduler, the thread context
switch (process state change) can only happen in a well-known set of
library functions. This set includes functions in which a thread may
"block":<TT> </TT>I/O functions (<TT>st_read(), st_write(), </TT>etc.),
sleep functions (<TT>st_sleep(), </TT>etc.), and thread synchronization
functions (<TT>st_thread_join(), st_cond_wait(), </TT>etc.). As a result,
process-specific global data need not to be protected by locks since a thread
cannot be rescheduled while in a critical section (and only one thread at a
time can access the same memory location). By the same token,
non thread-safe functions (in a traditional sense) can be safely used with
the State Threads. The library's mutex facilities are practically useless
for a correctly written application (no blocking functions in critical
section) and are provided mostly for completeness. This absence of locking
greatly simplifies an application design and provides a foundation for
scalability.
<P>
<A NAME="inter">
<H3>Inter-Process Synchronization</H3>
The State Threads library makes it possible to multiplex a large number
of simultaneous connections onto a much smaller number of separate
processes, where each process uses a many-to-one user-level threading
implementation (<B>N</B> of <B>M:1</B> mappings rather than one <B>M:N</B>
mapping used in native threading libraries on some platforms). This design
is key to the application's scalability. One can think about it as if a
set of all threads is partitioned into separate groups (processes) where
each group has a separate pool of resources (virtual address space, file
descriptors, etc.). An application designer has full control of how many
groups (processes) an application creates and what resources, if any,
are shared among different groups via standard UNIX inter-process
communication (IPC) facilities.<P>
There are several reasons for creating multiple processes:
<P>
<UL>
<LI>To take advantage of multiple hardware entities (CPUs, disks, etc.)
available in the system (hardware parallelism).</LI>
<P>
<LI>To reduce risk of losing a large number of user connections when one of
the processes crashes. For example, if <B>C</B> user connections (threads)
are multiplexed onto <B>P</B> processes and one of the processes crashes,
only a fraction (<B>C/P</B>) of all connections will be lost.</LI>
<P>
<LI>To overcome per-process resource limitations imposed by the OS. For
example, if <TT>select(2)</TT> is used for event polling, the number of
simultaneous connections (threads) per process is
limited by the <TT>FD_SETSIZE</TT> parameter (see <TT>select(2)</TT>).
If <TT>FD_SETSIZE</TT> is equal to 1024 and each connection needs one file
descriptor, then an application should create 10 processes to support 10,000
simultaneous connections.</LI>
</UL>
<P>
Ideally all user sessions are completely independent, so there is no need for
inter-process communication. It is always better to have several separate
smaller process-specific resources (e.g., data caches) than to have one large
resource shared (and modified) by all processes. Sometimes, however, there
is a need to share a common resource among different processes. In that case,
standard UNIX IPC facilities can be used. In addition to that, there is a way
to synchronize different processes so that only the thread accessing the
shared resource will be suspended (but not the entire process) if that resource
is unavailable. In the following code fragment a pipe is used as a counting
semaphore for inter-process synchronization:
<PRE>
#ifndef PIPE_BUF
#define PIPE_BUF 512 /* POSIX */
#endif
/* Semaphore data structure */
typedef struct ipc_sem {
st_netfd_t rdfd; /* read descriptor */
st_netfd_t wrfd; /* write descriptor */
} ipc_sem_t;
/* Create and initialize the semaphore. Should be called before fork(2). */
/* 'value' must be less than PIPE_BUF. */
/* If 'value' is 1, the semaphore works as mutex. */
ipc_sem_t *ipc_sem_create(int value)
{
ipc_sem_t *sem;
int p[2];
char b[PIPE_BUF];
/* Error checking is omitted for clarity */
sem = malloc(sizeof(ipc_sem_t));
/* Create the pipe */
pipe(p);
sem->rdfd = st_netfd_open(p[0]);
sem->wrfd = st_netfd_open(p[1]);
/* Initialize the semaphore: put 'value' bytes into the pipe */
write(p[1], b, value);
return sem;
}
/* Try to decrement the "value" of the semaphore. */
/* If "value" is 0, the calling thread blocks on the semaphore. */
int ipc_sem_wait(ipc_sem_t *sem)
{
char c;
/* Read one byte from the pipe */
if (st_read(sem->rdfd, &c, 1, ST_UTIME_NO_TIMEOUT) != 1)
return -1;
return 0;
}
/* Increment the "value" of the semaphore. */
int ipc_sem_post(ipc_sem_t *sem)
{
char c;
if (st_write(sem->wrfd, &c, 1, ST_UTIME_NO_TIMEOUT) != 1)
return -1;
return 0;
}
</PRE>
<P>
Generally, the following steps should be followed when writing an application
using the State Threads library:
<P>
<OL>
<LI>Initialize the library (<TT>st_init()</TT>).</LI>
<P>
<LI>Create resources that will be shared among different processes:
create and bind listening sockets, create shared memory segments, IPC
channels, synchronization primitives, etc.</LI>
<P>
<LI>Create several processes (<TT>fork(2)</TT>). The parent process should
either exit or become a "watchdog" (e.g., it starts a new process when
an existing one crashes, does a cleanup upon application termination,
etc.).</LI>
<P>
<LI>In each child process create a pool of threads
(<TT>st_thread_create()</TT>) to handle user connections.</LI>
</OL>
<P>
<A NAME="nonnet">
<H3>Non-Network I/O</H3>
The State Threads architecture uses non-blocking I/O on
<TT>st_netfd_t</TT> objects for concurrent processing of multiple user
connections. This architecture has a drawback: the entire process and
all its threads may block for the duration of a <I>disk</I> or other
non-network I/O operation, whether through State Threads I/O functions,
direct system calls, or standard I/O functions. (This is applicable
mostly to disk <I>reads</I>; disk <I>writes</I> are usually performed
asynchronously -- data goes to the buffer cache to be written to disk
later.) Fortunately, disk I/O (unlike network I/O) usually takes a
finite and predictable amount of time, but this may not be true for
special devices or user input devices (including stdin). Nevertheless,
such I/O reduces throughput of the system and increases response times.
There are several ways to design an application to overcome this
drawback:
<P>
<UL>
<LI>Create several identical main processes as described above (symmetric
architecture). This will improve CPU utilization and thus improve the
overall throughput of the system.</LI>
<P>
<LI>Create multiple "helper" processes in addition to the main process that
will handle blocking I/O operations (asymmetric architecture).
This approach was suggested for Web servers in a
<A HREF="http://www.cs.rice.edu/~vivek/flash99/">paper</A> by Peter
Druschel et al. In this architecture the main process communicates with
a helper process via an IPC channel (<TT>pipe(2), socketpair(2)</TT>).
The main process instructs a helper to perform the potentially blocking
operation. Once the operation completes, the helper returns a
notification via IPC.
</UL>
<P>
<A NAME="timeouts">
<H3>Timeouts</H3>
The <TT>timeout</TT> parameter to <TT>st_cond_timedwait()</TT> and the
I/O functions, and the arguments to <TT>st_sleep()</TT> and
<TT>st_usleep()</TT> specify a maximum time to wait <I>since the last
context switch</I> not since the beginning of the function call.
<P>The State Threads' time resolution is actually the time interval
between context switches. That time interval may be large in some
situations, for example, when a single thread does a lot of work
continuously. Note that a steady, uninterrupted stream of network I/O
qualifies for this description; a context switch occurs only when a
thread blocks.
<P>If a specified I/O timeout is less than the time interval between
context switches the function may return with a timeout error before
that amount of time has elapsed since the beginning of the function
call. For example, if eight milliseconds have passed since the last
context switch and an I/O function with a timeout of 10 milliseconds
blocks, causing a switch, the call may return with a timeout error as
little as two milliseconds after it was called. (On Linux,
<TT>select()</TT>'s timeout is an <I>upper</I> bound on the amount of
time elapsed before select returns.) Similarly, if 12 ms have passed
already, the function may return immediately.
<P>In almost all cases I/O timeouts should be used only for detecting a
broken network connection or for preventing a peer from holding an idle
connection for too long. Therefore for most applications realistic I/O
timeouts should be on the order of seconds. Furthermore, there's
probably no point in retrying operations that time out. Rather than
retrying simply use a larger timeout in the first place.
<P>The largest valid timeout value is platform-dependent and may be
significantly less than <TT>INT_MAX</TT> seconds for <TT>select()</TT>
or <TT>INT_MAX</TT> milliseconds for <TT>poll()</TT>. Generally, you
should not use timeouts exceeding several hours. Use
<tt>ST_UTIME_NO_TIMEOUT</tt> (<tt>-1</tt>) as a special value to
indicate infinite timeout or indefinite sleep. Use
<tt>ST_UTIME_NO_WAIT</tt> (<tt>0</tt>) to indicate no waiting at all.
<P>
<HR>
<P>
</BODY>
</HTML>
<HTML>
<HEAD>
<TITLE>State Threads for Internet Applications</TITLE>
</HEAD>
<BODY BGCOLOR=#FFFFFF>
<H2>State Threads for Internet Applications</H2>
<H3>Introduction</H3>
<P>
State Threads is an application library which provides a
foundation for writing fast and highly scalable Internet Applications
on UNIX-like platforms. It combines the simplicity of the multithreaded
programming paradigm, in which one thread supports each simultaneous
connection, with the performance and scalability of an event-driven
state machine architecture.</P>
<H3>1. Definitions</H3>
<P>
<A NAME="IA">
<H4>1.1 Internet Applications</H4>
</A>
<P>
An <I>Internet Application</I> (IA) is either a server or client network
application that accepts connections from clients and may or may not
connect to servers. In an IA the arrival or departure of network data
often controls processing (that is, IA is a <I>data-driven</I> application).
For each connection, an IA does some finite amount of work
involving data exchange with its peer, where its peer may be either
a client or a server.
The typical transaction steps of an IA are to accept a connection,
read a request, do some finite and predictable amount of work to
process the request, then write a response to the peer that sent the
request. One example of an IA is a Web server;
the most general example of an IA is a proxy server, because it both
accepts connections from clients and connects to other servers.</P>
<P>
We assume that the performance of an IA is constrained by available CPU
cycles rather than network bandwidth or disk I/O (that is, CPU
is a bottleneck resource).
<P>
<A NAME="PS">
<H4>1.2 Performance and Scalability</H4>
</A>
<P>
The <I>performance</I> of an IA is usually evaluated as its
throughput measured in transactions per second or bytes per second (one
can be converted to the other, given the average transaction size). There are
several benchmarks that can be used to measure throughput of Web serving
applications for specific workloads (such as
<A HREF="http://www.spec.org/osg/web96/">SPECweb96</A>,
<A HREF="http://www.mindcraft.com/webstone/">WebStone</A>,
<A HREF="http://www.zdnet.com/zdbop/webbench/">WebBench</A>).
Although there is no common definition for <I>scalability</I>, in general it
expresses the ability of an application to sustain its performance when some
external condition changes. For IAs this external condition is either the
number of clients (also known as "users," "simultaneous connections," or "load
generators") or the underlying hardware system size (number of CPUs, memory
size, and so on). Thus there are two types of scalability: <I>load
scalability</I> and <I>system scalability</I>, respectively.
<P>
The figure below shows how the throughput of an idealized IA changes with
the increasing number of clients (solid blue line). Initially the throughput
grows linearly (the slope represents the maximal throughput that one client
can provide). Within this initial range, the IA is underutilized and CPUs are
partially idle. Further increase in the number of clients leads to a system
saturation, and the throughput gradually stops growing as all CPUs become fully
utilized. After that point, the throughput stays flat because there are no
more CPU cycles available.
In the real world, however, each simultaneous connection
consumes some computational and memory resources, even when idle, and this
overhead grows with the number of clients. Therefore, the throughput of the
real world IA starts dropping after some point (dashed blue line in the figure
below). The rate at which the throughput drops depends, among other things, on
application design.
<P>
We say that an application has a good <I>load scalability</I> if it can
sustain its throughput over a wide range of loads.
Interestingly, the <A HREF="http://www.spec.org/osg/web99/">SPECweb99</A>
benchmark somewhat reflects the Web server's load scalability because it
measures the number of clients (load generators) given a mandatory minimal
throughput per client (that is, it measures the server's <I>capacity</I>).
This is unlike <A HREF="http://www.spec.org/osg/web96/">SPECweb96</A> and
other benchmarks that use the throughput as their main metric (see the figure
below).
<P>
<CENTER><IMG SRC="fig.gif" ALT="Figure: Throughput vs. Number of clients">
</CENTER>
<P>
<I>System scalability</I> is the ability of an application to sustain its
performance per hardware unit (such as a CPU) with the increasing number of
these units. In other words, good system scalability means that doubling the
number of processors will roughly double the application's throughput (dashed
green line). We assume here that the underlying operating system also scales
well. Good system scalability allows you to initially run an application on
the smallest system possible, while retaining the ability to move that
application to a larger system if necessary, without excessive effort or
expense. That is, an application need not be rewritten or even undergo a
major porting effort when changing system size.
<P>
Although scalability and performance are more important in the case of server
IAs, they should also be considered for some client applications (such as
benchmark load generators).
<P>
<A NAME="CONC">
<H4>1.3 Concurrency</H4>
</A>
<P>
Concurrency reflects the parallelism in a system. The two unrelated types
are <I>virtual</I> concurrency and <I>real</I> concurrency.
<UL>
<LI>Virtual (or apparent) concurrency is the number of simultaneous
connections that a system supports.
<BR><BR>
<LI>Real concurrency is the number of hardware devices, including
CPUs, network cards, and disks, that actually allow a system to perform
tasks in parallel.
</UL>
<P>
An IA must provide virtual concurrency in order to serve many users
simultaneously.
To achieve maximum performance and scalability in doing so, the number of
programming entities than an IA creates to be scheduled by the OS kernel
should be
kept close to (within an order of magnitude of) the real concurrency found on
the system. These programming entities scheduled by the kernel are known as
<I>kernel execution vehicles</I>. Examples of kernel execution vehicles
include Solaris lightweight processes and IRIX kernel threads.
In other words, the number of kernel execution vehicles should be dictated by
the system size and not by the number of simultaneous connections.
<P>
<H3>2. Existing Architectures</H3>
<P>
There are a few different architectures that are commonly used by IAs.
These include the <I>Multi-Process</I>,
<I>Multi-Threaded</I>, and <I>Event-Driven State Machine</I>
architectures.
<P>
<A NAME="MP">
<H4>2.1 Multi-Process Architecture</H4>
</A>
<P>
In the Multi-Process (MP) architecture, an individual process is
dedicated to each simultaneous connection.
A process performs all of a transaction's initialization steps
and services a connection completely before moving on to service
a new connection.
<P>
User sessions in IAs are relatively independent; therefore, no
synchronization between processes handling different connections is
necessary. Because each process has its own private address space,
this architecture is very robust. If a process serving one of the connections
crashes, the other sessions will not be affected. However, to serve many
concurrent connections, an equal number of processes must be employed.
Because processes are kernel entities (and are in fact the heaviest ones),
the number of kernel entities will be at least as large as the number of
concurrent sessions. On most systems, good performance will not be achieved
when more than a few hundred processes are created because of the high
context-switching overhead. In other words, MP applications have poor load
scalability.
<P>
On the other hand, MP applications have very good system scalability, because
no resources are shared among different processes and there is no
synchronization overhead.
<P>
The Apache Web Server 1.x (<A HREF=#refs1>[Reference 1]</A>) uses the MP
architecture on UNIX systems.
<P>
<A NAME="MT">
<H4>2.2 Multi-Threaded Architecture</H4>
</A>
<P>
In the Multi-Threaded (MT) architecture, multiple independent threads
of control are employed within a single shared address space. Like a
process in the MP architecture, each thread performs all of a
transaction's initialization steps and services a connection completely
before moving on to service a new connection.
<P>
Many modern UNIX operating systems implement a <I>many-to-few</I> model when
mapping user-level threads to kernel entities. In this model, an
arbitrarily large number of user-level threads is multiplexed onto a
lesser number of kernel execution vehicles. Kernel execution
vehicles are also known as <I>virtual processors</I>. Whenever a user-level
thread makes a blocking system call, the kernel execution vehicle it is using
will become blocked in the kernel. If there are no other non-blocked kernel
execution vehicles and there are other runnable user-level threads, a new
kernel execution vehicle will be created automatically. This prevents the
application from blocking when it can continue to make useful forward
progress.
<P>
Because IAs are by nature network I/O driven, all concurrent sessions block on
network I/O at various points. As a result, the number of virtual processors
created in the kernel grows close to the number of user-level threads
(or simultaneous connections). When this occurs, the many-to-few model
effectively degenerates to a <I>one-to-one</I> model. Again, like in
the MP architecture, the number of kernel execution vehicles is dictated by
the number of simultaneous connections rather than by number of CPUs. This
reduces an application's load scalability. However, because kernel threads
(lightweight processes) use fewer resources and are more light-weight than
traditional UNIX processes, an MT application should scale better with load
than an MP application.
<P>
Unexpectedly, the small number of virtual processors sharing the same address
space in the MT architecture destroys an application's system scalability
because of contention among the threads on various locks. Even if an
application itself is carefully
optimized to avoid lock contention around its own global data (a non-trivial
task), there are still standard library functions and system calls
that use common resources hidden from the application. For example,
on many platforms thread safety of memory allocation routines
(<TT>malloc(3)</TT>, <TT>free(3)</TT>, and so on) is achieved by using a single
global lock. Another example is a per-process file descriptor table.
This common resource table is shared by all kernel execution vehicles within
the same process and must be protected when one modifies it via
certain system calls (such as <TT>open(2)</TT>, <TT>close(2)</TT>, and so on).
In addition to that, maintaining the caches coherent
among CPUs on multiprocessor systems hurts performance when different threads
running on different CPUs modify data items on the same cache line.
<P>
In order to improve load scalability, some applications employ a different
type of MT architecture: they create one or more thread(s) <I>per task</I>
rather than one thread <I>per connection</I>. For example, one small group
of threads may be responsible for accepting client connections, another
for request processing, and yet another for serving responses. The main
advantage of this architecture is that it eliminates the tight coupling
between the number of threads and number of simultaneous connections. However,
in this architecture, different task-specific thread groups must share common
work queues that must be protected by mutual exclusion locks (a typical
producer-consumer problem). This adds synchronization overhead that causes an
application to perform badly on multiprocessor systems. In other words, in
this architecture, the application's system scalability is sacrificed for the
sake of load scalability.
<P>
Of course, the usual nightmares of threaded programming, including data
corruption, deadlocks, and race conditions, also make MT architecture (in any
form) non-simplistic to use.
<P>
<A NAME="EDSM">
<H4>2.3 Event-Driven State Machine Architecture</H4>
</A>
<P>
In the Event-Driven State Machine (EDSM) architecture, a single process
is employed to concurrently process multiple connections. The basics of this
architecture are described in Comer and Stevens
<A HREF=#refs2>[Reference 2]</A>.
The EDSM architecture performs one basic data-driven step associated with
a particular connection at a time, thus multiplexing many concurrent
connections. The process operates as a state machine that receives an event
and then reacts to it.
<P>
In the idle state the EDSM calls <TT>select(2)</TT> or <TT>poll(2)</TT> to
wait for network I/O events. When a particular file descriptor is ready for
I/O, the EDSM completes the corresponding basic step (usually by invoking a
handler function) and starts the next one. This architecture uses
non-blocking system calls to perform asynchronous network I/O operations.
For more details on non-blocking I/O see Stevens
<A HREF=#refs3>[Reference 3]</A>.
<P>
To take advantage of hardware parallelism (real concurrency), multiple
identical processes may be created. This is called Symmetric Multi-Process
EDSM and is used, for example, in the Zeus Web Server
(<A HREF=#refs4>[Reference 4]</A>). To more efficiently multiplex disk I/O,
special "helper" processes may be created. This is called Asymmetric
Multi-Process EDSM and was proposed for Web servers by Druschel
and others <A HREF=#refs5>[Reference 5]</A>.
<P>
EDSM is probably the most scalable architecture for IAs.
Because the number of simultaneous connections (virtual concurrency) is
completely decoupled from the number of kernel execution vehicles (processes),
this architecture has very good load scalability. It requires only minimal
user-level resources to create and maintain additional connection.
<P>
Like MP applications, Multi-Process EDSM has very good system scalability
because no resources are shared among different processes and there is no
synchronization overhead.
<P>
Unfortunately, the EDSM architecture is monolithic rather than based on the
concept of threads, so new applications generally need to be implemented from
the ground up. In effect, the EDSM architecture simulates threads and their
stacks the hard way.
<P>
<A NAME="ST">
<H3>3. State Threads Library</H3>
</A>
<P>
The State Threads library combines the advantages of all of the above
architectures. The interface preserves the programming simplicity of thread
abstraction, allowing each simultaneous connection to be treated as a separate
thread of execution within a single process. The underlying implementation is
close to the EDSM architecture as the state of each particular concurrent
session is saved in a separate memory segment.
<P>
<H4>3.1 State Changes and Scheduling</H4>
<P>
The state of each concurrent session includes its stack environment
(stack pointer, program counter, CPU registers) and its stack. Conceptually,
a thread context switch can be viewed as a process changing its state. There
are no kernel entities involved other than processes.
Unlike other general-purpose threading libraries, the State Threads library
is fully deterministic. The thread context switch (process state change) can
only happen in a well-known set of functions (at I/O points or at explicit
synchronization points). As a result, process-specific global data does not
have to be protected by mutual exclusion locks in most cases. The entire
application is free to use all the static variables and non-reentrant library
functions it wants, greatly simplifying programming and debugging while
increasing performance. This is somewhat similar to a <I>co-routine</I> model
(co-operatively multitasked threads), except that no explicit yield is needed
--
sooner or later, a thread performs a blocking I/O operation and thus surrenders
control. All threads of execution (simultaneous connections) have the
same priority, so scheduling is non-preemptive, like in the EDSM architecture.
Because IAs are data-driven (processing is limited by the size of network
buffers and data arrival rates), scheduling is non-time-slicing.
<P>
Only two types of external events are handled by the library's
scheduler, because only these events can be detected by
<TT>select(2)</TT> or <TT>poll(2)</TT>: I/O events (a file descriptor is ready
for I/O) and time events
(some timeout has expired). However, other types of events (such as
a signal sent to a process) can also be handled by converting them to I/O
events. For example, a signal handling function can perform a write to a pipe
(<TT>write(2)</TT> is reentrant/asynchronous-safe), thus converting a signal
event to an I/O event.
<P>
To take advantage of hardware parallelism, as in the EDSM architecture,
multiple processes can be created in either a symmetric or asymmetric manner.
Process management is not in the library's scope but instead is left up to the
application.
<P>
There are several general-purpose threading libraries that implement a
<I>many-to-one</I> model (many user-level threads to one kernel execution
vehicle), using the same basic techniques as the State Threads library
(non-blocking I/O, event-driven scheduler, and so on). For an example, see GNU
Portable Threads (<A HREF=#refs6>[Reference 6]</A>). Because they are
general-purpose, these libraries have different objectives than the State
Threads library. The State Threads library is <I>not</I> a general-purpose
threading library,
but rather an application library that targets only certain types of
applications (IAs) in order to achieve the highest possible performance and
scalability for those applications.
<P>
<H4>3.2 Scalability</H4>
<P>
State threads are very lightweight user-level entities, and therefore creating
and maintaining user connections requires minimal resources. An application
using the State Threads library scales very well with the increasing number
of connections.
<P>
On multiprocessor systems an application should create multiple processes
to take advantage of hardware parallelism. Using multiple separate processes
is the <I>only</I> way to achieve the highest possible system scalability.
This is because duplicating per-process resources is the only way to avoid
significant synchronization overhead on multiprocessor systems. Creating
separate UNIX processes naturally offers resource duplication. Again,
as in the EDSM architecture, there is no connection between the number of
simultaneous connections (which may be very large and changes within a wide
range) and the number of kernel entities (which is usually small and constant).
In other words, the State Threads library makes it possible to multiplex a
large number of simultaneous connections onto a much smaller number of
separate processes, thus allowing an application to scale well with both
the load and system size.
<P>
<H4>3.3 Performance</H4>
<P>
Performance is one of the library's main objectives. The State Threads
library is implemented to minimize the number of system calls and
to make thread creation and context switching as fast as possible.
For example, per-thread signal mask does not exist (unlike
POSIX threads), so there is no need to save and restore a process's
signal mask on every thread context switch. This eliminates two system
calls per context switch. Signal events can be handled much more
efficiently by converting them to I/O events (see above).
<P>
<H4>3.4 Portability</H4>
<P>
The library uses the same general, underlying concepts as the EDSM
architecture, including non-blocking I/O, file descriptors, and
I/O multiplexing. These concepts are available in some form on most
UNIX platforms, making the library very portable across many
flavors of UNIX. There are only a few platform-dependent sections in the
source.
<P>
<H4>3.5 State Threads and NSPR</H4>
<P>
The State Threads library is a derivative of the Netscape Portable
Runtime library (NSPR) <A HREF=#refs7>[Reference 7]</A>. The primary goal of
NSPR is to provide a platform-independent layer for system facilities,
where system facilities include threads, thread synchronization, and I/O.
Performance and scalability are not the main concern of NSPR. The
State Threads library addresses performance and scalability while
remaining much smaller than NSPR. It is contained in 8 source files
as opposed to more than 400, but provides all the functionality that
is needed to write efficient IAs on UNIX-like platforms.
<P>
<TABLE CELLPADDING=3>
<TR>
<TD></TD>
<TH>NSPR</TH>
<TH>State Threads</TH>
</TR>
<TR>
<TD><B>Lines of code</B></TD>
<TD ALIGN=RIGHT>~150,000</TD>
<TD ALIGN=RIGHT>~3000</TD>
</TR>
<TR>
<TD><B>Dynamic library size&nbsp;&nbsp;<BR>(debug version)</B></TD>
<TD></TD>
<TD></TD>
</TR>
<TR>
<TD>IRIX</TD>
<TD ALIGN=RIGHT>~700 KB</TD>
<TD ALIGN=RIGHT>~60 KB</TD>
</TR>
<TR>
<TD>Linux</TD>
<TD ALIGN=RIGHT>~900 KB</TD>
<TD ALIGN=RIGHT>~70 KB</TD>
</TR>
</TABLE>
<P>
<H3>Conclusion</H3>
<P>
State Threads is an application library which provides a foundation for
writing <A HREF=#IA>Internet Applications</A>. To summarize, it has the
following <I>advantages</I>:
<P>
<UL>
<LI>It allows the design of fast and highly scalable applications. An
application will scale well with both load and number of CPUs.
<P>
<LI>It greatly simplifies application programming and debugging because, as a
rule, no mutual exclusion locking is necessary and the entire application is
free to use static variables and non-reentrant library functions.
</UL>
<P>
The library's main <I>limitation</I>:
<P>
<UL>
<LI>All I/O operations on sockets must use the State Thread library's I/O
functions because only those functions perform thread scheduling and prevent
the application's processes from blocking.
</UL>
<P>
<H3>References</H3>
<OL>
<A NAME="refs1">
<LI> Apache Software Foundation,
<A HREF="http://www.apache.org">http://www.apache.org</A>.
<A NAME="refs2">
<LI> Douglas E. Comer, David L. Stevens, <I>Internetworking With TCP/IP,
Vol. III: Client-Server Programming And Applications</I>, Second Edition,
Ch. 8, 12.
<A NAME="refs3">
<LI> W. Richard Stevens, <I>UNIX Network Programming</I>, Second Edition,
Vol. 1, Ch. 15.
<A NAME="refs4">
<LI> Zeus Technology Limited,
<A HREF="http://www.zeus.co.uk/">http://www.zeus.co.uk</A>.
<A NAME="refs5">
<LI> Peter Druschel, Vivek S. Pai, Willy Zwaenepoel,
<A HREF="http://www.cs.rice.edu/~druschel/usenix99flash.ps.gz">
Flash: An Efficient and Portable Web Server</A>. In <I>Proceedings of the
USENIX 1999 Annual Technical Conference</I>, Monterey, CA, June 1999.
<A NAME="refs6">
<LI> GNU Portable Threads,
<A HREF="http://www.gnu.org/software/pth/">http://www.gnu.org/software/pth/</A>.
<A NAME="refs7">
<LI> Netscape Portable Runtime,
<A HREF="http://www.mozilla.org/docs/refList/refNSPR/">http://www.mozilla.org/docs/refList/refNSPR/</A>.
</OL>
<H3>Other resources covering various architectural issues in IAs</H3>
<OL START=8>
<LI> Dan Kegel, <I>The C10K problem</I>,
<A HREF="http://www.kegel.com/c10k.html">http://www.kegel.com/c10k.html</A>.
</LI>
<LI> James C. Hu, Douglas C. Schmidt, Irfan Pyarali, <I>JAWS: Understanding
High Performance Web Systems</I>,
<A HREF="http://www.cs.wustl.edu/~jxh/research/research.html">http://www.cs.wustl.edu/~jxh/research/research.html</A>.</LI>
</OL>
<P>
<HR>
<P>
<CENTER><FONT SIZE=-1>Portions created by SGI are Copyright &copy; 2000
Silicon Graphics, Inc. All rights reserved.</FONT></CENTER>
<P>
</BODY>
</HTML>
How the timeout heap works
As of version 1.5, the State Threads Library represents the queue of
sleeping threads using a heap data structure rather than a sorted
linked list. This improves performance when there is a large number
of sleeping threads, since insertion into a heap takes O(log N) time
while insertion into a sorted list takes O(N) time. For example, in
one test 1000 threads were created, each thread called st_usleep()
with a random time interval, and then all the threads where
immediately interrupted and joined before the sleeps had a chance to
finish. The whole process was repeated 1000 times, for a total of a
million sleep queue insertions and removals. With the old list-based
sleep queue, this test took 100 seconds; now it takes only 12 seconds.
Heap data structures are typically based on dynamically resized
arrays. However, since the existing ST code base was very nicely
structured around linking the thread objects into pointer-based lists
without the need for any auxiliary data structures, implementing the
heap using a similar nodes-and-pointers based approach seemed more
appropriate for ST than introducing a separate array.
Thus, the new ST timeout heap works by organizing the existing
_st_thread_t objects in a balanced binary tree, just as they were
previously organized into a doubly-linked, sorted list. The global
_ST_SLEEPQ variable, formerly a linked list head, is now simply a
pointer to the root of this tree, and the root node of the tree is the
thread with the earliest timeout. Each thread object has two child
pointers, "left" and "right", pointing to threads with later timeouts.
Each node in the tree is numbered with an integer index, corresponding
to the array index in an array-based heap, and the tree is kept fully
balanced and left-adjusted at all times. In other words, the tree
consists of any number of fully populated top levels, followed by a
single bottom level which may be partially populated, such that any
existing nodes form a contiguous block to the left and the spaces for
missing nodes form a contiguous block to the right. For example, if
there are nine threads waiting for a timeout, they are numbered and
arranged in a tree exactly as follows:
1
/ \
2 3
/ \ / \
4 5 6 7
/ \
8 9
Each node has either no children, only a left child, or both a left
and a right child. Children always time out later than their parents
(this is called the "heap invariant"), but when a node has two
children, their mutual order is unspecified - the left child may time
out before or after the right child. If a node is numbered N, its
left child is numbered 2N, and its right child is numbered 2N+1.
There is no pointer from a child to its parent; all pointers point
downward. Additions and deletions both work by starting at the root
and traversing the tree towards the leaves, going left or right
according to the binary digits forming the index of the destination
node. As nodes are added or deleted, existing nodes are rearranged to
maintain the heap invariant.
#
# Portions created by SGI are Copyright (C) 2000 Silicon Graphics, Inc.
# All Rights Reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
#
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# 3. Neither the name of Silicon Graphics, Inc. nor the names of its
# contributors may be used to endorse or promote products derived from
# this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# HOLDERS AND CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
# TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
##########################
# Supported OSes:
#
# AIX
# FREEBSD
# HPUX
# HPUX_64
# IRIX
# IRIX_64
# LINUX
# LINUX_IA64
# NETBSD
# OPENBSD
# OSF1
# SOLARIS
# SOLARIS_64
##########################
CC = cc
SHELL = /bin/sh
ECHO = /bin/echo
DEPTH = ..
BUILD =
TARGETDIR =
DEFINES =
CFLAGS =
OTHER_FLAGS =
OBJDIR = $(DEPTH)/$(TARGETDIR)
INCDIR = $(DEPTH)/$(TARGETDIR)
LIBST = $(OBJDIR)/libst.a
HEADER = $(INCDIR)/st.h
LIBRESOLV =
EXTRALIBS =
ifeq ($(OS),)
EXAMPLES = unknown
else
EXAMPLES = $(OBJDIR)/lookupdns $(OBJDIR)/proxy $(OBJDIR)/server
endif
##########################
# Platform section.
#
ifeq (DARWIN, $(findstring DARWIN, $(OS)))
LIBRESOLV = -lresolv
endif
ifeq (LINUX, $(findstring LINUX, $(OS)))
LIBRESOLV = -lresolv
endif
ifeq (SOLARIS, $(findstring SOLARIS, $(OS)))
LIBRESOLV = -lresolv
EXTRALIBS = -lsocket -lnsl
endif
#
# End of platform section.
##########################
all: $(EXAMPLES)
$(OBJDIR)/lookupdns: lookupdns.c $(OBJDIR)/res.o $(LIBST) $(HEADER)
$(CC) $(CFLAGS) -I$(INCDIR) lookupdns.c $(OBJDIR)/res.o $(LIBST) $(LIBRESOLV) $(EXTRALIBS) -o $@
$(OBJDIR)/proxy: proxy.c $(LIBST) $(HEADER)
$(CC) $(CFLAGS) -I$(INCDIR) proxy.c $(LIBST) $(EXTRALIBS) -o $@
$(OBJDIR)/server: server.c $(OBJDIR)/error.o $(LIBST) $(HEADER)
$(CC) $(CFLAGS) -I$(INCDIR) server.c $(OBJDIR)/error.o $(LIBST) $(EXTRALIBS) -o $@
$(OBJDIR)/%.o: %.c
$(CC) $(CFLAGS) -I$(INCDIR) -c $< -o $@
.DEFAULT:
@cd $(DEPTH); $(MAKE) $@
Portions created by SGI are Copyright (C) 2000 Silicon Graphics, Inc.
All Rights Reserved.
This directory contains three example programs.
---------------------------------------------------------------------------
PROGRAM
lookupdns
FILES
lookupdns.c
res.c
USAGE
lookupdns <hostname1> [<hostname2>] ...
DESCRIPTION
This program performs asynchronous DNS host name resolution and reports
IP address for each <hostname> specified as a command line argument.
One ST thread is created for each host name. All threads do host name
resolution concurrently.
---------------------------------------------------------------------------
PROGRAM
proxy
FILES
proxy.c
USAGE
proxy -l <local_addr> -r <remote_addr> [-p <num_processes>] [-S]
-l <local_addr> bind to local address specified as [<host>]:<port>
-r <remote_addr> connect to remote address specified as <host>:<port>
-p <num_processes> create specified number of processes
-S serialize accept() calls from different processes
on the same listening socket (if needed).
DESCRIPTION
This program acts as a generic gateway. It listens for connections to a
local address. Upon accepting a client connection, it connects to the
specified remote address and then just pumps the data through without any
modification.
---------------------------------------------------------------------------
PROGRAM
server
FILES
server.c
error.c
USAGE
server -l <log_directory> [<options>]
-l <log_directory> open all log files in specified directory.
Possible options:
-b <host>:<port> bind to specified address (multiple addresses
are permitted)
-p <num_processes> create specified number of processes
-t <min_thr>:<max_thr> specify thread limits per listening socket
across all processes
-u <user> change server's user id to specified value
-q <backlog> set max length of pending connections queue
-a enable access logging
-i run in interactive mode (useful for debugging)
-S serialize accept() calls from different processes
on the same listening socket (if needed).
DESCRIPTION
This program is a general server example. It accepts a client connection
and outputs a short HTML page. It can be easily adapted to provide
other services.
---------------------------------------------------------------------------
/*
* Portions created by SGI are Copyright (C) 2000 Silicon Graphics, Inc.
* All Rights Reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
*
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* 3. Neither the name of Silicon Graphics, Inc. nor the names of its
* contributors may be used to endorse or promote products derived from
* this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
* ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
* HOLDERS AND CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
* TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <stdarg.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <errno.h>
#include "st.h"
/*
* Simple error reporting functions.
* Suggested in W. Richard Stevens' "Advanced Programming in UNIX
* Environment".
*/
#define MAXLINE 4096 /* max line length */
static void err_doit(int, int, const char *, va_list);
/*
* Nonfatal error related to a system call.
* Print a message and return.
*/
void err_sys_report(int fd, const char *fmt, ...)
{
va_list ap;
va_start(ap, fmt);
err_doit(fd, 1, fmt, ap);
va_end(ap);
}
/*
* Fatal error related to a system call.
* Print a message and terminate.
*/
void err_sys_quit(int fd, const char *fmt, ...)
{
va_list ap;
va_start(ap, fmt);
err_doit(fd, 1, fmt, ap);
va_end(ap);
exit(1);
}
/*
* Fatal error related to a system call.
* Print a message, dump core, and terminate.
*/
void err_sys_dump(int fd, const char *fmt, ...)
{
va_list ap;
va_start(ap, fmt);
err_doit(fd, 1, fmt, ap);
va_end(ap);
abort(); /* dump core and terminate */
exit(1); /* shouldn't get here */
}
/*
* Nonfatal error unrelated to a system call.
* Print a message and return.
*/
void err_report(int fd, const char *fmt, ...)
{
va_list ap;
va_start(ap, fmt);
err_doit(fd, 0, fmt, ap);
va_end(ap);
}
/*
* Fatal error unrelated to a system call.
* Print a message and terminate.
*/
void err_quit(int fd, const char *fmt, ...)
{
va_list ap;
va_start(ap, fmt);
err_doit(fd, 0, fmt, ap);
va_end(ap);
exit(1);
}
/*
* Return a pointer to a string containing current time.
*/
char *err_tstamp(void)
{
static char *months[] = { "Jan", "Feb", "Mar", "Apr", "May", "Jun",
"Jul", "Aug", "Sep", "Oct", "Nov", "Dec" };
static char str[32];
static time_t lastt = 0;
struct tm *tmp;
time_t currt = st_time();
if (currt == lastt)
return str;
tmp = localtime(&currt);
sprintf(str, "[%02d/%s/%d:%02d:%02d:%02d] ", tmp->tm_mday,
months[tmp->tm_mon], 1900 + tmp->tm_year, tmp->tm_hour,
tmp->tm_min, tmp->tm_sec);
lastt = currt;
return str;
}
/*
* Print a message and return to caller.
* Caller specifies "errnoflag".
*/
static void err_doit(int fd, int errnoflag, const char *fmt, va_list ap)
{
int errno_save;
char buf[MAXLINE];
errno_save = errno; /* value caller might want printed */
strcpy(buf, err_tstamp()); /* prepend a message with time stamp */
vsprintf(buf + strlen(buf), fmt, ap);
if (errnoflag)
sprintf(buf + strlen(buf), ": %s\n", strerror(errno_save));
else
strcat(buf, "\n");
write(fd, buf, strlen(buf));
errno = errno_save;
}
/*
* Portions created by SGI are Copyright (C) 2000 Silicon Graphics, Inc.
* All Rights Reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
*
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* 3. Neither the name of Silicon Graphics, Inc. nor the names of its
* contributors may be used to endorse or promote products derived from
* this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
* ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
* HOLDERS AND CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
* TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/time.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <netdb.h>
#include "st.h"
#if !defined(NETDB_INTERNAL) && defined(h_NETDB_INTERNAL)
#define NETDB_INTERNAL h_NETDB_INTERNAL
#endif
/* Resolution timeout (in microseconds) */
#define TIMEOUT (2*1000000LL)
/* External function defined in the res.c file */
int dns_getaddr(const char *host, struct in_addr *addr, st_utime_t timeout);
void *do_resolve(void *host)
{
struct in_addr addr;
/* Use dns_getaddr() instead of gethostbyname(3) to get IP address */
if (dns_getaddr(host, &addr, TIMEOUT) < 0) {
fprintf(stderr, "dns_getaddr: can't resolve %s: ", (char *)host);
if (h_errno == NETDB_INTERNAL)
perror("");
else
herror("");
} else
printf("%-40s %s\n", (char *)host, inet_ntoa(addr));
return NULL;
}
/*
* Asynchronous DNS host name resolution. This program creates one
* ST thread for each host name (specified as command line arguments).
* All threads do host name resolution concurrently.
*/
int main(int argc, char *argv[])
{
int i;
if (argc < 2) {
fprintf(stderr, "Usage: %s <hostname1> [<hostname2>] ...\n", argv[0]);
exit(1);
}
if (st_init() < 0) {
perror("st_init");
exit(1);
}
for (i = 1; i < argc; i++) {
/* Create a separate thread for each host name */
if (st_thread_create(do_resolve, argv[i], 0, 0) == NULL) {
perror("st_thread_create");
exit(1);
}
}
st_thread_exit(NULL);
/* NOTREACHED */
return 1;
}
/*
* Portions created by SGI are Copyright (C) 2000 Silicon Graphics, Inc.
* All Rights Reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
*
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* 3. Neither the name of Silicon Graphics, Inc. nor the names of its
* contributors may be used to endorse or promote products derived from
* this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
* ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
* HOLDERS AND CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
* TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <signal.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <netdb.h>
#include "st.h"
#define IOBUFSIZE (16*1024)
#define IOV_LEN 256
#define IOV_COUNT (IOBUFSIZE / IOV_LEN)
#ifndef INADDR_NONE
#define INADDR_NONE 0xffffffff
#endif
static char *prog; /* Program name */
static struct sockaddr_in rmt_addr; /* Remote address */
static unsigned long testing;
#define TESTING_VERBOSE 0x1
#define TESTING_READV 0x2
#define TESTING_READ_RESID 0x4
#define TESTING_WRITEV 0x8
#define TESTING_WRITE_RESID 0x10
static void read_address(const char *str, struct sockaddr_in *sin);
static void start_daemon(void);
static int cpu_count(void);
static void set_concurrency(int nproc);
static void *handle_request(void *arg);
static void print_sys_error(const char *msg);
/*
* This program acts as a generic gateway. It listens for connections
* to a local address ('-l' option). Upon accepting a client connection,
* it connects to the specified remote address ('-r' option) and then
* just pumps the data through without any modification.
*/
int main(int argc, char *argv[])
{
extern char *optarg;
int opt, sock, n;
int laddr, raddr, num_procs, alt_ev, one_process;
int serialize_accept = 0;
struct sockaddr_in lcl_addr, cli_addr;
st_netfd_t cli_nfd, srv_nfd;
prog = argv[0];
num_procs = laddr = raddr = alt_ev = one_process = 0;
/* Parse arguments */
while((opt = getopt(argc, argv, "l:r:p:Saht:X")) != EOF) {
switch (opt) {
case 'a':
alt_ev = 1;
break;
case 'l':
read_address(optarg, &lcl_addr);
laddr = 1;
break;
case 'r':
read_address(optarg, &rmt_addr);
if (rmt_addr.sin_addr.s_addr == INADDR_ANY) {
fprintf(stderr, "%s: invalid remote address: %s\n", prog, optarg);
exit(1);
}
raddr = 1;
break;
case 'p':
num_procs = atoi(optarg);
if (num_procs < 1) {
fprintf(stderr, "%s: invalid number of processes: %s\n", prog, optarg);
exit(1);
}
break;
case 'S':
/*
* Serialization decision is tricky on some platforms. For example,
* Solaris 2.6 and above has kernel sockets implementation, so supposedly
* there is no need for serialization. The ST library may be compiled
* on one OS version, but used on another, so the need for serialization
* should be determined at run time by the application. Since it's just
* an example, the serialization decision is left up to user.
* Only on platforms where the serialization is never needed on any OS
* version st_netfd_serialize_accept() is a no-op.
*/
serialize_accept = 1;
break;
case 't':
testing = strtoul(optarg, NULL, 0);
break;
case 'X':
one_process = 1;
break;
case 'h':
case '?':
fprintf(stderr, "Usage: %s [options] -l <[host]:port> -r <host:port>\n",
prog);
fprintf(stderr, "options are:\n");
fprintf(stderr, " -p <num_processes> number of parallel processes\n");
fprintf(stderr, " -S serialize accepts\n");
fprintf(stderr, " -a use alternate event system\n");
#ifdef DEBUG
fprintf(stderr, " -t mask testing/debugging mode\n");
fprintf(stderr, " -X one process, don't daemonize\n");
#endif
exit(1);
}
}
if (!laddr) {
fprintf(stderr, "%s: local address required\n", prog);
exit(1);
}
if (!raddr) {
fprintf(stderr, "%s: remote address required\n", prog);
exit(1);
}
if (num_procs == 0)
num_procs = cpu_count();
fprintf(stderr, "%s: starting proxy daemon on %s:%d\n", prog,
inet_ntoa(lcl_addr.sin_addr), ntohs(lcl_addr.sin_port));
/* Start the daemon */
if (one_process)
num_procs = 1;
else
start_daemon();
if (alt_ev)
st_set_eventsys(ST_EVENTSYS_ALT);
/* Initialize the ST library */
if (st_init() < 0) {
print_sys_error("st_init");
exit(1);
}
/* Create and bind listening socket */
if ((sock = socket(PF_INET, SOCK_STREAM, 0)) < 0) {
print_sys_error("socket");
exit(1);
}
n = 1;
if (setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, (char *)&n, sizeof(n)) < 0) {
print_sys_error("setsockopt");
exit(1);
}
if (bind(sock, (struct sockaddr *)&lcl_addr, sizeof(lcl_addr)) < 0) {
print_sys_error("bind");
exit(1);
}
listen(sock, 128);
if ((srv_nfd = st_netfd_open_socket(sock)) == NULL) {
print_sys_error("st_netfd_open");
exit(1);
}
/* See the comment regarding serialization decision above */
if (num_procs > 1 && serialize_accept && st_netfd_serialize_accept(srv_nfd)
< 0) {
print_sys_error("st_netfd_serialize_accept");
exit(1);
}
/* Start server processes */
if (!one_process)
set_concurrency(num_procs);
for ( ; ; ) {
n = sizeof(cli_addr);
cli_nfd = st_accept(srv_nfd, (struct sockaddr *)&cli_addr, &n,
ST_UTIME_NO_TIMEOUT);
if (cli_nfd == NULL) {
print_sys_error("st_accept");
exit(1);
}
if (st_thread_create(handle_request, cli_nfd, 0, 0) == NULL) {
print_sys_error("st_thread_create");
exit(1);
}
}
/* NOTREACHED */
return 1;
}
static void read_address(const char *str, struct sockaddr_in *sin)
{
char host[128], *p;
struct hostent *hp;
unsigned short port;
strcpy(host, str);
if ((p = strchr(host, ':')) == NULL) {
fprintf(stderr, "%s: invalid address: %s\n", prog, host);
exit(1);
}
*p++ = '\0';
port = (unsigned short) atoi(p);
if (port < 1) {
fprintf(stderr, "%s: invalid port: %s\n", prog, p);
exit(1);
}
memset(sin, 0, sizeof(struct sockaddr_in));
sin->sin_family = AF_INET;
sin->sin_port = htons(port);
if (host[0] == '\0') {
sin->sin_addr.s_addr = INADDR_ANY;
return;
}
sin->sin_addr.s_addr = inet_addr(host);
if (sin->sin_addr.s_addr == INADDR_NONE) {
/* not dotted-decimal */
if ((hp = gethostbyname(host)) == NULL) {
fprintf(stderr, "%s: can't resolve address: %s\n", prog, host);
exit(1);
}
memcpy(&sin->sin_addr, hp->h_addr, hp->h_length);
}
}
#ifdef DEBUG
static void show_iov(const struct iovec *iov, int niov)
{
int i;
size_t total;
printf("iov %p has %d entries:\n", iov, niov);
total = 0;
for (i = 0; i < niov; i++) {
printf("iov[%3d] iov_base=%p iov_len=0x%lx(%lu)\n",
i, iov[i].iov_base, (unsigned long) iov[i].iov_len,
(unsigned long) iov[i].iov_len);
total += iov[i].iov_len;
}
printf("total 0x%lx(%ld)\n", (unsigned long) total, (unsigned long) total);
}
/*
* This version is tricked out to test all the
* st_(read|write)v?(_resid)? variants. Use the non-DEBUG version for
* anything serious. st_(read|write) are all this function really
* needs.
*/
static int pass(st_netfd_t in, st_netfd_t out)
{
char buf[IOBUFSIZE];
struct iovec iov[IOV_COUNT];
int ioviter, nw, nr;
if (testing & TESTING_READV) {
for (ioviter = 0; ioviter < IOV_COUNT; ioviter++) {
iov[ioviter].iov_base = &buf[ioviter * IOV_LEN];
iov[ioviter].iov_len = IOV_LEN;
}
if (testing & TESTING_VERBOSE) {
printf("readv(%p)...\n", in);
show_iov(iov, IOV_COUNT);
}
if (testing & TESTING_READ_RESID) {
struct iovec *riov = iov;
int riov_cnt = IOV_COUNT;
if (st_readv_resid(in, &riov, &riov_cnt, ST_UTIME_NO_TIMEOUT) == 0) {
if (testing & TESTING_VERBOSE) {
printf("resid\n");
show_iov(riov, riov_cnt);
printf("full\n");
show_iov(iov, IOV_COUNT);
}
nr = 0;
for (ioviter = 0; ioviter < IOV_COUNT; ioviter++)
nr += iov[ioviter].iov_len;
nr = IOBUFSIZE - nr;
} else
nr = -1;
} else
nr = (int) st_readv(in, iov, IOV_COUNT, ST_UTIME_NO_TIMEOUT);
} else {
if (testing & TESTING_READ_RESID) {
size_t resid = IOBUFSIZE;
if (st_read_resid(in, buf, &resid, ST_UTIME_NO_TIMEOUT) == 0)
nr = IOBUFSIZE - resid;
else
nr = -1;
} else
nr = (int) st_read(in, buf, IOBUFSIZE, ST_UTIME_NO_TIMEOUT);
}
if (testing & TESTING_VERBOSE)
printf("got 0x%x(%d) E=%d\n", nr, nr, errno);
if (nr <= 0)
return 0;
if (testing & TESTING_WRITEV) {
for (nw = 0, ioviter = 0; nw < nr;
nw += iov[ioviter].iov_len, ioviter++) {
iov[ioviter].iov_base = &buf[nw];
iov[ioviter].iov_len = nr - nw;
if (iov[ioviter].iov_len > IOV_LEN)
iov[ioviter].iov_len = IOV_LEN;
}
if (testing & TESTING_VERBOSE) {
printf("writev(%p)...\n", out);
show_iov(iov, ioviter);
}
if (testing & TESTING_WRITE_RESID) {
struct iovec *riov = iov;
int riov_cnt = ioviter;
if (st_writev_resid(out, &riov, &riov_cnt, ST_UTIME_NO_TIMEOUT) == 0) {
if (testing & TESTING_VERBOSE) {
printf("resid\n");
show_iov(riov, riov_cnt);
printf("full\n");
show_iov(iov, ioviter);
}
nw = 0;
while (--ioviter >= 0)
nw += iov[ioviter].iov_len;
nw = nr - nw;
} else
nw = -1;
} else
nw = st_writev(out, iov, ioviter, ST_UTIME_NO_TIMEOUT);
} else {
if (testing & TESTING_WRITE_RESID) {
size_t resid = nr;
if (st_write_resid(out, buf, &resid, ST_UTIME_NO_TIMEOUT) == 0)
nw = nr - resid;
else
nw = -1;
} else
nw = st_write(out, buf, nr, ST_UTIME_NO_TIMEOUT);
}
if (testing & TESTING_VERBOSE)
printf("put 0x%x(%d) E=%d\n", nw, nw, errno);
if (nw != nr)
return 0;
return 1;
}
#else /* DEBUG */
/*
* This version is the simple one suitable for serious use.
*/
static int pass(st_netfd_t in, st_netfd_t out)
{
char buf[IOBUFSIZE];
int nw, nr;
nr = (int) st_read(in, buf, IOBUFSIZE, ST_UTIME_NO_TIMEOUT);
if (nr <= 0)
return 0;
nw = st_write(out, buf, nr, ST_UTIME_NO_TIMEOUT);
if (nw != nr)
return 0;
return 1;
}
#endif
static void *handle_request(void *arg)
{
struct pollfd pds[2];
st_netfd_t cli_nfd, rmt_nfd;
int sock;
cli_nfd = (st_netfd_t) arg;
pds[0].fd = st_netfd_fileno(cli_nfd);
pds[0].events = POLLIN;
/* Connect to remote host */
if ((sock = socket(PF_INET, SOCK_STREAM, 0)) < 0) {
print_sys_error("socket");
goto done;
}
if ((rmt_nfd = st_netfd_open_socket(sock)) == NULL) {
print_sys_error("st_netfd_open_socket");
close(sock);
goto done;
}
if (st_connect(rmt_nfd, (struct sockaddr *)&rmt_addr,
sizeof(rmt_addr), ST_UTIME_NO_TIMEOUT) < 0) {
print_sys_error("st_connect");
st_netfd_close(rmt_nfd);
goto done;
}
pds[1].fd = sock;
pds[1].events = POLLIN;
/*
* Now just pump the data through.
* XXX This should use one thread for each direction for true full-duplex.
*/
for ( ; ; ) {
pds[0].revents = 0;
pds[1].revents = 0;
if (st_poll(pds, 2, ST_UTIME_NO_TIMEOUT) <= 0) {
print_sys_error("st_poll");
break;
}
if (pds[0].revents & POLLIN) {
if (!pass(cli_nfd, rmt_nfd))
break;
}
if (pds[1].revents & POLLIN) {
if (!pass(rmt_nfd, cli_nfd))
break;
}
}
st_netfd_close(rmt_nfd);
done:
st_netfd_close(cli_nfd);
return NULL;
}
static void start_daemon(void)
{
pid_t pid;
/* Start forking */
if ((pid = fork()) < 0) {
print_sys_error("fork");
exit(1);
}
if (pid > 0)
exit(0); /* parent */
/* First child process */
setsid(); /* become session leader */
if ((pid = fork()) < 0) {
print_sys_error("fork");
exit(1);
}
if (pid > 0) /* first child */
exit(0);
chdir("/");
umask(022);
}
/*
* Create separate processes ("virtual processors"). Since it's just an
* example, there is no watchdog - the parent just exits leaving children
* on their own.
*/
static void set_concurrency(int nproc)
{
pid_t pid;
int i;
if (nproc < 1)
nproc = 1;
for (i = 0; i < nproc; i++) {
if ((pid = fork()) < 0) {
print_sys_error("fork");
exit(1);
}
/* Child returns */
if (pid == 0)
return;
}
/* Parent just exits */
exit(0);
}
static int cpu_count(void)
{
int n;
#if defined (_SC_NPROCESSORS_ONLN)
n = (int) sysconf(_SC_NPROCESSORS_ONLN);
#elif defined (_SC_NPROC_ONLN)
n = (int) sysconf(_SC_NPROC_ONLN);
#elif defined (HPUX)
#include <sys/mpctl.h>
n = mpctl(MPC_GETNUMSPUS, 0, 0);
#else
n = -1;
errno = ENOSYS;
#endif
return n;
}
static void print_sys_error(const char *msg)
{
fprintf(stderr, "%s: %s: %s\n", prog, msg, strerror(errno));
}
/*
* Copyright (c) 1985, 1988, 1993
* The Regents of the University of California. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* 3. All advertising materials mentioning features or use of this software
* must display the following acknowledgement:
* This product includes software developed by the University of
* California, Berkeley and its contributors.
* 4. Neither the name of the University nor the names of its contributors
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*
* Portions created by SGI are Copyright (C) 2000 Silicon Graphics, Inc.
* All Rights Reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
*
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* 3. Neither the name of Silicon Graphics, Inc. nor the names of its
* contributors may be used to endorse or promote products derived from
* this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
* ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
* HOLDERS AND CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
* TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#if defined (DARWIN)
#define BIND_8_COMPAT
#endif
#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/nameser.h>
#include <resolv.h>
#include <netdb.h>
#include <errno.h>
#include "st.h"
#define MAXPACKET 1024
#if !defined(NETDB_INTERNAL) && defined(h_NETDB_INTERNAL)
#define NETDB_INTERNAL h_NETDB_INTERNAL
#endif
/* New in Solaris 7 */
#if !defined(_getshort) && defined(ns_get16)
#define _getshort(cp) ns_get16(cp)
#endif
typedef union {
HEADER hdr;
u_char buf[MAXPACKET];
} querybuf_t;
static int parse_answer(querybuf_t *ans, int len, struct in_addr *addr)
{
char buf[MAXPACKET];
HEADER *ahp;
u_char *cp, *eoa;
int type, n;
ahp = &ans->hdr;
eoa = ans->buf + len;
cp = ans->buf + sizeof(HEADER);
while (ahp->qdcount > 0) {
ahp->qdcount--;
cp += dn_skipname(cp, eoa) + QFIXEDSZ;
}
while (ahp->ancount > 0 && cp < eoa) {
ahp->ancount--;
if ((n = dn_expand(ans->buf, eoa, cp, buf, sizeof(buf))) < 0)
break;
cp += n;
type = _getshort(cp);
cp += 8;
n = _getshort(cp);
cp += 2;
if (type == T_CNAME) {
cp += n;
continue;
}
memcpy(addr, cp, n);
return 0;
}
h_errno = TRY_AGAIN;
return -1;
}
static int query_domain(st_netfd_t nfd, const char *name, struct in_addr *addr,
st_utime_t timeout)
{
querybuf_t qbuf;
u_char *buf = qbuf.buf;
HEADER *hp = &qbuf.hdr;
int blen = sizeof(qbuf);
int i, len, id;
for (i = 0; i < _res.nscount; i++) {
len = res_mkquery(QUERY, name, C_IN, T_A, NULL, 0, NULL, buf, blen);
if (len <= 0) {
h_errno = NO_RECOVERY;
return -1;
}
id = hp->id;
if (st_sendto(nfd, buf, len, (struct sockaddr *)&(_res.nsaddr_list[i]),
sizeof(struct sockaddr), timeout) != len) {
h_errno = NETDB_INTERNAL;
/* EINTR means interrupt by other thread, NOT by a caught signal */
if (errno == EINTR)
return -1;
continue;
}
/* Wait for reply */
do {
len = st_recvfrom(nfd, buf, blen, NULL, NULL, timeout);
if (len <= 0)
break;
} while (id != hp->id);
if (len < HFIXEDSZ) {
h_errno = NETDB_INTERNAL;
if (len >= 0)
errno = EMSGSIZE;
else if (errno == EINTR) /* see the comment above */
return -1;
continue;
}
hp->ancount = ntohs(hp->ancount);
hp->qdcount = ntohs(hp->qdcount);
if ((hp->rcode != NOERROR) || (hp->ancount == 0)) {
switch (hp->rcode) {
case NXDOMAIN:
h_errno = HOST_NOT_FOUND;
break;
case SERVFAIL:
h_errno = TRY_AGAIN;
break;
case NOERROR:
h_errno = NO_DATA;
break;
case FORMERR:
case NOTIMP:
case REFUSED:
default:
h_errno = NO_RECOVERY;
}
continue;
}
if (parse_answer(&qbuf, len, addr) == 0)
return 0;
}
return -1;
}
#define CLOSE_AND_RETURN(ret) \
{ \
n = errno; \
st_netfd_close(nfd); \
errno = n; \
return (ret); \
}
int dns_getaddr(const char *host, struct in_addr *addr, st_utime_t timeout)
{
char name[MAXDNAME], **domain;
const char *cp;
int s, n, maxlen, dots;
int trailing_dot, tried_as_is;
st_netfd_t nfd;
if ((_res.options & RES_INIT) == 0 && res_init() == -1) {
h_errno = NETDB_INTERNAL;
return -1;
}
if (_res.options & RES_USEVC) {
h_errno = NETDB_INTERNAL;
errno = ENOSYS;
return -1;
}
if (!host || *host == '\0') {
h_errno = HOST_NOT_FOUND;
return -1;
}
/* Create UDP socket */
if ((s = socket(PF_INET, SOCK_DGRAM, 0)) < 0) {
h_errno = NETDB_INTERNAL;
return -1;
}
if ((nfd = st_netfd_open_socket(s)) == NULL) {
h_errno = NETDB_INTERNAL;
n = errno;
close(s);
errno = n;
return -1;
}
maxlen = sizeof(name) - 1;
n = 0;
dots = 0;
trailing_dot = 0;
tried_as_is = 0;
for (cp = host; *cp && n < maxlen; cp++) {
dots += (*cp == '.');
name[n++] = *cp;
}
if (name[n - 1] == '.')
trailing_dot = 1;
/*
* If there are dots in the name already, let's just give it a try
* 'as is'. The threshold can be set with the "ndots" option.
*/
if (dots >= _res.ndots) {
if (query_domain(nfd, host, addr, timeout) == 0)
CLOSE_AND_RETURN(0);
if (h_errno == NETDB_INTERNAL && errno == EINTR)
CLOSE_AND_RETURN(-1);
tried_as_is = 1;
}
/*
* We do at least one level of search if
* - there is no dot and RES_DEFNAME is set, or
* - there is at least one dot, there is no trailing dot,
* and RES_DNSRCH is set.
*/
if ((!dots && (_res.options & RES_DEFNAMES)) ||
(dots && !trailing_dot && (_res.options & RES_DNSRCH))) {
name[n++] = '.';
for (domain = _res.dnsrch; *domain; domain++) {
strncpy(name + n, *domain, maxlen - n);
if (query_domain(nfd, name, addr, timeout) == 0)
CLOSE_AND_RETURN(0);
if (h_errno == NETDB_INTERNAL && errno == EINTR)
CLOSE_AND_RETURN(-1);
if (!(_res.options & RES_DNSRCH))
break;
}
}
/*
* If we have not already tried the name "as is", do that now.
* note that we do this regardless of how many dots were in the
* name or whether it ends with a dot.
*/
if (!tried_as_is) {
if (query_domain(nfd, host, addr, timeout) == 0)
CLOSE_AND_RETURN(0);
}
CLOSE_AND_RETURN(-1);
}
/*
* Portions created by SGI are Copyright (C) 2000 Silicon Graphics, Inc.
* All Rights Reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
*
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* 3. Neither the name of Silicon Graphics, Inc. nor the names of its
* contributors may be used to endorse or promote products derived from
* this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
* ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
* HOLDERS AND CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
* TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <time.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/socket.h>
#include <sys/wait.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <netdb.h>
#include <fcntl.h>
#include <signal.h>
#include <pwd.h>
#include "st.h"
/******************************************************************
* Server configuration parameters
*/
/* Log files */
#define PID_FILE "pid"
#define ERRORS_FILE "errors"
#define ACCESS_FILE "access"
/* Default server port */
#define SERV_PORT_DEFAULT 8000
/* Socket listen queue size */
#define LISTENQ_SIZE_DEFAULT 256
/* Max number of listening sockets ("hardware virtual servers") */
#define MAX_BIND_ADDRS 16
/* Max number of "spare" threads per process per socket */
#define MAX_WAIT_THREADS_DEFAULT 8
/* Number of file descriptors needed to handle one client session */
#define FD_PER_THREAD 2
/* Access log buffer flushing interval (in seconds) */
#define ACCLOG_FLUSH_INTERVAL 30
/* Request read timeout (in seconds) */
#define REQUEST_TIMEOUT 30
/******************************************************************
* Global data
*/
struct socket_info {
st_netfd_t nfd; /* Listening socket */
char *addr; /* Bind address */
unsigned int port; /* Port */
int wait_threads; /* Number of threads waiting to accept */
int busy_threads; /* Number of threads processing request */
int rqst_count; /* Total number of processed requests */
} srv_socket[MAX_BIND_ADDRS]; /* Array of listening sockets */
static int sk_count = 0; /* Number of listening sockets */
static int vp_count = 0; /* Number of server processes (VPs) */
static pid_t *vp_pids; /* Array of VP pids */
static int my_index = -1; /* Current process index */
static pid_t my_pid = -1; /* Current process pid */
static st_netfd_t sig_pipe[2]; /* Signal pipe */
/*
* Configuration flags/parameters
*/
static int interactive_mode = 0;
static int serialize_accept = 0;
static int log_access = 0;
static char *logdir = NULL;
static char *username = NULL;
static int listenq_size = LISTENQ_SIZE_DEFAULT;
static int errfd = STDERR_FILENO;
/*
* Thread throttling parameters (all numbers are per listening socket).
* Zero values mean use default.
*/
static int max_threads = 0; /* Max number of threads */
static int max_wait_threads = 0; /* Max number of "spare" threads */
static int min_wait_threads = 2; /* Min number of "spare" threads */
/******************************************************************
* Useful macros
*/
#ifndef INADDR_NONE
#define INADDR_NONE 0xffffffff
#endif
#define SEC2USEC(s) ((s)*1000000LL)
#define WAIT_THREADS(i) (srv_socket[i].wait_threads)
#define BUSY_THREADS(i) (srv_socket[i].busy_threads)
#define TOTAL_THREADS(i) (WAIT_THREADS(i) + BUSY_THREADS(i))
#define RQST_COUNT(i) (srv_socket[i].rqst_count)
/******************************************************************
* Forward declarations
*/
static void usage(const char *progname);
static void parse_arguments(int argc, char *argv[]);
static void start_daemon(void);
static void set_thread_throttling(void);
static void create_listeners(void);
static void change_user(void);
static void open_log_files(void);
static void start_processes(void);
static void wdog_sighandler(int signo);
static void child_sighandler(int signo);
static void install_sighandlers(void);
static void start_threads(void);
static void *process_signals(void *arg);
static void *flush_acclog_buffer(void *arg);
static void *handle_connections(void *arg);
static void dump_server_info(void);
static void Signal(int sig, void (*handler)(int));
static int cpu_count(void);
extern void handle_session(long srv_socket_index, st_netfd_t cli_nfd);
extern void load_configs(void);
extern void logbuf_open(void);
extern void logbuf_flush(void);
extern void logbuf_close(void);
/* Error reporting functions defined in the error.c file */
extern void err_sys_report(int fd, const char *fmt, ...);
extern void err_sys_quit(int fd, const char *fmt, ...);
extern void err_sys_dump(int fd, const char *fmt, ...);
extern void err_report(int fd, const char *fmt, ...);
extern void err_quit(int fd, const char *fmt, ...);
/*
* General server example: accept a client connection and do something.
* This program just outputs a short HTML page, but can be easily adapted
* to do other things.
*
* This server creates a constant number of processes ("virtual processors"
* or VPs) and replaces them when they die. Each virtual processor manages
* its own independent set of state threads (STs), the number of which varies
* with load against the server. Each state thread listens to exactly one
* listening socket. The initial process becomes the watchdog, waiting for
* children (VPs) to die or for a signal requesting termination or restart.
* Upon receiving a restart signal (SIGHUP), all VPs close and then reopen
* log files and reload configuration. All currently active connections remain
* active. It is assumed that new configuration affects only request
* processing and not the general server parameters such as number of VPs,
* thread limits, bind addresses, etc. Those are specified as command line
* arguments, so the server has to be stopped and then started again in order
* to change them.
*
* Each state thread loops processing connections from a single listening
* socket. Only one ST runs on a VP at a time, and VPs do not share memory,
* so no mutual exclusion locking is necessary on any data, and the entire
* server is free to use all the static variables and non-reentrant library
* functions it wants, greatly simplifying programming and debugging and
* increasing performance (for example, it is safe to ++ and -- all global
* counters or call inet_ntoa(3) without any mutexes). The current thread on
* each VP maintains equilibrium on that VP, starting a new thread or
* terminating itself if the number of spare threads exceeds the lower or
* upper limit.
*
* All I/O operations on sockets must use the State Thread library's I/O
* functions because only those functions prevent blocking of the entire VP
* process and perform state thread scheduling.
*/
int main(int argc, char *argv[])
{
/* Parse command-line options */
parse_arguments(argc, argv);
/* Allocate array of server pids */
if ((vp_pids = calloc(vp_count, sizeof(pid_t))) == NULL)
err_sys_quit(errfd, "ERROR: calloc failed");
/* Start the daemon */
if (!interactive_mode)
start_daemon();
/* Initialize the ST library */
if (st_init() < 0)
err_sys_quit(errfd, "ERROR: initialization failed: st_init");
/* Set thread throttling parameters */
set_thread_throttling();
/* Create listening sockets */
create_listeners();
/* Change the user */
if (username)
change_user();
/* Open log files */
open_log_files();
/* Start server processes (VPs) */
start_processes();
/* Turn time caching on */
st_timecache_set(1);
/* Install signal handlers */
install_sighandlers();
/* Load configuration from config files */
load_configs();
/* Start all threads */
start_threads();
/* Become a signal processing thread */
process_signals(NULL);
/* NOTREACHED */
return 1;
}
/******************************************************************/
static void usage(const char *progname)
{
fprintf(stderr, "Usage: %s -l <log_directory> [<options>]\n\n"
"Possible options:\n\n"
"\t-b <host>:<port> Bind to specified address. Multiple"
" addresses\n"
"\t are permitted.\n"
"\t-p <num_processes> Create specified number of processes.\n"
"\t-t <min_thr>:<max_thr> Specify thread limits per listening"
" socket\n"
"\t across all processes.\n"
"\t-u <user> Change server's user id to specified"
" value.\n"
"\t-q <backlog> Set max length of pending connections"
" queue.\n"
"\t-a Enable access logging.\n"
"\t-i Run in interactive mode.\n"
"\t-S Serialize all accept() calls.\n"
"\t-h Print this message.\n",
progname);
exit(1);
}
/******************************************************************/
static void parse_arguments(int argc, char *argv[])
{
extern char *optarg;
int opt;
char *c;
while ((opt = getopt(argc, argv, "b:p:l:t:u:q:aiSh")) != EOF) {
switch (opt) {
case 'b':
if (sk_count >= MAX_BIND_ADDRS)
err_quit(errfd, "ERROR: max number of bind addresses (%d) exceeded",
MAX_BIND_ADDRS);
if ((c = strdup(optarg)) == NULL)
err_sys_quit(errfd, "ERROR: strdup");
srv_socket[sk_count++].addr = c;
break;
case 'p':
vp_count = atoi(optarg);
if (vp_count < 1)
err_quit(errfd, "ERROR: invalid number of processes: %s", optarg);
break;
case 'l':
logdir = optarg;
break;
case 't':
max_wait_threads = (int) strtol(optarg, &c, 10);
if (*c++ == ':')
max_threads = atoi(c);
if (max_wait_threads < 0 || max_threads < 0)
err_quit(errfd, "ERROR: invalid number of threads: %s", optarg);
break;
case 'u':
username = optarg;
break;
case 'q':
listenq_size = atoi(optarg);
if (listenq_size < 1)
err_quit(errfd, "ERROR: invalid listen queue size: %s", optarg);
break;
case 'a':
log_access = 1;
break;
case 'i':
interactive_mode = 1;
break;
case 'S':
/*
* Serialization decision is tricky on some platforms. For example,
* Solaris 2.6 and above has kernel sockets implementation, so supposedly
* there is no need for serialization. The ST library may be compiled
* on one OS version, but used on another, so the need for serialization
* should be determined at run time by the application. Since it's just
* an example, the serialization decision is left up to user.
* Only on platforms where the serialization is never needed on any OS
* version st_netfd_serialize_accept() is a no-op.
*/
serialize_accept = 1;
break;
case 'h':
case '?':
usage(argv[0]);
}
}
if (logdir == NULL && !interactive_mode) {
err_report(errfd, "ERROR: logging directory is required\n");
usage(argv[0]);
}
if (getuid() == 0 && username == NULL)
err_report(errfd, "WARNING: running as super-user!");
if (vp_count == 0 && (vp_count = cpu_count()) < 1)
vp_count = 1;
if (sk_count == 0) {
sk_count = 1;
srv_socket[0].addr = "0.0.0.0";
}
}
/******************************************************************/
static void start_daemon(void)
{
pid_t pid;
/* Start forking */
if ((pid = fork()) < 0)
err_sys_quit(errfd, "ERROR: fork");
if (pid > 0)
exit(0); /* parent */
/* First child process */
setsid(); /* become session leader */
if ((pid = fork()) < 0)
err_sys_quit(errfd, "ERROR: fork");
if (pid > 0) /* first child */
exit(0);
umask(022);
if (chdir(logdir) < 0)
err_sys_quit(errfd, "ERROR: can't change directory to %s: chdir", logdir);
}
/******************************************************************
* For simplicity, the minimal size of thread pool is considered
* as a maximum number of spare threads (max_wait_threads) that
* will be created upon server startup. The pool size can grow up
* to the max_threads value. Note that this is a per listening
* socket limit. It is also possible to limit the total number of
* threads for all sockets rather than impose a per socket limit.
*/
static void set_thread_throttling(void)
{
/*
* Calculate total values across all processes.
* All numbers are per listening socket.
*/
if (max_wait_threads == 0)
max_wait_threads = MAX_WAIT_THREADS_DEFAULT * vp_count;
/* Assuming that each client session needs FD_PER_THREAD file descriptors */
if (max_threads == 0)
max_threads = (st_getfdlimit() * vp_count) / FD_PER_THREAD / sk_count;
if (max_wait_threads > max_threads)
max_wait_threads = max_threads;
/*
* Now calculate per-process values.
*/
if (max_wait_threads % vp_count)
max_wait_threads = max_wait_threads / vp_count + 1;
else
max_wait_threads = max_wait_threads / vp_count;
if (max_threads % vp_count)
max_threads = max_threads / vp_count + 1;
else
max_threads = max_threads / vp_count;
if (min_wait_threads > max_wait_threads)
min_wait_threads = max_wait_threads;
}
/******************************************************************/
static void create_listeners(void)
{
int i, n, sock;
char *c;
struct sockaddr_in serv_addr;
struct hostent *hp;
unsigned short port;
for (i = 0; i < sk_count; i++) {
port = 0;
if ((c = strchr(srv_socket[i].addr, ':')) != NULL) {
*c++ = '\0';
port = (unsigned short) atoi(c);
}
if (srv_socket[i].addr[0] == '\0')
srv_socket[i].addr = "0.0.0.0";
if (port == 0)
port = SERV_PORT_DEFAULT;
/* Create server socket */
if ((sock = socket(PF_INET, SOCK_STREAM, 0)) < 0)
err_sys_quit(errfd, "ERROR: can't create socket: socket");
n = 1;
if (setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, (char *)&n, sizeof(n)) < 0)
err_sys_quit(errfd, "ERROR: can't set SO_REUSEADDR: setsockopt");
memset(&serv_addr, 0, sizeof(serv_addr));
serv_addr.sin_family = AF_INET;
serv_addr.sin_port = htons(port);
serv_addr.sin_addr.s_addr = inet_addr(srv_socket[i].addr);
if (serv_addr.sin_addr.s_addr == INADDR_NONE) {
/* not dotted-decimal */
if ((hp = gethostbyname(srv_socket[i].addr)) == NULL)
err_quit(errfd, "ERROR: can't resolve address: %s",
srv_socket[i].addr);
memcpy(&serv_addr.sin_addr, hp->h_addr, hp->h_length);
}
srv_socket[i].port = port;
/* Do bind and listen */
if (bind(sock, (struct sockaddr *)&serv_addr, sizeof(serv_addr)) < 0)
err_sys_quit(errfd, "ERROR: can't bind to address %s, port %hu",
srv_socket[i].addr, port);
if (listen(sock, listenq_size) < 0)
err_sys_quit(errfd, "ERROR: listen");
/* Create file descriptor object from OS socket */
if ((srv_socket[i].nfd = st_netfd_open_socket(sock)) == NULL)
err_sys_quit(errfd, "ERROR: st_netfd_open_socket");
/*
* On some platforms (e.g. IRIX, Linux) accept() serialization is never
* needed for any OS version. In that case st_netfd_serialize_accept()
* is just a no-op. Also see the comment above.
*/
if (serialize_accept && st_netfd_serialize_accept(srv_socket[i].nfd) < 0)
err_sys_quit(errfd, "ERROR: st_netfd_serialize_accept");
}
}
/******************************************************************/
static void change_user(void)
{
struct passwd *pw;
if ((pw = getpwnam(username)) == NULL)
err_quit(errfd, "ERROR: can't find user '%s': getpwnam failed", username);
if (setgid(pw->pw_gid) < 0)
err_sys_quit(errfd, "ERROR: can't change group id: setgid");
if (setuid(pw->pw_uid) < 0)
err_sys_quit(errfd, "ERROR: can't change user id: setuid");
err_report(errfd, "INFO: changed process user id to '%s'", username);
}
/******************************************************************/
static void open_log_files(void)
{
int fd;
char str[32];
if (interactive_mode)
return;
/* Open access log */
if (log_access)
logbuf_open();
/* Open and write pid to pid file */
if ((fd = open(PID_FILE, O_CREAT | O_WRONLY | O_TRUNC, 0644)) < 0)
err_sys_quit(errfd, "ERROR: can't open pid file: open");
sprintf(str, "%d\n", (int)getpid());
if (write(fd, str, strlen(str)) != strlen(str))
err_sys_quit(errfd, "ERROR: can't write to pid file: write");
close(fd);
/* Open error log file */
if ((fd = open(ERRORS_FILE, O_CREAT | O_WRONLY | O_APPEND, 0644)) < 0)
err_sys_quit(errfd, "ERROR: can't open error log file: open");
errfd = fd;
err_report(errfd, "INFO: starting the server...");
}
/******************************************************************/
static void start_processes(void)
{
int i, status;
pid_t pid;
sigset_t mask, omask;
if (interactive_mode) {
my_index = 0;
my_pid = getpid();
return;
}
for (i = 0; i < vp_count; i++) {
if ((pid = fork()) < 0) {
err_sys_report(errfd, "ERROR: can't create process: fork");
if (i == 0)
exit(1);
err_report(errfd, "WARN: started only %d processes out of %d", i,
vp_count);
vp_count = i;
break;
}
if (pid == 0) {
my_index = i;
my_pid = getpid();
/* Child returns to continue in main() */
return;
}
vp_pids[i] = pid;
}
/*
* Parent process becomes a "watchdog" and never returns to main().
*/
/* Install signal handlers */
Signal(SIGTERM, wdog_sighandler); /* terminate */
Signal(SIGHUP, wdog_sighandler); /* restart */
Signal(SIGUSR1, wdog_sighandler); /* dump info */
/* Now go to sleep waiting for a child termination or a signal */
for ( ; ; ) {
if ((pid = wait(&status)) < 0) {
if (errno == EINTR)
continue;
err_sys_quit(errfd, "ERROR: watchdog: wait");
}
/* Find index of the exited child */
for (i = 0; i < vp_count; i++) {
if (vp_pids[i] == pid)
break;
}
/* Block signals while printing and forking */
sigemptyset(&mask);
sigaddset(&mask, SIGTERM);
sigaddset(&mask, SIGHUP);
sigaddset(&mask, SIGUSR1);
sigprocmask(SIG_BLOCK, &mask, &omask);
if (WIFEXITED(status))
err_report(errfd, "WARN: watchdog: process %d (pid %d) exited"
" with status %d", i, pid, WEXITSTATUS(status));
else if (WIFSIGNALED(status))
err_report(errfd, "WARN: watchdog: process %d (pid %d) terminated"
" by signal %d", i, pid, WTERMSIG(status));
else if (WIFSTOPPED(status))
err_report(errfd, "WARN: watchdog: process %d (pid %d) stopped"
" by signal %d", i, pid, WSTOPSIG(status));
else
err_report(errfd, "WARN: watchdog: process %d (pid %d) terminated:"
" unknown termination reason", i, pid);
/* Fork another VP */
if ((pid = fork()) < 0) {
err_sys_report(errfd, "ERROR: watchdog: can't create process: fork");
} else if (pid == 0) {
my_index = i;
my_pid = getpid();
/* Child returns to continue in main() */
return;
}
vp_pids[i] = pid;
/* Restore the signal mask */
sigprocmask(SIG_SETMASK, &omask, NULL);
}
}
/******************************************************************/
static void wdog_sighandler(int signo)
{
int i, err;
/* Save errno */
err = errno;
/* Forward the signal to all children */
for (i = 0; i < vp_count; i++) {
if (vp_pids[i] > 0)
kill(vp_pids[i], signo);
}
/*
* It is safe to do pretty much everything here because process is
* sleeping in wait() which is async-safe.
*/
switch (signo) {
case SIGHUP:
err_report(errfd, "INFO: watchdog: caught SIGHUP");
/* Reopen log files - needed for log rotation */
if (log_access) {
logbuf_close();
logbuf_open();
}
close(errfd);
if ((errfd = open(ERRORS_FILE, O_CREAT | O_WRONLY | O_APPEND, 0644)) < 0)
err_sys_quit(STDERR_FILENO, "ERROR: watchdog: open");
break;
case SIGTERM:
/* Non-graceful termination */
err_report(errfd, "INFO: watchdog: caught SIGTERM, terminating");
unlink(PID_FILE);
exit(0);
case SIGUSR1:
err_report(errfd, "INFO: watchdog: caught SIGUSR1");
break;
default:
err_report(errfd, "INFO: watchdog: caught signal %d", signo);
}
/* Restore errno */
errno = err;
}
/******************************************************************/
static void install_sighandlers(void)
{
sigset_t mask;
int p[2];
/* Create signal pipe */
if (pipe(p) < 0)
err_sys_quit(errfd, "ERROR: process %d (pid %d): can't create"
" signal pipe: pipe", my_index, my_pid);
if ((sig_pipe[0] = st_netfd_open(p[0])) == NULL ||
(sig_pipe[1] = st_netfd_open(p[1])) == NULL)
err_sys_quit(errfd, "ERROR: process %d (pid %d): can't create"
" signal pipe: st_netfd_open", my_index, my_pid);
/* Install signal handlers */
Signal(SIGTERM, child_sighandler); /* terminate */
Signal(SIGHUP, child_sighandler); /* restart */
Signal(SIGUSR1, child_sighandler); /* dump info */
/* Unblock signals */
sigemptyset(&mask);
sigaddset(&mask, SIGTERM);
sigaddset(&mask, SIGHUP);
sigaddset(&mask, SIGUSR1);
sigprocmask(SIG_UNBLOCK, &mask, NULL);
}
/******************************************************************/
static void child_sighandler(int signo)
{
int err, fd;
err = errno;
fd = st_netfd_fileno(sig_pipe[1]);
/* write() is async-safe */
if (write(fd, &signo, sizeof(int)) != sizeof(int))
err_sys_quit(errfd, "ERROR: process %d (pid %d): child's signal"
" handler: write", my_index, my_pid);
errno = err;
}
/******************************************************************
* The "main" function of the signal processing thread.
*/
/* ARGSUSED */
static void *process_signals(void *arg)
{
int signo;
for ( ; ; ) {
/* Read the next signal from the signal pipe */
if (st_read(sig_pipe[0], &signo, sizeof(int),
ST_UTIME_NO_TIMEOUT) != sizeof(int))
err_sys_quit(errfd, "ERROR: process %d (pid %d): signal processor:"
" st_read", my_index, my_pid);
switch (signo) {
case SIGHUP:
err_report(errfd, "INFO: process %d (pid %d): caught SIGHUP,"
" reloading configuration", my_index, my_pid);
if (interactive_mode) {
load_configs();
break;
}
/* Reopen log files - needed for log rotation */
if (log_access) {
logbuf_flush();
logbuf_close();
logbuf_open();
}
close(errfd);
if ((errfd = open(ERRORS_FILE, O_CREAT | O_WRONLY | O_APPEND, 0644)) < 0)
err_sys_quit(STDERR_FILENO, "ERROR: process %d (pid %d): signal"
" processor: open", my_index, my_pid);
/* Reload configuration */
load_configs();
break;
case SIGTERM:
/*
* Terminate ungracefully since it is generally not known how long
* it will take to gracefully complete all client sessions.
*/
err_report(errfd, "INFO: process %d (pid %d): caught SIGTERM,"
" terminating", my_index, my_pid);
if (log_access)
logbuf_flush();
exit(0);
case SIGUSR1:
err_report(errfd, "INFO: process %d (pid %d): caught SIGUSR1",
my_index, my_pid);
/* Print server info to stderr */
dump_server_info();
break;
default:
err_report(errfd, "INFO: process %d (pid %d): caught signal %d",
my_index, my_pid, signo);
}
}
/* NOTREACHED */
return NULL;
}
/******************************************************************
* The "main" function of the access log flushing thread.
*/
/* ARGSUSED */
static void *flush_acclog_buffer(void *arg)
{
for ( ; ; ) {
st_sleep(ACCLOG_FLUSH_INTERVAL);
logbuf_flush();
}
/* NOTREACHED */
return NULL;
}
/******************************************************************/
static void start_threads(void)
{
long i, n;
/* Create access log flushing thread */
if (log_access && st_thread_create(flush_acclog_buffer, NULL, 0, 0) == NULL)
err_sys_quit(errfd, "ERROR: process %d (pid %d): can't create"
" log flushing thread", my_index, my_pid);
/* Create connections handling threads */
for (i = 0; i < sk_count; i++) {
err_report(errfd, "INFO: process %d (pid %d): starting %d threads"
" on %s:%u", my_index, my_pid, max_wait_threads,
srv_socket[i].addr, srv_socket[i].port);
WAIT_THREADS(i) = 0;
BUSY_THREADS(i) = 0;
RQST_COUNT(i) = 0;
for (n = 0; n < max_wait_threads; n++) {
if (st_thread_create(handle_connections, (void *)i, 0, 0) != NULL)
WAIT_THREADS(i)++;
else
err_sys_report(errfd, "ERROR: process %d (pid %d): can't create"
" thread", my_index, my_pid);
}
if (WAIT_THREADS(i) == 0)
exit(1);
}
}
/******************************************************************/
static void *handle_connections(void *arg)
{
st_netfd_t srv_nfd, cli_nfd;
struct sockaddr_in from;
int fromlen;
long i = (long) arg;
srv_nfd = srv_socket[i].nfd;
fromlen = sizeof(from);
while (WAIT_THREADS(i) <= max_wait_threads) {
cli_nfd = st_accept(srv_nfd, (struct sockaddr *)&from, &fromlen,
ST_UTIME_NO_TIMEOUT);
if (cli_nfd == NULL) {
err_sys_report(errfd, "ERROR: can't accept connection: st_accept");
continue;
}
/* Save peer address, so we can retrieve it later */
st_netfd_setspecific(cli_nfd, &from.sin_addr, NULL);
WAIT_THREADS(i)--;
BUSY_THREADS(i)++;
if (WAIT_THREADS(i) < min_wait_threads && TOTAL_THREADS(i) < max_threads) {
/* Create another spare thread */
if (st_thread_create(handle_connections, (void *)i, 0, 0) != NULL)
WAIT_THREADS(i)++;
else
err_sys_report(errfd, "ERROR: process %d (pid %d): can't create"
" thread", my_index, my_pid);
}
handle_session(i, cli_nfd);
st_netfd_close(cli_nfd);
WAIT_THREADS(i)++;
BUSY_THREADS(i)--;
}
WAIT_THREADS(i)--;
return NULL;
}
/******************************************************************/
static void dump_server_info(void)
{
char *buf;
int i, len;
if ((buf = malloc(sk_count * 512)) == NULL) {
err_sys_report(errfd, "ERROR: malloc failed");
return;
}
len = sprintf(buf, "\n\nProcess #%d (pid %d):\n", my_index, (int)my_pid);
for (i = 0; i < sk_count; i++) {
len += sprintf(buf + len, "\nListening Socket #%d:\n"
"-------------------------\n"
"Address %s:%u\n"
"Thread limits (min/max) %d/%d\n"
"Waiting threads %d\n"
"Busy threads %d\n"
"Requests served %d\n",
i, srv_socket[i].addr, srv_socket[i].port,
max_wait_threads, max_threads,
WAIT_THREADS(i), BUSY_THREADS(i), RQST_COUNT(i));
}
write(STDERR_FILENO, buf, len);
free(buf);
}
/******************************************************************
* Stubs
*/
/*
* Session handling function stub. Just dumps small HTML page.
*/
void handle_session(long srv_socket_index, st_netfd_t cli_nfd)
{
static char resp[] = "HTTP/1.0 200 OK\r\nContent-type: text/html\r\n"
"Connection: close\r\n\r\n<H2>It worked!</H2>\n";
char buf[512];
int n = sizeof(resp) - 1;
struct in_addr *from = st_netfd_getspecific(cli_nfd);
if (st_read(cli_nfd, buf, sizeof(buf), SEC2USEC(REQUEST_TIMEOUT)) < 0) {
err_sys_report(errfd, "WARN: can't read request from %s: st_read",
inet_ntoa(*from));
return;
}
if (st_write(cli_nfd, resp, n, ST_UTIME_NO_TIMEOUT) != n) {
err_sys_report(errfd, "WARN: can't write response to %s: st_write",
inet_ntoa(*from));
return;
}
RQST_COUNT(srv_socket_index)++;
}
/*
* Configuration loading function stub.
*/
void load_configs(void)
{
err_report(errfd, "INFO: process %d (pid %d): configuration loaded",
my_index, my_pid);
}
/*
* Buffered access logging methods.
* Note that stdio functions (fopen(3), fprintf(3), fflush(3), etc.) cannot
* be used if multiple VPs are created since these functions can flush buffer
* at any point and thus write only partial log record to disk.
* Also, it is completely safe for all threads of the same VP to write to
* the same log buffer without any mutex protection (one buffer per VP, of
* course).
*/
void logbuf_open(void)
{
}
void logbuf_flush(void)
{
}
void logbuf_close(void)
{
}
/******************************************************************
* Small utility functions
*/
static void Signal(int sig, void (*handler)(int))
{
struct sigaction sa;
sa.sa_handler = handler;
sigemptyset(&sa.sa_mask);
sa.sa_flags = 0;
sigaction(sig, &sa, NULL);
}
static int cpu_count(void)
{
int n;
#if defined (_SC_NPROCESSORS_ONLN)
n = (int) sysconf(_SC_NPROCESSORS_ONLN);
#elif defined (_SC_NPROC_ONLN)
n = (int) sysconf(_SC_NPROC_ONLN);
#elif defined (HPUX)
#include <sys/mpctl.h>
n = mpctl(MPC_GETNUMSPUS, 0, 0);
#else
n = -1;
errno = ENOSYS;
#endif
return n;
}
/******************************************************************/
#
# Portions created by SGI are Copyright (C) 2000 Silicon Graphics, Inc.
# All Rights Reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
#
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# 3. Neither the name of Silicon Graphics, Inc. nor the names of its
# contributors may be used to endorse or promote products derived from
# this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# HOLDERS AND CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
# TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
CC = cc
SHELL = /bin/sh
ECHO = /bin/echo
DEPTH = ..
BUILD =
TARGETDIR = obj
DEFINES =
OTHER_FLAGS =
CFLAGS =
OBJDIR = $(DEPTH)/$(TARGETDIR)
INCDIR = $(DEPTH)/$(TARGETDIR)
LIBRESOLV =
EXTRALIBS =
SLIBRARY = $(OBJDIR)/libstx.a
OBJS = $(OBJDIR)/dnscache.o $(OBJDIR)/dnsres.o $(OBJDIR)/lrucache.o
CFLAGS += -Wall -I$(INCDIR)
AR = ar
ARFLAGS = rv
RANLIB = ranlib
##########################
# Platform section.
#
ifeq (LINUX, $(findstring LINUX, $(OS)))
LIBRESOLV = -lresolv
endif
ifeq ($(OS), SOLARIS)
LIBRESOLV = -lresolv
EXTRALIBS = -lsocket -lnsl
endif
#
# End of platform section.
##########################
all: $(SLIBRARY)
$(SLIBRARY): $(OBJS)
$(AR) $(ARFLAGS) $@ $(OBJS)
$(RANLIB) $@
$(OBJDIR)/%.o: %.c stx.h common.h
$(CC) $(CFLAGS) -c $< -o $@
clean:
rm -rf $(OBJS) $(SLIBRARY)
#.DEFAULT:
# @cd $(DEPTH); $(MAKE) $@
This directory contains extensions to the core State Threads Library
that were contributed by users. All files hereunder are not part of the
State Threads Library itself. They are provided as-is, without warranty
or support, and under whatever license terms their authors provided. To
contribute your own extensions, just mail them to the project
administrators or to one of the project's mailing lists; see
state-threads.sourceforge.net. Please indicate the license terms under
which the project may distribute your contribution.
========================================================================
stx_fileio
----------
Contributed by Jeff <jlb-st@houseofdistraction.com>, 4 Nov 2002.
Provides non-blocking random access file reading capability for
programs using the State Threads library. There is one public function:
ssize_t stx_file_read(st_netfd_t fd, off_t offset,
void *buf, size_t nbytes, st_utime_t timeout);
The implementation is not optimal in that the data is copied at least once
more than should be necessary. Its usefulness is limited to cases where
random access to a file is required and where starvation of other threads
is unacceptable.
The particular application which motivated this implementation was a UDP
file transfer protocol. Because the OS does very little buffering of UDP
traffic it is important that UDP transmission threads are not starved for
periods of time which are long relative to the interval required to
maintain a steady send rate.
Licensed under the same dual MPL/GPL as core State Threads.
========================================================================
stx_dns
-------
Documentation coming.
========================================================================
#ifndef _STX_COMMON_H_
#define _STX_COMMON_H_
#include <stddef.h>
#include <stdlib.h>
#define STX_BEGIN_MACRO {
#define STX_END_MACRO }
/*****************************************
* Circular linked list definitions
*/
typedef struct _stx_clist {
struct _stx_clist *next;
struct _stx_clist *prev;
} stx_clist_t;
/* Insert element "_e" into the list, before "_l" */
#define STX_CLIST_INSERT_BEFORE(_e,_l) \
STX_BEGIN_MACRO \
(_e)->next = (_l); \
(_e)->prev = (_l)->prev; \
(_l)->prev->next = (_e); \
(_l)->prev = (_e); \
STX_END_MACRO
/* Insert element "_e" into the list, after "_l" */
#define STX_CLIST_INSERT_AFTER(_e,_l) \
STX_BEGIN_MACRO \
(_e)->next = (_l)->next; \
(_e)->prev = (_l); \
(_l)->next->prev = (_e); \
(_l)->next = (_e); \
STX_END_MACRO
/* Append an element "_e" to the end of the list "_l" */
#define STX_CLIST_APPEND_LINK(_e,_l) STX_CLIST_INSERT_BEFORE(_e,_l)
/* Remove the element "_e" from it's circular list */
#define STX_CLIST_REMOVE_LINK(_e) \
STX_BEGIN_MACRO \
(_e)->prev->next = (_e)->next; \
(_e)->next->prev = (_e)->prev; \
STX_END_MACRO
/* Return the head/tail of the list */
#define STX_CLIST_HEAD(_l) (_l)->next
#define STX_CLIST_TAIL(_l) (_l)->prev
/* Return non-zero if the given circular list "_l" is empty, */
/* zero if the circular list is not empty */
#define STX_CLIST_IS_EMPTY(_l) \
((_l)->next == (_l))
/* Initialize a circular list */
#define STX_CLIST_INIT_CLIST(_l) \
STX_BEGIN_MACRO \
(_l)->next = (_l); \
(_l)->prev = (_l); \
STX_END_MACRO
/*****************************************
* Useful macros
*/
#ifndef offsetof
#define offsetof(type, identifier) ((size_t)&(((type *)0)->identifier))
#endif
#define STX_MIN(a, b) (((a) < (b)) ? (a) : (b))
#endif /* !_STX_COMMON_H_ */
#include "stx.h"
#include "common.h"
/*****************************************
* Basic types definitions
*/
typedef struct _stx_dns_data {
struct in_addr *addrs;
int num_addrs;
int cur;
time_t expires;
} stx_dns_data_t;
#define MAX_HOST_ADDRS 1024
static struct in_addr addr_list[MAX_HOST_ADDRS];
stx_cache_t *_stx_dns_cache = NULL;
extern int _stx_dns_ttl;
extern int _stx_dns_getaddrlist(const char *hostname, struct in_addr *addrs,
int *num_addrs, st_utime_t timeout);
static unsigned long hash_hostname(const void *key)
{
const char *name = (const char *)key;
unsigned long hash = 0;
while (*name)
hash = (hash << 4) - hash + *name++; /* hash = hash * 15 + *name++ */
return hash;
}
static void cleanup_entry(void *key, void *data)
{
if (key)
free(key);
if (data) {
if (((stx_dns_data_t *)data)->addrs)
free(((stx_dns_data_t *)data)->addrs);
free(data);
}
}
static int lookup_entry(const char *host, struct in_addr *addrs,
int *num_addrs, int rotate)
{
stx_cache_entry_t *entry;
stx_dns_data_t *data;
int n;
entry = stx_cache_entry_lookup(_stx_dns_cache, host);
if (entry) {
data = (stx_dns_data_t *)stx_cache_entry_getdata(entry);
if (st_time() <= data->expires) {
if (*num_addrs == 1) {
if (rotate) {
*addrs = data->addrs[data->cur++];
if (data->cur >= data->num_addrs)
data->cur = 0;
} else {
*addrs = data->addrs[0];
}
} else {
n = STX_MIN(*num_addrs, data->num_addrs);
memcpy(addrs, data->addrs, n * sizeof(*addrs));
*num_addrs = n;
}
stx_cache_entry_release(_stx_dns_cache, entry);
return 1;
}
/*
* Cache entry expired: decrement its refcount and purge it from cache.
*/
stx_cache_entry_release(_stx_dns_cache, entry);
stx_cache_entry_delete(_stx_dns_cache, entry);
}
return 0;
}
static void insert_entry(const char *host, struct in_addr *addrs, int count)
{
stx_cache_entry_t *entry;
stx_dns_data_t *data;
char *key;
size_t n;
if (_stx_dns_ttl > 0) {
key = strdup(host);
data = (stx_dns_data_t *)malloc(sizeof(stx_dns_data_t));
n = count * sizeof(*addrs);
if (data) {
data->addrs = (struct in_addr *)malloc(n);
if (data->addrs)
memcpy(data->addrs, addrs, n);
data->num_addrs = count;
data->cur = 0;
data->expires = st_time() + _stx_dns_ttl;
}
entry = stx_cache_entry_create(key, data, strlen(host) + 1 +
sizeof(stx_dns_data_t) + n +
stx_cache_entry_sizeof());
if (key && data && data->addrs && entry &&
stx_cache_entry_insert(_stx_dns_cache, entry) == 0) {
stx_cache_entry_release(_stx_dns_cache, entry);
return;
}
if (entry)
stx_cache_entry_delete(_stx_dns_cache, entry);
else
cleanup_entry(key, data);
}
}
int _stx_dns_cache_getaddrlist(const char *hostname, struct in_addr *addrs,
int *num_addrs, st_utime_t timeout,
int rotate)
{
char host[128];
int n, count;
if (!_stx_dns_cache)
return _stx_dns_getaddrlist(hostname, addrs, num_addrs, timeout);
for (n = 0; n < sizeof(host) - 1 && hostname[n]; n++) {
host[n] = tolower(hostname[n]);
}
host[n] = '\0';
if (lookup_entry(host, addrs, num_addrs, rotate))
return 0;
count = MAX_HOST_ADDRS;
if (_stx_dns_getaddrlist(host, addr_list, &count, timeout) < 0)
return -1;
n = STX_MIN(*num_addrs, count);
memcpy(addrs, addr_list, n * sizeof(*addrs));
*num_addrs = n;
insert_entry(host, addr_list, count);
return 0;
}
int stx_dns_cache_init(size_t max_size, size_t max_bytes, size_t hash_size)
{
_stx_dns_cache = stx_cache_create(max_size, max_bytes, hash_size,
hash_hostname,
(long (*)(const void *, const void *))strcmp,
cleanup_entry);
if (!_stx_dns_cache)
return -1;
return 0;
}
void stx_dns_cache_getinfo(stx_cache_info_t *info)
{
if (_stx_dns_cache)
stx_cache_getinfo(_stx_dns_cache, info);
else
memset(info, 0, sizeof(stx_cache_info_t));
}
int stx_dns_getaddrlist(const char *hostname, struct in_addr *addrs,
int *num_addrs, st_utime_t timeout)
{
return _stx_dns_cache_getaddrlist(hostname, addrs, num_addrs, timeout, 0);
}
int stx_dns_getaddr(const char *hostname, struct in_addr *addr,
st_utime_t timeout)
{
int n = 1;
return _stx_dns_cache_getaddrlist(hostname, addr, &n, timeout, 1);
}
/*
* Copyright (c) 1985, 1988, 1993
* The Regents of the University of California. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* 3. All advertising materials mentioning features or use of this software
* must display the following acknowledgement:
* This product includes software developed by the University of
* California, Berkeley and its contributors.
* 4. Neither the name of the University nor the names of its contributors
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*
* Portions created by SGI are Copyright (C) 2000 Silicon Graphics, Inc.
* All Rights Reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
*
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* 3. Neither the name of Silicon Graphics, Inc. nor the names of its
* contributors may be used to endorse or promote products derived from
* this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
* ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
* HOLDERS AND CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
* TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include "stx.h"
#define MAXPACKET 1024
#if !defined(NETDB_INTERNAL) && defined(h_NETDB_INTERNAL)
#define NETDB_INTERNAL h_NETDB_INTERNAL
#endif
/* New in Solaris 7 */
#if !defined(_getshort) && defined(ns_get16)
#define _getshort(cp) ns_get16(cp)
#define _getlong(cp) ns_get32(cp)
#endif
typedef union {
HEADER hdr;
u_char buf[MAXPACKET];
} querybuf_t;
int _stx_dns_ttl;
static int parse_answer(querybuf_t *ans, int len, struct in_addr *addrs,
int *num_addrs)
{
char buf[MAXPACKET];
HEADER *ahp;
u_char *cp, *eoa;
int type, n, i;
ahp = &ans->hdr;
eoa = ans->buf + len;
cp = ans->buf + sizeof(HEADER);
h_errno = TRY_AGAIN;
_stx_dns_ttl = -1;
i = 0;
while (ahp->qdcount > 0) {
ahp->qdcount--;
cp += dn_skipname(cp, eoa) + QFIXEDSZ;
}
while (ahp->ancount > 0 && cp < eoa && i < *num_addrs) {
ahp->ancount--;
if ((n = dn_expand(ans->buf, eoa, cp, buf, sizeof(buf))) < 0)
return -1;
cp += n;
if (cp + 4 + 4 + 2 >= eoa)
return -1;
type = _getshort(cp);
cp += 4;
if (type == T_A)
_stx_dns_ttl = _getlong(cp);
cp += 4;
n = _getshort(cp);
cp += 2;
if (type == T_A) {
if (n > sizeof(*addrs) || cp + n > eoa)
return -1;
memcpy(&addrs[i++], cp, n);
}
cp += n;
}
*num_addrs = i;
return 0;
}
static int query_domain(st_netfd_t nfd, const char *name,
struct in_addr *addrs, int *num_addrs,
st_utime_t timeout)
{
querybuf_t qbuf;
u_char *buf = qbuf.buf;
HEADER *hp = &qbuf.hdr;
int blen = sizeof(qbuf);
int i, len, id;
for (i = 0; i < _res.nscount; i++) {
len = res_mkquery(QUERY, name, C_IN, T_A, NULL, 0, NULL, buf, blen);
if (len <= 0) {
h_errno = NO_RECOVERY;
return -1;
}
id = hp->id;
if (st_sendto(nfd, buf, len, (struct sockaddr *)&(_res.nsaddr_list[i]),
sizeof(struct sockaddr), timeout) != len) {
h_errno = NETDB_INTERNAL;
/* EINTR means interrupt by other thread, NOT by a caught signal */
if (errno == EINTR)
return -1;
continue;
}
/* Wait for reply */
do {
len = st_recvfrom(nfd, buf, blen, NULL, NULL, timeout);
if (len <= 0)
break;
} while (id != hp->id);
if (len < HFIXEDSZ) {
h_errno = NETDB_INTERNAL;
if (len >= 0)
errno = EMSGSIZE;
else if (errno == EINTR) /* see the comment above */
return -1;
continue;
}
hp->ancount = ntohs(hp->ancount);
hp->qdcount = ntohs(hp->qdcount);
if ((hp->rcode != NOERROR) || (hp->ancount == 0)) {
switch (hp->rcode) {
case NXDOMAIN:
h_errno = HOST_NOT_FOUND;
break;
case SERVFAIL:
h_errno = TRY_AGAIN;
break;
case NOERROR:
h_errno = NO_DATA;
break;
case FORMERR:
case NOTIMP:
case REFUSED:
default:
h_errno = NO_RECOVERY;
}
continue;
}
if (parse_answer(&qbuf, len, addrs, num_addrs) == 0)
return 0;
}
return -1;
}
#define CLOSE_AND_RETURN(ret) \
{ \
n = errno; \
st_netfd_close(nfd); \
errno = n; \
return (ret); \
}
int _stx_dns_getaddrlist(const char *host, struct in_addr *addrs,
int *num_addrs, st_utime_t timeout)
{
char name[MAXDNAME], **domain;
const char *cp;
int s, n, maxlen, dots;
int trailing_dot, tried_as_is;
st_netfd_t nfd;
if ((_res.options & RES_INIT) == 0 && res_init() == -1) {
h_errno = NETDB_INTERNAL;
return -1;
}
if (_res.options & RES_USEVC) {
h_errno = NETDB_INTERNAL;
errno = ENOSYS;
return -1;
}
if (!host || *host == '\0') {
h_errno = HOST_NOT_FOUND;
return -1;
}
/* Create UDP socket */
if ((s = socket(PF_INET, SOCK_DGRAM, 0)) < 0) {
h_errno = NETDB_INTERNAL;
return -1;
}
if ((nfd = st_netfd_open_socket(s)) == NULL) {
h_errno = NETDB_INTERNAL;
n = errno;
close(s);
errno = n;
return -1;
}
maxlen = sizeof(name) - 1;
n = 0;
dots = 0;
trailing_dot = 0;
tried_as_is = 0;
for (cp = host; *cp && n < maxlen; cp++) {
dots += (*cp == '.');
name[n++] = *cp;
}
if (name[n - 1] == '.')
trailing_dot = 1;
/*
* If there are dots in the name already, let's just give it a try
* 'as is'. The threshold can be set with the "ndots" option.
*/
if (dots >= _res.ndots) {
if (query_domain(nfd, host, addrs, num_addrs, timeout) == 0)
CLOSE_AND_RETURN(0);
if (h_errno == NETDB_INTERNAL && errno == EINTR)
CLOSE_AND_RETURN(-1);
tried_as_is = 1;
}
/*
* We do at least one level of search if
* - there is no dot and RES_DEFNAME is set, or
* - there is at least one dot, there is no trailing dot,
* and RES_DNSRCH is set.
*/
if ((!dots && (_res.options & RES_DEFNAMES)) ||
(dots && !trailing_dot && (_res.options & RES_DNSRCH))) {
name[n++] = '.';
for (domain = _res.dnsrch; *domain; domain++) {
strncpy(name + n, *domain, maxlen - n);
if (query_domain(nfd, name, addrs, num_addrs, timeout) == 0)
CLOSE_AND_RETURN(0);
if (h_errno == NETDB_INTERNAL && errno == EINTR)
CLOSE_AND_RETURN(-1);
if (!(_res.options & RES_DNSRCH))
break;
}
}
/*
* If we have not already tried the name "as is", do that now.
* note that we do this regardless of how many dots were in the
* name or whether it ends with a dot.
*/
if (!tried_as_is) {
if (query_domain(nfd, host, addrs, num_addrs, timeout) == 0)
CLOSE_AND_RETURN(0);
}
CLOSE_AND_RETURN(-1);
}
#include "stx.h"
#include "common.h"
/*****************************************
* Basic types definitions
*/
struct _stx_centry {
void *key; /* key for doing lookups */
void *data; /* data in the cache */
size_t weight; /* "weight" of this entry */
struct _stx_centry *next; /* next entry */
struct _stx_centry **pthis;
stx_clist_t lru_link; /* for putting this entry on LRU list */
int ref_count; /* use count for this entry */
int delete_pending; /* pending delete flag */
};
struct _stx_cache {
size_t max_size; /* max size of cache */
size_t cur_size; /* current size of cache */
size_t max_weight; /* cache capacity */
size_t cur_weight; /* current total "weight" of all entries */
size_t hash_size; /* size of hash table */
stx_cache_entry_t **table; /* hash table for this cache */
stx_clist_t lru_list; /* least-recently-used list */
/* Cache stats */
unsigned long hits; /* num cache hits */
unsigned long lookups; /* num cache lookups */
unsigned long inserts; /* num inserts */
unsigned long deletes; /* num deletes */
/* Functions */
unsigned long (*key_hash_fn)(const void *);
long (*key_cmp_fn)(const void *, const void *);
void (*cleanup_fn)(void *, void *);
};
#define STX_CACHE_ENTRY_PTR(_qp) \
((stx_cache_entry_t *)((char *)(_qp) - offsetof(stx_cache_entry_t, lru_link)))
/*****************************************
* Cache methods
*/
stx_cache_t *stx_cache_create(size_t max_size, size_t max_weight,
size_t hash_size,
unsigned long (*key_hash_fn)(const void *key),
long (*key_cmp_fn)(const void *key1,
const void *key2),
void (*cleanup_fn)(void *key, void *data))
{
stx_cache_t *newcache;
newcache = (stx_cache_t *)calloc(1, sizeof(stx_cache_t));
if (newcache == NULL)
return NULL;
newcache->table = (stx_cache_entry_t **)calloc(hash_size,
sizeof(stx_cache_entry_t *));
if (newcache->table == NULL) {
free(newcache);
return NULL;
}
newcache->max_size = max_size;
newcache->max_weight = max_weight;
newcache->hash_size = hash_size;
STX_CLIST_INIT_CLIST(&(newcache->lru_list));
newcache->key_hash_fn = key_hash_fn;
newcache->key_cmp_fn = key_cmp_fn;
newcache->cleanup_fn = cleanup_fn;
return newcache;
}
void stx_cache_empty(stx_cache_t *cache)
{
size_t i;
stx_cache_entry_t *entry, *next_entry;
for (i = 0; i < cache->hash_size; i++) {
entry = cache->table[i];
while (entry) {
next_entry = entry->next;
stx_cache_entry_delete(cache, entry);
entry = next_entry;
}
}
}
void stx_cache_traverse(stx_cache_t *cache,
void (*callback)(void *key, void *data))
{
size_t i;
stx_cache_entry_t *entry;
for (i = 0; i < cache->hash_size; i++) {
for (entry = cache->table[i]; entry; entry = entry->next) {
if (!entry->delete_pending)
(*callback)(entry->key, entry->data);
}
}
}
void stx_cache_traverse_lru(stx_cache_t *cache,
void (*callback)(void *key, void *data),
unsigned int n)
{
stx_clist_t *q;
stx_cache_entry_t *entry;
for (q = STX_CLIST_HEAD(&cache->lru_list); q != &cache->lru_list && n;
q = q->next, n--) {
entry = STX_CACHE_ENTRY_PTR(q);
(*callback)(entry->key, entry->data);
}
}
void stx_cache_traverse_mru(stx_cache_t *cache,
void (*callback)(void *key, void *data),
unsigned int n)
{
stx_clist_t *q;
stx_cache_entry_t *entry;
for (q = STX_CLIST_TAIL(&cache->lru_list); q != &cache->lru_list && n;
q = q->prev, n--) {
entry = STX_CACHE_ENTRY_PTR(q);
(*callback)(entry->key, entry->data);
}
}
size_t stx_cache_getsize(stx_cache_t *cache)
{
return cache->cur_size;
}
size_t stx_cache_getweight(stx_cache_t *cache)
{
return cache->cur_weight;
}
void stx_cache_getinfo(stx_cache_t *cache, stx_cache_info_t *info)
{
info->max_size = cache->max_size;
info->max_weight = cache->max_weight;
info->hash_size = cache->hash_size;
info->cur_size = cache->cur_size;
info->cur_weight = cache->cur_weight;
info->hits = cache->hits;
info->lookups = cache->lookups;
info->inserts = cache->inserts;
info->deletes = cache->deletes;
}
/*****************************************
* Cache entry methods
*/
stx_cache_entry_t *stx_cache_entry_create(void *key, void *data,
size_t weight)
{
stx_cache_entry_t *newentry;
newentry = (stx_cache_entry_t *)calloc(1, sizeof(stx_cache_entry_t));
if (newentry == NULL)
return NULL;
newentry->key = key;
newentry->data = data;
newentry->weight = weight;
return newentry;
}
void stx_cache_entry_delete(stx_cache_t *cache, stx_cache_entry_t *entry)
{
entry->delete_pending = 1;
if (entry->ref_count > 0)
return;
if (entry->pthis) {
*entry->pthis = entry->next;
if (entry->next)
entry->next->pthis = entry->pthis;
cache->cur_size--;
cache->cur_weight -= entry->weight;
cache->deletes++;
STX_CLIST_REMOVE_LINK(&(entry->lru_link));
}
if (cache->cleanup_fn)
cache->cleanup_fn(entry->key, entry->data);
entry->pthis = NULL;
entry->key = NULL;
entry->data = NULL;
free(entry);
}
stx_cache_entry_t *stx_cache_entry_lookup(stx_cache_t *cache, const void *key)
{
unsigned long bucket;
stx_cache_entry_t *entry;
cache->lookups++;
bucket = cache->key_hash_fn(key) % cache->hash_size;
for (entry = cache->table[bucket]; entry; entry = entry->next) {
if (!entry->delete_pending && cache->key_cmp_fn(key, entry->key) == 0)
break;
}
if (entry) {
cache->hits++;
if (entry->ref_count == 0)
STX_CLIST_REMOVE_LINK(&(entry->lru_link));
entry->ref_count++;
}
return entry;
}
void stx_cache_entry_release(stx_cache_t *cache, stx_cache_entry_t *entry)
{
if (entry->ref_count == 0)
return;
entry->ref_count--;
if (entry->ref_count == 0) {
STX_CLIST_APPEND_LINK(&(entry->lru_link), &(cache->lru_list));
if (entry->delete_pending)
stx_cache_entry_delete(cache, entry);
}
}
int stx_cache_entry_insert(stx_cache_t *cache, stx_cache_entry_t *entry)
{
stx_cache_entry_t *old_entry;
unsigned long bucket;
/*
* If cache capacity is exceeded, try to remove LRU entries till there is
* enough room or LRU list is empty.
*/
while (cache->cur_weight + entry->weight > cache->max_weight) {
old_entry = stx_cache_entry_getlru(cache);
if (!old_entry) {
/* cache capacity is exceeded and all entries are in use */
return -1;
}
stx_cache_entry_delete(cache, old_entry);
}
/* If cache size is exceeded, remove LRU entry */
if (cache->cur_size >= cache->max_size) {
old_entry = stx_cache_entry_getlru(cache);
if (!old_entry) {
/* cache size is exceeded and all entries are in use */
return -1;
}
stx_cache_entry_delete(cache, old_entry);
}
/* Don't add duplicate entries in the cache */
bucket = cache->key_hash_fn(entry->key) % cache->hash_size;
for (old_entry = cache->table[bucket]; old_entry;
old_entry = old_entry->next) {
if (!old_entry->delete_pending &&
cache->key_cmp_fn(entry->key, old_entry->key) == 0)
break;
}
if (old_entry)
stx_cache_entry_delete(cache, old_entry);
/* Insert in the hash table */
entry->next = cache->table[bucket];
cache->table[bucket] = entry;
entry->pthis = &cache->table[bucket];
if (entry->next)
entry->next->pthis = &entry->next;
entry->ref_count++;
cache->inserts++;
cache->cur_size++;
cache->cur_weight += entry->weight;
return 0;
}
stx_cache_entry_t *stx_cache_entry_getlru(stx_cache_t *cache)
{
if (STX_CLIST_IS_EMPTY(&(cache->lru_list)))
return NULL;
return STX_CACHE_ENTRY_PTR(STX_CLIST_HEAD(&(cache->lru_list)));
}
int stx_cache_entry_sizeof(void)
{
return (int)sizeof(stx_cache_entry_t);
}
void *stx_cache_entry_getdata(stx_cache_entry_t *entry)
{
return entry->data;
}
void *stx_cache_entry_getkey(stx_cache_entry_t *entry)
{
return entry->key;
}
size_t stx_cache_entry_getweight(stx_cache_entry_t *entry)
{
return entry->weight;
}