Tokyo Tyrant, networking for Tokyo Cabinet

Mikio made a networking layer for Tokyo Cabinet, the fastest lightweight database engine, I've mentioned before.

Tokyo Tyrant is a packeage of network interfaces to the DBM called Tokyo Cabinet. Though the DBM has high performance, you might bother in case that multiple processes share the same database, or remote processes access the database. Thus, Tokyo Tyrant is provided for high concurrency connections to Tokyo Cabinet. It is composed of the server process managing a database and its access library for client applications.

The server features high concurrency due to thread-pool modeled implementation and the "epoll" mechanism of the modern Linux kernel. The server and its clients communicate with each other by two kinds of protocols. One has the original binary structure and provides higher efficiency. The other is based on HTTP and provides wider portability. The access library is implemented in C, Perl, Ruby, and Java.

Currently it works only on Linux, but Mikio is working on the BSD port as well.

Very simple web server for PHP development

I was looking for a lightweight, simple HTTP server to serve up static files form the current directory, for testing my ongoing web-development projects. It was pretty trivial to write one in Python, using SimpleHTTPServer class.

However when I had to test some PHP code, things got a little complicated. It was still quite easy to rewrite that small Python server with CGIHTTPServer to run a simple phpinfo() script, but that was far away to produce a complete CGI environment for running more complicated PHP scripts (with file uploads, for example).

I spent a whole evening looking for something similar that already implements it correctly. My big surprise, Ubuntu Gutsy is full of packaged simple web servers, but none of them supports PHP scripts with php-cgi out of the box.

The only one I found exciting is shttpd. It's available for Windows and Unix-like OSes. Compiling is trivial. Run it with:

shttpd -d /var/www -p 8080 -D 1 -c php -C /usr/lib/cgi-bin/php5 -l /dev/stdout -e /dev/stderr

Options:

-d /var/www

web root directory

-p 8080

listen on port 8080

-D 1

enable directory listing

-c php

CGI extensions

-C /usr/lib/cgi-bin/php5

CGI interpreter (php-cgi this time)

-l /dev/stdout

access log to stdout

-e /dev/stderr

error log to stderr

Tokyo Cabinet

Mikio Hirabayashi is known for QDBM embedded database library and Hyper Estrailer, an open-source search engine. He has a brand new project, called Tokyo Cabinet. It has several advantages over QDBM.

It produces smaller database files, processes them faster, has a more simple API, multi-threaded support, more robustness, no database corruption, 64-bit support. It still restricts a single process to access one database at a time, to user fixed key and value sizes.

Tokyo Cabinet is the fastest DB engine, I've ever seen. You really should check out the benchmarks here (PDF). A million of records can be stored in 1.5 seconds to a hash table, and in 2.2 seconds to a B+ tree. The Overhead is 16 bytes per record in hash, and 5 bytes for B+ tree.

Also Brian Aker is developing a MySQL storage engine on it.

Distributed Replicated Block Device

DRBD is a quite an interesting software for Linux. (It's now included in the "vanilla" kernel.) DRBD takes one of your block devices, and mirrors it to a remote computer's block device. It's something like a basic clustering on block level but with only two nodes.

You can create a journaling filesystem on top of a DRBD device, for example. When your main node dies, the secondary node can mount up that filesystem (after recovering it to a clean state from journal), and take over your task, as a cold backup. Once your ex-primary host come up again, it can now act as your slave, re-synchronizing the data for itself.

On DRBD, writes occur to the local and remote device simultaneously, however reads are made only on the local device, which is usually faster.

One can even use DRBD in a multi-master environment, if choosing shared-disk-filesystem, that supports simultaneous mounts on multiple hosts.

There is a nice article on Debian Administration, a good overview to demonstrate the possibilities when combining DRBD with other tools.