This is the home page for BitCluster, a small set of tools for doing distributed computing. Thanks for stopping in.

What is BitCluster?

• BitCluster is a set of handy tools for clustered computers running Linux. There isn't that much here yet, but some day this list might expand.

Are people discussing this?

• Yes. There is a mailing list maintained on BitMover. Send majordomo@bitmover.com and ask to join the "clusters" list. The achives of the mailing list are kept here.

What are the tools?

• ctelnet is a program which starts multiple xterms with a command console. You can type in the command console and that typing gets sent to all of the xterms. ctelnet is actually a perl script that starts up the xterms and calls xfanout and fanout to do the X work and type-in-each work. It's an incredible hack, but actually quite useful. There is a ~80Kbyte 1279x1022 screen shot of 25 nodes in the Google web search cluster available if you want to see what it looks like. I'm at the end of a 48Kbyte/sec DSL line so go for it if you have a fast link. Here's another one another one testing out ftp.varesearch.com's internet connection. That's 24 nodes get about 40+Kbytes/sec each (for a total of 960Kbytes/sec) from the same Linux server. Impressive.
• xfanout is a tiny window manager, really. It tries to manage the N xterms that are started by ctelnet. When you move the xfanout window, it grabs the xterms and moves them as well. It also lets you toggle input to each window by clicking on it's name. It has bugs, however, the interaction with virtual window managers isn't great. Still a useful tool though.
• fanout is the low level program which accepts input and fans it out to multiple ttys.
• rexd is a replacement (not a compatible one) for Sun's remote execution daemon. It is a remote command executor which preserves, if possible, your current working directory, your environment variables, and std{in,out,err}. rexd does have mechanisms for limiting the security risk, read the docs or the source.
• on is the client interface to rexd; you use it like so

  on host command

  or

  groff -mgs paper.ms | on host lpr

  That last example is for lazy people like me that don't bother to set up remote printing.

• any is like on except that it consults a scheduler to figure out where to run the command. This is quite useful for things like parallel make. Suppose you have 4 hosts, host1, host2, host3, host4, which you want to use for distributed builds. You place those hosts in the queue (see the man page for ond) and modify your make command like so

  make -j8 CC="any gcc"

  There are a couple of things to note:

  • you want to stick each host in the queue twice, that way you can have up to two jobs running on each host concurrently. This will help hide I/O latency.
  • you want the -j8 because you had 4 hosts but you doubled them up, so there are 8.
  • you want to run the make command on the file server where the data is. NFS has open to close cache consistency so the remote hosts should push the .o's through to the server when the compiler closes them. The only place you'll get a consistent view of them is on the file server. If your link line in your make file is a "$(CC)" type, change it to "$(NOREMOTE_CC)" and make a target that sas NO_REMOTECC=gcc (or whatever your C compiler is).
• cstat is sort of a vmstat replacement except it include packet counts as well. Like so:

  load free cach swap pgin  pgou  dk0 dk1 dk2 dk3 ipkt opkt  int  ctx  usr sys idl
  0.00  3.3 26.9  0.0  1.0K  1.0K   2   0   0   0  276  274  656  1.1K   1   3  96
  0.00  3.3 26.9  0.0    0     0    0   0   0   0  186  183  470  737    0   4  96
  0.00  3.3 26.9  0.0    0     0    0   0   0   0  171  170  442  678    1   3  96
  

  At some point in the future, I'll modify this to gather statistics from a cluster of hosts and display all of them.

How do I get the the tools?

• You can get an alpha version of BitCluster here.

Version 1.10 of index.html made on 2001/02/26 at 09:33:48