ClusterPUP - A Very Simple Cluster Implementation

ClusterPUP - A Very Simple Cluster Implementation

Note: This implemenation of a cluster is Using Puppy Linux

ClusterPUP - A Very Simple Cluster Implementation

Part 1
Introduction - A Plain Definition of a Cluster
Illustration of a Cluster
Part 2
How to get this implementation
Needed Package
Installation
Part 3
How to set up Puppy Cluster
Hardware
Setup a Cluster
References

Part 1
Introduction - A Plain Definition of a Cluster
This is NOT an official description. This is simply one view that attempts to describe what it means be a Cluster)

ClusterPUP - A Very Simple Cluster Implementation

A Cluster is a coupling of PCs together to work together as a team, in Parallel with each other in completing a
particular type of workload. Beowulf is one type of cluster. But, in general, all clusters bear some resemblance to
this type of cluster organization.
In the Beowulf world, typically you would have 1 PC that has a monitor, keyboard mouse and 2 LAN adapters,
while all of the other PCs DO NOT NEED THESE PERIPHERALS. They will only need a LAN card...ONLY!
The PC with the monitor, keyboard, mouse and 2 LAN adapters is referred to as the Master Node. This is because,
at this location, the Master Node, ALL workload that the Cluster team will work on, enters and exits the Cluster via
this Master Node. The Master Nodes primary function is to apportion the work out to its nodes on the LAN, stay in
touch with them, and collect (so to speak) their results from the work that they individually complete.
On the Hardware side, a very simple Cluster will have a Master Node which connects to a router somewhare on
your LAN; and additonally, it will have other node(s) which do NOT have monitors, keyboards, or mice attached
to them. All of a clusters PCs will have an Ethernet Switch which ONLY they are connected to. All of the clusters
work within the cluster is entered and exited the cluster via the Master Node.

ClusterPUP - A Very Simple Cluster Implementation

The Master Node and the cluster PCs keep constant communications with each other, physicallly thru the common
switch using a Protocol called MPI. This protocol gets implemented at each machines entry into the cluster via a
program called MPICH2. This protocol is a specially design protocol to be highly efficient in how these cluster
PCs stay in constant touch with each other.
Thats IT! No magic, Simply physically connect your PCs together on their own personal and separate switch, with
each of them running MPICH2.
Lastly, a clusters work/workload is different from your standard compiled programs. A clusters programs are
compiled in such a way that the program is modularized and components can be executed in parallel with each
other. This
affords the Master the ability to assign as many PCs it deems necessary to solve any individual problem or work. A
complied program will be started and the nodes will do all of the work and when complete, they will, all, check-in
the results of the work performed to the Master Node.

Part 2
How to get this implementation
Needed package and Installation
To turn puppy linux into beowulf cluster we will need some package installed, these are the needed package :
1. Passwordless ssh
2. Puppy linux devx.sfs
3. Python 2.6.4
4. fortran (optional)
5. MPICH2 (ver 1.3.2p1)
6. NFS
7. PXE
8. Script to set configuration
9. Petmaker to install sfs files

howto install them :

--------------------------------------1. Passwordless SSH
http://www.murga-linux.com/puppy/viewtopic.php?t=63523
1. install pet
2. extract tar.gz into root
----------------------------------------2. install Pet_Maker-1.7-3-430.pet
http://www.murga-linux.com/puppy/viewtopic.php?p=290171

ClusterPUP - A Very Simple Cluster Implementation

Using Pet_Maker you can install sfs files as easy as installing pet (the above version is what I use on
puppy 5.2, your puppy might need other version)
----------------------------------------3. devx.sfs
http://www.murga-linux.com/puppy/viewtopic.php?t=63251
----------------------------------------4. PYTHON (2.6.4)
http://www.murga-linux.com/puppy/viewtopic.php?p=385633&sid=c6c42672de38e064c800f31c1b1f2cef

----------------------------------------5. FORTRAN
- Install all .deb files included in the package
----------------------------------------6. MPICH2
http://www.mcs.anl.gov/research/projects/mpich2/
1. extract file mpic2-1.3.2p1 into root
2. open terminal, enter directory ----> cd ~/mpich2-1.3.2p1
3. type
./configure --with-pm=mpd --prefix=/root/mpich2-install 2>&1 | tee c.txt
make 2>&1 | tee m.txt
make V=1 2>&1 | tee m.txt
make install 2>&1 | tee mi.txt
4. add into .basrc (located at /root)
PATH=/root/mpich2-install/bin:$PATH ; export PATH
---------------------------------------7. NFS
UNFS (http://unfs3.sourceforge.net/)
and/or
Kernel NFS (http://www.murga-linux.com/puppy/viewtopic.php?t=45090)

- UNFS can be compiled to cluster mode...

1. extract UNFS source to root
2. open terminal, enter directory of extracted UNFS
3. type
./configure --enable-cluster
make
make install
4. copy or link /usr/local/sbin/unfsd into /root/my-applications/bin/
so program can be run.

ClusterPUP - A Very Simple Cluster Implementation

- KNFS
install pet
---------------------------------------8. PXE
http://www.murga-linux.com/puppy/viewtopic.php?t=64107
install pet package
---------------------------------------9. Script
To initialize automatically you need scrip,I have included my script which although simple but work, my
script only initialize mpdboot so if you want to use hydra then you should try build your own script.
(on puppy 5.2., please delete /root/tftpboot first then extract my script, so my script can overwrite it)
the last thing to do is put desired script into startup, please review the script, edit as necessary and put into
/root/Startup folder.
---------------------------------------I have all those packages ready to download here (except devx.sfs) :
http://hotfile.com/dl/107929613/844c9bc/Puppy_cluster_pack.zip.html
---------------------------------------- After all above program are installed you will need to remaster to create iso file, when remaster please
remember to answer no if remaster-program ask if you want to create separate zxxxxxxx file.
- Please remember that it is better to have client pc identical to the server so when client pc boot via pxe,
it will boot without prompt to choose graphic resolution.
- One last reminder, Puppy cluster has less security due to combination of passwordless ssh and root
user. but as this OS was intended for special purpose, boot from cd, used in closed network and hopefully all
mounted partition point to empty harddisk or flashdisk, so I think it is ok as long as you didnt connect to Internet
with it.

Part 3
How to setup cluster puppy
Installation of a Cluster
Software
If you didnt bother to create your own Puppy-cluster, I have created Puppy 5.2 with MPICH2-1.3.2 which can be
downloaded here :
http://hotfile.com/dl/107415479/f443227/clusterpup-v2.zip.html.html

ClusterPUP - A Very Simple Cluster Implementation

Hardware
One of the most simple beowulf cluster is a closed network consist of 1 ethernet device and few computer
attached, there is no outside/internet connection.
You should have PC that has 1 LAN adapter in it, along with its keyboard, monitor, and mouse.
Connect all PC adapter, typically the one designated eth0, to your ethernet router.
Make sure each client pc has lan-boot capabilities in their bios enabled and made as 1st priority :
Go into the Bios of the Master Node and set it to boot from the CD/DVD device.
Go into the Bios of the cluster nodes and set them to boor from their LAN adapter.
Power off all cluster nodes
Put the ISO you burned from the Download into the Master Nodes CD or DVD reader.
Then you boot your first pc, this is your Master Node.

How to run it
(On Server)
1. screen after booting dont click ok yet (mount sda1 and sda5 which has linux partition)

ClusterPUP - A Very Simple Cluster Implementation

2. select the mknetboot buried behind those popup to create humongous initrd

ClusterPUP - A Very Simple Cluster Implementation

3. after humongous initrd created click ok

ClusterPUP - A Very Simple Cluster Implementation

4. Boot up all client pc then after all pc boot up properly into desktop then click yes

ClusterPUP - A Very Simple Cluster Implementation

5. ip address configured instantly, close editor and click ok

ClusterPUP - A Very Simple Cluster Implementation

6. after setting complete... run cluster

ClusterPUP - A Very Simple Cluster Implementation

(On Client)
1. Do nothing, if you have remastered correctly, you will boot right into desktop and there is nothing you need to
do in the client Pc

ClusterPUP - A Very Simple Cluster Implementation

References
Here is a Document by the Author of Beowulf covering the definition and structure of Beowulf Clustering.... It is
described here.
It is descibed here.
There is an International Spec with defines how the PCs in the cluster are to communicate with each other. It is
described here.
Also, here is a Document covering the Installation, Setup, Testing, and Use of protocol which implements the
International Spec, Beowulf and also here