Setting up the cluster - Revision history

Wikiuser at 02:18, 13 April 2016

2016-04-13T02:18:44Z

← Older revision		Revision as of 02:18, 13 April 2016
Line 160:		Line 160:
	<h3>6. MPICH</h3>		<h3>6. MPICH</h3>
	<p>So there are two main options for setting up a message passing interface on your cluster as far as I can tell.  MPICH, and OpenMPI.  I am using MPICH.  Right before installing mpich2, we might want to install some other software.</p>		<p>So there are two main options for setting up a message passing interface on your cluster as far as I can tell.  MPICH, and OpenMPI.  I am using MPICH.  Right before installing mpich2, we might want to install some other software.</p>
	<p><code><span>beo@ub0:~$ </span>sudo apt-get install build-~~essentials~~ gfortran gfortran-multilib autoconf</code></p>		<p><code><span>beo@ub0:~$ </span>sudo apt-get install build-essential gfortran gfortran-multilib autoconf</code></p>
	<p>Install MPICH2.  Now.  There are two ways to do this.  The easy way, and the way that worked for me.  Here's the easy way.</p>		<p>Install MPICH2.  Now.  There are two ways to do this.  The easy way, and the way that worked for me.  Here's the easy way.</p>
	<p><code><span>beo@ub0:~$ </span>sudo apt-get install mpich2</code></p>		<p><code><span>beo@ub0:~$ </span>sudo apt-get install mpich2</code></p>

Wikiuser at 03:03, 24 February 2016

2016-02-24T03:03:32Z

← Older revision		Revision as of 03:03, 24 February 2016
Line 23:		Line 23:
	<ul>		<ul>
	<li>At the time of writing this, you can download Ubuntu 14.04 Server here.</li>		<li>At the time of writing this, you can download Ubuntu 14.04 Server here.</li>
			<li>http://www.ubuntu.com/download/server</li>
	<li>I'm sure that if a newer version comes out, it should work as well, although I tried initially with 15.10 (Desktop) and had some trouble with that.</li>		<li>I'm sure that if a newer version comes out, it should work as well, although I tried initially with 15.10 (Desktop) and had some trouble with that.</li>
	</ul>		</ul>

Wikiuser at 03:02, 24 February 2016

2016-02-24T03:02:38Z

← Older revision		Revision as of 03:02, 24 February 2016
Line 15:		Line 15:
	<h3>Let's Go</h3>		<h3>Let's Go</h3>
	<ol>		<ol>
	<li>Get a copy of Ubuntu 14.04 <b>Server</b>.  If you really want to, you can install a gui interface onto it later, like ~~<a href="http://~~unity~~.ubuntu.com/about">unity</a>~~.</li>		<li>Get a copy of Ubuntu 14.04 <b>Server</b>.  If you really want to, you can install a gui interface onto it later, like unity.</li>
	</ol>		</ol>
	<h3>1? Install Ubuntu</h3>		<h3>1? Install Ubuntu</h3>
Line 22:		Line 22:
	<li>Make the bootable USB.		<li>Make the bootable USB.
	<ul>		<ul>
	<li>At the time of writing this, you can download ~~<a href="http://www.ubuntu.com/download/alternative-downloads">~~Ubuntu 14.04 Server here~~</a>~~.</li>		<li>At the time of writing this, you can download Ubuntu 14.04 Server here.</li>
	<li>I'm sure that if a newer version comes out, it should work as well, although I tried initially with 15.10 (Desktop) and had some trouble with that.</li>		<li>I'm sure that if a newer version comes out, it should work as well, although I tried initially with 15.10 (Desktop) and had some trouble with that.</li>
	</ul>		</ul>
Line 166:		Line 166:
	<li>You will need a link to the mpich download.  At this time, the download is available at		<li>You will need a link to the mpich download.  At this time, the download is available at
	<ul>		<ul>
	<li>~~<a href="~~http://www.mpich.org/static/downloads/3.2/mpich-3.2.tar.gz~~">http://www.mpich.org/static/downloads/3.2/mpich-3.2.tar.gz</a>~~</li>		<li>http://www.mpich.org/static/downloads/3.2/mpich-3.2.tar.gz</li>
	</ul>		</ul>
	</li>		</li>
Line 179:		Line 179:
	ub3:4</pre>		ub3:4</pre>
	<p>Where we write the node name and how many processes we want to use separated by a colon.</p>		<p>Where we write the node name and how many processes we want to use separated by a colon.</p>
	<p>Great!  Now everything is set up and should be working, theoretically. Try to test it with this helloworld program from ~~<a href="https://help.ubuntu.com/community/MpichCluster">~~https://help.ubuntu.com/community/MpichCluster</a>.  Place the script in your home directory as "mpi_hello.c"</p>		<p>Great!  Now everything is set up and should be working, theoretically. Try to test it with this helloworld program from https://help.ubuntu.com/community/MpichCluster</a>.  Place the script in your home directory as "mpi_hello.c"</p>
	<pre>#include <stdio.h>		<pre>#include <stdio.h>
	#include <mpi.h>		#include <mpi.h>
Line 223:		Line 223:
	<ul>		<ul>
	<li>Installed manually.  Use the guide I posted.</li>		<li>Installed manually.  Use the guide I posted.</li>
	<li>~~<a href="~~http://www.mpich.org/static/downloads/3.1.3/mpich-3.1.3-installguide.pdf~~">http://www.mpich.org/static/downloads/3.1.3/mpich-3.1.3-installguide.pdf</a>~~</li>		<li>http://www.mpich.org/static/downloads/3.1.3/mpich-3.1.3-installguide.pdf</li>
	</ul>		</ul>
	</li>		</li>

Wikiuser at 18:58, 23 February 2016

2016-02-23T18:58:44Z

← Older revision		Revision as of 18:58, 23 February 2016
Line 7:		Line 7:
	<li>Also, if you want to look at other sources, these are two sources that were very helpful to me		<li>Also, if you want to look at other sources, these are two sources that were very helpful to me
	<ul>		<ul>
	<li>~~<a href="~~https://www-users.cs.york.ac.uk/~mjf/pi_cluster/src/Building_a_simple_Beowulf_cluster.html~~">https://www-users.cs.york.ac.uk/~mjf/pi_cluster/src/Building_a_simple_Beowulf_cluster.html</a>~~</li>		<li>https://www-users.cs.york.ac.uk/~mjf/pi_cluster/src/Building_a_simple_Beowulf_cluster.html</li>
	<li~~><a href="https://help.ubuntu.com/community/MpichCluster"~~>https://help.ubuntu.com/community/MpichCluster~~</a>~~</li>		<li>https://help.ubuntu.com/community/MpichCluster</li>
	</ul>		</ul>
	</li>		</li>
Line 112:		Line 112:
	<li>We will need to indicate which folder we want to share in our exports file.  Edit it with nano</li>		<li>We will need to indicate which folder we want to share in our exports file.  Edit it with nano</li>
	</ul>		</ul>
	<p~~><strong></strong~~><code>masternode: sudo nano /etc/exports</code></p>		<p><code>masternode: sudo nano /etc/exports</code></p>
	<pre>/home/beo *(rw, sync, no_subtree_check~~<span>~~)~~</span>~~</pre>		<pre>/home/beo *(rw, sync, no_subtree_check)</pre>
	<p>Add the above line to the bottom of /etc/exports and restart the server</p>		<p>Add the above line to the bottom of /etc/exports and restart the server</p>
	<p><code>masternode: sudo service nfs-kernel-server restart</code></p>		<p><code>masternode: sudo service nfs-kernel-server restart</code></p>
Line 151:		Line 151:
	<p>If you want to change the default port that is used for ssh, we have to make some changes to config files.  Unfortunately we have to do this in all nodes.</p>		<p>If you want to change the default port that is used for ssh, we have to make some changes to config files.  Unfortunately we have to do this in all nodes.</p>
	<p><code>allnodes: sudo nano /etc/ssh/ssh_config</code></p>		<p><code>allnodes: sudo nano /etc/ssh/ssh_config</code></p>
	<pre~~><span~~>Port xxxxx~~</span>~~</pre>		<pre>Port xxxxx</pre>
	<p></p>		<p></p>
	<p><code>allnodes: sudo nano /etc/ssh/sshd_config</code></p>		<p><code>allnodes: sudo nano /etc/ssh/sshd_config</code></p>
	<pre~~><code~~>Port xxxxx~~<br />~~PermitRootLogin no~~</code>~~</pre>		<pre>Port xxxxx
			PermitRootLogin no</pre>
	<p>Note that there is already a line that says "PermitRootLogin".  Replace that line with the above.</p>		<p>Note that there is already a line that says "PermitRootLogin".  Replace that line with the above.</p>
	<h3>6. MPICH</h3>		<h3>6. MPICH</h3>
Line 173:		Line 174:
	<p>Once mpich2 is installed we need to set up the machine file so that mpich2 knows how many processes to use in each node.  We can make this file somewhere in the shared directory.</p>		<p>Once mpich2 is installed we need to set up the machine file so that mpich2 knows how many processes to use in each node.  We can make this file somewhere in the shared directory.</p>
	<p><code>beo@ub0:~$ sudo nano /home/beo/machinefile</code></p>		<p><code>beo@ub0:~$ sudo nano /home/beo/machinefile</code></p>
	<pre~~><code~~>ub0:4~~<br />~~ub1:2~~<br />~~ub2:2~~<br />~~ub3:4~~</code>~~</pre>		<pre>ub0:4
			ub1:2
			ub2:2
			ub3:4</pre>
	<p>Where we write the node name and how many processes we want to use separated by a colon.</p>		<p>Where we write the node name and how many processes we want to use separated by a colon.</p>
	<p>Great!  Now everything is set up and should be working, theoretically. Try to test it with this helloworld program from <a href="https://help.ubuntu.com/community/MpichCluster">https://help.ubuntu.com/community/MpichCluster</a>.  Place the script in your home directory as "mpi_hello.c"</p>		<p>Great!  Now everything is set up and should be working, theoretically. Try to test it with this helloworld program from <a href="https://help.ubuntu.com/community/MpichCluster">https://help.ubuntu.com/community/MpichCluster</a>.  Place the script in your home directory as "mpi_hello.c"</p>
	<pre>#include <stdio.h>		<pre>#include <stdio.h>
	~~<span class="anchor" id="line-2-9"></span>~~#include <mpi.h>		#include <mpi.h>
	~~<span class="anchor" id="line-3-7"></span>~~
	~~<span class="anchor" id="line-4-5"></span>~~int main(int argc, char** argv) {		int main(int argc, char** argv) {
	~~<span class="anchor" id="line-5-4"></span>~~ int myrank, nprocs;		int myrank, nprocs;
	~~<span class="anchor" id="line-6-1"></span>~~
	~~<span class="anchor" id="line-7"></span>~~ MPI_Init(&argc, &argv);		MPI_Init(&argc, &argv);
	~~<span class="anchor" id="line-8"></span>~~ MPI_Comm_size(MPI_COMM_WORLD, &nprocs);		MPI_Comm_size(MPI_COMM_WORLD, &nprocs);
	~~<span class="anchor" id="line-9"></span>~~ MPI_Comm_rank(MPI_COMM_WORLD, &myrank);		MPI_Comm_rank(MPI_COMM_WORLD, &myrank);
	~~<span class="anchor" id="line-10"></span>~~
	~~<span class="anchor" id="line-11"></span>~~ printf("Hello from processor %d of %d\n", myrank, nprocs);		printf("Hello from processor %d of %d\n", myrank, nprocs);
	~~<span class="anchor" id="line-12"></span>~~
	~~<span class="anchor" id="line-13"></span>~~ MPI_Finalize();		MPI_Finalize();
	~~<span class="anchor" id="line-14"></span>~~ return 0;		return 0;
	~~<span class="anchor" id="line-15"></span>~~}</pre>		}</pre>
	<p>You need to compile it and then run it with...</p>		<p>You need to compile it and then run it with...</p>
	<p><code><span>beo@ub0:~$ mpicc mpi_hello.c -o mpi_hello</span></code></p>		<p><code><span>beo@ub0:~$ mpicc mpi_hello.c -o mpi_hello</span></code></p>

Wikiuser: Major changes - Brian Perrett

2016-02-23T18:50:24Z

Major changes - Brian Perrett

Show changes

Wikiuser at 23:24, 10 December 2013

2013-12-10T23:24:11Z

← Older revision		Revision as of 23:24, 10 December 2013
Line 8:		Line 8:
	2) Set Host name: ubx (where x is the number of the node, e.g., 0 for the master node.)		2) Set Host name: ubx (where x is the number of the node, e.g., 0 for the master node.)

	3) Set full name of new user as: ~~apl~~-~~admin~~		3) Set full name of new user as: new-user

	4) Set user name for account as: ~~apl~~-~~admin~~		4) Set user name for account as: new-user

	5) Do not encrypt the home directory.		5) Do not encrypt the home directory.
Line 43:		Line 43:

	2) In all nodes change etc/ssh/ssh_config to have the following line:		2) In all nodes change etc/ssh/ssh_config to have the following line:
	Port ~~41122~~		Port xxxxx

	3) In all nodes change etc/ssh/sshd_config, add the following line:		3) In all nodes change etc/ssh/sshd_config, add the following line:
	Port ~~41122~~		Port xxxxx
	PermitRootLogin no		PermitRootLogin no
	NOTE: this should replace the line which previously said "PermitRootLogin yes"		NOTE: this should replace the line which previously said "PermitRootLogin yes"

Admin: /* Settings and software */

2013-12-10T19:36:42Z

Settings and software

← Older revision		Revision as of 19:36, 10 December 2013
Line 47:		Line 47:
	3) In all nodes change etc/ssh/sshd_config, add the following line:		3) In all nodes change etc/ssh/sshd_config, add the following line:
	Port 41122		Port 41122
	PermitRootLogin no (NOTE: this should replace the line which previously said "PermitRootLogin yes")		PermitRootLogin no
			NOTE: this should replace the line which previously said "PermitRootLogin yes"

	4) Before installing MPICH2, run the following commands:		4) Before installing MPICH2, run the following commands:
	sudo apt-get install build-essentials gfortran gfortran-multilib autoconf		sudo apt-get install build-essentials gfortran gfortran-multilib autoconf

Admin: /* Settings and software */

2013-12-10T19:36:17Z

Settings and software

← Older revision		Revision as of 19:36, 10 December 2013
Line 46:		Line 46:

	3) In all nodes change etc/ssh/sshd_config, add the following line:		3) In all nodes change etc/ssh/sshd_config, add the following line:
	Port 41122		Port 41122
	PermitRootLogin no (NOTE: this should replace the line which previously said "PermitRootLogin yes")		PermitRootLogin no (NOTE: this should replace the line which previously said "PermitRootLogin yes")

	4) Before installing MPICH2, run the following commands:		4) Before installing MPICH2, run the following commands:
	sudo apt-get install build-essentials gfortran gfortran-multilib autoconf		sudo apt-get install build-essentials gfortran gfortran-multilib autoconf

Admin: Created page with " == Installing Ubuntu Server 12.04 == Always use default actions except for the following actions: 1) Do not use detect keyboard layout. Instead choose English/US manually. ..."

2013-12-10T19:35:25Z

Created page with " == Installing Ubuntu Server 12.04 == Always use default actions except for the following actions: 1) Do not use detect keyboard layout. Instead choose English/US manually. ..."

New page

== Installing Ubuntu Server 12.04 ==

Always use default actions except for the following actions:

1) Do not use detect keyboard layout. Instead choose English/US manually.

2) Set Host name: ubx (where x is the number of the node, e.g., 0 for the master node.)

3) Set full name of new user as: apl-admin

4) Set user name for account as: apl-admin

5) Do not encrypt the home directory.

6) Partition method: Guided-use entire disk

7) Do remove existing logical volume data.

8) Leave HTTP proxy information blank

9) Set update management to be: No automatic updates

10) Choose software to install: openSSH (nothing else)

11) Install GRUB: yes

12) Once completed update the system.

== Settings and software ==

Our cluster is based on this tutorial: https://help.ubuntu.com/community/MpichCluster

However, we have the following additions/exceptions:

1) To find IP addresses for nodes, log onto the node and type the following command:
ifconfig

Then look for the inet addr in the eth0 section.

1) When prompted to give an SSH-passphrase, leave this field blank.

2) In all nodes change etc/ssh/ssh_config to have the following line:
Port 41122

3) In all nodes change etc/ssh/sshd_config, add the following line:
Port 41122
PermitRootLogin no (NOTE: this should replace the line which previously said "PermitRootLogin yes")

4) Before installing MPICH2, run the following commands:
sudo apt-get install build-essentials gfortran gfortran-multilib autoconf