Skip to content

Commit 0929563

Browse files
dougalbdougalb
committed
Merge branch 'develop', v0.0.20 release
2 parents 0a9c545 + d49eca6 commit 0929563

28 files changed

+355
-113
lines changed

‎CHANGELOG.rst

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,16 @@
22
CHANGELOG
33
=========
44

5+
0.0.20
6+
======
7+
* feature:``cfncluster``: Support for D2 instances
8+
* updates:``ami``: Pulled latest CentOS6 errata
9+
* updates:``ami``: Pulled latest cfncluster-node package
10+
* updates:``ami``: Pulled latest ec2-udev-rules package
11+
* updates:``ami``: Pulled latest NVIDIA driver 346.47
12+
* updates:``ami``: Removed cfncluster-kernel repo and packages
13+
* updates:``ami``: Updated Chef client to 12.2.1 + berkshelf
14+
515
0.0.19
616
======
717
* feature:``cli``: Added configure command; easy config setup

‎README.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
cfncluster
22
==========
33

4-
cfncluster is a framework that deploys and maintains HPC clusters on AWS. It is reasonably agnostic to what the cluster is for and can easily be extended to support different frameworks. The CLI is stateless, everything is done using CloudFormation or resources within AWS.
4+
CfnCluster ("cloud formation cluster") is a framework that deploys and maintains high performance computing clusters on Amazon Web Services (AWS). Developed by AWS, CfnCluster facilitates both quick start proof of concepts (POCs) and production deployments. CfnCluster supports many different types of clustered applications and can easily be extended to support different frameworks. The CLI is stateless, everything is done using CloudFormation or resources within AWS.
55

66
Documentation
77
=============

‎amis.txt

Lines changed: 9 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,9 @@
1-
us-west-2 ami-ab7f599b
2-
eu-central-1 ami-b43705a9
3-
sa-east-1 ami-818b359c
4-
ap-northeast-1 ami-891bfb89
5-
eu-west-1 ami-87fd69f0
6-
us-east-1 ami-7660361e
7-
us-west-1 ami-86b0aac3
8-
ap-southeast-2 ami-7d730547
9-
ap-southeast-1 ami-fafcc8a8
10-
us-gov-west-1 ami-5d68097e
1+
us-west-2 ami-39d1fb09
2+
eu-central-1 ami-36eed22b
3+
sa-east-1 ami-19890c04
4+
ap-northeast-1 ami-e051abe0
5+
eu-west-1 ami-6f305018
6+
us-east-1 ami-f07f4598
7+
us-west-1 ami-a703e1e3
8+
ap-southeast-2 ami-4709747d
9+
ap-southeast-1 ami-d88bb98a

‎cli/setup.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,8 +20,8 @@ def read(fname):
2020
return open(os.path.join(os.path.dirname(__file__), fname)).read()
2121

2222
console_scripts = ['cfncluster = cfncluster.cli:main']
23-
version = "0.0.19"
24-
requires = ['boto>=2.36']
23+
version = "0.0.20"
24+
requires = ['boto>=2.38']
2525

2626
if sys.version_info[:2] == (2, 6):
2727
# For python2.6 we have to require argparse since it

‎cloudformation/cfncluster.cfn.json

Lines changed: 189 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"AWSTemplateFormatVersion" : "2010-09-09",
3-
"Description" : "AWS CloudFormation Sample Template cfncluster.cfn.json: Sample template showing an framework for deploying master + compute type clusters on AWS. **WARNING** This template creates AWS resources. You will be billed for the AWS resources used if you create a stack from this template. Version: ami-20141222-0 cfncluster-0.0.17",
3+
"Description" : "AWS CloudFormation Sample Template cfncluster.cfn.json: Sample template showing an framework for deploying master + compute type clusters on AWS. **WARNING** This template creates AWS resources. You will be billed for the AWS resources used if you create a stack from this template. Version: ami-20150415-0 cfncluster-0.0.20",
44
"Parameters" : {
55
"KeyName" : {
66
"Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instances",
@@ -42,7 +42,11 @@
4242
"g2.2xlarge",
4343
"t2.micro",
4444
"t2.small",
45-
"t2.medium"
45+
"t2.medium",
46+
"d2.8xlarge",
47+
"d2.4xlarge",
48+
"d2.2xlarge",
49+
"d2.xlarge"
4650
]
4751
},
4852
"ComputeInstanceType" : {
@@ -81,7 +85,11 @@
8185
"g2.2xlarge",
8286
"t2.micro",
8387
"t2.small",
84-
"t2.medium"
88+
"t2.medium",
89+
"d2.8xlarge",
90+
"d2.4xlarge",
91+
"d2.2xlarge",
92+
"d2.xlarge"
8593
]
8694
},
8795
"InitialQueueSize" : {
@@ -721,34 +729,34 @@
721729
},
722730
"AWSRegionOS2AMI" : {
723731
"us-west-2" : {
724-
"centos6" : "ami-ab7f599b"
732+
"centos6" : "ami-39d1fb09"
725733
},
726734
"eu-central-1" : {
727-
"centos6" : "ami-b43705a9"
735+
"centos6" : "ami-36eed22b"
728736
},
729737
"sa-east-1" : {
730-
"centos6" : "ami-818b359c"
738+
"centos6" : "ami-19890c04"
731739
},
732740
"ap-northeast-1" : {
733-
"centos6" : "ami-891bfb89"
741+
"centos6" : "ami-e051abe0"
734742
},
735743
"eu-west-1" : {
736-
"centos6" : "ami-87fd69f0"
744+
"centos6" : "ami-6f305018"
737745
},
738746
"us-east-1" : {
739-
"centos6" : "ami-7660361e"
747+
"centos6" : "ami-f07f4598"
740748
},
741749
"us-west-1" : {
742-
"centos6" : "ami-86b0aac3"
750+
"centos6" : "ami-a703e1e3"
743751
},
744752
"ap-southeast-2" : {
745-
"centos6" : "ami-7d730547"
753+
"centos6" : "ami-4709747d"
746754
},
747755
"ap-southeast-1" : {
748-
"centos6" : "ami-fafcc8a8"
756+
"centos6" : "ami-d88bb98a"
749757
},
750758
"us-gov-west-1" : {
751-
"centos6" : "ami-5d68097e"
759+
"centos6" : "ami-a7f99884"
752760
}
753761
},
754762
"ClusterUser" : {
@@ -1134,20 +1142,100 @@
11341142
},
11351143
"BlockDeviceMappings" : [
11361144
{
1137-
"DeviceName" : "/dev/sdb",
1145+
"DeviceName" : "/dev/xvdba",
11381146
"VirtualName" : "ephemeral0"
11391147
},
11401148
{
1141-
"DeviceName" : "/dev/sdc",
1149+
"DeviceName" : "/dev/xvdbb",
11421150
"VirtualName" : "ephemeral1"
11431151
},
11441152
{
1145-
"DeviceName" : "/dev/sdd",
1153+
"DeviceName" : "/dev/xvdbc",
11461154
"VirtualName" : "ephemeral2"
11471155
},
11481156
{
1149-
"DeviceName" : "/dev/sde",
1157+
"DeviceName" : "/dev/xvdbd",
11501158
"VirtualName" : "ephemeral3"
1159+
},
1160+
{
1161+
"DeviceName" : "/dev/xvdbe",
1162+
"VirtualName" : "ephemeral4"
1163+
},
1164+
{
1165+
"DeviceName" : "/dev/xvdbf",
1166+
"VirtualName" : "ephemeral5"
1167+
},
1168+
{
1169+
"DeviceName" : "/dev/xvdbg",
1170+
"VirtualName" : "ephemeral6"
1171+
},
1172+
{
1173+
"DeviceName" : "/dev/xvdbh",
1174+
"VirtualName" : "ephemeral7"
1175+
},
1176+
{
1177+
"DeviceName" : "/dev/xvdbi",
1178+
"VirtualName" : "ephemeral8"
1179+
},
1180+
{
1181+
"DeviceName" : "/dev/xvdbj",
1182+
"VirtualName" : "ephemeral9"
1183+
},
1184+
{
1185+
"DeviceName" : "/dev/xvdbk",
1186+
"VirtualName" : "ephemeral10"
1187+
},
1188+
{
1189+
"DeviceName" : "/dev/xvdbl",
1190+
"VirtualName" : "ephemeral11"
1191+
},
1192+
{
1193+
"DeviceName" : "/dev/xvdbm",
1194+
"VirtualName" : "ephemeral12"
1195+
},
1196+
{
1197+
"DeviceName" : "/dev/xvdbn",
1198+
"VirtualName" : "ephemeral13"
1199+
},
1200+
{
1201+
"DeviceName" : "/dev/xvdbo",
1202+
"VirtualName" : "ephemeral14"
1203+
},
1204+
{
1205+
"DeviceName" : "/dev/xvdbp",
1206+
"VirtualName" : "ephemeral15"
1207+
},
1208+
{
1209+
"DeviceName" : "/dev/xvdbq",
1210+
"VirtualName" : "ephemeral16"
1211+
},
1212+
{
1213+
"DeviceName" : "/dev/xvdbr",
1214+
"VirtualName" : "ephemeral17"
1215+
},
1216+
{
1217+
"DeviceName" : "/dev/xvdbs",
1218+
"VirtualName" : "ephemeral18"
1219+
},
1220+
{
1221+
"DeviceName" : "/dev/xvdbt",
1222+
"VirtualName" : "ephemeral19"
1223+
},
1224+
{
1225+
"DeviceName" : "/dev/xvdbu",
1226+
"VirtualName" : "ephemeral20"
1227+
},
1228+
{
1229+
"DeviceName" : "/dev/xvdbv",
1230+
"VirtualName" : "ephemeral21"
1231+
},
1232+
{
1233+
"DeviceName" : "/dev/xvdbw",
1234+
"VirtualName" : "ephemeral22"
1235+
},
1236+
{
1237+
"DeviceName" : "/dev/xvdbx",
1238+
"VirtualName" : "ephemeral23"
11511239
}
11521240
],
11531241
"KeyName" : {
@@ -1633,20 +1721,100 @@
16331721
},
16341722
"BlockDeviceMappings" : [
16351723
{
1636-
"DeviceName" : "/dev/sdb",
1724+
"DeviceName" : "/dev/xvdba",
16371725
"VirtualName" : "ephemeral0"
16381726
},
16391727
{
1640-
"DeviceName" : "/dev/sdc",
1728+
"DeviceName" : "/dev/xvdbb",
16411729
"VirtualName" : "ephemeral1"
16421730
},
16431731
{
1644-
"DeviceName" : "/dev/sdd",
1732+
"DeviceName" : "/dev/xvdbc",
16451733
"VirtualName" : "ephemeral2"
16461734
},
16471735
{
1648-
"DeviceName" : "/dev/sde",
1736+
"DeviceName" : "/dev/xvdbd",
16491737
"VirtualName" : "ephemeral3"
1738+
},
1739+
{
1740+
"DeviceName" : "/dev/xvdbe",
1741+
"VirtualName" : "ephemeral4"
1742+
},
1743+
{
1744+
"DeviceName" : "/dev/xvdbf",
1745+
"VirtualName" : "ephemeral5"
1746+
},
1747+
{
1748+
"DeviceName" : "/dev/xvdbg",
1749+
"VirtualName" : "ephemeral6"
1750+
},
1751+
{
1752+
"DeviceName" : "/dev/xvdbh",
1753+
"VirtualName" : "ephemeral7"
1754+
},
1755+
{
1756+
"DeviceName" : "/dev/xvdbi",
1757+
"VirtualName" : "ephemeral8"
1758+
},
1759+
{
1760+
"DeviceName" : "/dev/xvdbj",
1761+
"VirtualName" : "ephemeral9"
1762+
},
1763+
{
1764+
"DeviceName" : "/dev/xvdbk",
1765+
"VirtualName" : "ephemeral10"
1766+
},
1767+
{
1768+
"DeviceName" : "/dev/xvdbl",
1769+
"VirtualName" : "ephemeral11"
1770+
},
1771+
{
1772+
"DeviceName" : "/dev/xvdbm",
1773+
"VirtualName" : "ephemeral12"
1774+
},
1775+
{
1776+
"DeviceName" : "/dev/xvdbn",
1777+
"VirtualName" : "ephemeral13"
1778+
},
1779+
{
1780+
"DeviceName" : "/dev/xvdbo",
1781+
"VirtualName" : "ephemeral14"
1782+
},
1783+
{
1784+
"DeviceName" : "/dev/xvdbp",
1785+
"VirtualName" : "ephemeral15"
1786+
},
1787+
{
1788+
"DeviceName" : "/dev/xvdbq",
1789+
"VirtualName" : "ephemeral16"
1790+
},
1791+
{
1792+
"DeviceName" : "/dev/xvdbr",
1793+
"VirtualName" : "ephemeral17"
1794+
},
1795+
{
1796+
"DeviceName" : "/dev/xvdbs",
1797+
"VirtualName" : "ephemeral18"
1798+
},
1799+
{
1800+
"DeviceName" : "/dev/xvdbt",
1801+
"VirtualName" : "ephemeral19"
1802+
},
1803+
{
1804+
"DeviceName" : "/dev/xvdbu",
1805+
"VirtualName" : "ephemeral20"
1806+
},
1807+
{
1808+
"DeviceName" : "/dev/xvdbv",
1809+
"VirtualName" : "ephemeral21"
1810+
},
1811+
{
1812+
"DeviceName" : "/dev/xvdbw",
1813+
"VirtualName" : "ephemeral22"
1814+
},
1815+
{
1816+
"DeviceName" : "/dev/xvdbx",
1817+
"VirtualName" : "ephemeral23"
16501818
}
16511819
],
16521820
"KeyName" : {

‎docs/source/autoscaling.rst

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,16 @@
11
.. _autoscaling:
22

3-
cfncluster auto-scaling
3+
CfnCluster auto-scaling
44
=======================
55

6-
Clusters deployed with cfncluster are elastic in several ways. The first is by
6+
Clusters deployed with CfnCluster are elastic in several ways. The first is by
77
simply setting the initial_queue_size and max_queue_size parameters of a cluster
88
settings. The initial_queue_size sets minimum size value of the ComputeFleet
99
Auto Scaling Group(ASG) and also the desired capacity value . The max_queue_size
10-
sets maximum size value of the ComputeFleet ASG. As part of the cfncluster, two
10+
sets maximum size value of the ComputeFleet ASG. As part of the CfnCluster, two
1111
Amazon CloudWatch alarms are created. These alarms monitor a custom Amazon
1212
CloudWatch metric[1] that is published by the MasterServer of each cluster, this
13-
is the second elastic nature of cfncluster. This metric is called pending and is
13+
is the second elastic nature of CfnCluster. This metric is called pending and is
1414
created per Stack and unique to each cluster. These Amazon CloudWatch alarms
1515
call ScaleUp policies associated with the ComputeFleet ASG. This is what handles
1616
the automatic addition of compute nodes when there is pending tasks in the
@@ -32,7 +32,7 @@ affecting any running jobs and also enables an elastic cluster with a fixed base
3232
number of instances.
3333

3434
The value of the auto scaling is the same for HPC as with any other workloads,
35-
the only difference here is cfncluster has code to specifically make it interact
35+
the only difference here is CfnCluster has code to specifically make it interact
3636
in a more intelligent manner. If a static cluster is required, this can be
3737
achieved by setting initial_queue_size and max_queue_size parameters to the size
3838
of cluster required and also setting the maintain_initial_size parameter to

0 commit comments

Comments
 (0)