Set Up Jenkins EC2 Slaves

This page will describe how to set up AWS and Jenkins such that when a build is executed in Jenkins an AWS EC2 instance will be started and the required code run on it.

For this example the instance will terminate after 10 minutes

Create the AWS User – Jenkins

Using IAM create the user:
  Type: programmatic access type.
     Username: Jenkins
     Policy: AmazonSSMFullAccess

Save the security credentials (access key and secret access key) in a csv file called Jenkins.csv

Build the AWS AMI

For this example I created a free tier instance in the default VPC on eu-west-2 (London).

When required create a new key-pair and save the details in JenkinsLDN.pem 

When the instance has started connect and execute the following on it.

 yum install docker -y
 yum install java -y
 yum install python -y
 yum install git -y
 yum update -y

 usermod -aG docker $USER
 usermod -aG wheel $USER
 chkconfig docker on
 service docker start

 amazon-linux-extras install epel
 curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.34.0/install.sh | bash
 . ~/.nvm/nvm.sh
 nvm install node

 vi /etc/sudoers
   change to the following:
   ## Allows people in group wheel to run all commands
   #%wheel ALL=(ALL)       ALL
  
   ## Same thing without a password
   %wheel  ALL=(ALL)       NOPASSWD: ALL 

Stop the above instance and create an AMI image from it.

Note the image id

e.g. “ami-0a400a260ee69f0c2”

Configure Jenkins

Install the “Amazon EC2 plugin”

Open the “Manage Jenkins” -> “Configure System” screen and scroll to the Cloud section.

In the “Add a new cloud” drop-down list select “Amazon EC2” 
Name the EC2 “ec2-slave”

For the “Amazon EC2 Credentials” select ADD

In the pop up window enter the credentials from the Jenkins.csv file we downloaded earlier i.e. copy&paste values into the Access Key ID and the Secret Access Key fields. Select Save.

For “Region” select the same region you created the EC2 AMI in.

Paste the contents of the jenkins.pem file into the EC2 Key Pair’s Private Key field.

Use the Test Connection button to verify that everything has been set correctly.

Now add an AMI by selecting the “Add” button.

Note the "instance type" set must match the instance type of the ami.
Enter the AWS AMI ID in the "AMI ID" field.
Select the "instance type" from the drop down list, in this case t2micro.
Enter the AWS Security group to be used by the instance, in this case Jenkins.
Enter the image user directory and user name e.g. /home/ec2-user, ec2-user.
Enter the AMI type, unix
Enter the ssh port to be used, normally 22
Set the labels that will identify this image to the Jenkins jobs
Select "Advanced"
Select "Associate Public IP" and select "Public IP" in the Connection Strategy dropdown box. 
Ignore the warning as it all works for me and I do not know why this appears at this time.

Create the Jenkins item

Select "New Item" from the Jenkins top page.
Enter the Item name, select "Freestyle Project" and then "OK"
In the General tab of the item description page select the 
"Restrict where this project can be run" box and enter "ec2" in the popup box. 
This entry should be one of the words entered in the "Labels" box of the "Add AMI" screen above.
Select "Add Build Step" in the Build section and enter some dummy code; e.g.
!/bin/bash
 echo "HELLO EC2 2"
Select "Save"

Run

Build the project and you should see the ec2 image created and then deleted after 10 minutes or so.

Checking the “Console output” from the Jenkins build you should get something similar to:

Started by user Paul Hallam 
Running as SYSTEM 
Building remotely on EC2 (ec2-slave) - fc2 (i-034e9995fbbb654f7) (fc22) in workspace /home/ec2-user/workspace/ec22 
[ec22] $ /bin/bash /tmp/jenkins12990021641701246009.sh 
HELLO EC2 2 
Finished: SUCCESS 

Next we will configure this process as a Blue Ocean Pipeline from Git.