Create a new AWS account#
When we create a new AWS account, we create it to be a AWS Member Account to
our AWS Management Account
2i2c-sandbox. We then grant permissions to a group
of IAM users in the management account to manage the created member account.
Like this, we can sign in to manage the member accounts using users defined in
More information on these terms can be found in AWS Access.
Login at https://2i2c.awsapps.com/start/#
Visit the Management Console of
2i2c-sandbox, the AWS Management Account
Visit the Organizations Accounts Console and click “Add an AWS account”
You can find this page by searching “organizations” in the search bar once you’re authenticated.
Enter an AWS account name
2i2cin the account name in case the user decides to exercise their right to replicate at some point.
Enter an email address for the account’s owner
support+aws-<aws account name>@2i2c.org, like
email@example.com. It will still be delivered to
firstname.lastname@example.org still function as a unique username identifier. This is called subaddressing.
Click “Create AWS account”
Once the new account is created, visit the AWS accounts section of the IAM Identity Center
To add the new account to our SSO:
Select the checkbox next to the new account and then click the “Assign users or groups” button
On the “Groups” tab, select the “2i2c-engineers” group. Click “Next”.
On the “Permission Set” page, select “AdministratorAccess”. Click “Next”.
On the “Review and submit assignments” page, click “Submit”.
You have successfully created a new AWS account and connected it to our AWS Organization’s Management Account! Now, setup a new cluster inside it via Terraform.
Checking quotas and requesting increases#
Cloud providers like AWS require their users to request a Service Quota increase for any substantial use of their services. Quotas act as an upper bound of for example the number of CPUs from a certain machine type and the amount of public IPs that the account can acquire.
When an AWS account is created under our AWS Organization, a Service Quota increase request is automatically submitted thanks to what AWS refer to “Organization templates”, “Quota request template”, and “Template association”.
Following account creation, make sure to check our emails to see what is being requested and if its approved.
We typically need to increase three kinds of quotas described below. The values of these are all ‘Total CPUs’ and hence larger nodes consume more quota.
Standard instance quota (
Running On-Demand Standard (A, C, D, H, I, M, R, T, Z) instances)
These instances are what we use for everything besides the exceptions noted below.
All our hubs will require an increase in this quota.
Spot instance quota (
All Standard (A, C, D, H, I, M, R, T, Z) Spot Instance Requests)
A spot instance is a cheaper instance not guaranteed to be available like standard instances are. We configure these to be used by dask worker pods as created for dask-gateway provided clusters.
daskhubhubs will require an increase in this quota.
GPU instance or high memory instance quota
A GPU instance quota (
Running On-Demand G and VT instances,
Running On-Demand P instances) or a High Memory instance quota (
Running On-Demand High Memory instances) is requested specifically to be able to use GPU powered machines or machines with high amounts of RAM memory.
Our custom tailored hubs will require an increase in this quota.
Manually requesting a quota increase#
Visit the Service Quotas console and select “AWS services” from the left-hand side menu
Search for the service you would like to manage the quotas for, e.g., “Amazon Elastic Kubernetes Service (Amazon EKS)”
Select the quota you would like to manage, e.g., “Nodes per managed node group”
Click the “Request quota increase” button in the “Recent quota increase requests” section of the page
Fill in the form that pops up and change the quota value (must be greater than the current quota value), then click “Request”