Skip to main content

Machines

Here are the machines (also known as instance types) you can use to create sessions and runs. The syntax for specifying an instance type is numberOfAccelerators_acceleratorType_availableMemory. Below is an example utilizing this syntax to create a session on a machine with 8 v100 gpus and 32 GB memory.

grid session create --instance_type 8_v100_32gb

The below table provides AWS instance types to numberOfAccelerators_acceleratorType_availableMemory mapping.

NameCPUGPUMemoryAcceleratornumberOfAccelerators acceleratorType availableMemoryException Instance Type Used
t2.medium204CPU2_CPU_4GB
t3.medium204CPU2_CPU_4GBt2.medium
t2.large208CPU2_CPU_8GB
t3.large208CPU2_CPU_8GBt2.large
m5a.large208CPU2_CPU_8GBt2.large
t2.xlarge4016CPU4_CPU_16GB
t3.xlarge4016CPU4_CPU_16GBt2.xlarge
m5a.xlarge4016CPU4_CPU_16GBt2.xlarge
t2.2xlarge8032CPU8_CPU_32GB
t3.2xlarge8032CPU8_CPU_32GBt2.2xlarge
m5a.2xlarge8032CPU8_CPU_32GBt2.2xlarge
m5a.4xlarge16064CPU16_CPU_64GB
m5a.8xlarge320128CPU32_CPU_128GB
m5a.12xlarge480192CPU48_CPU_192GB
m5a.16xlarge640256CPU64_CPU_256GB
m5a.24xlarge960384CPU96_CPU_384GB
g2.8xlarge32460K5204_K520_60GB
p2.xlarge4161K801_K80_61GB
p2.8xlarge328488K808_K80_488GB
p2.16xlarge6416732K8016_K80_732GB
g3s.xlarge4130.5M601_M60_30.5GB
g3.4xlarge161122M601_M60_122GB
g3.8xlarge322244M602_M60_244GB
g3.16xlarge644488M604_M60_488GB
g4dn.xlarge4116T41_T4_16GB
g4dn.2xlarge8132T41_T4_32GB
g4dn.4xlarge16164T41_T4_64GB
g4dn.8xlarge321128T41_T4_128GB
g4dn.16xlarge641256T41_T4_256GB
g4dn.12xlarge484192T44_T4_192GB
g4dn.metal968384T48_T4_384GB
p3.2xlarge8161V1001_V100_61GB
p3.8xlarge324244V1004_V100_244GB
p3.16xlarge648488V1008_V100_488GB
p3dn.24xlarge968768V1008_V100_768GB

Accepted Conventions

There are two accepted conventions for specifying the machine types:

  1. numberOfAccelerators_acceleratorType_availableMemory
  2. instance.size as taken from the official aws documentation. An example of this can be found in the table here. ​​3. when numberOfAccelerators_acceleratorType_availableMemory results in duplicate, the Exception Instance Type Used column contains the actual instance_type used.

Below are examples utilizing both the numberOfAccelerators_acceleratorType_availableMemory syntax and instance.size syntax to create a session on a machine with 8 v100 gpus and 32 GB memory.

numberOfAccelerators_acceleratorType_availableMemory syntax

grid session create --instance_type 8_v100_32gb

instance.size syntax

grid run --instance_type p3.16xlarge hello.py --gpus 8

Show Instance Types available in Cluster

To show all the available instance types in your cluster from the CLI you can submit:

grid instance-types

This will provide the instance size names from the AWS convention that you have available for use by your cluster.

Some instances show a label Recommended in the UI (this is not shown in the CLI yet).These instances have higher availability than instances without this label and can allow for faster session start up times.

Spot Instances

Spot instances give you access to cheaper compute instances, but they can take longer to start, depending on how popular they are at the moment requested.

You can also try using spot instances with Sessions [BETA], if you don't mind waiting for the instance to be available or if you don't mind your Session being interrupted when the instance becomes unavailable.

To select a spot instance in the Grid UI, select the "spot instance" checkbox in the Session configuration page:

In the CLI, add the option --use_spot to your session create command:

grid session create --use_spot
grid run --use_spot