Hyperparameter Sweeps
Grid allows running hyperparameter sweeps without changing a single line of code! Just make sure your script can take arguments:
1
python main.py --layers 32 --learning_rate 0.01
Copied!
Note: This page provides a list of arguments specific to running hyperparameter sweeps. For the full Grid CLI API reference, visit this page.

Python flags

boolean

Passing in a flag without values is treated like a boolean
1
grid run main.py --do_something
Copied!

float

Grid supports standard float notation and scientific notation
1
grid run main.py --learning_rate 0.01
2
grid run main.py --learning_rate 1e-2
Copied!

int

Grid supports standard integer notation
1
grid run main.py --batch_size 32
Copied!

string

Grid supports standard string notation
1
grid run main.py --model bert
2
grid run main.py --model 'bert'
3
grid run main.py --model "bert"
Copied!

Grid extensions

distribution

Calls the script with each value sampled from any numpy distribution.
1
grid run main.py --alpha "uniform(1e-5, 1e-1, 3)"
Copied!
Grid will submit 3 jobs here:
1
# job 1
2
grid run main.py --alpha 0.03977392,
3
4
# job 2
5
grid run main.py --alpha 0.04835479
6
7
# job 3
8
grid run main.py --alpha 0.05200016
Copied!
This is equivalent to:
1
>>> np.random.uniform(1e-5, 1e-1, 3)
2
array([0.03977392, 0.04835479, 0.05200016])
Copied!

list

Calls the script with each value in the list
1
grid run main.py --layers "[32, 64, 128]"
Copied!
Grid will submit 3 jobs here:
1
# job 1
2
grid run main.py --layers 32
3
4
# job 2
5
grid run main.py --layers 64
6
7
# job 3
8
grid run main.py --layers 128
Copied!

range

Calls the script with each value in the range.
1
grid run main.py --layers "range(1, 10, 2)"
Copied!
Grid will submit 3 jobs here:
1
# job 1
2
grid run main.py --layers 1
3
4
# job 2
5
grid run main.py --layers 3
6
7
# job 3
8
grid run main.py --layers 5
9
10
# job 4
11
grid run main.py --layers 7
12
13
# job 5
14
grid run main.py --layers 9
Copied!
This is equivalent to the Python Range syntax
1
>>> list(range(1, 10, 2))
2
[1, 3, 5, 7, 9]
Copied!

string list

Calls the script with each value in the list
1
grid run main.py --model "['resnet18' 'transformer', 'resnet50']"
Copied!
Grid will submit 3 jobs here:
1
# job 1
2
grid run main.py --model resnet18
3
4
# job 2
5
grid run main.py --model transformer
6
7
# job 3
8
grid run main.py --model resnet50
Copied!

Grid Search

To generate combinations (ie: a Grid Search), pass in multiple flags
1
grid run main.py --alpha "uniform(1e-5, 1e-1, 3)" --beta "[1, 2, 3, 4]"
Copied!
Generates 3 x 4 = 12 experiments
1
grid run main.py --alpha 0.03977392 --beta 1
2
grid run main.py --alpha 0.03977392 --beta 2
3
grid run main.py --alpha 0.03977392 --beta 3
4
grid run main.py --alpha 0.03977392 --beta 4
5
6
grid run main.py --alpha 0.04835479 --beta 1
7
grid run main.py --alpha 0.04835479 --beta 2
8
grid run main.py --alpha 0.04835479 --beta 3
9
grid run main.py --alpha 0.04835479 --beta 4
10
11
grid run main.py --alpha 0.05200016 --beta 1
12
grid run main.py --alpha 0.05200016 --beta 2
13
grid run main.py --alpha 0.05200016 --beta 3
14
grid run main.py --alpha 0.05200016 --beta 4
Copied!
To run only a sample of these, choose the Random search option.

Random Search

The following command, generates 12 experiments.
1
grid run main.py --alpha "uniform(1e-5, 1e-1, 3)" --beta "[1, 2, 3, 4]"
Copied!
Generates 3 x 4 = 12 experiments
1
grid run main.py --alpha 0.03977392 --beta 1
2
grid run main.py --alpha 0.03977392 --beta 2
3
grid run main.py --alpha 0.03977392 --beta 3
4
grid run main.py --alpha 0.03977392 --beta 4
5
6
grid run main.py --alpha 0.04835479 --beta 1
7
grid run main.py --alpha 0.04835479 --beta 2
8
grid run main.py --alpha 0.04835479 --beta 3
9
grid run main.py --alpha 0.04835479 --beta 4
10
11
grid run main.py --alpha 0.05200016 --beta 1
12
grid run main.py --alpha 0.05200016 --beta 2
13
grid run main.py --alpha 0.05200016 --beta 3
14
grid run main.py --alpha 0.05200016 --beta 4
Copied!
Using Random Search you can get close to optimal performance without needing to run all possible combinations. To use random search with Grid, simply choose that option
1
grid run --strategy random_search \
2
--trials 3 \
3
main.py \
4
--alpha "uniform(1e-5, 1e-1, 3)" \
5
--beta "[1, 2, 3, 4]"
Copied!
This will choose a subset of the combinations generated and only run 3 experiments.
1
grid run main.py --alpha 0.03977392 --beta 1
2
grid run main.py --alpha 0.03977392 --beta 2
3
grid run main.py --alpha 0.03977392 --beta 3 # <-- chosen (randomly)
4
grid run main.py --alpha 0.03977392 --beta 4
5
6
grid run main.py --alpha 0.04835479 --beta 1 # <-- chosen (randomly)
7
grid run main.py --alpha 0.04835479 --beta 2
8
grid run main.py --alpha 0.04835479 --beta 3 # <-- chosen (randomly)
9
grid run main.py --alpha 0.04835479 --beta 4
10
11
grid run main.py --alpha 0.05200016 --beta 1
12
grid run main.py --alpha 0.05200016 --beta 2
13
grid run main.py --alpha 0.05200016 --beta 3
14
grid run main.py --alpha 0.05200016 --beta 4
Copied!
Last modified 1mo ago