Designing Your Parameter Space
The parameter space is the foundation of your experiment. Getting it right makes optimization faster and more reliable. This article covers the key decisions: how many parameters, what types, and how to keep the space manageable.
Limit the Number of Parameters
Every additional parameter increases the dimensionality of the optimization problem. More dimensions means the optimizer needs more experiments to find good solutions.
Rule of thumb: If there is strong evidence from previous data or literature that a parameter has very little effect on your objectives, fix it at a sensible value and leave it out of the optimization. Only include parameters that are genuinely uncertain or impactful.
Add Descriptors to Categorical Variables
Categorical parameters are inherently harder for the optimizer because there is no built-in notion of similarity between options. The model must learn the structure entirely from data.
You can make this much easier by providing properties — numerical properties attached to each category (e.g. molecular weight, boiling point, cost). Descriptors encode known structure so the optimizer can generalize across categories it has not yet tested.
Without descriptors: the algorithm stays in initial design until every categorical option has been seen at least once (when there are no constraints). This can be expensive for large option sets.
With descriptors: the model can make predictions for untested categories based on their descriptor values, enabling model-guided optimization much sooner.
Categorical for Unevenly Spaced Numerical Choices
If you need a numerical parameter with unevenly spaced options (e.g. temperature values of 30, 60, 75, 80 °C), create a categorical parameter with a single numerical descriptor per option rather than using exclusion constraints on a numerical parameter.
Example:
Category | Temperature (°C) |
Low | 30 |
Medium | 60 |
High-1 | 75 |
High-2 | 80 |
This keeps the search space simpler than a discrete numerical parameter with exclusion constraints, while the descriptor preserves the order and spacing information.
Homogeneous vs Mixed Spaces
Homogeneous parameter spaces — all continuous or all categorical/discrete — are easier to optimize than mixed spaces where continuous and categorical variables coexist.
When it is physically reasonable, try to convert the space toward homogeneous:
Toward all-continuous: reduce or remove step sizes on numerical parameters so they have many grid points and are treated as continuous.
Toward all-discrete: increase step sizes on numerical parameters so they have few grid points and are treated as discrete, matching your categorical variables.
This is not always possible — many real problems have both types. But when you have a choice, a homogeneous space will generally optimize more efficiently.
Summary
Decision | Guideline |
Number of parameters | As few as justified — fix what does not matter |
Categorical variables | Add properties whenever possible |
Uneven numerical choices | Use a categorical parameter with a numerical descriptor |
Space type | Prefer homogeneous (all-continuous or all-discrete) when possible |