aiaccel.hpo.optuna.samplers.NelderMeadSampler#

class aiaccel.hpo.optuna.samplers.NelderMeadSampler(search_space: dict[str, tuple[int | float, int | float]], seed: int | None = None, rng: RandomState | None = None, coeff: NelderMeadCoefficient | None = None, block: bool = False, sub_sampler: BaseSampler | None = None)[source]#

Sampler using the NelderMead algorithm

Only the initial point and shrink (number of parameters - 1) can be calculated in parallel. Others are basically series calculations. (Even if set by e.g. optuna.optimize(n_jobs=2), the calculation is performed in series except in initial and shrink.) If parallelisation is enabled, set block = True.

When using optuna.enqueue_trial(), the enqueued parameters are calculated separately from the parameters determined by NelderMeadSampler and are taken into NelderMead if a good result is obtained. (Simplex is reconstituted). The enqueued parameters are calculated in parallel with the parameters determined by NelderMead if parallelisation is enabled.

Example

An example of a single-objective optimization is as follows:

import optuna
from aiaccel.hpo.optuna.samplers.nelder_mead_sampler import NelderMeadSampler


def objective(trial):
    x = trial.suggest_float("x", -10, 10)
    return x**2


search_space = {"x": {"low": -10, "high": 10}}
study = optuna.create_study(sampler=NelderMeadSampler(search_space=search_space, seed=42))
study.optimize(objective, n_trials=10)
Parameters:

  • search_space – dict[str, tuple[float, float]] Parameter names and corresponding lower and upper limits. Must be set separately from suggest_uniform (as the parameters must be determined at the time of before_trial).

  • seed – int | None = None Random seed used for initial point calculation.

  • rng – np.random.RandomState | None = None RandomState used for initial point calculation. If specified with seed, rng takes precedence.

  • coeff – NelderMeadCoefficient | None = None Parameters used in NelderMead Algorism.

  • block – bool = False Indicates whether the queue used internally is blocked or not. If parallelisation by optuna.optimize is enabled, it must be set with block = True

  • sub_sampler – optuna.samplers.BaseSampler | None = None Sampler to output parameters when NelderMead cannot output parameters. Mainly intended for use on free computation nodes in parallel. If the sub_sampler function is enabled, it must be set with block = False.

nm#

NelderMeadAlgorism Instance of a class that manages the NelderMead algorithm.

__init__(search_space: dict[str, tuple[int | float, int | float]], seed: int | None = None, rng: RandomState | None = None, coeff: NelderMeadCoefficient | None = None, block: bool = False, sub_sampler: BaseSampler | None = None) → None[source]#

Methods

__init__(search_space[, seed, rng, coeff, ...])

after_trial(study, trial, state, values)

Trial post-processing.

before_trial(study, trial)

Trial pre-processing.

infer_relative_search_space(study, trial)

Infer the search space that will be used by relative sampling in the target trial.

reseed_rng()

Reseed sampler's random number generator.

sample_independent(study, trial, param_name, ...)

Sample a parameter

sample_relative(study, trial, search_space)

Sample parameters in a given search space.