optimizer_helper

Adam

class ding.torch_utils.optimizer_helper.Adam(params: Iterable, lr: float = 0.001, betas: Tuple[float, float] = (0.9, 0.999), eps: float = 1e-08, weight_decay: float = 0, amsgrad: bool = False, optim_type: str = 'adam', grad_clip_type: Optional[str] = None, clip_value: Optional[float] = None, clip_coef: float = 5, clip_norm_type: float = 2.0, clip_momentum_timestep: int = 100, grad_norm_type: Optional[str] = None, grad_ignore_type: Optional[str] = None, ignore_value: Optional[float] = None, ignore_coef: float = 5, ignore_norm_type: float = 2.0, ignore_momentum_timestep: int = 100)

Overview:

Rewrited Adam optimizer to support more features.

Interface:

__init__, step

step(closure: Optional[Callable] = None)

Overview:

Performs a single optimization step

Arguments:

• closure (callable): A closure that reevaluates the model and returns the loss, default set to None

RMSprop

class ding.torch_utils.optimizer_helper.RMSprop(params: Iterable, lr: float = 0.01, alpha: float = 0.99, eps: float = 1e-08, weight_decay: float = 0, momentum: float = 0, centered: bool = False, grad_clip_type: Optional[str] = None, clip_value: Optional[float] = None, clip_coef: float = 5, clip_norm_type: float = 2.0, clip_momentum_timestep: int = 100, grad_norm_type: Optional[str] = None, grad_ignore_type: Optional[str] = None, ignore_value: Optional[float] = None, ignore_coef: float = 5, ignore_norm_type: float = 2.0, ignore_momentum_timestep: int = 100)

Overview:

Rewrited RMSprop optimizer to support more features.

Interface:

__init__, step

step(closure: Optional[Callable] = None)

Overview:

Performs a single optimization step

Arguments:

• closure (callable): A closure that reevaluates the model and returns the loss, default set to None