A2C

A2CPolicy

class ding.policy.a2c.A2CPolicy(cfg: dict, model: Optional[Union[type, torch.nn.modules.module.Module]] = None, enable_field: Optional[List[str]] = None)

Overview:

Policy class of A2C algorithm.

_forward_collect(data: dict) → dict

Overview:

Forward function for collect mode

Arguments:

• data (dict): Dict type data, including at least [‘obs’].

Returns:

• data (dict): The collected data

_forward_eval(data: dict) → dict

Overview:

Forward function for eval mode, similar to self._forward_collect.

Arguments:

• data (dict): Dict type data, including at least [‘obs’].

Returns:

• output (dict): Dict type data, including at least inferred action according to input obs.

_forward_learn(data: dict) → Dict[str, Any]

Overview:

Forward and backward function of learn mode.

Arguments:

• data (dict): Dict type data, including at least [‘obs’, ‘action’, ‘reward’, ‘next_obs’,’adv’]

Returns:

• info_dict (Dict[str, Any]): Including current lr and loss.

_get_train_sample(data: collections.deque) → Union[None, List[Any]]

Overview:

Get the trajectory and the n step return data, then sample from the n_step return data

Arguments:

• data (deque): The trajectory’s cache

Returns:

• samples (dict): The training samples generated

_init_collect() → None

Overview:

Collect mode init method. Called by self.__init__. Init traj and unroll length, collect model.

_init_eval() → None

Overview:

Evaluate mode init method. Called by self.__init__. Init eval model with argmax strategy.

_init_learn() → None

Overview:

Learn mode init method. Called by self.__init__. Init the optimizer, algorithm config, main and target models.

_process_transition(obs: Any, model_output: dict, timestep: collections.namedtuple) → dict

Overview:

Generate dict type transition data from inputs.

Arguments:

• obs (Any): Env observation

• model_output (dict): Output of collect model, including at least [‘action’]

• timestep (namedtuple): Output after env step, including at least [‘obs’, ‘reward’, ‘done’]

  (here ‘obs’ indicates obs after env step).

Returns:

• transition (dict): Dict type transition data.