cache

Please Reference ding/utils/data/tests/test_cache.py for usage

Cache

class ding.utils.data.structure.cache.Cache(maxlen: int, timeout: float, monitor_interval: float = 1.0, _debug: bool = False)

Overview:

Data cache for reducing concurrent pressure, with timeout and full queue eject mechanism

Interface:

__init__, push_data, get_cached_data_iter, run, close

Property:

remain_data_count

__init__(maxlen: int, timeout: float, monitor_interval: float = 1.0, _debug: bool = False) → None

Overview:

Initialize the cache object.

Arguments:

• maxlen (int): Maximum length of the cache queue.

• timeout (float): Maximum second of the data can remain in the cache.

• monitor_interval (float): Interval of the timeout monitor thread checks the time.

• _debug (bool): Whether to use debug mode or not, which enables debug print info.

close() → None

Overview:

Shut down the cache internal thread and send the end flag to send queue’s iterator.

get_cached_data_iter() → callable_iterator

Overview:

Get the iterator of the send queue. Once a data is pushed into send queue, it can be accessed by this iterator. ‘STOP’ is the end flag of this iterator.

Returns:

• iterator (callable_iterator) The send queue iterator.

push_data(data: Any) → None

Overview:

Push data into receive queue, if the receive queue is full(after push), then push all the data in receive queue into send queue.

Arguments:

• data (Any): The data which needs to be added into receive queue

Tip

thread-safe

run() → None

Overview:

Launch the cache internal thread, e.g. timeout monitor thread.