markovchain package¶

Submodules¶

markovchain.base module¶

class markovchain.base.Markov(scanner=None, parser=None, storage=None)[source]¶

Bases: object

Markov chain generator base class.

DEFAULT_SCANNER¶

Default scanner class.

Type: type

DEFAULT_PARSER¶

Default parser class.

Type: type

DEFAULT_STORAGE¶

Default storage class.

Type: type

scanner¶

Type: markovchain.scanner.Scanner

parser¶

Type: markovchain.parser.ParserBase

storage¶

Type: markovchain.storage.Storage

DEFAULT_PARSER¶: alias of markovchain.parser.Parser

DEFAULT_SCANNER¶: alias of markovchain.scanner.Scanner

DEFAULT_STORAGE¶: alias of markovchain.storage.json.JsonStorage

__init__(scanner=None, parser=None, storage=None)[source]¶

Markov chain generator base class constructor.

Parameters

scanner (dict or markovchain.scanner.Scanner, optional) – Scanner (default: DEFAULT_SCANNER()).
parser (dict or markovchain.parser.ParserBase, optional) – Parser (default: DEFAULT_PARSER()).
storage (markovchain.storage.Storage, optional) – Parser (default: DEFAULT_STORAGE()).

close()[source]¶: Close.

data(data, part=False, dataset='')[source]¶

Parse data and update links.

Parameters

data – Data to parse.
part (bool, optional) – True if data is partial (default: False).
dataset (str, optional) – Dataset key prefix (default: ‘’).

classmethod from_file(fp, storage=None)[source]¶

Load from file.

Parameters

fp (str or file) – File or path.
storage (type, optional) – Storage class (default: cls.DEFAULT_STORAGE)

Return type

markovchain.Markov

classmethod from_settings(settings=None, storage=None)[source]¶

Create from settings.

Parameters

settings (dict, optional) – Settings (default: None).
storage (type, optional) – Storage class (default: cls.DEFAULT_STORAGE)

Return type

markovchain.Markov

classmethod from_storage(storage)[source]¶

Load from storage.

Parameters: storage (markovchain.storage.Storage) –
Return type: markovchain.Markov

generate(state_size=None, start=(), dataset='', backward=False)[source]¶

Generate a sequence.

Parameters

state_size (int, optional) – State size (default: parser.state_sizes[0]).
start (str or iterable of str, optional) – Initial state (default: ()).
dataset (str, optional) – Dataset key prefix.
backward (bool, optional) – Link direction.

Returns

State generator.

Return type

generator of str

get_settings_json()[source]¶

Convert generator settings to JSON.

Returns: JSON data.
Return type: dict

save(fp=None)[source]¶

Save to file.

Parameters: fp (file, optional) – Output file.

markovchain.info module¶

markovchain.parser module¶

class markovchain.parser.LevelParser(levels=1, parsers=None)[source]¶

Bases: markovchain.parser.ParserBase

Multilevel parser class.

classes¶

Type: dict

Parser class group.

Examples

>>> parse = ParserBase(lambda data: zip(data, data[1:]))
>>> list(parse(['a', 'b', 'c']))
[('a', 'b'), ('b', 'c')]

__call__(data, part=False, dataset='')[source]¶

Parse tokens.

Parameters

data (generator of generator of (str or markovchain.scanner.Scanner.END or (markovchain.scanner.Scanner.START, str))) – Levels to parse.
part (bool, optional) – True if data is partial (default: False).
dataset (str, optional) – Dataset key prefix (default: ‘’).

Returns

Link generator.

Return type

generator of (str, islice of str, str)

__init__(levels=1, parsers=None)[source]¶

Multilevel parser constructor.

Parameters

levels (int) – Number of levels.
parsers (list of markovchain.parser.ParserBase) – Level parsers.

property levels¶

Number of levels.

Type: int

property parsers¶

Level parsers.

Type: list of markovchain.parser.ParserBase

reset()[source]¶: Reset parser state.

save()[source]¶

Convert to JSON.

Returns: JSON data.
Return type: dict

class markovchain.parser.Parser(state_sizes=None, reset_on_sentence_end=True)[source]¶

Bases: markovchain.parser.ParserBase

Default parser class.

state¶

Parser state.

Type: deque of str

state_size¶

Maximum parser state size.

Type: int

reset_on_sentence_end¶

Reset parser state on markovchain.scanner.Scanner.END token.

Type: bool

end¶

True if a sentence is not started.

Type: bool

Examples

>>> tokens = ['a', 'b', '.', Scanner.END, 'c', '.', Scanner.END]
>>> parse = Parser()
>>> [(list(state), next) for state, next in parse(tokens)]
[([''], 'a'), (['a'], 'b'), (['b'], '.'),
 ([''], 'c'), (['c'], '.')]
>>> parse.state_sizes = 2
>>> [(list(state), next) for state, next in parse(tokens)]
[(['', ''], 'a'), (['', 'a'], 'b'), (['a', 'b'], '.'),
 (['', ''], 'c'), (['', 'c'], '.')]
>>> parse.state_sizes = [1, 2]
>>> [(list(state), next) for state, next in parse(tokens)]
[([''], 'a'), (['', ''], 'a'),
 (['a'], 'b'), (['', 'a'], 'b'),
 (['b'], '.'), (['a', 'b'], '.'),
 ([''], 'c'), (['', ''], 'c'),
 (['c'], '.'), (['', 'c'], '.')]

__call__(data, part=False, dataset='')[source]¶

Parse tokens.

Parameters

data (generator of (str or markovchain.scanner.Scanner.END or (markovchain.scanner.Scanner.START, str))) – Tokens to parse.
part (bool, optional) – True if data is partial (default: False).
dataset (str, optional) – Dataset key prefix (default: ‘’).

Returns

Link generator.

Return type

generator of (str, islice of str, str)

__init__(state_sizes=None, reset_on_sentence_end=True)[source]¶

Default parser constructor.

Parameters

state_sizes (int or list of int, optional) – Parser state size(s) (default: [1]).
reset_on_sentence_end (bool, optional) – Reset parser state on markovchain.scanner.Scanner.END token (default: True).

reset(state_size_changed=False)[source]¶

Reset parser state.

Parameters: state_size_changed (bool, optional) – True if maximum state size changed (default: False).

save()[source]¶

Convert to JSON.

Returns: JSON data.
Return type: dict

property state_sizes¶

Parser state sizes.

Type: list of int

class markovchain.parser.ParserBase(parse=None)[source]¶

Bases: markovchain.util.SaveLoad

Base parser class.

classes¶

Type: dict

Parser class group.

Examples

>>> parse = ParserBase(lambda data: zip(data, data[1:]))
>>> list(parse(['a', 'b', 'c']))
[('a', 'b'), ('b', 'c')]

__call__(data, part=False, dataset='')[source]¶

Parse data.

Parameters

data – Data to parse.
part (bool, optional) – True if data is partial (default: False).
dataset (str, optional) – Dataset key prefix (default: ‘’).

Returns

self.parse(data)

Return type

object

__init__(parse=None)[source]¶

Base parser constructor.

Parameters: parse (function, optional) –

classes = {'LevelParser': <class 'markovchain.parser.LevelParser'>, 'Parser': <class 'markovchain.parser.Parser'>}¶

markovchain.scanner module¶

class markovchain.scanner.Scanner(scan=None)[source]¶

Bases: markovchain.util.SaveLoad

Base scanner class.

classes¶

Scanner class group.

Type: dict

START¶: Sentence start token.

END¶: Sentence end token.

Examples

>>> scan = Scanner(lambda data: data.split())
>>> scan('a b c')
['a', 'b', 'c']

END = None¶

START = None¶

__call__(data, part=False)[source]¶

Scan data.

Parameters

data – Data to scan.
part (bool, optional) – True if data is partial (default: False).

Returns

self.scan(data)

Return type

object

__init__(scan=None)[source]¶

Base scanner constructor.

Parameters: scan (function, optional) –

classes = {}¶

reset()[source]¶: Reset scanner state.

markovchain.util module¶

class markovchain.util.ObjectWrapper(obj)[source]¶

Bases: object

Base class for wrapping objects.

Example

>>> class Object:
...     def method(self):
...         return 2
...
>>> class Wrapper(ObjectWrapper):
...     def method(self):
...         return super().method() * 2
...
>>> obj = Object()
>>> wrapped = Wrapper(obj)
>>> wrapped.method()
4

__init__(obj)[source]¶

class markovchain.util.SaveLoad[source]¶

Bases: object

Base class for converting to/from JSON.

classes¶

Class group.

Type: dict

Examples

>>> class SaveLoadGroup(SaveLoad):
...     classes = {}
...
>>> class SaveLoadObject(SaveLoadGroup):
...     def __init__(self, attr=None):
...         self.attr = attr
...     def save(self):
...         data = super().save()
...         data['attr'] = self.attr
...         return data
...
>>> SaveLoadGroup.add_class(SaveLoadObject)
>>> SaveLoadGroup.classes
{'SaveLoadObject': <class '__main__.SaveLoadObject'>}
>>> obj = SaveLoadObject(0)
>>> data = obj.save()
>>> data
{'attr': 0, '__class__': 'SaveLoadObject'}
>>> obj2 = SaveLoadGroup.load(data)
>>> type(obj2)
<class '__main__.SaveLoadObject'>
>>> obj2.attr
0

classmethod add_class(*args)[source]¶

Add classes to the group.

Parameters: *args (type) – Classes to add.

classes = {}¶

classmethod load(data)[source]¶

Create an object from JSON data.

Parameters: data (dict) – JSON data.
Returns: Created object.
Return type: object
Raises: KeyError – If data does not have the ‘__class__’ key or the necessary class is not in the class group.

classmethod remove_class(*args)[source]¶

Remove classes from the group.

Parameters: *args (type) – Classes to remove.

save()[source]¶

Convert an object to JSON.

Returns: JSON data.
Return type: dict

markovchain.util.const(x)[source]¶

Return a function that takes any arguments and returns the specified value.

Parameters: x – Value to return.
Return type: function

markovchain.util.extend(dst, *args)[source]¶

Recursively update a dictionary.

Parameters

dst (dict) – Dictionary to update.
*args (dict) –

Returns

Updated dictionary.

Return type

dict

Examples

>>> extend({'x': {'y': 0}}, {'x': {'z': 1}})
{'x': {'y': 0, 'z': 1}}

markovchain.util.fill(xs, length, copy=False)[source]¶

Convert a value to a list of specified length.

If the input is too short, fill it with its last element.

Parameters

xs – Input list or value.
length (int) – Output list length.
copy (bool, optional) – Deep copy the last element to fill the list (default: False).

Return type

list

Raises

ValueError – If xs is empty and length > 0

Examples

>>> fill(0, 3)
[0, 0, 0]
>>> fill((x ** 2 for x in range(3)), 1)
[0]
>>> x = [{'x': 0}, {'x': 1}]
>>> y = fill(x, 4)
>>> y
[{'x': 0}, {'x': 1}, {'x': 1}, {'x': 1}]
>>> y[2] is y[1]
True
>>> y[3] is y[2]
True
>>> y = fill(x, 4, True)
>>> y
[{'x': 0}, {'x': 1}, {'x': 1}, {'x': 1}]
>>> y[2] is y[1]
False
>>> y[3] is y[2]
False

markovchain.util.int_enum(cls, val)[source]¶

Get int enum value.

Parameters

cls (type) – Int enum class.
val (int or str) – Name or value.

Return type

IntEnum

Raises

ValueError –

markovchain.util.level_dataset(lv)[source]¶

Get dataset key part for level.

Parameters: lv (int) – Level.
Returns: Dataset key part.
Return type: str

markovchain.util.load(obj, cls, default_factory)[source]¶

Create or load an object if necessary.

Parameters

obj (object or dict or None) –
cls (type) –
default_factory (function) –

Return type

object

markovchain.util.state_size_dataset(sz)[source]¶

Get dataset key part for state size.

Parameters: sz (int) – State size.
Returns: Dataset key part.
Return type: str

markovchain.util.to_list(x)[source]¶

Convert a value to a list.

Parameters: x – Value.
Return type: list

Examples

>>> to_list(0)
[0]
>>> to_list({'x': 0})
[{'x': 0}]
>>> to_list(x ** 2 for x in range(3))
[0, 1, 4]
>>> x = [1, 2, 3]
>>> to_list(x)
[1, 2, 3]
>>> _ is x
True

markovchain.util.truncate(string, maxlen, end=True)[source]¶

Truncate a string.

Parameters

string (str) – String to truncate.
maxlen (int) – Maximum string length.
end (boolean, optional) – Remove characters from the end (default: True).

Raises

ValueError – If maxlen <= 3.

Returns

Truncated string.

Return type

str

Examples

>>> truncate('str', 6)
'str'
>>> truncate('long string', 8)
'long ...'
>>> truncate('long string', 8, False)
'...tring'

markovchain package¶

Subpackages¶

Submodules¶

markovchain.base module¶

markovchain.info module¶

markovchain.parser module¶

markovchain.scanner module¶

markovchain.util module¶

Module contents¶