markovchain package

Submodules

markovchain.base module

class markovchain.base.Markov(scanner=None, parser=None, storage=None)[source]

Bases: object

Markov chain generator base class.

DEFAULT_SCANNER

Default scanner class.

Type

type

DEFAULT_PARSER

Default parser class.

Type

type

DEFAULT_STORAGE

Default storage class.

Type

type

scanner
Type

markovchain.scanner.Scanner

parser
Type

markovchain.parser.ParserBase

storage
Type

markovchain.storage.Storage

DEFAULT_PARSER

alias of markovchain.parser.Parser

DEFAULT_SCANNER

alias of markovchain.scanner.Scanner

DEFAULT_STORAGE

alias of markovchain.storage.json.JsonStorage

__init__(scanner=None, parser=None, storage=None)[source]

Markov chain generator base class constructor.

Parameters
close()[source]

Close.

data(data, part=False, dataset='')[source]

Parse data and update links.

Parameters
  • data – Data to parse.

  • part (bool, optional) – True if data is partial (default: False).

  • dataset (str, optional) – Dataset key prefix (default: ‘’).

classmethod from_file(fp, storage=None)[source]

Load from file.

Parameters
  • fp (str or file) – File or path.

  • storage (type, optional) – Storage class (default: cls.DEFAULT_STORAGE)

Return type

markovchain.Markov

classmethod from_settings(settings=None, storage=None)[source]

Create from settings.

Parameters
  • settings (dict, optional) – Settings (default: None).

  • storage (type, optional) – Storage class (default: cls.DEFAULT_STORAGE)

Return type

markovchain.Markov

classmethod from_storage(storage)[source]

Load from storage.

Parameters

storage (markovchain.storage.Storage) –

Return type

markovchain.Markov

generate(state_size=None, start=(), dataset='', backward=False)[source]

Generate a sequence.

Parameters
  • state_size (int, optional) – State size (default: parser.state_sizes[0]).

  • start (str or iterable of str, optional) – Initial state (default: ()).

  • dataset (str, optional) – Dataset key prefix.

  • backward (bool, optional) – Link direction.

Returns

State generator.

Return type

generator of str

get_settings_json()[source]

Convert generator settings to JSON.

Returns

JSON data.

Return type

dict

save(fp=None)[source]

Save to file.

Parameters

fp (file, optional) – Output file.

markovchain.info module

markovchain.parser module

class markovchain.parser.LevelParser(levels=1, parsers=None)[source]

Bases: markovchain.parser.ParserBase

Multilevel parser class.

classes
Type

dict

Parser class group.

Examples

>>> parse = ParserBase(lambda data: zip(data, data[1:]))
>>> list(parse(['a', 'b', 'c']))
[('a', 'b'), ('b', 'c')]
__call__(data, part=False, dataset='')[source]

Parse tokens.

Parameters
Returns

Link generator.

Return type

generator of (str, islice of str, str)

__init__(levels=1, parsers=None)[source]

Multilevel parser constructor.

Parameters
property levels

Number of levels.

Type

int

property parsers

Level parsers.

Type

list of markovchain.parser.ParserBase

reset()[source]

Reset parser state.

save()[source]

Convert to JSON.

Returns

JSON data.

Return type

dict

class markovchain.parser.Parser(state_sizes=None, reset_on_sentence_end=True)[source]

Bases: markovchain.parser.ParserBase

Default parser class.

state

Parser state.

Type

deque of str

state_size

Maximum parser state size.

Type

int

reset_on_sentence_end

Reset parser state on markovchain.scanner.Scanner.END token.

Type

bool

end

True if a sentence is not started.

Type

bool

Examples

>>> tokens = ['a', 'b', '.', Scanner.END, 'c', '.', Scanner.END]
>>> parse = Parser()
>>> [(list(state), next) for state, next in parse(tokens)]
[([''], 'a'), (['a'], 'b'), (['b'], '.'),
 ([''], 'c'), (['c'], '.')]
>>> parse.state_sizes = 2
>>> [(list(state), next) for state, next in parse(tokens)]
[(['', ''], 'a'), (['', 'a'], 'b'), (['a', 'b'], '.'),
 (['', ''], 'c'), (['', 'c'], '.')]
>>> parse.state_sizes = [1, 2]
>>> [(list(state), next) for state, next in parse(tokens)]
[([''], 'a'), (['', ''], 'a'),
 (['a'], 'b'), (['', 'a'], 'b'),
 (['b'], '.'), (['a', 'b'], '.'),
 ([''], 'c'), (['', ''], 'c'),
 (['c'], '.'), (['', 'c'], '.')]
__call__(data, part=False, dataset='')[source]

Parse tokens.

Parameters
Returns

Link generator.

Return type

generator of (str, islice of str, str)

__init__(state_sizes=None, reset_on_sentence_end=True)[source]

Default parser constructor.

Parameters
  • state_sizes (int or list of int, optional) – Parser state size(s) (default: [1]).

  • reset_on_sentence_end (bool, optional) – Reset parser state on markovchain.scanner.Scanner.END token (default: True).

reset(state_size_changed=False)[source]

Reset parser state.

Parameters

state_size_changed (bool, optional) – True if maximum state size changed (default: False).

save()[source]

Convert to JSON.

Returns

JSON data.

Return type

dict

property state_sizes

Parser state sizes.

Type

list of int

class markovchain.parser.ParserBase(parse=None)[source]

Bases: markovchain.util.SaveLoad

Base parser class.

classes
Type

dict

Parser class group.

Examples

>>> parse = ParserBase(lambda data: zip(data, data[1:]))
>>> list(parse(['a', 'b', 'c']))
[('a', 'b'), ('b', 'c')]
__call__(data, part=False, dataset='')[source]

Parse data.

Parameters
  • data – Data to parse.

  • part (bool, optional) – True if data is partial (default: False).

  • dataset (str, optional) – Dataset key prefix (default: ‘’).

Returns

self.parse(data)

Return type

object

__init__(parse=None)[source]

Base parser constructor.

Parameters

parse (function, optional) –

classes = {'LevelParser': <class 'markovchain.parser.LevelParser'>, 'Parser': <class 'markovchain.parser.Parser'>}

markovchain.scanner module

class markovchain.scanner.Scanner(scan=None)[source]

Bases: markovchain.util.SaveLoad

Base scanner class.

classes

Scanner class group.

Type

dict

START

Sentence start token.

END

Sentence end token.

Examples

>>> scan = Scanner(lambda data: data.split())
>>> scan('a b c')
['a', 'b', 'c']
END = None
START = None
__call__(data, part=False)[source]

Scan data.

Parameters
  • data – Data to scan.

  • part (bool, optional) – True if data is partial (default: False).

Returns

self.scan(data)

Return type

object

__init__(scan=None)[source]

Base scanner constructor.

Parameters

scan (function, optional) –

classes = {}
reset()[source]

Reset scanner state.

markovchain.util module

class markovchain.util.ObjectWrapper(obj)[source]

Bases: object

Base class for wrapping objects.

Example

>>> class Object:
...     def method(self):
...         return 2
...
>>> class Wrapper(ObjectWrapper):
...     def method(self):
...         return super().method() * 2
...
>>> obj = Object()
>>> wrapped = Wrapper(obj)
>>> wrapped.method()
4
__init__(obj)[source]
class markovchain.util.SaveLoad[source]

Bases: object

Base class for converting to/from JSON.

classes

Class group.

Type

dict

Examples

>>> class SaveLoadGroup(SaveLoad):
...     classes = {}
...
>>> class SaveLoadObject(SaveLoadGroup):
...     def __init__(self, attr=None):
...         self.attr = attr
...     def save(self):
...         data = super().save()
...         data['attr'] = self.attr
...         return data
...
>>> SaveLoadGroup.add_class(SaveLoadObject)
>>> SaveLoadGroup.classes
{'SaveLoadObject': <class '__main__.SaveLoadObject'>}
>>> obj = SaveLoadObject(0)
>>> data = obj.save()
>>> data
{'attr': 0, '__class__': 'SaveLoadObject'}
>>> obj2 = SaveLoadGroup.load(data)
>>> type(obj2)
<class '__main__.SaveLoadObject'>
>>> obj2.attr
0
classmethod add_class(*args)[source]

Add classes to the group.

Parameters

*args (type) – Classes to add.

classes = {}
classmethod load(data)[source]

Create an object from JSON data.

Parameters

data (dict) – JSON data.

Returns

Created object.

Return type

object

Raises

KeyError – If data does not have the ‘__class__’ key or the necessary class is not in the class group.

classmethod remove_class(*args)[source]

Remove classes from the group.

Parameters

*args (type) – Classes to remove.

save()[source]

Convert an object to JSON.

Returns

JSON data.

Return type

dict

markovchain.util.const(x)[source]

Return a function that takes any arguments and returns the specified value.

Parameters

x – Value to return.

Return type

function

markovchain.util.extend(dst, *args)[source]

Recursively update a dictionary.

Parameters
  • dst (dict) – Dictionary to update.

  • *args (dict) –

Returns

Updated dictionary.

Return type

dict

Examples

>>> extend({'x': {'y': 0}}, {'x': {'z': 1}})
{'x': {'y': 0, 'z': 1}}
markovchain.util.fill(xs, length, copy=False)[source]

Convert a value to a list of specified length.

If the input is too short, fill it with its last element.

Parameters
  • xs – Input list or value.

  • length (int) – Output list length.

  • copy (bool, optional) – Deep copy the last element to fill the list (default: False).

Return type

list

Raises

ValueError – If xs is empty and length > 0

Examples

>>> fill(0, 3)
[0, 0, 0]
>>> fill((x ** 2 for x in range(3)), 1)
[0]
>>> x = [{'x': 0}, {'x': 1}]
>>> y = fill(x, 4)
>>> y
[{'x': 0}, {'x': 1}, {'x': 1}, {'x': 1}]
>>> y[2] is y[1]
True
>>> y[3] is y[2]
True
>>> y = fill(x, 4, True)
>>> y
[{'x': 0}, {'x': 1}, {'x': 1}, {'x': 1}]
>>> y[2] is y[1]
False
>>> y[3] is y[2]
False
markovchain.util.int_enum(cls, val)[source]

Get int enum value.

Parameters
  • cls (type) – Int enum class.

  • val (int or str) – Name or value.

Return type

IntEnum

Raises

ValueError

markovchain.util.level_dataset(lv)[source]

Get dataset key part for level.

Parameters

lv (int) – Level.

Returns

Dataset key part.

Return type

str

markovchain.util.load(obj, cls, default_factory)[source]

Create or load an object if necessary.

Parameters
  • obj (object or dict or None) –

  • cls (type) –

  • default_factory (function) –

Return type

object

markovchain.util.state_size_dataset(sz)[source]

Get dataset key part for state size.

Parameters

sz (int) – State size.

Returns

Dataset key part.

Return type

str

markovchain.util.to_list(x)[source]

Convert a value to a list.

Parameters

x – Value.

Return type

list

Examples

>>> to_list(0)
[0]
>>> to_list({'x': 0})
[{'x': 0}]
>>> to_list(x ** 2 for x in range(3))
[0, 1, 4]
>>> x = [1, 2, 3]
>>> to_list(x)
[1, 2, 3]
>>> _ is x
True
markovchain.util.truncate(string, maxlen, end=True)[source]

Truncate a string.

Parameters
  • string (str) – String to truncate.

  • maxlen (int) – Maximum string length.

  • end (boolean, optional) – Remove characters from the end (default: True).

Raises

ValueError – If maxlen <= 3.

Returns

Truncated string.

Return type

str

Examples

>>> truncate('str', 6)
'str'
>>> truncate('long string', 8)
'long ...'
>>> truncate('long string', 8, False)
'...tring'

Module contents