markovchain package

Submodules

markovchain.base module

class markovchain.base.Markov(scanner=None, parser=None, storage=None)[source]

Bases: object

Markov chain generator base class.

DEFAULT_SCANNER

type – Default scanner class.

DEFAULT_PARSER

type – Default parser class.

DEFAULT_STORAGE

type – Default storage class.

scanner

markovchain.scanner.Scanner

parser

markovchain.parser.ParserBase

storage

markovchain.storage.Storage

DEFAULT_PARSER

alias of Parser

DEFAULT_SCANNER

alias of Scanner

DEFAULT_STORAGE

alias of JsonStorage

__init__(scanner=None, parser=None, storage=None)[source]

Markov chain generator base class constructor.

Parameters:
close()[source]

Close.

data(data, part=False, dataset='')[source]

Parse data and update links.

Parameters:
  • data – Data to parse.
  • part (bool, optional) – True if data is partial (default: False).
  • dataset (str, optional) – Dataset key prefix (default: ‘’).
classmethod from_file(fp, storage=None)[source]

Load from file.

Parameters:
  • fp (str or file) – File or path.
  • storage (type, optional) – Storage class (default: cls.DEFAULT_STORAGE)
Returns:

Return type:

markovchain.Markov

classmethod from_settings(settings=None, storage=None)[source]

Create from settings.

Parameters:
  • settings (dict, optional) – Settings (default: None).
  • storage (type, optional) – Storage class (default: cls.DEFAULT_STORAGE)
Returns:

Return type:

markovchain.Markov

classmethod from_storage(storage)[source]

Load from storage.

Parameters:storage (markovchain.storage.Storage) –
Returns:
Return type:markovchain.Markov
generate(state_size=None, start=(), dataset='', backward=False)[source]

Generate a sequence.

Parameters:
  • state_size (int, optional) – State size (default: parser.state_sizes[0]).
  • start (str or iterable of str, optional) – Initial state (default: ()).
  • dataset (str, optional) – Dataset key prefix.
  • backward (bool, optional) – Link direction.
Returns:

State generator.

Return type:

generator of str

get_settings_json()[source]

Convert generator settings to JSON.

Returns:JSON data.
Return type:dict
save(fp=None)[source]

Save to file.

Parameters:fp (file, optional) – Output file.

markovchain.info module

markovchain.parser module

class markovchain.parser.LevelParser(levels=1, parsers=None)[source]

Bases: markovchain.parser.ParserBase

Multilevel parser class.

classes

dict – Parser class group.

Examples

>>> parse = ParserBase(lambda data: zip(data, data[1:]))
>>> list(parse(['a', 'b', 'c']))
[('a', 'b'), ('b', 'c')]
__call__(data, part=False, dataset='')[source]

Parse tokens.

Parameters:
Returns:

Link generator.

Return type:

generator of (str, islice of str, str)

__init__(levels=1, parsers=None)[source]

Multilevel parser constructor.

Parameters:
levels

int – Number of levels.

parsers

list of markovchain.parser.ParserBase – Level parsers.

reset()[source]

Reset parser state.

save()[source]

Convert to JSON.

Returns:JSON data.
Return type:dict
class markovchain.parser.Parser(state_sizes=None, reset_on_sentence_end=True)[source]

Bases: markovchain.parser.ParserBase

Default parser class.

state

deque of str – Parser state.

state_size

int – Maximum parser state size.

reset_on_sentence_end

bool – Reset parser state on markovchain.scanner.Scanner.END token.

end

bool – True if a sentence is not started.

Examples

>>> tokens = ['a', 'b', '.', Scanner.END, 'c', '.', Scanner.END]
>>> parse = Parser()
>>> [(list(state), next) for state, next in parse(tokens)]
[([''], 'a'), (['a'], 'b'), (['b'], '.'),
 ([''], 'c'), (['c'], '.')]
>>> parse.state_sizes = 2
>>> [(list(state), next) for state, next in parse(tokens)]
[(['', ''], 'a'), (['', 'a'], 'b'), (['a', 'b'], '.'),
 (['', ''], 'c'), (['', 'c'], '.')]
>>> parse.state_sizes = [1, 2]
>>> [(list(state), next) for state, next in parse(tokens)]
[([''], 'a'), (['', ''], 'a'),
 (['a'], 'b'), (['', 'a'], 'b'),
 (['b'], '.'), (['a', 'b'], '.'),
 ([''], 'c'), (['', ''], 'c'),
 (['c'], '.'), (['', 'c'], '.')]
__call__(data, part=False, dataset='')[source]

Parse tokens.

Parameters:
Returns:

Link generator.

Return type:

generator of (str, islice of str, str)

__init__(state_sizes=None, reset_on_sentence_end=True)[source]

Default parser constructor.

Parameters:
  • state_sizes (int or list of int, optional) – Parser state size(s) (default: [1]).
  • reset_on_sentence_end (bool, optional) – Reset parser state on markovchain.scanner.Scanner.END token (default: True).
reset(state_size_changed=False)[source]

Reset parser state.

Parameters:state_size_changed (bool, optional) – True if maximum state size changed (default: False).
save()[source]

Convert to JSON.

Returns:JSON data.
Return type:dict
state_sizes

list of int – Parser state sizes.

class markovchain.parser.ParserBase(parse=None)[source]

Bases: markovchain.util.SaveLoad

Base parser class.

classes

dict – Parser class group.

Examples

>>> parse = ParserBase(lambda data: zip(data, data[1:]))
>>> list(parse(['a', 'b', 'c']))
[('a', 'b'), ('b', 'c')]
__call__(data, part=False, dataset='')[source]

Parse data.

Parameters:
  • data – Data to parse.
  • part (bool, optional) – True if data is partial (default: False).
  • dataset (str, optional) – Dataset key prefix (default: ‘’).
Returns:

self.parse(data)

Return type:

object

__init__(parse=None)[source]

Base parser constructor.

Parameters:parse (function, optional) –
classes = {'Parser': <class 'markovchain.parser.Parser'>, 'LevelParser': <class 'markovchain.parser.LevelParser'>}

markovchain.scanner module

class markovchain.scanner.Scanner(scan=None)[source]

Bases: markovchain.util.SaveLoad

Base scanner class.

classes

dict – Scanner class group.

START

Sentence start token.

END

Sentence end token.

Examples

>>> scan = Scanner(lambda data: data.split())
>>> scan('a b c')
['a', 'b', 'c']
END = None
START = None
__call__(data, part=False)[source]

Scan data.

Parameters:
  • data – Data to scan.
  • part (bool, optional) – True if data is partial (default: False).
Returns:

self.scan(data)

Return type:

object

__init__(scan=None)[source]

Base scanner constructor.

Parameters:scan (function, optional) –
classes = {}
reset()[source]

Reset scanner state.

markovchain.util module

class markovchain.util.ObjectWrapper(obj)[source]

Bases: object

Base class for wrapping objects.

Example

>>> class Object:
...     def method(self):
...         return 2
...
>>> class Wrapper(ObjectWrapper):
...     def method(self):
...         return super().method() * 2
...
>>> obj = Object()
>>> wrapped = Wrapper(obj)
>>> wrapped.method()
4
__init__(obj)[source]
class markovchain.util.SaveLoad[source]

Bases: object

Base class for converting to/from JSON.

classes

dict – Class group.

Examples

>>> class SaveLoadGroup(SaveLoad):
...     classes = {}
...
>>> class SaveLoadObject(SaveLoadGroup):
...     def __init__(self, attr=None):
...         self.attr = attr
...     def save(self):
...         data = super().save()
...         data['attr'] = self.attr
...         return data
...
>>> SaveLoadGroup.add_class(SaveLoadObject)
>>> SaveLoadGroup.classes
{'SaveLoadObject': <class '__main__.SaveLoadObject'>}
>>> obj = SaveLoadObject(0)
>>> data = obj.save()
>>> data
{'attr': 0, '__class__': 'SaveLoadObject'}
>>> obj2 = SaveLoadGroup.load(data)
>>> type(obj2)
<class '__main__.SaveLoadObject'>
>>> obj2.attr
0
classmethod add_class(*args)[source]

Add classes to the group.

Parameters:*args (type) – Classes to add.
classes = {}
classmethod load(data)[source]

Create an object from JSON data.

Parameters:data (dict) – JSON data.
Returns:Created object.
Return type:object
Raises:KeyError – If data does not have the ‘__class__’ key or the necessary class is not in the class group.
classmethod remove_class(*args)[source]

Remove classes from the group.

Parameters:*args (type) – Classes to remove.
save()[source]

Convert an object to JSON.

Returns:JSON data.
Return type:dict
markovchain.util.const(x)[source]

Return a function that takes any arguments and returns the specified value.

Parameters:x – Value to return.
Returns:
Return type:function
markovchain.util.extend(dst, *args)[source]

Recursively update a dictionary.

Parameters:
  • dst (dict) – Dictionary to update.
  • *args (dict) –
Returns:

Updated dictionary.

Return type:

dict

Examples

>>> extend({'x': {'y': 0}}, {'x': {'z': 1}})
{'x': {'y': 0, 'z': 1}}
markovchain.util.fill(xs, length, copy=False)[source]

Convert a value to a list of specified length.

If the input is too short, fill it with its last element.

Parameters:
  • xs – Input list or value.
  • length (int) – Output list length.
  • copy (bool, optional) – Deep copy the last element to fill the list (default: False).
Returns:

Return type:

list

Raises:

ValueError – If xs is empty and length > 0

Examples

>>> fill(0, 3)
[0, 0, 0]
>>> fill((x ** 2 for x in range(3)), 1)
[0]
>>> x = [{'x': 0}, {'x': 1}]
>>> y = fill(x, 4)
>>> y
[{'x': 0}, {'x': 1}, {'x': 1}, {'x': 1}]
>>> y[2] is y[1]
True
>>> y[3] is y[2]
True
>>> y = fill(x, 4, True)
>>> y
[{'x': 0}, {'x': 1}, {'x': 1}, {'x': 1}]
>>> y[2] is y[1]
False
>>> y[3] is y[2]
False
markovchain.util.int_enum(cls, val)[source]

Get int enum value.

Parameters:
  • cls (type) – Int enum class.
  • val (int or str) – Name or value.
Returns:

Return type:

IntEnum

Raises:

ValueError

markovchain.util.level_dataset(lv)[source]

Get dataset key part for level.

Parameters:lv (int) – Level.
Returns:Dataset key part.
Return type:str
markovchain.util.load(obj, cls, default_factory)[source]

Create or load an object if necessary.

Parameters:
  • obj (object or dict or None) –
  • cls (type) –
  • default_factory (function) –
Returns:

Return type:

object

markovchain.util.state_size_dataset(sz)[source]

Get dataset key part for state size.

Parameters:sz (int) – State size.
Returns:Dataset key part.
Return type:str
markovchain.util.to_list(x)[source]

Convert a value to a list.

Parameters:x – Value.
Returns:
Return type:list

Examples

>>> to_list(0)
[0]
>>> to_list({'x': 0})
[{'x': 0}]
>>> to_list(x ** 2 for x in range(3))
[0, 1, 4]
>>> x = [1, 2, 3]
>>> to_list(x)
[1, 2, 3]
>>> _ is x
True
markovchain.util.truncate(string, maxlen, end=True)[source]

Truncate a string.

Parameters:
  • string (str) – String to truncate.
  • maxlen (int) – Maximum string length.
  • end (boolean, optional) – Remove characters from the end (default: True).
Raises:

ValueError – If maxlen <= 3.

Returns:

Truncated string.

Return type:

str

Examples

>>> truncate('str', 6)
'str'
>>> truncate('long string', 8)
'long ...'
>>> truncate('long string', 8, False)
'...tring'

Module contents