angelos.archive7.tree¶

exception angelos.archive7.tree.BPlusTreeError¶

Bases: RuntimeError

Errors that happens inside the B+Trees.

CLEAVE_ERROR = ('Failed splitting node, to few entries', 37)¶

CONFIGURATION_ERROR = ("Class configuration doesn't match saved tree.", 30)¶

CONFIG_ORDER_ERROR = ('Order can never be less than 4.', 32)¶

CONFIG_SIZE_ERROR = ('Page size is to small, mode is needed.', 31)¶

ITER_BACKWARD_ERROR = ('Cannot iterate backwards', 36)¶

ITER_CUSTOM_ERROR = ('Cannot iterate with a custom step.', 35)¶

PAGE_ITER_ERROR = ('Iteration over pages or values in pages not complete', 39)¶

SEARCH_ERROR = ('Page search error, can not be None.', 34)¶

VALUE_SIZE_ERROR = ('Value is larger than allowed size', 38)¶

WRONG_NODE_KIND = ('Got unknown node type.', 33)¶

class angelos.archive7.tree.Blob(conf: importlib._bootstrap.Configuration, data: bytes, items: list = [])¶

Bases: angelos.archive7.tree.Entry

Blob entry for opaque data.

data¶

dump() → bytes¶: Dump data from storage.

load(data: bytes)¶: Load data into storage.

class angelos.archive7.tree.Comparable¶

Bases: object

Compare keys.

class angelos.archive7.tree.Comparator¶

Bases: angelos.archive7.tree.Comparable

Comparable dummy.

key¶

class angelos.archive7.tree.DataLoaderDumper¶

Bases: abc.ABC

Data load/dump data base class.

abstract dump() → bytes¶: Abstract data serialization and dump method.

abstract load(data: bytes)¶: Abstract data load and deserialization method.

class angelos.archive7.tree.DataNode(conf: importlib._bootstrap.Configuration, data: bytes = None, page: int = None, next_: int = - 1)¶

Bases: angelos.archive7.tree.StackNode, angelos.archive7.tree.DataLoaderDumper

Node class for arbitrary data.

ENTRY_CLASS¶: alias of angelos.archive7.tree.Blob

MAX_ENTRIES = 1¶

NODE_KIND = b'D'¶

blob¶

dump() → bytes¶: Packing data consisting of node meta and entries.

load(data: bytes)¶: Unpack data consisting of node meta and entries.

class angelos.archive7.tree.EmptyNode(conf: importlib._bootstrap.Configuration, data: bytes = None, page: int = None, next_: int = - 1)¶

Bases: angelos.archive7.tree.StackNode, angelos.archive7.tree.DataLoaderDumper

Node class for recycled nodes.

NODE_KIND = b'E'¶

dump() → bytes¶: Packing data consisting of node meta and entries.

load(data: bytes)¶: Unpack data consisting of node meta and entries.

class angelos.archive7.tree.Entry(conf: importlib._bootstrap.Configuration, data: bytes = None)¶

Bases: angelos.archive7.tree.DataLoaderDumper

Entry base class.

exception angelos.archive7.tree.EntryNotFound¶

Bases: RuntimeWarning

Entry not found in node based on key.

class angelos.archive7.tree.HierarchyNode(conf: importlib._bootstrap.Configuration, data: bytes = None, page: int = None, next_: int = - 1, parent: angelos.archive7.tree.Node = None)¶

Bases: angelos.archive7.tree.Node, angelos.archive7.tree.DataLoaderDumper

Node class for managing the btree hierarchy.

delete_entry(key: uuid.UUID)¶: Delete entry by key,

dump() → bytes¶: Packing data consisting of node meta and entries.

entries¶

get_entry(key: uuid.UUID) → angelos.archive7.tree.Entry ¶: Get entry by key.

insert_entry(entry: angelos.archive7.tree.Entry)¶: Insert an entry.

is_not_empty() → bool¶: Can entries be deleted.

is_not_full() → bool¶: Can entries be added.

largest_entry() → angelos.archive7.tree.Entry ¶: Get largest entry

largest_key() → uuid.UUID¶: Get largest key

least_entry() → angelos.archive7.tree.Entry ¶: Get least entry

least_key() → uuid.UUID¶: Get least key

length() → int¶: Number of entries

load(data: bytes)¶: Unpack data consisting of node meta and entries.

max¶

min¶

parent¶

pop_least() → angelos.archive7.tree.Entry ¶: Remove and return the least entry.

split_entries() → list¶: Split an entry in two halves and return half of all entries.

class angelos.archive7.tree.ItemsNode(conf: importlib._bootstrap.Configuration, data: bytes = None, page: int = None, next_: int = - 1)¶

Bases: angelos.archive7.tree.StackNode, angelos.archive7.tree.DataLoaderDumper

Node class for recycled nodes.

NODE_KIND = b'I'¶

dump() → bytes¶: Packing data consisting of node meta and entries.

load(data: bytes)¶: Unpack data consisting of node meta and entries.

class angelos.archive7.tree.LeafNode(conf: importlib._bootstrap.Configuration, data: bytes = None, page: int = None, next_: int = - 1, parent: angelos.archive7.tree.Node = None)¶

Bases: angelos.archive7.tree.RecordNode

Node class that is used as leaf node of records.

NODE_KIND = b'L'¶

class angelos.archive7.tree.MultiBTree(fileobj: _io.FileIO, conf: importlib._bootstrap.Configuration)¶

Bases: angelos.archive7.tree.Tree

BTree that handles multi-item values.

TREE_KIND = b'M'¶

clear(key: uuid.UUID)¶

Clear all values from entry in the tree.

Parameters: key –
Returns

classmethod config(order: int, value_size: int, page_size: Optional[int] = None) → importlib._bootstrap.Configuration¶: Generate configuration class.

delete(key: uuid.UUID)¶

Delete entry from the tree using key.

Parameters: key –
Returns

get(key: uuid.UUID) → list¶

Get values from the tree using key.

Parameters: key –
Returns

insert(key: uuid.UUID, value: Union[set, list])¶

Insert key and values into the tree.

Parameters

key –
value –

Returns

traverse(key) → angelos.archive7.tree.MultiItemIterator ¶: Like get but returns an iterator.

update(key: uuid.UUID, insertions: list = [], deletions: set = {})¶

Update key with values to be inserted and deleted in the tree.

Parameters

key –
insertions –
deletions –

Returns

class angelos.archive7.tree.MultiItemIterator(tree: angelos.archive7.tree.Tree, record: angelos.archive7.tree.Record)¶

Bases: collections.abc.Iterator

Iterator that iterates over a multi item generator.

class angelos.archive7.tree.Node(conf: importlib._bootstrap.Configuration, data: bytes, page: int, next_: int = - 1)¶

Bases: object

Node base class

ENTRY_CLASS = None¶

NODE_KIND = b''¶

data¶

next¶

page¶

class angelos.archive7.tree.Pager(fileobj: _io.FileIO, size: int, meta: int = 0)¶

Bases: collections.abc.Mapping

Pager that wraps pages written to a file object, indexed like a list.

append(data: Union[bytes, bytearray]) → int¶: Append a page of data to the end of the list.

close()¶: Close file descriptor.

meta(data: Optional[bytes] = None) → bytes¶: Read or write meta-data chunk.

read(index: int)¶: Read a page of data from an existing index.

write(data: Union[bytes, bytearray], index: int)¶: Write a page of data to an existing index.

exception angelos.archive7.tree.PagerError¶

Bases: RuntimeError

Pager related errors.

META_SIZE_INVALID = ('Data is not of meta size.', 17)¶

OUT_OF_BOUNDS = ('Out of bounds', 18)¶

PAGE_SIZE_INVALID = ('Data size different from page size.', 19)¶

SEEK_OFFEST_ERROR = ('Failed to seek to offset', 20)¶

UNEVEN_LENGTH = ('File of uneven length compared to page siz.', 16)¶

WRITE_FAILED = ("Didn't write all data", 21)¶

class angelos.archive7.tree.Record(conf: importlib._bootstrap.Configuration, data: bytes = None, key: uuid.UUID = None, value: Union[bytes, int] = None, page: int = - 1)¶

Bases: angelos.archive7.tree.Entry, angelos.archive7.tree.Comparable

Record entry using key/value-pair.

dump() → bytes¶: Packing data consisting of page number, key and value.

key¶

load(data: bytes)¶: Unpack data consisting of page number, key and value.

page¶

value¶

exception angelos.archive7.tree.RecordError¶

Bases: RuntimeWarning

Record lookup failure.

class angelos.archive7.tree.RecordNode(conf: importlib._bootstrap.Configuration, data: bytes = None, page: int = None, next_: int = - 1, parent: angelos.archive7.tree.Node = None)¶

Bases: angelos.archive7.tree.HierarchyNode

Node class that is used as leaf node of records.

ENTRY_CLASS¶: alias of angelos.archive7.tree.Record

class angelos.archive7.tree.Reference(conf: importlib._bootstrap.Configuration, data: bytes = None, key: uuid.UUID = None, before: int = - 1, after: int = - 1)¶

Bases: angelos.archive7.tree.Entry, angelos.archive7.tree.Comparable

Reference entry for internal structure.

after¶

before¶

dump() → bytes¶: Packing data consisting of before, after and key.

key¶

load(data: bytes)¶: Unpack data consisting of before, after and key.

class angelos.archive7.tree.ReferenceNode(conf: importlib._bootstrap.Configuration, data: bytes = None, page: int = None, parent: angelos.archive7.tree.Node = None)¶

Bases: angelos.archive7.tree.HierarchyNode

Node class for holding references higher up than leaf nodes.

ENTRY_CLASS¶: alias of angelos.archive7.tree.Reference

class angelos.archive7.tree.RootNode(conf: importlib._bootstrap.Configuration, data: bytes = None, page: int = None, parent: angelos.archive7.tree.Node = None)¶

Bases: angelos.archive7.tree.ReferenceNode

When there is several nodes this is the root.

NODE_KIND = b'R'¶

convert()¶: Convert root node to structure node.

class angelos.archive7.tree.SimpleBTree(fileobj: _io.FileIO, conf: importlib._bootstrap.Configuration)¶

Bases: angelos.archive7.tree.Tree

BTree that handles single item values.

TREE_KIND = b'S'¶

classmethod config(order: int, value_size: int, page_size: Optional[int] = None) → importlib._bootstrap.Configuration¶: Generate configuration class.

delete(key: uuid.UUID)¶

Delete entry from the tree using key.

Parameters: key –
Returns

get(key: uuid.UUID) → bytes¶

Get value from the tree using key.

Parameters: key –
Returns

insert(key: uuid.UUID, value: bytes)¶

Insert key and value into the tree.

Parameters

key –
value –

Returns

update(key: uuid.UUID, value: bytes)¶

Update key with value in the tree.

Parameters

key –
value –

Returns

class angelos.archive7.tree.StackNode(conf: importlib._bootstrap.Configuration, data: bytes = None, page: int = None, next_: int = - 1)¶

Bases: angelos.archive7.tree.Node

Node class for nodes that are part of a stack.

MAX_ENTRIES = 0¶

class angelos.archive7.tree.StartNode(conf: importlib._bootstrap.Configuration, data: bytes = None, page: int = None, parent: angelos.archive7.tree.Node = None)¶

Bases: angelos.archive7.tree.RecordNode

When there only is one node this is the root.

NODE_KIND = b'S'¶

convert()¶: Convert start node to normal node.

class angelos.archive7.tree.StructureNode(conf: importlib._bootstrap.Configuration, data: bytes = None, page: int = None, parent: angelos.archive7.tree.Node = None)¶

Bases: angelos.archive7.tree.ReferenceNode

Node class for references that isn’t root node.

NODE_KIND = b'F'¶

insert_entry(entry: angelos.archive7.tree.Reference)¶

Make sure that after of a reference matches before of the next one.

Probably very inefficient approach.

length() → int¶: Entries count.

class angelos.archive7.tree.Transact¶

Bases: contextlib.ContextDecorator, contextlib.AbstractContextManager

BTree transaction context.

class angelos.archive7.tree.Tree(fileobj: _io.FileIO, conf: importlib._bootstrap.Configuration)¶

Bases: abc.ABC

Base tree class.

FORMAT_BLOB = <Struct object>¶

FORMAT_META = <Struct object>¶

FORMAT_NODE = <Struct object>¶

FORMAT_REFERENCE = <Struct object>¶

NODE_KINDS = {b'D': <class 'angelos.archive7.tree.DataNode'>, b'E': <class 'angelos.archive7.tree.EmptyNode'>, b'F': <class 'angelos.archive7.tree.StructureNode'>, b'I': <class 'angelos.archive7.tree.ItemsNode'>, b'L': <class 'angelos.archive7.tree.LeafNode'>, b'R': <class 'angelos.archive7.tree.RootNode'>, b'S': <class 'angelos.archive7.tree.StartNode'>}¶

TREE_KIND = None¶

close()¶: Close memory.

abstract classmethod config(order: int, value_size: int) → importlib._bootstrap.Configuration¶: Generate configuration class.

abstract delete(key: uuid.UUID) → bytes¶

Delete entry from the tree using key.

Parameters: key –
Returns

classmethod factory(fileobj: _io.FileIO, order: int, value_size: int, page_size: Optional[int] = None) → angelos.archive7.tree.Tree ¶: Create a new BTree instance.

abstract get(key: uuid.UUID) → bytes¶

Get value from the tree using key.

Parameters: key –
Returns

abstract insert(key: uuid.UUID, value: Union[bytes, memoryview, set, list])¶

Insert key and value into the tree.

Parameters

key –
value –

Returns

abstract update(key: uuid.UUID, value: Union[bytes, memoryview, set, list])¶

Update key with value in the tree.

Parameters

key –
value –

Returns

class angelos.archive7.tree.TreeAnalyzer(fileobj: _io.FileIO, tree_cls: Type[angelos.archive7.tree.Tree])¶

Bases: object

Analyzer class that analyzes existing trees.

counter() → importlib._bootstrap.NodeCount¶: Count nodes and entries for statistics.

iterate_records(node: angelos.archive7.tree.RecordNode)¶: Iterator for entries in a record node

iterator()¶: Iterator for pages which yields (page, data).

kind_from_data(data: bytes) → bytes¶: Extract node kind.

load_pairs() → importlib._bootstrap.RecordBundle¶: Load key/value-pairs

page_to_node(page: int, data: bytes) → angelos.archive7.tree.Node ¶: Structure node from data.

print_stats()¶: Print BTree meta and statistics information.

records()¶: Iterate all records.

references()¶: Iterate all references.

scanner(keys: set = {}) → importlib._bootstrap.NodeCount¶: Print the page where a record is found by key.

exception angelos.archive7.tree.TreeNodeError¶

Bases: RuntimeError

Error within the tree nodes, entries and items.

BLOB_SIZE_INVALID = ('Blob size larger than fits in page data', 12)¶

CHECKSUM_ERROR = ('Record checksum mismatch.', 2)¶

DATA_TO_LARGE = ('Data larger than page size.', 7)¶

ENTRY_COUNT_DATA_TO_BIG = ('Entry count higher than fits in page data.', 9)¶

ENTRY_COUNT_ERROR = ('Page has a higher count than the allowed order.', 6)¶

ITEM_COUNT_ERROR = ('Item count higher than specified order.', 3)¶

ITEM_TO_MANY = ('More items than fits according to specified order.', 8)¶

ITEM_WRONG_SIZE = ('Item not of expected item size.', 10)¶

NODE_SPLIT_ERROR = ('At least 4 entries in order to split a node', 11)¶

PAGE_LENGTH_INVALID = ('Page data is not of set page size.', 4)¶

PAGE_NOT_SET = ('Page index not set.', 1)¶

WRONG_NODE_KIND = ('Can not load data for wrong node type.', 5)¶

class angelos.archive7.tree.TreeRescue(fileobj: _io.FileIO, tree_cls: Type[angelos.archive7.tree.Tree])¶

Bases: object

Rescue BTree by reading from TreeAnalyzer but inserting to new file.

rescue(database: _io.FileIO)¶: Scans the btree and outputs a rescue copy to database.