maci

A Python-Styled Serialization Language & Thin Wrapper Library

maci is an easy to use library for data serialization. It can parse native python data types from any plain file, which is safer than using an executable .py file for your stored or configuration data. There are useful language features built-in like creating realistic constants for your name/value pairs by locking them, mapping a name to another to follow its value similar to a pointer, and much more.

Its focus is to reduce boilerplate by removing repetitive code implementation, like code written for common file handling, or common libraries used like JSON, YAML, TOML, etc. maci on its own is a pure Python-based library, and I've used variations of this library on projects for many companies and decided I wanted to make a robust and stable public version. It has made common needs less painful, and has solved simplicity in many ways. Hope it helps you

🎓 tutorials & docs:

quick start: tutorial video

full tutorials: all videos

docs: maci docs

changelog: update history

readme
installing
basic usage: maci
basic usage: thin libs
exceptions, hints, and built-in tools
performance
testing & release
previous support

🍨 install flavors

full --> maci, standard library, and 3rd-party packages

pip install maci

standard lib --> maci and standard library based packages only

pip install maci-std

just maci --> maci package only

pip install maci-only

back to top

📖 basic usage

maci

Example file "my.file" with maci (Python-styled) data

# Example maci data "my.file"
data1 = 'my data'
data2 = 1
data3 = [1,2,3]
data4 = {'k1': 1}
data5 = True
data6 = (1,2,3)
data7 = {1,2,3}
data8 = 1.0
data9 = None
data10 = b'\ndata\n'

Load

load maci data from file

maci_data = maci.load('my.file')
maci_data.data1  # access data with attr name

load raw data from file

raw_data = maci.loadraw('my.file')  # returns string (default)

load attributes names and their values back into your object from file

maci.loadattrs('my.file', my_obj)  # loads in-place
my_obj.data4  # access data in your object with attr name

load as dict data from file

dict_data = maci.loaddict('my.file')
dict_data['data3']  # access data as dict key name

load maci data from string

maci_data = maci.loadstr('data1 = "data"')
maci_data.data1  # access data with attr name

load as dict data from string

dict_data = maci.loadstrdict('data3 = "data"')
dict_data['data3']  # access data as dict key name

Dump

dump data to file from maci object, dict, or your own object with attrs

maci.dump('my.file', maci_data or dict_data or my_obj)
# creates new file with data formatted as maci syntax

dump raw data to file

maci.dumpraw('my.file', 'my data')
# creates new file with data raw as-is to file

dump data to string from maci object, dict, or your own object with attrs

str_data = maci.dumpstr(maci_data or dict_data or my_obj)
# returns string with data formatted as maci syntax

Build

build maci data in code

maci_data = maci.build()
maci_data.data1 = 'my data'
maci_data.data2 = [1,2,3]
maci_data.data3 = 1
maci_data.data4 = True

In-File Language Features

maci supports varying in-file features. Here are some examples using a file named "my.file":

Lock an attr from re-assignment using a lock glyph

# Example maci data in "my.file"
data1 +l= 'my data'

Hard Lock an attr from re-assignment, deletion, and unlocking using a hard lock glyph

# Example maci data in "my.file"
data1 +h= 'my data'

Reference and follow another attr's value with an attr (like a pointer) using a map glyph

# Example maci data in "my.file"
data1 = 'my data'
data2 +m= data1

Date and time parsing

# Example maci data in "my.file"
# Multiple options -> returns datetime, date, or time object
date_time1 = 2023-03-13 22:06:00
date_time2 = 2023-03-13 22:06:00.50
time_date1 = 22:06:00 2023-03-13
time_date2 = 22:06:00.50 2023-03-13
time1 = 22:06:00
time2 = 22:06:00.50
date = 2023-03-13
date_time_iso8601 = 2023-03-13T22:06:00

In-Code Language Features

The in-file language features can also be handled in code with a maci object

maci_data.lock_attr('data1')
maci_data.hard_lock_attr('data2')
maci_data.map_attr('data3', 'data4')

You may unlock attrs, unmap attrs, and much more with a maci object

Note: if you dump your maci object back to a file, all language features will be retained and represented appropriately in the file

back to top

📖 basic usage: thin libs

json -> based on json standard library

load json data from file

data = maci.jsonload('file.json')

load json data from string

data = maci.jsonloadstr('{"k1": "data"}')

dump python data to file as json data

maci.jsondump('file.json', data)

dump data to string as json data

json_data = maci.jsondumpstr(data)

yaml -> based on pyyaml framework

load yaml data from file

data = maci.yamlload('file.yaml')

load yaml data from string

data = maci.yamlloadstr('k1: data')

dump python data to file as yaml data

maci.yamldump('file.yaml', data)

dump data to string as yaml data

yaml_data = maci.yamldumpstr(data)

There are also "loadall" and "dumpall" for multiple yaml docs in a file

toml -> based on tomli libraries

load toml data from file

data = maci.tomlload('file.toml')

load toml data from string

data = maci.tomlloadstr('data1 = "data1"')

dump python data to file as toml data

maci.tomldump('file.toml', data)

dump data to string as toml data

toml_data = maci.tomldumpstr(data)

ini -> based on configparser standard library

load ini data from file

configparser_data = maci.iniload('file.ini')

dump configparser data to file as ini data

maci.inidump('file.ini', configparser_data)

build ini data to configparser data automatically - learn more about configparser objects

configparser_data = maci.inibuildauto({'section1': {'k1': 'value1'}})

build configparser data manually - learn more about configparser objects

configparser_data = maci.inibuildmanual()

xml -> based on xmltodict module & xml etree from standard library

Dict (easiest)

load xml data from file as dict

dict_data = maci.xmlloaddict('file.xml')

load xml data from string as dict

dict_data = maci.xmlloadstrdict('<tag>data</tag>')

dump dict data to file as xml data

maci.xmldumpdict('file.xml', dict_data)

dump dict data to string as xml data

xml_data = maci.xmldumpstrdict(dict_data)

ElementTree - learn more about element tree objects

load xml data from file as element tree object

et_data = maci.xmlload('file.xml')

load xml data from string as element tree object

et_data = maci.xmlloadstr('<tag>data</tag>')

dump element tree data to file as xml data

maci.xmldump('file.xml', et_data)

dump element tree data to string as xml data

xml_data = maci.xmldumpstr(et_data)

build element tree data manually

et_data = maci.xmlbuildmanual()

back to top

🪄 helpful extras

exceptions

All exceptions/errors thrown by maci and its thin wrapper libraries are conveniently accessible here:

maci.error

Examples of different load exceptions

maci.error.Load
maci.error.JsonLoad
maci.error.YamlLoad
maci.error.TomlLoad

To catch/suppress all maci exceptions, use its base exception

maci.error.MaciError

hinting

For type hinting/annotation needs, you can conveniently access the respective object types here:

maci.hint

Examples of different hint objects

maci.hint.MaciDataObj
maci.hint.ConfigParser
maci.hint.ElementTree
maci.hint.Element

useful tools

cleanformat

format nested data cleanly

str_data = maci.cleanformat([1,{'k1': 1, 'k2': 2},2])

print(str_data)

Output -->
[
    1,
    {
        'k1': 1,
        'k2': 2,
    },
    2,
]

pickling

pickle your objects using a non-executable file concept with maci

# Dump to file
maci_data.pickle_data = maci.pickledumpbytes(my_obj)
maci.dump('my.data', maci_data)

# Load back from file
maci_data = maci.load('my.data')
my_obj = maci.pickleloadbytes(maci_data.pickle_data)

This is better than having your whole file having the ability to be unpickled, especially if you cannot trust the file's integrity. More on this from python pickle docs. Though this may help improve pickling needs, still use methods to verify integrity of your pickled data if required

hashing

Easily generate hash of a file and store hash - default hash is sha256

maci.createfilehash('my.data', 'my.data.hashed')
# always returns string of file hash

Now simply compare the hash of the source file to check integrity when needed

maci.comparefilehash('my.data', 'my.data.hashed')
# returns bool if hash is a valid match

Create hash of data - default hash is sha256

maci.createhash('data')  # returns string of hash

back to top

⏳️ performance

Performance tests each library loading 100,000 lines of data each in their natural usage

Tests are done by loading a file with 100 lines of data 1000 times with the proper file syntax for each library. You may also consider this test about loading 1000 files within the time taken as well

Results vary based on system spec, but you may simulate or prove the same difference in test results for your needs from the "perf" dir in this repo. Results below is running the test 3 times consecutively

libs tested: json, pyyaml, tomli, xmltodict, maci

---

Notes

XML ElementTree type and INI Configparser tests were left out for now

pyyaml loads much faster using its c-based safe loader, but using the native out of the box methods/functions provided as tests for fairness and potential compatibility issues for needing LibYAML bindings

---

# Test 1
$ python3 perf_load.py 
Performance tests: "load" - loading file 1000 times with 100 lines of data

xml: 0.225348
json: 0.016725
yaml: 3.625997
toml: 0.23937
maci: 0.807448

# Test 2
$ python3 perf_load.py 
Performance tests: "load" - loading file 1000 times with 100 lines of data

xml: 0.22595
json: 0.016566
yaml: 3.652053
toml: 0.242974
maci: 0.806545

# Test 3
$ python3 perf_load.py 
Performance tests: "load" - loading file 1000 times with 100 lines of data

xml: 0.225579
json: 0.01695
yaml: 3.611955
toml: 0.239593
maci: 0.802843

place	lib
🥇 1st	json - avg 0.016s
🥈 2nd	xmltodict - avg 0.225s
🥉 3rd	tomli - avg 0.240s
4th	maci - avg 0.805s
5th	pyyaml - avg 3.630s (4th if using CLoader)

Current differences in load time results for 100k lines of data from maci compared to popular or modern libraries

Looking to continually improve maci's performance and update the results, but so far, not bad for pure python.

back to top

🚀 testing & release

300+ tests and counting ⚡️

A maci release is only deployed/released if all qa tests pass, and if the revision number is incremented.

All coverage testing must be at 100% or test pipeline will fail (badge is not auto-updated, and just indicates confidence in testing at 100%).

back to top

⏪ previous project support

Project maci is derived from an older project called sfcparse that is no longer supported, and still provides forward ported support for most of the older API names as a courtesy. sfcparse uses the MIT license, and therefore, maci does not really need to associate itself with that older project, but out of notice for the reason of having the forward ported support is it being mentioned if desiring to migrate.

Reason for sfcparse's deprecation was merely for desire of re-branding and scrapping the old to make usage simpler and anew, thus, maci.

Though maci does support the older API names as a courtesy, some names being attempted to use may throw exceptions. Also, functionality in a lot of the forward connected API names may require different parameter positional args or kwargs. See these files for API matched names and where they point to

function names: __init__.py under __getattr__

exception names: error.py under __getattr__

back to top