jsonextended.edict module

a module to manipulate python dictionary like objects

class jsonextended.edict.LazyLoad(obj, ignore_regexes=('.*', '_*'), recursive=True, parent=None, key_paths=True, list_of_dicts=False, parse_errors=True, **parser_kwargs)

Bases: object

lazy load a dict_like object or file structure as a pseudo dictionary (works with all edict functions) supplies tab completion of keys

Parameters:

• obj (dict or str or object) – file like object or path to file
• ignore_regexes (list[str]) – ignore files and folders matching these regexes (can contain *, ? and [] wildcards)
• recursive (bool) – if True, load subdirectories
• parent (object) – the parent object of this instance
• key_paths (bool) – indicates if the keys of the object can be resolved as file/folder paths (to ensure strings do not get unintentionally treated as paths)
• list_of_dicts (bool) – treat list of dicts as additional branches
• parse_errors (bool) – if True, if parsing a file fails then an IOError will be raised if False, if parsing a file fails then only a logging.error will be made and the value will be returned as None
• parser_kwargs (dict) – additional keywords for parser plugins read_file method, (loaded decoder plugins are parsed by default)

Examples

>>> from jsonextended import plugins
>>> plugins.load_builtin_plugins()
[]

>>> l = LazyLoad({'a':{'b':2},3:4})
>>> print(l)
{3:..,a:..}
>>> l['a']
{b:..}
>>> l[['a','b']]
2
>>> l.a.b
2
>>> l.i3
4

>>> from jsonextended.utils import get_test_path
>>> from jsonextended.edict import pprint

>>> lazydict = LazyLoad(get_test_path())
>>> pprint(lazydict,depth=2)
dir1:
  dir1_1: {...}
  file1.json: {...}
  file2.json: {...}
dir2:
  file1.csv: {...}
  file1.json: {...}
dir3:
file1.keypair:
  key1: val1
  key2: val2
  key3: val3

>>> 'dir1' in lazydict
True

>>> sorted(lazydict.keys())
['dir1', 'dir2', 'dir3', 'file1.keypair']

>>> sorted(lazydict.values())
[{}, {key1:..,key2:..,key3:..}, {file1.csv:..,file1.json:..}, {dir1_1:..,file1.json:..,file2.json:..}]

>>> lazydict.dir1.file1_json
{initial:..,meta:..,optimised:..,units:..}

>>> ldict = lazydict.dir1.file1_json.to_dict()
>>> isinstance(ldict,dict)
True
>>> pprint(ldict,depth=1)
initial: {...}
meta: {...}
optimised: {...}
units: {...}

>>> lazydict = LazyLoad(get_test_path(),recursive=False)
>>> lazydict
{file1.keypair:..}

>>> lazydict = LazyLoad([{'a':{'b':{'c':1}}},{'a':2}],
...                     list_of_dicts=True)
>>> lazydict.i0.a.b.c
1

>>> LazyLoad([1,2,3])
Traceback (most recent call last):
 ...
ValueError: not an expandable object: [1, 2, 3]

>>> plugins.unload_all_plugins()

items() → list of D's (key, value) pairs, as 2-tuples

keys() → iter of D's keys

to_df(**kwargs)

return the (fully loaded) structure as a pandas.DataFrame

to_dict()

return the (fully loaded) structure as a nested dictionary

to_obj()

return the internal object

values() → list of D's values

jsonextended.edict.apply(d, leaf_key, func, new_name=None, remove_lkey=True, list_of_dicts=False, unflatten_level=0, deepcopy=True, **kwargs)

apply a function to all values with a certain leaf (terminal) key

Parameters:

• d (dict) –
• leaf_key (str) – name of leaf key
• func (callable) – function to apply
• new_name (str) – if not None, rename leaf_key
• remove_lkey (bool) – whether to remove original leaf_key (if new_name is not None)
• list_of_dicts (bool) – treat list of dicts as additional branches
• unflatten_level (int or None) – the number of levels to leave unflattened before combining, for instance if you need dicts as inputs
• deepcopy (bool) – deepcopy values
• kwargs (dict) – additional keywords to parse to function

Examples

>>> from pprint import pprint
>>> d = {'a':1,'b':1}
>>> func = lambda x: x+1
>>> pprint(apply(d,'a',func))
{'a': 2, 'b': 1}
>>> pprint(apply(d,'a',func,new_name='c'))
{'b': 1, 'c': 2}
>>> pprint(apply(d,'a',func,new_name='c', remove_lkey=False))
{'a': 1, 'b': 1, 'c': 2}

>>> test_dict = {"a":[{"b":[{"c":1, "d": 2}, {"e":3, "f": 4}]}, {"b":[{"c":5, "d": 6}, {"e":7, "f": 8}]}]}
>>> pprint(apply(test_dict, "b", lambda x: x[-1], list_of_dicts=True, unflatten_level=2))
{'a': [{'b': {'e': 3, 'f': 4}}, {'b': {'e': 7, 'f': 8}}]}

jsonextended.edict.combine_apply(d, leaf_keys, func, new_name, unflatten_level=1, remove_lkeys=True, overwrite=False, list_of_dicts=False, deepcopy=True, **kwargs)

combine values with certain leaf (terminal) keys by a function

Parameters:

• d (dict) –
• leaf_keys (list) – names of leaf keys
• func (callable) – function to apply, must take at least len(leaf_keys) arguments
• new_name (str) – new key name
• unflatten_level (int or None) – the number of levels to leave unflattened before combining, for instance if you need dicts as inputs (None means all)
• remove_lkeys (bool) – whether to remove original leaf_keys
• overwrite (bool) – whether to overwrite any existing new_name key
• list_of_dicts (bool) – treat list of dicts as additional branches
• deepcopy (bool) – deepcopy values
• kwargs (dict) – additional keywords to parse to function

Examples

>>> from pprint import pprint
>>> d = {'a':1,'b':2}
>>> func = lambda x,y: x+y
>>> pprint(combine_apply(d,['a','b'],func,'c'))
{'c': 3}
>>> pprint(combine_apply(d,['a','b'],func,'c',remove_lkeys=False))
{'a': 1, 'b': 2, 'c': 3}

>>> d = {1:{'a':1,'b':2},2:{'a':4,'b':5},3:{'a':1}}
>>> pprint(combine_apply(d,['a','b'],func,'c'))
{1: {'c': 3}, 2: {'c': 9}, 3: {'a': 1}}

>>> func2 = lambda x: sorted(list(x.keys()))
>>> d2 = {'d':{'a':{'b':1,'c':2}}}
>>> pprint(combine_apply(d2,['a'],func2,'a',unflatten_level=2))
{'d': {'a': ['b', 'c']}}

jsonextended.edict.combine_lists(d, keys=None, deepcopy=True)

combine lists of dicts

Parameters:

• d (dict or list[dict]) –
• keys (list) – keys to combine (all if None)
• deepcopy (bool) – deepcopy values

Example

>>> from pprint import pprint
>>> d = {'path_key': {'a': 1, 'split': [{'x': 1, 'y': 3}, {'x': 2, 'y': 4}]}}
>>> pprint(combine_lists(d,['split']))
{'path_key': {'a': 1, 'split': {'x': [1, 2], 'y': [3, 4]}}}

>>> combine_lists([{"a":2}, {"a":1}])
{'a': [2, 1]}

jsonextended.edict.convert_type(d, intype, outtype, convert_list=True, in_place=True)

convert all values of one type to another

Parameters:

• d (dict) –
• intype (type_class) –
• outtype (type_class) –
• convert_list (bool) – whether to convert instances inside lists and tuples
• in_place (bool) – if True, applies conversions to original dict, else returns copy

Examples

>>> from pprint import pprint

>>> d = {'a':'1','b':'2'}
>>> pprint(convert_type(d,str,float))
{'a': 1.0, 'b': 2.0}

>>> d = {'a':['1','2']}
>>> pprint(convert_type(d,str,float))
{'a': [1.0, 2.0]}

>>> d = {'a':[('1','2'),[3,4]]}
>>> pprint(convert_type(d,str,float))
{'a': [(1.0, 2.0), [3, 4]]}

jsonextended.edict.diff(new_dict, old_dict, iter_prefix='__iter__', np_allclose=False, **kwargs)

return the difference between two dict_like objects

Parameters:

• new_dict (dict) –
• old_dict (dict) –
• iter_prefix (str) – prefix to use for list and tuple indexes
• np_allclose (bool) – if True, try using numpy.allclose to assess differences
• **kwargs – keyword arguments to parse to numpy.allclose

Returns:

outcome – Containing none or more of:

• ”insertions” : list of (path, val)
• ”deletions” : list of (path, val)
• ”changes” : list of (path, (val1, val2))
• ”uncomparable” : list of (path, (val1, val2))

Return type:

dict

Examples

>>> from pprint import pprint

>>> diff({'a':1},{'a':1})
{}

>>> pprint(diff({'a': 1, 'b': 2, 'c': 5},{'b': 3, 'c': 4, 'd': 6}))
{'changes': [(('b',), (2, 3)), (('c',), (5, 4))],
 'deletions': [(('d',), 6)],
 'insertions': [(('a',), 1)]}

>>> pprint(diff({'a': [{"b":1}, {"c":2}, 1]},{'a': [{"b":1}, {"d":2}, 2]}))
{'changes': [(('a', '__iter__2'), (1, 2))],
 'deletions': [(('a', '__iter__1', 'd'), 2)],
 'insertions': [(('a', '__iter__1', 'c'), 2)]}

>>> diff({'a':1}, {'a':1+1e-10})
{'changes': [(('a',), (1, 1.0000000001))]}

>>> diff({'a':1}, {'a':1+1e-10}, np_allclose=True)
{}

jsonextended.edict.dump(dct, jfile, overwrite=False, dirlevel=0, sort_keys=True, indent=2, default_name='root.json', **kwargs)

output dict to json

Parameters:

• dct (dict) –
• jfile (str or file_like) – if file_like, must have write method
• overwrite (bool) – whether to overwrite existing files
• dirlevel (int) – if jfile is path to folder, defines how many key levels to set as sub-folders
• sort_keys (bool) – if true then the output of dictionaries will be sorted by key
• indent (int) – if non-negative integer, then JSON array elements and object members will be pretty-printed on new lines with that indent level spacing.
• kwargs (dict) – keywords for json.dump

jsonextended.edict.extract(d, path=None)

extract section of dictionary

Parameters:

• d (dict) –
• path (list[str]) – keys to section

Returns:

• new_dict (dict) – original, without extracted section
• extract_dict (dict) – extracted section

Examples

>>> from pprint import pprint
>>> d = {1:{"a":"A"},2:{"b":"B",'c':'C'}}
>>> pprint(extract(d,[2,'b']))
({1: {'a': 'A'}, 2: {'c': 'C'}}, {'b': 'B'})

jsonextended.edict.filter_keyfuncs(d, keyfuncs, logic='OR', keep_siblings=False, list_of_dicts=False, deepcopy=True)

filters leaf nodes key:func(val) pairs of nested dictionary, where func(val) -> True/False

Parameters:

• d (dict) –
• keyfuncs (dict or list[tuple]) – (key,funcs) pairs to filter by
• logic (str) – “OR” or “AND” for matching pairs
• keep_siblings (bool) – keep all sibling paths
• list_of_dicts (bool) – treat list of dicts as additional branches
• deepcopy (bool) – deepcopy values

Examples

>>> from pprint import pprint

>>> d = {'a':{'b':1,'c':2,'d':3}, 'e':4}
>>> func1 = lambda v: v <= 2

>>> pprint(filter_keyfuncs(d, {'b': func1, 'e': func1}, logic="OR", keep_siblings=False))
{'a': {'b': 1}}

>>> pprint(filter_keyfuncs(d,[('b',func1), ('d', func1)], logic="OR", keep_siblings=True))
{'a': {'b': 1, 'c': 2, 'd': 3}}

>>> pprint(filter_keyfuncs(d, {'b': func1, 'e': func1}, logic="AND", keep_siblings=False))
{}

>>> pprint(filter_keyfuncs(d, {'b': func1, 'd': func1}, logic="AND", keep_siblings=False))
{}

>>> pprint(filter_keyfuncs(d, {'b': func1, 'c': func1}, logic="AND", keep_siblings=False))
{'a': {'b': 1, 'c': 2}}

>>> pprint(filter_keyfuncs(d,[('b',func1), ('c',func1)], logic="AND", keep_siblings=True))
{'a': {'b': 1, 'c': 2, 'd': 3}}

jsonextended.edict.filter_keys(d, keys, use_wildcards=False, list_of_dicts=False, deepcopy=True)

filter dict by certain keys

Parameters:

• d (dict) –
• keys (list) –
• use_wildcards (bool) – if true, can use * (matches everything) and ? (matches any single character)
• list_of_dicts (bool) – treat list of dicts as additional branches
• deepcopy (bool) – deepcopy values

Examples

>>> from pprint import pprint

>>> d = {1:{"a":"A"},2:{"b":"B"},4:{5:{6:'a',7:'b'}}}
>>> pprint(filter_keys(d,['a',6]))
{1: {'a': 'A'}, 4: {5: {6: 'a'}}}

>>> d = {1:{"axxxx":"A"},2:{"b":"B"}}
>>> pprint(filter_keys(d,['a*'],use_wildcards=True))
{1: {'axxxx': 'A'}}

jsonextended.edict.filter_keyvals(d, keyvals, logic='OR', keep_siblings=False, list_of_dicts=False, deepcopy=True)

filters leaf nodes key:value pairs of nested dictionary

Parameters:

• d (dict) –
• keyvals (dict or list[tuple]) – (key,value) pairs to filter by
• logic (str) – “OR” or “AND” for matching pairs
• keep_siblings (bool) – keep all sibling paths
• list_of_dicts (bool) – treat list of dicts as additional branches
• deepcopy (bool) – deepcopy values

Examples

>>> from pprint import pprint

>>> d = {1:{6:'a'},3:{7:'a'},2:{6:"b"},4:{5:{6:'a'}}}
>>> pprint(filter_keyvals(d,[(6,'a')]))
{1: {6: 'a'}, 4: {5: {6: 'a'}}}

>>> d2 = {'a':{'b':1,'c':2,'d':3}, 'e':4}

>>> pprint(filter_keyvals(d2, {'b': 1, 'e': 4}, logic="OR", keep_siblings=False))
{'a': {'b': 1}, 'e': 4}

>>> pprint(filter_keyvals(d2,[('b',1)], logic="OR", keep_siblings=True))
{'a': {'b': 1, 'c': 2, 'd': 3}}

>>> pprint(filter_keyvals(d2, {'b': 1, 'e': 4}, logic="AND", keep_siblings=False))
{}

>>> pprint(filter_keyvals(d2, {'b': 1, 'c': 2}, logic="AND", keep_siblings=False))
{'a': {'b': 1, 'c': 2}}

>>> pprint(filter_keyvals(d2,[('b',1), ('c',2)], logic="AND", keep_siblings=True))
{'a': {'b': 1, 'c': 2, 'd': 3}}

>>> d3 = {"a": {"b": 1, "f": {"d": 3}}, "e": {"b": 1, "c": 2, "f": {"d": 3}}, "g": 5}
>>> pprint(filter_keyvals(d3,[('b',1), ('c', 2)], logic="OR", keep_siblings=True))
{'a': {'b': 1, 'f': {'d': 3}}, 'e': {'b': 1, 'c': 2, 'f': {'d': 3}}}

>>> pprint(filter_keyvals(d3,[('b',1), ('c', 2)], logic="AND", keep_siblings=True))
{'e': {'b': 1, 'c': 2, 'f': {'d': 3}}}

jsonextended.edict.filter_paths(d, paths, list_of_dicts=False, deepcopy=True)

filter dict by certain paths containing key sets

Parameters:

• d (dict) –
• paths (list[str] or list[tuple]) –
• list_of_dicts (bool) – treat list of dicts as additional branches
• deepcopy (bool) – deepcopy values

Examples

>>> from pprint import pprint
>>> d = {'a':{'b':1,'c':{'d':2}},'e':{'c':3}}
>>> filter_paths(d,[('c','d')])
{'a': {'c': {'d': 2}}}

>>> d2 = {'a':[{'b':1,'c':3},{'b':1,'c':2}]}
>>> pprint(filter_paths(d2,["b"],list_of_dicts=False))
{}

>>> pprint(filter_paths(d2,["c"],list_of_dicts=True))
{'a': [{'c': 3}, {'c': 2}]}

jsonextended.edict.filter_values(d, vals=None, list_of_dicts=False, deepcopy=True)

filters leaf nodes of nested dictionary

Parameters:

• d (dict) –
• vals (list) – values to filter by
• list_of_dicts (bool) – treat list of dicts as additional branches
• deepcopy (bool) – deepcopy values

Examples

>>> d = {1:{"a":"A"},2:{"b":"B"},4:{5:{6:'a'}}}
>>> filter_values(d,['a'])
{4: {5: {6: 'a'}}}

jsonextended.edict.flatten(d, key_as_tuple=True, sep='.', list_of_dicts=None, all_iters=None)

get nested dict as flat {key:val,…}, where key is tuple/string of all nested keys

Parameters:

• d (object) –
• key_as_tuple (bool) – whether keys are list of nested keys or delimited string of nested keys
• sep (str) – if key_as_tuple=False, delimiter for keys
• list_of_dicts (str or None) – if not None, flatten lists of dicts using this prefix
• all_iters (str or None) – if not None, flatten all lists and tuples using this prefix

Examples

>>> from pprint import pprint

>>> d = {1:{"a":"A"}, 2:{"b":"B"}}
>>> pprint(flatten(d))
{(1, 'a'): 'A', (2, 'b'): 'B'}

>>> d = {1:{"a":"A"},2:{"b":"B"}}
>>> pprint(flatten(d,key_as_tuple=False))
{'1.a': 'A', '2.b': 'B'}

>>> d = [{'a':1},{'b':[1, 2]}]
>>> pprint(flatten(d,list_of_dicts='__list__'))
{('__list__0', 'a'): 1, ('__list__1', 'b'): [1, 2]}

>>> d = [{'a':1},{'b':[1, 2]}]
>>> pprint(flatten(d,all_iters='__iter__'))
{('__iter__0', 'a'): 1,
 ('__iter__1', 'b', '__iter__0'): 1,
 ('__iter__1', 'b', '__iter__1'): 2}

jsonextended.edict.flatten2d(d, key_as_tuple=True, delim='.', list_of_dicts=None)

get nested dict as {key:dict,…}, where key is tuple/string of all-1 nested keys

NB: is same as flattennd(d,1,key_as_tuple,delim)

Parameters:

• d (dict) –
• key_as_tuple (bool) – whether keys are list of nested keys or delimited string of nested keys
• delim (str) – if key_as_tuple=False, delimiter for keys
• list_of_dicts (str or None) – if not None, flatten lists of dicts using this prefix

Examples

>>> from pprint import pprint

>>> d = {1:{2:{3:{'b':'B','c':'C'},4:'D'}}}
>>> pprint(flatten2d(d))
{(1, 2): {4: 'D'}, (1, 2, 3): {'b': 'B', 'c': 'C'}}

>>> pprint(flatten2d(d,key_as_tuple=False,delim=','))
{'1,2': {4: 'D'}, '1,2,3': {'b': 'B', 'c': 'C'}}

jsonextended.edict.flattennd(d, levels=0, key_as_tuple=True, delim='.', list_of_dicts=None)

get nested dict as {key:dict,…}, where key is tuple/string of all-n levels of nested keys

Parameters:

• d (dict) –
• levels (int) – the number of levels to leave unflattened
• key_as_tuple (bool) – whether keys are list of nested keys or delimited string of nested keys
• delim (str) – if key_as_tuple=False, delimiter for keys
• list_of_dicts (str or None) – if not None, flatten lists of dicts using this prefix

Examples

>>> from pprint import pprint

>>> d = {1:{2:{3:{'b':'B','c':'C'},4:'D'}}}
>>> pprint(flattennd(d,0))
{(1, 2, 3, 'b'): 'B', (1, 2, 3, 'c'): 'C', (1, 2, 4): 'D'}

>>> pprint(flattennd(d,1))
{(1, 2): {4: 'D'}, (1, 2, 3): {'b': 'B', 'c': 'C'}}

>>> pprint(flattennd(d,2))
{(1,): {2: {4: 'D'}}, (1, 2): {3: {'b': 'B', 'c': 'C'}}}

>>> pprint(flattennd(d,3))
{(): {1: {2: {4: 'D'}}}, (1,): {2: {3: {'b': 'B', 'c': 'C'}}}}

>>> pprint(flattennd(d,4))
{(): {1: {2: {3: {'b': 'B', 'c': 'C'}, 4: 'D'}}}}

>>> pprint(flattennd(d,5))
{(): {1: {2: {3: {'b': 'B', 'c': 'C'}, 4: 'D'}}}}

>>> pprint(flattennd(d,1,key_as_tuple=False,delim='.'))
{'1.2': {4: 'D'}, '1.2.3': {'b': 'B', 'c': 'C'}}

>>> test_dict = {"a":[{"b":[{"c":1, "d": 2}, {"e":3, "f": 4}]}, {"b":[{"c":5, "d": 6}, {"e":7, "f": 8}]}]}
>>> pprint(flattennd(test_dict, list_of_dicts="__list__", levels=2))
{('a', '__list__0', 'b'): [{'c': 1, 'd': 2}, {'e': 3, 'f': 4}],
 ('a', '__list__1', 'b'): [{'c': 5, 'd': 6}, {'e': 7, 'f': 8}]}

>>> pprint(flattennd(test_dict, list_of_dicts="__list__", levels=3))
{('a', '__list__0'): {'b': [{'c': 1, 'd': 2}, {'e': 3, 'f': 4}]},
 ('a', '__list__1'): {'b': [{'c': 5, 'd': 6}, {'e': 7, 'f': 8}]}}

jsonextended.edict.indexes(dic, keys=None)

index dictionary by multiple keys

Parameters:

• dic (dict) –
• keys (list) –

Examples

>>> d = {1:{"a":"A"},2:{"b":"B"}}
>>> indexes(d,[1,'a'])
'A'

jsonextended.edict.is_dict_like(obj, attr=('keys', 'items'))

test if object is dict like

jsonextended.edict.is_iter_non_string(obj)

test if object is a list or tuple

jsonextended.edict.is_list_of_dict_like(obj, attr=('keys', 'items'))

test if object is a list only containing dict like items

jsonextended.edict.is_path_like(obj, attr=('name', 'is_file', 'is_dir', 'iterdir'))

test if object is pathlib.Path like

jsonextended.edict.list_to_dict(lst, key=None, remove_key=True)

convert a list of dicts to a dict with root keys

Parameters:

• lst (list[dict]) –
• key (str or None) – a key contained by all of the dicts if None use index number string
• remove_key (bool) – remove key from dicts in list

Examples

>>> from pprint import pprint
>>> lst = [{'name':'f','b':1},{'name':'g','c':2}]
>>> pprint(list_to_dict(lst))
{'0': {'b': 1, 'name': 'f'}, '1': {'c': 2, 'name': 'g'}}

>>> pprint(list_to_dict(lst,'name'))
{'f': {'b': 1}, 'g': {'c': 2}}

jsonextended.edict.merge(dicts, overwrite=False, append=False, list_of_dicts=False)

merge dicts, starting with dicts[1] into dicts[0]

Parameters:

• dicts (list[dict]) – list of dictionaries
• overwrite (bool) – if true allow overwriting of current data
• append (bool) – if true and items are both lists, then add them
• list_of_dicts (bool) – treat list of dicts as additional branches

Examples

>>> from pprint import pprint

>>> d1 = {1:{"a":"A"},2:{"b":"B"}}
>>> d2 = {1:{"a":"A"},2:{"c":"C"}}
>>> pprint(merge([d1,d2]))
{1: {'a': 'A'}, 2: {'b': 'B', 'c': 'C'}}

>>> d1 = {1:{"a":["A"]}}
>>> d2 = {1:{"a":["D"]}}
>>> pprint(merge([d1,d2],append=True))
{1: {'a': ['A', 'D']}}

>>> d1 = {1:{"a":"A"},2:{"b":"B"}}
>>> d2 = {1:{"a":"X"},2:{"c":"C"}}
>>> merge([d1,d2],overwrite=False)
Traceback (most recent call last):
...
ValueError: different data already exists at "1.a": old: A, new: X

>>> merge([{},{}],overwrite=False)
{}
>>> merge([{},{'a':1}],overwrite=False)
{'a': 1}
>>> pprint(merge([{},{'a':1},{'a':1},{'b':2}]))
{'a': 1, 'b': 2}
>>> pprint(merge([{'a':[{"b": 1}, {"c": 2}]}, {'a':[{"d": 3}]}]))
Traceback (most recent call last):
 ...
ValueError: different data already exists at "a": old: [{'b': 1}, {'c': 2}], new: [{'d': 3}]
>>> pprint(merge([{'a':[{"b": 1}, {"c": 2}]}, {'a':[{"d": 3}]}], list_of_dicts=True))
Traceback (most recent call last):
 ...
ValueError: list of dicts are of different lengths at "a": old: [{'b': 1}, {'c': 2}], new: [{'d': 3}]
>>> pprint(merge([{'a':[{"b": 1}, {"c": 2}]}, {'a':[{"d": 3}, {"e": 4}]}], list_of_dicts=True))
{'a': [{'b': 1, 'd': 3}, {'c': 2, 'e': 4}]}

jsonextended.edict.pprint(d, lvlindent=2, initindent=0, delim=':', max_width=80, depth=3, no_values=False, align_vals=True, print_func=None, keycolor=None, compress_lists=None, round_floats=None, _dlist=False)

print a nested dict in readable format

(- denotes an element in a list of dictionaries)

Parameters:

• d (object) –
• lvlindent (int) – additional indentation spaces for each level
• initindent (int) – initial indentation spaces
• delim (str) – delimiter between key and value nodes
• max_width (int) – max character width of each line
• depth (int or None) – maximum levels to display
• no_values (bool) – whether to print values
• align_vals (bool) – whether to align values for each level
• print_func (callable or None) – function to print strings (print if None)
• keycolor (None or str) – if str, color keys by this color, allowed: red, green, yellow, blue, magenta, cyan, white
• compress_lists (int) –

  compress lists/tuples longer than this,

  e.g. [1,1,1,1,1,1] -> [1, 1,…, 1]

• round_floats (int) – significant figures for floats

Examples

>>> d = {'a':{'b':{'c':'Å','de':[4,5,[7,'x'],9]}}}
>>> pprint(d,depth=None)
a:
  b:
    c:  Å
    de: [4, 5, [7, x], 9]
>>> pprint(d,max_width=17,depth=None)
a:
  b:
    c:  Å
    de: [4, 5,
        [7, x],
        9]
>>> pprint(d,no_values=True,depth=None)
a:
  b:
    c:
    de:
>>> pprint(d,depth=2)
a:
  b: {...}
>>> pprint({'a':[1,1,1,1,1,1,1,1]},
...        compress_lists=3)
a: [1, 1, 1, ...(x5)]

jsonextended.edict.remove_keys(d, keys=None, use_wildcards=True, list_of_dicts=False, deepcopy=True)

remove certain keys from nested dict, retaining preceeding paths

Parameters:

• keys (list) –
• use_wildcards (bool) – if true, can use * (matches everything) and ? (matches any single character)
• list_of_dicts (bool) – treat list of dicts as additional branches
• deepcopy (bool) –

Examples

>>> from pprint import pprint
>>> d = {1:{"a":"A"},"a":{"b":"B"}}
>>> pprint(remove_keys(d,['a']))
{1: 'A', 'b': 'B'}

>>> pprint(remove_keys({'abc':1},['a*'],use_wildcards=False))
{'abc': 1}
>>> pprint(remove_keys({'abc':1},['a*'],use_wildcards=True))
{}

jsonextended.edict.remove_keyvals(d, keyvals=None, list_of_dicts=False, deepcopy=True)

remove paths with at least one branch leading to certain (key,value) pairs from dict

Parameters:

• d (dict) –
• keyvals (dict or list[tuple]) – (key,value) pairs to remove
• list_of_dicts (bool) – treat list of dicts as additional branches

Examples

>>> from pprint import pprint
>>> d = {1:{"b":"A"},"a":{"b":"B","c":"D"},"b":{"a":"B"}}
>>> pprint(remove_keyvals(d,[("b","B")]))
{1: {'b': 'A'}, 'b': {'a': 'B'}}

>>> d2 = {'a':[{'b':1,'c':1},{'b':1,'c':2}]}
>>> pprint(remove_keyvals(d2,[("b",1)]))
{'a': [{'b': 1, 'c': 1}, {'b': 1, 'c': 2}]}

>>> pprint(remove_keyvals(d2,[("b",1)],list_of_dicts=True))
{}

jsonextended.edict.remove_paths(d, keys, list_of_dicts=False, deepcopy=True)

remove paths containing certain keys from dict

Parameters:

• d (dict) –
• keys (list) – list of keys to find and remove path
• list_of_dicts (bool) – treat list of dicts as additional branches
• deepcopy (bool) – deepcopy values

Examples

>>> from pprint import pprint
>>> d = {1:{"a":"A"},2:{"b":"B"},4:{5:{6:'a',7:'b'}}}
>>> pprint(remove_paths(d,[6,'a']))
{2: {'b': 'B'}, 4: {5: {7: 'b'}}}

>>> d = {1:{2: 3}, 1:{4: 5}}
>>> pprint(remove_paths(d,[(1, 2)]))
{1: {4: 5}}

>>> d2 = {'a':[{'b':1,'c':{'b':3}},{'b':1,'c':2}]}
>>> pprint(remove_paths(d2,["b"],list_of_dicts=False))
{'a': [{'b': 1, 'c': {'b': 3}}, {'b': 1, 'c': 2}]}

>>> pprint(remove_paths(d2,["b"],list_of_dicts=True))
{'a': [{'c': 2}]}

jsonextended.edict.rename_keys(d, keymap=None, list_of_dicts=False, deepcopy=True)

rename keys in dict

Parameters:

• d (dict) –
• keymap (dict) – dictionary of key name mappings
• list_of_dicts (bool) – treat list of dicts as additional branches
• deepcopy (bool) – deepcopy values

Examples

>>> from pprint import pprint
>>> d = {'a':{'old_name':1}}
>>> pprint(rename_keys(d,{'old_name':'new_name'}))
{'a': {'new_name': 1}}

jsonextended.edict.split_key(d, key, new_keys, before=True, list_of_dicts=False, deepcopy=True)

split an existing key(s) into multiple levels

Parameters:

• d (dict) – or dict like
• key (str) – existing key value
• new_keys (list[str]) – new levels to add
• before (bool) – add level before existing key (else after)
• list_of_dicts (bool) – treat list of dicts as additional branches

Examples

>>> from pprint import pprint
>>> d = {'a':1,'b':2}
>>> pprint(split_key(d,'a',['c','d']))
{'b': 2, 'c': {'d': {'a': 1}}}

>>> pprint(split_key(d,'a',['c','d'],before=False))
{'a': {'c': {'d': 1}}, 'b': 2}

>>> d2 = [{'a':1},{'a':2},{'a':3}]
>>> pprint(split_key(d2,'a',['b'],list_of_dicts=True))
[{'b': {'a': 1}}, {'b': {'a': 2}}, {'b': {'a': 3}}]

jsonextended.edict.split_lists(d, split_keys, new_name='split', check_length=True, deepcopy=True)

split_lists key:list pairs into dicts for each item in the lists NB: will only split if all split_keys are present

Parameters:

• d (dict) –
• split_keys (list) – keys to split
• new_name (str) – top level key for split items
• check_length (bool) – if true, raise error if any lists are of a different length
• deepcopy (bool) – deepcopy values

Examples

>>> from pprint import pprint

>>> d = {'path_key':{'x':[1,2],'y':[3,4],'a':1}}
>>> new_d = split_lists(d,['x','y'])
>>> pprint(new_d)
{'path_key': {'a': 1, 'split': [{'x': 1, 'y': 3}, {'x': 2, 'y': 4}]}}

>>> split_lists(d,['x','a'])
Traceback (most recent call last):
...
ValueError: "a" data at the following path is not a list ('path_key',)

>>> d2 = {'path_key':{'x':[1,7],'y':[3,4,5]}}
>>> split_lists(d2,['x','y'])
Traceback (most recent call last):
...
ValueError: lists at the following path do not have the same size ('path_key',)

class jsonextended.edict.to_html(obj, depth=2, max_length=20, max_height=600, sort=True, local=True, uniqueid=None)

Bases: object

Pretty display dictionary in collapsible format with indents

Parameters:

• obj (str or dict) – dict or json
• depth (int) – Depth of the json tree structure displayed, the rest is collapsed.
• max_length (int) – Maximum number of characters of a string displayed as preview, longer string appear collapsed.
• max_height (int) – Maxium height in pixels of containing box.
• sort (bool) – Whether the json keys are sorted alphabetically.
• local (bool) – use local version of javascript file
• uniqueid (str) – unique identifier (if None, auto-created)

Examples

>>> dic = {'sape': {'value': 22}, 'jack': 4098, 'guido': 4127}
>>> obj = to_html(dic, depth=1, max_length=10, sort=False, local=True, uniqueid='123')
>>> print(obj._repr_html_())
<style>
    .renderjson a              { text-decoration: none; }
    .renderjson .disclosure    { color: red;
                                 font-size: 125%; }
    .renderjson .syntax        { color: darkgrey; }
    .renderjson .string        { color: black; }
    .renderjson .number        { color: black; }
    .renderjson .boolean       { color: purple; }
    .renderjson .key           { color: royalblue; }
    .renderjson .keyword       { color: orange; }
    .renderjson .object.syntax { color: lightseagreen; }
    .renderjson .array.syntax  { color: lightseagreen; }
    </style><div id="123" style="max-height: 600px; width:100%%;"></div>
                <script>
require(["jsonextended/renderjson.js"], function() {
    document.getElementById("123").appendChild(
        renderjson.set_max_string_length(10)
            //.set_icons(circled plus, circled minus)
            .set_icons(String.fromCharCode(8853), String.fromCharCode(8854))
            .set_sort_objects(false)
            .set_show_to_level(1)({"guido": 4127, "jack": 4098, "sape": {"value": 22}}))
});</script>

jsonextended.edict.to_json(dct, jfile, overwrite=False, dirlevel=0, sort_keys=True, indent=2, default_name='root.json', **kwargs)

output dict to json

Parameters:

• dct (dict) –
• jfile (str or file_like) – if file_like, must have write method
• overwrite (bool) – whether to overwrite existing files
• dirlevel (int) – if jfile is path to folder, defines how many key levels to set as sub-folders
• sort_keys (bool) – if true then the output of dictionaries will be sorted by key
• indent (int) – if non-negative integer, then JSON array elements and object members will be pretty-printed on new lines with that indent level spacing.
• kwargs (dict) – keywords for json.dump

Examples

>>> from jsonextended.utils import MockPath
>>> file_obj = MockPath('test.json',is_file=True,exists=False)
>>> dct = {'a':{'b':1}}
>>> to_json(dct, file_obj)
>>> print(file_obj.to_string())
File("test.json") Contents:
{
  "a": {
    "b": 1
  }
}

>>> from jsonextended.utils import MockPath
>>> folder_obj = MockPath()
>>> dct = {'x':{'a':{'b':1},'c':{'d':3}}}
>>> to_json(dct, folder_obj, dirlevel=0,indent=None)
>>> print(folder_obj.to_string(file_content=True))
Folder("root")
  File("x.json") Contents:
   {"a": {"b": 1}, "c": {"d": 3}}

>>> folder_obj = MockPath()
>>> to_json(dct, folder_obj, dirlevel=1,indent=None)
>>> print(folder_obj.to_string(file_content=True))
Folder("root")
  Folder("x")
    File("a.json") Contents:
     {"b": 1}
    File("c.json") Contents:
     {"d": 3}

jsonextended.edict.unflatten(d, key_as_tuple=True, delim='.', list_of_dicts=None, deepcopy=True)

unflatten dictionary with keys as tuples or delimited strings

Parameters:

• d (dict) –
• key_as_tuple (bool) – if true, keys are tuples, else, keys are delimited strings
• delim (str) – if keys are strings, then split by delim
• list_of_dicts (str or None) – if key starts with this treat as a list

Examples

>>> from pprint import pprint

>>> d = {('a','b'):1,('a','c'):2}
>>> pprint(unflatten(d))
{'a': {'b': 1, 'c': 2}}

>>> d2 = {'a.b':1,'a.c':2}
>>> pprint(unflatten(d2,key_as_tuple=False))
{'a': {'b': 1, 'c': 2}}

>>> d3 = {('a','__list__1', 'a'): 1, ('a','__list__0', 'b'): 2}
>>> pprint(unflatten(d3,list_of_dicts='__list__'))
{'a': [{'b': 2}, {'a': 1}]}

>>> unflatten({('a','b','c'):1,('a','b'):2})
Traceback (most recent call last):
...
KeyError: "child conflict for path: ('a', 'b'); 2 and {'c': 1}"