pathtree: Cache keys in a flat dict for fast lookup
This knocks about a second off of the depth stress tests:
$ python -m obmc.utils.testpathtree
Depth tests:
1: 5.18097305298
2: 6.18765211105
3: 7.3427259922
4: 8.33903598785
5: 9.55236506462
6: 10.7165560722
7: 12.1563079357
8: 13.3570129871
9: 14.7827298641
10: 16.0967490673
Width tests:
1: 1.20649981499
2: 1.20889282227
4: 1.20902085304
8: 1.22063994408
16: 1.22446990013
32: 1.21649098396
64: 1.21024799347
128: 1.21360588074
256: 1.22187900543
512: 1.23233604431
1024: 1.21607708931
2048: 1.21069407463
4096: 1.22389698029
8192: 1.20828580856
16384: 1.21290493011
32768: 1.21552395821
65536: 1.23201704025
131072: 1.21459794044
262144: 1.24190402031
524288: 1.21342992783
1048576: 1.21248602867
However, we get just over a 300% improvement in the host reboot
scenario, which is captured by some separate tools used to analyse the
PathTree's performance issues:
$ # Before
$ ./replay.py -n 1000 ptinit.log ptreboot.log
deep copy: 1000 iterations took 3.0744523939938517s
ops script: 1000 iterations took 10.03650910500437s
net: 1000 iterations took 6.9620567110105185s
$ # After
$ ./replay.py -n 1000 ptinit.log ptreboot.log
deep copy: 1000 iterations took 6.784729664999759s
ops script: 1000 iterations took 9.0453162470003s
net: 1000 iterations took 2.260586582000542s
Change-Id: I6826b99950fa1561d292908c4ffff1137ecaa1bc
Signed-off-by: Andrew Jeffery <andrew@aj.id.au>
diff --git a/obmc/utils/pathtree.py b/obmc/utils/pathtree.py
index da56cfb..dc48609 100644
--- a/obmc/utils/pathtree.py
+++ b/obmc/utils/pathtree.py
@@ -94,6 +94,7 @@
class PathTree:
def __init__(self):
self.root = {}
+ self.cache = {}
def _try_delete_parent(self, elements):
if len(elements) == 1:
@@ -131,6 +132,7 @@
return True
def __delitem__(self, key):
+ del self.cache[key]
kids = 'children'
elements = ['/'] + list(filter(bool, key.split('/')))
d = self.root
@@ -144,6 +146,7 @@
self._try_delete_parent(elements)
def __setitem__(self, key, value):
+ self.cache[key] = value
kids = 'children'
elements = ['/'] + list(filter(bool, key.split('/')))
d = self.root
@@ -154,7 +157,7 @@
d[elements[-1]].update({kids: children, 'data': value})
def __getitem__(self, key):
- return self._get_node(key).get('data')
+ return self.cache[key]
def setdefault(self, key, default):
if not self.get(key):