'Re: [tor-dev] Trouble with onionperf visualize and S61 performance experiments'

[prev in list] [next in list] [prev in thread] [next in thread] 

List:       tor-dev
Subject:    Re: [tor-dev] Trouble with onionperf visualize and S61 performance experiments
From:       Karsten Loesing <karsten () torproject ! org>
Date:       2020-12-07 7:56:58
Message-ID: c4c09271-b8a7-698b-9375-ef1164d75496 () torproject ! org
[Download RAW message or body]

On 2020-12-02 20:23, George Kadianakis wrote:
> Karsten Loesing <karsten@torproject.org> writes:
> 
> > On 2020-11-03 15:01, George Kadianakis wrote:
> > > Hello Karsten,
> > 
> > Hi George!
> > 
> > > hope you are doing well!
> > > 
> > > I've been working on the S61 performance experiments [0] and I would appreciate
> > > some help with onionperf.
> > > 
> > <snip>
> > 
> > > Then for another onionperf run (attached as 'onionperf-44028.json.xz') it gave \
> > > me a different error: 
> > > $ onionperf visualize --data onionperf.analysis.json.xz "Test Measurements"
> > > 2020-11-03 15:50:03 1604411403.946028 [onionperf] [INFO] loading analysis \
> > > results from /user/tmp/onionperf/analysis/onionperf.analysis.json.xz 2020-11-03 \
> > > 15:50:03 1604411403.976088 [onionperf] [INFO] done! Traceback (most recent call \
> > > last): File "/user/.local/bin/onionperf", line 4, in <module>
> > > __import__('pkg_resources').run_script('OnionPerf==0.8', 'onionperf')
> > > File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 650, in \
> > > run_script self.require(requires)[0].run_script(script_name, ns)
> > > File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 1453, in \
> > > run_script exec(script_code, namespace, namespace)
> > > File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", \
> > > line 622, in <module> File \
> > > "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", \
> > > line 382, in main File \
> > > "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", \
> > > line 522, in visualize File \
> > > "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py", \
> > > line 38, in plot_all File \
> > > "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py", \
> > >                 line 129, in __extract_data_frame
> > > KeyError: '44028'
> > > 
> > > Because of the two errors above I'm unable to proceed with producing any
> > > results.  Any idea what I might be doing wrong? Is there any other data
> > > you might need to help me with this?
> > 
> > In this case it's failing even before writing a CSV file. Still, when
> > looking at the .xz file I'm seeing only errors, too, so it's a related
> > issue here.
> > 
> 
> Hello again Karsten,

Hi George!

> your help was valuable last time and I managed to overcome those issues.
> 
> I've been doing more experiments recently (see
> https://gitlab.torproject.org/tpo/core/tor/-/issues/40157#note_2716591)
> and while the data collection is still ongoing, I decided to do some
> visualizations to make sure that I'm not collecting useless data all
> this time.
> 
> So I took collected onionperf data but when I tried to visualize them I
> got a similar error like the one from my previous email:
> 
> """
> $ onionperf visualize  --data onionperf.analysis.json.xz "bla" 
> 2020-12-02 21:19:04 1606936744.475013 [onionperf] [INFO] loading analysis results \
> from /home/user/tmp/onionperf/onionperf-data-2558933-close/onionperf.analysis.json.xz
>  2020-12-02 21:19:05 1606936745.245664 [onionperf] [INFO] done!
> Traceback (most recent call last):
> File "/home/user/.local/bin/onionperf", line 4, in <module>
> __import__('pkg_resources').run_script('OnionPerf==0.8', 'onionperf')
> File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 650, in \
> run_script self.require(requires)[0].run_script(script_name, ns)
> File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 1453, in \
> run_script exec(script_code, namespace, namespace)
> File "/home/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", \
> line 622, in <module> File \
> "/home/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", \
> line 382, in main File \
> "/home/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", \
> line 522, in visualize File \
> "/home/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py", \
> line 38, in plot_all File \
> "/home/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py", \
>                 line 129, in __extract_data_frame
> KeyError: '36178'
> """
> 
> However this time the onionperf.analysis.json.xz file looks more
> reasonable. I'm attaching it to this email just in case you have the
> time to check it out and tell me what might be going wrong.

This is related to the issues you saw before. Please try out the patch
that I wrote for #40012 last week for the visualization part (no need to
re-do the measurements):

$ git remote add karsten https://gitlab.torproject.org/karsten/onionperf.git

$ git fetch karsten

$ git checkout -b task-40012 karsten/task-40012

If you want, please comment on the ticket whether this worked for you!

All the best,
Karsten

> 
> FWIW, the data was collected in a manner similar to this:
> $ onionperf measure -i --tgen ./tgen --tor ./tor --torclient-conf-file ./torrc \
> --tgen-connect-ip 1.2.3.4 --tgen-listen-port 443 --tgen-connect-port 443 \
> --tgen-pause-initial 20 --tgen-pause-between 2 --tgen-transfer-size '50 KiB' 
> Thanks a lot!
> 

_______________________________________________
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

[prev in list] [next in list] [prev in thread] [next in thread] 