WebPagetest Forums
wpt_batch.py and custom connectivity - Printable Version

+- WebPagetest Forums (https://www.webpagetest.org/forums)
+-- Forum: WebPagetest (/forumdisplay.php?fid=7)
+--- Forum: Bugs/Issues (/forumdisplay.php?fid=10)
+--- Thread: wpt_batch.py and custom connectivity (/showthread.php?tid=11882)



wpt_batch.py and custom connectivity - andydavies - 11-14-2012 09:43 PM

Hi Pat,

I'm trying to submit some batch jobs with custom connectivity profiles...

If I use the default profile everything is OK, but if I try to set a custom profile e.g.

Quote:python wpt_batch.py -u test_urls.txt -o Sydney:Chrome --connectivity=custom --bwup=384 --bwdown=2000 --latency=50 --plr=1 -k apikey -s http://www.webpagetest.org/ -r 1

I get

Code:
<bwDown>0</bwDown>
<bwUp>0</bwUp>
<latency>0</latency>
<plr>('1',)</plr>

A similar thing happens if I leave the --plr out

Code:
<bwDown>0</bwDown>
<bwUp>0</bwUp>
<latency>0</latency>
<plr>('0',)</plr>

Any ideas what I'm missing?


RE: wpt_batch.py and custom connectivity - pmeenan - 11-20-2012 12:50 AM

Any chance you can log the runtest.php url before it sends it out (or ping me and run it again so I can grab it from the access logs)?

Looking at the runtest.php code it looks like the bandwidth and latency params are extracted as ints and the plr is extracted as a string so the strange plr is probably happening to all of them and the int versions are just getting set to zero.

If I had to guess I would say that something on the python size is causing the test_params dictionary to encode the query params in a strange way but I'm not a big python guy so it may take me a little while to look into it.


RE: wpt_batch.py and custom connectivity - andydavies - 11-21-2012 12:12 AM

Edit: Got time this week so dug into it and found the issue - guess no-one has used custom bandwidth with the batch submitter!

The URL being sent is:

Code:
http://www.webpagetest.org/runtest.php?bwIn=(2000,)&plr=(1,)&private=1&video=0&latency=(50,)&runs=1&f=xml&url=http://www.debenhams.com&k=apikey&priority=6&bwOut=(384,)&mv=1&location=Sydney:Chrome.custom&fvonly=1

The comma on the end of the following lines converts the value to a tuple

Code:
test_params['bwOut'] = options.bwup,
test_params['bwIn'] = options.bwdown,
test_params['latency'] = options.latency,
test_params['plr'] = options.plr,
test_params['location'] = options.location + '.custom'

Removing the commas i.e.

Code:
test_params['bwOut'] = options.bwup
test_params['bwIn'] = options.bwdown
test_params['latency'] = options.latency
test_params['plr'] = options.plr
test_params['location'] = options.location + '.custom'

Will give the correct query string values URL:

Code:
http://www.webpagetest.org/runtest.php?bwIn=2000&plr=1&private=1&video=0&latency=50&runs=1&f=xml&url=http://www.debenhams.com&k=apikey&priority=6&bwOut=384&mv=1&location=Sydney:Chrome.custom&fvonly=1



RE: wpt_batch.py and custom connectivity - pmeenan - 11-21-2012 02:12 AM

Awesome. Thanks for digging into it. I just pushed the fix.