Problems with non-Trident agents?

[PauloIntechnica]

Are there supposed to be problems with non-Trident agents? I've been trying lots of different scripts in IE7/8/9, Firefox and Chrome and it appears that important commands (setValue, clickAndWait) do not work at all in Firefox/Chrome driving. Also, is IE driving expected to fail from time to time or is it supposed to work pretty reliably?

The behaviour I'm showing is always 100% consistent in Firefox/Chrome both in WebpageTest.org and in our private instances that we're testing. In IE, it works almost always, only occasionally failing and exhibiting similar behavior as Firefox/Chrome.

Code:

navigate    http://www.intechnica.co.uk/
clickAndWait    className=performance
clickAndWait    innerText=Products

IE GETs 1st page, then navigates to 2nd link, then to 3rd link.
Firefox/Chrome GET 1st page, then seem to timeout.

(IE) http://www.webpagetest.org/result/130418_25_JCM/
(Ch) http://www.webpagetest.org/result/130418_AJ_JCV/
(Ff) http://www.webpagetest.org/result/130418_22_JD1/

Code:

navigate    http://casa.sapo.pt/EN/For-sale/Apartments/Ovar/?sa=1
setValue    id=txtPrecoMin    40000
setValue    id=txtPrecoMax    70000
clickAndWait    innerText=Search

IE GETs 1st page, fills the boxes with the values, then submits the form and gets the 2nd page.
Firefox/Chrome GET 1st page, then seem to timeout - don't even fill the boxes with min/max price.

(IE) http://www.webpagetest.org/result/130418_8Q_JPE/
(Ch) http://www.webpagetest.org/result/130418_DV_JPH/
(Ff) http://www.webpagetest.org/result/130418_P7_JPM/

(IE PT) http://www.webpagetest.org/result/130418_SH_HQ4/
(Ch PT) http://www.webpagetest.org/result/130418_MJ_HXD/
(Ff PT) http://www.webpagetest.org/result/130418_41_J2C/

Code:

navigate    http://casa.sapo.pt/EN/For-sale/Apartments/Ovar/?sa=1
exec    document.getElementById("txtPrecoMin").value="40000";
exec    document.getElementById("txtPrecoMax").value="70000";
execAndWait    Submeter();

IE behaves as expected (as previous script). IE failed once, which happens from time to time.
Firefox/Chrome behave as in previous script except now they manage to fill the min/max price boxes.

(IE Fail) http://www.webpagetest.org/result/130418_1B_JPS/
(IE) http://www.webpagetest.org/result/130418_2B_JVT/
(Ch) http://www.webpagetest.org/result/130418_MA_JPT/
(Ff) http://www.webpagetest.org/result/130418_E8_JPZ/

(IE PT) http://www.webpagetest.org/result/130418_X0_J7Z/
(Ch PT) http://www.webpagetest.org/result/130418_KF_J82/
(Ff PT) http://www.webpagetest.org/result/130418_XJ_J86/

[pmeenan]

IE is supposed to work pretty reliably and the non-IE agent's scripting still lags IE's by a fair bit (though I thought it was in better state than you are seeing). Bringing the Chrome and Firefox test agents up to feature parity with the IE agents is being worked on through the year (hopefully this quarter but it will be incremental and over time).

[PauloIntechnica]

Thanks for the answers.

We are investigating WPT - its capabilities, its reliability etc. So we are bound, of course, to run into many (most or all, if I do my investigation well) of its limitations.

I can't say I'm not somewhat disappointed at a few things. But the truth is that WPT seems to have been designed for slightly different goals (deep analysis of single http GETs) than those we are looking for (user journeys performance overview). It does suit really well its goals and I'm kind of thinking it will still end up being suitable enough for what we're intending.

[pmeenan]

If you are doing more automation it may be worth looking into the jode.js agent (in the source tree under agents/js). It is limited to Chrome but works across platforms and can run full webdriver scripts.

[pmeenan]

The js agent is still quite alpha - just a warning because a lot of other stuff isn't there for it yet (optimization checks and most of the regular advanced options).