WPTDriver NTLM authentication

[ash]

Hi,

I just migrated one of my agents from URLBlast do WPTDriver.

Unfortunately some of the tests started to fail. This applies only to tested sites that require NTLM authentication. The problem is that in waterfall I now can see all the responses of the server to the browser that require to provide credentials:

HTTP/1.1 401 Unauthorized
WWW-Authenticate: NTLM

HTTP/1.1 401 Unauthorized
WWW-Authenticate: NTLM TlRMTVNTUAACAAAADAAMADgAAAAFgomiFHd18hyttBgAAAAAAAA[...]

After that browser sends some hashes to the server and page loads as expected. All subsequent connections to the server are authenticated.

This is great because UrlBlaster does not separate this behavior from reported http requests. And it is impossible to see how fast it is done and TTFB is reported incorrectly.



The only problem is that after that (in my opinion normal behavior of the server and browser) the test has error status 401.



Is this big deal to correct this? My automated tools that use WPT Server have problem with reporting statuses of tests. They are not able to distinguish described case from real problem with authentication.

Regards,

[ash]

I investigated this issue deeper and noticed that for described case when result for run is other than "0" average is not calculated. Only median. Tools that rely on average of runs have problem in case of "401" responses from the server.

I have following in xml result:

Code:

<response><statusCode>200</statusCode><statusText>Ok</statusText>
...
<average><firstView></firstView></average>
...
<median><firstView><run>1</run><URL>https://my.url.com/</URL><loadTime>3460</loadTime><TTFB>1202</TTFB><bytesOut>89710</bytesOut><bytesOutDoc>74140</bytesOutDoc><bytesIn>2460365</bytesIn><bytesInDoc>2077324</bytesInDoc><connections>6</connections><requests>62</requests><requestsDoc>55</requestsDoc><responses_200>58</responses_200><responses_404>0</responses_404><responses_other>4</responses_other><result>401</result><render>2483</render><fullyLoaded>4373</fullyLoaded>
...
<run><id>1</id><firstView><results><URL>https://my.url.com/</URL><loadTime>3460</loadTime><TTFB>1202</TTFB><bytesOut>89710</bytesOut><bytesOutDoc>74140</bytesOutDoc><bytesIn>2460365</bytesIn><bytesInDoc>2077324</bytesInDoc><connections>6</connections><requests>62</requests><requestsDoc>55</requestsDoc><responses_200>58</responses_200><responses_404>0</responses_404><responses_other>4</responses_other><result>401</result><render>2483</render><fullyLoaded>4373</fullyLoaded><cached>0</cached><docTime>3460</docTime><domTime>0</domTime>

[ash]

There is another problem with tests of pages that require authentication by sending 401 responses. Repeat view test is not performed:

Code:

2014/04/18 11:54:02 - Test Created
2014/04/18 11:54:06 - Starting test (initiated by tester PLKRK-L-R9A1FAP-10.3.52.208)
2014/04/18 11:54:37 - Test Run Complete. Run: 1, Cached: 0, Done: , Tester: xxx-10.3.52.208
2014/04/18 11:54:37 - Test Run Complete. Run: 1, Cached: 1, Done: , Tester: xxx-10.3.52.208, Test Run Error: "Skipped repeat view, first view failed: Test run failed with result code 401"
2014/04/18 11:54:37 - Test Run Complete. Run: 1, Cached: 1, Done: 1, Tester: xxx-10.3.52.208, Test Run Error: "Skipped repeat view, first view failed: Test run failed with result code 401"
2014/04/18 11:54:37 - Test Complete
2014/04/18 11:54:38 - Test post-processing complete

[jaythos]

Any updates on this bug? We're looking at implementing a private instance to test our internal web sites, most of which use NTLM auth. We noticed the same problems.

It's valuable for us to capture the 401s but as mentioned before we don't want the test to report as an failure.

Thanks.

[pmeenan]

Is it just the 401's or is there also a functional problem getting to the page? It should be easy enough for me to treat 401's more like redirects which will make them non-errors and report a correct TTFB.

[jaythos]

In my case it's just the 401s, the first test completes as you would expect (captures the entire waterfall) just reports as an error and skips the repeat test (plus the other issues noted by OP). Treating them more like redirects sounds like an appropriate solution.

Thanks!

[ash]

(05-15-2014 12:43 AM)pmeenan Wrote:  Is it just the 401's or is there also a functional problem getting to the page? It should be easy enough for me to treat 401's more like redirects which will make them non-errors and report a correct TTFB.

Treating them like redirects would be perfect IMO.

Thanks!

[ash]

(05-15-2014 12:43 AM)pmeenan Wrote:  Is it just the 401's or is there also a functional problem getting to the page? It should be easy enough for me to treat 401's more like redirects which will make them non-errors and report a correct TTFB.

Patrick, is it possible to predict when this change can be implemented in WPTDriver?

[pmeenan]

I just pushed 177 which has an attempt at a fix (I don't have a test rig with NTLM auth so I can't verify it is working). I think I got all of the places in the code where it looks for a > 400 response code and have it treating 401's as 301 and 302's: https://github.com/WPO-Foundation/webpag...4d9f4b81f7

Here's how to grab the latest binary: https://sites.google.com/a/webpagetest.o...est-Agents

Let me know if it still doesn't work.

[ash]

Works perfectly now.

Thanks a lot for your work, Patrick.