[prev in list] [next in list] [prev in thread] [next in thread] 

List:       httpclient-commons-dev
Subject:    [jira] [Commented] (HTTPCLIENT-2301) BasicHttpClientConnectionManager returns response of previous r
From:       "Pierre N. (Jira)" <jira () apache ! org>
Date:       2023-09-28 14:59:00
Message-ID: JIRA.13552285.1695894925000.110269.1695913140091 () Atlassian ! JIRA
[Download RAW message or body]


    [ https://issues.apache.org/jira/browse/HTTPCLIENT-2301?page=com.atlassian.jira.pl \
ugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17770097#comment-17770097 \
] 

Pierre N. commented on HTTPCLIENT-2301:
---------------------------------------

I expected this kind of response because that's exactly what I thought when I \
encountered this bug...

For the story, I discovered this bug in production with \
[Serposcope|http://atlassian.com]. One of my user had [inverted \
response|https://forum.serposcope.com/d/101-error-with-scrapingant-api], when sending \
a request to Google it would sometimes have the response of a previous request.

It was hell to debug... To the point I suspected, like you, the server was \
responsible for this. I would never have expected a library I have been using for a \
decade to have such bug. 

But in my production code I don't use Wiremock... I just used it to troubleshooting \
and reproduce this bug.

But here you go, the same bug using httpbin.org, no mocking : 

{quote}
internal class HttpClientDebug {

    @Test
    fun bugbug() {
        val client = HttpClients.custom()
            .setConnectionManager(BasicHttpClientConnectionManager())
//            .setConnectionManager(PoolingHttpClientConnectionManager())
            .setDefaultRequestConfig(
                RequestConfig.custom()
                    .setResponseTimeout(Timeout.ofSeconds(1))
                    .build()
            )
            .build()

        client.executeAndLog("https://httpbin.org/anything?1")
        client.executeAndLog("https://httpbin.org/delay/2?2")
        client.executeAndLog("https://httpbin.org/delay/2?2")
        client.executeAndLog("https://httpbin.org/anything?3")
        client.executeAndLog("https://httpbin.org/anything?3")
        client.executeAndLog("https://httpbin.org/anything?3")
    }

    private fun CloseableHttpClient.executeAndLog(uri: String) {
        try {
            execute(HttpGet(uri)) { println("$uri -> \
${EntityUtils.toString(it.entity).split("\n").filter { it.contains("url") }}") }  } \
catch (ex: Exception) {  println("$uri -> $ex")
        }
    }
}
{quote}

It returns: 

{quote}
https://httpbin.org/anything?1 -> [  "url": "https://httpbin.org/anything?1"]
https://httpbin.org/delay/2?2 -> java.net.SocketTimeoutException: Read timed out
https://httpbin.org/delay/2?2 -> java.net.SocketTimeoutException: Read timed out
https://httpbin.org/anything?3 -> java.net.SocketTimeoutException: Read timed out
https://httpbin.org/anything?3 -> [  "url": "https://httpbin.org/delay/2?2"]
https://httpbin.org/anything?3 -> [  "url": "https://httpbin.org/anything?3"]
{quote}

Replace with PoolingHttpClientConnectionManager and it will work correctly : 

{quote}
https://httpbin.org/anything?1 -> [  "url": "https://httpbin.org/anything?1"]
https://httpbin.org/delay/2?2 -> java.net.SocketTimeoutException: Read timed out
https://httpbin.org/delay/2?2 -> java.net.SocketTimeoutException: Read timed out
https://httpbin.org/anything?3 -> [  "url": "https://httpbin.org/anything?3"]
https://httpbin.org/anything?3 -> [  "url": "https://httpbin.org/anything?3"]
https://httpbin.org/anything?3 -> [  "url": "https://httpbin.org/anything?3"]
{quote} 

Do you think the bug is also on httpbin.org ? :p

> BasicHttpClientConnectionManager returns response of previous requests
> ----------------------------------------------------------------------
> 
> Key: HTTPCLIENT-2301
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2301
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
> Affects Versions: 5.2.1, 5.3-alpha1
> Environment: JAVA 18
> kotlin 1.7.22
> Ubuntu 22.04 x64
> Reporter: Pierre N.
> Priority: Major
> Attachments: test.kt
> 
> 
> BasicHttpClientConnectionManager doesn't handle request timeout correctly and \
> return the response of a previous request. The unit test below should output:
> {quote}
> http://localhost:41751/1 -> 1
> http://localhost:41751/2 -> java.net.SocketTimeoutException: Read timed out
> http://localhost:41751/2 -> java.net.SocketTimeoutException: Read timed out
> http://localhost:41751/3 -> 3
> http://localhost:41751/3 -> 3
> http://localhost:41751/3 -> 3
> {quote}
> But instead it returns:
> {quote}
> http://localhost:33875/1 -> 1
> http://localhost:33875/2 -> java.net.SocketTimeoutException: Read timed out
> http://localhost:33875/2 -> java.net.SocketTimeoutException: Read timed out
> http://localhost:33875/3 -> java.net.SocketTimeoutException: Read timed out
> http://localhost:33875/3 -> 2
> http://localhost:33875/3 -> 3
> {quote}
> As you can see it returns 2 when requesting the uri /3 which returned 3.
> Also it timeout on the first request to /3 while it shouldn't.
> Replace with PoolingHttpClientConnectionManager() and it works as expected.
> Kotlin unit test to reproduce :
> {quote}
> import com.github.tomakehurst.wiremock.client.WireMock.*
> import com.github.tomakehurst.wiremock.junit5.WireMockRuntimeInfo
> import com.github.tomakehurst.wiremock.junit5.WireMockTest
> import org.apache.hc.client5.http.classic.methods.HttpGet
> import org.apache.hc.client5.http.config.RequestConfig
> import org.apache.hc.client5.http.impl.classic.CloseableHttpClient
> import org.apache.hc.client5.http.impl.classic.HttpClients
> import org.apache.hc.client5.http.impl.io.BasicHttpClientConnectionManager
> import org.apache.hc.core5.http.io.entity.EntityUtils
> import org.apache.hc.core5.util.Timeout
> import org.junit.jupiter.api.Test
> @WireMockTest
> internal class HttpClientRaceConditionDebug {
> @Test
> fun `debugging z`(wm: WireMockRuntimeInfo) {
> stubFor(get(urlEqualTo("/1")).willReturn(aResponse().withBody("1")))
> stubFor(get(urlEqualTo("/2")).willReturn(aResponse().withFixedDelay(2000).withBody("2")))
>  stubFor(get(urlEqualTo("/3")).willReturn(aResponse().withBody("3")))
> val client = HttpClients.custom()
> .setConnectionManager(BasicHttpClientConnectionManager())
> //            .setConnectionManager(PoolingHttpClientConnectionManager())
> .setDefaultRequestConfig(
> RequestConfig.custom()
> .setResponseTimeout(Timeout.ofSeconds(1))
> .build()
> )
> .build()
> client.executeAndLog("${wm.httpBaseUrl}/1")
> client.executeAndLog("${wm.httpBaseUrl}/2")
> client.executeAndLog("${wm.httpBaseUrl}/2")
> client.executeAndLog("${wm.httpBaseUrl}/3")
> client.executeAndLog("${wm.httpBaseUrl}/3")
> client.executeAndLog("${wm.httpBaseUrl}/3")
> }
> private fun CloseableHttpClient.executeAndLog(uri: String) {
> try {
> execute(HttpGet(uri)) { println("$uri -> ${EntityUtils.toString(it.entity)}") }
> } catch (ex: Exception) {
> println("$uri -> $ex")
> }
> }
> }
> {quote}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org


[prev in list] [next in list] [prev in thread] [next in thread]