'Fwd: Arista Networks: Caching the BODY of a POST request for subsequent GETs'

[prev in list] [next in list] [prev in thread] [next in thread] 

List: varnish-misc
Subject: Fwd: Arista Networks: Caching the BODY of a POST request for subsequent GETs
From: Sidhartha Agrawal <sid () arista ! com>
Date: 2020-05-17 6:42:37
Message-ID: CAPyp3mCzrcYhE_0drb632ZO_0W8V01XHqYFZPTktjH0sip3VXw () mail ! gmail ! com
[Download RAW message or body]

[Attachment #2 (multipart/alternative)]

Hi Varnish Community,

I am a software developer at Arista Network in our Tools and Infrastructure
Group. We use Varnish(6.3.1) to cache our build objects such as RPMS and
images and Varnish is an integral part of our internal build system.

Some of these objects are of the order of ~500 MB, so a GET of multiple
such objects can put a significant load on the backend(OpenSwift). Since
there is an adequate temporal locality(within a day) between the POST of
the large object and the subsequent GET, we would like to cache the object
when the POST happens, so that the subsequent GET is guaranteed to find it
in the cache(unless evicted or expired).

As I understand, Varnish bypasses the cache for POST and passes it straight
to the backend. We would like to change this behavior of POST to also add
an entry for the object in the cache if the write to the backend was
successful.

A cursory google search for "Caching Post requests on Varnish" lead to me
this article
<https://docs.varnish-software.com/tutorials/caching-post-requests/#step-1-prepare-the-vcl-for-caching-post-requests>.
 Though I do not think this is exactly what we need. The article talks about
avoiding a write to the backend if another POST request is made with the
same body. That is different from what we are trying to do.

I would appreciate any insights you can provide in to:

   -

   What changes do I need to make to the VCL to achieve our desired behavior
   -

   Comments, on whether this is a bad idea for some reason.

I have been following the VCL state machine from this website
<https://varnish-cache.org/docs/6.3/reference/states.html#reference-states>.
I appreciate the help.

-Sid

[Attachment #5 (text/html)]

<div dir="ltr"><div class="gmail_default" \
style="font-family:monospace,monospace">Hi \
Varnish Community, </div><div class="gmail_quote"><div dir="ltr"><div \
style="font-family:monospace,monospace">I \
am a software developer at Arista Network in our Tools and Infrastructure Group. We \
use Varnish(6.3.1) to cache our build objects such as RPMS and images and Varnish is \
an integral part of our internal build system. Some \
of these objects are of the order of ~500 MB, so a GET of multiple such objects can \
put a significant load on the backend(OpenSwift). Since there is an adequate temporal \
locality(within a day) between the POST of the large object and the subsequent GET, \
we would like to cache the object when the POST happens, so that the subsequent GET \
is guaranteed to find it in the cache(unless evicted or expired). As \
I understand, Varnish bypasses the cache for POST and passes it straight to the \
backend. We would like to change this behavior of POST to also add an entry for the \
object in the cache if the write to the backend was successful. A \
cursory google search for "<span \
style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent; \
font-weight:700;font-style:italic;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Caching \
Post requests on Varnish" \
lead to me this <a \
href="https://docs.varnish-software.com/tutorials/caching-post-requests/#step-1-prepare-the-vcl-for-caching-post-requests" \
style="text-decoration-line:none" target="_blank"><span \
style="font-size:11pt;font-family:Arial;background-color:transparent;font-variant-nume \
ric:normal;font-variant-east-asian:normal;text-decoration-line:underline;vertical-align:baseline;white-space:pre-wrap">article</a>. \
Though I do not think this is exactly what we need. The article talks about avoiding \
a write to the backend if another POST request is made with the same body. That is \
different from what we are trying to do. I \
would appreciate any insights you can provide in to:<ul \
style="margin-top:0px;margin-bottom:0px"><li dir="ltr" \
style="list-style-type:disc;font-size:11pt;font-family:Arial;color:rgb(0,0,0);backgrou \
nd-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">What \
changes do I need to make to the VCL to achieve our desired \
behavior</li><li dir="ltr" \
style="list-style-type:disc;font-size:11pt;font-family:Arial;color:rgb(0,0,0);backgrou \
nd-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Comments, \
on whether this is a bad idea for some reason.</li></ul> I \
have been following the VCL state machine from this <span \
style="font-size:11pt;font-family:Arial;background-color:transparent;font-variant-nume \
ric:normal;font-variant-east-asian:normal;text-decoration-line:underline;vertical-align:baseline;white-space:pre-wrap"><a \
href="https://varnish-cache.org/docs/6.3/reference/states.html#reference-states" \
target="_blank">website</a>. \
I \
appreciate the help. -Sid </div></div>
 </div></div>

_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

[prev in list] [next in list] [prev in thread] [next in thread] 