Configure ATS reverse proxy with live HLS origin returning Cache-Control: no-cache

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Configure ATS reverse proxy with live HLS origin returning Cache-Control: no-cache

ezko
Hi,
I am trying to configure ATS reverse proxy with a live HLS origin which
returns Cache-Control: no-cache on everything.
By default ATS doesn't cache anything because the origin server objects have
response headers with Cache-Control: no-cache (this is documented).

The only way i found to get it to work was pinning the objects by adding the
following to cache.config:

url_regex=<path to origin> ttl-in-cache=10s

i choose 10 seconds because that is the target duration in the HLS playlist
(#EXT-X-TARGETDURATION)

But this seems like a very manual and verbose method , is there any better
way to configure live content from an origin that returns no-cache ?
Thanks.




--
Sent from: http://apache-traffic-server.24303.n7.nabble.com/
Reply | Threaded
Open this post in threaded view
|

Re: Configure ATS reverse proxy with live HLS origin returning Cache-Control: no-cache

Miles Libbey
We tend to use the header_rewrite plugin for this in our remap rules.

map http://inbound.example.com \
    http://origin.example.com \
    @plugin=header_rewrite.so @pparam=hr.config


With hr.config containing:

cond %{READ_RESPONSE_HDR_HOOK} [AND]
cond %{STATUS} >199 [AND]
cond %{STATUS} <400
    rm-header Pragma
    rm-header Expires
    set-header Cache-Control "max-age=604800, public"

(so that we only cache the success not the missing objects).

miles

On Sun, Dec 3, 2017 at 8:54 AM, ezko <[hidden email]> wrote:

> Hi,
> I am trying to configure ATS reverse proxy with a live HLS origin which
> returns Cache-Control: no-cache on everything.
> By default ATS doesn't cache anything because the origin server objects have
> response headers with Cache-Control: no-cache (this is documented).
>
> The only way i found to get it to work was pinning the objects by adding the
> following to cache.config:
>
> url_regex=<path to origin> ttl-in-cache=10s
>
> i choose 10 seconds because that is the target duration in the HLS playlist
> (#EXT-X-TARGETDURATION)
>
> But this seems like a very manual and verbose method , is there any better
> way to configure live content from an origin that returns no-cache ?
> Thanks.
>
>
>
>
> --
> Sent from: http://apache-traffic-server.24303.n7.nabble.com/
Reply | Threaded
Open this post in threaded view
|

Re: Configure ATS reverse proxy with live HLS origin returning Cache-Control: no-cache

Burak Sarp
In reply to this post by ezko
Hi, 

You have to  handle headers on origin servers. In addition to that there are some parameters in records.config ingore no cache both client and servers.

Sarp


Sent from Yahoo Mail for iPhone

On Sunday, December 3, 2017, 8:54 AM, ezko <[hidden email]> wrote:

Hi,
I am trying to configure ATS reverse proxy with a live HLS origin which
returns Cache-Control: no-cache on everything.
By default ATS doesn't cache anything because the origin server objects have
response headers with Cache-Control: no-cache (this is documented).

The only way i found to get it to work was pinning the objects by adding the
following to cache.config:

url_regex=<path to origin> ttl-in-cache=10s

i choose 10 seconds because that is the target duration in the HLS playlist
(#EXT-X-TARGETDURATION)

But this seems like a very manual and verbose method , is there any better
way to configure live content from an origin that returns no-cache ?
Thanks.




--
Reply | Threaded
Open this post in threaded view
|

Re: Configure ATS reverse proxy with live HLS origin returning Cache-Control: no-cache

ezko
In reply to this post by Miles Libbey
Thanks !!!

But can you explain why you used such a large max-age ?
When i tried it with your max-age value it didn't work (i kept getting the
same old playlist and segments).
So I tried changing max-age to 10 seconds and now it seems to work much
better.

If my HLS playlist has 3 segments and each segment is 10 second each , then
why should i cache anything for more then 30 seconds ?

For live HLS , since everything is short lived and assuming i only have a
small number of streams and  enough RAM , is there a way to configure ATS to
work with the in-memory cache only ?

thanks again.



Miles Libbey wrote

> We tend to use the header_rewrite plugin for this in our remap rules.
>
> map http://inbound.example.com \
>     http://origin.example.com \
>     @plugin=header_rewrite.so @pparam=hr.config
>
>
> With hr.config containing:
>
> cond %{READ_RESPONSE_HDR_HOOK} [AND]
> cond %{STATUS} >199 [AND]
> cond %{STATUS} <400
>     rm-header Pragma
>     rm-header Expires
>     set-header Cache-Control "max-age=604800, public"
>
> (so that we only cache the success not the missing objects).
>
> miles
>
> On Sun, Dec 3, 2017 at 8:54 AM, ezko &lt;

> erez.koler@

> &gt; wrote:
>> Hi,
>> I am trying to configure ATS reverse proxy with a live HLS origin which
>> returns Cache-Control: no-cache on everything.
>> By default ATS doesn't cache anything because the origin server objects
>> have
>> response headers with Cache-Control: no-cache (this is documented).
>>
>> The only way i found to get it to work was pinning the objects by adding
>> the
>> following to cache.config:
>>
>> url_regex=
> <path to origin>
>  ttl-in-cache=10s
>>
>> i choose 10 seconds because that is the target duration in the HLS
>> playlist
>> (#EXT-X-TARGETDURATION)
>>
>> But this seems like a very manual and verbose method , is there any
>> better
>> way to configure live content from an origin that returns no-cache ?
>> Thanks.
>>
>>
>>
>>
>> --
>> Sent from: http://apache-traffic-server.24303.n7.nabble.com/





--
Sent from: http://apache-traffic-server.24303.n7.nabble.com/
Reply | Threaded
Open this post in threaded view
|

Re: Configure ATS reverse proxy with live HLS origin returning Cache-Control: no-cache

Miles Libbey
On Sun, Dec 3, 2017 at 10:44 AM, ezko <[hidden email]> wrote:
> Thanks !!!
>
> But can you explain why you used such a large max-age ?

It's just an example.

> When i tried it with your max-age value it didn't work (i kept getting the
> same old playlist and segments).
> So I tried changing max-age to 10 seconds and now it seems to work much
> better.
>
> If my HLS playlist has 3 segments and each segment is 10 second each , then
> why should i cache anything for more then 30 seconds ?

I think there are 2 distinct HLS cases. For live streaming, the
playlist will change frequently (but usually, the encoded segments
won't). For already recorded content, neither likely change. You might
distinguish between the 2 cases in your remap rule. Or, you could add
extra header_rewrite rules like

cond %{CLIENT-URL:PATH} /\.m3u8/

or perhaps look at the mime type like

cond %{HEADER:Content-Type} /mpegURL/

(though, now re-reading, perhaps you only have the live streaming case.)

> For live HLS , since everything is short lived and assuming i only have a
> small number of streams and  enough RAM , is there a way to configure ATS to
> work with the in-memory cache only ?

No, the ram cache is too intertwined to the disk cache to do that.
miles


> thanks again.
>
>
>
> Miles Libbey wrote
>> We tend to use the header_rewrite plugin for this in our remap rules.
>>
>> map http://inbound.example.com \
>>     http://origin.example.com \
>>     @plugin=header_rewrite.so @pparam=hr.config
>>
>>
>> With hr.config containing:
>>
>> cond %{READ_RESPONSE_HDR_HOOK} [AND]
>> cond %{STATUS} >199 [AND]
>> cond %{STATUS} <400
>>     rm-header Pragma
>>     rm-header Expires
>>     set-header Cache-Control "max-age=604800, public"
>>
>> (so that we only cache the success not the missing objects).
>>
>> miles
>>
>> On Sun, Dec 3, 2017 at 8:54 AM, ezko &lt;
>
>> erez.koler@
>
>> &gt; wrote:
>>> Hi,
>>> I am trying to configure ATS reverse proxy with a live HLS origin which
>>> returns Cache-Control: no-cache on everything.
>>> By default ATS doesn't cache anything because the origin server objects
>>> have
>>> response headers with Cache-Control: no-cache (this is documented).
>>>
>>> The only way i found to get it to work was pinning the objects by adding
>>> the
>>> following to cache.config:
>>>
>>> url_regex=
>> <path to origin>
>>  ttl-in-cache=10s
>>>
>>> i choose 10 seconds because that is the target duration in the HLS
>>> playlist
>>> (#EXT-X-TARGETDURATION)
>>>
>>> But this seems like a very manual and verbose method , is there any
>>> better
>>> way to configure live content from an origin that returns no-cache ?
>>> Thanks.
>>>
>>>
>>>
>>>
>>> --
>>> Sent from: http://apache-traffic-server.24303.n7.nabble.com/
>
>
>
>
>
> --
> Sent from: http://apache-traffic-server.24303.n7.nabble.com/
Reply | Threaded
Open this post in threaded view
|

Re: Configure ATS reverse proxy with live HLS origin returning Cache-Control: no-cache

ezko
got it.
i will add an extra header_rewrite for .m3u8
thanks again.


On Sun, Dec 3, 2017 at 10:44 AM, ezko &lt;erez.koler@&gt; wrote:
> Thanks !!!
>
> But can you explain why you used such a large max-age ?

It's just an example.

> When i tried it with your max-age value it didn't work (i kept getting the
> same old playlist and segments).
> So I tried changing max-age to 10 seconds and now it seems to work much
> better.
>
> If my HLS playlist has 3 segments and each segment is 10 second each ,
> then
> why should i cache anything for more then 30 seconds ?

I think there are 2 distinct HLS cases. For live streaming, the
playlist will change frequently (but usually, the encoded segments
won't). For already recorded content, neither likely change. You might
distinguish between the 2 cases in your remap rule. Or, you could add
extra header_rewrite rules like

cond %{CLIENT-URL:PATH} /\.m3u8/

or perhaps look at the mime type like

cond %{HEADER:Content-Type} /mpegURL/

(though, now re-reading, perhaps you only have the live streaming case.)

> For live HLS , since everything is short lived and assuming i only have a
> small number of streams and  enough RAM , is there a way to configure ATS
> to
> work with the in-memory cache only ?

No, the ram cache is too intertwined to the disk cache to do that.
miles






--
Sent from: http://apache-traffic-server.24303.n7.nabble.com/