child always re-validates object fetched from parent

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

child always re-validates object fetched from parent

Patrick O'Brien
Hello,

I am testing out hierarchical caching in ATS 7.1.1. Everything seems
to work, although it looks like the child node is always needing to
revalidate objects fetched from the parent node. Here are the headers
on an example GET:
https://gist.github.com/poblahblahblah/928aaa88c24ef3c05366433a514f1a28

The parent has the object stored in RAM and doesn't require
validation, but the child thinks the response is stale. Here is the
cache.config file on both the parent and the child:
https://gist.github.com/poblahblahblah/36b54ac8b7d75071c1342da6694d3c51

The problem with this is that the connection between the child and
parent is pretty flaky and when the parent is down we are met with a
502: https://gist.github.com/poblahblahblah/d50657354c815040faae50539fa45836

Here is the parent.config on the child:
https://gist.github.com/poblahblahblah/9f1dcb4cf674c21c80a27173bd0ef81b

Relevant configs from records.config for the parent:
https://gist.github.com/poblahblahblah/e6ac573e9f9cd63088c2e06e6544d801
Relevant configs from records.config for the child:
https://gist.github.com/poblahblahblah/5a6dc02a88b24cafca0e8c70d353b92e

Any guidance on this would be much appreciated.

-patrick
Reply | Threaded
Open this post in threaded view
|

Re: child always re-validates object fetched from parent

Jeremy Payne
Can you post the complete records.config from your child cache ?
If I had to guess, it looks like heurstics or fuzzytimer may be set too low ???
But if you provide  a company sanitized version of your records.config
that should give us a better understanding of your setup.






On Wed, Jan 3, 2018 at 6:10 PM, Patrick O'Brien
<[hidden email]> wrote:

> Hello,
>
> I am testing out hierarchical caching in ATS 7.1.1. Everything seems
> to work, although it looks like the child node is always needing to
> revalidate objects fetched from the parent node. Here are the headers
> on an example GET:
> https://gist.github.com/poblahblahblah/928aaa88c24ef3c05366433a514f1a28
>
> The parent has the object stored in RAM and doesn't require
> validation, but the child thinks the response is stale. Here is the
> cache.config file on both the parent and the child:
> https://gist.github.com/poblahblahblah/36b54ac8b7d75071c1342da6694d3c51
>
> The problem with this is that the connection between the child and
> parent is pretty flaky and when the parent is down we are met with a
> 502: https://gist.github.com/poblahblahblah/d50657354c815040faae50539fa45836
>
> Here is the parent.config on the child:
> https://gist.github.com/poblahblahblah/9f1dcb4cf674c21c80a27173bd0ef81b
>
> Relevant configs from records.config for the parent:
> https://gist.github.com/poblahblahblah/e6ac573e9f9cd63088c2e06e6544d801
> Relevant configs from records.config for the child:
> https://gist.github.com/poblahblahblah/5a6dc02a88b24cafca0e8c70d353b92e
>
> Any guidance on this would be much appreciated.
>
> -patrick
Reply | Threaded
Open this post in threaded view
|

Re: child always re-validates object fetched from parent

Patrick O'Brien
Hey Jeremy,

Here's the full config from the child node:
https://gist.github.com/poblahblahblah/a1abdad3d64ebdfba003c75e50c9fe00

Thanks for looking!

On Sun, Jan 7, 2018 at 8:13 PM, Jeremy Payne <[hidden email]> wrote:

> Can you post the complete records.config from your child cache ?
> If I had to guess, it looks like heurstics or fuzzytimer may be set too low ???
> But if you provide  a company sanitized version of your records.config
> that should give us a better understanding of your setup.
>
>
>
>
>
>
> On Wed, Jan 3, 2018 at 6:10 PM, Patrick O'Brien
> <[hidden email]> wrote:
>> Hello,
>>
>> I am testing out hierarchical caching in ATS 7.1.1. Everything seems
>> to work, although it looks like the child node is always needing to
>> revalidate objects fetched from the parent node. Here are the headers
>> on an example GET:
>> https://gist.github.com/poblahblahblah/928aaa88c24ef3c05366433a514f1a28
>>
>> The parent has the object stored in RAM and doesn't require
>> validation, but the child thinks the response is stale. Here is the
>> cache.config file on both the parent and the child:
>> https://gist.github.com/poblahblahblah/36b54ac8b7d75071c1342da6694d3c51
>>
>> The problem with this is that the connection between the child and
>> parent is pretty flaky and when the parent is down we are met with a
>> 502: https://gist.github.com/poblahblahblah/d50657354c815040faae50539fa45836
>>
>> Here is the parent.config on the child:
>> https://gist.github.com/poblahblahblah/9f1dcb4cf674c21c80a27173bd0ef81b
>>
>> Relevant configs from records.config for the parent:
>> https://gist.github.com/poblahblahblah/e6ac573e9f9cd63088c2e06e6544d801
>> Relevant configs from records.config for the child:
>> https://gist.github.com/poblahblahblah/5a6dc02a88b24cafca0e8c70d353b92e
>>
>> Any guidance on this would be much appreciated.
>>
>> -patrick
Reply | Threaded
Open this post in threaded view
|

Re: child always re-validates object fetched from parent

Jeremy Payne
Can you modify the below to reflect non-zero values ?

##############################################################################
# Heuristic cache expiration. Docs:
# https://docs.trafficserver.apache.org/records.config#heuristic-expiration
##############################################################################
# Setting all of these to zero in order for everything to be considered stale
# by ATS. If we want to change this in the future, this is where to do it.
CONFIG proxy.config.http.cache.heuristic_min_lifetime INT 0
CONFIG proxy.config.http.cache.heuristic_max_lifetime INT 0
CONFIG proxy.config.http.cache.heuristic_lm_factor FLOAT 0.000000
##############################################################################

On Mon, Jan 8, 2018 at 10:24 AM, Patrick O'Brien
<[hidden email]> wrote:

> Hey Jeremy,
>
> Here's the full config from the child node:
> https://gist.github.com/poblahblahblah/a1abdad3d64ebdfba003c75e50c9fe00
>
> Thanks for looking!
>
> On Sun, Jan 7, 2018 at 8:13 PM, Jeremy Payne <[hidden email]> wrote:
>> Can you post the complete records.config from your child cache ?
>> If I had to guess, it looks like heurstics or fuzzytimer may be set too low ???
>> But if you provide  a company sanitized version of your records.config
>> that should give us a better understanding of your setup.
>>
>>
>>
>>
>>
>>
>> On Wed, Jan 3, 2018 at 6:10 PM, Patrick O'Brien
>> <[hidden email]> wrote:
>>> Hello,
>>>
>>> I am testing out hierarchical caching in ATS 7.1.1. Everything seems
>>> to work, although it looks like the child node is always needing to
>>> revalidate objects fetched from the parent node. Here are the headers
>>> on an example GET:
>>> https://gist.github.com/poblahblahblah/928aaa88c24ef3c05366433a514f1a28
>>>
>>> The parent has the object stored in RAM and doesn't require
>>> validation, but the child thinks the response is stale. Here is the
>>> cache.config file on both the parent and the child:
>>> https://gist.github.com/poblahblahblah/36b54ac8b7d75071c1342da6694d3c51
>>>
>>> The problem with this is that the connection between the child and
>>> parent is pretty flaky and when the parent is down we are met with a
>>> 502: https://gist.github.com/poblahblahblah/d50657354c815040faae50539fa45836
>>>
>>> Here is the parent.config on the child:
>>> https://gist.github.com/poblahblahblah/9f1dcb4cf674c21c80a27173bd0ef81b
>>>
>>> Relevant configs from records.config for the parent:
>>> https://gist.github.com/poblahblahblah/e6ac573e9f9cd63088c2e06e6544d801
>>> Relevant configs from records.config for the child:
>>> https://gist.github.com/poblahblahblah/5a6dc02a88b24cafca0e8c70d353b92e
>>>
>>> Any guidance on this would be much appreciated.
>>>
>>> -patrick
Reply | Threaded
Open this post in threaded view
|

Re: child always re-validates object fetched from parent

Patrick O'Brien
Good catch!

This didn't fix the issue by itself, but it did remind me that
proxy.config.http.cache.when_to_revalidate was set to 2 (Always
Stale).

I guess on our non-chained proxies everything worked because of our
rules in cache.config?

I appreciate the advice, Jeremy. Thank you!

On Mon, Jan 8, 2018 at 8:29 AM, Jeremy Payne <[hidden email]> wrote:

> Can you modify the below to reflect non-zero values ?
>
> ##############################################################################
> # Heuristic cache expiration. Docs:
> # https://docs.trafficserver.apache.org/records.config#heuristic-expiration
> ##############################################################################
> # Setting all of these to zero in order for everything to be considered stale
> # by ATS. If we want to change this in the future, this is where to do it.
> CONFIG proxy.config.http.cache.heuristic_min_lifetime INT 0
> CONFIG proxy.config.http.cache.heuristic_max_lifetime INT 0
> CONFIG proxy.config.http.cache.heuristic_lm_factor FLOAT 0.000000
> ##############################################################################
>
> On Mon, Jan 8, 2018 at 10:24 AM, Patrick O'Brien
> <[hidden email]> wrote:
>> Hey Jeremy,
>>
>> Here's the full config from the child node:
>> https://gist.github.com/poblahblahblah/a1abdad3d64ebdfba003c75e50c9fe00
>>
>> Thanks for looking!
>>
>> On Sun, Jan 7, 2018 at 8:13 PM, Jeremy Payne <[hidden email]> wrote:
>>> Can you post the complete records.config from your child cache ?
>>> If I had to guess, it looks like heurstics or fuzzytimer may be set too low ???
>>> But if you provide  a company sanitized version of your records.config
>>> that should give us a better understanding of your setup.
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Wed, Jan 3, 2018 at 6:10 PM, Patrick O'Brien
>>> <[hidden email]> wrote:
>>>> Hello,
>>>>
>>>> I am testing out hierarchical caching in ATS 7.1.1. Everything seems
>>>> to work, although it looks like the child node is always needing to
>>>> revalidate objects fetched from the parent node. Here are the headers
>>>> on an example GET:
>>>> https://gist.github.com/poblahblahblah/928aaa88c24ef3c05366433a514f1a28
>>>>
>>>> The parent has the object stored in RAM and doesn't require
>>>> validation, but the child thinks the response is stale. Here is the
>>>> cache.config file on both the parent and the child:
>>>> https://gist.github.com/poblahblahblah/36b54ac8b7d75071c1342da6694d3c51
>>>>
>>>> The problem with this is that the connection between the child and
>>>> parent is pretty flaky and when the parent is down we are met with a
>>>> 502: https://gist.github.com/poblahblahblah/d50657354c815040faae50539fa45836
>>>>
>>>> Here is the parent.config on the child:
>>>> https://gist.github.com/poblahblahblah/9f1dcb4cf674c21c80a27173bd0ef81b
>>>>
>>>> Relevant configs from records.config for the parent:
>>>> https://gist.github.com/poblahblahblah/e6ac573e9f9cd63088c2e06e6544d801
>>>> Relevant configs from records.config for the child:
>>>> https://gist.github.com/poblahblahblah/5a6dc02a88b24cafca0e8c70d353b92e
>>>>
>>>> Any guidance on this would be much appreciated.
>>>>
>>>> -patrick