How to purge all cached negative responses

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

How to purge all cached negative responses

Veiko Kukk
Hi,

We had configuration mistake that enforced Cache-Contro: max-age=157784630 also to negative responses that then got cached.

Now, after fixing config, we need to purge all those objects from cache. It's even good enough if I could get list of objects with certain HTTP status code, then I could write script that purges those objects one by one.

How to do mass purge based on HTTP status code or just get list of objects based on HTTP status code?


Veiko

Reply | Threaded
Open this post in threaded view
|

Re: How to purge all cached negative responses

David Carlin-2
Can you grab the list of objects from the log files?   Thats only thing I can think of.

The cache inspector exists, but I've never had any luck with it.  I think our cache is too large for it:

https://docs.trafficserver.apache.org/en/latest/admin-guide/storage/index.en.html#inspecting-the-cache

David

On Thu, Dec 21, 2017 at 5:40 AM, Veiko Kukk <[hidden email]> wrote:
Hi,

We had configuration mistake that enforced Cache-Contro: max-age=157784630 also to negative responses that then got cached.

Now, after fixing config, we need to purge all those objects from cache. It's even good enough if I could get list of objects with certain HTTP status code, then I could write script that purges those objects one by one.

How to do mass purge based on HTTP status code or just get list of objects based on HTTP status code?


Veiko


Reply | Threaded
Open this post in threaded view
|

Re: How to purge all cached negative responses

Veiko Kukk
Thank you for your answer.

Sure I can find those failures from logs, but it's no good because, well, they have then already failed for client.
I've read about cache inspector, but it does not seem to be able to filter based on HTTP status code.

Veiko


2017-12-21 14:21 GMT+02:00 David Carlin <[hidden email]>:
Can you grab the list of objects from the log files?   Thats only thing I can think of.

The cache inspector exists, but I've never had any luck with it.  I think our cache is too large for it:

https://docs.trafficserver.apache.org/en/latest/admin-guide/storage/index.en.html#inspecting-the-cache

David

On Thu, Dec 21, 2017 at 5:40 AM, Veiko Kukk <[hidden email]> wrote:
Hi,

We had configuration mistake that enforced Cache-Contro: max-age=157784630 also to negative responses that then got cached.

Now, after fixing config, we need to purge all those objects from cache. It's even good enough if I could get list of objects with certain HTTP status code, then I could write script that purges those objects one by one.

How to do mass purge based on HTTP status code or just get list of objects based on HTTP status code?


Veiko



Reply | Threaded
Open this post in threaded view
|

Re: How to purge all cached negative responses

David Carlin-2
I'm suggesting finding the failed requests from the logs, and purging those URLs so future requests work.

David

On Thu, Dec 21, 2017 at 9:20 AM, Veiko Kukk <[hidden email]> wrote:
Thank you for your answer.

Sure I can find those failures from logs, but it's no good because, well, they have then already failed for client.
I've read about cache inspector, but it does not seem to be able to filter based on HTTP status code.

Veiko


2017-12-21 14:21 GMT+02:00 David Carlin <[hidden email]>:
Can you grab the list of objects from the log files?   Thats only thing I can think of.

The cache inspector exists, but I've never had any luck with it.  I think our cache is too large for it:

https://docs.trafficserver.apache.org/en/latest/admin-guide/storage/index.en.html#inspecting-the-cache

David

On Thu, Dec 21, 2017 at 5:40 AM, Veiko Kukk <[hidden email]> wrote:
Hi,

We had configuration mistake that enforced Cache-Contro: max-age=157784630 also to negative responses that then got cached.

Now, after fixing config, we need to purge all those objects from cache. It's even good enough if I could get list of objects with certain HTTP status code, then I could write script that purges those objects one by one.

How to do mass purge based on HTTP status code or just get list of objects based on HTTP status code?


Veiko




Reply | Threaded
Open this post in threaded view
|

Re: How to purge all cached negative responses

Sudheer Vinukonda-2
Another option is to write a simple plugin that hooks at Cache Lookup Complete state to overwrite the cache status from hit to stale for negative responses (status > 399). Unfortunately, I think header rewrite doesn’t expose the cache lookup Complete hooks, so this has to be written as a plugin (perhaps, might be simple with Lua?).

This approach won’t purge these objects instantly, but prevents them from returning to clients from the cache and they eventually get removed automatically due to cyclic cache.

Thanks,

Sudheer

On Dec 21, 2017, at 6:39 AM, David Carlin <[hidden email]> wrote:

I'm suggesting finding the failed requests from the logs, and purging those URLs so future requests work.

David

On Thu, Dec 21, 2017 at 9:20 AM, Veiko Kukk <[hidden email]> wrote:
Thank you for your answer.

Sure I can find those failures from logs, but it's no good because, well, they have then already failed for client.
I've read about cache inspector, but it does not seem to be able to filter based on HTTP status code.

Veiko


2017-12-21 14:21 GMT+02:00 David Carlin <[hidden email]>:
Can you grab the list of objects from the log files?   Thats only thing I can think of.

The cache inspector exists, but I've never had any luck with it.  I think our cache is too large for it:

https://docs.trafficserver.apache.org/en/latest/admin-guide/storage/index.en.html#inspecting-the-cache

David

On Thu, Dec 21, 2017 at 5:40 AM, Veiko Kukk <[hidden email]> wrote:
Hi,

We had configuration mistake that enforced Cache-Contro: max-age=157784630 also to negative responses that then got cached.

Now, after fixing config, we need to purge all those objects from cache. It's even good enough if I could get list of objects with certain HTTP status code, then I could write script that purges those objects one by one.

How to do mass purge based on HTTP status code or just get list of objects based on HTTP status code?


Veiko




Reply | Threaded
Open this post in threaded view
|

Re: How to purge all cached negative responses

Miles Libbey
How about a header_rewrite like
If status>399
Turn off cache. 

You might just be better off dropping the entire cache or changing the cache generation number for that remap ruleset. 
miles

On Thursday, December 21, 2017, Sudheer Vinukonda <[hidden email]> wrote:
Another option is to write a simple plugin that hooks at Cache Lookup Complete state to overwrite the cache status from hit to stale for negative responses (status > 399). Unfortunately, I think header rewrite doesn’t expose the cache lookup Complete hooks, so this has to be written as a plugin (perhaps, might be simple with Lua?).

This approach won’t purge these objects instantly, but prevents them from returning to clients from the cache and they eventually get removed automatically due to cyclic cache.

Thanks,

Sudheer

On Dec 21, 2017, at 6:39 AM, David Carlin <[hidden email]> wrote:

I'm suggesting finding the failed requests from the logs, and purging those URLs so future requests work.

David

On Thu, Dec 21, 2017 at 9:20 AM, Veiko Kukk <[hidden email]> wrote:
Thank you for your answer.

Sure I can find those failures from logs, but it's no good because, well, they have then already failed for client.
I've read about cache inspector, but it does not seem to be able to filter based on HTTP status code.

Veiko


2017-12-21 14:21 GMT+02:00 David Carlin <[hidden email]>:
Can you grab the list of objects from the log files?   Thats only thing I can think of.

The cache inspector exists, but I've never had any luck with it.  I think our cache is too large for it:

https://docs.trafficserver.apache.org/en/latest/admin-guide/storage/index.en.html#inspecting-the-cache

David

On Thu, Dec 21, 2017 at 5:40 AM, Veiko Kukk <[hidden email]> wrote:
Hi,

We had configuration mistake that enforced Cache-Contro: max-age=157784630 also to negative responses that then got cached.

Now, after fixing config, we need to purge all those objects from cache. It's even good enough if I could get list of objects with certain HTTP status code, then I could write script that purges those objects one by one.

How to do mass purge based on HTTP status code or just get list of objects based on HTTP status code?


Veiko




Reply | Threaded
Open this post in threaded view
|

Re: How to purge all cached negative responses

Sudheer Vinukonda-2


On Dec 21, 2017, at 7:43 AM, Miles Libbey <[hidden email]> wrote:

How about a header_rewrite like
If status>399
Turn off cache. 

It’s too “late” to turn off cache at this point.


You might just be better off dropping the entire cache or changing the cache generation number for that remap ruleset. 
miles

On Thursday, December 21, 2017, Sudheer Vinukonda <[hidden email]> wrote:
Another option is to write a simple plugin that hooks at Cache Lookup Complete state to overwrite the cache status from hit to stale for negative responses (status > 399). Unfortunately, I think header rewrite doesn’t expose the cache lookup Complete hooks, so this has to be written as a plugin (perhaps, might be simple with Lua?).

This approach won’t purge these objects instantly, but prevents them from returning to clients from the cache and they eventually get removed automatically due to cyclic cache.

Thanks,

Sudheer

On Dec 21, 2017, at 6:39 AM, David Carlin <[hidden email]> wrote:

I'm suggesting finding the failed requests from the logs, and purging those URLs so future requests work.

David

On Thu, Dec 21, 2017 at 9:20 AM, Veiko Kukk <[hidden email]> wrote:
Thank you for your answer.

Sure I can find those failures from logs, but it's no good because, well, they have then already failed for client.
I've read about cache inspector, but it does not seem to be able to filter based on HTTP status code.

Veiko


2017-12-21 14:21 GMT+02:00 David Carlin <[hidden email]>:
Can you grab the list of objects from the log files?   Thats only thing I can think of.

The cache inspector exists, but I've never had any luck with it.  I think our cache is too large for it:

https://docs.trafficserver.apache.org/en/latest/admin-guide/storage/index.en.html#inspecting-the-cache

David

On Thu, Dec 21, 2017 at 5:40 AM, Veiko Kukk <[hidden email]> wrote:
Hi,

We had configuration mistake that enforced Cache-Contro: max-age=157784630 also to negative responses that then got cached.

Now, after fixing config, we need to purge all those objects from cache. It's even good enough if I could get list of objects with certain HTTP status code, then I could write script that purges those objects one by one.

How to do mass purge based on HTTP status code or just get list of objects based on HTTP status code?


Veiko




Reply | Threaded
Open this post in threaded view
|

Re: How to purge all cached negative responses

Sudheer Vinukonda-2


On Dec 21, 2017, at 7:54 AM, Sudheer Vinukonda <[hidden email]> wrote:



On Dec 21, 2017, at 7:43 AM, Miles Libbey <[hidden email]> wrote:

How about a header_rewrite like
If status>399
Turn off cache. 

It’s too “late” to turn off cache at this point (using header_rewrite with the hooks it currently allows)


You might just be better off dropping the entire cache or changing the cache generation number for that remap ruleset. 
miles

On Thursday, December 21, 2017, Sudheer Vinukonda <[hidden email]> wrote:
Another option is to write a simple plugin that hooks at Cache Lookup Complete state to overwrite the cache status from hit to stale for negative responses (status > 399). Unfortunately, I think header rewrite doesn’t expose the cache lookup Complete hooks, so this has to be written as a plugin (perhaps, might be simple with Lua?).

This approach won’t purge these objects instantly, but prevents them from returning to clients from the cache and they eventually get removed automatically due to cyclic cache.

Thanks,

Sudheer

On Dec 21, 2017, at 6:39 AM, David Carlin <[hidden email]> wrote:

I'm suggesting finding the failed requests from the logs, and purging those URLs so future requests work.

David

On Thu, Dec 21, 2017 at 9:20 AM, Veiko Kukk <[hidden email]> wrote:
Thank you for your answer.

Sure I can find those failures from logs, but it's no good because, well, they have then already failed for client.
I've read about cache inspector, but it does not seem to be able to filter based on HTTP status code.

Veiko


2017-12-21 14:21 GMT+02:00 David Carlin <[hidden email]>:
Can you grab the list of objects from the log files?   Thats only thing I can think of.

The cache inspector exists, but I've never had any luck with it.  I think our cache is too large for it:

https://docs.trafficserver.apache.org/en/latest/admin-guide/storage/index.en.html#inspecting-the-cache

David

On Thu, Dec 21, 2017 at 5:40 AM, Veiko Kukk <[hidden email]> wrote:
Hi,

We had configuration mistake that enforced Cache-Contro: max-age=157784630 also to negative responses that then got cached.

Now, after fixing config, we need to purge all those objects from cache. It's even good enough if I could get list of objects with certain HTTP status code, then I could write script that purges those objects one by one.

How to do mass purge based on HTTP status code or just get list of objects based on HTTP status code?


Veiko