diff --git a/README.md b/README.md index e10ffc9..d727cb6 100644 --- a/README.md +++ b/README.md @@ -39,27 +39,66 @@ md_tools.sh Bash script ``cget.sh`` retrieves and caches HTTP resources on disk. A previously cached resource is retrieved via HTTP Conditional GET [RFC 7232]. If the web server responds with HTTP 200 OK, the resource is cached and written to stdout. If the web server responds with 304 Not Modified, the cached resource is output instead. +To illustrate, define a couple of HTTP resources and a cache: + ```Shell -$ MD_LOCATION=http://md.incommon.org/InCommon/InCommon-metadata.xml +$ url1=http://md.incommon.org/InCommon/InCommon-metadata-preview.xml +$ url2=http://md.incommon.org/InCommon/InCommon-metadata-fallback.xml $ export CACHE_DIR=/tmp/http_cache -$ $BIN_DIR/cget.sh $MD_LOCATION > /dev/null +``` + +GET the first resource: + +```Shell +$ $BIN_DIR/cget.sh $url1 > /dev/null $ echo $? 0 -$ ls -1 $CACHE_DIR -885d2eedcad7d0355da5bb9e648833ec_content -885d2eedcad7d0355da5bb9e648833ec_headers -$ cat $CACHE_DIR/885d2eedcad7d0355da5bb9e648833ec_headers +$ ls -1 $CACHE_DIR +1e6b844a49d1850b82feded72cf83ed7_content +1e6b844a49d1850b82feded72cf83ed7_headers +$ cat $CACHE_DIR/1e6b844a49d1850b82feded72cf83ed7_headers HTTP/1.1 200 OK -Date: Mon, 17 Oct 2016 17:26:58 GMT +Date: Sat, 29 Oct 2016 18:48:29 GMT Server: Apache -Last-Modified: Fri, 14 Oct 2016 19:07:55 GMT -ETag: "190004-24ddf06-53ed7f18d18c0" +Last-Modified: Fri, 28 Oct 2016 19:11:49 GMT +ETag: "1f802b-252f7f2-53ff1a146e740" Accept-Ranges: bytes -Content-Length: 38657798 +Content-Length: 38991858 Connection: close Content-Type: application/samlmetadata+xml ``` +Assuming the resource doesn't change on the server, subsequent requests will return the cached resource. To bypass the network altogether, use the ``-C`` option: + +```Shell +$ $BIN_DIR/cget.sh -C $url1 | wc -c + 38991858 +``` + +Of course the ``-C`` option will fail if the resource is not cached: + +```Shell +$ $BIN_DIR/cget.sh -C $url2 +$ echo $? +1 +``` + +OTOH, the ``-F`` option forces the return of a fresh resource from the server. If the resource is cached and unchanged on the server (304), such a request will fail, however: + +```Shell +$ $BIN_DIR/cget.sh -F $url1 +$ echo $? +1 +$ $BIN_DIR/cget.sh -F $url2 > /dev/null +$ echo $? +0 +$ ls -1 $CACHE_DIR +1727196e5b7593f3b7528c539e7169d2_content +1727196e5b7593f3b7528c539e7169d2_headers +1e6b844a49d1850b82feded72cf83ed7_content +1e6b844a49d1850b82feded72cf83ed7_headers +``` + See the inline help file for details: ```Shell