Skip to content

Commit

Permalink
Add more examples
Browse files Browse the repository at this point in the history
  • Loading branch information
Tom Scavo authored Oct 29, 2016
1 parent 63bf820 commit 8a147b4
Showing 1 changed file with 49 additions and 10 deletions.
59 changes: 49 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,27 +39,66 @@ md_tools.sh

Bash script ``cget.sh`` retrieves and caches HTTP resources on disk. A previously cached resource is retrieved via HTTP Conditional GET [RFC 7232]. If the web server responds with HTTP 200 OK, the resource is cached and written to stdout. If the web server responds with 304 Not Modified, the cached resource is output instead.

To illustrate, define a couple of HTTP resources and a cache:

```Shell
$ MD_LOCATION=http://md.incommon.org/InCommon/InCommon-metadata.xml
$ url1=http://md.incommon.org/InCommon/InCommon-metadata-preview.xml
$ url2=http://md.incommon.org/InCommon/InCommon-metadata-fallback.xml
$ export CACHE_DIR=/tmp/http_cache
$ $BIN_DIR/cget.sh $MD_LOCATION > /dev/null
```

GET the first resource:

```Shell
$ $BIN_DIR/cget.sh $url1 > /dev/null
$ echo $?
0
$ ls -1 $CACHE_DIR
885d2eedcad7d0355da5bb9e648833ec_content
885d2eedcad7d0355da5bb9e648833ec_headers
$ cat $CACHE_DIR/885d2eedcad7d0355da5bb9e648833ec_headers
$ ls -1 $CACHE_DIR
1e6b844a49d1850b82feded72cf83ed7_content
1e6b844a49d1850b82feded72cf83ed7_headers
$ cat $CACHE_DIR/1e6b844a49d1850b82feded72cf83ed7_headers
HTTP/1.1 200 OK
Date: Mon, 17 Oct 2016 17:26:58 GMT
Date: Sat, 29 Oct 2016 18:48:29 GMT
Server: Apache
Last-Modified: Fri, 14 Oct 2016 19:07:55 GMT
ETag: "190004-24ddf06-53ed7f18d18c0"
Last-Modified: Fri, 28 Oct 2016 19:11:49 GMT
ETag: "1f802b-252f7f2-53ff1a146e740"
Accept-Ranges: bytes
Content-Length: 38657798
Content-Length: 38991858
Connection: close
Content-Type: application/samlmetadata+xml
```

Assuming the resource doesn't change on the server, subsequent requests will return the cached resource. To bypass the network altogether, use the ``-C`` option:

```Shell
$ $BIN_DIR/cget.sh -C $url1 | wc -c
38991858
```

Of course the ``-C`` option will fail if the resource is not cached:

```Shell
$ $BIN_DIR/cget.sh -C $url2
$ echo $?
1
```

OTOH, the ``-F`` option forces the return of a fresh resource from the server. If the resource is cached and unchanged on the server (304), such a request will fail, however:

```Shell
$ $BIN_DIR/cget.sh -F $url1
$ echo $?
1
$ $BIN_DIR/cget.sh -F $url2 > /dev/null
$ echo $?
0
$ ls -1 $CACHE_DIR
1727196e5b7593f3b7528c539e7169d2_content
1727196e5b7593f3b7528c539e7169d2_headers
1e6b844a49d1850b82feded72cf83ed7_content
1e6b844a49d1850b82feded72cf83ed7_headers
```

See the inline help file for details:

```Shell
Expand Down

0 comments on commit 8a147b4

Please sign in to comment.