|
virtual void | add_tracking_query_param () |
| Modify this chunk's data URL so that it includes tracking info. More...
|
|
| Chunk () |
| Get an empty chunk. More...
|
|
| Chunk (const Chunk &h4bs) |
|
| Chunk (const std::string &data_url, unsigned long long size, unsigned long long offset, const std::vector< unsigned int > &pia_vec) |
| Get a chunk initialized with values. More...
|
|
| Chunk (const std::string &data_url, unsigned long long size, unsigned long long offset, std::string pia_str="") |
| Get a chunk initialized with values. More...
|
|
virtual void | dump (std::ostream &strm) const |
|
virtual unsigned long long | get_bytes_read () const |
| Get the number of bytes read so far for this Chunk. More...
|
|
virtual std::string | get_curl_range_arg_string () |
| Returns a curl range argument. The libcurl requires a string argument for range-ge activitys, this method constructs one in the required syntax from the offset and size information for this byteStream. More...
|
|
virtual std::string | get_data_url () const |
| Get the data url string for this Chunk's data block. More...
|
|
virtual bool | get_is_inflated () const |
|
virtual unsigned long long | get_offset () const |
| Get the offset to this Chunk's data block. More...
|
|
virtual const std::vector< unsigned int > & | get_position_in_array () const |
|
virtual char * | get_rbuf () |
|
virtual unsigned long long | get_rbuf_size () const |
|
virtual unsigned long long | get_size () const |
| Get the size of this Chunk's data block on disk. More...
|
|
virtual void | inflate_chunk (bool deflate, bool shuffle, unsigned int chunk_size, unsigned int elem_width) |
| Decompress data in the chunk, managing the Chunk's data buffers. More...
|
|
Chunk & | operator= (const Chunk &rhs) |
|
virtual void | read_chunk () |
|
virtual void | set_bytes_read (unsigned long long bytes_read) |
| Set the size of this Chunk's data block. More...
|
|
virtual void | set_data_url (const std::string &data_url) |
| Get the data url string for this Chunk's data block. More...
|
|
virtual void | set_is_inflated (bool state) |
|
virtual void | set_is_read (bool state) |
|
virtual void | set_position_in_array (const std::string &pia) |
| parse the chunk position string More...
|
|
virtual void | set_position_in_array (const std::vector< unsigned int > &pia) |
| Set the chunk's position in the Array. More...
|
|
virtual void | set_rbuf (char *buf, unsigned int size) |
| Set the read buffer. More...
|
|
virtual void | set_rbuf_to_size () |
| Allocates the internal read buffer to be d_size bytes. More...
|
|
virtual std::string | to_string () const |
|
This class is used to encapsulate the state and behavior needed for reading chunked data associated with a DAP variable. In particular it is based on the semantics of an hdf4:Chunk object, which is used to represent a chunk of data in a (potentially complex) HDF4/HDF5 file.
Definition at line 43 of file Chunk.h.
void dmrpp::Chunk::add_tracking_query_param |
( |
| ) |
|
|
virtual |
Modify this chunk's data URL so that it includes tracking info.
Add information to the Query string of a URL, intended primarily to aid in tracking the origin of requests when reading data from S3. The information added to the query string comes from a BES Context command sent to the BES by a client (e.g., the OLFS). The addition takes the form "?tracking_context=<context value>".
- Note
- This is only added to data URLs that reference S3.
Cloudydap test hack where we tag the S3 URLs with a query string for the S3 log in order to track S3 requests. The tag is submitted as a BESContext with the request. Here we check to see if the request is for an AWS S3 object, if it is AND we have the magic BESContext "cloudydap" then we add a query parameter to the S3 URL for tracking purposes.
Should this be a function? FFS why? This is the ONLY place where this needs happen, as close to the curl call as possible and we can just turn it off down the road. - ndp 1/20/17 (EOD)
Well, it's a function now... ;-) jhrg 8/6/18
Definition at line 401 of file Chunk.cc.
void dmrpp::Chunk::inflate_chunk |
( |
bool |
deflate, |
|
|
bool |
shuffle, |
|
|
unsigned int |
chunk_size, |
|
|
unsigned int |
elem_width |
|
) |
| |
|
virtual |
Decompress data in the chunk, managing the Chunk's data buffers.
This method tracks if a chunk has already been decompressed, so, like read_chunk() it can be called for a chunk that has already been decompressed without error.
- Parameters
-
deflate | True if the chunk should be 'inflated' |
shuffle | True if the chunk should be 'unshuffled' |
chunk_size | The expected chunk size, in elements; used to allocate storage |
elem_width | The number of bytes per element |
Definition at line 472 of file Chunk.cc.