|
void | add_tracking_query_param () |
| Modify this chunk's data URL so that it includes tracking info.
|
|
| Chunk ()=default |
| Get an empty chunk.
|
|
| Chunk (const Chunk &h4bs) |
|
| Chunk (std::shared_ptr< http::url > data_url, std::string order, unsigned long long size, unsigned long long offset, const std::string &pia_str="") |
| Get a chunk initialized with values.
|
|
| Chunk (std::shared_ptr< http::url > data_url, std::string order, unsigned long long size, unsigned long long offset, const std::vector< unsigned long long > &pia_vec) |
| Get a chunk initialized with values.
|
|
| Chunk (std::string order, std::string fill_value, libdap::Type fv_type, unsigned long long chunk_size, std::vector< unsigned long long > pia) |
|
| Chunk (std::string order, unsigned long long size, unsigned long long offset, const std::string &pia_str="") |
| Get a chunk initialized with values, the data URL will not be set.
|
|
| Chunk (std::string order, unsigned long long size, unsigned long long offset, const std::vector< unsigned long long > &pia_vec) |
| Get a chunk initialized with values, the data URl will not be set.
|
|
virtual void | dump (std::ostream &strm) const |
|
virtual void | filter_chunk (const std::string &filters, unsigned long long chunk_size, unsigned long long elem_width) |
| filter data in the chunk
|
|
virtual std::string | get_byte_order () |
|
virtual unsigned long long | get_bytes_read () const |
|
virtual std::string | get_curl_range_arg_string () |
| Returns a curl range argument. The libcurl requires a string argument for range-ge activitys, this method constructs one in the required syntax from the offset and size information for this byteStream.
|
|
virtual std::shared_ptr< http::url > | get_data_url () const |
|
virtual std::string | get_fill_value () const |
|
virtual bool | get_is_read () |
|
virtual unsigned long long | get_offset () const |
|
virtual const std::vector< unsigned long long > & | get_position_in_array () const |
|
virtual char * | get_rbuf () |
|
virtual unsigned long long | get_rbuf_size () const |
|
virtual std::string | get_response_content_type () |
| Get the response type of the last response.
|
|
virtual unsigned long long | get_size () const |
|
virtual bool | get_uses_fill_value () const |
|
virtual void | load_fill_values () |
| Load the chunk with fill values - temporary implementation.
|
|
Chunk & | operator= (const Chunk &rhs) |
|
virtual void | read_chunk () |
|
virtual void | set_bytes_read (unsigned long long bytes_read) |
| Set the size of this Chunk's data block.
|
|
virtual void | set_data_url (std::shared_ptr< http::url > data_url) |
| Set the data url for this Chunk's data block.
|
|
virtual void | set_is_read (bool state) |
|
void | set_position_in_array (const std::string &pia) |
| parse the chunk position string
|
|
void | set_position_in_array (const std::vector< unsigned long long > &pia) |
| Set the chunk's position in the Array.
|
|
virtual void | set_rbuf_to_size () |
| Allocates the internal read buffer to be d_size bytes.
|
|
void | set_read_buffer (char *buf, unsigned long long buf_size, unsigned long long bytes_read=0, bool assume_ownership=true) |
| Set the target read buffer for this chunk.
|
|
void | set_response_content_type (const std::string &ct) |
| Set the response type of the last response.
|
|
virtual std::string | to_string () const |
|
This class is used to encapsulate the state and behavior needed for reading chunked data associated with a DAP variable. In particular it is based on the semantics of an hdf4:Chunk object, which is used to represent a chunk of data in a (potentially complex) HDF4/HDF5 file.
Definition at line 77 of file Chunk.h.
void dmrpp::Chunk::add_tracking_query_param |
( |
| ) |
|
Modify this chunk's data URL so that it includes tracking info.
Add information to the Query string of a URL, intended primarily to aid in tracking the origin of requests when reading data from S3. The information added to the query string comes from a BES Context command sent to the BES by a client (e.g., the OLFS). The addition takes the form "tracking_context=<context value>". The method checks to see if the URL already has a query string, if not it adds one: "?tracking_context=<context value>" And if so it appends an additional parameter: "&tracking_context=<context value>"
- Note
- This is only added to data URLs that reference an S3 bucket.
-
This adds a significant cost to the run-time behavior of the DMR++ code and should only be used for testing and, even then, probably rarely since the performance of the server will be reduced significantly.
Cloudydap test hack where we tag the S3 URLs with a query string for the S3 log in order to track S3 requests. The tag is submitted as a BESContext with the request. Here we check to see if the request is for an AWS S3 object, if it is AND we have the magic BESContext "cloudydap" then we add a query parameter to the S3 URL for tracking purposes.
Should this be a function? FFS why? This is the ONLY place where this needs happen, as close to the curl call as possible and we can just turn it off down the road. - ndp 1/20/17 (EOD)
Well, it's a function now... ;-) jhrg 8/6/18
Definition at line 486 of file Chunk.cc.
virtual void dmrpp::Chunk::set_rbuf_to_size |
( |
| ) |
|
|
inlinevirtual |
Allocates the internal read buffer to be d_size bytes.
The memory of the read buffer is managed internally by this method.
The class maintains an internal flag, d_read_buffer_is_mine, which controls if the currently held read buffer memory is released (via a call to 'delete[]') when an this method is invoked.
If the CHunk owns the read buffer, then calling this method will release any previously allocated read buffer memory and then allocate a new memory block. The bytes_read counter is reset to zero.
Definition at line 322 of file Chunk.h.