|
bool | cache_enabled () const |
|
virtual bool | cache_too_big (unsigned long long current_size) const |
| look at the cache size; is it too large? Look at the cache size and see if it is too big. More...
|
|
virtual bool | can_be_cached (libdap::DDS *dds, const std::string &constraint) |
|
virtual bool | create_and_lock (const std::string &target, int &fd) |
| Create a file in the cache and lock it for write access. More...
|
|
void | disable () |
| Disable the cache. More...
|
|
virtual void | dump (std::ostream &strm) const |
| dumps information about this object More...
|
|
void | enable () |
| Enable the cache. More...
|
|
virtual void | exclusive_to_shared_lock (int fd) |
| Transfer from an exclusive lock to a shared lock. More...
|
|
const std::string | get_cache_directory () |
|
virtual std::string | get_cache_file_name (const std::string &src, bool mangle=true) |
|
const std::string | get_cache_file_prefix () |
|
virtual unsigned long long | get_cache_size () |
| Get the cache size. More...
|
|
virtual bool | get_exclusive_lock (const std::string &target, int &fd) |
|
virtual bool | get_exclusive_lock_nb (const std::string &target, int &fd) |
|
virtual libdap::DDS * | get_or_cache_dataset (libdap::DDS *dds, const std::string &constraint) |
| Return a DDS loaded with data that can be serialized back to a client. More...
|
|
virtual bool | get_read_lock (const std::string &target, int &fd) |
| Get a read-only lock on the file if it exists. More...
|
|
void | initialize (const std::string &cache_dir, const std::string &prefix, unsigned long long size) |
| Initialize an instance of FileLockingCache. More...
|
|
bool | is_unlimited () const |
| Is this cache allowed to store as much as it wants? More...
|
|
virtual void | lock_cache_read () |
|
virtual void | lock_cache_write () |
|
virtual void | purge_file (const std::string &file) |
| Purge a single file from the cache. More...
|
|
virtual void | unlock_and_close (const std::string &target) |
|
virtual void | unlock_cache () |
|
virtual void | update_and_purge (const std::string &new_file) |
| Purge files from the cache. More...
|
|
virtual unsigned long long | update_cache_info (const std::string &target) |
| Update the cache info file to include 'target'. More...
|
|
|
static bool | dir_exists (const std::string &dir) |
|
static string | get_cache_dir_from_config () |
|
static string | get_cache_prefix_from_config () |
|
static unsigned long | get_cache_size_from_config () |
|
|
Get an instance of the BESDapFunctionResponseCache object. This class is a singleton, so the first call to any of three 'get_instance()' methods makes an instance and subsequent calls return a pointer to that instance.
- Note
- If the cache_dir parameter is null, this will return null for the pointer to the singleton and caching is disabled.
- Parameters
-
cache_dir_key | Key to use to get the value of the cache directory. If this is the empty stirng, return null right away. |
prefix_key | Key for the item/file prefix. Each file added to the cache uses this as a prefix so cached items can be easily identified when /tmp is used for the cache. |
size_key | How big should the cache be, in megabytes |
- Returns
- A pointer to a BESDapFunctionResponseCache object
|
static BESDapFunctionResponseCache * | get_instance (const string &cache_dir, const string &prefix, unsigned long long size) |
|
static BESDapFunctionResponseCache * | get_instance () |
|
Cache the results from server functions.
Serve-side functions build new datasets and can be quite large. This code caches those results so that when clients ask for a suite of responses from the function calls, the computations are run only once (in the best case) and subsequent requests for data or metadata are satisfied using information in this case.
- Note
- Cache entry collisions: This cache must hold objects that are identified by the combination of a dataset and a constraint expression. The CE can be quite large and contain a number of 'special' characters like '()' and so on. Instead of building cache IDs using a simple concatenation of the dataset and CE, we use the C++ std::hash class to generate a hash code. However, it's possible that two different dataset/CE combinations will have the same hash values. We use a simple collision resolution system where a suffix is appended to the hash value. After a number of collisions, we give up and simply do not cache the response (providing no worse performance than if the cache did not exist - but currently we throw an exception - see load_from_cache and the constant 'max_collisions').
-
Cache entry format: The cache uses a specially formated 'response object' that is more efficient to read and write than a typical DAP2 or DAP4 response object. DAP2 serializes data using network byte order while the cache uses native machine order. DAP4 computes checksums; the cache does not. In addition, each cache entry contains the resource id as its first line so that the correct entry can be identified.
- Author
- ndp, jhrg
Definition at line 72 of file BESDapFunctionResponseCache.h.
BESDapFunctionResponseCache::BESDapFunctionResponseCache |
( |
const string & |
cache_dir, |
|
|
const string & |
prefix, |
|
|
unsigned long long |
size |
|
) |
| |
|
inlineprotected |
Protected constructor that takes as arguments keys to the cache directory, file prefix, and size of the cache to be looked up a configuration file.
The keys specified are looked up in the specified keys object. If not found or not set correctly then an exception is thrown. I.E., if the cache directory is empty, the size is zero, or the prefix is empty.
- Parameters
-
cache_dir_key | key to look up in the keys file to find cache dir |
prefix_key | key to look up in the keys file to find the cache prefix |
size_key | key to look up in the keys file to find the cache size (in MBytes) |
- Exceptions
-
BESSyntaxUserError | if keys not set, cache dir or prefix empty, size is 0, or if cache dir does not exist. |
Definition at line 123 of file BESDapFunctionResponseCache.h.
bool BESFileLockingCache::create_and_lock |
( |
const std::string & |
target, |
|
|
int & |
fd |
|
) |
| |
|
virtualinherited |
Create a file in the cache and lock it for write access.
If the file does not exist, make it, open it for read-write access and get an exclusive lock on it. The locking operation blocks, although that should never happen.
- Parameters
-
target | The name of the file to make/open/lock |
fd | Value-result param that holds the file descriptor of the opened file |
- Returns
- True if the operation was successful, false otherwise. This method will return false if the file already existed (the file won't be locked and the descriptor reference is undefined - but likely -1).
- Exceptions
-
BESBESInternalErroror | if any error except EEXIST is returned by open(2) or if fcntl(2) returns an error. |
Definition at line 599 of file BESFileLockingCache.cc.
void BESFileLockingCache::exclusive_to_shared_lock |
( |
int |
fd | ) |
|
|
virtualinherited |
Transfer from an exclusive lock to a shared lock.
If the file has an exclusive write lock on it, change that to a shared read lock. This is an atomic operation. If the call to fcntl(2) is protected by locking the cache, a dead lock will result given typical use of this class. This method exists to help with the situation where one process has the cache locked and is blocking on a shared read lock for a file that a second process has locked exclusively (for writing). By changing the exclusive lock to a shared lock, the first process can get its shared lock and then release the cache.
- Parameters
-
fd | The file descriptor that is exclusively locked and which, on exit, will have a shared lock. |
Definition at line 630 of file BESFileLockingCache.cc.
DDS * BESDapFunctionResponseCache::get_or_cache_dataset |
( |
libdap::DDS * |
dds, |
|
|
const std::string & |
constraint |
|
) |
| |
|
virtual |
Return a DDS loaded with data that can be serialized back to a client.
Given a DDS and a DAP2 constraint expression that contains only projection function calls, either pull a cached DDS* that is the result of evaluating those functions, or evaluate, cache and return the result. This is the main API cacll for this class.
- Note
- This method controls the cache lock, ensuring that the cache is unlocked when it returns.
-
The code that evaluates the function expression (when needed) could be sped up by using a thread to handle the process of writing the DDS to the cache, but this will be complicated until we have shared pointers (because the DDS* could be deleted while the cache code is still writing it).
- Parameters
-
- Returns
Definition at line 320 of file BESDapFunctionResponseCache.cc.
void BESFileLockingCache::initialize |
( |
const std::string & |
cache_dir, |
|
|
const std::string & |
prefix, |
|
|
unsigned long long |
size |
|
) |
| |
|
inherited |
Initialize an instance of FileLockingCache.
Initialize and instance of FileLockingCache using the passed values for the cache directory, item prefix and max cache size. This will ignore the value of enable_cache() (but will correctly (re)set it based on the directory, ..., values). This provides a way for clients to re-initialize caches on the fly.
- Parameters
-
cache_dir | The directory into which the cache files will be written. |
prefix | The prefix that will be added to each cache file. |
size | The size of the cache in MBytes |
- Exceptions
-
BESInternalError | If the cache_dir does not exist or is not writable. size is 0, or if cache dir does not exist. |
BESError | If the parameters (directory, ...) are invalid. |
Definition at line 116 of file BESFileLockingCache.cc.
void BESFileLockingCache::purge_file |
( |
const std::string & |
file | ) |
|
|
virtualinherited |
Purge a single file from the cache.
Purge a single file from the cache. The file might be old, etc., and need to be removed. Don't use this to shrink the cache when it gets too big, use update_and_purge() instead since that file optimizes accesses to the cache control file for several changes in a row.
- Parameters
-
file | The name of the file to purge. |
Definition at line 1085 of file BESFileLockingCache.cc.