OPeNDAP Hyrax Back End Server (BES)
Updated for version 3.8.3
|
Implementation of a caching mechanism for compressed data. More...
#include <BESFileLockingCache.h>
Public Member Functions | |
virtual bool | cache_too_big (unsigned long long current_size) const |
look at the cache size; is it too large? Look at the cache size and see if it is too big. More... | |
virtual bool | create_and_lock (const string &target, int &fd) |
Create a file in the cache and lock it for write access. More... | |
virtual void | dump (ostream &strm) const |
dumps information about this object More... | |
virtual void | exclusive_to_shared_lock (int fd) |
Transfer from an exclusive lock to a shared lock. More... | |
virtual string | get_cache_file_name (const string &src, bool mangle=true) |
Build the name of file that will holds the uncompressed data from 'src' in the cache. More... | |
virtual unsigned long long | get_cache_size () |
Get the cache size. More... | |
virtual bool | get_read_lock (const string &target, int &fd) |
Get a read-only lock on the file if it exists. More... | |
const string | getCacheDirectory () |
const string | getCacheFilePrefix () |
virtual void | lock_cache_read () |
Get a shared lock on the 'cache info' file. More... | |
virtual void | lock_cache_write () |
Get an exclusive lock on the 'cache info' file. More... | |
virtual void | purge_file (const string &file) |
Purge a single file from the cache. More... | |
virtual void | unlock_and_close (const string &target) |
Unlock the named file. More... | |
virtual void | unlock_and_close (int fd) |
Unlock the file. More... | |
virtual void | unlock_cache () |
Unlock the cache info file. More... | |
virtual void | update_and_purge (const string &new_file) |
Purge files from the cache. More... | |
virtual unsigned long long | update_cache_info (const string &target) |
Update the cache info file to include 'target'. More... | |
Protected Member Functions | |
BESFileLockingCache () | |
BESFileLockingCache (const string &cache_dir, const string &prefix, unsigned long long size) | |
Protected constructor that takes as arguments keys to the cache directory, file prefix, and size of the cache to be looked up a configuration file. More... | |
void | initialize (const string &cache_dir, const string &prefix, unsigned long long size) |
virtual | ~BESFileLockingCache () |
Implementation of a caching mechanism for compressed data.
This cache uses simple advisory locking found on most modern unix file systems. Compressed files are uncompressed and stored in a cache where they can be used over and over until removed from the cache. Several processes can share the cache with each reading from files. At the same time, new files can be added and the cache can be purged, without disrupting the existing read operations.
How it works. When a file is added to the cache, the cache is locked - no other processes can add, read or remove files. Once a file has been added, the cache size is examined and, if needed, the cache is purged so that its size is 80% of the maximum size. Then the cache is unlocked. When a process looks to see if a file is already in the cache, the entire cache is locked. If the file is present, a shared read lock is obtained and the cache is unlocked.
Methods: create_and_lock() and get_read_lock() open and lock files; the former creates the file and locks it exclusively iff it does not exist, while the latter obtains a shared lock iff the file already exists. The unlock() methods unlock a file. The lock_cache_info() and unlock_cache_info() are used to control access to the whole cache - with the open + lock and close + unlock operations performed atomically. Other methods that operate on the cache info file must only be called when the lock has been obtained.
Definition at line 71 of file BESFileLockingCache.h.
|
inlineprotected |
Definition at line 109 of file BESFileLockingCache.h.
|
protected |
Protected constructor that takes as arguments keys to the cache directory, file prefix, and size of the cache to be looked up a configuration file.
The keys specified are looked up in the specified keys object. If not found or not set correctly then an exception is thrown. I.E., if the cache directory is empty, the size is zero, or the prefix is empty.
cache_dir | The directory into which the cache files will be written. |
prefix | The prefix that will be added to each cache file. |
size | The size of the cache in MBytes |
BESInternalError | If the cache_dir does not exist or is not writable. size is 0, or if cache dir does not exist. |
Definition at line 80 of file BESFileLockingCache.cc.
|
inlineprotectedvirtual |
Definition at line 112 of file BESFileLockingCache.h.
|
virtual |
look at the cache size; is it too large? Look at the cache size and see if it is too big.
Definition at line 660 of file BESFileLockingCache.cc.
Referenced by BESDapResponseCache::cache_dataset(), BESStoredDapResultCache::cache_dataset(), and update_and_purge().
|
virtual |
Create a file in the cache and lock it for write access.
If the file does not exist, make it, open it for read-write access and get an exclusive lock on it. The locking operation blocks, although that should never happen.
target | The name of the file to make/open/lock |
fd | Value-result param that holds the file descriptor of the opened file |
BESBESInternalErroror | if any error except EEXIST is returned by open(2) or if fcntl(2) returns an error. |
Definition at line 481 of file BESFileLockingCache.cc.
References BESDEBUG, lock_cache_write(), and unlock_cache().
Referenced by BESDapResponseCache::cache_dataset(), and BESStoredDapResultCache::cache_dataset().
|
virtual |
dumps information about this object
Displays the pointer value of this instance along with information about this cache.
strm | C++ i/o stream to dump the information to |
Implements BESObj.
Definition at line 898 of file BESFileLockingCache.cc.
References BESIndent::Indent(), BESIndent::LMarg(), and BESIndent::UnIndent().
|
virtual |
Transfer from an exclusive lock to a shared lock.
If the file has an exclusive write lock on it, change that to a shared read lock. This is an atomic operation. If the call to fcntl(2) is protected by locking the cache, a dead lock will result given typical use of this class. This method exists to help with the situation where one process has the cache locked and is blocking on a shared read lock for a file that a second process has locked exclusively (for writing). By changing the exclusive lock to a shared lock, the first process can get its shared lock and then release the cache.
fd | The file descriptor that is exclusively locked and which, on exit, will have a shared lock. |
Definition at line 511 of file BESFileLockingCache.cc.
Referenced by BESDapResponseCache::cache_dataset(), and BESStoredDapResultCache::cache_dataset().
|
virtual |
Build the name of file that will holds the uncompressed data from 'src' in the cache.
src | The source name to cache |
mangle | if True, assume the name is a file pathname and mangle it. If false, do not mangle the name (assume the caller has sent a suitable string) but do turn the string into a pathname located in the cache directory with the cache prefix. the 'mangle' param is true by default. |
Reimplemented in BESStoredDapResultCache.
Definition at line 412 of file BESFileLockingCache.cc.
References BESDEBUG.
Referenced by BESDapResponseCache::cache_dataset().
|
virtual |
Get the cache size.
Read the size information from the cache info file and return it. This methods locks the cache.
Definition at line 672 of file BESFileLockingCache.cc.
References lock_cache_read(), and unlock_cache().
Referenced by purge_file().
|
virtual |
Get a read-only lock on the file if it exists.
Try to get a read-only lock on the file, blocking until we can get it. If the file does not exist, return false.
src | src file that will be cached eventually |
target | a value-result parameter set to the resulting cached file |
Error | if the attempt to get the (shared) lock failed for any reason other than that the file does/did not exist. |
Definition at line 453 of file BESFileLockingCache.cc.
References BESDEBUG, lock_cache_read(), and unlock_cache().
Referenced by BESDapResponseCache::cache_dataset(), and BESStoredDapResultCache::cache_dataset().
const string BESFileLockingCache::getCacheDirectory | ( | ) |
Definition at line 885 of file BESFileLockingCache.cc.
Referenced by BESStoredDapResultCache::get_cache_file_name().
const string BESFileLockingCache::getCacheFilePrefix | ( | ) |
Definition at line 881 of file BESFileLockingCache.cc.
Referenced by BESStoredDapResultCache::get_cache_file_name().
|
protected |
Definition at line 88 of file BESFileLockingCache.cc.
|
virtual |
Get a shared lock on the 'cache info' file.
Definition at line 545 of file BESFileLockingCache.cc.
References BESDEBUG.
Referenced by get_cache_size(), and get_read_lock().
|
virtual |
Get an exclusive lock on the 'cache info' file.
The 'cache info' file is used to control certain cache actions, ensuring that they are atomic. These include making sure that the create_and_lock() and read_and_lock() operations are atomic as well as the purge and related operations.
Definition at line 533 of file BESFileLockingCache.cc.
References BESDEBUG.
Referenced by create_and_lock(), purge_file(), update_and_purge(), and update_cache_info().
|
virtual |
Purge a single file from the cache.
Purge a single file from the cache. The file might be old, etc., and need to be removed. Don't use this to shrink the cache when it gets too big, use update_and_purge() instead since that file optimizes accesses to the cache control file for several changes in a row.
file | The name of the file to purge. |
Definition at line 839 of file BESFileLockingCache.cc.
References BESDEBUG, get_cache_size(), lock_cache_write(), and unlock_cache().
Referenced by BESDapResponseCache::cache_dataset(), and BESStoredDapResultCache::cache_dataset().
|
virtual |
Unlock the named file.
This does not do any name mangling; it just closes and unlocks whatever is named (or throws BESBESInternalErroror if the file cannot be closed). If the file was opened more than once, all descriptors are closed. If you need to close a specific descriptor, use the other version of unlock_and_close().
file_name | The name of the file to unlock. |
BESBESInternalErroror |
Definition at line 583 of file BESFileLockingCache.cc.
References BESDEBUG.
|
virtual |
Unlock the file.
fd | The descriptor of the file to unlock. |
BESBESInternalErroror |
Definition at line 599 of file BESFileLockingCache.cc.
References BESDEBUG.
|
virtual |
Unlock the cache info file.
Definition at line 559 of file BESFileLockingCache.cc.
References BESDEBUG.
Referenced by BESDapResponseCache::cache_dataset(), BESStoredDapResultCache::cache_dataset(), create_and_lock(), get_cache_size(), get_read_lock(), purge_file(), update_and_purge(), and update_cache_info().
|
virtual |
Purge files from the cache.
Purge files, oldest to newest, if the current size of the cache exceeds the size of the cache specified in the constructor. This method uses an exclusive lock on the cache for the duration of the purge process.
new_file | The name of a file this process just added to the cache. Using fcntl(2) locking there is no way this process can detect its own lock, so the shared read lock on the new file won't keep this process from deleting it (but will keep other processes from deleting it). |
Definition at line 755 of file BESFileLockingCache.cc.
References BESDEBUG, BESISDEBUG, cache_too_big(), lock_cache_write(), and unlock_cache().
Referenced by BESDapResponseCache::cache_dataset(), and BESStoredDapResultCache::cache_dataset().
|
virtual |
Update the cache info file to include 'target'.
Add the size of the named file to the total cache size recorded in the cache info file. The cache info file is exclusively locked by this method for its duration. This updates the cache info file and returns the new size.
target | The name of the file |
Definition at line 618 of file BESFileLockingCache.cc.
References BESDEBUG, lock_cache_write(), and unlock_cache().
Referenced by BESDapResponseCache::cache_dataset(), and BESStoredDapResultCache::cache_dataset().