Package gov.usgs.earthquake.distribution

Class FileProductStorage

java.lang.Object
gov.usgs.util.DefaultConfigurable
gov.usgs.earthquake.distribution.FileProductStorage
All Implemented Interfaces:
ProductStorage, Configurable
Direct Known Subclasses:
ConsolidatedFileProductStorage, DYFILegacyStorage, HashFileProductStorage, URLProductStorage
public class FileProductStorage extends DefaultConfigurable implements ProductStorage
Store products in the file system. This implementation of ProductStorage extracts products into directories. The FileProductStorage implements the Configurable interface and can use the following configuration parameters:
directory
(Optional, default = storage) The base directory where products are stored. Each product is stored in a separate directory within this directory.
verifySignatures
(Optional, default = off) Whether or not to verify signatures:
off
no verification
test
test but accept invalid signatures
anything else
reject invalid signatures.
keychain
(Optional) List of key section names to load for signature verification.
An attempt is made to make storage operations atomic by using read and write locks. While a write operation (store or remove) is in progress, read operations will block. It is possible for a remove operation to occur between the time getProduct() returns and the time when product contents are actually loaded from a file. Users who are concerned about this should use the getInMemoryProduct() method, which holds a read lock until all product files are read. To override the directory structure or format, override one or more of the following methods: 
 String getProductPath(ProductId)
 ProductSource getProductSourceFormat(File)
 ProductOutput getProductHandlerFormat(File)

  • Field Details

    • STORAGE_LISTENER_PROPERTY

      public static final String STORAGE_LISTENER_PROPERTY
      Property for configured listeners
      See Also:

    • DIRECTORY_PROPERTY_NAME

      public static final String DIRECTORY_PROPERTY_NAME
      Storage path property name used by Configurable interface.
      See Also:

    • DEFAULT_DIRECTORY

      public static final String DEFAULT_DIRECTORY
      Default storage path if none is provided.
      See Also:

    • USE_ATOMIC_DOWNLOAD_PROPERTY

      public static final String USE_ATOMIC_DOWNLOAD_PROPERTY
      See Also:

    • USE_ATOMIC_DOWNLOAD_DEFAULT

      public static final boolean USE_ATOMIC_DOWNLOAD_DEFAULT
      See Also:

    • USE_HASH_PATHS_PROPERTY

      public static final String USE_HASH_PATHS_PROPERTY
      Property for whether or not to hash file paths.
      See Also:

    • USE_HASH_PATHS_DEFAULT

      public static final boolean USE_HASH_PATHS_DEFAULT
      Do not use hashes (Default).
      See Also:

    • LEGACY_STORAGES_PROPERTY

      public static final String LEGACY_STORAGES_PROPERTY
      Property for legacyStorages.
      See Also:

    • DIRECTORY_NAME_LENGTH

      public static final int DIRECTORY_NAME_LENGTH
      This is chosen because 16^3 = 4096 < 32000, which is the ext3 subdirectory limit.
      See Also:

  • Constructor Details

    • FileProductStorage

      public FileProductStorage()
      Create a new FileProductStorage using the default storage path.

    • FileProductStorage

      public FileProductStorage(File baseDirectory)
      Create a new FileProductStorage.
      Parameters:
      baseDirectory - the base directory for all products being stored.

  • Method Details

    • getStorageLocks

      public ObjectLock<ProductId> getStorageLocks()
      Returns:
      the storageLocks

    • configure

      public void configure(Config config) throws Exception
      Configure this object. Expects a key named "directory".
      Specified by:
      configure in interface Configurable
      Overrides:
      configure in class DefaultConfigurable
      Parameters:
      config - the Config object with settings.
      Throws:
      Exception - if configuration exceptions occur.

    • notifyListeners

      public void notifyListeners(StorageEvent event)
      Description copied from interface: ProductStorage
      Notifies StorageListeners of the change to the ProductStorage.
      Specified by:
      notifyListeners in interface ProductStorage
      Parameters:
      event - StorageEvent

    • addStorageListener

      public void addStorageListener(StorageListener listener)
      Description copied from interface: ProductStorage
      Adds a StorageListener to be notified when a change occurs in this ProductStorage.
      Specified by:
      addStorageListener in interface ProductStorage
      Parameters:
      listener - The listener to notify of changes.

    • removeStorageListener

      public void removeStorageListener(StorageListener listener)
      Description copied from interface: ProductStorage
      Removes a StorageListener from being notified when a change occurs in this ProductStorage.
      Specified by:
      removeStorageListener in interface ProductStorage
      Parameters:
      listener - The listener to remove

    • getProductPath

      public String getProductPath(ProductId id)
      A method for subclasses to override the storage path. The returned path is appended to the base directory when storing and retrieving products.
      Parameters:
      id - the product id to convert.
      Returns:
      the directory used to store id.

    • getHashedProductPath

      protected String getHashedProductPath(ProductId id)
      Parameters:
      id - Specific productID
      Returns:
      string buffer of hashed product path

    • getNormalProductPath

      public String getNormalProductPath(ProductId id)
      Parameters:
      id - ProductId
      Returns:
      string buffer of normal product path

    • getProductHandlerFormat

      protected ProductHandler getProductHandlerFormat(File file) throws Exception
      A method for subclasses to override the storage format. When overriding this method, the method getProductSourceFormat should also be overridden.
      Parameters:
      file - a file that should be converted into a ProductHandler.
      Returns:
      the ProductHandler.
      Throws:
      Exception - if error occurs

    • getProductSourceFormat

      protected ProductSource getProductSourceFormat(File file) throws Exception
      A method for subclasses to override the storage format. When overriding this method, the method getProductHandlerFormat should also be overridden.
      Parameters:
      file - a file that should be converted into a ProductSource.
      Returns:
      the ProductSource.
      Throws:
      Exception - if error occurs

    • getProductFile

      public File getProductFile(ProductId id)
      Get the file or directory used to store a specific product.
      Parameters:
      id - which product.
      Returns:
      a file or directory where the product would be stored.

    • getTempProductFile

      public File getTempProductFile(ProductId id)
      Get the temp file or directory used to store a specific product.
      Parameters:
      id - which product.
      Returns:
      a file or directory where the product would be stored.

    • getProduct

      public Product getProduct(ProductId id) throws Exception
      Get a product from storage. Calls the getProductSource method, and uses ObjectProductHandler to convert the ProductSource into a Product.
      Specified by:
      getProduct in interface ProductStorage
      Parameters:
      id - the product to retrieve.
      Returns:
      the product, or null if not in this storage.
      Throws:
      Exception - if errors occur while retrieving product.

    • getInMemoryProduct

      public Product getInMemoryProduct(ProductId id) throws Exception
      Get a product from storage, loading all file contents into memory. This method may cause memory problems if product contents are large.
      Parameters:
      id - the product to retrieve.
      Returns:
      the loaded product.
      Throws:
      Exception - if error occurs

    • getProductSource

      public ProductSource getProductSource(ProductId id) throws Exception
      Get a ProductSource from storage.
      Specified by:
      getProductSource in interface ProductStorage
      Parameters:
      id - the product to retrieve.
      Returns:
      a ProductSource for the product, or null if not in this storage.
      Throws:
      Exception - if any errors occur while getting the ProductInput.

    • hasProduct

      public boolean hasProduct(ProductId id) throws Exception
      Check whether a product exists in storage.
      Specified by:
      hasProduct in interface ProductStorage
      Parameters:
      id - the product to check.
      Returns:
      true if the product exists, false otherwise.
      Throws:
      Exception - if an error occurs while checking.

    • removeProduct

      public void removeProduct(ProductId id) throws Exception
      Remove a product from storage.
      Specified by:
      removeProduct in interface ProductStorage
      Parameters:
      id - product to remove.
      Throws:
      Exception - if errors occur while removing product.

    • storeProduct

      public ProductId storeProduct(Product product) throws Exception
      Store a product in storage. Same as storeProductSource(new ObjectProductSource(product)).
      Specified by:
      storeProduct in interface ProductStorage
      Parameters:
      product - the product to store.
      Returns:
      the id of the stored product.
      Throws:
      Exception - if errors occur while storing product.

    • storeProductSource

      public ProductId storeProductSource(ProductSource source) throws Exception
      Store a ProductSource to storage. If any exceptions occur while storing a product (other than the product already existing in storage) the incompletely stored product is removed.
      Specified by:
      storeProductSource in interface ProductStorage
      Parameters:
      source - the ProductSource to store.
      Returns:
      the id of the stored product.
      Throws:
      Exception - if errors occur while storing product.

    • shutdown

      public void shutdown() throws Exception
      Called at client shutdown to free resources.
      Specified by:
      shutdown in interface Configurable
      Overrides:
      shutdown in class DefaultConfigurable
      Throws:
      Exception - if exceptions occur while starting.

    • startup

      public void startup() throws Exception
      Called after client configuration to begin processing.
      Specified by:
      startup in interface Configurable
      Overrides:
      startup in class DefaultConfigurable
      Throws:
      Exception - if exceptions occur while starting.

    • getBaseDirectory

      public File getBaseDirectory()
      Returns:
      the baseDirectory

    • setBaseDirectory

      public void setBaseDirectory(File baseDirectory)
      Parameters:
      baseDirectory - the baseDirectory to set

    • isRejectInvalidSignatures

      public boolean isRejectInvalidSignatures()
      Returns:
      the rejectInvalidSignatures

    • setRejectInvalidSignatures

      public void setRejectInvalidSignatures(boolean rejectInvalidSignatures)
      Parameters:
      rejectInvalidSignatures - the rejectInvalidSignatures to set

    • isTestSignatures

      public boolean isTestSignatures()
      Returns:
      the testSignatures

    • setTestSignatures

      public void setTestSignatures(boolean testSignatures)
      Parameters:
      testSignatures - the testSignatures to set

    • getKeychain

      public ProductKeyChain getKeychain()
      Returns:
      the keychain

    • setKeychain

      public void setKeychain(ProductKeyChain keychain)
      Parameters:
      keychain - the keychain to set

    • getLegacyStorages

      public List<ProductStorage> getLegacyStorages()
      Returns:
      the legacyStorages.