Package org.apache.storm.blobstore

## Class BlobStore

- java.lang.Object
- - org.apache.storm.blobstore.BlobStore

- All Implemented Interfaces:
    `AutoCloseable`, `Shutdownable`
  Direct Known Subclasses:
    `HdfsBlobStore`, `LocalFsBlobStore`

      public abstract class BlobStore
      extends Object
      implements Shutdownable, AutoCloseable

  Provides a way to store blobs that can be downloaded. Blobs must be able to be uploaded and listed from Nimbus, and downloaded from the Supervisors. It is a key value based store. Key being a string and value being the blob data.

  ACL checking must take place against the provided subject. If the blob store does not support Security it must validate that all ACLs set are always WORLD, everything.

  The users can upload their blobs through the blob store command line. The command line also allows us to update and delete blobs.

  Modifying the replication factor only works for HdfsBlobStore as for the LocalFsBlobStore the replication is dependent on the number of Nimbodes available.

- - ### Nested Class Summary

    Nested Classes
    Modifier and Type | Class | Description
    --- | --- | ---
    `protected class` | `BlobStore.BlobStoreFileInputStream` | Input stream implementation used for writing both the metadata containing the acl information and the blob data.
    `protected class` | `BlobStore.BlobStoreFileOutputStream` | Output stream implementation used for reading the metadata and data information.
    `static class` | `BlobStore.KeyTranslationIterator` | Blob store implements its own version of iterator to list the blobs.

  - ### Field Summary

    Fields
    Modifier and Type | Field | Description
    --- | --- | ---
    `protected static String` | `BASE_BLOBS_DIR_NAME` |

  - ### Constructor Summary

    Constructors
    Constructor | Description
    --- | ---
    `BlobStore()` |

  - ### Method Summary

    All Methods Static Methods Instance Methods Abstract Methods Concrete Methods
    Modifier and Type | Method | Description
    --- | --- | ---
    `void` | `close()` |
    `void` | `createBlob​(String key, byte[] data, SettableBlobMeta meta, Subject who)` | Wrapper called to create the blob which contains the byte data.
    `void` | `createBlob​(String key, InputStream in, SettableBlobMeta meta, Subject who)` | Wrapper called to create the blob which contains the byte data.
    `abstract AtomicOutputStream` | `createBlob​(String key, SettableBlobMeta meta, Subject who)` | Creates the blob.
    `abstract void` | `deleteBlob​(String key, Subject who)` | Deletes the blob data and metadata.
    `<R> Set<R>` | `filterAndListKeys​(KeyFilter<R> filter)` | Filters keys based on the KeyFilter passed as the argument.
    `abstract InputStreamWithMeta` | `getBlob​(String key, Subject who)` | Gets the InputStream to read the blob details.
    `abstract ReadableBlobMeta` | `getBlobMeta​(String key, Subject who)` | Gets the current version of metadata for a blob to be viewed by the user or downloaded by the supervisor.
    `abstract int` | `getBlobReplication​(String key, Subject who)` | Gets the replication factor of the blob.
    `abstract Iterator<String>` | `listKeys()` | Returns an iterator with all the list of keys currently available on the blob store.
    `abstract void` | `prepare​(Map<String,​Object> conf, String baseDir, NimbusInfo nimbusInfo, ILeaderElector leaderElector)` | Allows us to initialize the blob store.
    `byte[]` | `readBlob​(String key, Subject who)` | Wrapper around readBlobTo which returns a ByteArray output stream.
    `void` | `readBlobTo​(String key, OutputStream out, Subject who)` | Reads the blob from the blob store and writes it into the output stream.
    `abstract void` | `setBlobMeta​(String key, SettableBlobMeta meta, Subject who)` | Sets the metadata with renewed acls for the blob.
    `abstract void` | `setLeaderElector​(ILeaderElector leaderElector)` | Sets leader elector (only used by LocalFsBlobStore to help sync blobs between Nimbi.
    `void` | `startSyncBlobs()` | Start the syncing blobs between the local running instance of the BlobStore and others.
    `Set<String>` | `storedTopoIds()` | Get IDs stored in blob store.
    `void` | `updateBlob​(String key, byte[] data, Subject who)` | Wrapper called to create the blob which contains the byte data.
    `abstract AtomicOutputStream` | `updateBlob​(String key, Subject who)` | Updates the blob data.
    `abstract int` | `updateBlobReplication​(String key, int replication, Subject who)` | Modifies the replication factor of the blob.
    `void` | `updateLastBlobUpdateTime()` | Updates the last update time of existing blobs in the blobstore to the current time.
    `void` | `validateBlobUpdateTime()` | Validates that the blob update time of the blobstore is up to date with the current existing blobs.
    `static void` | `validateKey​(String key)` | Validates key checking for potentially harmful patterns.

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

    - ### Methods inherited from interface org.apache.storm.daemon.Shutdownable

      `shutdown`

- - ### Field Detail

    - #### BASE_BLOBS_DIR_NAME

          protected static final String BASE_BLOBS_DIR_NAME

      See Also:
        Constant Field Values

  - ### Constructor Detail

    - #### BlobStore

          public BlobStore()

  - ### Method Detail

    - #### validateKey

          public static final void validateKey​(String key)
                                        throws IllegalArgumentException

      Validates key checking for potentially harmful patterns.
      Parameters:
        `key` - Key for the blob
      Throws:
        `IllegalArgumentException`

    - #### prepare

          public abstract void prepare​(Map<String,​Object> conf,
                                       String baseDir,
                                       NimbusInfo nimbusInfo,
                                       ILeaderElector leaderElector)

      Allows us to initialize the blob store.
      Parameters:
        `conf` - The storm configuration
        `baseDir` - The directory path to store the blobs
        `nimbusInfo` - Contains the nimbus host, port and leadership information

    - #### startSyncBlobs

          public void startSyncBlobs()
                              throws KeyNotFoundException,
                                     AuthorizationException

      Start the syncing blobs between the local running instance of the BlobStore and others. A no-op for the HdfsBlobStore where HDFS itself does the syncing but for the LocalFsBlobStore ZK state updates are run periodically here.
      Throws:
        `KeyNotFoundException`
        `AuthorizationException`

    - #### createBlob

          public abstract AtomicOutputStream createBlob​(String key,
                                                        SettableBlobMeta meta,
                                                        Subject who)
                                                 throws AuthorizationException,
                                                        KeyAlreadyExistsException

      Creates the blob.
      Parameters:
        `key` - Key for the blob
        `meta` - Metadata which contains the acls information
        `who` - Is the subject creating the blob
      Returns:
        AtomicOutputStream returns a stream into which the data can be written
      Throws:
        `AuthorizationException`
        `KeyAlreadyExistsException`

    - #### createBlob

          public void createBlob​(String key,
                                 byte[] data,
                                 SettableBlobMeta meta,
                                 Subject who)
                          throws AuthorizationException,
                                 KeyAlreadyExistsException,
                                 IOException

      Wrapper called to create the blob which contains the byte data.
      Parameters:
        `key` - Key for the blob
        `data` - Byte data that needs to be uploaded
        `meta` - Metadata which contains the acls information
        `who` - Is the subject creating the blob
      Throws:
        `AuthorizationException`
        `KeyAlreadyExistsException`
        `IOException`

    - #### createBlob

          public void createBlob​(String key,
                                 InputStream in,
                                 SettableBlobMeta meta,
                                 Subject who)
                          throws AuthorizationException,
                                 KeyAlreadyExistsException,
                                 IOException

      Wrapper called to create the blob which contains the byte data.
      Parameters:
        `key` - Key for the blob
        `in` - InputStream from which the data is read to be written as a part of the blob
        `meta` - Metadata which contains the acls information
        `who` - Is the subject creating the blob
      Throws:
        `AuthorizationException`
        `KeyAlreadyExistsException`
        `IOException`

    - #### updateBlob

          public abstract AtomicOutputStream updateBlob​(String key,
                                                        Subject who)
                                                 throws AuthorizationException,
                                                        KeyNotFoundException

      Updates the blob data.
      Parameters:
        `key` - Key for the blob
        `who` - Is the subject having the write privilege for the blob
      Returns:
        AtomicOutputStream returns a stream into which the data can be written
      Throws:
        `AuthorizationException`
        `KeyNotFoundException`

    - #### updateBlob

          public void updateBlob​(String key,
                                 byte[] data,
                                 Subject who)
                          throws AuthorizationException,
                                 IOException,
                                 KeyNotFoundException

      Wrapper called to create the blob which contains the byte data.
      Parameters:
        `key` - Key for the blob
        `data` - Byte data that needs to be uploaded
        `who` - Is the subject creating the blob
      Throws:
        `AuthorizationException`
        `IOException`
        `KeyNotFoundException`

    - #### getBlobMeta

          public abstract ReadableBlobMeta getBlobMeta​(String key,
                                                       Subject who)
                                                throws AuthorizationException,
                                                       KeyNotFoundException

      Gets the current version of metadata for a blob to be viewed by the user or downloaded by the supervisor.
      Parameters:
        `key` - Key for the blob
        `who` - Is the subject having the read privilege for the blob
      Returns:
        AtomicOutputStream returns a stream into which the data can be written
      Throws:
        `AuthorizationException`
        `KeyNotFoundException`

    - #### setLeaderElector

          public abstract void setLeaderElector​(ILeaderElector leaderElector)

      Sets leader elector (only used by LocalFsBlobStore to help sync blobs between Nimbi.

    - #### setBlobMeta

          public abstract void setBlobMeta​(String key,
                                           SettableBlobMeta meta,
                                           Subject who)
                                    throws AuthorizationException,
                                           KeyNotFoundException

      Sets the metadata with renewed acls for the blob.
      Parameters:
        `key` - Key for the blob
        `meta` - Metadata which contains the updated acls information
        `who` - Is the subject having the write privilege for the blob
      Throws:
        `AuthorizationException`
        `KeyNotFoundException`

    - #### deleteBlob

          public abstract void deleteBlob​(String key,
                                          Subject who)
                                   throws AuthorizationException,
                                          KeyNotFoundException

      Deletes the blob data and metadata.
      Parameters:
        `key` - Key for the blob
        `who` - Is the subject having write privilege for the blob
      Throws:
        `AuthorizationException`
        `KeyNotFoundException`

    - #### getBlob

          public abstract InputStreamWithMeta getBlob​(String key,
                                                      Subject who)
                                               throws AuthorizationException,
                                                      KeyNotFoundException

      Gets the InputStream to read the blob details.
      Parameters:
        `key` - Key for the blob
        `who` - Is the subject having the read privilege for the blob
      Returns:
        InputStreamWithMeta has the additional file length and version information
      Throws:
        `AuthorizationException`
        `KeyNotFoundException`

    - #### listKeys

          public abstract Iterator<String> listKeys()

      Returns an iterator with all the list of keys currently available on the blob store.
      Returns:
        `Iterator<String>`

    - #### getBlobReplication

          public abstract int getBlobReplication​(String key,
                                                 Subject who)
                                          throws Exception

      Gets the replication factor of the blob.
      Parameters:
        `key` - Key for the blob
        `who` - Is the subject having the read privilege for the blob
      Returns:
        BlobReplication object containing the replication factor for the blob
      Throws:
        `Exception`

    - #### updateBlobReplication

          public abstract int updateBlobReplication​(String key,
                                                    int replication,
                                                    Subject who)
                                             throws AuthorizationException,
                                                    KeyNotFoundException,
                                                    IOException

      Modifies the replication factor of the blob.
      Parameters:
        `key` - Key for the blob
        `replication` - The replication factor the blob has to be set
        `who` - Is the subject having the update privilege for the blob
      Returns:
        BlobReplication object containing the updated replication factor for the blob
      Throws:
        `AuthorizationException`
        `KeyNotFoundException`
        `IOException`

    - #### close

          public void close()

      Specified by:
        `close` in interface `AutoCloseable`

    - #### filterAndListKeys

          public <R> Set<R> filterAndListKeys​(KeyFilter<R> filter)

      Filters keys based on the KeyFilter passed as the argument.
      Parameters:
        `filter` - KeyFilter
      Returns:
        Set of filtered keys

    - #### readBlobTo

          public void readBlobTo​(String key,
                                 OutputStream out,
                                 Subject who)
                          throws IOException,
                                 KeyNotFoundException,
                                 AuthorizationException

      Reads the blob from the blob store and writes it into the output stream.
      Parameters:
        `key` - Key for the blob
        `out` - Output stream
        `who` - Is the subject having read privilege for the blob
      Throws:
        `IOException`
        `KeyNotFoundException`
        `AuthorizationException`

    - #### readBlob

          public byte[] readBlob​(String key,
                                 Subject who)
                          throws IOException,
                                 KeyNotFoundException,
                                 AuthorizationException

      Wrapper around readBlobTo which returns a ByteArray output stream.
      Parameters:
        `key` - Key for the blob
        `who` - Is the subject having the read privilege for the blob
      Throws:
        `IOException`
        `KeyNotFoundException`
        `AuthorizationException`

    - #### storedTopoIds

          public Set<String> storedTopoIds()

      Get IDs stored in blob store.
      Returns:
        a set of all of the topology ids with special data stored in the blob store.

    - #### updateLastBlobUpdateTime

          public void updateLastBlobUpdateTime()
                                        throws IOException

      Updates the last update time of existing blobs in the blobstore to the current time.
      Throws:
        `IOException` - on any error

    - #### validateBlobUpdateTime

          public void validateBlobUpdateTime()
                                      throws IOException

      Validates that the blob update time of the blobstore is up to date with the current existing blobs.
      Throws:
        `IOException` - on any error