Package org.apache.storm.localizer

## Class LocallyCachedBlob

- java.lang.Object
- - org.apache.storm.localizer.LocallyCachedBlob

- Direct Known Subclasses:
    `LocalizedResource`, `LocallyCachedTopologyBlob`

      public abstract class LocallyCachedBlob
      extends Object

  Represents a blob that is cached locally on disk by the supervisor.

- - ### Field Summary

    Fields
    Modifier and Type | Field | Description
    --- | --- | ---
    `protected long` | `localUpdateTime` |
    `static long` | `NOT_DOWNLOADED_VERSION` |

  - ### Constructor Summary

    Constructors
    Modifier | Constructor | Description
    --- | --- | ---
    `protected` | `LocallyCachedBlob​(String blobDescription, String blobKey, StormMetricsRegistry metricsRegistry)` | Create a new LocallyCachedBlob.

  - ### Method Summary

    All Methods Static Methods Instance Methods Abstract Methods Concrete Methods
    Modifier and Type | Method | Description
    --- | --- | ---
    `void` | `addReference​(PortAndAssignment pna, BlobChangingCallback cb)` | Mark that a given port and assignment are using this.
    `abstract void` | `cleanupOrphanedData()` | Clean up any temporary files.
    `protected abstract void` | `commitNewVersion​(long version)` | Commit the new version and make it available for the end user.
    `abstract void` | `completelyRemove()` | Completely remove anything that is cached locally for this blob and all tracking files also stored for it.
    `protected org.apache.storm.localizer.LocallyCachedBlob.DownloadMeta` | `fetch​(ClientBlobStore store, String key, IOFunction<Long,​Path> pathSupplier, IOFunction<File,​OutputStream> outStreamSupplier)` | Helper function to download blob from blob store.
    `abstract long` | `fetchUnzipToTemp​(ClientBlobStore store)` | Download the latest version to a temp location.
    `Collection<PortAndAssignment>` | `getDependencies()` |
    `String` | `getKey()` | Get the key for this blob.
    `long` | `getLastUsed()` | Get the last time that this used for LRU calculations.
    `abstract long` | `getLocalVersion()` | Get the version of the blob cached locally.
    `abstract long` | `getRemoteVersion​(ClientBlobStore store)` | Get the version of the blob in the blob store.
    `abstract long` | `getSizeOnDisk()` | Get the amount of disk space that is used by this blob.
    `protected static long` | `getSizeOnDisk​(Path p)` | Get the size of p in bytes.
    `void` | `informReferencesAndCommitNewVersion​(long newVersion)` | Inform all of the callbacks that a change is going to happen and then wait for them to all get back that it is OK to make that change.
    `abstract boolean` | `isFullyDownloaded()` |
    `boolean` | `isUsed()` | Return true if this blob is actively being used, else false (meaning it can be deleted, but might not be).
    `boolean` | `removeReference​(PortAndAssignment pna)` | Removes a reservation for this blob from a given slot and assignemnt.
    `protected void` | `touch()` | Updates the last updated time.
    `void` | `update​(ClientBlobStore blobStore, long remoteBlobstoreUpdateTime)` | Checks and downloads a blob locally as necessary.

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### NOT_DOWNLOADED_VERSION

          public static final long NOT_DOWNLOADED_VERSION

      See Also:
        Constant Field Values

    - #### localUpdateTime

          protected long localUpdateTime

  - ### Constructor Detail

    - #### LocallyCachedBlob

          protected LocallyCachedBlob​(String blobDescription,
                                      String blobKey,
                                      StormMetricsRegistry metricsRegistry)

      Create a new LocallyCachedBlob.
      Parameters:
        `blobDescription` - a description of the blob this represents. Typically it should at least be the blob key, but ideally also include if it is an archive or not, what user or topology it is for, or if it is a storm.jar etc.

  - ### Method Detail

    - #### fetch

          protected org.apache.storm.localizer.LocallyCachedBlob.DownloadMeta fetch​(ClientBlobStore store,
                                                                                    String key,
                                                                                    IOFunction<Long,​Path> pathSupplier,
                                                                                    IOFunction<File,​OutputStream> outStreamSupplier)
                                                                             throws KeyNotFoundException,
                                                                                    AuthorizationException,
                                                                                    IOException

      Helper function to download blob from blob store.
      Parameters:
        `store` - Blob store to fetch blobs from
        `key` - Key to retrieve blobs
        `pathSupplier` - A function that supplies the download destination of a blob. It guarantees the validity of path or throws `IOException`
        `outStreamSupplier` - A function that supplies the `OutputStream` object
      Returns:
        The metadata of the download session, including blob's version and download destination
      Throws:
        `KeyNotFoundException` - Thrown if key to retrieve blob is invalid
        `AuthorizationException` - Thrown if the retrieval is not under security authorization
        `IOException` - Thrown if any IO error occurs

    - #### getLocalVersion

          public abstract long getLocalVersion()

      Get the version of the blob cached locally. If the version is unknown or it has not been downloaded NOT_DOWNLOADED_VERSION should be returned. PRECONDITION: this can only be called with a lock on this instance held.

    - #### getRemoteVersion

          public abstract long getRemoteVersion​(ClientBlobStore store)
                                         throws KeyNotFoundException,
                                                AuthorizationException

      Get the version of the blob in the blob store. PRECONDITION: this can only be called with a lock on this instance held.
      Throws:
        `KeyNotFoundException`
        `AuthorizationException`

    - #### fetchUnzipToTemp

          public abstract long fetchUnzipToTemp​(ClientBlobStore store)
                                         throws IOException,
                                                KeyNotFoundException,
                                                AuthorizationException

      Download the latest version to a temp location. This may also include unzipping some or all of the data to a temp location. PRECONDITION: this can only be called with a lock on this instance held.
      Parameters:
        `store` - the store to us to download the data.
      Returns:
        the version that was downloaded.
      Throws:
        `IOException`
        `KeyNotFoundException`
        `AuthorizationException`

    - #### commitNewVersion

          protected abstract void commitNewVersion​(long version)
                                            throws IOException

      Commit the new version and make it available for the end user. PRECONDITION: uncompressToTempLocationIfNeeded will have been called. PRECONDITION: this can only be called with a lock on this instance held.
      Parameters:
        `version` - the version of the blob to commit.
      Throws:
        `IOException`

    - #### cleanupOrphanedData

          public abstract void cleanupOrphanedData()
                                            throws IOException

      Clean up any temporary files. This will be called after updating a blob, either successfully or if an error has occured. The goal is to find any files that may be left over and remove them so space is not leaked. PRECONDITION: this can only be called with a lock on this instance held.
      Throws:
        `IOException`

    - #### completelyRemove

          public abstract void completelyRemove()
                                         throws IOException

      Completely remove anything that is cached locally for this blob and all tracking files also stored for it. This will be called after the blob was determined to no longer be needed in the cache. PRECONDITION: this can only be called with a lock on this instance held.
      Throws:
        `IOException`

    - #### getSizeOnDisk

          public abstract long getSizeOnDisk()

      Get the amount of disk space that is used by this blob. If the blob is uncompressed it should be the sum of the space used by all of the uncompressed files. In general this will not be called with any locks held so it is a good idea to cache it and updated it when committing a new version.

    - #### getSizeOnDisk

          protected static long getSizeOnDisk​(Path p)
                                       throws IOException

      Get the size of p in bytes.
      Parameters:
        `p` - the path to read.
      Returns:
        the size of p in bytes.
      Throws:
        `IOException`

    - #### touch

          protected void touch()

      Updates the last updated time. This should be called when references are added or removed.

    - #### getLastUsed

          public long getLastUsed()

      Get the last time that this used for LRU calculations.

    - #### isUsed

          public boolean isUsed()

      Return true if this blob is actively being used, else false (meaning it can be deleted, but might not be).

    - #### addReference

          public void addReference​(PortAndAssignment pna,
                                   BlobChangingCallback cb)

      Mark that a given port and assignment are using this.
      Parameters:
        `pna` - the slot and assignment that are using this blob.
        `cb` - an optional callback indicating that they want to know/synchronize when a blob is updated.

    - #### removeReference

          public boolean removeReference​(PortAndAssignment pna)

      Removes a reservation for this blob from a given slot and assignemnt.
      Parameters:
        `pna` - the slot + assignment that no longer needs this blob.
      Returns:
        false if a reference was failed to be removed

    - #### informReferencesAndCommitNewVersion

          public void informReferencesAndCommitNewVersion​(long newVersion)
                                                   throws IOException

      Inform all of the callbacks that a change is going to happen and then wait for them to all get back that it is OK to make that change. Commit the new version once all callbacks are ready. Finally inform all callbacks that the commit is complete.
      Throws:
        `IOException`

    - #### getKey

          public String getKey()

      Get the key for this blob.

    - #### getDependencies

          public Collection<PortAndAssignment> getDependencies()

    - #### isFullyDownloaded

          public abstract boolean isFullyDownloaded()

    - #### update

          public void update​(ClientBlobStore blobStore,
                             long remoteBlobstoreUpdateTime)
                      throws KeyNotFoundException,
                             AuthorizationException,
                             IOException

      Checks and downloads a blob locally as necessary.
      Parameters:
        `blobStore` - the client blobstore
        `remoteBlobstoreUpdateTime` - last update time of remote blobstore
      Throws:
        `KeyNotFoundException` - if the remote blob is missing
        `AuthorizationException` - if authorization is failed
        `IOException` - on errors