public class HdfsBlobStore extends BlobStore
Provides a HDFS file system backed blob store implementation. Note that this provides an api for having HDFS be the backing store for the blobstore, it is not a service/daemon.
We currently have NIMBUS_ADMINS and SUPERVISOR_ADMINS configuration. NIMBUS_ADMINS are given READ, WRITE and ADMIN access whereas the SUPERVISOR_ADMINS are given READ access in order to read and download the blobs form the nimbus.
The ACLs for the blob store are validated against whether the subject is a NIMBUS_ADMIN, SUPERVISOR_ADMIN or USER who has read, write or admin privileges in order to perform respective operations on the blob.
For hdfs blob store 1. The USER interacts with nimbus to upload and access blobs through NimbusBlobStore Client API. Here, unlike local blob store which stores the blobs locally, the nimbus talks to HDFS to upload the blobs. 2. The USER sets the ACLs, and the blob access is validated against these ACLs. 3. The SUPERVISOR interacts with nimbus through HdfsClientBlobStore to download the blobs. Here, unlike local blob store the supervisor interacts with HDFS directly to download the blobs. The call to HdfsBlobStore is made as a “null” subject. The blobstore gets the hadoop user and validates permissions for the supervisor.
BlobStore.BlobStoreFileInputStream, BlobStore.BlobStoreFileOutputStream, BlobStore.KeyTranslationIterator| Modifier and Type | Field and Description |
|---|---|
static org.slf4j.Logger |
LOG |
BASE_BLOBS_DIR_NAME| Constructor and Description |
|---|
HdfsBlobStore() |
| Modifier and Type | Method and Description |
|---|---|
AtomicOutputStream |
createBlob(String key,
SettableBlobMeta meta,
Subject who) |
void |
deleteBlob(String key,
Subject who) |
void |
fullCleanup(long age) |
InputStreamWithMeta |
getBlob(String key,
Subject who) |
ReadableBlobMeta |
getBlobMeta(String key,
Subject who) |
int |
getBlobReplication(String key,
Subject who) |
Iterator<String> |
listKeys() |
void |
prepare(Map conf,
String overrideBase,
NimbusInfo nimbusInfo) |
protected void |
prepareInternal(Map conf,
String overrideBase,
org.apache.hadoop.conf.Configuration hadoopConf)
Allow a Hadoop Configuration to be passed for testing.
|
void |
setBlobMeta(String key,
SettableBlobMeta meta,
Subject who) |
void |
shutdown() |
AtomicOutputStream |
updateBlob(String key,
Subject who) |
int |
updateBlobReplication(String key,
int replication,
Subject who) |
void |
writeMetadata(String key,
SettableBlobMeta meta) |
createBlob, createBlob, filterAndListKeys, readBlob, readBlobTo, validateKeypublic void prepare(Map conf, String overrideBase, NimbusInfo nimbusInfo)
protected void prepareInternal(Map conf, String overrideBase, org.apache.hadoop.conf.Configuration hadoopConf)
Allow a Hadoop Configuration to be passed for testing. If it’s null then the hadoop configs must be in your classpath.
public AtomicOutputStream createBlob(String key, SettableBlobMeta meta, Subject who) throws AuthorizationException, KeyAlreadyExistsException
createBlob in class BlobStoreAuthorizationExceptionKeyAlreadyExistsExceptionpublic AtomicOutputStream updateBlob(String key, Subject who) throws AuthorizationException, KeyNotFoundException
updateBlob in class BlobStoreAuthorizationExceptionKeyNotFoundExceptionpublic ReadableBlobMeta getBlobMeta(String key, Subject who) throws AuthorizationException, KeyNotFoundException
getBlobMeta in class BlobStoreAuthorizationExceptionKeyNotFoundExceptionpublic void setBlobMeta(String key, SettableBlobMeta meta, Subject who) throws AuthorizationException, KeyNotFoundException
setBlobMeta in class BlobStoreAuthorizationExceptionKeyNotFoundExceptionpublic void deleteBlob(String key, Subject who) throws AuthorizationException, KeyNotFoundException
deleteBlob in class BlobStoreAuthorizationExceptionKeyNotFoundExceptionpublic InputStreamWithMeta getBlob(String key, Subject who) throws AuthorizationException, KeyNotFoundException
getBlob in class BlobStoreAuthorizationExceptionKeyNotFoundExceptionpublic void shutdown()
public int getBlobReplication(String key, Subject who) throws AuthorizationException, KeyNotFoundException
getBlobReplication in class BlobStoreAuthorizationExceptionKeyNotFoundExceptionpublic int updateBlobReplication(String key, int replication, Subject who) throws AuthorizationException, KeyNotFoundException
updateBlobReplication in class BlobStoreAuthorizationExceptionKeyNotFoundExceptionpublic void writeMetadata(String key, SettableBlobMeta meta) throws AuthorizationException, KeyNotFoundException
public void fullCleanup(long age)
throws IOException
IOExceptionCopyright © 2018 The Apache Software Foundation. All Rights Reserved.