This repository has been archived by the owner on Jan 17, 2022. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 33
ID API
hojo edited this page Sep 26, 2012
·
1 revision
This API allows a caller to specify objects by their IDs.
- dpl_post_id() Adds objects to storage using the HTTP POST method, and returns object IDs.
- dpl_post_buffered_id() Adds objects to storage from a buffered stream using the POST method, and returns object IDs.
- dpl_put_id() Adds objects to storage using the PUT method, and returns object IDs.
- dpl_put_buffered_id() Adds objects to storage from a buffered stream using the PUT method, and returns object IDs.
- dpl_get_id() Returns objects from storage.
- dpl_get_range_id() Returns data chunks containing subportions of objects from storage.
- dpl_get_buffered_id() Returns objects from storage in a buffered stream.
- dpl_head_id() Returns user metadata.
- dpl_head_all_id() Returns user and system metadata.
- dpl_head_sysmd_id() Returns system metadata.
- dpl_delete_id() Marks objects in storage for deletion.
- dpl_copy_id() Copies objects from one storage location to another.
- dpl_gen_id_from_oid() Generates unique keys from arbitrary large integers.
- ctx The droplet context.
- bucket Container for objects in storage.
- subresource Component of an object in storage.
- object_type Context type of an object in storage.
- metadata Metadata of an object in storage.
- sysmd System metadata of a stored object.
- data_buf The object body.
- data_len The object length.
- query_params Additional variables passed through POST queries.
- id Arbitrary large integer uniquely identifying an object.
- condition Condition for copying an object from one bucket to another.
- start Start of a range for a data chunk.
- end End of a range for a data chunk.
- header_func Callback function for every header found in the HTTP response.
- buffer_func Callback function for every chunk of data found in the HTTP response.
- cb_arg User-defined callback argument (closure).
- src_bucket Source bucket for object copy operations.
- src_id Source ID for object copy operations.
- src_subresource Source name for component included in object copy operations.
- dst_bucket Destination bucket for object copy operations.
- dst_id Destination ID for object copy operations.
- dst_subresource Destination name for component included in object copy operations.
- metadata_directive Header describing how to process metadata for copy operations.
- oid Arbitrary large integer.
- storage_class Class of service indicating the number of replicas of an object in storage.
- resource_idp Returns ID of the object.
- connp Returns connection handles. Caller must free resource after use by calling dpl_conn_release() or dpl_conn_terminate().
- data_bufp Returns object body. Caller must release resource after use by calling dpl_free().
- data_lenp Returns object length.
- metadatap Returns object metadata. Caller must release resource after use by calling dpl_dict_free().
dpl_status_t
dpl_post_id(dpl_ctx_t *ctx,
char *bucket,
char *subresource,
dpl_ftype_t object_type,
dpl_dict_t *metadata,
dpl_sysmd_t *sysmd,
char *data_buf,
unsigned int data_len,
dpl_dict_t *query_params,
char **resource_idp);
dpl_status_t
dpl_post_buffered_id(dpl_ctx_t *ctx,
char *bucket,
char *subresource,
dpl_ftype_t object_type,
dpl_dict_t *metadata,
dpl_sysmd_t *sysmd,
unsigned int data_len,
dpl_dict_t *query_params,
dpl_conn_t **connp);
dpl_status_t
dpl_put_id(dpl_ctx_t *ctx,
char *bucket,
char *id,
char *subresource,
dpl_ftype_t object_type,
dpl_dict_t *metadata,
dpl_sysmd_t *sysmd,
char *data_buf,
unsigned int data_len);
dpl_status_t
dpl_put_buffered_id(dpl_ctx_t *ctx,
char *bucket,
char *id,
char *subresource,
dpl_ftype_t object_type,
dpl_dict_t *metadata,
dpl_sysmd_t *sysmd,
unsigned int data_len,
dpl_conn_t **connp);
dpl_status_t
dpl_get_id(dpl_ctx_t *ctx,
char *bucket,
char *id,
char *subresource,
dpl_ftype_t object_type,
dpl_condition_t *condition,
char **data_bufp,
unsigned int *data_lenp,
dpl_dict_t **metadatap);
dpl_status_t
dpl_get_range_id(dpl_ctx_t *ctx,
char *bucket,
char *id,
char *subresource,
dpl_ftype_t object_type,
dpl_condition_t *condition,
int start,
int end,
char **data_bufp,
unsigned int *data_lenp,
dpl_dict_t **metadatap);
dpl_status_t
dpl_get_buffered_id(dpl_ctx_t *ctx,
char *bucket,
char *id,
char *subresource,
dpl_ftype_t object_type,
dpl_condition_t *condition,
dpl_header_func_t header_func,
dpl_buffer_func_t buffer_func,
void *cb_arg);
dpl_status_t dpl_head_id(dpl_ctx_t *ctx,
char *bucket,
char *id,
char *subresource,
dpl_condition_t *condition,
dpl_dict_t **metadatap);
dpl_status_t
dpl_head_all_id(dpl_ctx_t *ctx,
char *bucket,
char *id,
char *subresource,
dpl_condition_t *condition,
dpl_dict_t **metadatap);
dpl_status_t
dpl_head_sysmd_id(dpl_ctx_t *ctx,
char *bucket,
char *id,
char *subresource,
dpl_condition_t *condition,
dpl_sysmd_t *sysmdp,
dpl_dict_t **metadatap);
dpl_status_t
dpl_delete_id(dpl_ctx_t *ctx,
char *bucket,
char *id,
char *subresource);
dpl_status_t
dpl_copy_id(dpl_ctx_t *ctx,
char *src_bucket,
char *src_id,
char *src_subresource,
char *dst_bucket,
char *dst_id,
char *dst_subresource,
dpl_ftype_t object_type,
dpl_metadata_directive_t metadata_directive,
dpl_dict_t *metadata,
dpl_sysmd_t *sysmd,
dpl_condition_t *condition);
dpl_status_t
dpl_gen_id_from_oid(dpl_ctx_t *ctx,
uint64_t oid,
dpl_storage_class_t storage_class,
char **resource_idp);
To uniquely identify an object, its location and how many copies of that object should exist, a 20-byte (160 bits) key is divided into predefined parts. At the highest level, there are three parts:
- Hashing/Dispersion: the first 24 bits of all keys is an MD5 hash of the payload. This makes sure that adjacent objects are dispersed across all storage nodes.
- Payload: the next 128 bits uniquely identify an object. The payload includes 64 bits for the object ID and 32 bits for the volume ID.
- Class of service: 8 bits, 4 bits for the class itself, and 4 bits to specify the replica number of a particular key. This ensures that a replica of an object cannot clash with the key of another object.
The SRWS-specific functions of the ID API are included to permit key generation required by the storage system.
- dpl_srws_gen_key() Generates keys for objects without the class-of-service bits.
- dpl_srws_set_class() Sets the class of service (number of replicas) for objects in storage.
- id (IN/OUT) Unique key as computed for an object in storage. The key must be pre-allocated in a BN_new() call.
- oid Arbitrary 64-bit large integer unique for given volid.
- volid 32-bit volume ID.
- serviceid 8-bit ID for the application interfacing with the storage system.
- specific 24-bit ID specific to the application, such as a mailbox ID for a webmail application.
- class Class of service indicating the number of replicas of an object in storage.
dpl_status_t
dpl_srws_gen_key(BIGNUM *id,
uint64_t oid,
uint32_t volid,
uint8_t serviceid,
uint32_t specific);
dpl_status_t
dpl_srws_set_class(BIGNUM *id,
int class);