This class represents a hierarchical cloud storage dataset. More...

#include <SDHierarchicalDataset.h>

Public Member Functions
	SDHierarchicalDataset (SDManager *sdmanager, const std::string &sdfilename, const bool log=false)
	Constructor. More...

	SDHierarchicalDataset (SDManager *sdmanager, const std::string &sdfilename, const std::string &filetype, const bool log=false)
	Constructor with dataset type. More...

	~SDHierarchicalDataset ()
	Destructor. More...

	SDHierarchicalDataset (SDHierarchicalDataset &&sdhd) noexcept
	Move Constructor. More...

SDHierarchicalDataset &	operator= (SDHierarchicalDataset &&sdhd) noexcept
	Move Assignment Operator. More...

	SDHierarchicalDataset (const SDHierarchicalDataset &op)
	Copy Constructor. More...

SDHierarchicalDataset &	operator= (const SDHierarchicalDataset &op)
	Copy Assignment Operator. More...

void	open (const SDDatasetDisposition disposition)
	Opens the dataset whose name was specified the constructor. More...

void	open (const SDDatasetDisposition disposition, const std::string &legaltag)
	Opens the dataset whose name was specified the constructor. More...

void	open (const SDDatasetDisposition disposition, const std::string &legaltag, const std::string &wid)
	Opens the dataset whose name was specified the constructor. More...

void	open (const SDDatasetDisposition disposition, const std::unordered_map< std::string, std::string > &args)
	Main method to open a seismic store dataset. More...

void	close ()
	Close an opened dataset. More...

void	flush ()
	Flushes the file metadata. More...

bool	exist () const
	Deprecated. More...

bool	exists () const
	Check if the dataset exists. More...

bool	isOpen ()
	Return the dataset opening state. More...

bool	objectExists (const std::string &blockName) const
	Checks if an object exists. More...

SDHierarchicalDatasetAccessor	getDatasetAccessor () const
	gets the hierarchical dataset accessor More...

void	mergeDatasetAccessor (const SDHierarchicalDatasetAccessor &dsaccessor)
	Merges the accessor metadata into the dataset. More...

void	lock ()
	Acquire a lockfile on the dataset. More...

void	waitLock ()
	Wait until lockfile on the dataset is removed. More...

void	unlock ()
	Release a lockfile on the dataset. More...

std::string	getConsistencyID ()
	Gets the locking session id. More...

int	getConsistencyCounter ()
	Gets process number that is locking the dataset. More...

void	setTags (const std::vector< std::string > &tags) const
	Add dataset tags. More...

std::vector< std::string >	getTags () const
	Get dataset tags. More...

std::string	getLegalTag () const
	Gets the legal-tag. More...

std::string	getCreatedDate () const
	Get dataset created date. More...

std::string	getType () const
	Gets the dataset type. More...

void	setSeismicMeta (const std::string &seismicmeta)
	Add seismic metadata to the dataset This method can be use to set the seismic metadata field in the dataset's JSON. More...

std::string	getSeismicMeta () const
	Get the seismic metadata from the dataset. More...

void	setHttpContext (const HttpContext *context=nullptr)
	Set connection parameters. More...

Friends
class	SDHierarchicalDatasetTest

Detailed Description

This class represents a hierarchical cloud storage dataset.

This is a dataset class to read/write cloud storage objects with a hierarchical structure.

Constructor & Destructor Documentation

◆ SDHierarchicalDataset() [1/4]

seismicdrive::SDHierarchicalDataset::SDHierarchicalDataset	(	SDManager *	sdmanager,
		const std::string &	sdfilename,
		const bool	log = `false`
	)

Constructor.

Parameters

sdmanager	is the seismic drive service manager
sdfilename	name of the dataset. It must be in the form /seimsic-drive/[tenant_name]/[subproject_name]/[path]/[dataset_name]. The field [path] rapresent the logic dataset hierarchy and it's optional. example sd://fatty-acid/sproj01/a/b/c/dataset_01
log	is a boolean value that enable the debug logger (false by default)

◆ SDHierarchicalDataset() [2/4]

seismicdrive::SDHierarchicalDataset::SDHierarchicalDataset	(	SDManager *	sdmanager,
		const std::string &	sdfilename,
		const std::string &	filetype,
		const bool	log = `false`
	)

Constructor with dataset type.

Parameters

sdmanager	is the seismic drive service manager
sdfilename	name of the dataset. It must be in the form /seimsic-drive/[tenant_name]/[subproject_name]/[path]/[dataset_name]. The field [path] rapresent the logic dataset hierarchy and it's optional. example sd://fatty-acid/sproj01/a/b/c/dataset_01
filetype	is the dataset type (ex. dio, bolt, txt, segy... )
log	is a boolean value that enable the debug logger (false by default)

◆ ~SDHierarchicalDataset()

seismicdrive::SDHierarchicalDataset::~SDHierarchicalDataset ( )

Destructor.

If the file is opened for writing (not READ_ONLY), it wll be closed.

◆ SDHierarchicalDataset() [3/4]

seismicdrive::SDHierarchicalDataset::SDHierarchicalDataset ( SDHierarchicalDataset && sdhd )

noexcept

Move Constructor.

This contructor moves the SDHierarchicalDataset object from the source to the new object and destroy the source object.

◆ SDHierarchicalDataset() [4/4]

seismicdrive::SDHierarchicalDataset::SDHierarchicalDataset ( const SDHierarchicalDataset & op )

Copy Constructor.

This contructor makes a copy of the SDHierarchicalDataset object from the source to the new object without modifying the source object.

Member Function Documentation

◆ close()

void seismicdrive::SDHierarchicalDataset::close ( )

Close an opened dataset.

This method closes a dataset and marks it as closed. File must be opened.

◆ exist()

bool seismicdrive::SDHierarchicalDataset::exist ( ) const

Deprecated.

Use exists() instead

◆ exists()

bool seismicdrive::SDHierarchicalDataset::exists ( ) const

Check if the dataset exists.

Checks if the dataset exists in cloud storage

Returns: true if the dataset exists

◆ flush()

void seismicdrive::SDHierarchicalDataset::flush ( )

Flushes the file metadata.

This method flushes the metadata of the object into the dataset.

◆ getConsistencyCounter()

int seismicdrive::SDHierarchicalDataset::getConsistencyCounter ( )

Gets process number that is locking the dataset.

This method is an accessor used to get the process number that is locking the current dataset.

Returns: the consistency process counter

◆ getConsistencyID()

std::string seismicdrive::SDHierarchicalDataset::getConsistencyID ( )

Gets the locking session id.

This method is an accessor used to get the locking session ID of the current dataset.

Returns: the locking id

◆ getCreatedDate()

std::string seismicdrive::SDHierarchicalDataset::getCreatedDate ( ) const

Get dataset created date.

This method can be used to retrieve the dataset creation date.

usage example:

SDManager sdmanager;
SDGenericDataset dataset(&sdmanager, "sd://tenant/subproject/data");
dataset.open(SDDatasetDisposition::READ_ONLY);
auto cdate = dataset.getCreatedDate();
dataset.close();

Returns: The dataset created date

◆ getDatasetAccessor()

SDHierarchicalDatasetAccessor seismicdrive::SDHierarchicalDataset::getDatasetAccessor ( ) const

gets the hierarchical dataset accessor

This method generates an accessor for the current hierarchical dataset.

Returns: the Hierarchical Dataset Accessor.

◆ getLegalTag()

std::string seismicdrive::SDHierarchicalDataset::getLegalTag ( ) const

Gets the legal-tag.

This method gets the legal-tag.

Returns: the legal-tag.

◆ getSeismicMeta()

std::string seismicdrive::SDHierarchicalDataset::getSeismicMeta ( ) const

Get the seismic metadata from the dataset.

This method can be used to retrieve the seismic metadata field from the dataset's JSON

usage example:

SDManager sdmanager; SDHierarchicalDataset dataset(&sdmanager, "sd://tenant/subproject/data"); dataset.open(SDDatasetDisposition::READ_ONLY); auto smeta = dataset.getSeismicMeta(); dataset.close()

Returns: the stringify json seismic metadata

◆ getTags()

std::vector<std::string> seismicdrive::SDHierarchicalDataset::getTags ( ) const

Get dataset tags.

This method can be used to retrieve the list of seismic store dataset tags.

usage example:

SDManager sdmanager;
SDGenericDataset dataset(&sdmanager, "sd://tenant/subproject/data");
dataset.open(SDDatasetDisposition::READ_ONLY);
auto tags = dataset.getTags();
dataset.close();

Returns: The list of seismic store dataset tags

◆ getType()

std::string seismicdrive::SDHierarchicalDataset::getType ( ) const

Gets the dataset type.

This method gets the dataset type.

Returns: the dataset type.

◆ isOpen()

bool seismicdrive::SDHierarchicalDataset::isOpen ( )

Return the dataset opening state.

This method return whether or not a dataset is already opened.

Returns: true if the dataset is opened, false if not

◆ lock()

void seismicdrive::SDHierarchicalDataset::lock ( )

Acquire a lockfile on the dataset.

This methos locks the current dataset.

◆ mergeDatasetAccessor()

void seismicdrive::SDHierarchicalDataset::mergeDatasetAccessor ( const SDHierarchicalDatasetAccessor & dsaccessor )

Merges the accessor metadata into the dataset.

This method merges the metadata of a Hierarchical Dataset Accessor into the Hierarchical Dataset.

Parameters

dsaccessor is the Hierarchical Dataset Accessor

◆ objectExists()

bool seismicdrive::SDHierarchicalDataset::objectExists ( const std::string & blockName ) const

Checks if an object exists.

Checks if an object exists in this dataset

Parameters

blockName is the name of the block (object)

Returns: true if the object exists

◆ open() [1/4]

void seismicdrive::SDHierarchicalDataset::open ( const SDDatasetDisposition disposition )

Opens the dataset whose name was specified the constructor.

This method is the most basic open function. It opens a dataset with a given open mode.

usage example:

SDManager sdmanager;
SDGenericDataset dataset(&sdmanager, "sd://tenant/subproject/data");
dataset.open(SDDatasetDisposition::CREATE);
dataset.close();

Parameters

disposition open mode

◆ open() [2/4]

void seismicdrive::SDHierarchicalDataset::open	(	const SDDatasetDisposition	disposition,
		const std::string &	legaltag
	)

Opens the dataset whose name was specified the constructor.

This method allows opening a dataset with a given open mode and legaltag.

usage example:

SDManager sdmanager;
SDGenericDataset dataset(&sdmanager, "sd://tenant/subproject/data");
dataset.open(SDDatasetDisposition::CREATE, { {api::json::Constants::kLegalTagLabel, "legal-tag"});
dataset.close();

Parameters

disposition	open mode
legaltag	the dataset legaltag

◆ open() [3/4]

void seismicdrive::SDHierarchicalDataset::open	(	const SDDatasetDisposition	disposition,
		const std::string &	legaltag,
		const std::string &	wid
	)

Opens the dataset whose name was specified the constructor.

This method allows opening a dataset with a given open mode and legaltag that has already been locked for writing.

usage example:

SDManager sdmanager;
SDGenericDataset dataset(&sdmanager, "sd://tenant/subproject/data");
dataset.open(SDDatasetDisposition::CREATE, { {api::json::Constants::kLegalTagLabel, "legal-tag"}, {api::json::Constants::kWriteIdLabel, "wid"});
dataset.close();

Parameters

disposition	open mode
wid	is the write opening id. This must be specified if you wnat to open a dataset already locked for write.
legaltag	the dataset legaltag

◆ open() [4/4]

void seismicdrive::SDHierarchicalDataset::open	(	const SDDatasetDisposition	disposition,
		const std::unordered_map< std::string, std::string > &	args
	)

Main method to open a seismic store dataset.

This method opens a seismic data dataset in seismic store.

The method support different opening dispotion modes:

CREATE: create a dataset and open it for write
OVERWRITE: create a dataset and open it for write overriding the existing one.
READ_WRITE: open an existing dataset for read and write.
READ_ONLY: open an existing dataset for read only.

The method accepts these optional opening arguments (defined in Constants.h);

kLegalTagLabel: the legal tag,
kWriteIdLabel: the open write session id
kDDMSSeismicMetadataLabel: the seismic storage metadata.
kFlushManifest: define if the file-metadata* (manifest) have be saved along with the dataset metadata. = "enable" -> the file-metadata, if exist, will be saved. = "disable" -> the file-metadata, if exist, will be saved.
KPedantic: define if enable or disable the pedantic mode to automatically flush the file-metadata in a locking session. = "enable" -> the file-metadata, if exist, will be saved. = "disable" -> the file-metadata, if exist, will be saved ONLY if the kWriteIdLabel is not set (not in a locking session)

The file-metadata, in the SDHierarchicalDataset accessor, is an object metadata associated to the dataset. It contains information on nodes, number of objects that compose the node and the total size of the node in bytes: file-metadata = { type: "HIERARCHICAL", nodes: { name: <node_name> nobjects: <number_of_node_objects> size: <dataset_size> } }

These properties are collected at every block written and saved during the close operation. In the case of a "Master" node non pedantic (see table below) the file-metadata is not collected at every block written, but it is computed during the close by iterating on the saved object on the storage (high costs)

The file-metadata can be useful if the dataset is saved with a hierarchical structure (like a normal filesystem) These informations are saved during the close dataset operataion. If is not required, is advisable to disable the file-metadata saving. This will immprove performance by reducing the latencies while closing datasets.

The optional argument KFlushManifest can be tuned based on the consumer workflow to disable the file-metadata saving. Below reported all possible combination with: saved = YES (save the file metadata) NO (do not save the file metadata) cost = number of call to perform to the SDMS service during the close

|-kFlushManifest-|-saved-|-cost-| |-ENABLE------—|-YES—|-1-—| |-DISABLE-----—|-NO-—|-0— |

usage example:

SDManager sdmanager;
SDGenericDataset dataset(&sdmanager, "sd://tenant/subproject/data");
dataset.open(SDDatasetDisposition::CREATE, { {api::json::Constants::kLegalTagLabel, "legal-tag"}, {api::json::Constants::kWriteIdLabel, "wid"}, {api::json::Constants::kDDMSSeismicMetadataLabel, "{\"data":"value"}"} {api::json::Constants::kPedantic, api::json::Constants::KEnableProperty || api::json::Constants::KDisableProperty} }); {api::json::Constants::kFlushManifest, api::json::Constants::KEnableProperty || api::json::Constants::KDisableProperty} });
dataset.close();

Parameters

disposition	The opening disposition open mode
args	The list of optional arguments

◆ operator=() [1/2]

SDHierarchicalDataset& seismicdrive::SDHierarchicalDataset::operator= ( SDHierarchicalDataset && sdhd )

noexcept

Move Assignment Operator.

Same functionality as the move constructor.

◆ operator=() [2/2]

SDHierarchicalDataset& seismicdrive::SDHierarchicalDataset::operator= ( const SDHierarchicalDataset & op )

Copy Assignment Operator.

Same functionality as the copy constructor.

◆ setHttpContext()

void seismicdrive::SDHierarchicalDataset::setHttpContext ( const HttpContext * context = nullptr )

Set connection parameters.

This method sets connection parameters for the underlying HTTP connection to the server.

Parameters

context The connection parameters to be used for this dataset

usage example:

HttpContext context; context.timeoutSecs = 3600; context.tcpKeepAlive = true;

SDManager sdmanager; SDHierarchicalDataset dataset(&sdmanager, "sd://tenant/subproject/data"); dataset.setHttpContext(&context); dataset.open(SDDatasetDisposition::READ_ONLY); auto smeta = dataset.getSeismicMeta(); dataset.close()

◆ setSeismicMeta()

void seismicdrive::SDHierarchicalDataset::setSeismicMeta ( const std::string & seismicmeta )

Add seismic metadata to the dataset This method can be use to set the seismic metadata field in the dataset's JSON.

usage example:

SDManager sdmanager; SDHierarchicalDataset dataset(&sdmanager, "sd://tenant/subproject/data"); dataset.open(SDDatasetDisposition::READ_WRITE); dataset.setSeismicMeta("{\"description":""}"); dataset.close();

Parameters

seismicmeta is the json stringify seismic metadata. The schema for seismic meta can be found at https://developer.delfi.slb.com/solutions/dataecosystem/tutorials/storageservice

Exceptions

error::dataset::Error Invalid JSON or seismic meta field does not exist after patch request

◆ setTags()

void seismicdrive::SDHierarchicalDataset::setTags ( const std::vector< std::string > & tags ) const

Add dataset tags.

This method can be used to set a list of tags to a seismic store dataset. Tags are just simple strings (not key/value pairs).

usage example:

SDManager sdmanager;
SDGenericDataset dataset(&sdmanager, "sd://tenant/subproject/data");
dataset.open(SDDatasetDisposition::READ_WRITE);
dataset.setTags({"TestTag01", "TestTag02", "TestTag03"});
dataset.close();

Parameters

tags	The list of tags to add to the opened seismic store dataset

◆ unlock()

void seismicdrive::SDHierarchicalDataset::unlock ( )

Release a lockfile on the dataset.

This method removes the lock from the current dataset.

◆ waitLock()

void seismicdrive::SDHierarchicalDataset::waitLock ( )

Wait until lockfile on the dataset is removed.

This method waits until the current dataset is not locked by another process.

Friends And Related Function Documentation

◆ SDHierarchicalDatasetTest

friend class SDHierarchicalDatasetTest

friend

The documentation for this class was generated from the following file:

/osdu-sdapi-cpp/src/src/core/SDHierarchicalDataset.h

Public Member Functions

Friends

Detailed Description

Constructor & Destructor Documentation

◆ SDHierarchicalDataset() [1/4]

◆ SDHierarchicalDataset() [2/4]

◆ ~SDHierarchicalDataset()

◆ SDHierarchicalDataset() [3/4]

◆ SDHierarchicalDataset() [4/4]

Member Function Documentation

◆ close()

◆ exist()

◆ exists()

◆ flush()

◆ getConsistencyCounter()

◆ getConsistencyID()

◆ getCreatedDate()

◆ getDatasetAccessor()

◆ getLegalTag()

◆ getSeismicMeta()

◆ getTags()

◆ getType()

◆ isOpen()

◆ lock()

◆ mergeDatasetAccessor()

◆ objectExists()

◆ open() [1/4]

◆ open() [2/4]

◆ open() [3/4]

◆ open() [4/4]

◆ operator=() [1/2]

◆ operator=() [2/2]

◆ setHttpContext()

◆ setSeismicMeta()

◆ setTags()

◆ unlock()

◆ waitLock()

Friends And Related Function Documentation

◆ SDHierarchicalDatasetTest