The Data Mastering Configuration APIs allow the user to create/update the Match and Merge Rules configurations required by Data Mastering workflow. Data Mastering Service considers these configurations to decide the source priorities, the source record kind and unique attribute to match based on conditions while merging the Master record.
More details on these configurations are mentioned in subsequent sections.
The Data Mastering configuration has the following endpoints:
- PUT /data-mastering/v1/match-rules: Creates/Updates the Match Rules configuration.
- GET /data-mastering/v1/match-rules/{kind}/{source}: Fetch the Match Rules configuration based on the source and kind.
- PUT /data-mastering/v1/merge-rules: Creates/Updates the Merge Rules configuration.
- GET /data-mastering/v1/merge-rules/{kind}: Fetch the Merge Rules configuration based on kind.
You can create/update the Match Rules configuration using the PUT /data-mastering/v1/match-rules endpoint. The following parameters are used to create this configuration:
| Attribute | Mandatory/Optional | Datatype | Description | Validation |
|---|---|---|---|---|
| kind | Mandatory | String | This is the wks kind specifying only major version; i.e., the source kind of the data. | As per current scope, creation of Match Rules is allowed only for "master-data" group type entities(ex: osdu:wks:master-data--Field:1.., osdu:wks:master-data--Basin:1.. etc). Also, Kind is a case sensitive attribute and the format of kind should be authority:source:entity-type:majorversion... |
| source | Mandatory | String | This defines the source of the wks record from where it is derived/contributed. | source is a mandatory and case sensitive attribute. "system" is reserved as a value for default Match Rules. The maximum length of the source attribute value allowed is 25 characters. |
| matchRules | Mandatory | List<MatchRule> | This will contain a list of match rules as per the raw data source. | matchRules cannot be empty. |
| matchKey | Mandatory | String | It is used to search the existing Master record based on andCondition/orCondition(if given). | matchKey is a mandatory attribute. Here, [] denotes array operation. Allowed operation is [#]; which denotes any index of the array. '#' is evaluated using a specified condition mentioned in "andCondition"/"orCondition". |
| andCondition/orCondition | Optional | Array | Conditions on which AND/OR operation is to be performed. | Either andCondition or orCondition can be used with same matchKey. |
| type | Optional | String | Type of condition. | Allowed values are equals and contains. |
| attribute | Optional | String | Attribute for which the condition is applicable. | If matchKey is holding nested structure as per schema(ex: NameAliases, VerticalMeasurements etc) then attribute should be of same nested structure. |
| value | Optional | String | Value of the attribute. |
Following is the sample Match Rule with "andCondition".
Sample Payload to Create/update Match Rules with "andCondition"
{
"kind": "osdu:wks:master-data--Wellbore:1.*.*",
"source": "Source-A",
"matchRules": [
{
"priority": 1,
"matchKeys": [
{
"matchKey": "data.FacilityName"
},
{
"matchKey": "data.NameAliases[#].AliasName",
"andCondition": [
{
"type": "equals",
"attribute": "data.NameAliases[#].AliasNameTypeID",
"value": "{datapartition}:reference-data--AliasNameType:UniqueIdentifier:"
},
{
"type": "contains",
"attribute": "data.NameAliases[#].DefinitionOrganisationID",
"value": "SLB_EXPLORATION_LTD"
}
]
}
]
},
{
"priority": 2,
"matchKeys": [
{
"matchKey": "data.FacilityID"
}
]
},
{
"priority": 3,
"matchKeys": [
{
"matchKey": "data.SpatialLocation.AsIngestedCoordinates.features[#].geometry.coordinates"
}
]
}
]
}
Here, the set of match keys (data.FacilityName and data.NameAliases[#].AliasName) will be considered as first priority to uniquely identify the existing data master record.
AliasName is selected where ‘AliasNameTypeID’ is equal to ‘{datapartition}:reference-data--AliasNameType:UniqueIdentifier:’ AND ‘DefinitionOrganisationID’ contains ‘SLB_EXPLORATION_LTD’.
If there are multiple Data Master records found with this combination of MatchKeys(priority 1) then combination of MatchKeys present in priority 2 will be considered along with priority 1.
Hence,at priority 2, data.FacilityName, data.NameAliases[#].AliasName and data.FacilityID will be considered to uniquely identify the existing data master record.
Similarly,at priority 3, data.FacilityName, data.NameAliases[#].AliasName, data.FacilityID and data.SpatialLocation.AsIngestedCoordinates.features[#].geometry.coordinates will be considered to uniquely identify the existing data master record.
For example,"FacilityName": "SLB-A-101" and "AliasName": "API-101" from following sample WKS record will be considered by as first priority by Data Mastering service to search existing master record.
Sample WKS record under process
{
"data": {
"NameAliases": [
{
"AliasName": "API-101",
"AliasNameTypeID": "{datapartition}:reference-data--AliasNameType:UniqueIdentifier:",
"DefinitionOrganisationID": "SLB_EXPLORATION_LTD",
"EffectiveDateTime": "2021-02-13T09:13:15.55Z",
"TerminationDateTime": "2023-03-13T09:13:15.55Z"
},
{
"AliasName": "SLB-Regulatory-101",
"AliasNameTypeID": "{datapartition}:reference-data--AliasNameType:RegulatoryIdentifier:",
"DefinitionOrganisationID": "{datapartition}:master-data--Organisation:c5a035f54c7646449fe1a87c9606de3e:",
"EffectiveDateTime": "2021-02-13T09:13:15.55Z",
"TerminationDateTime": "2023-03-13T09:13:15.55Z"
}
],
"FacilityName": "SLB-A-101",
"FacilitySpecifications": [
{
"FacilitySpecificationText": "Example FacilitySpecificationText",
"ParameterTypeID": "{datapartition}:reference-data--ParameterType:SlotName",
"EffectiveDateTime": "2022-01-15T09:13:15.55Z",
"FacilitySpecificationDateTime": "2022-03-22T09:13:15.55Z",
"FacilitySpecificationIndicator": "False"
},
{
"FacilitySpecificationText": "Example FacilitySpecificationText",
"ParameterTypeID": "{datapartition}:reference-data--ParameterType:PermitNumber",
"TerminationDateTime": "2022-02-15T09:13:15.55Z",
"EffectiveDateTime": "2021-01-10T09:13:15.55Z",
"FacilitySpecificationDateTime": "2021-01-10T09:13:15.55Z",
"FacilitySpecificationIndicator": "False"
}
],
"FacilityStates": [
{
"EffectiveDateTime": "2021-02-13T08:13:15.55Z",
"TerminationDateTime": "2021-09-10T10:15:15.55Z",
"FacilityStateTypeID": "{datapartition}:reference-data--FacilityStateType:Planning"
}
],
"FacilityEvents": [
{
"FacilityEventTypeID": "{datapartition}:reference-data--FacilityEventType:WorkoverFinish",
"EffectiveDateTime": "2021-08-25T11:45:25.45Z"
}
],
"FacilityOperators": [
{
"FacilityOperatorID": "SLB-Trusted-Operator",
"FacilityOperatorOrganisationID": "{datapartition}:master-data--Organisation:48428bc276a94fbab522c3e8dac2d11a",
"EffectiveDateTime": "2021-02-13T09:13:15.55Z",
"TerminationDateTime": "2022-03-25T09:13:15.55Z"
}
],
"SpatialLocation": {
"AsIngestedCoordinates": {
"CoordinateReferenceSystemID": "namespace:reference-data--CoordinateReferenceSystem:BoundCRS.SLB.32021.15852:",
"features": [
{
"geometry": {
"coordinates": [
-73.771724,
42.516771
],
"type": "AnyCrsPoint"
},
"type": "AnyCrsFeature",
"properties": {}
}
],
"persistableReferenceCrs": "{\\lateBoundCRS\\\":{\\\"wkt\\\":\\\"PROJCS[\\\\\\\"NAD_1927_StatePlane_North_Dakota_South_FIPS_3302\\\\\\GEOGCS[\\\\\\\"GCS_North_American_1927\\\\\\DATUM[\\\\\\\"D_North_American_1927\\\\\\SPHEROID[\\\\\\\"Clarke_1866\\\\\\6378206.4,294.9786982]],PRIMEM[\\\\\\\"Greenwich\\\\\\0.0],UNIT[\\\\\\\"Degree\\\\\\0.0174532925199433]],PROJECTION[\\\\\\\"Lambert_Conformal_Conic\\\\\\\"],PARAMETER[\\\\\\\"False_Easting\\\\\\2000000.0],PARAMETER[\\\\\\\"False_Northing\\\\\\0.0],PARAMETER[\\\\\\\"Central_Meridian\\\\\\-100.5],PARAMETER[\\\\\\\"Standard_Parallel_1\\\\\\46.1833333333333],PARAMETER[\\\\\\\"Standard_Parallel_2\\\\\\47.4833333333333],PARAMETER[\\\\\\\"Latitude_Of_Origin\\\\\\45.6666666666667],UNIT[\\\\\\\"Foot_US\\\\\\0.304800609601219],AUTHORITY[\\\\\\\"EPSG\\\\\\32021]]\\\\\"ver\\\":\\\"PE_10_3_1\\\\\"name\\\":\\\"NAD_1927_StatePlane_North_Dakota_South_FIPS_3302\\\\\"authCode\\\":{\\\"auth\\\":\\\"EPSG\\\\\"code\\\":\\\"32021\\\"},\\\"type\\\":\\\"LBC\\\"},\\\"singleCT\\\":{\\\"wkt\\\":\\\"GEOGTRAN[\\\\\\\"NAD_1927_To_WGS_1984_79_CONUS\\\\\\GEOGCS[\\\\\\\"GCS_North_American_1927\\\\\\DATUM[\\\\\\\"D_North_American_1927\\\\\\SPHEROID[\\\\\\\"Clarke_1866\\\\\\6378206.4,294.9786982]],PRIMEM[\\\\\\\"Greenwich\\\\\\0.0],UNIT[\\\\\\\"Degree\\\\\\0.0174532925199433]],GEOGCS[\\\\\\\"GCS_WGS_1984\\\\\\DATUM[\\\\\\\"D_WGS_1984\\\\\\SPHEROID[\\\\\\\"WGS_1984\\\\\\6378137.0,298.257223563]],PRIMEM[\\\\\\\"Greenwich\\\\\\0.0],UNIT[\\\\\\\"Degree\\\\\\0.0174532925199433]],METHOD[\\\\\\\"NADCON\\\\\\\"],PARAMETER[\\\\\\\"Dataset_conus\\\\\\0.0],AUTHORITY[\\\\\\\"EPSG\\\\\\15851]]\\\\\"ver\\\":\\\"PE_10_3_1\\\\\"name\\\":\\\"NAD_1927_To_WGS_1984_79_CONUS\\\\\"authCode\\\":{\\\"auth\\\":\\\"EPSG\\\\\"code\\\":\\\"15851\\\"},\\\"type\\\":\\\"ST\\\"},\\\"ver\\\":\\\"PE_10_3_1\\\\\"name\\\":\\\"NAD27 * OGP-Usa Conus / North Dakota South [32021,15851]\\\\\"authCode\\\":{\\\"auth\\\":\\\"SLB\\\\\"code\\\":\\\"32021079\\\"},\\\"type\\\":\\\"EBC\\\"}\"",
"persistableReferenceUnitZ": "{\\scaleOffset\\\":{\\\"scale\\\":1.0,\\\"offset\\\":0.0},\\\"symbol\\\":\\\"m\\\\\"baseMeasurement\\\":{\\\"ancestry\\\":\\\"Length\\\\\"type\\\":\\\"UM\\\"},\\\"type\\\":\\\"USO\\\"}\"",
"persistableReferenceVerticalCrs": "{\\authCode\\\":{\\\"auth\\\":\\\"EPSG\\\\\"code\\\":\\\"5773\\\"},\\\"type\\\":\\\"LBC\\\\\"ver\\\":\\\"PE_10_3_1\\\\\"name\\\":\\\"EGM96_Geoid\\\\\"wkt\\\":\\\"VERTCS[\\\\\\\"EGM96_Geoid\\\\\\VDATUM[\\\\\\\"EGM96_Geoid\\\\\\\"],PARAMETER[\\\\\\\"Vertical_Shift\\\\\\0.0],PARAMETER[\\\\\\\"Direction\\\\\\1.0],UNIT[\\\\\\\"Meter\\\\\\1.0],AUTHORITY[\\\\\\\"EPSG\\\\\\5773]]\\\"}\"",
"type": "AnyCrsFeatureCollection",
"VerticalCoordinateReferenceSystemID": "namespace:reference-data--CoordinateReferenceSystem:VerticalCRS.EPSG.5773:"
},
"CoordinateQualityCheckPerformedBy": "Example CoordinateQualityCheckPerformedBy",
"CoordinateQualityCheckDateTime": "2020-02-13T09:13:15.55Z",
"SpatialLocationCoordinatesDate": "2020-02-13T09:13:15.55Z"
},
"Source": "Source-A",
"FacilityID": "SLB-Trusted-External-Facility-A",
"VerticalMeasurements": [
{
"VerticalMeasurement": 1001.54,
"VerticalMeasurementTypeID": "{datapartition}:reference-data--VerticalMeasurementType:KB",
"EffectiveDateTime": "2021-02-13T09:13:15.55Z"
},
{
"VerticalMeasurement": 2001.54,
"VerticalMeasurementTypeID": "{datapartition}:reference-data--VerticalMeasurementType:RB",
"EffectiveDateTime": "2021-02-13T09:13:15.55Z"
}
],
"KickOffWellbore": "{datapartition}:master-data--Wellbore:8cf5990a5f434cd189212b1fbf66c513:"
},
"meta": [
{
"kind": "Unit",
"name": "m",
"persistableReference": "{\"scaleOffset\":{\"scale\":1.0,\"offset\":0.0},\"symbol\":\"m\",\"baseMeasurement\":{\"ancestry\":\"L\",\"type\":\"UM\"},\"type\":\"USO\"}",
"unitOfMeasureID": "{datapartition}:reference-data--UnitOfMeasure:ft",
"propertyNames": [
"VerticalMeasurements[0].VerticalMeasurement",
"VerticalMeasurements[1].VerticalMeasurement"
]
}
],
"id": "{datapartition}:master-data--Wellbore:33655890e0054c4991f12fd3c1edd089",
"version": 1648625779422061,
"kind": "osdu:wks:master-data--Wellbore:1.1.0",
"acl": {
"viewers": [
"<acl_viewers>"
],
"owners": [
"<acl_owners>"
]
},
"legal": {
"legaltags": [
"<legal_tags>"
],
"otherRelevantDataCountries": [
"US"
],
"status": "compliant"
}
}
Following is the sample Match Rule with "orCondition".
Sample Payload to Create/update Match Rules with "orCondition"
{
"kind": "osdu:wks:master-data--Wellbore:1.*.*",
"source": "Source-A",
"matchRules": [
{
"priority": 1,
"matchKeys": [
{
"matchKey": "data.NameAliases[#].AliasName",
"orCondition": [
{
"type": "equals",
"attribute": "data.NameAliases[#].AliasNameTypeID",
"value": "{datapartition}:reference-data--AliasNameType:UniqueIdentifier:"
},
{
"type": "equals",
"attribute": "data.NameAliases[#].AliasNameTypeID",
"value": "{datapartition}:reference-data--AliasNameType:WellboreNumber:"
}
]
}
]
},
{
"priority": 2,
"matchKeys": [
{
"matchKey": "data.FacilityID"
}
]
}
]
}
As seen in the above match rule, first match key ‘NameAliases[#].AliasName’ is selected when either ‘AliasNameTypeID’ is equal to ‘{datapartition}:reference-data--AliasNameType:UniqueIdentifier:’ OR ‘AliasNameTypeID’ is equal to ‘{datapartition}:reference-data--AliasNameType:WellboreNumber:’. Second match key is ‘FacilityID’ at priority-2.
For example, "AliasName": "API-101" from following sample WKS record will be considered by Data Mastering service as first priority to search existing master record. If user does not find unique WKE with "AliasName": "API-101", mastering will try to find unique WKE with "AliasName": "API-101" and "FacilityID": "SLB-Trusted-External-Facility-A"
Sample WKS record under process
{
"data": {
"NameAliases": [
{
"AliasName": "WB-101",
"AliasNameTypeID": "{datapartition}:reference-data--AliasNameType:WellboreNumber:",
"DefinitionOrganisationID": "SLB_EXPLORATION_LTD",
"EffectiveDateTime": "2021-02-13T09:13:15.55Z",
"TerminationDateTime": "2023-03-13T09:13:15.55Z"
},
{
"AliasName": "Regulatory Identifier",
"AliasNameTypeID": "{datapartition}:reference-data--AliasNameType:RegulatoryIdentifier:",
"DefinitionOrganisationID": "{datapartition}:master-data--Organisation:c5a035f54c7646449fe1a87c9606de3e:",
"EffectiveDateTime": "2021-02-13T09:13:15.55Z",
"TerminationDateTime": "2023-03-13T09:13:15.55Z"
}
],
"FacilityName": "SLB-A-101",
"FacilitySpecifications": [
{
"FacilitySpecificationText": "Example FacilitySpecificationText",
"ParameterTypeID": "{datapartition}:reference-data--ParameterType:SlotName",
"EffectiveDateTime": "2022-01-15T09:13:15.55Z",
"FacilitySpecificationDateTime": "2022-03-22T09:13:15.55Z",
"FacilitySpecificationIndicator": "False"
},
{
"FacilitySpecificationText": "Example FacilitySpecificationText",
"ParameterTypeID": "{datapartition}:reference-data--ParameterType:PermitNumber",
"TerminationDateTime": "2022-02-15T09:13:15.55Z",
"EffectiveDateTime": "2021-01-10T09:13:15.55Z",
"FacilitySpecificationDateTime": "2021-01-10T09:13:15.55Z",
"FacilitySpecificationIndicator": "False"
}
],
"FacilityStates": [
{
"EffectiveDateTime": "2021-02-13T08:13:15.55Z",
"TerminationDateTime": "2021-09-10T10:15:15.55Z",
"FacilityStateTypeID": "{datapartition}:reference-data--FacilityStateType:Planning"
}
],
"FacilityEvents": [
{
"FacilityEventTypeID": "{datapartition}:reference-data--FacilityEventType:WorkoverFinish",
"EffectiveDateTime": "2021-08-25T11:45:25.45Z"
}
],
"FacilityOperators": [
{
"FacilityOperatorID": "SLB-Trusted-Operator",
"FacilityOperatorOrganisationID": "{datapartition}:master-data--Organisation:48428bc276a94fbab522c3e8dac2d11a",
"EffectiveDateTime": "2021-02-13T09:13:15.55Z",
"TerminationDateTime": "2022-03-25T09:13:15.55Z"
}
],
"SpatialLocation": {
"AsIngestedCoordinates": {
"CoordinateReferenceSystemID": "namespace:reference-data--CoordinateReferenceSystem:BoundCRS.SLB.32021.15852:",
"features": [
{
"geometry": {
"coordinates": [
-73.771724,
42.516771
],
"type": "AnyCrsPoint"
},
"type": "AnyCrsFeature",
"properties": {}
}
],
"persistableReferenceCrs": "{\\lateBoundCRS\\\":{\\\"wkt\\\":\\\"PROJCS[\\\\\\\"NAD_1927_StatePlane_North_Dakota_South_FIPS_3302\\\\\\GEOGCS[\\\\\\\"GCS_North_American_1927\\\\\\DATUM[\\\\\\\"D_North_American_1927\\\\\\SPHEROID[\\\\\\\"Clarke_1866\\\\\\6378206.4,294.9786982]],PRIMEM[\\\\\\\"Greenwich\\\\\\0.0],UNIT[\\\\\\\"Degree\\\\\\0.0174532925199433]],PROJECTION[\\\\\\\"Lambert_Conformal_Conic\\\\\\\"],PARAMETER[\\\\\\\"False_Easting\\\\\\2000000.0],PARAMETER[\\\\\\\"False_Northing\\\\\\0.0],PARAMETER[\\\\\\\"Central_Meridian\\\\\\-100.5],PARAMETER[\\\\\\\"Standard_Parallel_1\\\\\\46.1833333333333],PARAMETER[\\\\\\\"Standard_Parallel_2\\\\\\47.4833333333333],PARAMETER[\\\\\\\"Latitude_Of_Origin\\\\\\45.6666666666667],UNIT[\\\\\\\"Foot_US\\\\\\0.304800609601219],AUTHORITY[\\\\\\\"EPSG\\\\\\32021]]\\\\\"ver\\\":\\\"PE_10_3_1\\\\\"name\\\":\\\"NAD_1927_StatePlane_North_Dakota_South_FIPS_3302\\\\\"authCode\\\":{\\\"auth\\\":\\\"EPSG\\\\\"code\\\":\\\"32021\\\"},\\\"type\\\":\\\"LBC\\\"},\\\"singleCT\\\":{\\\"wkt\\\":\\\"GEOGTRAN[\\\\\\\"NAD_1927_To_WGS_1984_79_CONUS\\\\\\GEOGCS[\\\\\\\"GCS_North_American_1927\\\\\\DATUM[\\\\\\\"D_North_American_1927\\\\\\SPHEROID[\\\\\\\"Clarke_1866\\\\\\6378206.4,294.9786982]],PRIMEM[\\\\\\\"Greenwich\\\\\\0.0],UNIT[\\\\\\\"Degree\\\\\\0.0174532925199433]],GEOGCS[\\\\\\\"GCS_WGS_1984\\\\\\DATUM[\\\\\\\"D_WGS_1984\\\\\\SPHEROID[\\\\\\\"WGS_1984\\\\\\6378137.0,298.257223563]],PRIMEM[\\\\\\\"Greenwich\\\\\\0.0],UNIT[\\\\\\\"Degree\\\\\\0.0174532925199433]],METHOD[\\\\\\\"NADCON\\\\\\\"],PARAMETER[\\\\\\\"Dataset_conus\\\\\\0.0],AUTHORITY[\\\\\\\"EPSG\\\\\\15851]]\\\\\"ver\\\":\\\"PE_10_3_1\\\\\"name\\\":\\\"NAD_1927_To_WGS_1984_79_CONUS\\\\\"authCode\\\":{\\\"auth\\\":\\\"EPSG\\\\\"code\\\":\\\"15851\\\"},\\\"type\\\":\\\"ST\\\"},\\\"ver\\\":\\\"PE_10_3_1\\\\\"name\\\":\\\"NAD27 * OGP-Usa Conus / North Dakota South [32021,15851]\\\\\"authCode\\\":{\\\"auth\\\":\\\"SLB\\\\\"code\\\":\\\"32021079\\\"},\\\"type\\\":\\\"EBC\\\"}\"",
"persistableReferenceUnitZ": "{\\scaleOffset\\\":{\\\"scale\\\":1.0,\\\"offset\\\":0.0},\\\"symbol\\\":\\\"m\\\\\"baseMeasurement\\\":{\\\"ancestry\\\":\\\"Length\\\\\"type\\\":\\\"UM\\\"},\\\"type\\\":\\\"USO\\\"}\"",
"persistableReferenceVerticalCrs": "{\\authCode\\\":{\\\"auth\\\":\\\"EPSG\\\\\"code\\\":\\\"5773\\\"},\\\"type\\\":\\\"LBC\\\\\"ver\\\":\\\"PE_10_3_1\\\\\"name\\\":\\\"EGM96_Geoid\\\\\"wkt\\\":\\\"VERTCS[\\\\\\\"EGM96_Geoid\\\\\\VDATUM[\\\\\\\"EGM96_Geoid\\\\\\\"],PARAMETER[\\\\\\\"Vertical_Shift\\\\\\0.0],PARAMETER[\\\\\\\"Direction\\\\\\1.0],UNIT[\\\\\\\"Meter\\\\\\1.0],AUTHORITY[\\\\\\\"EPSG\\\\\\5773]]\\\"}\"",
"type": "AnyCrsFeatureCollection",
"VerticalCoordinateReferenceSystemID": "namespace:reference-data--CoordinateReferenceSystem:VerticalCRS.EPSG.5773:"
},
"CoordinateQualityCheckPerformedBy": "Example CoordinateQualityCheckPerformedBy",
"CoordinateQualityCheckDateTime": "2020-02-13T09:13:15.55Z",
"SpatialLocationCoordinatesDate": "2020-02-13T09:13:15.55Z"
},
"Source": "Source-A",
"FacilityID": "SLB-Trusted-External-Facility-A",
"VerticalMeasurements": [
{
"VerticalMeasurement": 1001.54,
"VerticalMeasurementTypeID": "{datapartition}:reference-data--VerticalMeasurementType:KB",
"EffectiveDateTime": "2021-02-13T09:13:15.55Z"
},
{
"VerticalMeasurement": 2001.54,
"VerticalMeasurementTypeID": "{datapartition}:reference-data--VerticalMeasurementType:RB",
"EffectiveDateTime": "2021-02-13T09:13:15.55Z"
}
],
"KickOffWellbore": "{datapartition}:master-data--Wellbore:8cf5990a5f434cd189212b1fbf66c513:"
},
"meta": [
{
"kind": "Unit",
"name": "m",
"persistableReference": "{\"scaleOffset\":{\"scale\":1.0,\"offset\":0.0},\"symbol\":\"m\",\"baseMeasurement\":{\"ancestry\":\"L\",\"type\":\"UM\"},\"type\":\"USO\"}",
"unitOfMeasureID": "{datapartition}:reference-data--UnitOfMeasure:ft",
"propertyNames": [
"VerticalMeasurements[0].VerticalMeasurement",
"VerticalMeasurements[1].VerticalMeasurement"
]
}
],
"id": "{datapartition}:master-data--Wellbore:33655890e0054c4991f12fd3c1edd089",
"version": 1648625779422061,
"kind": "osdu:wks:master-data--Wellbore:1.1.0",
"acl": {
"viewers": [
"<acl_viewers>"
],
"owners": [
"<acl_owners>"
]
},
"legal": {
"legaltags": [
"<legal_tags>"
],
"otherRelevantDataCountries": [
"US"
],
"status": "compliant"
}
}
To enable Data Mastering workflow when user defined Match Rules is not present in the system, Data Mastering service bootstraps default/system match rules. Data Mastering service considers this default match rules until user defined Match Rules for respective kind and source is not defined.
Following are some samples of Default Match Rules which are being bootstrapped -
Sample Default Match Rules for Wellbore Entity Type
{
"kind": "osdu:wks:master-data--Wellbore:1.*.*",
"source": "system",
"matchRules": [
{
"priority": 1,
"matchKeys": [
{
"matchKey": "data.FacilityID"
}
]
}
]
}
Sample Default Match Rules for Field Entity Type
{
"kind": "osdu:wks:master-data--Field:1.*.*",
"source": "system",
"matchRules": [
{
"priority": 1,
"matchKeys": [
{
"matchKey": "data.FieldID"
}
]
}
]
}
Sample Default Match Rules for Basin Entity Type
{
"kind": "osdu:wks:master-data--Basin:1.*.*",
"source": "system",
"matchRules": [
{
"priority": 1,
"matchKeys": [
{
"matchKey": "data.BasinID"
}
]
}
]
}
Things to consider while creating Match Rules:
- Create/update Match Rules for all the Sources which are contributing to Master record and are part of "priorities" in Merge Rules.
- Create/update Match Rules with only those attributes which are part of OSDU Schema of respective kind(i.e osdu:wks:master-data--Wellbore:1.0.0, osdu:wks:master-data--Field:1.0.0).
- Do not create Match Rules with any attributes which are part of "ExtensionProperties" of the record. These attributes are not searchable via Search Service.
- Here, operation between two matchKey is always AND.
You can fetch the Match Rules configuration based on source and kind using the GET /data-mastering/v1/match-rules/{kind}/{source} endpoint. The following parameters are used to fetch the related data:
| Attribute | Datatype | Description | Validation |
|---|---|---|---|
| kind | String | This is the wks kind; i.e. the source kind of the data. | Kind is a case sensitive attribute. The format of kind should be authority:source:entity-type:version. |
| source | String | This defines the source for which Match Template is created. | source is a case sensitive attribute. |
Sample Response
{
"kind": "osdu:wks:master-data--Wellbore:1.*.*",
"source": "Source-A",
"matchRules": [
{
"priority": 1,
"matchKeys": [
{
"matchKey": "data.FacilityName"
},
{
"matchKey": "data.NameAliases[#].AliasName",
"andCondition": [
{
"type": "equals",
"attribute": "data.NameAliases[#].AliasNameTypeID",
"value": "{datapartition}:reference-data--AliasNameType:UniqueIdentifier:"
},
{
"type": "contains",
"attribute": "data.NameAliases[#].DefinitionOrganisationID",
"value": "SLB_EXPLORATION_LTD"
}
]
}
]
},
{
"priority": 2,
"matchKeys": [
{
"matchKey": "data.FacilityID"
}
]
},
{
"priority": 3,
"matchKeys": [
{
"matchKey": "data.SpatialLocation.AsIngestedCoordinates.features[#].geometry.coordinates"
}
]
}
]
}
You can create/update the Merge Rules configuration using the PUT /data-mastering/v1/merge-rules endpoint. The following parameters are used to create this configuration:
| Attribute | Datatype | Description | Validation |
|---|---|---|---|
| kind | String | This is the wks kind specifying only major version; i.e., the source kind of the data. | Same as Match Rules, creation of Merge Rules is also allowed only for "master-data" group type entities(ex: osdu:wks:master-data--Field:1.*.*, osdu:wks:master-data--Basin:1.*.* etc). Also, Kind is a case sensitive attribute and the format of kind should be authority:source:entity-type:majorversion... |
| mergeRules | Array | List of mergerules which contains attributes and rules. | mergeRules should not be empty. |
| attributes | String | Attribute name for which the rule is applicable | Duplicate attributes are not allowed. Here, [] denotes array operation. Allowed operations are [#] and [*]; which denote any index of the array and all index of the array respectively. '#' is evaluated using a specified condition mentioned in "andCondition". |
| rules | Array | Priority rules for the attribute | Rules can not be empty. |
| priorities | Array | Priority list of contributing sources for respective attribute. | Priorities can not be empty and the maximum length of the source value allowed is 25 characters. |
| andCondition | Array | Conditions on which AND operation is to be performed. | Only andCondition can be used in Merge Rule. orCondition is not allowed. |
Sample Payload
{
"kind": "osdu:wks:master-data--Wellbore:1.*.*",
"mergeRules": [
{
"attribute": "data.FacilityName",
"rules": [
{
"priorities": [
"Source-A",
"Source-B"
]
}
]
},
{
"attribute": "data.FacilityID",
"rules": [
{
"priorities": [
"Source-B",
"Source-A"
]
}
]
},
{
"attribute": "data.FacilitySpecifications[*]",
"rules": [
{
"priorities": [
"Source-A",
"Source-B"
]
}
]
},
{
"attribute": "data.FacilityStates[*]",
"rules": [
{
"priorities": [
"Source-B",
"Source-A"
]
}
]
},
{
"attribute": "data.NameAliases[*]",
"rules": [
{
"priorities": [
"Source-A",
"Source-B"
],
"andCondition": [
{
"type": "equals",
"attribute": "data.NameAliases[#].AliasNameTypeID",
"values": [
"{datapartition}:reference-data--AliasNameType:LeaseIdentifier:",
"{datapartition}:reference-data--AliasNameType:UniqueIdentifier:"
]
}
]
},
{
"priorities": [
"Source-B",
"Source-A"
],
"andCondition": [
{
"type": "equals",
"attribute": "data.NameAliases[#].AliasNameTypeID",
"values": [
"{datapartition}:reference-data--AliasNameType:RegulatoryIdentifier:"
]
}
]
}
]
},
{
"attribute": "data.VerticalMeasurements[*]",
"rules": [
{
"priorities": [
"Source-B",
"Source-A"
],
"andCondition": [
{
"type": "notEquals",
"attribute": "data.NameAliases[#].VerticalMeasurementTypeID",
"values": [
"{datapartition}:reference-data--VerticalMeasurementType:KB:",
"{datapartition}:reference-data--VerticalMeasurementType:GL:"
]
}
]
}
]
},
{
"attribute": "data.SpatialLocation",
"rules": [
{
"priorities": [
"Source-B",
"Source-A"
]
}
]
},
{
"attribute": "data.ExtensionProperties.SLBWellContexts.Country",
"rules": [
{
"priorities": [
"Source-A",
"Source-B"
]
}
]
},
{
"attribute": "data.ExtensionProperties.SLBWellContexts.EwDirection",
"rules": [
{
"priorities": [
"Source-B",
"Source-A"
]
}
]
},
{
"attribute": "data.ExtensionProperties.SLBWellContexts.PrimaryKeyNodeID",
"rules": [
{
"priorities": [
"Source-A",
"Source-B"
]
}
]
},
{
"attribute": "data.ExtensionProperties.SLBWellContexts.Township",
"rules": [
{
"priorities": [
"Source-B",
"Source-A"
]
}
]
},
{
"attribute": "data.*",
"rules": [
{
"priorities": [
"Source-B",
"Source-A"
]
}
]
}
]
}
Sample Response
{
"message": "osdu:wks:master-data--Wellbore:1.*.*"
}
You can fetch the Merge Rules configuration based on kind using the GET /data-mastering/v1/merge-rules/{kind} endpoint. The following parameters are used to fetch the related data:
| Attribute | Datatype | Description | Validation |
|---|---|---|---|
| kind | String | This is the wks kind specifying the major version; i.e., the source kind of the data. | Kind is a case sensitive attribute. The format of kind should be authority:source:entity-type:majorversion... |
Sample Response
{
"kind": "osdu:wks:master-data--Wellbore:1.*.*",
"mergeRules": [
{
"attribute": "data.FacilityName",
"rules": [
{
"priorities": [
"Source-A",
"Source-B"
]
}
]
},
{
"attribute": "data.FacilityID",
"rules": [
{
"priorities": [
"Source-B",
"Source-A"
]
}
]
},
{
"attribute": "data.FacilitySpecifications[*]",
"rules": [
{
"priorities": [
"Source-A",
"Source-B"
]
}
]
},
{
"attribute": "data.FacilityStates[*]",
"rules": [
{
"priorities": [
"Source-B",
"Source-A"
]
}
]
},
{
"attribute": "data.NameAliases[*]",
"rules": [
{
"priorities": [
"Source-A",
"Source-B"
],
"andCondition": [
{
"type": "equals",
"attribute": "data.NameAliases[#].AliasNameTypeID",
"values": [
"{datapartition}:reference-data--AliasNameType:LeaseIdentifier:",
"{datapartition}:reference-data--AliasNameType:UniqueIdentifier:"
]
}
]
},
{
"priorities": [
"Source-B",
"Source-A"
],
"andCondition": [
{
"type": "equals",
"attribute": "data.NameAliases[#].AliasNameTypeID",
"values": [
"{datapartition}:reference-data--AliasNameType:RegulatoryIdentifier:"
]
}
]
}
]
},
{
"attribute": "data.VerticalMeasurements[*]",
"rules": [
{
"priorities": [
"Source-B",
"Source-A"
],
"andCondition": [
{
"type": "notEquals",
"attribute": "data.NameAliases[#].VerticalMeasurementTypeID",
"values": [
"{datapartition}:reference-data--VerticalMeasurementType:KB:",
"{datapartition}:reference-data--VerticalMeasurementType:GL:"
]
}
]
}
]
},
{
"attribute": "data.SpatialLocation",
"rules": [
{
"priorities": [
"Source-B",
"Source-A"
]
}
]
},
{
"attribute": "data.ExtensionProperties.SLBWellContexts.Country",
"rules": [
{
"priorities": [
"Source-A",
"Source-B"
]
}
]
},
{
"attribute": "data.ExtensionProperties.SLBWellContexts.EwDirection",
"rules": [
{
"priorities": [
"Source-B",
"Source-A"
]
}
]
},
{
"attribute": "data.ExtensionProperties.SLBWellContexts.PrimaryKeyNodeID",
"rules": [
{
"priorities": [
"Source-A",
"Source-B"
]
}
]
},
{
"attribute": "data.ExtensionProperties.SLBWellContexts.Township",
"rules": [
{
"priorities": [
"Source-B",
"Source-A"
]
}
]
},
{
"attribute": "data.*",
"rules": [
{
"priorities": [
"Source-B",
"Source-A"
]
}
]
}
]
}
You can access this API here.