Class: Transform

datarestructor.Transform(descriptions)

new Transform(descriptions)

Main class for the data transformation.

Parameters:
Name Type Description
descriptions Array.<module:datarestructor.PropertyStructureDescription>
Source:

Members

(protected) config :module:datarestructor.TransformConfig

Configuration for the transformation.

Type:
Source:

descriptions :Array.<module:datarestructor.DescribedEntry>

Descriptions of the input data that define the behaviour of the transformation.

Type:
Source:

Methods

enableDebugMode() → {module:datarestructor.Transform}

Enables debug mode. Logs additional information.

Source:
Returns:
Type
module:datarestructor.Transform

processJson(data) → {Array.<module:datarestructor.DescribedEntry>}

"Assembly line", that takes the (pared JSON) data and processes it using all given descriptions in their given order.

Parameters:
Name Type Description
data object

parsed JSON data or any other data object

Source:
Returns:
Type
Array.<module:datarestructor.DescribedEntry>
Example
var allDescriptions = [];
allDescriptions.push(summariesDescription());
allDescriptions.push(detailsDescription());
var result = new datarestructor.Transform(allDescriptions).processJson(jsonData);

setMaxRecursionDepth(value) → {module:datarestructor.Transform}

Sets the maximum recursion depth. Defaults to 8 if not set.

Parameters:
Name Type Description
value number

non negative number.

Source:
Returns:
Type
module:datarestructor.Transform

setRemoveDuplicationAboveRecursionDepth(value) → {module:datarestructor.Transform}

Sets the recursion depth above which duplication will be removed. Duplications below it remain unchanged. Defaults to 1.

Since fields can contain groups of fields that can contain groups of fields..., cyclic data structures are possible by nature and will lead to duplications. Some of them might be intended e.g. to take one (sub-)field with all (duplicated) groups. To restrict duplications and improve performance it is beneficial to define a recursion depth, above which further duplication won't be used and should be removed/avoided.

Parameters:
Name Type Description
value number

non negative number.

Source:
Returns:
Type
module:datarestructor.Transform

(protected, static) applyGroupDestinationPattern(groupedObject) → {Array.<module:datarestructor.DescribedEntry>}

Takes already grouped module:datarestructor.DescribedEntry objects and uses their "_identifier.groupDestinationId" (if exists) to move groups to the given destination.

This is useful, if separately described groups like "summary" and "detail" should be put together, so that every summery contains a group with the regarding details.

Parameters:
Name Type Description
groupedObject Array.<module:datarestructor.DescribedEntry>

already grouped entries

Source:
Returns:
Type
Array.<module:datarestructor.DescribedEntry>

(protected, static) asIdBasedObject(elements, idOfElementFunction)

Converts the given elements to an object, that provides these entries by their id. For example, [{id: A, value: 1}] becomes result['A'] = 1.

Parameters:
Name Type Description
elements Array.<module:datarestructor.DescribedEntry>

of DescribedEntry elements

idOfElementFunction module:datarestructor.stringFieldOfDescribedEntryFunction

returns the id of an DescribedEntry

Source:
Returns:

{module:datarestructor.DescribedEntry[] entries indexed by id

(protected, static) deduplicateFlattenedData(entries, entriesToMerge, idOfElementFunction)

Takes two arrays of objects, e.g. [{id: B, value: 2},{id: C, value: 3}] and [{id: A, value: 1},{id: B, value: 4}] and merges them into one: [{id: C, value: 3},{id: A, value: 1},{id: B, value: 4}]

Entries with the same id ("duplicates") will be overwritten. Only the last element with the same id remains. The order is determined by the order of the array elements, whereas the first array comes before the second one. This means, that entries with the same id in the second array overwrite entries in the first array, and entries occurring later in the array overwrite earlier ones, if they have the same id.

"entriesToMerge" will be returned directly, if "entries" is null or empty.

The id is extracted from every element using their deduplication pattern (if available).

Parameters:
Name Type Description
entries Array.<module:datarestructor.DescribedEntry>
entriesToMerge Array.<module:datarestructor.DescribedEntry>
idOfElementFunction module:datarestructor.stringFieldOfDescribedEntryFunction

returns the id of an DescribedEntry

Source:
See:
  • mergeFlattenedData

(protected, static) extractEntriesByDescription(flattenedData) → {Array.<module:datarestructor.DescribedEntry>}

Extracts entries out of "flattened" JSON data and provides an array of objects.

Parameters:
Name Type Description
flattenedData Array.<Object>

flattened json from search query result

Properties
Name Type Description
name string

name of the property in hierarchical order separated by points

value string

value of the property as string

module:datarestructor.PropertyStructureDescription

description of structure of the entries that should be extracted

Source:
Returns:
Type
Array.<module:datarestructor.DescribedEntry>

(protected, static) fillInArrayValues()

Fills in extra "_comma_separated_values" properties into the flattened data for properties that end with an array. E.g. response.hits.hits.tags[0]="active" and response.hits.hits.tags[0]="ready" will lead to the extra element "response.hits.hits.tags_comma_separated_values="active, ready".

Source:
Returns:

flattened data with filled in "_comma_separated_values" properties

(protected, static) forEachGroupEntry(rootEntry, onFoundEntry)

Traverses through all groups and their entries and calls the given function on every found entry with the group name and the entry itself as parameters.

Parameters:
Name Type Description
rootEntry module:datarestructor.DescribedEntry
onFoundEntry module:datarestructor.onEntryFoundFunction
Source:

(protected, static) groupById(elements, groupNameOfElementFunction, groupIdOfElementFunction)

Converts the given elements into an object, that provides these entries by their id. For example, [{id: A, value: 1}] becomes result['A'] = 1. Furthermore, this function creates a group property (with the name ) and collects all related elements (specified by their group pattern) in it.

Parameters:
Name Type Description
elements Array.<module:datarestructor.DescribedEntry>

of DescribedEntry elements

groupNameOfElementFunction module:datarestructor.stringFieldOfDescribedEntryFunction

function, that returns the name of the group property that will be created inside the "main" element.

groupIdOfElementFunction module:datarestructor.stringFieldOfDescribedEntryFunction

returns the group id of an DescribedEntry

Source:
Returns:

{module:datarestructor.DescribedEntry[] entries indexed by id

(protected, static) groupFlattenedData(elements)

Converts the given elements into an object, that provides these entries by their id (determined by the entry's groupPattern). For example, [{id: A, value: 1}] becomes result['A'] = 1.

Furthermore, this function creates a group property (determined by the entry's groupName) and collects all related elements (specified by their group pattern) in it.

Parameters:
Name Type Description
elements Array.<module:datarestructor.DescribedEntry>

of DescribedEntry elements

Source:
Returns:

{module:datarestructor.DescribedEntry[] entries indexed by id

(protected, static) mergeFlattenedData(entries, entriesToMerge, idOfElementFunction)

Takes two arrays of objects, e.g. [{id: B, value: 2},{id: C, value: 3}] and [{id: A, value: 1},{id: B, value: 4}] and merges them into one: [{id: C, value: 3},{id: A, value: 1},{id: B, value: 4}]

Entries with the same id ("duplicates") will be overwritten. Only the last element with the same id remains. The order is determined by the order of the array elements, whereas the first array comes before the second one. This means, that entries with the same id in the second array overwrite entries in the first array, and entries that occur later in the array overwrite earlier ones, if they have the same id.

The id is extracted from every element using the given function.

Parameters:
Name Type Description
entries Array.<module:datarestructor.DescribedEntry>
entriesToMerge Array.<module:datarestructor.DescribedEntry>
idOfElementFunction module:datarestructor.stringFieldOfDescribedEntryFunction

returns the id of an DescribedEntry

Source:

(protected, static) processJsonUsingDescriptions(jsonData, descriptions, config) → {Array.<module:datarestructor.DescribedEntry>}

"Assembly line", that takes the jsonData and processes it using all given descriptions in their given order.

Parameters:
Name Type Description
jsonData object

parsed JSON data or any other data object

descriptions Array.<module:datarestructor.PropertyStructureDescription>

already grouped entries

config module:datarestructor.TransformConfig

configuration for the data transformation

Source:
Returns:
Type
Array.<module:datarestructor.DescribedEntry>

(protected, static) toDescribedField(entry, recursionContext) → {module:described_field.DescribedDataField}

Converts a internal described entry to a newly created public described field. Since the structure of a described field is hierarchical, this function is called recursively. Because the internal described entries may very likely contain cyclic references, the depth of recursion needs to be limited. Therefore, the current recursion depth is taken as second parameter and the maximum recursion depth is taken as third parameter.

Parameters:
Name Type Description
entry module:datarestructor.DescribedEntry

the internal entry that will be converted

recursionContext module:datarestructor.DescribedFieldRecursionContext

context contains everything that needs to be accessible throughout the recursion.

Source:
Returns:
Type
module:described_field.DescribedDataField

(protected, static) toDescribedFields(describedEntries, config) → {Array.<module:described_field.DescribedDataField>}

Converts described entries (internal data structure) to described fields (external data structure). Since the structure of a described field is hierarchical, every field needs to be converted in a recursive manner. The maximum recursion depth is taken as the second parameter.

Parameters:
Name Type Description
describedEntries Array.<module:datarestructor.DescribedEntry>
config module:datarestructor.TransformConfig

configuration for the data transformation

Source:
Returns:
Type
Array.<module:described_field.DescribedDataField>