Skip to content

Commit

Permalink
Merge pull request #25 from data-solution-automation-engine/dev/sj-md…
Browse files Browse the repository at this point in the history
…-updates

Update metadata
  • Loading branch information
RoelantVos authored Apr 28, 2024
2 parents 2472710 + 5bdbc70 commit 1d5e96d
Show file tree
Hide file tree
Showing 19 changed files with 81 additions and 66 deletions.
13 changes: 13 additions & 0 deletions docs/build.bat
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
@echo off

REM Make sure we're running this from the context of the script file location
pushd %~dp0

REM Build through docfx, using the docfx.json configuration file
REM This will generate the static site in the _site folder
REM note that docfx needs to be installed and available in the PATH

REM Install docfx: dotnet tool install -g docfx
REM Docos: https://dotnet.github.io/docfx/tutorial/docfx_getting_started.html

docfx docfx.json
14 changes: 7 additions & 7 deletions docs/faq/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Why have a schema definition for Data Warehouse Automation?

The reason work on the schema definition started is based on a desire to collaborate with various likeminded professionals in the industry, but who all had different (proprietary) ways to record metadata. The challenge was how to collaborate on at least the patterns, without necessarily agreeing on ‘the best’ back-end solution to store metadata.
The reason work on the schema definition started is based on a desire to collaborate with various like-minded professionals in the industry, but who all had different (proprietary) ways to record metadata. The challenge was how to collaborate on at least the patterns, without necessarily agreeing on ‘the best’ back-end solution to store metadata.

The interface for Data Warehouse Automation was started as an initiative that would support each to use the back-end of their choosing, while still be able to cooperate on the design and improvement of Data Warehouse implementation patterns and ETL generation concepts.

Expand All @@ -16,11 +16,11 @@ The decision to start the format with an array / list that contains potentially

For example, consider the loading of a Core Business Concept (‘Hub’) type entity from various different data sources. If you would use these different mappings to generate ETL processes you would create one physical ETL object for each mapping. However, if you are seeking to generate a view that represents the target table you would use the collection (list) of mappings to generate separate statements that are 'unioned' in a single view object.

Example: [https://github.com/RoelantVos/Data-Warehouse-Automation-Metadata-Schema/blob/master/ClassLibrary/DataWarehouseAutomation/Sample_Metadata/sampleBasic.json].
Example: [https://github.com/RoelantVos/Data-Warehouse-Automation-Metadata-Schema/blob/master/ClassLibrary/DataWarehouseAutomation/Sample_Metadata/sampleBasic.json](https://github.com/RoelantVos/Data-Warehouse-Automation-Metadata-Schema/blob/master/ClassLibrary/DataWarehouseAutomation/Sample_Metadata/sampleBasic.json).

Or below, an even more simplified example of a single Data Object Mapping (with only one source, one target and a single Data Item Mapping) being part of the list dataObjectMappings.

````json
```json
{
"dataObjectMappings": [
{
Expand Down Expand Up @@ -48,7 +48,7 @@ Or below, an even more simplified example of a single Data Object Mapping (with
}
]
}
`````
```

## Why are Data Objects and Data Items defined as arrays when they are used as sources?

Expand All @@ -58,7 +58,7 @@ The downside is that either a loop or array index needs to be used to pinpoint t

Example: using Handlebars will give you the name of the first source object:

```JSON
```handlebars
{{sourceDataObjects.0.name}}
```

Expand All @@ -76,7 +76,7 @@ The easiest way to get started is to either copy and modify the example project,

A simple C# example to generate some quick ETL (taken from the example project):

```json
```cs
// Load a template (pattern) from file
stringTemplate = File.ReadAllText(AppDomain.CurrentDomain.BaseDirectory + @"..\\..\\TemplateSampleBasic.handlebars");

Expand All @@ -94,4 +94,4 @@ result = template(deserialisedMapping);

// Display the results to the user
Console.WriteLine(result);
``````
```
6 changes: 3 additions & 3 deletions docs/handlebars/extension_exists.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,21 +8,21 @@ Only the *multiActiveKey* and *targetDataItem* properties are currently supporte

## Usage

``` handlebars
```handlebars
{{#exists "<category / json segment>" "<optional value>"}}
```

## Example

``` handlebars
```handlebars
{{#exists multiActiveKey}}There is a multi-active key!{{else}}No multi-active key is found in this data object mapping.{{/exists}}
{{#exists multiActiveKey "DATE_OF_BIRTH"}}There is a multi-active key which is not DATE_OF_BIRTH{{else}}No multi-active key with DATE_OF_BIRTH is found in this data object mapping.{{/exists}}
{{#exists targetDataItem}}There is a target data item in this mapping!{{else}}No target data items are defined in this mapping.{{/exists}}
```

Depending on the metadata, this may result in:

```dotnetcli
```text
There is a multi-active key!
No multi-active key with DATE_OF_BIRTH is found in this data object mapping.
There is a target data item in this mapping!
Expand Down
6 changes: 3 additions & 3 deletions docs/handlebars/extension_lookupextension.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,18 +8,18 @@ Pass in the extensions list and the string key value.

## Usage

``` handlebars
```handlebars
{{lookupExtension <extension list> "<string value>"}}
```

## Example

``` handlebars
```handlebars
The lookup value for the 'type' extension is '{{lookupExtension sourceDataObjects.0.extensions "type"}}'.
```

Depending on the metadata, this may result in:

```dotnetcli
```text
The lookup value for the 'type' extension is 'procedure'.
```
4 changes: 2 additions & 2 deletions docs/handlebars/extension_now.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,13 @@ Get the date and time at the moment of execution.

## Usage

``` handlebars
```handlebars
{{now}}
```

## Example

``` handlebars
```handlebars
The time is {{now}}.
```

Expand Down
6 changes: 3 additions & 3 deletions docs/handlebars/extension_randomdate.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,18 +6,18 @@ This was originally added to generate test data, and is used for referential-int

## Usage

``` handlebars
```handlebars
{{randomdate 2020}}
```

## Example

``` handlebars
```handlebars
Here is a random date: {{randomdate 2020}}.
```

This may return:

```dotnetcli
```text
2022-08-03T00:00:00.000000
```
6 changes: 3 additions & 3 deletions docs/handlebars/extension_randomnumber.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,18 +6,18 @@ This was originally added to generate test data, and is used for referential-int

## Usage

``` handlebars
```handlebars
{{randomnumber 5}}
```

## Example

``` handlebars
```handlebars
And here is a random string: {{randomnumber 5}}.
```

This may return:

```dotnetcli
```text
1346
```
6 changes: 3 additions & 3 deletions docs/handlebars/extension_randomstring.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,18 @@ Generate a random string no longer than the provided characters as integer input

## Usage

``` handlebars
```handlebars
{{randomstring 10}}
```

## Example

``` handlebars
```handlebars
And here is a random string: {{randomstring 10}}.
```

This returns:

```dotnetcli
```text
Cg8uKuBFS5
```
6 changes: 3 additions & 3 deletions docs/handlebars/extension_replicate.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,21 +6,21 @@ This is often used to generate test data.

## Usage

``` handlebars
```handlebars
{{#replicate 10}}
```

## Example

``` handlebars
```handlebars
{{#replicate 3}}
This value is replicated 3 times!
{{/replicate}}
```

This returns:

```dotnetcli
```text
This value is replicated 3 times!
This value is replicated 3 times!
This value is replicated 3 times!
Expand Down
6 changes: 3 additions & 3 deletions docs/handlebars/extension_space.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,18 @@ Space will space out (!?) the name of the input value to 30 characters using tab

## Usage

``` handlebars
```handlebars
{{space "<value>"}}
```

## Example

``` handlebars
```handlebars
This is {{space "Hello World"}} ...spaced out.
```

This results in:

```dotnetcli
```text
This is Hello World ...spaced out.
```
6 changes: 3 additions & 3 deletions docs/handlebars/extension_stringcompare.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,18 @@ Block helper to evaluate if the input matches a specified string. Also optionall

## Usage

``` handlebars
```handlebars
{{#stringcompare string1 string2}} do something {{else}} do something else {{/stringcompare}}
```

## Example

``` handlebars
```handlebars
A and B {{#stringcompare "A" "B"}}are the same. {{else}}are not the same. {{/stringcompare}}
```

This results in:

```dotnetcli
```text
A and B are not the same.
```
6 changes: 3 additions & 3 deletions docs/handlebars/extension_stringdiff.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,18 @@ Block helper to evaluate if the input does *not* match a specified string. Also

## Usage

``` handlebars
```handlebars
{{#stringdiff string1 string2}} do something {{else}} do something else {{/stringdiff}}
```

## Example

``` handlebars
```handlebars
A and B {{#stringdiff "A" "B"}}are not the same. {{else}}are the same. {{/stringdiff}}
```

This results in:

```dotnetcli
```text
A and B are not the same.
```
6 changes: 3 additions & 3 deletions docs/handlebars/extension_stringreplace.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,18 @@ Parses an input string value, and replaces a specified part of the contents with

## Usage

``` handlebars
```handlebars
{{stringreplace "<input string value>" "<lookup character>" "<replacement character>"}}
```

## Example

``` handlebars
```handlebars
This replaces the o's in Hello Word with an !: {{StringReplace "Hello World" "o" "!"}}
```

This results in:

```dotnetcli
```text
This replaces the o's in Hello Word with an !: Hell! W!rld
```
6 changes: 3 additions & 3 deletions docs/handlebars/extension_stringwrap.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,18 @@ Wraps an input character string with two specified characters.

## Usage

``` handlebars
```handlebars
{{stringwrap "<start character>" "<end character>"}}
```

## Example

``` handlebars
```handlebars
This adds brackets around the string value: {{stringwrap "Example" "[" "]"}}
```

This results in:

```dotnetcli
```text
[Example]
```
6 changes: 3 additions & 3 deletions docs/handlebars/extension_targetdataitemexists.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,18 +8,18 @@ This is used in some cases to handle special column names.

## Usage

``` handlebars
```handlebars
{{stringwrap "<character value>"}}
```

## Example

``` handlebars
```handlebars
{{#targetDataItemExists "FIRST_NAME"}}FIRST_NAME exists{{else}}FIRST_NAME does not exist{{/targetDataItemExists}}
```

This results in:

```dotnetcli
```text
FIRST_NAME exists
```
2 changes: 1 addition & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,4 +38,4 @@ Across most, if not all, metadata models there is a core set of information that

### Working guidelines

For any change, create a new branch (no direct commits to master branch).
For any change, create a new branch (no direct commits to main branch).
Loading

0 comments on commit 1d5e96d

Please sign in to comment.