Skip to content

Commit 4f778f8

Browse files
michaelamadimichaelamadi
committed
Exposed the Data Package functions for direct use in Power Query M expressions. Updated README.md to cover the Power Query M functions.
1 parent 4f69826 commit 4f778f8

File tree

4 files changed

+155
-14
lines changed

4 files changed

+155
-14
lines changed

‎DataPackage/DataPackage.pq

+17-10
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
2222
SOFTWARE.
2323
*/
2424

25-
[Version = "1.0.1"]
25+
[Version = "1.1.0"]
2626
section DataPackage;
2727

2828
//
@@ -71,11 +71,17 @@ Extension.LoadFunction = (name as text) =>
7171
Expression.Evaluate(asText, #shared);
7272

7373
// Load the DataPackage.Tables module
74-
DataPackage.Tables = Extension.LoadFunction("DataPackageTables.pqm");
74+
75+
shared DataPackage.Helper = Extension.LoadFunction("DataPackageHelper.pqm");
76+
shared DataPackage.Table = Extension.LoadFunction("DataPackageTable.pqm");
77+
shared DataPackage.Tables = Extension.LoadFunction("DataPackageTables.pqm");
78+
7579

7680
[DataSource.Kind="DataPackage", Publish="DataPackage.Publish"]
7781
shared DataPackage.Load = Value.ReplaceType(DataPackage.LoadContents, DataPackage.LoadContentsWithDocumentation);
7882

83+
84+
7985
DataPackage.LoadContentsWithDocumentation = type function (
8086
dataPackageIdentifier as (
8187
type text meta [
@@ -108,16 +114,17 @@ DataPackage.LoadContents = (
108114

109115
// Rename columns to meet the navigation table requirements
110116
NavigationReadyDataResources = Table.RenameColumns(
111-
DataResources,
112-
{
113-
{"name", "Name"},
114-
{"data", "Data"}
115-
}
116-
),
117+
DataResources,
118+
{
119+
{"name", "Name"},
120+
{"data", "Data"}
121+
}
122+
),
117123
NonNullData = Table.SelectRows(NavigationReadyDataResources, each [Data] <> null and [Data] <> ""),
118124
TableWithItemKind = Table.AddColumn(NonNullData, "ItemKind", each "Table", Text.Type),
119-
TableWithIsLeaf = Table.AddColumn(TableWithItemKind, "IsLeaf", each "True", Text.Type),
120-
NavTable = Table.ToNavigationTable(TableWithItemKind, {"Name"}, "Name", "Data", "ItemKind", "ItemKind", "IsLeaf")
125+
TableWithItemName = Table.AddColumn(TableWithItemKind, "ItemName", each "Table", Text.Type),
126+
TableWithIsLeaf = Table.AddColumn(TableWithItemName, "IsLeaf", each "True", Text.Type),
127+
NavTable = Table.ToNavigationTable(TableWithIsLeaf, {"Name"}, "Name", "Data", "ItemKind", "ItemName", "IsLeaf")
121128
in
122129
NavTable;
123130

‎README.md

+138-4
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Data Package Connector (datapackage-connector)
22

3-
A Power BI [Custom Connector](https://docs.microsoft.com/en-us/power-bi/desktop-connector-extensibility#custom-connectors) based on the [Data Package M (datapackage-m)](https://github.com/nimblelearn/datapackage-m) functions. Data Package Connector enables you to load one or more tables from [Tabular Data Packages](http://frictionlessdata.io/docs/tabular-data-package/) directly into Power BI through the familiar 'Get Data' experience.
3+
A Power BI [Custom Connector](https://docs.microsoft.com/en-us/power-bi/desktop-connector-extensibility#custom-connectors) based on the [Data Package M (datapackage-m)](https://github.com/nimblelearn/datapackage-m) functions. Data Package Connector enables you to load one or more tables from [Tabular Data Packages](http://frictionlessdata.io/docs/tabular-data-package/) directly into Power BI through the familiar 'Get Data' experience. It also includes the Data Package M functions library.
44

55
Not heard of Power BI? You can learn about it [here](https://docs.microsoft.com/en-us/power-bi/power-bi-overview).
66

@@ -36,13 +36,14 @@ Visit [https://frictionlessdata.io](https://frictionlessdata.io) to learn more.
3636
![Microsoft Power BI UI: GDP Tabular Data Package Data Refresh](./images/power-bi/datapackage-connector-power-bi-illustration-06.png)
3737
*Full support for on demand and scheduled data refresh in the Power BI service (requires a Data Gateway)*
3838

39+
![Microsoft Power BI UI: Data Package M functions](./images/power-bi/datapackage-connector-power-bi-illustration-07.png)
40+
*Includes [Data Package M](https://github.com/nimblelearn/datapackage-m) functions for more advanced scenarios*
3941

4042
## Data Package Connector Functions
4143

4244
| Function Name | Description |
4345
| :----------------- | :-------------------------------------------------------------------------- |
44-
| DataPackage.Load | Returns a Navigation Table (i.e. `table`) that lists the [Data Resources](https://frictionlessdata.io/specs/data-resource/) contained within a Data Package. This function wraps DataPackage.Tables but expects the `ignoreTableSchemaTypes` parameter to be a `text` representation of a logical value instead of an actual `logical` value. The `text` value is casted to a `logical` value before calling DataPackage.Tables. See [Data Package M (datapackage-m)](https://github.com/nimblelearn/datapackage-m) to learn more about the DataPackage.Tables function.|
45-
46+
| DataPackage.Load | Returns a Navigation Table (i.e. `table`) that lists the [Data Resources](https://frictionlessdata.io/specs/data-resource/) contained within a Data Package. This function wraps DataPackage.Tables but expects the `ignoreTableSchemaTypes` parameter to be a `text` representation of a logical value instead of an actual `logical` value. The `text` value is casted to a `logical` value before calling DataPackage.Tables.|
4647

4748
### DataPackage.Load
4849

@@ -53,6 +54,55 @@ Visit [https://frictionlessdata.io](https://frictionlessdata.io) to learn more.
5354

5455
Only data resources that are detected as being tabular (i.e. contain a `table` value in their 'Data' column) are returned by this function.
5556

57+
## Data Package M Functions
58+
59+
| Function Name | Description |
60+
| :----------------- | :-------------------------------------------------------------------------- |
61+
| DataPackage.Table | Returns a [Tabular Data Resource](https://frictionlessdata.io/specs/tabular-data-resource/) as a `table` |
62+
| DataPackage.Tables | Returns a `table` that lists the [Data Resources](https://frictionlessdata.io/specs/data-resource/) contained within a Data Package |
63+
| DataPackage.Helper | Returns a Data Package helper function as a `function` |
64+
65+
### DataPackage.Table
66+
67+
| Parameter | Type | Description |
68+
| :--------------------- | :----------- | :------------------------------------------------------- |
69+
| dataPackageIdentifier | text | A valid [Data Package Identifier](https://frictionlessdata.io/specs/data-package-identifier/) |
70+
| dataResourceIndex | number | A valid Data Resource index |
71+
| dataResourceName | text | A valid Data Resource name |
72+
| ignoreTableSchemaTypes | logical | Controls whether the Table Schema is applied to the data |
73+
74+
### DataPackage.Tables
75+
76+
| Parameter | Type | Description |
77+
| :--------------------- | :----------- | :------------------------------------------------------- |
78+
| dataPackageIdentifier | text | A valid [Data Package Identifier](https://frictionlessdata.io/specs/data-package-identifier/) |
79+
| ignoreTableSchemaTypes | logical | Controls whether the [Table Schema](https://frictionlessdata.io/specs/table-schema/) is applied to the data |
80+
81+
Any Data Resource that is detected as being tabular will contain the data table in the 'data' column. Data Package properties that are inferred and added by Data Package M have their name preceded by double underscore e.g. '__fullpath'.
82+
83+
### DataPackage.Helper
84+
85+
| Parameter | Type | Description |
86+
| :--------------------- | :----------- | :------------------------------------------------------- |
87+
| functionName | text | A valid Data Package helper function name |
88+
89+
This is a special function that acts as a library of Data Package helper functions. As the returned functions are only used as helpers for the `DataPackage.Table` and `DataPackage.Tables` functions, please see the comments inline with the [Power Query M expressions](./DataPackage/DataPackageHelper.pqm) to understand how they work. Advanced Power Query M users may wish to use these helper functions to work with Data Package metadata more directly.
90+
91+
## Table Schema Type Conversions
92+
93+
Type conversion is attempted for the most common [Table Schema](https://frictionlessdata.io/specs/table-schema/) types:
94+
95+
| Table Schema Type | M Type |
96+
| :-----------------| :------- |
97+
| string | text |
98+
| number | number |
99+
| integer | number |
100+
| boolean | logical |
101+
| date | date |
102+
| datetime | datetime |
103+
| time | time |
104+
105+
Unhandled types are defaulted to the `text` type. Setting the `ignoreTableSchemaTypes` property to `true` when invoking `DataPackage.Load`, `DataPackage.Table`, or `DataPackage.Tables` will stop the Table Schema from being applied. This can be useful when one or more values in a column cause an error when the Table Schema type conversions are attempted.
56106

57107
## Setup
58108

@@ -71,7 +121,91 @@ The thumbprint to use when setting up the Data Package Connector as a trusted th
71121
3. Follow [these](https://docs.microsoft.com/en-us/power-bi/desktop-connector-extensibility) instructions on how to use Power BI Custom Connectors with Power BI Desktop.
72122
4. Follow [these](https://docs.microsoft.com/en-us/power-bi/service-gateway-custom-connectors) instructions on how to use Power BI Custom Connectors with the Power BI On-premises Data Gateway.
73123

74-
124+
## Power Query M Expression Examples
125+
126+
The following examples show the *recommended* way to invoke the Data Package M functions when using them in your Power Query M expressions.
127+
128+
#### Getting the List of Resources from a Data Package (Remote)
129+
130+
```text
131+
// Invoke the function
132+
Source = DataPackage.Tables("https://datahub.io/core/gdp/datapackage.json")
133+
in
134+
Source
135+
```
136+
137+
#### Getting the List of Resources from a Data Package (Local)
138+
139+
```text
140+
let
141+
// Invoke the function
142+
Source = DataPackage.Tables("C:\gdp\datapackage.json")
143+
in
144+
Source
145+
```
146+
147+
#### Getting the Data for a Resource Using Its Index (Remote)
148+
149+
```text
150+
let
151+
// Invoke the function
152+
Source = DataPackage.Table("https://datahub.io/core/gdp/datapackage.json", 0)
153+
in
154+
Source
155+
```
156+
157+
#### Getting the Data for a Resource Using Its Index (Local)
158+
159+
```text
160+
let
161+
// Invoke the function
162+
Source = DataPackage.Table("C:\gdp\datapackage.json", 0)
163+
in
164+
Source
165+
```
166+
167+
#### Getting the Data for a Resource Using Its Name (Remote)
168+
169+
```text
170+
let
171+
// Invoke the function
172+
Source = DataPackage.Table("https://datahub.io/core/gdp/datapackage.json", null, "gdp")
173+
in
174+
Source
175+
```
176+
177+
#### Getting the Data for a Resource Using Its Name (Local)
178+
179+
```text
180+
let
181+
// Invoke the function
182+
Source = DataPackage.Table("C:\gdp\datapackage.json", null, "gdp")
183+
in
184+
Source
185+
```
186+
187+
#### Getting the Data for a Resource Without Table Schema Type Conversion
188+
189+
```text
190+
let
191+
// Invoke the function
192+
Source = DataPackage.Table("https://datahub.io/core/gdp/datapackage.json", null, "gdp", true)
193+
in
194+
Source
195+
```
196+
197+
#### Invoking a Data Package Helper Function Directly
198+
199+
```text
200+
let
201+
// Get the required helper function by name
202+
DataPackage.Package = DataPackage.Helper("DataPackage.Package"),
203+
204+
// Invoke the helper function
205+
Source = DataPackage.Package("https://datahub.io/core/gdp/datapackage.json")
206+
in
207+
Source
208+
```
75209

76210
## Try Data Package Connector with the Core Datasets
77211

‎distributable/DataPackage.pqx

46 Bytes
Binary file not shown.

‎images/power-bi/datapackage-connector-power-bi-illustration-07.png

16.6 KB
Loading

0 commit comments

Comments
 (0)