Skip to content

Commit 97d163e

Browse files
SumoSourabhSumoSourabh
authored
CP-8874 Script to build the VSDK project with various commands (#453)
* CP-8874 Script to build the VSDK project with various commands * DLPX-83062 Update engine version to 1.11.16 in develop branch * DLPX-83062 Update engine version to 1.11.16 in develop branch
1 parent 26c8ba8 commit 97d163e

File tree

8 files changed

+340
-176
lines changed

8 files changed

+340
-176
lines changed

‎.bumpversion.cfg

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,16 +3,17 @@ current_version = 4.0.6.dev0
33
commit = False
44
tag = False
55
parse = (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+)(\.(?P<release>[a-z]+)(?P<dev>\d+))?
6-
serialize =
6+
serialize =
77
{major}.{minor}.{patch}.{release}{dev}
88
{major}.{minor}.{patch}
99

1010
[bumpversion:part:release]
11-
optional_value = prod
11+
optional_value = _
1212
first_value = dev
13-
values =
13+
values =
1414
dev
15-
prod
15+
rc
16+
_
1617

1718
[bumpversion:part:dev]
1819

@@ -26,3 +27,7 @@ values =
2627

2728
[bumpversion:file:./tools/src/main/python/dlpx/virtualization/_internal/VERSION]
2829

30+
[bumpversion:file:./tools/src/test/python/dlpx/virtualization/_internal/test_package_util.py]
31+
search = DVP_VERSION = '{current_version}'
32+
replace = DVP_VERSION = '{new_version}'
33+

‎.github/workflows/pre-commit.yml

Lines changed: 34 additions & 126 deletions
Original file line numberDiff line numberDiff line change
@@ -1,158 +1,66 @@
1-
name: Pre-commit actions for Delphix Virtualization SDK
1+
#
2+
# Copyright (c) 2020, 2022 by Delphix. All rights reserved.
3+
#
24

3-
on: [pull_request]
5+
name: "Pre-commit actions for Delphix Virtualization SDK"
46

5-
jobs:
6-
pytest27:
7-
name: Test ${{ matrix.package }} on ${{ matrix.os }} using pytest (Python 2.7)
8-
runs-on: ${{ matrix.os }}
9-
strategy:
10-
max-parallel: 4
11-
matrix:
12-
python-version: [2.7]
13-
os: [ubuntu-latest, macos-10.15, windows-latest]
14-
package: [common, libs, platform]
15-
16-
steps:
17-
- name: Checkout ${{ matrix.package }} project
18-
uses: actions/checkout@v3
19-
20-
- name: Set up Python ${{ matrix.python-version }}
21-
uses: actions/setup-python@v3
22-
with:
23-
python-version: ${{ matrix.python-version }}
24-
25-
- name: Install ${{ matrix.package }} dependencies
26-
working-directory: ${{ matrix.package }}
27-
run: |
28-
python -m pip install --upgrade pip
29-
pip install -r requirements.txt --find-links https://test.pypi.org/simple/dvp-api/
30-
31-
- name: Install ${{ matrix.package }} project
32-
working-directory: ${{ matrix.package }}
33-
run: |
34-
pip install . --find-links https://test.pypi.org/simple/dvp-api/
35-
36-
- name: Test ${{ matrix.package }} project with pytest
37-
working-directory: ${{ matrix.package }}
38-
run: |
39-
python -m pytest src/test/python
7+
on:
8+
# It is important to check the commits done to master, develop and release branches only.
9+
pull_request:
10+
branches:
11+
- master
12+
- develop
13+
- release
4014

15+
jobs:
4116
pytest38:
4217
name: Test ${{ matrix.package }} on ${{ matrix.os }} using pytest (Python 3.8)
4318
runs-on: ${{ matrix.os }}
4419
strategy:
4520
max-parallel: 4
4621
matrix:
47-
python-version: [3.8]
48-
os: [ubuntu-latest, macos-latest, windows-latest]
49-
package: [common, libs, platform, tools]
50-
51-
steps:
52-
- name: Checkout ${{ matrix.package }} project
53-
uses: actions/checkout@v1
54-
55-
- name: Set up Python ${{ matrix.python-version }}
56-
uses: actions/setup-python@v1
57-
with:
58-
python-version: ${{ matrix.python-version }}
59-
60-
- name: Install ${{ matrix.package }} dependencies
61-
working-directory: ${{ matrix.package }}
62-
run: |
63-
python -m pip install --upgrade pip
64-
pip install -r requirements.txt --find-links https://test.pypi.org/simple/dvp-api/
65-
66-
- name: Install ${{ matrix.package }} project
67-
working-directory: ${{ matrix.package }}
68-
run: |
69-
pip install . --find-links https://test.pypi.org/simple/dvp-api/
70-
71-
- name: Test ${{ matrix.package }} project with pytest
72-
working-directory: ${{ matrix.package }}
73-
run: |
74-
python -m pytest src/test/python
75-
76-
lintpython27:
77-
name: Lint ${{ matrix.package }} - Python27
78-
79-
runs-on: ubuntu-latest
80-
strategy:
81-
max-parallel: 4
82-
matrix:
83-
package: [common, libs, platform]
22+
python-version: [ 3.8 ]
23+
os: [ ubuntu-latest, macos-latest, windows-latest ]
24+
package: [ common, libs, platform, tools ]
8425

8526
steps:
86-
- name: Checkout ${{ matrix.package }}
87-
uses: actions/checkout@v1
27+
- name: Checkout ${{ matrix.package }} project
28+
uses: actions/checkout@v3
8829

89-
- name: Set up Python 2.7
90-
uses: actions/setup-python@v1
30+
- name: Set up Python ${{ matrix.python-version }}
31+
uses: actions/setup-python@v4
9132
with:
92-
python-version: 2.7
33+
python-version: ${{ matrix.python-version }}
9334

94-
- name: Install flake8
35+
- name: Install ${{ matrix.package }} dependencies
36+
working-directory: ${{ matrix.package }}
9537
run: |
9638
python -m pip install --upgrade pip
97-
pip install flake8
39+
pip install -r requirements.txt --find-links https://test.pypi.org/simple/dvp-api/
9840
99-
- name: Run flake8 on src directory
41+
- name: Install ${{ matrix.package }} project
10042
working-directory: ${{ matrix.package }}
101-
run: python -m flake8 src/main/python --max-line-length 88
43+
run: |
44+
pip install . --find-links https://test.pypi.org/simple/dvp-api/
10245
103-
- name: Run flake8 on test directory
46+
# Run all the test cases part of the package.
47+
- name: Test ${{ matrix.package }} project with pytest
10448
working-directory: ${{ matrix.package }}
105-
run: python -m flake8 src/test/python --max-line-length 88
106-
107-
lintpython38:
108-
name: Lint ${{ matrix.package }} - Python38
109-
110-
runs-on: ubuntu-latest
111-
strategy:
112-
max-parallel: 4
113-
matrix:
114-
package: [common, libs, platform, tools]
115-
116-
steps:
117-
- name: Checkout ${{ matrix.package }}
118-
uses: actions/checkout@v1
119-
120-
- name: Set up Python 3.8
121-
uses: actions/setup-python@v1
122-
with:
123-
python-version: 3.8
49+
run: |
50+
python -m pytest src/test/python
12451
52+
# Install flake8 and run linting on src and test for linting if OS is ubuntu.
12553
- name: Install flake8
54+
if: ${{ matrix.os == 'ubuntu-latest' }}
12655
run: |
127-
python -m pip install --upgrade pip
12856
pip install flake8
12957
13058
- name: Run flake8 on src directory
59+
if: ${{ matrix.os == 'ubuntu-latest' }}
13160
working-directory: ${{ matrix.package }}
13261
run: python -m flake8 src/main/python --max-line-length 88
13362

13463
- name: Run flake8 on test directory
64+
if: ${{ matrix.os == 'ubuntu-latest' }}
13565
working-directory: ${{ matrix.package }}
13666
run: python -m flake8 src/test/python --max-line-length 88
137-
138-
#format:
139-
#name: Check format ${{ matrix.package}}
140-
141-
#runs-on: ubuntu-latest
142-
#strategy:
143-
#max-parallel: 4
144-
#matrix:
145-
#package: [common, libs, platform, tools]
146-
147-
#steps:
148-
#- uses: actions/checkout@v1
149-
150-
#- name: Check src format
151-
#uses: lgeiger/black-action@v1.0.1
152-
#with:
153-
#args: "${{ matrix.package }}/src/main/python -t py27 --check"
154-
155-
#- name: Check test format
156-
#uses: lgeiger/black-action@v1.0.1
157-
#with:
158-
#args: "${{ matrix.package }}/src/test/python -t py27 --check"

‎.github/workflows/publish-docs.yml

Lines changed: 35 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,7 @@
1+
#
2+
# Copyright (c) 2020, 2022 by Delphix. All rights reserved.
3+
#
4+
15
name: Publish docs to GitHub pages
26

37
on:
@@ -16,42 +20,52 @@ jobs:
1620
runs-on: ubuntu-latest
1721
strategy:
1822
matrix:
19-
python-version: [3.7]
20-
package: [docs]
21-
repository: ['delphix/virtualization-sdk']
23+
python-version: [ 3.7 ]
24+
package: [ docs ]
25+
repository: [ 'delphix/virtualization-sdk' ]
2226

2327
steps:
28+
###
29+
# Only a single commit is fetched by default, for the ref/SHA that triggered the workflow.
30+
# Set fetch-depth: 0 to fetch all history for all branches and tags.
31+
###
2432
- uses: actions/checkout@v2
2533
with:
26-
# Only a single commit is fetched by default, for the ref/SHA that triggered the workflow.
27-
# Set fetch-depth: 0 to fetch all history for all branches and tags.
2834
fetch-depth: 0
35+
2936
- name: Set up Python ${{ matrix.python-version }}
3037
uses: actions/setup-python@v2
3138
with:
3239
python-version: ${{ matrix.python-version }}
40+
41+
###
42+
# Set CURRENT_BRANCH environment variable to the current branch name. Refrain from using 'set-env' as
43+
# GitHub has identified the command as a moderate security vulnerability.
44+
###
3345
- name: Set the current branch environment variable
3446
working-directory: ${{ matrix.package }}
3547
run: |
3648
CURRENT_BRANCH_VAR=${GITHUB_REF#refs/heads/}
3749
echo "Current branch: $CURRENT_BRANCH_VAR"
38-
# Set CURRENT_BRANCH environment variable to the current branch name. Refrain from using 'set-env' as
39-
# GitHub has identified the command as a moderate security vulnerability.
4050
echo "CURRENT_BRANCH=$(echo $CURRENT_BRANCH_VAR)" >> $GITHUB_ENV
51+
52+
# Display all remote branches
4153
- name: Display all remote branches
4254
working-directory: ${{ matrix.package }}
43-
run: |
44-
# Display all remote branches
45-
git branch -r
55+
run: git branch -r
56+
57+
###
58+
# Get only docs branches, extract the "docs/x.y.z" part, sort them in descending order, and get the first one.
59+
# Set the LATEST_DOCS_BRANCH environment variable. Refrain from using 'set-env' as GitHub has identified the
60+
# command as a moderate security vulnerability.
61+
###
4662
- name: Set the latest docs branch environment variable
4763
working-directory: ${{ matrix.package }}
4864
run: |
49-
# Get only docs branches, extract the "docs/x.y.z" part, sort them in descending order, and get the first one.
5065
LATEST_DOCS_BRANCH_VAR=$(git branch -r | grep -e ".*\/*docs\/[0-9].[0-9].[0-9]" | sed -n "s/.*\/*\(docs\/[0-9].[0-9].[0-9]\).*/\1/p" | sort -r | head -n 1)
5166
echo "Latest docs branch: $LATEST_DOCS_BRANCH_VAR"
52-
# Set the LATEST_DOCS_BRANCH environment variable. Refrain from using 'set-env' as GitHub has identified the
53-
# command as a moderate security vulnerability.
5467
echo "LATEST_DOCS_BRANCH=$(echo $LATEST_DOCS_BRANCH_VAR)" >> $GITHUB_ENV
68+
5569
- name: Check that the current branch is the latest docs branch, fail otherwise
5670
working-directory: ${{ matrix.package }}
5771
run: |
@@ -61,16 +75,19 @@ jobs:
6175
else
6276
echo "The action is running on the latest docs branch."
6377
fi
78+
6479
- name: Install Pipenv
6580
uses: dschep/install-pipenv-action@v1
81+
6682
- name: Install dependencies for building documenation
6783
working-directory: ${{ matrix.package }}
68-
run: |
69-
pipenv install
84+
run: pipenv install
85+
7086
- name: Build documentation
7187
working-directory: ${{ matrix.package }}
72-
run: |
73-
pipenv run mkdocs build --clean
88+
run: pipenv run mkdocs build --clean
89+
90+
# Docs will be pushed to developer.delphix.com if the repository is delphix/virtualization-sdk.
7491
- name: Deploy the contents of docs/site to gh-pages (developer.delphix.com) 🚀
7592
if: ${{ github.repository == matrix.repository }}
7693
uses: peaceiris/actions-gh-pages@v3
@@ -81,6 +98,7 @@ jobs:
8198
user_name: "github-actions[bot]"
8299
user_email: "github-actions[bot]@users.noreply.github.com"
83100
cname: developer.delphix.com
101+
84102
- name: Deploy the contents of docs/site to personal gh-pages 🚀
85103
if: ${{ github.repository != matrix.repository }}
86104
uses: peaceiris/actions-gh-pages@v3

‎README-dev.md

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Copyright (c) 2019 by Delphix. All rights reserved.
1+
# Copyright (c) 2019, 2022 by Delphix. All rights reserved.
22

33
# Delphix Virtualization SDK
44

@@ -63,9 +63,8 @@ To install the SDK, follow these steps:
6363
One of the SDK dependencies - dvp-api - is currently hosted on [TestPyPi](https://test.pypi.org/project/dvp-api/).
6464
By default `pip` looks at pypi.org for packages to install. In order to successfully install the SDK, you have to
6565
configure pip to search an additional package repository - test.pypi.org.
66-
2. Go into one of the package directories (common, dvp, libs, platform, tools) and run the commands below.
67-
3. Install the package's development dependencies: `pip install -r requirements.txt`.
68-
4. Install the package itself (use `-e` flag if you want to install the package in editable mode): `pip install .`.
66+
2. To build the project, run `sh bin/build_project.sh -b`.
67+
3. For more information on the script options, use `sh bin/build_project.sh -h`.
6968
7069
7170
### CLI changes
@@ -111,9 +110,8 @@ Go into one of the package directories (common, dvp, libs, platform, tools) and
111110
112111
1. Install the package's development dependencies and package itself by following the directions in [the SDK installation section](#installing-the-sdk-from-source).
113112
2. Run unit tests: `python -m pytest src/test/python`.
114-
115-
There's no way to locally run unit tests in all packages with one command. However, they will be run automatically
116-
through GitHub Actions when you open a pull request. You can always open a draft pull request
113+
3. To run unit tests in all packages with one command, run `sh bin/build_project.sh -t`.
114+
4. For more information on the script options, use `sh bin/build_project.sh -h`.
117115
118116
### Manual testing
119117

0 commit comments

Comments
 (0)