Merge pull request #542 from SumoSourabh/CP-11887

CP-11887 Merge Python 3.11 changes to master from develop branch

Author: SumoSourabh

.bumpversion.cfg

@@ -1,5 +1,5 @@
 [bumpversion]
-current_version = 4.1.0
+current_version = 5.0.0
 commit = False
 tag = False
 parse = (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+)(\.(?P<release>[a-z]+)(?P<dev>\d+))?

.github/workflows/pre-commit.yml

@@ -13,13 +13,13 @@ on:
       - release
 
 jobs:
-  pytest38:
-    name: Test ${{ matrix.package }} on ${{ matrix.os }} using pytest (Python 3.8)
+  pytest311:
+    name: Test ${{ matrix.package }} on ${{ matrix.os }} using pytest (Python 3.11)
     runs-on: ${{ matrix.os }}
     strategy:
       max-parallel: 4
       matrix:
-        python-version: [ 3.8 ]
+        python-version: [ 3.11 ]
         os: [ ubuntu-latest, macos-latest, windows-latest ]
         package: [ common, libs, platform, tools ]
 

.github/workflows/publish-docs.yml

@@ -20,7 +20,7 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: [ 3.8 ]
+        python-version: [ 3.11 ]
         package: [ docs ]
         repository: [ 'delphix/virtualization-sdk' ]
 

.github/workflows/publish-python-packages.yml

@@ -17,7 +17,7 @@ jobs:
     strategy:
       max-parallel: 5
       matrix:
-        python-version: [3.8]
+        python-version: [3.11]
         package: [common, dvp, libs, platform, tools]
 
     steps:

README.md

@@ -11,7 +11,8 @@ The latest user documentation can be found [here](https://developer.delphix.com)
 
 - macOS 10.14+, Ubuntu 16.04+, or Windows 10
 - Python 2.7 (vSDK 3.1.0 and earlier)
-- Python 3.8 (vSDK 4.0.0 and later)
+- Python 3.8 (vSDK 4.1.0 and earlier)
+- Python 3.11 (vSDK 5.0.0 and later)
 - Java 7+
 - A Delphix Engine of an [appropriate version](/References/Version_Compatibility.md)
 

common/setup.cfg

@@ -12,9 +12,9 @@ long_description_content_type: text/markdown
 classifiers:
     Development Status :: 5 - Production/Stable
     Programming Language :: Python
-    Programming Language :: Python :: 3.8
+    Programming Language :: Python :: 3.11
     License :: OSI Approved :: Apache Software License
     Operating System :: OS Independent
 
 [options]
-requires_python: >=3.8, <3.9
+requires_python: >=3.11, <3.12

common/setup.py

@@ -4,7 +4,7 @@
 PYTHON_SRC = 'src/main/python'
 
 install_requires = [
-    "dvp-api == 1.8.0",
+    "dvp-api == 1.9.0",
     "six >= 1.16, < 1.17",
 ]
 
@@ -16,5 +16,5 @@
                  install_requires=install_requires,
                  package_dir={'': PYTHON_SRC},
                  packages=setuptools.find_packages(PYTHON_SRC),
-                 python_requires='>=3.8, <3.9',
+                 python_requires='>=3.11, <3.12',
                  )

common/src/main/python/dlpx/virtualization/common/VERSION

@@ -1 +1 @@
-4.1.0
+5.0.0

docs/Pipfile

@@ -10,6 +10,6 @@ markdown-include = "==0.8.1"
 mkdocs-awesome-pages-plugin = "==2.9.1"
 
 [requires]
-python_version = "3.8"
+python_version = "3.11"
 
 [dev-packages]

docs/Pipfile.lock

@@ -2,7 +2,7 @@
 
 All Python modules inside of `srcDir` can be imported just as they would be if the plugin was executing locally. When a plugin operation is executed `srcDir` is the current working directory so all imports need to be relative to `srcDir` regardless of the path of the module doing the import.
 
-Please refer to Python's [documentation on modules](https://docs.python.org/3.8/tutorial/modules.html#modules) to learn more about modules and imports.
+Please refer to Python's [documentation on modules](https://docs.python.org/3.11/tutorial/modules.html#modules) to learn more about modules and imports.
 
 ## Example
 

docs/docs/Best_Practices/Code_Sharing.md

@@ -4,7 +4,7 @@ To execute a PowerShell or Bash script or Expect script on a remote host, you mu
 
 [pkgutil](https://docs.python.org/2/library/pkgutil.html) is part of the standard Python library. The method that is applicable to resources is [pkgutil.get_data](https://docs.python.org/2/library/pkgutil.html#pkgutil.get_data).
 
-When developing a plugin in Python3, it is instead suggested to use the newer `importlib.resources`. This package is part of the standard Python 3 library. The method that is applicable to resources is [resources.read_text](https://docs.python.org/3.8/library/importlib.html#importlib.resources.read_text), which accepts the same arguments as `pkgutil.get_data`.
+When developing a plugin in Python3, it is instead suggested to use the newer `importlib.resources`. This package is part of the standard Python 3 library. The method that is applicable to resources is [resources.read_text](https://docs.python.org/3.11/library/importlib.resources.html#importlib.resources.read_text), which accepts the same arguments as `pkgutil.get_data`.
 
 ### Basic Usage
 
@@ -106,7 +106,7 @@ The contents of `src/resources/platform/get_date.sh` can be retrieved with:
 script_content = pkgutil.get_data('resources.platform', 'get_date.sh')
 ```
 
-In a Python 3.8 plugin, the suggested approach is:
+In a Python 3.8 and Python 3.11 plugin, the suggested approach is:
 
 ```python
 script_content = resources.read_text('resources.platform', 'get_date.sh')

docs/docs/Best_Practices/Managing_Scripts_For_Remote_Execution.md

@@ -13,7 +13,8 @@ The platform and libs modules expose objects and methods needed to develop a plu
 
 - macOS 10.14+, Ubuntu 16.04+, or Windows 10
 - Python 2.7 (vSDK 3.1.0 and earlier)
-- Python 3.8 (vSDK 4.0.0 and later)
+- Python 3.8 (vSDK 4.1.0 and earlier)
+- Python 3.11 (vSDK 5.0.0 and later)
 - Java 7+
 - A Delphix Engine of an [appropriate version](References/Version_Compatibility.md)
 - An active internet connection to download packages from [PyPI](https://pypi.org/)
@@ -33,11 +34,13 @@ $ pip install dvp
 
      If using vSDK 3.1.0 or earlier, the virtual environment needs to use Python 2.7.
 
-     If using vSDK 4.0.0 or earlier, the virtual environment needs to use Python 3.8.
+     If using vSDK 4.1.0 or earlier, the virtual environment needs to use Python 3.8.
+
+     If using vSDK 5.0.0 or later, the virtual environment needs to use Python 3.11.
 
 	 This is configured when creating the virtualenv:
 
-	 ```$ virtualenv -p /path/to/python2.7/binary ENV``` or ```$ virtualenv -p /path/to/python3.8/binary ENV```
+	 ```$ virtualenv -p /path/to/python2.7/binary ENV``` or ```$ virtualenv -p /path/to/python3.8/binary ENV``` or ```$ virtualenv -p /path/to/python3.11/binary ENV```
 
 To install a specific version of the SDK run:
 

docs/docs/Getting_Started.md

@@ -6,10 +6,10 @@ The Virtualization Platform keeps plugin-specific log files. A plugin can, at an
 
 ## Overview
 
-The Virtualization Platform integrates with Python's built-in [logging framework](https://docs.python.org/3.8/library/logging.html). A special [Handler](https://docs.python.org/3.8/library/logging.html#handler-objects) is exposed by the platform at `dlpx.virtualization.libs.PlatformHandler`. This handler needs to be added to the Python logger your plugin creates. Logging statements made through Python's logging framework will then be routed to the platform.
+The Virtualization Platform integrates with Python's built-in [logging framework](https://docs.python.org/3.11/library/logging.html). A special [Handler](https://docs.python.org/3.11/library/logging.html#handler-objects) is exposed by the platform at `dlpx.virtualization.libs.PlatformHandler`. This handler needs to be added to the Python logger your plugin creates. Logging statements made through Python's logging framework will then be routed to the platform.
 
 ## Basic Setup
- Below is the absolute minimum needed to setup logging for the platform. Please refer to Python's [logging documentation](https://docs.python.org/3.8/library/logging.html) and the [example below](#example) to better understand how it can be customized.
+ Below is the absolute minimum needed to setup logging for the platform. Please refer to Python's [logging documentation](https://docs.python.org/3.11/library/logging.html) and the [example below](#example) to better understand how it can be customized.
 
 ```python
 import logging
@@ -40,7 +40,7 @@ logger.setLevel(logging.DEBUG)
     There is a limit to how much data can be stored within a log message. See [Message Limits](../Best_Practices/Message_Limits.md) for details.
 
 ## Usage
-Once the `PlatformHandler` has been added to the logger, logging is done with Python's [Logger](https://docs.python.org/3.8/library/logging.html#logger-objects) object. Below is a simple example including the basic setup code used above:
+Once the `PlatformHandler` has been added to the logger, logging is done with Python's [Logger](https://docs.python.org/3.11/library/logging.html#logger-objects) object. Below is a simple example including the basic setup code used above:
 
 ```python
 import logging
@@ -181,7 +181,7 @@ Download a support bundle by going to **Help** > **Support Logs**  and select **
 
 ## Logging Levels
 
-Python has a number of [preset logging levels](https://docs.python.org/3.8/library/logging.html#logging-levels) and allows for custom ones as well. Since logging on the Virtualization Platform uses the `logging` framework, log statements of all levels are supported.
+Python has a number of [preset logging levels](https://docs.python.org/3.11/library/logging.html#logging-levels) and allows for custom ones as well. Since logging on the Virtualization Platform uses the `logging` framework, log statements of all levels are supported.
 
 However, the Virtualization Platform will map all logging levels into three files: `debug.log`, `info.log`, and `error.log` in the following way:
 

docs/docs/References/Logging.md

@@ -111,7 +111,7 @@ response = libs.run_bash(connection, command)
  # Execute script on remote host
  response = libs.run_bash(direct_source.connection, script_content)
 ```
-###### Python 3.8 recommended approach
+###### Python 3.8 and Python 3.11 recommended approach
 ```python
 
  from importlib import resources

docs/docs/References/Platform_Libraries.md

@@ -5,22 +5,22 @@ The name of the file can be specified during the build. By default, the build lo
 
 ## Fields
 
-|Field Name|Required|Type|Description|
-|----------|:------:|:--:|-----------|
-|id|Y|string|The unique id of the plugin in a valid UUID format.|
-|name|N|string|The display name of the plugin. This will be used in the UI. If it is not specified name will be equal to id.|
-|externalVersion|N|string|The plugin's [external version](../Versioning_And_Upgrade/Versioning.md#external-version). This is a freeform string. If it is not supplied, the build number is used as an external version.
-|buildNumber|Y|string|The plugin's [build number](../Versioning_And_Upgrade/Versioning.md#build-number). This string must conform to the format described [here](../Versioning_And_Upgrade/Versioning.md#build-number-format-rules).
-|hostTypes|Y|list|The host type that the plugin supports. Either `UNIX` or `WINDOWS`.|
-|schemaFile|Y|string|The path to the JSON file that contains the [plugin's schema definitions](Schemas.md).<br><br>This path can be absolute or relative to the directory containing the plugin config file.|
-|srcDir|Y|string|The path to the directory that contains the source code for the plugin. During execution of a plugin operation, this directory will be the current working directory of the Python interpreter. Any modules or resources defined outside of this directory will be inaccessible at runtime.<br><br>This path can be absolute or relative to the directory containing the plugin config file.|
-|entryPoint|Y|string|A fully qualified Python symbol that points to the `dlpx.virtualization.platform.Plugin` object that defines the plugin.<br><br>It must be in the form `importable.module:object_name` where `importable.module` is in `srcDir`.|
-|manualDiscovery|N|boolean|True if the plugin supports manual discovery of source config objects. The default value is `true`.|
-|pluginType|Y|enum|The ingestion strategy of the plugin. Can be either `STAGED` or `DIRECT`.|
-|language|Y|enum|Must be `PYTHON38`.|
-|defaultLocale|N|enum|The locale to be used by the plugin if the Delphix user does not specify one. Plugin messages will be displayed in this locale by default. The default value is `en-us`.|
-|rootSquashEnabled|N|boolean|This dictates whether "root squash" is enabled on NFS mounts for the plugin (i.e. whether the `root` user on remote hosts has access to the NFS mounts). Setting this to `false` allows processes usually run as `root`, like Docker daemons, access to the NFS mounts. The default value is `true`. This field only applies to Unix hosts.|
-|extendedStartStopHooks|N|boolean|This controls whether the user's pre-start and post-start hooks will run during enable operations (and, likewise, whether pre-stop and post-stop hooks will run during disable operations). The default value is `false`.|
+|Field Name|Required|Type| Description                                                                                                                                                                                                                                                                                                                                                                                  |
+|----------|:------:|:--:|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+|id|Y|string| The unique id of the plugin in a valid UUID format.                                                                                                                                                                                                                                                                                                                                          |
+|name|N|string| The display name of the plugin. This will be used in the UI. If it is not specified name will be equal to id.                                                                                                                                                                                                                                                                                |
+|externalVersion|N|string| The plugin's [external version](../Versioning_And_Upgrade/Versioning.md#external-version). This is a freeform string. If it is not supplied, the build number is used as an external version.                                                                                                                                                                                                
+|buildNumber|Y|string| The plugin's [build number](../Versioning_And_Upgrade/Versioning.md#build-number). This string must conform to the format described [here](../Versioning_And_Upgrade/Versioning.md#build-number-format-rules).                                                                                                                                                                               
+|hostTypes|Y|list| The host type that the plugin supports. Either `UNIX` or `WINDOWS`.                                                                                                                                                                                                                                                                                                                          |
+|schemaFile|Y|string| The path to the JSON file that contains the [plugin's schema definitions](Schemas.md).<br><br>This path can be absolute or relative to the directory containing the plugin config file.                                                                                                                                                                                                      |
+|srcDir|Y|string| The path to the directory that contains the source code for the plugin. During execution of a plugin operation, this directory will be the current working directory of the Python interpreter. Any modules or resources defined outside of this directory will be inaccessible at runtime.<br><br>This path can be absolute or relative to the directory containing the plugin config file. |
+|entryPoint|Y|string| A fully qualified Python symbol that points to the `dlpx.virtualization.platform.Plugin` object that defines the plugin.<br><br>It must be in the form `importable.module:object_name` where `importable.module` is in `srcDir`.                                                                                                                                                             |
+|manualDiscovery|N|boolean| True if the plugin supports manual discovery of source config objects. The default value is `true`.                                                                                                                                                                                                                                                                                          |
+|pluginType|Y|enum| The ingestion strategy of the plugin. Can be either `STAGED` or `DIRECT`.                                                                                                                                                                                                                                                                                                                    |
+|language|Y|enum| Must be `PYTHON311`.                                                                                                                                                                                                                                                                                                                                                                         |
+|defaultLocale|N|enum| The locale to be used by the plugin if the Delphix user does not specify one. Plugin messages will be displayed in this locale by default. The default value is `en-us`.                                                                                                                                                                                                                     |
+|rootSquashEnabled|N|boolean| This dictates whether "root squash" is enabled on NFS mounts for the plugin (i.e. whether the `root` user on remote hosts has access to the NFS mounts). Setting this to `false` allows processes usually run as `root`, like Docker daemons, access to the NFS mounts. The default value is `true`. This field only applies to Unix hosts.                                                  |
+|extendedStartStopHooks|N|boolean| This controls whether the user's pre-start and post-start hooks will run during enable operations (and, likewise, whether pre-stop and post-stop hooks will run during disable operations). The default value is `false`.                                                                                                                                                                    |
 
 ## Example
 Assume the following basic plugin structure:
@@ -52,7 +52,7 @@ entryPoint: mongo_runner:mongodb
 srcDir: src/
 schemaFile: schema.json
 pluginType: DIRECT
-language: PYTHON38
+language: PYTHON311
 buildNumber: 0.1.0
 ```
 This is a valid plugin config for the plugin with `manualDiscovery` set to `false` and an `externalVersion` set:
@@ -67,7 +67,7 @@ srcDir: src/
 schemaFile: schema.json
 manualDiscovery: false
 pluginType: DIRECT
-language: PYTHON38
+language: PYTHON311
 externalVersion: "MongoDB 1.0"
 buildNumber: "1"
 ```