Skip to content
This repository was archived by the owner on Feb 27, 2025. It is now read-only.

Upgrade to spark 3.5 #269

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 13 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
@@ -9,7 +9,7 @@ This library contains the source code for the Apache Spark Connector for SQL Ser

[Apache Spark](https://spark.apache.org/) is a unified analytics engine for large-scale data processing.

There are three version sets of the connector available through Maven, a 2.4.x, a 3.0.x and a 3.1.x compatible version. All versions can be found [here](https://search.maven.org/search?q=spark-mssql-connector) and can be imported using the coordinates below:
All available versions can be found [here](https://search.maven.org/search?q=spark-mssql-connector) and can be imported using the coordinates below:

| Connector | Maven Coordinate | Scala Version |
|----------------------------------|--------------------------------------------------------|---------------|
@@ -18,6 +18,7 @@ There are three version sets of the connector available through Maven, a 2.4.x,
| Spark 3.1.x compatible connector | `com.microsoft.azure:spark-mssql-connector_2.12:1.2.0` | 2.12 |
| Spark 3.3.x compatible connector | `com.microsoft.azure:spark-mssql-connector_2.12:1.3.0` | 2.12 |
| Spark 3.4.x compatible connector | `com.microsoft.azure:spark-mssql-connector_2.12:1.4.0` | 2.12 |
| Spark 3.5.x compatible connector | `com.microsoft.azure:spark-mssql-connector_2.12:1.5.0` | 2.12 |

## Current Releases

@@ -27,6 +28,10 @@ The latest Spark 3.0.x compatible connector is on v1.1.0.

The latest Spark 3.1.x compatible connector is on v1.2.0.

The latest Spark 3.4.x compatible connector is on v1.4.0.

The latest Spark 3.5.x compatible connector is on v1.5.0.

For main changes from previous releases and known issues please refer to [CHANGELIST](docs/CHANGELIST.md)

## Supported Features
@@ -37,13 +42,13 @@ For main changes from previous releases and known issues please refer to [CHANGE
* Reliable connector support for Sql Server Single Instance


| Component | Versions Supported |
|--------------------------------------|----------------------------|
| Apache Spark | 2.4.x, 3.0.x, 3.1.x, 3.3.x |
| Scala | 2.11, 2.12 |
| Microsoft JDBC Driver for SQL Server | 8.4.1 |
| Microsoft SQL Server | SQL Server 2008 or later |
| Azure SQL Databases | Supported |
| Component | Versions Supported |
|--------------------------------------|------------------------------------------|
| Apache Spark | 2.4.x, 3.0.x, 3.1.x, 3.3.x, 3.4.x, 3.5.x |
| Scala | 2.11, 2.12 |
| Microsoft JDBC Driver for SQL Server | 8.4.1 |
| Microsoft SQL Server | SQL Server 2008 or later |
| Azure SQL Databases | Supported |

*Note: Azure Synapse (Azure SQL DW) use is not tested with this connector. While it may work, there may be unintended consequences.*

12 changes: 6 additions & 6 deletions pom.xml
Original file line number Diff line number Diff line change
@@ -2,9 +2,9 @@
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0">
<modelVersion>4.0.0</modelVersion>
<groupId>com.microsoft.azure</groupId>
<artifactId>spark-mssql-connector</artifactId>
<artifactId>spark-mssql-connector_${scala.binary.version}</artifactId>
<packaging>jar</packaging>
<version>1.4.0</version>
<version>1.5.0</version>
<name>${project.groupId}:${project.artifactId}</name>
<description>The Apache Spark Connector for SQL Server and Azure SQL is a high-performance connector that enables you to use transactional data in big data analytics and persists results for ad-hoc queries or reporting.</description>
<url>https://github.com/microsoft/sql-spark-connector</url>
@@ -201,20 +201,20 @@
</build>
<profiles>
<profile>
<id>spark34</id>
<id>spark35</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<scala.binary.version>2.12</scala.binary.version>
<scala.version>2.12.11</scala.version>
<spark.version>3.4.0</spark.version>
<scala.version>2.12.20</scala.version>
<spark.version>3.5.0</spark.version>
</properties>
<dependencies>
<dependency>
<groupId>org.scalatest</groupId>
<artifactId>scalatest_${scala.binary.version}</artifactId>
<version>3.2.6</version>
<version>3.2.19</version>
<scope>test</scope>
</dependency>
<dependency>
6 changes: 3 additions & 3 deletions test/scala_test/pom.xml
Original file line number Diff line number Diff line change
@@ -123,14 +123,14 @@
</build>
<profiles>
<profile>
<id>spark34</id>
<id>spark35</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<scala.binary.version>2.12</scala.binary.version>
<scala.version>2.12.11</scala.version>
<spark.version>3.4.0</spark.version>
<scala.version>2.12.20</scala.version>
<spark.version>3.5.0</spark.version>
</properties>
</profile>
</profiles>