Skip to content

Support VECTOR data type #1549

Open
Open
@bgrainger

Description

@bgrainger

Discussed in #1548, originally posted by @harrison314.

Add support for the VECTOR(n) type, which is available since MariaDB 11.7.

https://mariadb.com/kb/en/vector-overview/

I've solved it with a workaround:

    internal static class MySqlExtensions
    {
        public static MySqlParameter AddWithVectorValue(this MySqlParameterCollection parameters, string name, ReadOnlySpan<float> vector)
        {
            byte[] buffer = new byte[vector.Length * 4];

            for (int i = 0; i < vector.Length; i++)
            {
                BinaryPrimitives.WriteSingleLittleEndian(buffer.AsSpan(i * 4), vector[i]);
            }

            MySqlParameter p = parameters.Add(name, System.Data.DbType.Binary);
            p.Value = buffer;

            return p;
        }
    }
```</div>

Activity

changed the title [-]Support MariaDB `VECTOR` data type[/-] [+]Support `VECTOR` data type[/+] on Feb 23, 2025
bgrainger

bgrainger commented on Feb 23, 2025

@bgrainger
MemberAuthor
bgrainger

bgrainger commented on Feb 23, 2025

@bgrainger
MemberAuthor

The underlying type should probably be float[] (not ReadOnlySpan<float> as that can't be boxed for MySqlParameter.Value).

Possibly could add GetFieldValue<ReadOnlySpan<float>> to get a view of the row that's been retrieved.

added a commit that references this issue on Feb 23, 2025
bgrainger

bgrainger commented on Feb 24, 2025

@bgrainger
MemberAuthor

The VECTOR data type was added in Connector/NET 8.4.0.

bgrainger

bgrainger commented on Feb 24, 2025

@bgrainger
MemberAuthor

Preliminary tests with MariaDB 11.7 indicate that it doesn't use a dedicated on-the-wire type for VECTOR(n), so all results come back as BLOBs. (Possibly waiting on https://jira.mariadb.org/browse/MDEV-35831.) This means that MySqlDataReader.GetValue(n) will return byte[] and users will have to use Buffer.BlockCopy or MemoryMarshal.Cast<> to get a float[].

self-assigned this
on Feb 24, 2025
harrison314

harrison314 commented on Feb 24, 2025

@harrison314

Possibly could add GetFieldValue<ReadOnlySpan<float>> to get a view of the row that's been retrieved.

I think it's enough to implement float[], because the array will always be allocated anyway. I used ReadOnlySpan<float> in my helper method because it is a generic type. But it may not be suitable for implementing methods in the MySQL connector.

This means that MySqlDataReader.GetValue(n) will return byte[] and users will have to use Buffer.BlockCopy or MemoryMarshal.Cast<> to get a float[].

MemoryMarshal.Cast<> may not work on all processors due to big and little endian.

harrison314

harrison314 commented on May 1, 2025

@harrison314

Don't you plan to release at least a preview nuget with vector support in the near future?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

    Development

    No branches or pull requests

      Participants

      @bgrainger@harrison314

      Issue actions

        Support `VECTOR` data type · Issue #1549 · mysql-net/MySqlConnector