Skip to content

feat(sdk): Add a KAS allowlist #2085

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 26 commits into
base: main
Choose a base branch
from
Open

Conversation

elizabethhealy
Copy link
Member

@elizabethhealy elizabethhealy commented Apr 18, 2025

Proposed Changes

  • adds a concept of an allowlist for kas urls during decrypt, if a url from a kao is not allowed then error
  • the allowlist takes in url formatted strings and reformats to scheme://host:port and adds that to a set, if no scheme provided, will default to https
  • the default contents of the kas allow list is the list of kases from the key access registry + the platform url
  • this also adds tdf reader options: withKasAllowlist and WithIgnoreAllowlist -- withKasAllowlist lets you manually specify the kases in your allowlist (not using the registry), and WithIgnoreAllowlist lets you ignore the allowlist entirely
  • currently we are not implementing a cache and just creating the allowlist on each request (expected slow down with queries to key access registry service)

Other additions to support the above:

  • the concept of nano reader options - we have nano options on encrypt but not decrypt, add nano reader config and reader options similar to tdf3
  • ability to pass in decrypt options from bulk decrypt -- previously bulk decrypt did not support decrypt options, ive added ability to specify either tdf3 or nano decrypt options which are passed down to the decrypter

Things to note:

  • the platform url passed into the sdk should now include the scheme, otherwise the allowlist will default to https causing decrypt failures if youre using the platform kas on http

Checklist

  • I have added or updated unit tests
  • I have added or updated integration tests (if appropriate)
  • I have added or updated documentation

Testing Instructions

Copy link
Contributor

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Metric Value
Total Decrypts 100
Successful Decrypts 100
Failed Decrypts 0
Total Time 617.981158ms
Throughput 161.82 requests/second

Copy link
Contributor

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Metric Value
Total Decrypts 100
Successful Decrypts 100
Failed Decrypts 0
Total Time 618.949102ms
Throughput 161.56 requests/second

Copy link
Contributor

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Metric Value
Total Decrypts 100
Successful Decrypts 100
Failed Decrypts 0
Total Time 373.586656ms
Throughput 267.68 requests/second

@opentdf opentdf deleted a comment from github-actions bot Apr 18, 2025
@opentdf opentdf deleted a comment from github-actions bot Apr 18, 2025
@opentdf opentdf deleted a comment from github-actions bot Apr 18, 2025
@opentdf opentdf deleted a comment from github-actions bot Apr 18, 2025
@opentdf opentdf deleted a comment from github-actions bot Apr 18, 2025
Copy link
Contributor

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Metric Value
Total Decrypts 100
Successful Decrypts 100
Failed Decrypts 0
Total Time 364.188788ms
Throughput 274.58 requests/second

@elizabethhealy elizabethhealy marked this pull request as ready for review April 18, 2025 20:39
@elizabethhealy elizabethhealy requested review from a team as code owners April 18, 2025 20:39
@elizabethhealy elizabethhealy requested a review from Copilot April 21, 2025 15:20
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds a KAS allowlist feature to the SDK decryption functionality to restrict which KAS URLs may be used during decryption. Key changes include:

  • Implementation of functions to construct and manage the allowlist in both TDF3 and Nano TDF reader configurations.
  • Integration of allowlist checks in the decryption flows, including BulkDecrypt.
  • Addition of extensive tests to validate the intended allowlist behavior.

Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
service/rttests/rt_test.go Tests that decryption fails when using a disallowed KAS URL.
sdk/tdf_test.go Introduces tests for KAS allowlist functionality in TDF decryption.
sdk/tdf_config_test.go Updates tests to verify correct processing of valid and invalid KAS URLs.
sdk/tdf_config.go Implements allowlist functions and new reader options.
sdk/tdf.go Integrates KAS allowlist checking in the TDF decryption flow.
sdk/nanotdf_config_test.go Adds tests for Nano TDF allowlist options and ignore settings.
sdk/nanotdf_config.go Implements NanoTDFReaderConfig support for KAS allowlist.
sdk/nanotdf.go Adds allowlist verification in the NanoTDF decryption flow.
sdk/bulk.go Updates bulk decryption to incorporate new KAS allowlist options.
Comments suppressed due to low confidence (1)

sdk/tdf_config.go:293

  • The function getKasAddress returns an empty string for URLs without a scheme, which may lead to unintended errors when valid host:port values are provided without a scheme. Consider handling such cases explicitly by inferring a default scheme or adjusting the parsing logic.
if parsedURL.Host == "" && parsedURL.Port() == "" {

Copy link
Contributor

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Metric Value
Total Decrypts 100
Successful Decrypts 100
Failed Decrypts 0
Total Time 362.601427ms
Throughput 275.78 requests/second

Copy link
Contributor

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Metric Value
Total Decrypts 100
Successful Decrypts 100
Failed Decrypts 0
Total Time 384.798199ms
Throughput 259.88 requests/second

Copy link
Contributor

@jentfoo jentfoo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A good start, but I have some questions before merging

Copy link
Contributor

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Metric Value
Total Decrypts 100
Successful Decrypts 100
Failed Decrypts 0
Total Time 617.786445ms
Throughput 161.87 requests/second

Copy link
Contributor

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Metric Value
Total Decrypts 100
Successful Decrypts 100
Failed Decrypts 0
Total Time 379.071221ms
Throughput 263.80 requests/second

Copy link
Contributor

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Metric Value
Total Decrypts 100
Successful Decrypts 100
Failed Decrypts 0
Total Time 366.481696ms
Throughput 272.86 requests/second

Copy link
Contributor

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Metric Value
Total Decrypts 100
Successful Decrypts 100
Failed Decrypts 0
Total Time 380.149503ms
Throughput 263.05 requests/second

Copy link
Contributor

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Metric Value
Total Decrypts 100
Successful Decrypts 0
Failed Decrypts 100
Total Time 11.462754ms
Throughput 0.00 requests/second

Error Summary

Error Message Occurrences
splitKey.unable to reconstruct split key: map[{http://localhost:8080 }:kao unwrap failed for split {http://localhost:8080 }: KasAllowlist: kas url http://localhost:8080 is not allowed]
kao unwrap failed for split {http://localhost:8080 }: KasAllowlist: kas url http://localhost:8080 is not allowed 100 occurrences

Copy link
Contributor

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Metric Value
Total Decrypts 100
Successful Decrypts 100
Failed Decrypts 0
Total Time 373.939186ms
Throughput 267.42 requests/second

Copy link
Contributor

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Metric Value
Total Decrypts 100
Successful Decrypts 100
Failed Decrypts 0
Total Time 398.865113ms
Throughput 250.71 requests/second

Copy link
Contributor

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Metric Value
Total Decrypts 100
Successful Decrypts 100
Failed Decrypts 0
Total Time 379.184402ms
Throughput 263.72 requests/second

Copy link
Contributor

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Metric Value
Total Decrypts 100
Successful Decrypts 100
Failed Decrypts 0
Total Time 366.916276ms
Throughput 272.54 requests/second

Copy link
Contributor

@jentfoo jentfoo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks generally pretty good, remaining comments are relatively minor

@elizabethhealy elizabethhealy requested a review from jentfoo April 25, 2025 15:00
Copy link
Contributor

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Metric Value
Total Decrypts 100
Successful Decrypts 100
Failed Decrypts 0
Total Time 373.721998ms
Throughput 267.58 requests/second

jentfoo
jentfoo previously approved these changes Apr 25, 2025
@policy-bot-opentdf policy-bot-opentdf bot dismissed jentfoo’s stale review April 25, 2025 20:03

Invalidated by push of bdf6ed4

Copy link
Contributor

Benchmark results, click to expand

Bulk Benchmark Results

Metric Value
Total Decrypts 100
Successful Decrypts 100
Failed Decrypts 0
Total Time 364.411097ms
Throughput 274.42 requests/second

TDF3 Benchmark Results:

Metric Value
Total Requests 5000
Successful Requests 5000
Failed Requests 0
Concurrent Requests 50
Total Time 1m19.318506016s
Average Latency 789.15631ms
Throughput 63.04 requests/second

NANOTDF Benchmark Results:

Metric Value
Total Requests 5000
Successful Requests 0
Failed Requests 5000
Concurrent Requests 50
Total Time 4.442067092s
Throughput 0.00 requests/second

Error Summary:

Error Message Occurrences
ReadNanoTDF error: KasAllowlist: kas url http://localhost:8080/kas is not allowed 5000 occurrences

Standard Benchmark Metrics Skipped or Failed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants