Skip to content

Ensure fetch_ctk action can be used everywhere #600

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 33 commits into from
May 4, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
e7627e3
test installing git for windows
leofang May 2, 2025
562b558
shrink CI
leofang May 2, 2025
c54332b
also install 7z...
leofang May 2, 2025
270b681
skip cuda.bindings for now
leofang May 2, 2025
9e665db
fix
leofang May 2, 2025
38830b2
fix
leofang May 2, 2025
4a2bb6f
fix
leofang May 2, 2025
2bb32a9
skip
leofang May 2, 2025
dc633dc
skip
leofang May 2, 2025
cd69de3
more skip
leofang May 2, 2025
52adef2
add quotes
leofang May 2, 2025
d5c7be2
try this
leofang May 2, 2025
efd6036
try this
leofang May 2, 2025
1c8d7a8
try this
leofang May 2, 2025
4cb69d6
don't escape?
leofang May 2, 2025
10de33c
try this
leofang May 2, 2025
3cfc4e3
--version does not exist
leofang May 2, 2025
fffe85c
switch to fetch_ctk
leofang May 2, 2025
daf26ad
try not to hang
leofang May 2, 2025
0a578be
try not to hang
leofang May 2, 2025
f3fa596
try not to hang; composite action does not support setting default sh…
leofang May 2, 2025
fa6abf0
Merge branch 'main' into fix_win
leofang May 3, 2025
0418045
self extracting + run post install
leofang May 3, 2025
869ff14
block extraction to ensure file visibility
leofang May 3, 2025
ffa239e
clean up
leofang May 3, 2025
1b02e55
ensure cuda related paths are set correctly on windows
leofang May 3, 2025
7a0e053
fix normpath (again) to escape slash
leofang May 3, 2025
96ecae0
DEBUG
leofang May 3, 2025
a73564c
perhaps git-bash is not the right exe...
leofang May 3, 2025
78d0113
install zstd for win local ctk test
leofang May 4, 2025
390995a
fix copy pasta
leofang May 4, 2025
65e19b2
forward-port fetch_ctk fix
leofang May 4, 2025
6ade91b
ready for full test!
leofang May 4, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 14 additions & 3 deletions .github/actions/fetch_ctk/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,8 @@ runs:
unzip $1 -d $_TEMP_DIR_
cp -r $_TEMP_DIR_/*/* $CACHE_TMP_DIR
rm -rf $_TEMP_DIR_
# see commit NVIDIA/cuda-python@69410f1d9228e775845ef6c8b4a9c7f37ffc68a5
chmod 644 $CACHE_TMP_DIR/LICENSE
}
fi
function populate_cuda_path() {
Expand Down Expand Up @@ -146,8 +148,17 @@ runs:
- name: Set output environment variables
shell: bash --noprofile --norc -xeuo pipefail {0}
run: |
CUDA_PATH=$(realpath "./cuda_toolkit")
# mimics actual CTK installation
if [[ "${{ inputs.host-platform }}" == linux* ]]; then
CUDA_PATH=$(realpath "./cuda_toolkit")
echo "LD_LIBRARY_PATH=${LD_LIBRARY_PATH:-}:${CUDA_PATH}/lib:${CUDA_PATH}/nvvm/lib64" >> $GITHUB_ENV
elif [[ "${{ inputs.host-platform }}" == win* ]]; then
function normpath() {
echo "$(echo $(cygpath -w $1) | sed 's/\\/\\\\/g')"
}
CUDA_PATH=$(normpath $(realpath "./cuda_toolkit"))
echo "$(normpath ${CUDA_PATH}/bin)" >> $GITHUB_PATH
echo "$(normpath $CUDA_PATH/nvvm/bin)" >> $GITHUB_PATH
fi
echo "CUDA_PATH=${CUDA_PATH}" >> $GITHUB_ENV
echo "CUDA_HOME=${CUDA_PATH}" >> $GITHUB_ENV
echo "${CUDA_PATH}/bin" >> $GITHUB_PATH
echo "LD_LIBRARY_PATH=${LD_LIBRARY_PATH:-}:${CUDA_PATH}/lib:${CUDA_PATH}/nvvm/lib64" >> $GITHUB_ENV
2 changes: 1 addition & 1 deletion .github/workflows/build-and-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,7 @@ jobs:
LIBRARY_PATH=/host/${{ env.CUDA_PATH }}/lib
CUDA_BINDINGS_PARALLEL_LEVEL=${{ env.CUDA_BINDINGS_PARALLEL_LEVEL }}
CIBW_ENVIRONMENT_WINDOWS: >
CUDA_HOME="$(cygpath -w ${{ env.CUDA_PATH }})"
CUDA_HOME=${{ env.CUDA_PATH }}
LIB="${CUDA_HOME}\\lib\\x64;${LIB}"
CUDA_BINDINGS_PARALLEL_LEVEL=${{ env.CUDA_BINDINGS_PARALLEL_LEVEL }}
with:
Expand Down
56 changes: 34 additions & 22 deletions .github/workflows/test-wheel-windows.yml
Original file line number Diff line number Diff line change
Expand Up @@ -57,14 +57,6 @@ jobs:
$SKIP_CUDA_BINDINGS_TEST = 0
}

if ('${{ inputs.local-ctk }}' -eq '1') {
if ($TEST_CUDA_MAJOR -eq '12') {
$MINI_CTK_DEPS = '["nvcc", "nvrtc", "nvjitlink", "thrust"]'
} else {
$MINI_CTK_DEPS = '["nvcc", "nvrtc", "thrust"]'
}
}

# Make outputs from the previous job as env vars
$CUDA_CORE_ARTIFACT_BASENAME = "cuda-core-python${PYTHON_VERSION_FORMATTED}-${{ inputs.host-platform }}"
"PYTHON_VERSION_FORMATTED=${PYTHON_VERSION_FORMATTED}" >> $env:GITHUB_ENV
Expand All @@ -76,7 +68,6 @@ jobs:
"CUDA_BINDINGS_ARTIFACT_NAME=${CUDA_BINDINGS_ARTIFACT_BASENAME}-${{ github.sha }}" >> $env:GITHUB_ENV
"CUDA_BINDINGS_ARTIFACTS_DIR=$($ExecutionContext.SessionState.Path.GetUnresolvedProviderPathFromPSPath("$REPO_DIR\cuda_bindings\dist"))" >> $env:GITHUB_ENV
"SKIP_CUDA_BINDINGS_TEST=${SKIP_CUDA_BINDINGS_TEST}" >> $env:GITHUB_ENV
"MINI_CTK_DEPS=${MINI_CTK_DEPS}" >> $env:GITHUB_ENV

- name: Download cuda-python build artifacts
if: ${{ env.SKIP_CUDA_BINDINGS_TEST == '0'}}
Expand Down Expand Up @@ -107,6 +98,36 @@ jobs:
}
gh --version

- name: Install Git for Windows
# the GPU runner image does not have Git Bash pre-installed...
if: ${{ inputs.local-ctk == '1' }}
env:
# doesn't seem there's an easy way to avoid hard-coding it?
GFW_EXE_URL: https://github.com/git-for-windows/git/releases/download/v2.49.0.windows.1/PortableGit-2.49.0-64-bit.7z.exe
run: |
Invoke-WebRequest -Uri "$env:GFW_EXE_URL" -OutFile "PortableGit.7z.exe"
# Self-extracting, see https://gitforwindows.org/zip-archives-extracting-the-released-archives.html
Start-Process .\PortableGit.7z.exe -Wait -Verbose -ArgumentList '-y -gm2'
ls -l PortableGit
echo "$((Get-Location).Path)\\PortableGit\\bin" >> $env:GITHUB_PATH
$env:Path += ";$((Get-Location).Path)\\PortableGit\\bin"
bash --version

- name: Install zstd
# the GPU runner image does not have zstd pre-installed... and it's needed by actions/cache
if: ${{ inputs.local-ctk == '1' }}
env:
# doesn't seem there's an easy way to avoid hard-coding it?
ZSTD_URL: https://github.com/facebook/zstd/releases/download/v1.5.7/zstd-v1.5.7-win64.zip
ZSTD_DIR: zstd-v1.5.7-win64
run: |
Invoke-WebRequest -Uri "$env:ZSTD_URL" -OutFile "zstd-win64.zip"
Expand-Archive -Path "zstd-win64.zip" -DestinationPath .
ls -l $env:ZSTD_DIR
echo "$((Get-Location).Path)\\$env:ZSTD_DIR" >> $env:GITHUB_PATH
$env:Path += ";$((Get-Location).Path)\\$env:ZSTD_DIR"
zstd --version

- name: Download cuda-python & cuda.bindings build artifacts from the prior branch
if: ${{ env.SKIP_CUDA_BINDINGS_TEST == '1'}}
env:
Expand Down Expand Up @@ -160,20 +181,11 @@ jobs:

- name: Set up mini CTK
if: ${{ inputs.local-ctk == '1' }}
# Note: The GH-hosted Windows GPU runner does not have Git for Windows pre-installed,
# so we cannot use our own fetch_ctk action unfortunately...
uses: Jimver/cuda-toolkit@v0.2.21
uses: ./.github/actions/fetch_ctk
continue-on-error: false
with:
cuda: ${{ inputs.cuda-version }}
method: 'network'
sub-packages: ${{ env.MINI_CTK_DEPS }}

- name: Update PATH
if: ${{ inputs.local-ctk == '1' }}
run: |
# mimics actual CTK installation
echo $PATH
echo "$env:CUDA_PATH\nvvm\bin" >> $env:GITHUB_PATH
host-platform: ${{ inputs.host-platform }}
cuda-version: ${{ inputs.cuda-version }}

- name: Run cuda.bindings tests
if: ${{ env.SKIP_CUDA_BINDINGS_TEST == '0' }}
Expand Down
Loading