Fix tie breaking and handle missing evals #8
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Hi @piojanu, thanks for bringing my attention to this! It seems like the script is a previous version during development that even has the variable names messed around. I have pushed our correct script for this calculation, which aligns with your fix here. However, I am not sure about the missing evals part -- there isn't supposed to be any missing evaluation log? Would you mind explaining a bit what issue did you have with this part and why you want to add that? Thanks! |
|
I didn't debug it throughly really, but whenever docker instance eval doesn't finish, you put an empty line for such an eval in the |
|
@ronch99 Here is the example error that causes the evaluation to fail and produce an empty line in the Here are the example generated program by llama-70b that produced the error: |
|
@piojanu Thanks for the details! I wonder have you tried changing the dockerfile according to this post? @flyhero99 or @btyu can maybe help to do a quick check as well |
|
@ronch99 Nope, I didn't. Here is another error, this time during the instance docker build: Here is the program generated by llama-70b: import os
import numpy as np
from osgeo import gdal
# Define the classification dictionaries
landCover_classification = {
11:10,#Open Water
21:8,#Developed, Open Space
22:7,#Developed, Low Intensity
23:8,#Developed, Medium Intensity
24:9,#Developed, High Intensity
31:6,#Barren Land
41:2,#Deciduous Forest
42:1,#Evergreen Forest
43:2,#Mixed Forest
52:3,#Shrub/Scrub
71:3,#Grassland/Herbaceous
72:3,#Sedge/Herbaceous
81:4,#Hay/Pasture
82:6,#Cultivated Crops
90:4,#Woody Wetlands
95:4,#Emergent Herbaceous Wetlands
255:10
}
protected_status_classification = {
0:1,
1:3,
2:6,
3:9,
4:10,
255:10
}
# Define the input and output file paths
input_landCover_file = 'benchmark/datasets/MountainLionNew/landCover.tif'
input_protected_status_file = 'benchmark/datasets/MountainLionNew/Protected_Status.tif'
output_landCover_file = 'pred_results/landCover_reclassified.tif'
output_protected_status_file = 'pred_results/protected_status_reclassified.tif'
# Open the input files
landCover_dataset = gdal.Open(input_landCover_file)
protected_status_dataset = gdal.Open(input_protected_status_file)
# Get the input data
landCover_data = landCover_dataset.GetRasterBand(1).ReadAsArray()
protected_status_data = protected_status_dataset.GetRasterBand(1).ReadAsArray()
# Reclassify the data
reclassified_landCover_data = np.vectorize(landCover_classification.get)(landCover_data)
reclassified_protected_status_data = np.vectorize(protected_status_classification.get)(protected_status_data)
# Create output files
driver = gdal.GetDriverByName('GTiff')
output_landCover_dataset = driver.CreateCopy(output_landCover_file, landCover_dataset, 1, [gdal.GDT_Byte])
output_protected_status_dataset = driver.CreateCopy(output_protected_status_file, protected_status_dataset, 1, [gdal.GDT_Byte])
# Write the reclassified data to the output files
output_landCover_dataset.GetRasterBand(1).WriteArray(reclassified_landCover_data)
output_protected_status_dataset.GetRasterBand(1).WriteArray(reclassified_protected_status_data)
# Close the datasets
landCover_dataset = None
protected_status_dataset = None
output_landCover_dataset = None
output_protected_status_dataset = None |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Before, when tie breaking, the suboptimal run could be picked (e.g. with best cost, but invalid program).
Missing evals can happen when there are problems with docker run evaluation (see the examples in my comments below).