Custom datasets

[caihaunqai]

Hello, I have followed the Tutorial 2 to customize my own dataset. It can find all the pictures and train normally. However, the accuracy is 99.99% at the first iter, since it estimate all pixels to one class (two class in total). I use some saved model to test and find that it really estimate all pixels to one class. I checked my dataset and the labels for GT are 0 or 1， which are correct. But I still can not get the true results. Do you know why? Thank you~

[caihaunqai]

I print one dataset and it shows like the following picture. I don't know why it has 255 since I check my dataset and label for GT are 0 or 1.

[MeowZheng]

Might check ignore_index in decode_head cfg, as if ignore_index=0 will ingore all 0 label

[caihaunqai]

Thank you for your reply. I have check the dataset file in "mmseg/datasets" and found ignore_index is 255 in default. Is it the variable that you mentioned?   

…

------------------ 原始邮件 ------------------ 发件人: "open-mmlab/mmsegmentation" ***@***.***>; 发送时间: 2022年5月6日(星期五) 下午4:51 ***@***.***>; ***@***.******@***.***>; 主题: Re: [open-mmlab/mmsegmentation] Custom datasets (Issue #1558) Might check ignore_index in decode_head cfg, as if ignore_index=0 will ingore all 0 label — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

[MeowZheng]

I print one dataset and it shows like the following picture. I don't know why it has 255 since I check my dataset and label for GT are 0 or 1.

I found the label of dataset is 255, it might be the problem of dataset pre-processes. You might try to covert the color label to seg map label, ref

mmsegmentation/tools/convert_datasets/isaid.py

Lines 14 to 45 in ee67c80

	iSAID_palette = \
	{
	0: (0, 0, 0),
	1: (0, 0, 63),
	2: (0, 63, 63),
	3: (0, 63, 0),
	4: (0, 63, 127),
	5: (0, 63, 191),
	6: (0, 63, 255),
	7: (0, 127, 63),
	8: (0, 127, 127),
	9: (0, 0, 127),
	10: (0, 0, 191),
	11: (0, 0, 255),
	12: (0, 191, 127),
	13: (0, 127, 191),
	14: (0, 127, 255),
	15: (0, 100, 155)
	}
	iSAID_invert_palette = {v: k for k, v in iSAID_palette.items()}
	def iSAID_convert_from_color(arr_3d, palette=iSAID_invert_palette):
	"""RGB-color encoding to grayscale labels."""
	arr_2d = np.zeros((arr_3d.shape[0], arr_3d.shape[1]), dtype=np.uint8)
	for c, i in palette.items():
	m = np.all(arr_3d == np.array(c).reshape(1, 1, 3), axis=2)
	arr_2d[m] = i
	return arr_2d

[caihaunqai]

Hello, I double check my dataset and it only has two channels with pixels, which are 0 or 1. Is the dataset correct? I didn't modify any pre-processes before. Do you mean that there are some pre-processes that I need to modify by myself? Thank you.

…

------------------ 原始邮件 ------------------ 发件人: "open-mmlab/mmsegmentation" ***@***.***>; 发送时间: 2022年5月6日(星期五) 下午5:44 ***@***.***>; ***@***.******@***.***>; 主题: Re: [open-mmlab/mmsegmentation] Custom datasets (Issue #1558) I print one dataset and it shows like the following picture. I don't know why it has 255 since I check my dataset and label for GT are 0 or 1. I found the label of dataset is 255, it might be the problem of dataset pre-processes. You might try to covert the color label to seg map label, ref https://github.com/open-mmlab/mmsegmentation/blob/ee67c808b2ac9dad38e082d168741dd1097ffc93/tools/convert_datasets/isaid.py#L14-L45 — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

[MeowZheng]

I print one dataset and it shows like the following picture. I don't know why it has 255 since I check my dataset and label for GT are 0 or 1.

I think you might check why label has 255, which might need to be reverted from color image to label.

ref: this revert script mmsegmentation/tools/convert_datasets/isaid.py

[luyang-NWPU]

请问你解决这个问题了吗？我也遇到了同样的问题。和你一样，我自己的数据集除了背景就只有1个类。

[aymanaboghonim]

@MeowZheng could you help me how to check that my dataset has only 0 & 1 ?
If it has other value, how to apply the isaid converter ?? I am using custom dataset where both images and labels are rgb.

[MeowZheng]

do you set reduce_zero_label=0?