Skip to content

Fix problem due to HBaseRowDataLookupFunction returned to reuse objects #20

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

Jackson-xky
Copy link

HBaseRowDataLookupFunction.lookup中返回的结果是Collections.singletonList(serde.convertToNewRow(result)),这里默认都是用convertToNewRow方法,内部是在复用同一个对象,当开启了lookup.cache的时候,会导致缓存的结果只都是同一个对象,从而导致结果错乱,由于这里目前不能判断是否使用cache,因此只能默认使用convertToNewRow返回一个新对象来避免上面的情况

…returned is a reused object. If lookup.cache is enabled, the result encapsulated in the reused object will be cached externally, resulting in all cached values being the same object
@boring-cyborg
Copy link

boring-cyborg bot commented Sep 15, 2023

Thanks for opening this pull request! Please check out our contributing guidelines. (https://flink.apache.org/contributing/how-to-contribute.html)

@Jackson-xky Jackson-xky changed the title fix: Fix problem due to HBaseRowDataLookupFunction returned to reuse objects Fix problem due to HBaseRowDataLookupFunction returned to reuse objects Sep 15, 2023
@ferenc-csaky
Copy link
Contributor

Thank you for your contribution! Would you mind creating a bug jira with a description about the bug and then modify this PR with that ticket number and also an English description? Thank you in advance!

@Jackson-xky
Copy link
Author

HBaseRowDataLookupFunction. The returned result is the Collections in the lookup. SingletonList (serde. ConvertToNewRow (result). The default convertToNewRow method is used here, and the same object is reused internally. When lookup.cache is enabled, the cache results will only be the same object, resulting in confusion. You can avoid this by default by using convertToNewRow to return a new object

@MartijnVisser
Copy link
Contributor

@ferenc-csaky WDYT?

@Tan-JiaLiang
Copy link
Contributor

@Jackson-xky This is a serious problem in the hbase lookup join. Are you still working on it? I'd be happy to help.

@Tan-JiaLiang
Copy link
Contributor

@MartijnVisser @ferenc-csaky It doesn't look like he's had time to deal with the problem, and I'd like to take it on.

@ferenc-csaky
Copy link
Contributor

@Tan-JiaLiang I think it is okay to take over. We can keep the separate commits, so the original change will keep its author, even the commit msg can be modified while the original author remains the same.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants