numo-narray: error: implicit declaration of function ‘RBASIC’

[puni073]

ruby --version

truffleruby 24.1.1, like ruby 3.2.4, Oracle GraalVM Native [x86_64-linux]

gem install numo-narray

compiling narray.c
narray.c: In function ‘nary_copy_flags’:
narray.c:948:5: error: implicit declaration of function ‘RBASIC’ [-Werror=implicit-function-declaration]
  948 |     RBASIC(dst)->flags |= (RBASIC(src)->flags) &
      |     ^~~~~~
narray.c:948:16: error: invalid type argument of ‘->’ (have ‘int’)
  948 |     RBASIC(dst)->flags |= (RBASIC(src)->flags) &
      |                ^~
narray.c:948:39: error: invalid type argument of ‘->’ (have ‘int’)
  948 |     RBASIC(dst)->flags |= (RBASIC(src)->flags) &
      |                                       ^~
cc1: some warnings being treated as errors
make: *** [Makefile:583: narray.o] Error 1

[eregon]

Thanks for the report.
This needs to be addressed upstream, see #3118 (comment) for the replacement.

[ankane]

Hi @eregon, I applied the following patch to numo-narray locally.

--- a/ext/numo/narray/narray.c
+++ b/ext/numo/narray/narray.c
@@ -7,6 +7,14 @@
 #include <ruby.h>
 #include <assert.h>
 
+#ifndef RBASIC_FLAGS
+#define RBASIC_FLAGS(obj) (RBASIC(obj)->flags)
+#endif
+
+#ifndef RBASIC_SET_FLAGS
+#define RBASIC_SET_FLAGS(obj, flags_to_set) (RBASIC(obj)->flags = (flags_to_set))
+#endif
+
 /* global variables within this module */
 VALUE numo_cNArray;
 VALUE rb_mNumo;
@@ -945,8 +953,8 @@ na_copy_flags(VALUE src, VALUE dst)
     na2->flag[0] = na1->flag[0];
     //na2->flag[1] = NA_FL1_INIT;
 
-    RBASIC(dst)->flags |= (RBASIC(src)->flags) &
-        (FL_USER1|FL_USER2|FL_USER3|FL_USER4|FL_USER5|FL_USER6|FL_USER7);
+    RBASIC_SET_FLAGS(dst, RBASIC_FLAGS(dst) | (RBASIC_FLAGS(src) &
+        (FL_USER1|FL_USER2|FL_USER3|FL_USER4|FL_USER5|FL_USER6|FL_USER7)));
 }

Compilation succeeded, but seeing an exception with this minimal script.

require "numo/narray"

Numo::NArray.cast([1, 2, 3]).to_a

Exception

array.c:23:in `rb_tr_rarray_aref': Index 0 does not exist. (Polyglot::ForeignException)
    from /Users/user/.rbenv/versions/truffleruby-24.2.0/lib/truffle/truffle/cext.rb:1852:in `<unknown>'
    from /Users/user/.rbenv/versions/truffleruby-24.2.0/lib/truffle/truffle/cext.rb:1852:in `<unknown>'
    from /Users/user/.rbenv/versions/truffleruby-24.2.0/lib/truffle/truffle/cext.rb:1852:in `CExt#rb_ensure'
    from exception.c:114:in `rb_ensure'
    from /Users/user/.rbenv/versions/truffleruby-24.2.0/lib/truffle/truffle/cext_ruby.rb:38:in `<unknown>'
    from /Users/user/.rbenv/versions/truffleruby-24.2.0/lib/truffle/truffle/cext_ruby.rb:38:in `<unknown>'
    from /Users/user/.rbenv/versions/truffleruby-24.2.0/lib/truffle/truffle/cext_ruby.rb:38:in `Numo::Int32#to_a'

Any idea what would cause this?

[eregon]

It means the C extension is doing RARRAY_AREF(ary, 0) but ary is empty so that seems potentially a bug of the C extension, or somehow that array not having the elements it should have.

Could you make a PR to numo-narray with your patch? It looks good to me.

[ankane]

Thanks @eregon, will try to see why CRuby isn't throwing an error.

There are other errors with the test suite with TruffleRuby as well that don't happen with CRuby.

Error 1

Numo::DFloat[1..3]

/Users/user/.rbenv/versions/truffleruby-24.2.0/lib/truffle/truffle/cext.rb:1347:in `rb_ivar_get': TruffleRuby doesn't have a case for the org.truffleruby.core.support.TypeNodesFactory$ObjectIVarGetPrimitiveNodeFactory$ObjectIVarGetPrimitiveNodeGen node with values of type org.truffleruby.core.range.RubyIntRange org.truffleruby.core.symbol.RubySymbol (TypeError)
    from org.truffleruby.core.support.TypeNodesFactory$ObjectIVarGetPrimitiveNodeFactory$ObjectIVarGetPrimitiveNodeGen.executeAndSpecialize(TypeNodesFactory.java:2299)
    from org.truffleruby.core.support.TypeNodesFactory$ObjectIVarGetPrimitiveNodeFactory$ObjectIVarGetPrimitiveNodeGen.execute(TypeNodesFactory.java:2238)
    from org.truffleruby.language.control.SequenceNode.execute(SequenceNode.java:35)
    from org.truffleruby.language.RubyMethodRootNode.execute(RubyMethodRootNode.java:65)
    from ivar.c:39:in `rb_ivar_get'
    from /Users/user/.rbenv/versions/truffleruby-24.2.0/lib/truffle/truffle/cext_ruby.rb:38:in `[]'

Error 2

Numo::DFloat[-4.step(-1)]

/Users/user/.rbenv/versions/truffleruby-24.2.0/lib/truffle/truffle/cext.rb:1315:in `rb_exc_raise': wrong argument type (not T_DATA) (TypeError)
	from exception.c:29:in `rb_exc_raise'
	from /Users/user/.rbenv/versions/truffleruby-24.2.0/lib/truffle/truffle/cext_ruby.rb:38:in `[]'

Will submit a PR to numo-narray once everything is passing.

[ankane]

It looks like CRuby doesn't check the array length for RARRAY_AREF.

https://github.com/ruby/ruby/blob/d0b7e5b6a04bde21ca483d20a1546b28b401c2d4/include/ruby/internal/core/rarray.h#L395-L403

Fixed with:

--- a/ext/numo/narray/ndloop.c
+++ b/ext/numo/narray/ndloop.c
@@ -1218,6 +1218,9 @@ ndloop_extract(VALUE results, ndfunc_t *nf)
     case 0:
         return Qnil;
     case 1:
+        if (RARRAY_LEN(results) == 0) {
+            return Qnil;
+        }
         x = RARRAY_AREF(results,0);
         if (NDF_TEST(nf,NDF_EXTRACT)) {
             if (IsNArray(x)){

[ankane]

Error 3

Numo::DFloat[1, 2, 3][1..]

/Users/user/.rbenv/versions/truffleruby-24.2.0/lib/truffle/truffle/cext.rb:1315:in `rb_exc_raise': no implicit conversion from nil (TypeError)
	from exception.c:29:in `rb_exc_raise'
	from numeric.c:78:in `rb_num2ll'
	from /Users/user/.rbenv/versions/truffleruby-24.2.0/lib/truffle/truffle/cext.rb:1852:in `rb_ensure'
	from exception.c:115:in `rb_ensure'
	from /Users/user/.rbenv/versions/truffleruby-24.2.0/lib/truffle/truffle/cext_ruby.rb:38:in `[]'

[eregon]

Will submit a PR to numo-narray once everything is passing.

It would be helpful to already submit a PR with what you found.
Notably that RARRAY_LEN could cause a potential segfault or memory corruption on CRuby.

Probably more things need to be changed in both numo-narray and truffleruby, but incremental fixes are good and avoid redoing the work later.

[ankane]

I think it'll be better to submit it all at once, as there isn't a lot of activity on the repo (but feel free to submit it earlier if you feel differently).