Need an ownership_holds for non-malloced memory

[fefe17]

I have an API where you can add a buffer to a writev-style array and then later repeatedly ask the API to write what's left of the total buffers to a file descriptor.

There are two functions to add a buffer to the list. One says "free the buffer when you are done", the other says "don't free the buffer when you are done".

Usually you will call the latter one with a string constant, but you might also call it with something malloced, and then I would like to get an error if I free that malloced buffer after adding it to the batch, and I would like to tell clang when the data structure is deinitialized so it can know that it is now OK to free the buffer you gave it.

The first half could be done with a magic identifier for ownership_holds, like currently malloc is. Maybe generic?

But the second half would need a totally new attribute, like ownership_release or so.

[llvmbot]

@llvm/issue-subscribers-clang-static-analyzer

Author: None (fefe17)

I have an API where you can add a buffer to a writev-style array and then later repeatedly ask the API to write what's left of the total buffers to a file descriptor.

There are two functions to add a buffer to the list. One says "free the buffer when you are done", the other says "don't free the buffer when you are done".

Usually you will call the latter one with a string constant, but you might also call it with something malloced, and then I would like to get an error if I free that malloced buffer after adding it to the batch, and I would like to tell clang when the data structure is deinitialized so it can know that it is now OK to free the buffer you gave it.

The first half could be done with a magic identifier for ownership_holds, like currently malloc is. Maybe generic?

But the second half would need a totally new attribute, like ownership_release or so.

[steakhal]

Hi,
Could you give some naive code examples to illustrate the use case?
I think that would help the discussion to formulate the next steps.

[fefe17]

My API works like this:

struct io_batch b;
iob_init(&b,10);
iob_addbuf(&b,"hello, world!\n", 13);
iob_addbuf_free(&b,some_buffer_we_malloced,bytes_in_said_buffer); // hand buffer ownership over to iob struct
...
free(some_buffer_we_malloced); // I want clang to warn about this, because it's like a double free
...
iob_reset(&b); // will call free on the buffer we gave it ownership of
// I want clang to warn if I do anything with the buffer here because it's like a use-after-free

[steakhal]

Thank you for the example. It was very illustrative. I migrated the code to CompilerExplorer to play with it: https://godbolt.org/z/d5o3xYf1e

On the example, after adding the relevant ownership_holds to iob_addbuf_free, I'd get the expected (and desired) error for the subsequent free(some_buffer_we_malloced).

From what I can tell that doesn't work as expected is the iob_reset(&b). It's because the ownership_holds attribute does not encode until what point and what object holds ownership. I think you are right that this is a common pattern, and the attribute should support this use case.

One idea that would work for this use case would be to say when calling iob_addbuf_free(&b,some_buffer_we_malloced,bytes_in_said_buffer), then the object &b would hold ownership until calling iob_reset. I'm not sure it would generalize well though. I'm not really familiar how C APIs are generally designed for ownership.

WDYT?

[fefe17]

C APIs are generally not designed for ownership considerations, that's why we have so many security problems with them :-)

The reason for this API's existence is to write a number of buffers and/or file contents to a socket that can be non-blocking. You call a function that sends as much as it can, and then you can call it again when the socket is writable again, and it will write more.
That's why it needs to take ownership of the buffers until their data has been written.

I'm not sure we really need more fine-grained attributes. For now it would be enough if we could say "this is a container, this function takes ownership of the function argument 3" and then a way to say "this function gives ownership back to you" and a way to say "this function destructs all ownership held in the container". The give back part would be nice, for example, if you have a work queue.

While I'm talking to you: It would be nice if we had something like ownership_gives but for functions who return the data in a function argument. For example, if you have a malloc wrapper like this:

errno_t mymalloc(void** result,size_t len);

Or, less theoretical, asprintf (a GNU invention widely available on Linux).

int asprintf(char **restrict strp, const char *restrict fmt, ...);

That attribute could also be useful, for example, for a "read one line from a file and return it in malloced memory" type function.
Or let's say you have a work queue where you add tasks and then pop one off the start. It would be nice if I could tell the compiler than popping one gives me ownership and I need to free the data when I'm done.