Data pointers#

How pointers are used#

In C, pointer arguments are used for differenty purposes. It is important to distinguish these use cases because Koffi provides different ways to deal with each of them:

  • Struct pointers: Use of struct pointers by C libraries fall in two cases: avoid (potentially) expensive copies, and to let the function change struct contents (output or input/output arguments).

  • Opaque pointers: the library does not expose the contents of the structs, and only provides you with a pointer to it (e.g. FILE *). Only the functions provided by the library can do something with this pointer, in Koffi we call this an opaque type. This is usually done for ABI-stability reason, and to prevent library users from messing directly with library internals.

  • Pointers to primitive types: This is more rare, and generally used for output or input/output arguments. The Win32 API has a lot of these.

  • Arrays: in C, you dynamically-sized arrays are usually passed to functions with pointers, either NULL-terminated (or any other sentinel value) or with an additional length argument.

Pointer types#

Struct pointers#

The following Win32 example uses GetCursorPos() (with an output parameter) to retrieve and show the current cursor position.

 1// ES6 syntax: import koffi from 'koffi';
 2const koffi = require('koffi');
 3
 4const lib = koffi.load('user32.dll');
 5
 6// Type declarations
 7const POINT = koffi.struct('POINT', {
 8    x: 'long',
 9    y: 'long'
10});
11
12// Functions declarations
13const GetCursorPos = lib.func('int __stdcall GetCursorPos(_Out_ POINT *pos)');
14
15// Get and show cursor position
16let pos = {};
17if (!GetCursorPos(pos))
18    throw new Error('Failed to get cursor position');
19console.log(pos);

Opaque pointers#

New in Koffi 2.0

Some C libraries use handles, which behave as pointers to opaque structs. An example of this is the HANDLE type in the Win32 API. If you want to reproduce this behavior, you can define a named pointer type to an opaque type, like so:

1const HANDLE = koffi.pointer('HANDLE', koffi.opaque());
2
3// And now you get to use it this way:
4const GetHandleInformation = lib.func('bool __stdcall GetHandleInformation(HANDLE h, _Out_ uint32_t *flags)');
5const CloseHandle = lib.func('bool __stdcall CloseHandle(HANDLE h)');

Pointer to primitive types#

In javascript, it is not possible to pass a primitive value by reference to another function. This means that you cannot call a function and expect it to modify the value of one of its number or string parameter.

However, arrays and objects (among others) are reference type values. Assigning an array or an object from one variable to another does not invole any copy. Instead, as the following example illustrates, the new variable references the same array as the first:

1let list1 = [1, 2];
2let list2 = list1;
3
4list2[1] = 42;
5
6console.log(list1); // Prints [1, 42]

All of this means that C functions that are expected to modify their primitive output values (such as an int * parameter) cannot be used directly. However, thanks to Koffi’s transparent array support, you can use Javascript arrays to approximate reference semantics with single-element arrays.

Below, you can find an example of an addition function where the result is stored in an int * input/output parameter and how to use this function from Koffi.

1void AddInt(int *dest, int add)
2{
3    *dest += add;
4}

You can simply pass a single-element array as the first argument:

1const AddInt = lib.func('void AddInt(_Inout_ int *dest, int add)');
2
3let sum = [36];
4AddInt(sum, 6);
5
6console.log(sum[0]); // Prints 42

Dynamic arrays#

In C, dynamically-sized arrays are usually passed around as pointers. The length is either passed as an additional argument, or inferred from the array content itself, for example with a terminating sentinel value (such as a NULL pointers in the case of an array of strings).

Koffi can translate JS arrays and TypedArrays to pointer arguments. However, because C does not have a proper notion of dynamically-sized arrays (fat pointers), you need to provide the length or the sentinel value yourself depending on the API.

Here is a simple example of a C function taking a NULL-terminated list of strings as input, to calculate the total length of all strings.

 1// Build with: clang -fPIC -o length.so -shared length.c -Wall -O2
 2
 3#include <stdlib.h>
 4#include <stdint.h>
 5#include <string.h>
 6
 7int64_t ComputeTotalLength(const char **strings)
 8{
 9    int64_t total = 0;
10
11    for (const char **ptr = strings; *ptr; ptr++) {
12        const char *str = *ptr;
13        total += strlen(str);
14    }
15
16    return total;
17}

 1// ES6 syntax: import koffi from 'koffi';
 2const koffi = require('koffi');
 3
 4const lib = koffi.load('./length.so');
 5
 6const ComputeTotalLength = lib.func('int64_t ComputeTotalLength(const char **strings)');
 7
 8let strings = ['Get', 'Total', 'Length', null];
 9let total = ComputeTotalLength(strings);
10
11console.log(total); // Prints 14

By default, just like for objects, array arguments are copied from JS to C but not vice-versa. You can however change the direction as documented in the section on output parameters.

Handling void pointers#

New in Koffi 2.1

Many C functions use void * parameters in order to pass polymorphic objects and arrays, meaning that the data format changes can change depending on one other argument, or on some kind of struct tag member.

Koffi provides two features to deal with this:

  • You can use koffi.as(value, type) to tell Koffi what kind of type is actually expected, as shown in the example below.

  • Buffers and typed JS arrays can be used as values in place everywhere a pointer is expected. See dynamic arrays for more information, for input or output.

The example below shows the use of koffi.as() to read the header of a PNG file with fread() directly to a JS object.

 1// ES6 syntax: import koffi from 'koffi';
 2const koffi = require('koffi');
 3
 4const lib = koffi.load('libc.so.6');
 5
 6const FILE = koffi.opaque('FILE');
 7
 8const PngHeader = koffi.pack('PngHeader', {
 9    signature: koffi.array('uint8_t', 8),
10    ihdr: koffi.pack({
11        length: 'uint32_be_t',
12        chunk: koffi.array('char', 4),
13        width: 'uint32_be_t',
14        height: 'uint32_be_t',
15        depth: 'uint8_t',
16        color: 'uint8_t',
17        compression: 'uint8_t',
18        filter: 'uint8_t',
19        interlace: 'uint8_t',
20        crc: 'uint32_be_t'
21    })
22});
23
24const fopen = lib.func('FILE *fopen(const char *path, const char *mode)');
25const fclose = lib.func('int fclose(FILE *fp)');
26const fread = lib.func('size_t fread(_Out_ void *ptr, size_t size, size_t nmemb, FILE *fp)');
27
28let filename = process.argv[2];
29if (filename == null)
30    throw new Error('Usage: node png.js <image.png>');
31
32let hdr = {};
33{
34    let fp = fopen(filename, 'rb');
35    if (!fp)
36        throw new Error(`Failed to open '${filename}'`);
37
38    try {
39        let len = fread(koffi.as(hdr, 'PngHeader *'), 1, koffi.sizeof(PngHeader), fp);
40        if (len < koffi.sizeof(PngHeader))
41            throw new Error('Failed to read PNG header');
42    } finally {
43        fclose(fp);
44    }
45}
46
47console.log('PNG header:', hdr);

Disposable types#

New in Koffi 2.0

Disposable types allow you to register a function that will automatically called after each C to JS conversion performed by Koffi. This can be used to avoid leaking heap-allocated strings, for example.

Some C functions return heap-allocated values directly or through output parameters. While Koffi automatically converts values from C to JS (to a string or an object), it does not know when something needs to be freed, or how.

For opaque types, such as FILE, this does not matter because you will explicitly call fclose() on them. But some values (such as strings) get implicitly converted by Koffi, and you lose access to the original pointer. This creates a leak if the string is heap-allocated.

To avoid this, you can instruct Koffi to call a function on the original pointer once the conversion is done, by creating a disposable type with koffi.dispose(name, type, func). This creates a type derived from another type, the only difference being that func gets called with the original pointer once the value has been converted and is not needed anymore.

The name can be omitted to create an anonymous disposable type. If func is omitted or is null, Koffi will use koffi.free(ptr) (which calls the standard C library free function under the hood).

1const AnonHeapStr = koffi.disposable('str'); // Anonymous type (cannot be used in function prototypes)
2const NamedHeapStr = koffi.disposable('HeapStr', 'str'); // Same thing, but named so usable in function prototypes
3const ExplicitFree = koffi.disposable('HeapStr16', 'str16', koffi.free); // You can specify any other JS function

The following example illustrates the use of a disposable type derived from str.

 1// ES6 syntax: import koffi from 'koffi';
 2const koffi = require('koffi');
 3
 4const lib = koffi.load('libc.so.6');
 5
 6const HeapStr = koffi.disposable('str');
 7const strdup = lib.cdecl('strdup', HeapStr, ['str']);
 8
 9let copy = strdup('Hello!');
10console.log(copy); // Prints Hello!

When you declare functions with the prototype-like syntax, you can either use named disposable types or use the ‘!’ shortcut qualifier with compatibles types, as shown in the example below. This qualifier creates an anonymous disposable type that calls koffi.free(ptr).

 1// ES6 syntax: import koffi from 'koffi';
 2const koffi = require('koffi');
 3
 4const lib = koffi.load('libc.so.6');
 5
 6// You can also use: const strdup = lib.func('const char *! strdup(const char *str)')
 7const strdup = lib.func('str! strdup(const char *str)');
 8
 9let copy = strdup('World!');
10console.log(copy); // Prints World!

Disposable types can only be created from pointer or string types.

Warning

Be careful on Windows: if your shared library uses a different CRT (such as msvcrt), the memory could have been allocated by a different malloc/free implementation or heap, resulting in undefined behavior if you use koffi.free().

Unwrap pointers#

You can use koffi.address(ptr) on a pointer to get the numeric value as a BigInt object.