I've had this idea for a while now and might put effort into it next. I understand that this would be a massive task but the first 30% of it is already accomplished by infix existing in its current state.

Here's my proposal:

The design of this will be based on the same principles as the rest of infix: one-time setup for performance, declarative signature string, and clean separation between the generic, user-facing API and the messy platform-specific backends.

The Core API: A Familiar, Handle-Based Pattern

Instead of a single, syscall() function, we would mirror the trampoline pattern by creating a handle. This allows infix to perform the expensive lookup and JIT-compilation once.

/**
 * @brief An opaque handle to a JIT-compiled system call stub.
 */
typedef struct infix_syscall_t infix_syscall_t;

/**
 * @brief Creates a handle to a specific system call.
 *
 * This function looks up the syscall by its platform-specific name, parses the
 * signature, and generates a JIT stub to invoke it. Like a bound trampoline.
 *
 * @param[out] out_handle Receives the created handle.
 * @param[in] syscall_specifier A string identifying the syscall, using a
 *            platform namespace (e.g., "linux:write", "win:NtCreateFile").
 * @param[in] signature The infix signature of the syscall's arguments and return type.
 * @param[in] registry An optional registry for any named types used in the signature.
 * @return INFIX_SUCCESS on success.
 */
infix_status infix_syscall_create(
    infix_syscall_t ** out_handle,
    const char * syscall_specifier,
    const char * signature,
    infix_registry_t * registry
);

/**
 * @brief Invokes the system call.
 *
 * This function is the syscall version of of `infix_cif_func`. It executes
 * the JIT-compiled stub.
 *
 * @param handle The handle created by infix_syscall_create.
 * @param out_return_value A pointer to a buffer to receive the raw return value
 *        (e.g., an intptr_t). This value may indicate an error, which must be
 *        interpreted according to the OS's conventions (e.g., negative value for errno).
 * @param args An array of pointers to the argument values.
 */
void infix_syscall_call( infix_syscall_t * handle, void * out_return_value, void ** args );

/**
 * @brief Destroys a syscall handle and frees its resources.
 */
void infix_syscall_destroy( infix_syscall_t * handle );

A Portability 'Solution': The "Syscall Specifier"

The biggest challenge is naming. write means different things on different OSes so my proposed solution is a namespaced string: "os_name::syscall_name".

This approach acknowledges that syscalls are not truly portable at a semantic level and requires the developer to know their target OS. I might even need to add a version number to this somehow. Either way, this naming system makes the backend implementation clean and manageable, as each namespace can have its own simple name-to-number lookup table.

Example Usage: Writing to stdout

Here's how you'd call write(1, "hello\n", 6) on Linux:

#include <infix/infix.h>
#include <stdio.h>
#include <unistd.h> // For ssize_t

void portable_syscall_example_linux() {
    infix_syscall_t * handle = NULL;
    const char* specifier = "linux::write";
    // The signature for ssize_t write(int fd, const void *buf, size_t count);
    const char * signature = "(int, *void, uint64) -> sint64"; // Same signature language as infix...

    // Create the handle once.
    infix_status status = infix_syscall_create(&handle, specifier, signature, NULL);
    if (status != INFIX_SUCCESS)
        // Handle error...
        return;
  
    // Prepare arguments for a specific call.
    int fd = 1; // stdout
    const char * message = "Hello from a direct syscall!\n";
    uint64_t count = 29;
    void * args[] = { &fd, &message, &count };
    int64_t return_value;

    // Call the syscall. This is the fast part.
    infix_syscall_call( handle, &return_value, args );

    if (return_value < 0)
        printf("Syscall failed with errno: %lld\n", (long long) 0-return_value);
    else 
        printf("Syscall returned %lld (bytes written).\n", (long long)return_value);
    
    infix_syscall_destroy(handle);
}

Internals

I'll probably add a new v-table specific to system calls. Probably something like this in infix_internals.h:

typedef struct {
    // Looks up a name like "write" and returns its number for this OS.
    long (*lookup_syscall_number)(const char * name);
    // The ABI for syscalls (which registers to use for number and args).
    // This could even reuse parts of the main ABI spec.
    infix_abi_spec * syscall_abi;
    // The specific instruction to emit (e.g., the bytes for `syscall` or `svc`).
    void (* generate_syscall_instruction)(code_buffer * buf);
} infix_syscall_spec;

When infix_syscall_create is called (assume linux)...

Before I write any actual code, I've tried to piece together a few solid resources for finding the system call tables and conventions for each major supported platform.

Linux

Linux has a stable and well-documented syscall ABI for each architecture. The syscall number is placed in a specific register (rax on x86-64, x8 on AArch64), and then the syscall (or svc) instruction is executed.

Sources:

Windows (Native API)

Windows does not have a stable, numbered syscall interface in the same way as Linux. Instead, the lowest-level supported interface is the Native API, which consists of functions exported by ntdll.dll (e.g., NtCreateFile, NtWriteFile). While these functions ultimately use a syscall or sysenter instruction, the numbers can change between Windows versions and even service packs. I'll need to enhance the OS detection in infix_config.h considerably. The only 'stable' interface is the function in ntdll.dll.

Sources:

macOS (XNU Kernel)

macOS uses a combination of POSIX-style syscalls and lower-level "Mach traps". It's based on FreeBSD and the numbers are defined in the kernel source code which is open source: https://github.com/apple-oss-distributions/xnu.

FreeBSD

FreeBSD also uses a central file in the kernel source to define its syscalls: https://cgit.freebsd.org/src/tree/sys/kern/syscalls.master