Foreign Function Interfaces (FFI) are the holy grail of language interoperability. They allow us to call native C, Rust, or Go libraries directly from Perl without writing a single line of XS boilerplate. But while crossing the boundary to call a function has gotten incredibly fast, accessing the data returned by that function has always carried a heavy toll.

I'll call this the FFI tax.

If a C function returns a pointer to a complex, nested struct, how does Perl interact with it? Historically, the evolution of Perl FFI has gone through three distinct stages to solve this problem.

Stage 1: The Copying Approach (unpack)

In the earliest days of FFI, the only way to read C memory was to treat it as a raw byte string and unpack it into a Perl list or hash.

The problem: This is a static snapshot. If the C library updates the struct in the background, your Perl data is oblivious. If you update the Perl data, C sees nothing until you manually pack the bytes and overwrite the C memory.

Stage 2: The Hybrid Pin Model (Tied Accessors)

To solve the synchronization problem, modern FFIs (like FFI::Platypus::Record) introduced the Hybrid Pin model. You wrap the C pointer in a generated class that provides accessor methods. When you access $struct->x(), a Perl method fires, calculating the byte offset and returning the value.

The problem: Method dispatch overhead. Every time you call $struct->x(), Perl has to push a frame onto the call stack, look up the method, execute Perl opcodes, and tear down the stack. In a tight loop, this dispatch overhead destroys your performance.

Stage 3: The Affix Breakthrough (Perl Magic)

Affix threw out both approaches. By diving deep into Perl's internal magic system, I've built a zero-copy, zero-method-dispatch memory mapper that feels like pure Perl but runs at native C speed.

Here is how I did it.

Enter Perl Magic (PERL_MAGIC_ext)

Many Perl developers use Perl's magic every day without realizing it. When you read from $!, Perl doesn't just read a string; it executes an internal C function to fetch the current system error. When you write to %ENV, Perl automatically updates the C-level environment variables.

Under the hood, this is powered by a C struct called MAGIC and a Virtual Table (MGVTBL) attached to the Perl Scalar (SV). A VTable contains pointers to C functions that fire when a scalar is read (svt_get), written to (svt_set), or destroyed (svt_free).

Instead of writing a Perl-level class to manage pointers (as in Stage 2), Affix uses PERL_MAGIC_ext (custom extension magic) to attach C memory addresses directly to standard Perl scalars.

The Pinned Value

To understand the performance win, we need to review how Affix constructs a C struct:

Instead of creating a magical, tied hash (where every key lookup triggers a Perl method), Affix creates a standard, native Perl Hash (HV) with typical keys but values that are all magical scalars (I also refer to these as "pins" because you're pinning it to a specific memory address). Because the hash itself is native, Perl's highly optimized hv_fetch routine locates the key at full C speed. Method dispatch is completely bypassed. Only when the specific value is read or written does the Affix C-level VTable quietly take over.

Let's look at the actual C code inside Affix that handles a 32bit integer. It's generated by a macro (MAKE_PRIMITIVE_DISPATCH) so it isn't hand written per type, but get_sint32 expands to exactly this:

int get_sint32(pTHX_ SV * sv, MAGIC * mg) {
    Affix_Pin_2_Point_Oh * im = (Affix_Pin_2_Point_Oh *)mg->mg_ptr;
    SvSMAGICAL_off(sv);
    if (!im->ptr)
        sv_setsv(sv, &PL_sv_undef);
    else
        sv_setiv(sv, *(int32_t *)im->ptr);
    SvSMAGICAL_on(sv);
    return 0;
}

And its mirror image set_sint32 writes the Perl integer straight back into native memory:

int set_sint32(pTHX_ SV * sv, MAGIC * mg) {
    Affix_Pin_2_Point_Oh * im = (Affix_Pin_2_Point_Oh *)mg->mg_ptr;
    if (im->readonly)
        croak("Modification of a read-only C value attempted");
    if (!im->ptr)
        return 0;
    SvGMAGICAL_off(sv);
    *(int32_t *)im->ptr = (int32_t)SvIV(sv);
    SvGMAGICAL_on(sv);
    return 0;
}

When you write to a pinned variable, set_sint32 fires, writing the Perl integer directly into the native memory block.

The Proof is in the Benchmarks

By eliminating the method dispatch overhead, the performance gains are staggering. In a direct benchmark against FFI::Platypus (the current CPAN standard for FFI), Affix's magic system dramatically changes the landscape.

When continuously reading and writing to a single integer pointer, Affix reaches 13 million operations per second—more than 3x faster than Platypus's Record accessors (~4M/s).

When dealing with structs (writing to x, writing to y, and reading the sum), the gap remains vast. Affix hits 5.2 million operations per second, doubling Platypus's 2.4M/s. While it is expectedly slower than a pure, native Perl hash (~8.7M/s) due to the cross-runtime synchronization, Affix successfully eliminates the traditional FFI tax, bringing live C memory manipulation closer to native speeds than ever before.

These numbers are from a point-in-time benchmark on specific hardware. Always measure on your own target machines. The qualitative point stands: no Perl-level method dispatch sits in the hot path.

Deep Access: Looking at the Ergonomics

Because we are mapping memory using native Hashes (HV) and native Arrays (AV), nested C structures behave exactly like deep Perl data structures.

Let's say we have a C library with a complex hierarchy:

typedef struct {
    int id;
    char name[16];
} Task;

typedef struct {
    char name[32];
    Task tasks[2];
} Employee;

typedef struct {
    Employee *manager;
    int budget;
} Company;

In Affix, you define these exact types using the AST (Abstract Syntax Tree) builder, allocate the memory, and interact with it as if it were a JSON payload:

use v5.40;
use Affix qw[:all];

typedef Task     => Struct [ id      => Int, name => Array[ Char, 16 ] ];
typedef Employee => Struct [ name    => Array [ Char, 32 ], tasks => Array[ Task(), 2 ] ];
typedef Company  => Struct [ manager => Pointer [ Employee() ], budget => Int ];

# Allocate the pieces
my $emp_mem = alloc_owned( sizeof( Employee() ) );
my $emp     = cast( $emp_mem, Employee() );

my $comp_mem = alloc_owned( sizeof( Company() ) );
my $comp     = cast( $comp_mem, Company() );

# Wire the manager pointer to the Employee we just allocated
$comp->{manager} = $emp;

# Deep, native, zero-copy write!
$comp->{manager}{tasks}[0]{id} = 999;
say $comp->{manager}{tasks}[0]{id};    # 999

When $comp->{manager}{tasks}[0]{id} = 999; executes:

  1. Perl does a fast hash lookup for manager.
  2. Perl does a fast hash lookup for tasks.
  3. Perl does a fast array lookup for index 0.
  4. Perl does a fast hash lookup for id.
  5. The svt_set C-hook fires, updating the exact 4 bytes in the malloc'd memory.

No tie related FETCH methods. No object overhead. Just raw memory access dressed up in native Perl syntax.A Pointer member that was never assigned reads back as undef; alloc_owned zeroes its memory, so a null pointer is exactly that. Allocate the child struct, cast it, and assign it into the pointer member before reaching through it. I did that above with $comp->{manager} = $emp;.

Lifeline Tracking: Preventing Use-After-Free

C developers are intimately familiar with 'use after free' segmentation faults. If you free a parent structure while holding a pointer to one of its children, accessing that child will crash your program.

In a garbage collected language like Perl, this is a massive hazard. What happens if we do this?

use v5.40;
use Affix qw[:all];

typedef Task     => Struct [ id      => Int, name => Array[ Char, 16 ] ];
typedef Employee => Struct [ name    => Array [ Char, 32 ], tasks => Array[ Task(), 2 ] ];
typedef Company  => Struct [ manager => Pointer [ Employee() ], budget => Int ];

my $task;
{
    my $emp_mem = alloc_owned( sizeof( Employee() ) );
    my $emp     = cast( $emp_mem, Employee() );
    $emp->{tasks}[0]{id} = 100;

    my $comp_mem = alloc_owned( sizeof( Company() ) );
    my $comp     = cast( $comp_mem, Company() );
    $comp->{manager} = $emp;

    # Extract a deeply nested child pointer
    $task = $comp->{manager}{tasks}[0];
}
# $comp, $emp, and both *_mem go out of scope here. Is $task dangling?
say $task->{id};

In Affix, the answer is no, it is not freed. Affix implements a concept I called Lifelines.

When alloc_owned is called, it creates an Affix::Memory root object. As Affix generates magical hashes and arrays representing the struct members, it stores a reference to that root Affix::Memory object inside the mg->mg_obj slot of every single child scalar.

As long as the $task variable exists, Perl's internal reference counting keeps the root $mem allocation alive. Once $task falls out of scope, the reference count drops to zero, and the C memory is finally, safely freed. ($task->{id} prints 100 even though its allocating scopes have long since closed.)

The "Giant Array" Dilemma: Cheap Headers and Snapshots

Mapping a struct with 5 fields to a magical hash is incredibly fast. But what if a C struct contains an array of 10,000 integers? If Affix had to copy 10,000 Perl values back and forth the moment you cast the pointer, memory usage would skyrocket and your program would stall.

Affix solves this in two ways.

First, the array header is native but the elements are lazy bindings. When you cast an Array[Int, 10000], Affix returns a native Perl array reference (AV) filled with one lightweight magical element scalar per slot: this is an O(1) header, no bulk value copy. Each element is a binding, not a value: reading $arr->[500] fires svt_get to fetch those exact 4 bytes from C memory, and writing fires svt_set. Whether the array has 5 elements or 10,000, Perl's fast av_fetch/av_store path finds the slot, and C memory is only touched on demand.

But what if you need to iterate over all 10,000 elements in a loop?

My benchmarks revealed a fascinating truth: performing 10,000 individual VTable magic lookups per loop iteration is slightly slower than having the FFI simply bulk copy the array into a native Perl list. If you need peak speed for bulk operations, "Live" memory access actually becomes a bottleneck.

To solve this, I've added a pair of powerful escape hatches: Affix::snapshot() and Affix::raw().

use v5.40;
use Affix qw[:all];

typedef BigArray => Array [ Int, 10000 ];

my $mem = alloc_owned( sizeof( BigArray() ) );
my $arr = cast( $mem, BigArray() );
$arr->[500] = 7;

my $snap = Affix::snapshot($arr);          # pure Perl list, no magic
my $blob = Affix::raw( $arr, 40_000 );     # 10,000 * 4 bytes, ready for unpack('l<*')
say $snap->[500];                          # 7
say unpack( 'l<', substr( $blob, 500 * 4, 4 ) );    # 7

The Array Iteration Benchmark (Summing 10,000 elements):

Conclusion

By abandoning method-based FFI wrappers and embracing Perl's low-level Magic system, Affix achieves the best of both worlds. You get the strict memory layout and blazing speed of C, alongside the beautiful, dynamic syntax of Perl. And when you need to crunch massive arrays, Affix provides the low-level memory extraction tools to run neck-and-neck with native Perl speeds.