Plugin Developer's Guide
Overview
Beginning with OPAE C library version 1.2.0, OPAE implements a plugin-centric model. This guide serves as a reference to define the makeup of an OPAE C API plugin and to describe a sequence of steps that one may follow when constructing an OPAE C API plugin.
Plugin Required Functions
An OPAE C API plugin is a runtime-loadable shared object library, also known as a module. On Linux systems, the dl family of APIs from libdl are used to interact with shared objects. Refer to "man dlopen" and "man dlsym" for examples of using the libdl API.
An OPAE C API plugin implements one required function. This function is required to have C linkage, so that its name is not mangled.
During initialization, the OPAE plugin manager component loads each plugin,
searching for its opae_plugin_configure function. If none is found, then
the plugin manager rejects that plugin. When it is found, opae_plugin_configure
is called passing a pointer to a freshly-created opae_api_adapter_table and
a buffer consisting of configuration data for the plugin.
The job of the opae_plugin_configure function is to populate the given adapter
table with each of the plugin's API entry points and to consume and comprehend
the given configuration data in preparation for initialization.
OPAE API Adapter Table
The adapter table is a data structure that contains function pointer entry points for each of the OPAE APIs implemented by a plugin. In this way, it adapts the plugin-specific behavior to the more general case of a flat C API. Note that OPAE applications are only required to link with opae-c. In other words, the name of the plugin library should not appear on the linker command line. In this way, plugins are truly decoupled from the OPAE C API, and they are required to adapt to the strict API specification by populating the adapter table only. No other linkage is required nor recommended.
adapter.h contains the definition of the opae_api_adapter_table. An abbreviated
version is depicted below, along with supporting type opae_plugin:
typedef struct _opae_plugin {
char *path;
void *dl_handle;
} opae_plugin;
typedef struct _opae_api_adapter_table {
struct _opae_api_adapater_table *next;
opae_plugin plugin;
fpga_result (*fpgaOpen)(fpga_token token, fpga_handle *handle,
int flags);
fpga_result (*fpgaClose)(fpga_handle handle);
...
fpga_result (*fpgaEnumerate)(const fpga_properties *filters,
uint32_t num_filters, fpga_token *tokens,
uint32_t max_tokens,
uint32_t *num_matches);
...
// configuration functions
int (*initialize)(void);
int (*finalize)(void);
// first-level query
bool (*supports_device)(const char *device_type);
bool (*supports_host)(const char *hostname);
} opae_api_adapter_table;
Some points worth noting are that the adapter tables are organized in memory by
adding them to a linked list data structure. This is the use of the next
structure member. (The list management is handled by the plugin manager.)
The plugin structure member contains the handle to the shared object instance,
as created by dlopen. This handle is used in the plugin's opae_plugin_configure
to load plugin entry points. A plugin need only implement the portion of the
OPAE C API that a target application needs. Any API entry points that are not
supported should be left as NULL pointers (the default) in the adapter table.
When an OPAE API that has no associated entry point in the adapter table is
called, the result for objects associated with that plugin will be
FPGA_NOT_SUPPORTED.
The following code illustrates a portion of the opae_plugin_configure for
a theoretical OPAE C API plugin libfoo.so:
/* foo_plugin.c */
int opae_plugin_configure(opae_api_adapter_table *table, const char *config)
{
adapter->fpgaOpen = dlsym(adapter->plugin.dl_handle, "foo_fpgaOpen");
adapter->fpgaClose =
dlsym(adapter->plugin.dl_handle, "foo_fpgaClose");
...
adapter->fpgaEnumerate =
dlsym(adapter->plugin.dl_handle, "foo_fpgaEnumerate");
...
return 0;
}
Notice that the implementations of the API entry points for plugin libfoo.so
are prefixed with foo_. This is the recommended practice to avoid name
collisions and to enhance the debugability of the application. Upon successful
configuration, opae_plugin_configure returns 0 to indicate success. A
non-zero return value indicates failure and causes the plugin manager to
reject the plugin from futher consideration.
Plugin Optional Functions
Once the plugin manager loads and configures each plugin, it uses the adapter
table to call back into the plugin so that it can be made ready for runtime.
This is the job of the opae_plugin_initialize entry point, whose signature
is defined as:
The function takes no parameters, as the configuration data was already given
to the plugin by opae_plugin_configure. opae_plugin_initialize returns 0
if no errors were encountered during initialization. A non-zero return code
indicates that plugin initialization failed. A plugin makes its
opae_plugin_initialize available to the plugin manager by populating the
adapter table's initialize entry point as shown:
/* foo_plugin.c */
int foo_plugin_initialize(void)
{
...
return 0; /* success */
}
int opae_plugin_configure(opae_api_adapter_table *table, const char *config)
{
...
adapter->initialize =
dlsym(adapter->plugin.dl_handle, "foo_plugin_initialize");
...
return 0;
}
If a plugin does not implement an opae_plugin_initialize entry point, then
the initialize member of the adapter table should be left uninitialized.
During plugin initialization, if a plugin has no opae_plugin_initialize
entry in its adapter table, the plugin initialization step will be skipped,
and the plugin will be considered to have initialized successfully.
Once plugin initialization is complete for all loaded plugins, the system is considered to be running and fully functional.
During teardown, the plugin manager uses the adapter table to call into each
plugin's opae_plugin_finalize entry point, whose signature is defined as:
opae_plugin_finalize returns 0 if no errors were encountered during teardown.
A non-zero return code indicates that plugin teardown failed. A plugin makes
its opae_plugin_finalize available to the plugin manager by populating the
adapter table's finalize entry point as shown:
/* foo_plugin.c */
int foo_plugin_finalize(void)
{
...
return 0; /* success */
}
int opae_plugin_configure(opae_api_adapter_table *table, const char *config)
{
...
adapter->finalize =
dlsym(adapter->plugin.dl_handle, "foo_plugin_finalize");
...
return 0;
}
If a plugin does not implement an opae_plugin_finalize entry point, then
the finalize member of the adapter table should be left uninitialized.
During plugin cleanup, if a plugin has no opae_plugin_finalize entry
point in its adapter table, the plugin finalize step will be skipped, and
the plugin will be considered to have finalized successfully.
In addition to initialize and finalize, an OPAE C API plugin has two
further optional entry points that relate to device enumeration. During
enumeration, when a plugin is being considered for a type of device, the
plugin may provide input on that decision by exporting an
opae_plugin_supports_device entry point in the adapter table:
opae_plugin_supports_device returns true if the given device type is
supported and false if it is not. A false return value from
opae_plugin_supports_device causes device enumeration to skip the
plugin.
Populating the opae_plugin_supports_device is done as:
/* foo_plugin.c */
bool foo_plugin_supports_device(const char *device_type)
{
if (/* device_type is supported */)
return true;
...
return false;
}
int opae_plugin_configure(opae_api_adapter_table *table, const char *config)
{
...
adapter->supports_device =
dlsym(adapter->plugin.dl_handle, "foo_plugin_supports_device");
...
return 0;
}
.. note::
The `opae_plugin_supports_device` mechanism serves as a placeholder only.
It is not implemented in the current version of the OPAE C API.
Similarly to determining whether a plugin supports a type of device, a plugin
may also answer questions about network host support by populating an
opae_plugin_supports_host entry point in the adapter table:
opae_plugin_supports_host returns true if the given hostname is supported
and false if it is not. A false return value from opae_plugin_supports_host
causes device enumeration to skip the plugin.
Populating the opae_plugin_supports_host is done as:
/* foo_plugin.c */
bool foo_plugin_supports_host(const char *hostname)
{
if (/* hostname is supported */)
return true;
...
return false;
}
int opae_plugin_configure(opae_api_adapter_table *table, const char *config)
{
...
adapter->supports_host =
dlsym(adapter->plugin.dl_handle, "foo_plugin_supports_host");
...
return 0;
}
.. note::
The `opae_plugin_supports_host` mechanism serves as a placeholder only.
It is not implemented in the current version of the OPAE C API.
Plugin Construction
The steps required to implement an OPAE C API plugin, libfoo.so, are:
- Create foo_plugin.c: implements
opae_plugin_configure,opae_plugin_initialize,opae_plugin_finalize,opae_plugin_supports_device, andopae_plugin_supports_hostas described in the previous sections. - Create foo_plugin.h: implements function prototypes for each of the plugin-specific OPAE C APIs.
/* foo_plugin.h */
fpga_result foo_fpgaOpen(fpga_token token, fpga_handle *handle,
int flags);
fpga_result foo_fpgaClose(fpga_handle handle);
...
fpga_result foo_fpgaEnumerate(const fpga_properties *filters,
uint32_t num_filters, fpga_token *tokens,
uint32_t max_tokens,
uint32_t *num_matches);
...
- Create foo_types.h: implements plugin-specific types for opaque data structures.
/* foo_types.h */
struct _foo_token {
...
};
struct _foo_handle {
...
};
struct _foo_event_handle {
...
};
struct _foo_object {
...
};
- Create foo_enum.c: implements
foo_fpgaEnumerate,foo_fpgaCloneToken, andfoo_fpgaDestroyToken. - Create foo_open.c: implements
foo_fpgaOpen. - Create foo_close.c: implements
foo_fpgaClose. - Create foo_props.c: implements
foo_fpgaGetProperties,foo_fpgaGetPropertiesFromHandle,foo_fpgaUpdateProperties - Create foo_mmio.c: implements
foo_fpgaMapMMIO,foo_fpgaUnmapMMIOfoo_fpgaWriteMMIO64,foo_fpgaReadMMIO64,foo_fpgaWriteMMIO32,foo_fpgaReadMMIO32. - Create foo_buff.c: implements
foo_fpgaPrepareBuffer,foo_fpgaReleaseBuffer,foo_fpgaGetIOAddress. - Create foo_error.c: implements
foo_fpgaReadError,foo_fpgaClearError,foo_fpgaClearAllErrors,foo_fpgaGetErrorInfo. - Create foo_event.c: implements
foo_fpgaCreateEventHandle,foo_fpgaDestroyEventHandle,foo_fpgaGetOSObjectFromEventHandle,foo_fpgaRegisterEvent,foo_fpgaUnregisterEvent. - Create foo_reconf.c: implements
foo_fpgaReconfigureSlot. - Create foo_obj.c: implements
foo_fpgaTokenGetObject,foo_fpgaHandleGetObject,foo_fpgaObjectGetObject,foo_fpgaDestroyObject,foo_fpgaObjectGetSize,foo_fpgaObjectRead,foo_fpgaObjectRead64,foo_fpgaObjectWrite64. - Create foo_clk.c: implements
foo_fpgaSetUserClock,foo_fpgaGetUserClock.