Profiling Hooks

This page documents the interface between Kokkos and the profiling library. Every function prototype on this page is an interface hook. Profiling libraries may define any subset of the hooks listed here; hooks which are not defined by the library will be silently ignored by Kokkos. The hooks have C linkage, and we emphasize this with the extern "C" required to define such symbols in C++. If the profiling library is written in C, the extern "C" should be omitted.

KokkosPDeviceInfo

struct KokkosPDeviceInfo {
  uint32_t deviceID;
};

SpaceHandle

struct SpaceHandle {
  char name[64];                                                                        
};

This handle describes a Kokkos memory space. The name member is a zero-padded string which currently can take the values "Host" or "Cuda".

kokkosp_init_library

extern "C" void kokkosp_init_library(int loadseq, uint64_t version, uint32_t ndevinfos, KokkosPDeviceInfo* devinfos);

This function will be called only once, prior to calling any other hooks in the profiling library. Currently the only argument which is non-zero is version, which will specify the version of the interface (which will allow future changes to the interface). The version is an integer encoding a date as ((year*100)+month)*100, and the current interface version is 20150628.

kokkosp_print_help

extern "C" void kokkosp_print_help(char* progName)

This function will only be called at most once. If a user invokes their application with --kokkos-tools-help, this will be invoked, and you should print a help message detailing the options the user can pass to --kokkos-tools-args (displayed below)

kokkosp_parse_args

extern "C" void kokkosp_parse_args(int, char**)`

This function will only be called once. A user may call their application with --kokkos-tools-args=[args], and those args will be forwarded to your tool in this function. This can be used to configure your tool to use (or not use) certain capabilities

kokkosp_finalize_library

extern "C" void kokkosp_finalize_library();

This function will be called only once, after all other calls to profiling hooks.

kokkosp_begin_parallel_*

extern "C" void kokkosp_begin_parallel_for(const char* name, uint32_t devid, uint64_t* kernid);
extern "C" void kokkosp_begin_parallel_reduce(const char* name, uint32_t devid, uint64_t* kernid);
extern "C" void kokkosp_begin_parallel_scan(const char* name, uint32_t devid, uint64_t* kernid);
extern "C" void kokkosp_begin_fence(const char* name, uint32_t devid, uint64_t* kernid);

These functions are called before their respective parallel constructs execute (Kokkos::parallel_for, Kokkos::parallel_reduce, Kokkos::parallel_scan, Kokkos::fence/Kokkos::ExecutionSpace.fence()). The name argument is the name given by the user to the parallel construct, or in the case no name was given it is the compiler-dependent type name of the functor or lambda given to the construct. For fences, the name is either user-provided or derived from the type being fenced. Currently devid is always zero. kernid is an output variable: the profiling library assigns a value to this, and that value will be given to the corresponding kokkosp_end_* call at the end of the parallel construct or fence.

kokkosp_end_parallel_*

extern "C" void kokkosp_end_parallel_for(uint64_t kernid);
extern "C" void kokkosp_end_parallel_reduce(uint64_t kernid);
extern "C" void kokkosp_end_parallel_scan(uint64_t kernid);
extern "C" void kokkosp_end_fence(uint64_t kernid);

kokkosp_push_profile_region

extern "C" void kokkosp_push_profile_region(const char* name);

This function will be called by Kokkos::Profiling::pushRegion(const std::string& cpp_name). The name passed to it is the C equivalent of the cpp_name given to Kokkos::Profiling::pushRegion. As the function names imply, regions are meant to be treated in a stack fashion, ideally consistent with the calling stack of the application. One natural way to use them is to call pushRegion at the beginning of an application function, and call popRegion at the end of the application function. This helps the profiling library group other events like parallel_for calls and memory allocations, and organize them according to the higher-level flow of the application for better presentation to the user.

kokkosp_pop_profile_region

extern "C" void kokkosp_pop_profile_region();

This function will be called by Kokkos::Profiling::popRegion(). In accordance with the stack convention, the region being popped is the one named by the last call to pushRegion.

kokkosp_allocate_data

extern "C" void kokkosp_allocate_data(SpaceHandle handle, const char* name, void* ptr, uint64_t size);

This function will be called whenever a shared allocation is created to support a Kokkos::View. The handle refers to the Kokkos MemorySpace where the memory resides, the name is the name given by the user to the View. The ptr and size parameters describe the block of memory as its starting pointer and size in bytes.

kokkosp_deallocate_data

extern "C" void kokkosp_deallocate_data(SpaceHandle handle, const char* name, void* ptr, uint64_t size);

This function will be called whenever a shared allocation is destroyed. The handle refers to the Kokkos MemorySpace where the memory resides, the name is the name given by the user to the View. The ptr and size parameters describe the block of memory as its starting pointer and size in bytes.

kokkosp_begin_deep_copy

extern "C" void kokkosp_begin_deep_copy(
    SpaceHandle dst_handle, const char* dst_name, const void* dst_ptr,
    SpaceHandle src_handle, const char* src_name, const void* src_ptr,
    uint64_t size);

This function will be called whenever a Kokkos::deep_copy function is called on a contiguous view (i.e. it is not a remapping operation from for example one layout to another). The dst_handle and src_handle refer to the MemorySpace of the destination and source allocations respectively. dst_name and src_name are the user provided names of the allocations, while dst_ptr and src_ptr are the respective data pointers. size is the size in bytes of the allocations.

kokkosp_end_deep_copy

extern "C" void kokkosp_end_deep_copy();

This function marks the end of a Kokkos::deep_copy call following a kokkosp_begind_deep_copy call.

kokkosp_create_profile_section

extern "C" void kokkosp_create_profile_section( const char* name, uint32_t* sec_id);

Create a profiling section handle. Sections can overlap with each other in contrast to Regions which behave like a stack. name is a user provided name for the section and sec_id is used to return a section identifier to the user. Note that sec_id should be unique for an open section, but name may not be unique. Hence there may be multiple sections with the same name.

kokkosp_start_profile_section

extern "C" void kokkosp_start_profile_section( uint32_t sec_id);

Start a profiling section using a previously created section id. A profiling section can be started multiple times, assuming it was first stopped each time.

kokkosp_stop_profile_section

extern "C" void kokkosp_stop_profile_section( uint32_t sec_id);

Stop a profiling section using a previously created section id.

kokkosp_destroy_profile_section

extern "C" void kokkosp_destroy_profile_section( uint32_t sec_id);

Destroy a previously created profiling section.

kokkosp_profile_event

extern "C" void kokkosp_profile_event( const char* name );

Marks an event during an application with a user provided name.