How is the execution configuration different in Warp than CUDA?

[AkeelMedina22]

Hello everyone,

I'm a beginner to CUDA and Warp, and I was wondering how concepts such as Blocks and Threads work in Warp. For example, if I wanted to do a simple dot product, in CUDA I would use several blocks with an ideal number of threads as per the occupancy calculator. Would launching a multidimensional grid be appropriate to mimic this? Or does this happen automatically with a 1-dimensional execution configuration?

[mmacklin]

Hi @AkeelMedina22 , Warp takes care of decomposing kernel thread grids into blocks for you - we currently use a fixed block dimension of 256 threads (regardless of single / multi-dimension launches).

If we were to expose a way set the block dimensions inside wp.launch() would that be useful for you?

[nishadgothoskar]

if the block dimension is always 256, how does the number of blocks get selected? I see the generated cuda code is set up using a for loop, following Grid-Stride Loops, but I haven't been able to identify how the number of blocks is selected.

[shi-eric]

In Warp 1.5.0, block_dim will be exposed as an argument to wp.launch() to control the number of threads per block.

The number of blocks gets selected in the following code:

warp/warp/native/warp.cu

Lines 3150 to 3202 in 1bab08d

	size_t cuda_launch_kernel(void* context, void* kernel, size_t dim, int max_blocks, int block_dim, int shared_memory_bytes, void** args, void* stream)
	{
	ContextGuard guard(context);
	if (block_dim <= 0)
	{
	#if defined(_DEBUG)
	fprintf(stderr, "Warp warning: Launch got block_dim %d. Setting to 256.\n", dim, block_dim);
	#endif
	block_dim = 256;
	}
	// CUDA specs up to compute capability 9.0 says the max x-dim grid is 2**31-1, so
	// grid_dim is fine as an int for the near future
	int grid_dim = (dim + block_dim - 1)/block_dim;
	if (max_blocks <= 0) {
	max_blocks = 2147483647;
	}
	if (grid_dim < 0)
	{
	#if defined(_DEBUG)
	fprintf(stderr, "Warp warning: Overflow in grid dimensions detected for %zu total elements and 256 threads "
	"per block.\n Setting block count to %d.\n", dim, max_blocks);
	#endif
	grid_dim = max_blocks;
	}
	else
	{
	if (grid_dim > max_blocks)
	{
	grid_dim = max_blocks;
	}
	}
	begin_cuda_range(WP_TIMING_KERNEL, stream, context, get_cuda_kernel_name(kernel));
	CUresult res = cuLaunchKernel_f(
	(CUfunction)kernel,
	grid_dim, 1, 1,
	block_dim, 1, 1,
	shared_memory_bytes,
	static_cast<CUstream>(stream),
	args,
	0);
	check_cu(res);
	end_cuda_range(WP_TIMING_KERNEL, stream);
	return res;
	}

We compute the number of blocks that would be required using to process one element of the Warp launch grid per thread. If this results in too many blocks than supported by the CUDA device, then we use grid-stride loops to process more than one element per thread.