15.1. Image Operations Overview

Image Operations are steps performed by SPIR-V image instructions, where those instructions which take an OpTypeImage (representing a VkImageView) or OpTypeSampledImage (representing a (VkImageView, VkSampler) pair) and texel coordinates as operands, and return a value based on one or more neighboring texture elements (texels) in the image.

[Note]Note

Texel is a term which is a combination of the words texture and element. Early interactive computer graphics supported texture operations on textures, a small subset of the image operations on images described here. The discrete samples remain essentially equivalent, however, so we retain the historical term texel to refer to them.

SPIR-V Image Instructions include the following functionality:

15.1.1. Texel Coordinate Systems

Images are addressed by texel coordinates. There are three texel coordinate systems:

  • normalized texel coordinates (coordinates ranging from 0 to 1 span the image),
  • unnormalized texel coordinates (floating point coordinates ranging from 0 to width/height/depth span the image), and
  • integer texel coordinates (integer coordinates ranging from 0 to width-1/height-1/depth-1 address the texels within the image).

SPIR-V OpImageFetch, OpImageSparseFetch, OpImageRead, OpImageSparseRead, and OpImageWrite instructions use integer texel coordinates. Other image instructions can use either normalized or unnormalized texel coordinates (selected by the unnormalizedCoordinates state of the sampler used in the instruction), but there are limitations on what operations, image state, and sampler state is supported. Normalized coordinates are logically converted to unnormalized as part of image operations, and certain steps are only performed on normalized coordinates. The array layer coordinate is always treated as unnormalized even when other coordinates are normalized.

Normalized texel coordinates are referred to as $(s,t,r,q,a)$ , with the coordinates having the following meanings:

  • s: Coordinate in the first dimension of an image.
  • t: Coordinate in the second dimension of an image.
  • r: Coordinate in the third dimension of an image.

    • (s,t,r) are interpreted as a direction vector for Cube images.
  • q: Fourth coordinate, for homogeneous (projective) coordinates.
  • a: Coordinate for array layer.

The coordinates are extracted from the SPIR-V operand based on the dimensionality of the image variable and type of instruction. For Proj instructions, the components are in order (s, [t,] [r,] q) with t and r being conditionally present based on the Dim of the image. For non-Proj instructions, the coordinates are (s [,t] [,r] [,a]), with t and r being conditionally present based on the Dim of the image and a being conditionally present based on the Arrayed property of the image. Projective image instructions are not supported on Arrayed images.

Unnormalized texel coordinates are referred to as $(u,v,w,a)$ , with the coordinates having the following meanings:

  • u: Coordinate in the first dimension of an image.
  • v: Coordinate in the second dimension of an image.
  • w: Coordinate in the third dimension of an image.
  • a: Coordinate for array layer.

Only the u and v coordinates are directly extracted from the SPIR-V operand, because only 1D and 2D (non-Arrayed) dimensionalities support unnormalized coordinates. The components are in order (u [,v]), with v being conditionally present when the dimensionality is 2D. When normalized coordinates are converted to unnormalized coordinates, all four coordinates are used.

Integer texel coordinates are referred to as $(i,j,k,l,n)$ , and the first four in that order have the same meanings as unnormalized texel coordinates. They are extracted from the SPIR-V operand in order (i, [,j], [,k], [,l]), with j and k conditionally present based on the Dim of the image, and l conditionally present based on the Arrayed property of the image. n is the sample index and is taken from the Sample image operand.

For all coordinate types, unused coordinates are assigned a value of zero.

images/vulkantexture0.png

The Texel Coordinate Systems - For the example shown of an 8x4 texel two dimensional image.

  • Normalized texel coordinates:

    • The s coordinate goes from 0.0 to 1.0, left to right.
    • The t coordinate goes from 0.0 to 1.0, top to bottom.
  • Unnormalized texel coordinates:

    • The u coordinate goes from -1.0 to 9.0, left to right. The u coordinate within the range 0.0 to 8.0 is within the image, otherwise it is within the border.
    • The v coordinate goes from -1.0 to 5.0, top to bottom. The v coordinate within the range 0.0 to 4.0 is within the image, otherwise it is within the border.
  • Integer texel coordinates:

    • The i coordinate goes from -1 to 8, left to right. The i coordinate within the range 0 to 7 addresses texels within the image, otherwise it addresses a border texel.
    • The j coordinate goes from -1 to 5, top to bottom. The j coordinate within the range 0 to 3 addresses texels within the image, otherwise it addresses a border texel.
  • Also shown for linear filtering:

    • Given the unnormalized coordinates (u,v), the four texels selected are i0j0, i1j0, i0j1 and i1j1.
    • The weights $\alpha$ and $\beta$ .
    • Given the offset $\Delta_{i}$ and $\Delta_{j}$ , the four texels selected by the offset are i0j0', i1j0', i0j1' and i1j1'.
images/vulkantexture1.png

The Texel Coordinate Systems - For the example shown of an 8x4 texel two dimensional image.

  • Texel coordinates as above. Also shown for nearest filtering:

    • Given the unnormalized coordinates (u,v), the texel selected is ij.
    • Given the offset $\Delta_{i}$ and $\Delta_{j}$ , the texel selected by the offset is ij'.