15.3. Texel Input Operations

Texel input instructions are SPIR-V image instructions that read from an image. Texel input operations are a set of steps that are performed on state, coordinates, and texel values while processing a texel input instruction, and which are common to some or all texel input instructions. They include the following steps, which are performed in the listed order:

For texel input instructions involving multiple texels (for sampling or gathering), these steps are applied for each texel that is used in the instruction. Depending on the type of image instruction, other steps are conditionally performed between these steps or involving multiple coordinate or texel values.

15.3.1. Texel Input Validation Operations

Texel input validation operations inspect instruction/image/sampler state or coordinates, and in certain circumstances cause the texel value to be replaced or become undefined. There are a series of validations that the texel undergoes.

Instruction/Sampler/Image Validation

There are a number of cases where a SPIR-V instruction can mismatch with the sampler, the image, or both. There are a number of cases where the sampler can mismatch with the image. In such cases the value of the texel returned is undefined.

These cases include:

  • The sampler borderColor is an integer type and the image format is not one of the VkFormat integer types or a stencil component of a depth/stencil format.
  • The sampler borderColor is a float type and the image format is not one of the VkFormat float types or a depth component of a depth/stencil format.
  • The sampler borderColor is one of the opaque black colors (VK_BORDER_COLOR_FLOAT_OPAQUE_BLACK or VK_BORDER_COLOR_INT_OPAQUE_BLACK) and the image VkComponentSwizzle for any of the VkComponentMapping components is not VK_COMPONENT_SWIZZLE_IDENTITY.
  • If the instruction is OpImageRead or OpImageSparseRead and the shaderStorageImageReadWithoutFormat feature is not enabled, or the instruction is OpImageWrite and the shaderStorageImageWriteWithoutFormat feature is not enabled, then the SPIR-V Image Format must be compatible with the image view’s format.
  • The sampler unnormalizedCoordinates is VK_TRUE and any of the limitations of unnormalized coordinates are violated.
  • The SPIR-V instruction is one of the OpImage*Dref* instructions and the sampler compareEnable is VK_FALSE
  • The SPIR-V instruction is not one of the OpImage*Dref* instructions and the sampler compareEnable is VK_TRUE
  • The SPIR-V instruction is one of the OpImage*Dref* instructions and the image format is not one of the depth/stencil formats with a depth component, or the image aspect is not VK_IMAGE_ASPECT_DEPTH_BIT.
  • The SPIR-V instruction’s image variable’s properties are not compatible with the image view:

    • Rules for viewType:

      • VK_IMAGE_VIEW_TYPE_1D must have Dim = 1D, Arrayed = 0, MS = 0.
      • VK_IMAGE_VIEW_TYPE_2D must have Dim = 2D, Arrayed = 0.
      • VK_IMAGE_VIEW_TYPE_3D must have Dim = 3D, Arrayed = 0, MS = 0.
      • VK_IMAGE_VIEW_TYPE_CUBE must have Dim = Cube, Arrayed = 0, MS = 0.
      • VK_IMAGE_VIEW_TYPE_1D_ARRAY must have Dim = 1D, Arrayed = 1, MS = 0.
      • VK_IMAGE_VIEW_TYPE_2D_ARRAY must have Dim = 2D, Arrayed = 1.
      • VK_IMAGE_VIEW_TYPE_CUBE_ARRAY must have Dim = Cube, Arrayed = 1, MS = 0.
    • If the image’s samples is not equal to VK_SAMPLE_COUNT_1_BIT, the instruction must have MS = 1.

Integer Texel Coordinate Validation

Integer texel coordinates are validated against the size of the image level, and the number of layers and number of samples in the image. For SPIR-V instructions that use integer texel coordinates, this is performed directly on the integer coordinates. For instructions that use normalized or unnormalized texel coordinates, this is performed on the coordinates that result after conversion to integer texel coordinates.

If the integer texel coordinates satisfy any of the conditions

\begin{align*} i & < 0 & i \geq w_{s} \\ j & < 0 & j \geq h_{s} \\ k & < 0 & k \geq d_{s} \\ l & < 0 & l \geq layers \\ n & < 0 & n \geq samples \end{align*}

where:

\begin{align*} & w_{s} & = \textrm{width of the image level} \\ & h_{s} & = \textrm{height of the image level} \\ & d_{s} & = \textrm{depth of the image level} \\ & layers & = \textrm{number of layers in the image} \\ & samples & = \textrm{number of samples per texel in the image} \end{align*}

then the texel fails integer texel coordinate validation.

There are four cases to consider:

  • Valid Texel Coordinates

    • If the texel coordinates pass validation (that is, the coordinates lie within the image),

      then the texel value comes from the value in image memory.

  • Border Texel

    • If the texel coordinates fail validation, and
    • If the read is the result of an image sample instruction or image gather instruction, and
    • If the image is not a cube image,

      then the texel is a border texel and texel replacement is performed.

  • Invalid Texel

    • If the texel coordinates fail validation, and
    • If the read is the result of an image fetch instruction, image read instruction, or atomic instruction,

      then the texel is an invalid texel and texel replacement is performed.

  • Cube Map Edge or Corner

    • Otherwise the texel coordinates lie on the borders along the edges and corners of a cube map image, and Cube map edge handling is performed.

Cube Map Edge Handling

If the texel coordinates lie on the borders along the edges and corners of a cube map image, the following steps are performed. Note that this only occurs when using VK_FILTER_LINEAR filtering within a miplevel, since VK_FILTER_NEAREST is treated as using VK_SAMPLER_ADDRESS_MODE_CLAMP_TO_EDGE.

  • Cube Map Edge Texel

    • If the texel lies along the border in either only $i$ or only $j$

      then the texel lies along an edge, so the coordinates $(i,j)$ and the array layer $l$ are transformed to select the adjacent texel from the appropriate neighboring face.

  • Cube Map Corner Texel

    • If the texel lies along the border in both $i$ and $j$

      then the texel lies at the corner and there is no unique neighboring face from which to read that texel. The texel should be replaced by the average of the three values of the adjacent texels in each incident face. However, implementations may replace the cube map corner texel by other methods, subject to the constraint that if the three available samples have the same value, the replacement texel also has that value.

Sparse Validation

If the texel reads from an unbound region of a sparse image, the texel is a sparse unbound texel, and processing continues with texel replacement.

15.3.2. Format Conversion

Texels undergo a format conversion from the VkFormat of the image view to a vector of either floating point or signed or unsigned integer components, with the number of components based on the number of components present in the format.

  • Color formats have one, two, three, or four components, according to the format.
  • Depth/stencil formats are one component. The depth or stencil component is selected by the aspectMask of the image view.

Each component is converted based on its type and size (as defined in the Format Definition section for each VkFormat), using the appropriate equations in 16-Bit Floating-Point Numbers, Unsigned 11-Bit Floating-Point Numbers, Unsigned 10-Bit Floating-Point Numbers, Fixed-Point Data Conversion, and Shared Exponent to RGB. Signed integer components smaller than 32 bits are sign-extended.

If the image format is sRGB, the color components are first converted as if they are UNORM, and then sRGB to linear conversion is applied to the R, G, and B components as described in the “KHR_DF_TRANSFER_SRGB” section of the Khronos Data Format Specification. The A component, if present, is unchanged.

If the image view format is block-compressed, then the texel value is first decoded, then converted based on the type and number of components defined by the compressed format.

15.3.3. Texel Replacement

A texel is replaced if it is one (and only one) of:

  • a border texel, or
  • an invalid texel, or
  • a sparse unbound texel.

Border texels are replaced with a value based on the image format and the borderColor of the sampler. The border color is:

Table 15.1. Border Color $B$

Sampler borderColor Corresponding Border Color

VK_BORDER_COLOR_FLOAT_TRANSPARENT_BLACK

$B = (0.0, 0.0, 0.0, 0.0)$

VK_BORDER_COLOR_FLOAT_OPAQUE_BLACK

$B = (0.0, 0.0, 0.0, 1.0)$

VK_BORDER_COLOR_FLOAT_OPAQUE_WHITE

$B = (1.0, 1.0, 1.0, 1.0)$

VK_BORDER_COLOR_INT_TRANSPARENT_BLACK

$B = (0, 0, 0, 0)$

VK_BORDER_COLOR_INT_OPAQUE_BLACK

$B = (0, 0, 0, 1)$

VK_BORDER_COLOR_INT_OPAQUE_WHITE

$B = (1, 1, 1, 1)$


[Note]Note

The names TRANSPARENT_BLACK, OPAQUE_BLACK, and OPAQUE_WHITE are meant to describe which components are zeros and ones in the vocabulary of compositing, and are not meant to imply that the numerical value of WHITE is a saturating value for integers.

This is substituted for the texel value by replacing the number of components in the image format

Table 15.2. Border Texel Components After Replacement

Texel Aspect or Format Component Assignment

Depth aspect

$D = (B_{r})$

Stencil aspect

$S = (B_{r})$

One component color format

$C_{r} = (B_{r})$

Two component color format

$C_{rg} = (B_{r},B_{g})$

Three component color format

$C_{rgb} = (B_{r},B_{g},B_{b})$

Four component color format

$C_{rgba} = (B_{r},B_{g},B_{b},B_{a})$


If the read operation is from a buffer resource, and the robustBufferAccess feature is enabled, an invalid texel is replaced as described here.

If the robustBufferAccess feature is not enabled, the value of an invalid texel is undefined.

[Note]editing-note

(Bill) This is not currently catching this significant case.

For opImageFetch, which fetches from an image not a buffer, the result is defined if robustBufferAccess is enabled.

If the VkPhysicalDeviceSparseProperties property residencyNonResidentStrict is true, a sparse unbound texel is replaced with 0 or 0.0 values for integer and floating-point components of the image format, respectively.

If residencyNonResidentStrict is false, the read must be safe, but the value of the sparse unbound texel is undefined.

15.3.4. Depth Compare Operation

If the image view has a depth/stencil format, the depth component is selected by the aspectMask, and the operation is a Dref instruction, a depth comparison is performed. The value of the result $D$ is $1.0$ if the result of the compare operation is $true$ , and $0.0$ otherwise. The compare operation is selected by the compareOp member of the sampler.

\begin{align*} D & = 1.0 & \begin{cases} D_{ref} \leq D & \textrm{for LEQUAL} \\ D_{ref} \geq D & \textrm{for GEQUAL} \\ D_{ref} < D & \textrm{for LESS} \\ D_{ref} > D & \textrm{for GREATER} \\ D_{ref} = D & \textrm{for EQUAL} \\ D_{ref} \neq D & \textrm{for NOTEQUAL} \\ true & \textrm{for ALWAYS} \\ false & \textrm{for NEVER} \end{cases} \\ D & = 0.0 & \textrm{otherwise} \end{align*}

where, in the depth comparison:

\begin{align*} & D_{ref} = shaderOp.D_{ref} & \textrm{(from optional SPIR-V operand)} \\ & D & \textrm{texel depth value} \end{align*}

15.3.5. Conversion to RGBA

The texel is expanded from one, two, or three to four components based on the image base color:

Table 15.3. Texel Color After Conversion To RGBA

Texel Aspect or Format RGBA Color

Depth aspect

$C_{rgba} = (D,0,0,one)$

Stencil aspect

$C_{rgba} = (S,0,0,one)$

One component color format

$C_{rgba} = (C_{r},0,0,one)$

Two component color format

$C_{rgba} = (C_{rg},0,one)$

Three component color format

$C_{rgba} = (C_{rgb},one)$

Four component color format

$C_{rgba} = C_{rgba}$


where $one = 1.0f$ for floating-point formats and depth aspects, and $one = 1$ for integer formats and stencil aspects.

15.3.6. Component Swizzle

All texel input instructions apply a swizzle based on the VkComponentSwizzle enums in the components member of the VkImageViewCreateInfo structure for the image being read. The swizzle can rearrange the components of the texel, or substitute zero and one for any components. It is defined as follows for the R component, and operates similarly for the other components.

\begin{align*} C'_{rgba}[R] & = \begin{cases} C_{rgba}[R] & \textrm{for RED swizzle} \\ C_{rgba}[G] & \textrm{for GREEN swizzle} \\ C_{rgba}[B] & \textrm{for BLUE swizzle} \\ C_{rgba}[A] & \textrm{for ALPHA swizzle} \\ 0 & \textrm{for ZERO swizzle} \\ one & \textrm{for ONE swizzle} \\ C_{rgba}[R] & \textrm{for IDENTITY swizzle} \end{cases} \end{align*}

where:

\begin{align*} C_{rgba}[R] & \textrm{is the RED component} \\ C_{rgba}[G] & \textrm{is the GREEN component} \\ C_{rgba}[B] & \textrm{is the BLUE component} \\ C_{rgba}[A] & \textrm{is the ALPHA component} \\ one & = 1.0\textrm{f} & \textrm{for floating point components} \\ one & = 1 & \textrm{for integer components} \end{align*}

For each component this is applied to, the VK_COMPONENT_SWIZZLE_IDENTITY swizzle selects the corresponding component from $C_{rgba}$ .

If the border color is one of the VK_BORDER_COLOR_*_OPAQUE_BLACK enums and the VkComponentSwizzle is not VK_COMPONENT_SWIZZLE_IDENTITY for all components (or the equivalent identity mapping), the value of the texel after swizzle is undefined.

15.3.7. Sparse Residency

OpImageSparse* instructions return a structure which includes a residency code indicating whether any texels accessed by the instruction are sparse unbound texels. This code can be interpreted by the OpImageSparseTexelsResident instruction which converts the residency code to a boolean value.