Get the tuples and use those to index into the last two dims -
r, c = np.triu_indices(2) out = foo[: , r, c]
Alternatively, one-liner with Ellipsis that works for both 3D and 2D arrays -
foo[(Ellipsis, ) + np.triu_indices(2)]It will work for 2D arrays similarly -
out = foo[r, c] # foo as 2 D input array
2D array case
foo[~np.tri(2, k = -1, dtype = bool)]NumPy’s broadcasting rule relaxes this constraint when the arrays’ shapes meet certain constraints. The simplest broadcasting example occurs when an array and a scalar value are combined in an operation:,When operating on two arrays, NumPy compares their shapes element-wise. It starts with the trailing (i.e. rightmost) dimensions and works its way left. Two dimensions are compatible when,If these conditions are not met, a ValueError: operands could not be broadcast together exception is thrown, indicating that the arrays have incompatible shapes. The size of the resulting array is the size that is not 1 along each axis of the inputs.,In the following example, both the A and B arrays have axes with length one that are expanded to a larger size during the broadcast operation:
>>> a = np.array([1.0, 2.0, 3.0]) >>> b = np.array([2.0, 2.0, 2.0]) >>> a * b array([2., 4., 6.])
>>> a = np.array([1.0, 2.0, 3.0]) >>> b = 2.0 >>> a * b array([2., 4., 6.])
Image(3 d array): 256 x 256 x 3
Scale(1 d array): 3
Result(3 d array): 256 x 256 x 3A(4 d array): 8 x 1 x 6 x 1
B(3 d array): 7 x 1 x 5
Result(4 d array): 8 x 7 x 6 x 5A(2 d array): 5 x 4
B(1 d array): 1
Result(2 d array): 5 x 4
A(2 d array): 5 x 4
B(1 d array): 4
Result(2 d array): 5 x 4
A(3 d array): 15 x 3 x 5
B(3 d array): 15 x 1 x 5
Result(3 d array): 15 x 3 x 5
A(3 d array): 15 x 3 x 5
B(2 d array): 3 x 5
Result(3 d array): 15 x 3 x 5
A(3 d array): 15 x 3 x 5
B(2 d array): 3 x 1
Result(3 d array): 15 x 3 x 5A(1 d array): 3
B(1 d array): 4 # trailing dimensions do not match
A(2 d array): 2 x 1
B(3 d array): 8 x 4 x 3 # second from last dimensions mismatchedGet the tuples and use those to index anycodings_python into the last two dims -,How can we broadcast tuple indexing along anycodings_multidimensional-array the batch dimensions?,RuntimeError: The expanded size of the tensor (0) must match the existing size (5) at non-singleton dimension 0,and I want to obtain the upper triangular anycodings_multidimensional-array indices of each matrix, the naive thing to anycodings_multidimensional-array try would be:
For example, anycodings_multidimensional-array np.array([[1,2],[3,4]])[np.triu_indices(2)] anycodings_multidimensional-array has shape (3,), being a flattened list of anycodings_multidimensional-array the upper triangular entries. However, if I anycodings_multidimensional-array have a batch of 2x2 matrices:
foo = np.repeat(np.array([
[
[1, 2],
[3, 4]
]
]), 30, axis = 0)and I want to obtain the upper triangular anycodings_multidimensional-array indices of each matrix, the naive thing to anycodings_multidimensional-array try would be:
foo[: , np.triu_indices(2)]Get the tuples and use those to index anycodings_python into the last two dims -
r, c = np.triu_indices(2) out = foo[: , r, c]
Alternatively, one-liner with Ellipsis anycodings_python that works for both 3D and 2D arrays -
foo[(Ellipsis, ) + np.triu_indices(2)]It will work for 2D arrays similarly -
out = foo[r, c] # foo as 2 D input array
2D array case
foo[~np.tri(2, k = -1, dtype = bool)]We saw in the previous section how NumPy's universal functions can be used to vectorize operations and thereby remove slow Python loops. Another means of vectorizing operations is to use NumPy's broadcasting functionality. Broadcasting is simply a set of rules for applying binary ufuncs (e.g., addition, subtraction, multiplication, etc.) on arrays of different sizes.,Broadcasting allows these types of binary operations to be performed on arrays of different sizes–for example, we can just as easily add a scalar (think of it as a zero-dimensional array) to an array:,One place that broadcasting is very useful is in displaying images based on two-dimensional functions. If we want to define a function $z = f(x, y)$, broadcasting can be used to compute the function across the grid:,Also note that while we've been focusing on the + operator here, these broadcasting rules apply to any binary ufunc. For example, here is the logaddexp(a, b) function, which computes log(exp(a) + exp(b)) with more precision than the naive approach:
import numpy as npa = np.array([0, 1, 2]) b = np.array([5, 5, 5]) a + b
array([5, 6, 7])a + 5M = np.ones((3, 3)) M
array([ [1., 1., 1.], [1., 1., 1.], [1., 1., 1.] ])
TensorFlow follows standard Python indexing rules, similar to indexing a list or a string in Python, and the basic rules for NumPy indexing.,The base tf.Tensor class requires tensors to be "rectangular"---that is, along each axis, every element is the same size. However, there are specialized types of tensors that can handle different shapes:,Read the tensor slicing guide to learn how you can apply indexing to manipulate individual elements in your tensors.,The strings are atomic and cannot be indexed the way Python strings are. The length of the string is not one of the axes of the tensor. See tf.strings for functions to manipulate them.
import tensorflow as tf
import numpy as npHere is a "scalar" or "rank-0" tensor . A scalar contains a single value, and no "axes".
# This will be an int32 tensor by default; see "dtypes" below. rank_0_tensor = tf.constant(4) print(rank_0_tensor)
# This will be an int32 tensor by default; see "dtypes" below.
rank_0_tensor = tf.constant(4)
print(rank_0_tensor)
tf.Tensor(4, shape = (), dtype = int32)
# Let's make this a float tensor.
rank_1_tensor = tf.constant([2.0, 3.0, 4.0])
print(rank_1_tensor)
tf.Tensor([2. 3. 4.], shape = (3, ), dtype = float32)
A "matrix" or "rank-2" tensor has two axes:
# If you want to be specific, you can set the dtype(see below) at creation time rank_2_tensor = tf.constant([ [1, 2], [3, 4], [5, 6] ], dtype = tf.float16) print(rank_2_tensor)
bidirectional - the broadcast rule is similar to numpy.array(input) \* numpy.ones(target_shape). Dimensions are right alignment. Two corresponding dimension must have the same value, or one of them is equal to 1. If this attribute value is used, then the 3rd input for the operation shouldn’t be provided.,In case if mode is set to bidirectional, then the broadcast rule is similar to numpy.array(input) \* numpy.ones(target_shape). Dimensions are right alignment. Two corresponding dimension must have the same value, or one of them is equal to 1. If this attribute value is used, then the 3rd input for the operation shouldn’t be provided. The behaviour is described in Bidirectional Broadcast Rules.,numpy - numpy broadcasting rules, aligned with ONNX Broadcasting. Description is available in ONNX docs.; only one-directional broadcasting is applied from data to target_shape. If this attribute value is used, then the 3rd input for the operation shouldn’t be provided.,Short description : Broadcast replicates data on the first input to fit a given shape on the second input.
<layer ... type="Broadcast" ...>
<data mode="numpy" />
<input>
<port id="0">
<dim>16</dim>
<dim>1</dim>
<dim>1</dim>
</port>
<port id="1">
<dim>4</dim> <!--The tensor contains 4 elements: [1, 16, 50, 50] -->
</port>
<!-- the 3rd input shouldn't be provided with mode="numpy" -->
</input>
<output>
<port id="2">
<dim>1</dim>
<dim>16</dim>
<dim>50</dim>
<dim>50</dim>
</port>
</output>
</layer>
<layer ... type="Broadcast" ...>
<data mode="explicit" />
<input>
<port id="0">
<dim>16</dim>
</port>
<port id="1">
<dim>4</dim> <!--The tensor contains 4 elements: [1, 16, 50, 50] -->
</port>
<port id="1">
<dim>1</dim> <!--The tensor contains 1 elements: [1] -->
</port>
</input>
<output>
<port id="2">
<dim>1</dim>
<dim>16</dim>
<dim>50</dim>
<dim>50</dim>
</port>
</output>
</layer>
<layer ... type="Broadcast" ...>
<data mode="explicit" />
<input>
<port id="0">
<dim>50</dim>
<dim>50</dim>
</port>
<port id="1">
<dim>4</dim> <!--The tensor contains 4 elements: [1, 50, 50, 16] -->
</port>
<port id="1">
<dim>2</dim> <!--The tensor contains 2 elements: [1, 2] -->
</port>
</input>
<output>
<port id="2">
<dim>1</dim>
<dim>50</dim>
<dim>50</dim>
<dim>16</dim>
</port>
</output>
</layer>
<layer ... type="Broadcast" ...>
<data mode="bidirectional" />
<input>
<port id="0">
<dim>16</dim>
<dim>1</dim>
<dim>1</dim>
</port>
<port id="1">
<dim>4</dim> <!--The tensor contains 4 elements: [1, 1, 50, 50] -->
</port>
<!-- the 3rd input shouldn't be provided with mode="bidirectional" -->
</input>
<output>
<port id="2">
<dim>1</dim>
<dim>16</dim>
<dim>50</dim>
<dim>50</dim>
</port>
</output>
</layer>Then, for each dimension size, the resulting dimension size is the max of the sizes of x and y along that dimension.,When iterating over the dimension sizes, starting at the trailing dimension, the dimension sizes must either be equal, one of them is 1, or one of them does not exist.,In short, if a PyTorch operation supports broadcast, then its Tensor arguments can be automatically expanded to be of equal sizes (without making copies of the data).,Many PyTorch operations support NumPy’s broadcasting semantics. See https://numpy.org/doc/stable/user/basics.broadcasting.html for details.
>>> x = torch.empty(5, 7, 3) >>> y = torch.empty(5, 7, 3) # same shapes are always broadcastable(i.e.the above rules always hold) >>> x = torch.empty((0, )) >>> y = torch.empty(2, 2) # x and y are not broadcastable, because x does not have at least 1 dimension # can line up trailing dimensions >>> x = torch.empty(5, 3, 4, 1) >>> y = torch.empty(3, 1, 1) # x and y are broadcastable. # 1 st trailing dimension: both have size 1 # 2n d trailing dimension: y has size 1 # 3 rd trailing dimension: x size == y size # 4 th trailing dimension: y dimension doesn 't exist # but: >>> x = torch.empty(5, 2, 4, 1) >>> y = torch.empty(3, 1, 1) # x and y are not broadcastable, because in the 3 rd trailing dimension 2 != 3
# can line up trailing dimensions to make reading easier >>> x = torch.empty(5, 1, 4, 1) >>> y = torch.empty(3, 1, 1) >>> (x + y).size() torch.Size([5, 3, 4, 1]) # but not necessary: >>> x = torch.empty(1) >>> y = torch.empty(3, 1, 7) >>> (x + y).size() torch.Size([3, 1, 7]) >>> x = torch.empty(5, 2, 4, 1) >>> y = torch.empty(3, 1, 1) >>> (x + y).size() RuntimeError: The size of tensor a(2) must match the size of tensor b(3) at non - singleton dimension 1
>>> x = torch.empty(5, 3, 4, 1) >>> y = torch.empty(3, 1, 1) >>> (x.add_(y)).size() torch.Size([5, 3, 4, 1]) # but: >>> x = torch.empty(1, 3, 1) >>> y = torch.empty(3, 1, 7) >>> (x.add_(y)).size() RuntimeError: The expanded size of the tensor(1) must match the existing size(7) at non - singleton dimension 2.
>>> torch.add(torch.ones(4, 1), torch.randn(4))
>>> torch.utils.backcompat.broadcast_warning.enabled = True >>>
torch.add(torch.ones(4, 1), torch.ones(4))
__main__: 1: UserWarning: self and other do not have the same shape, but are broadcastable, and have the same number of elements.
Changing behavior in a backwards incompatible manner to broadcasting rather than viewing as 1 - dimensional.