make numpy matrix with insufficient length of data

I'm sure there are various ways of doing this. My first inclination is to make a output array filled with the 'fill', and copy the data to it. Since the fill is 'ragged', not a full column or row, I'd start out 1d and reshape to the final shape.

In[730]: row, col = 3, 4
In[731]: data = [0, 4, 1, 3, 2, 5, 9, 6, 7, 8]
In[732]: output = np.zeros(row * col, dtype = int) - 1
In[733]: output[: len(data)] = data
In[734]: output = output.reshape(3, 4)
In[735]: output
Out[735]:
   array([
      [0, 4, 1, 3],
      [2, 5, 9, 6],
      [7, 8, -1, -1]
   ])

Use np.ndarray.flat to index into the flattened version of the array.

data = [0, 4, 1, 3, 2, 5, 9, 6, 7, 8]
default_value = -1
desired_shape = (3, 4)
output = default_value * np.ones(desired_shape)
output.flat[: len(data)] = data

# output is now:
   # array([
      [0., 4., 1., 3.],
      #[2., 5., 9., 6.],
      #[7., 8., -1., -1.]
   ])

M. Klugerford's solution (icreasing the data and reshaping):

% timeit data = [x
   for x in range(100000)
];
col = 15000;
row = 15000;
data += [-1] * (row * col - len(data));
output = np.array(data).reshape((row, col))

Psidom's solution (using np.pad):

 % timeit
 import numpy as np;
 data = [x
    for x in range(100000)
 ];
 col = 15000;
 row = 15000;
 np.pad(data, (0, row * col - len(data)), 'constant', constant_values = -1).reshape(row, col)

Praveen's solution (using np.ndarray.flat):

% timeit
import numpy as np;
data = [x
   for x in range(100000)
];
col = 15000;
row = 15000;
output = -1 * np.ones((col, row));
output.flat[: len(data)] = data

Here is one option using numpy.pad, pad the data with -1 at the end of array and then reshape it:

import numpy as np
data = [0, 4, 1, 3, 2, 5, 9, 6, 7, 8]
row, col = 3, 4

np.pad(data, (0, row * col - len(data)), 'constant', constant_values = -1).reshape(row, col)

# array([
   [0, 4, 1, 3],
   #[2, 5, 9, 6],
   #[7, 8, -1, -1]
])

Suggestion : 2

NumPy is the fundamental package for scientific computing in Python. It is a Python library that provides a multidimensional array object, various derived objects (such as masked arrays and matrices), and an assortment of routines for fast operations on arrays, including mathematical, logical, shape manipulation, sorting, selecting, I/O, discrete Fourier transforms, basic linear algebra, basic statistical operations, random simulation and much more.,NumPy arrays facilitate advanced mathematical and other types of operations on large numbers of data. Typically, such operations are executed more efficiently and with less code than is possible using Python’s built-in sequences.,At the core of the NumPy package, is the ndarray object. This encapsulates n-dimensional arrays of homogeneous data types, with many operations being performed in compiled code for performance. There are several important differences between NumPy arrays and the standard Python sequences:,The elements in a NumPy array are all required to be of the same data type, and thus will be the same size in memory. The exception: one can have arrays of (Python, including NumPy) objects, thereby allowing for arrays of different sized elements.

c = []
for i in range(len(a)):
   c.append(a[i] * b[i])

for (i = 0; i < rows; i++) {
   c[i] = a[i] * b[i];
}

for (i = 0; i < rows; i++) {
   for (j = 0; j < columns; j++) {
      c[i][j] = a[i][j] * b[i][j];
   }
}

c = a * b

Suggestion : 3

Here we see how to speed up NumPy array processing using Cython. By explicitly declaring the "ndarray" data type, your array processing can be 1250x faster. ,This tutorial will show you how to speed up the processing of NumPy arrays using Cython. By explicitly specifying the data types of variables in Python, Cython can give drastic speed increases at runtime.,This tutorial discussed using Cython for manipulating NumPy arrays with a speed of more than 1000x times Python processing alone. The key for reducing the computational time is to specify the data types for the variables, and to index the array rather than iterate through it.,The datatype of the array elements is int and defined according to the line below. The numpy imported using cimport has a type corresponding to each type in NumPy but with _t at the end. For example, int in regular NumPy corresponds to int_t in Cython.

We'll start with the same code as in the previous tutorial, except here we'll iterate through a NumPy array rather than a list. The NumPy array is created in the arr variable using the arrange() function, which returns one billion numbers starting from 0 with a step of 1.

import time
import numpy

total = 0
arr = numpy.arange(1000000000)

t1 = time.time()

for k in arr:
   total = total + k
print("Total = ", total)

t2 = time.time()
t = t2 - t1
print("%.20f" % t)

Let's see how much time it takes to complete after editing the Cython script created in the previous tutorial, as given below. The only change is the inclusion of the NumPy array in the for loop. Note that you have to rebuild the Cython script using the command below before using it.

python setup.py build_ext--inplace

The maxval variable is set equal to the length of the NumPy array. We can start by creating an array of length 10,000 and increase this number later to compare how Cython improves compared to Python.

import time
import numpy
cimport numpy

cdef unsigned long long int maxval
cdef unsigned long long int total
cdef int k
cdef double t1, t2, t
cdef numpy.ndarray arr

maxval = 10000
arr = numpy.arange(maxval)

t1 = time.time()

for k in arr:
   total = total + k
print "Total =", total

t2 = time.time()
t = t2 - t1
print("%.20f" % t)

The argument is ndim, which specifies the number of dimensions in the array. It is set to 1 here. Note that its default value is also 1, and thus can be omitted from our example. If more dimensions are being used, we must specify it.

cdef numpy.ndarray[numpy.int_t, ndim = 1] arr

Note that we defined the type of the variable arr to be numpy.ndarray, but do not forget that this is the type of the container. This container has elements and these elements are translated as objects if nothing else is specified. To force these elements to be integers, the dtype argument is set to numpy.int according to the next line.

arr = numpy.arange(maxval, dtype = numpy.int)

Suggestion : 4

NumPy arrays have a fixed size at creation, unlike Python lists (which can grow dynamically). Changing the size of an ndarray will create a new array and delete the original.,The elements in a NumPy array are all required to be of the same data type, and thus will be the same size in memory. The exception: one can have arrays of (Python, including NumPy) objects, thereby allowing for arrays of different sized elements.,NumPy arrays facilitate advanced mathematical and other types of operations on large numbers of data. Typically, such operations are executed more efficiently and with less code than is possible using Python’s built-in sequences.,15 - Using a from part 14, create a new list c by calling a = c. Now change the shape of c. Comment on your results.

1._

In[1]:

   import numpy as np

2._

import numpy as np

import numpy as np

1._

In[2]:

   b = np.arange(1, 3).reshape(2, 1) # Using reshape
#b = np.arange(1, 3)[: , np.newaxis] # Using newaxis
print(b)

2._

b = np.arange(1, 3).reshape(2, 1) # Using reshape
#b = np.arange(1, 3)[: , np.newaxis] # Using newaxis
print(b)

[
   [1]
   [2]
]

[
   [1]
   [2]
]

1._

In[3]:

   X = np.arange(1, 5).reshape(2, 2)
print(X)

2._

X = np.arange(1, 5).reshape(2, 2)
print(X)

[
   [1 2]
   [3 4]
]

Suggestion : 5

The numpy.array.reshape() function can be called with an “unknown dimension”.,This is possible by specifying -1 as the unspecified dimension.,To recap, -1 is an “unknown dimension”. The reshape() function calculates this dimension for you based on the context.,Again, the reshape() function treats the -1 as an unknown dimension.

For example:

import numpy as np

A = np.array([
   [1, 2],
   [3, 4]
])

B = A.reshape(-1) # result: [1, 2, 3, 4]

For instance, let’s convert a 1D array to a 3D array with 2×2 elements:

import numpy as np

A = np.array([1, 2, 3, 4, 5, 6, 7, 8])

B = A.reshape(2, 2, -1)

print(B)

Output:

Anyway, let’s see an example of reshaping in action by converting a 1D array to a 2D array.

import numpy as np

# 1 x 8 matrix
A = np.array([1, 2, 3, 4, 5, 6, 7, 8])

# 2 x 4 matrix
B = A.reshape(2, 4)

print(B)

Here is how it looks in code:

import numpy as np

# 1 x 12 matrix
A = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12])

# A matrix with three 2 x 2 matrices(a tensor)
B = A.reshape(3, 2, 2)

print(B)