simpler way to filter out nan or invalid's in python array or a numpy array?

  • Last Update :
  • Techknowledgy :

It works exactly like Matlab but with slightly different syntax

heights = heights[~np.isnan(heights)]

Demo

>>>
import numpy as np
   >>>
   heights = np.array([1.0, 3.0, np.nan, -10.0]) >>>
   np.asarray([h
      for h in heights
      if not np.isnan(h)
   ])
array([1., 3., -10.]) >>>
   heights[~np.isnan(heights)]
array([1., 3., -10.]) >>>

This works:

heights[(10 <= heights) & (heights <= 90)]

This fails with an error about ambiguous truth value

heights[10 <= heights & heights <= 90]

You should have specified the error, not just say it 'fails'. The error message gives us some clues as to why it fails. It usually means that it's trying to do a simple 'if then' evaluation with a boolean array, though it isn't obvious in this case how that is happening.

10 <= heights & heights <= 90 # also fails(10 <= heights) & (heights <= 90) # fine
heights & heights <= 90 # ok
20 <= (heights & heights) <= 80 # same error
20 <= heights <= 80 # error
20 <= heights[0] <= 80 # True

Suggestion : 2

I have numpy array heights which may have nan's in it. I clean it by doing:, 6 days ago 1. I have numpy array heights which may have nan 's in it. I clean it by doing: heights = numpy.asarray ( [ h for h in heights if not numpy.isnan (h) ]) This seems like a rather verbose way to express such a simple/common thing. I often have to do this as well for filtering my arrays in other ways and have to fall back on array building, which ... ,fails in python, where heights is still a numpy array. I fall back to doing...,This seems like a rather verbose way to express such a simple/common thing. I often have to do this as well for filtering my arrays in other ways and have to fall back on array building, which works but I bet there is a better way to do it. E.g. filtering by range...


heights = numpy.asarray([h
   for h in heights
   if not numpy.isnan(h)
])

heights[(10 <= heights) & (heights <= 90)]
heights = numpy.asarray([h
   for h in heights
   if not numpy.isnan(h)
])
heights = numpy.asarray(heights[lowerBound <= heights & heights < upperBound])
heights = numpy.asarray(heights[[h
   for h in heights
   if lowerBound <= h and h < upperBound
]])
heights = heights(~isnan(heights));

Suggestion : 3

Return a new “bytes” object which is an immutable sequence of integers in the range 0 <= x < 256. bytes is an immutable version of bytearray – it has the same non-mutating methods and the same indexing and slicing behavior.,Convert an integer number to a binary string prefixed with “0b”. The result is a valid Python expression. If x is not a Python int object, it has to define an __index__() method that returns an integer. Some examples:,Convert an integer number to a lowercase hexadecimal string prefixed with “0x”. If x is not a Python int object, it has to define an __index__() method that returns an integer. Some examples:,Convert an integer number to an octal string prefixed with “0o”. The result is a valid Python expression. If x is not a Python int object, it has to define an __index__() method that returns an integer. For example:

def all(iterable):
   for element in iterable:
   if not element:
   return False
return True
def any(iterable):
   for element in iterable:
   if element:
   return True
return False
>>> bin(3)
'0b11' >>>
bin(-10)
'-0b1010'
>>> format(14, '#b'), format(14, 'b')
   ('0b1110', '1110') >>>
   f '{14:#b}', f '{14:b}'
   ('0b1110', '1110')
class C:
   @classmethod
def f(cls, arg1, arg2): ...
>>>
import struct
   >>>
   dir() # show the names in the module namespace['__builtins__', '__name__', 'struct'] >>>
   dir(struct) # show the names in the struct module['Struct', '__all__', '__builtins__', '__cached__', '__doc__', '__file__',
      '__initializing__', '__loader__', '__name__', '__package__',
      '_clearcache', 'calcsize', 'error', 'pack', 'pack_into',
      'unpack', 'unpack_from'] >>>
   class Shape:
   ...def __dir__(self):
   ...
   return ['area', 'perimeter', 'location'] >>>
      s = Shape() >>>
      dir(s)['area', 'location', 'perimeter']

Suggestion : 4

To mask an array where invalid values occur (NaNs or infs), use the numpy.ma.masked_invalid() method. Here, we will set the interval i.e. to mask between 55 and 90 −,To mask an array where invalid values occur (NaNs or infs), use the numpy.ma.masked_invalid() method in Python Numpy. This function is a shortcut to masked_where, with condition = ~(np.isfinite(a)). Any pre-existing mask is conserved. Only applies to arrays with a dtype where NaNs or infs make sense (i.e. floating point types), but accepts any array_like object.,A masked array is the combination of a standard numpy.ndarray and a mask. A mask is either nomask, indicating that no value of the associated array is invalid, or an array of booleans that determines for each element of the associated array whether the value is valid or not.,Mask array elements greater than or equal to a given value in Numpy

At first, import the required library −

import numpy as np
import numpy.ma as ma

Create an array with float elements using the numpy.array() method −

arr = np.array([91.6, 73.8, 29.2, 49.9, 39.7, 73.5, 87.6, 51.1])
print("Array...\n", arr)

Get the type pf array −

print("\nArray type...\n", arr.dtype)

Get the shape of the Array −

print("\nOur Array Shape...\n", arr.shape)

Get the number of elements of the Array −

print("\nNumber of Elements in the Array...\n", arr.size)