0x90 is indeed out of range for ASCII, which only covers 0x00 to 0x7f. The file may be in some Unicode encoding, or it may be in some 8-bit encoding, in the ISO-8859 family. Once you have found that out, open your file with the codecs module. Supposing your encoding is ISO-8859-1:
with codecs.open('class1.csv', encoding = 'iso-8859-1') as handle:
reader = csv.reader(handle)f = gzip.open('neural-networks-and-deep-learning/data/mnist.pkl.gz', 'rb') training_data, validation_data, test_data = pickle.load(f,encoding='bytes') f.close(),with gzip.open('neural-networks-and-deep-learning/data/mnist.pkl.gz', 'rb') as ff: u=pickle._Unpickler(ff) u.encoding='latin1',f = gzip.open('mnist.pkl.gz', 'rb') training_data, validation_data, test_data = pickle.load(f, encoding="latin1") f.close(),For the people, who still struggling with it, just change the code into: f = gzip.open('../data/mnist.pkl.gz', 'rb') u = cPickle.Unpickler(file=f, encoding='latin1') training_data, validation_data, test_data = u.load() f.close() return (training_data, validation_data, test_data) return (training_data, validation_data, test_data)
!git clone https: //github.com/mnielsen/neural-networks-and-deep-learning.git
import pickle
import gzip
import numpy as np
f = gzip.open("./neural-networks-and-deep-learning/data/mnist.pkl.gz")
training_data, validation_data, test_data = pickle.load(f, encoding = "latin1")
print(training_data)!git clone https: //github.com/MichalDanielDobrzanski/DeepLearningPython35.git
!ls
import pickle
import gzip
import numpy as np
f = gzip.open("./DeepLearningPython35/mnist.pkl.gz")
training_data, validation_data, test_data = pickle.load(f, encoding = "latin1")
print(training_data)Last updated: May 1, 2022
Copied!๐แธ๐ข๐ฏูค แธิะว hello world
Copied!#โ๏ธ UnicodeDecodeError: 'ascii'
codec can 't decode byte 0xf0 in position 0: ordinal not in range(128)
with open('example.txt', 'r', encoding = 'ascii') as f:
lines = f.readlines()
print(lines)Copied!#๐๏ธ set encoding to utf - 8
with open('example.txt', 'r', encoding = 'utf-8') as f:
lines = f.readlines()
print(lines) #๐๏ธ['๐แธ๐ข๐ฏูคแธิะว\n', 'hello world']Copied!my_text = '๐แธ๐ข๐ฏูคแธิะว'
my_binary_data = my_text.encode('utf-8')
#โ๏ธ UnicodeDecodeError: 'ascii'
codec can 't decode byte 0xf0 in position 0: ordinal not in range(128)
my_text_again = my_binary_data.decode('ascii')Copied!my_text = '๐แธ๐ข๐ฏูคแธิะว'
my_binary_data = my_text.encode('utf-8')
#๐๏ธ b '\xf0\x9d\x98\x88\xe1\xb8\x86\xf0\x9d\x96\xa2\xf0\x9d\x95\xaf\xd9\xa4\xe1\xb8\x9e\xd4\x8d\xd0\x9d\xc7\x8f'
print(my_binary_data)
#โ
specify correct encoding
my_text_again = my_binary_data.decode('utf-8')
print(my_text_again) #๐๏ธ '๐แธ๐ข๐ฏูคแธิะว'Copied!my_text = '๐แธ๐ข๐ฏูคแธิะว'
my_binary_data = my_text.encode('utf-8')
#๐๏ธ set errors to ignore
my_text_again = my_binary_data.decode('utf-8', errors = 'ignore')
print(my_text_again)See: UnicodeEncodeError: 'ascii' codec can't encode character., 1 Have you messed with the default python installation (for example, symlinking python3.2 to /usr/bin/python?) - apt should be using the system default python2.7 on 12.04. Can you add the output of ls -l $(which python) to your post? โย steeldriver Jul 1, 2014 at 19:25 , lrwxrwxrwx 1 root root 9 Apr 10 2013 /usr/bin/python -> python2.7 I did install the packages in order to get apt-add-repo (which messes with Python iiirc) โย James Heald Jul 2, 2014 at 10:32 ,Trying to install the latest PHP5 packages and so I add the repo ( sudo add-apt-repository ppa:ondrej/php5) I need and this is the outcome:
A better workaround was pointed out in the issue tracker that uses specific unicode locale when adding the repository
LC_ALL = C.UTF - 8 add - apt - repository - y ppa: ondrej / php5 - 5.6
Try installing a language pack which may correct your issues with encoding, e.g.
sudo apt - get install language - pack - en
Otherwise set the locale settings manually, e.g.
$ locale - a | grep "^en_.\+UTF-8"
en_GB.UTF - 8
en_US.UTF - 8
$
export LC_ALL = en_GB.UTF - 8
$
export LANG = en_GB.UTF - 8If you are in docker, this worked for me:
RUN LC_ALL = C.UTF - 8 add - apt - repository - y ppa: ondrej / php
An after:
RUN apt - get update RUN apt - get install - y php7 .2
For me adding php5-compat before php solved my problem on ubuntu 16.
sudo LC_ALL = C.UTF - 8 add - apt - repository - y ppa: ondrej / php5 - compat
and then
sudo LC_ALL = C.UTF - 8 add - apt - repository - y ppa: ondrej / php