Situation:
Reading a text file in python3 (csv or txt) there is a character that can be appreciated using "more" in terminal but in python3 the situation is more complicated.
Example:
$ more epa.csv
<U+FEFF>the text
Problem:
Python3 reads the file well, it doesn't throw an error, but that invisible "character" remains in the variables, the texts, etc. and can cause some inconvenience.
Solution:
The solution is to read the file and specify the encoding, something as simple as:
FILENAME="epa.csv"
with open(FILENAME, encoding='utf-8-sig') as file:
for line in file:
print(line)
Explanation (taken from: https://stackoverflow.com/questions/17912307/u-ufeff-in-python-string):
The Unicode character U+FEFF is the byte order mark, or BOM, and is used to tell the difference between big- and little-endian UTF-16 encoding.
Good luck,
