[Solved] Processing non-english text

Question

It’s common question.
Seems that you’re using cmd which doesn’t support unicode, so error occurs during translation of output to the encoding, which your cmd runs. And as unicode has a wider charset, than encoding used in cmd, it gives an error

IDLE is built ontop of tkinter’s Text widget, which perfectly supports Python strings in unicode.

And, finally, when you specify a file you’d like to open, the open function assumes that it’s in platform default (per locale.getpreferredencoding()). So if your file encoding differs, you should exactly mention it in keyword arg encoding to open func.

Accepted Answer

It’s common question.
Seems that you’re using cmd which doesn’t support unicode, so error occurs during translation of output to the encoding, which your cmd runs. And as unicode has a wider charset, than encoding used in cmd, it gives an error

IDLE is built ontop of tkinter’s Text widget, which perfectly supports Python strings in unicode.

And, finally, when you specify a file you’d like to open, the open function assumes that it’s in platform default (per locale.getpreferredencoding()). So if your file encoding differs, you should exactly mention it in keyword arg encoding to open func.