As already mentioned there is no certain way to detect encoding. But there is a large amount of heuristics that allow to do a smart guess about file encoding.
If there is no way for you to get to know the encoding for sure you may have a look at Apache Tika project and EncodingDetector there.
2
solved How to identify the encoding charset of a file in Java?