String.split() would be the ideal choice – it takes a regular expressions — which can be used to define everything from the very simplest of patterns to the most complex ones .
As per Java API Doc –
StringTokenizer is a legacy class that is retained for compatibility reasons although its use is discouraged in new code. It is recommended that anyone seeking this functionality use the split method of String or the java.util.regex package instead.
solved Search Tokens in java [closed]