According to the documentation, it’s not returning a byte array but an Array.
Just type ByteArray_to_Hex(problem) and let Visual Studio generate the method. Then you’ll see what type it returns. Perhaps you can call ByteArray_to_Hex((byte[])problem) to explicitly cast it.
solved type of variable in Visual C#