You can define the matrix like so (semicolons can be used for line breaks)
A = [1 2 3; 4 5 6; 7 8 0]
A =
1 2 3
4 5 6
7 8 0
Then you calculate the determinate by using DET
det(A)
ans =
27
8
solved MATLAB: how to calculate the ‘determinant’ of a matrix N*N in MATLAB [closed]