If you don’t already have a custom 404 page, simply generate the template file in your WordPress theme.
https://codex.wordpress.org/Creating_an_Error_404_Page
Regardless of whether or not you are going to prompt users to log in to view your private posts or not, creating a 404 page that’s styled to match your site’s brand is strongly recommended.
As for prompting users to log in to view your private posts, it is possible, but will require (either) some substantial theme customizations, or a 3rd party plugin.
Based on the requirements you described, here’s a plugin that should do the trick:
solved WordPress Prevent 404 page for private posts