What is public private key exchange encryption and how they work in SSH

Public-Key encryption can be confusing sometimes. Here is a quick summary to explain what it means and how it works in SSH.