What is a public and a private key?
Before you get a certificate and start encrypting or signing documents, you need to request it. In this request you will need to specify for what you need a certificate (email, web server security, digital signature and etc.). At the moment you submit the request, your operating system will generate two cryptographic keys, private key and public key. They are called "keys" because they are used as input data for any cryptographic operation and they are also unique and paired.
Private key
Private key is crucial protected element, and it is always should be protected with all possible means. Private key is never revealed. Using private key you will decrypt emails send to you, create your signature. It is the main element for your operations. There are lots of different ways to protect private keys using hardware USB keys, smart cards, passwords and etc.
Generation of private key is a task of a machine from which the request is done. It means that the private key never leaves user computer, thus never is exposed to external network.
Public key
It can be distributed to everybody. Using this key your friends/colleagues/partners will be able to encrypt data that is for you (you can decrypt data using only private key), check your signature (only checking your signature). It means that there is always a cloud of public keys in your infrastructure, and if you need to send encrypted data to anybody, you just his/her public key and encrypt the data, and you can be sure that only this person will be able to read it.