Say you have your regular SSH public-private key pair for most of what you do, and you need to have another SSH public-private key pair for specific servers (this doesn't happen often, but there are some edge cases where there is no easy alternative).
start by creating a new directory in your home folder, called .ssh-whatever, and put your public private key pair (or just your private key, that works too) in there.
Now, each time you want to access a server with the key in .ssh-whatever, type in "ssh-agent bash" followed by "ssh-add ~/.ssh-whatever/id_rsa". Now you will be able to access any server with the key in .ssh-whatever.