Penny post, or penny postage, was an invention of Sir Rowland Hill in Victorian Britain.
In the early 1800s, postage was very expensive, and in 1837 Hill published a pamphlet advocating pre-payment of postage. In other words, he had come up with the idea of postage stamps.
The first stamps cost a penny, hence the name.
In 1839 the British Post Office adopted Hill's plan, and for a brief period Hill was put in charge of implementing it.
Before Hill's innovation, postage was generally paid by the recipient, not the sender of a letter. The reforms that became known as the penny post essentially established the framework used by post offices to this day.