Back in 1913 this country had pretty much the same coins and bills that we have today; there were a few more though. There was a 50 cent piece (which I recall still being in circulation as late as the 1970's) and there was a silver $1 piece. In 1913 there were $5 gold pieces, $10 gold pieces and $20 gold pieces -- surely not in common circulation but at least recently minted.
A lot has changed since 1913 when automobiles were still a novelty, indoor plumbing and electricity and the telephone a rarity and commercial airplane travel unimagined. But our coins are still pretty much the same as they were then. But what if our coins had kept up with inflation? We would have a coin that looks like a penny but it would be worth 25 cents. We would have a coin that looks like the nickel but it would be worth $2.40 and we would have one that looks like a dime but it would be worth twice that. We'd have a coin that looks like a quarter but it would be worth $6 and we'd have the now rarely seen 50 cent coin that would be valued at just over $12. And if we had one to replace the silver dollar (or perhaps the Susan B. Anthony dollar) that would be valued at $24. That scarce $2 bill would now be valued at $48.
It really was not so long ago that coins were actually something more than the nuisance they are today. I recall that at late as the 1970's I rarely got my wallet out. Sure, a restaurant meal would mean opening the wallet as would most clothing purchases but a cup of coffee and a donut would be pocket change as would have been a meal at McDonalds. Today, nearly everyone carries a heavy load of coins that cannot buy anything; coins are useful only for paying those fine gradations of money that really don't mean much to anyone. Even that $1 bill in your wallet will buy only what 4 cents would have bought in 1913; granted, you could have bought a couple candy bars for that much back then.
So what you might say? We are now all accustomed to carrying around a half-pound of fairly useless coins and we don't think much about it.
There are costs for minting these fairly useless coins but the coins do last a long time -- much longer than bills last. But there is the expense of printing, re-printing and re-printing bills that could have been coins instead. But this is a rich country, we can surely afford to waste a little money like this, just to preserve the great tradition of our coinage. What does it cost every years? Millions surely, billions maybe. What does it matter? As a country we're rich.
Another cost is when you travel abroad compared to when others travel here. Generally speaking you can exchange only paper money; that pocket full of change you just cannot trade in for your own domestic cash; you can keep it as a souvenir or you can save it for your next trip. That's hardly a problem for people visiting the U.S. The change they are force to take home is probably not worth much.
But, for example, an American visiting Japan may well find themselves at Narita Airport with a pocket full of change, maybe four or five 1000 Yen coins and as many 500 coins along with a mix of smaller coins all totaling perhaps 7500 Yen. That is no heavier in the pocket than the $2.50 in change that the tourist leaving America keeps but it is valued at something close to $75. Narita airport does thoughtfully provide an opportunity to donate that excess pocket change to charity before leaving the country, however.