These two words aren't intended to be interchangeable as much as they are misused that way, so this one is just fun for me.
Ultimate has a circuitous etymology. It derives its meaning from the Latin "ultima" for "the last syllable of a word", a feminine derivation of "ultimus", the superlative comparative of "ulterior" which means "more distant or farther", and which is itself related to the prefix "ultra" for "located beyond" or "on the far side of" and the comparative derivation "ulter" for "beyond". I'll save "ulterior" for another day, and leave this one with the understanding that the use of "ult-" implies distance. Now, by applying it to a "last syllable", it goes from being a great distance beyond what can be observed or perhaps even comprehended (I'll foreshadow "ulterior") to simply last, where the distance is still present, but not required to be great. And from there, the physical distance requirement was dropped, leaving only the connotation that the thing had come to an end. It has to be the last by whatever distinction of last you choose to employ (e.g., maximum, highest, total, final)
Scientists have hypothesized on the ultimate boundary of the universe. A bit esoteric, but if you've ever studied astronomy, you may understand that the universe is expanding, and at some point, it may stop expanding. It should be noted that this is a mixed usage of time and space, so not quite a pure distance application. The ultimate loser of the marathon was the last person to cross the line, or arguably, the first person to quit running. Don't confuse the distance in the example of the footrace for a physical distance attached to ultimate. The "last" element of this example is time. Still trying for just distance. So much easier with the abstracts. Many believe the ultimate measure of power is money. Correct, but not necessarily true. Plaintiff's counsel's ultimate goal is to settle the case fast, even at the cost of getting more money for his client, therefore, the ultimate outcome of his incompetence was being fired. Note that a goal is not necessarily an end, so there is no redundancy, whereas saying "the ultimate end" would be. The ultimate point in the climb was the top of the mountain. Ok, it's distance, but you still have to get down the mountain, so shouldn't the ultimate point of any trip be to get back home? The term "ultimate destinations" is bandied about so freely as euphemisms for luxury destinations, but a true ultimate destination is really the ICU ward, or the hospice! Yeah, abstracts are much easier.
Meanwhile, penultimate is an odd little word that isn't used enough. So, now that we've been through "ultimate", the only difference is the Latin prefix "paene" meaning "almost", which, of course, gives us the word for "the next to the last" because what is "almost last" must be so close to last as to be the next to last.
Plaintiff's counsel's penultimate goal is to make his client happy, or perhaps not be reported for his ethical violations, while the ultimate goal is to make millions for himself. The penultimate point of my trip when I pass through U.S. Customs & Immigration successfully with all my international purchases. Ok, so I've gotten distance out of the way, but it doesn't sound quite right. Penultimate seems to have become solely attached to abstracts. Y is the penultimate letter of the alphabet. Pedantic and uninspired. Students whose last names start with Y were frequently the ultimate victim of Socratic method, but occasionally could be granted a reprieve to penultimate status if a Z last name enrolled. Italian is distinguished from French by the abundance of penultimate versus ultimate stresses. Try it. You'll see. Well, I think you get the picture. Basically, if it can end, it can be just before the end, and subject to myriad interpretations and usages.
So, I hope this is not the ultimate posting for this year, but as I have less than 24 hours and a small party to host, which will still require grocery shopping and cooking, it may well be. At best, if I can scrounge another easy comparison, one that doesn't go through the Middle Ages or require the magnifying glass with OED, this may be the penultimate post. But in any event, Happy New Year!!!