Why can calling a scalar user defined function within a query against a large table cause significant performance problems?

A scalar function called within a query is typically executed once for every single row being processed, and since older versions of SQL Server treated scalar functions as a kind of black box the optimizer could not see inside of, this row by row execution can become extremely slow on large tables compared to an equivalent inline calculation the optimizer could otherwise handle in a fully set based manner.

-- Calling a scalar function per row can be slow on large tables
SELECT OrderId, dbo.CalculateTax(Amount) AS Tax FROM Orders;

-- An inline calculation often performs much better
SELECT OrderId, Amount * 0.08 AS Tax FROM Orders;