When the markets were booming, billionaire colleges like Harvard University, the Massachusetts Institute of Technology and Stanford University tapped their swelling endowments and launched spending binges on faculty, buildings and scholarships.

Now, they’re seeing firsthand the one downside to relying on a huge nest-egg: The market crash has them confronting the sharpest budget cuts in memory.

A new survey released Tuesday reports college endowments fell 3 percent in the fiscal year ending June 30. In a follow-up, a smaller group estimated declines averaging 23 percent in the first five months of fiscal 2009, which began in July.

That decline is nearly twice as big as any full-year return since endowment performance was first tracked in 1974, said Brett Hammond, chief investment strategist at TIAA-CREF, which collected the figures with NACUBO, a college business officers group, and Commonfund Institute.

The survey of 791 colleges accounts for virtually all of the endowed savings of American public and private colleges — some $522 billion last June. But the losses since then would erase nearly $120 billion. Colleges typically spend around 5 percent of their endowments annually.

The challenge for colleges with eroded endowments is that many of the faculty they hired now have tenure, all those new buildings still need heating — and financial aid demand is rising.

In recent years, Dartmouth College in New Hampshire spent $1 billion on new facilities and more than doubled its financial aid budget. But with its endowment down $700 million, staff cuts are inevitable. The college needs to slice $60 million from next year's $700 million budget.

"I don't think anybody believes there aren't going to be big consequences," said Adam Keller, who oversees Dartmouth's finance and administration. "At the same time, people have begun to think hard what priorities exist for them and ways they can do their job better."