Wouldn't wallet encryption, which is already implemented, have made this a nightmare of the past? (honest question, not trying to be a dick)

I don't think so as wallet encryption in an automated system would still have required the password to be placed in some script to function. Even if it required sysadmin intervention after reboot to init a password there would still be a running process with access privileges. While it may be harder to hijack that a user (hacker) who has gained root could likely manage it. I'm not even sure the bitcoind  rpc calls are limited to the process that unlocked the wallet since it may not even be able to tell which process made an rpc request. It may be as simple as polling the rpc calls until a valid  request unlocks it and then firing a malicious request.

I would think a more defensive approach would be only maintaining enough bitcoin in an active wallet to handle X minutes demand activity and transfer excess receipts to an offline address. Any demand in excess of average could fire an SMS message to a sysadmin who would use an offline wallet to topup the active wallet.

Security is not only about protecting access but also about limiting damage.

Edit: It's also conceivable to build traps into the system. eg. On a hardened system (with properly configured sudoers) root login should never be needed in daily activity so if root ever logs in the wallet could be immediately shredded such that a legitimate sysadmin must audit the system and re-init the wallet from an offline backup.

Yes, offline wallets + transactions would be more useful imho.

1 Bitcoin node (A) that collects incoming coins + outgoing requests.
1 Bitcoin node (B) completely offline.

Every x minutes, A puts on an USB stick: Current blockchain (can be just a delta from the last time...), outgoing transactions to be confirmed, transaction to send all incoming coins to a address controlled by B

B then gets just this data, someone looks over it, if it seems legit and if it is so, B signs the payout transactions on the USB stick. A imports these and broadcasts them to the network.

Refinements could be that X transactions below Y BTC don't require confirmation and are paid from the "hot" wallet or similar.

Multisig would just mean that you need to compromise more internet machines than one and that you are screwed if you require a signature from a device that has gone belly up.
multisig also looks easier to DoS to me...

That would just mean, you need a more persistent exploit that waits until a wallet gets decrypted and then sends all it's funds to some address. Less efficient, yes but it still might take a while until this comes out, especially if the backend shows no loss in balance or if you can make it look like a user initiated payout.

Why would I need to log on to deposit after the first time, if I then know at least 1 address from my own private wallet?

Sorry to say this, but, no matter what, it is simply not excusable to keep a large amount like that on off-the-shelf virtual servers.

Encryption would have been a saviour in this case because the exploit required a reboot of the servers in question. But multisig would allow to setup an even more secure system.

Gavin posted about this in his blog and explained one way of doing it (regarding the faucet): http://gavintech.blogspot.com/2012/03/bitcoin-faucet-hacked.html

Currently P2SH support is around 36%, all we need is Deepbit support and it's done. I hope Tycho finally gives in. This feature IS needed.

Mine at p2pool, you won't lose your btc because you mine them on your address

Tycho has expressed his opinion several times on the forums. Because he is the biggest pool he CANNOT make a decision, if he does, and the majority of miners vote against him, he runs the risk of forcing a fork of the blockchain and then being the biggest miner on the new blockchain, meaning he'd be mining useless coin (the blockchain would be subject to 50% attacks regularly and no one would want to use it).

Tycho seems to have made it known that he'll wait till BIP16 or BIP17 has a clear majority, and then follow suit accordingly with the majority opinion.

This isn't really true. Multisig will not stop bitcoins from being stolen. If it's an online wallet that supports automated withdrawl then hackers can just do whatever the application normally does to process withdrawls. Maybe the webapp makes an rpc call to some other server to process the second signature.. fine, you just do that. easy. If you can make automated/instant withdrawls then so can a hacker - there's no way around this fundamental issue.

True, you wouldn't want to just expose "withdraw X bitcoins to address Y" as an RPC call. You need to tie the withdrawals to specific accounts and validate the amount against the account balance so that no one RPC call can wipe out the entire wallet.

Ideally, you would have one or more cheap, low-security systems to handle the front-end traffic. These systems would not hold any private keys or sensitive customer databases. A separate, secure, low-traffic system would deal with authenticating users and withdrawing coins (or you could use two separate servers). Authentication should include some form of challenge-response protocol (e.g. a public-key certificate) or one-time password to avoid exposing the user's credentials to anyone breaking into the front-end server.

When someone logs in, their credentials would be forwarded to the authentication server, which would then issue a time-limited one-time authentication token. When they want to withdraw money from their account, the front-end server would send that token, the amount, and the destination address to the secure wallet server, which would validate the token and the amount and then sign a transaction to send the coins to the destination address. A compromise of the front-end systems would expose, at most, the balances of those accounts which were active at the time.

For even better security, you could require the login credentials to be re-entered for each withdrawal (limiting the exposure to those accounts making withdrawals), and/or only allow withdrawal to pre-registered addresses. This last option is more cumbersome, but it also almost completely eliminates the incentive to break in to the front-end systems, leaving only the more hardened secure servers as a worthwhile target.