veni vidi vici

Could it maybe be my use of GPUmax occasionally?

Any progress pinpointing the source of those stales?
Mint is hitting 1% stale rate at 103 GHash/s - it doesn't look like GPUMAX is to blame...

EDIT:: I've been getting slow LPs lately - maybe that's some sort of a clue.

Very low stales - great fix?

Ummm... the pool has been down since 12:02 UTC.

Long post Warning:
(hit the scroll wheel down if not interesting in my ramblings on Bitminter, p2pool and a stronger/decentralized network)

BitMinter has been a great place to mine. Dr. Haribo is an awesome pool op and one I have no problems trusting with large sums of coins.  I have however finished migrating my farm to p2pool.  I joined BitMinter (way back when we had 30GH/s and I was 10 of them ) because I was concerned about the risk large pools represented.  I decided to put my hashing power where my mouth was and it has been an interesting experience.

It wasn't an easy decision as I have grown attached to BitMinter but I have moved to p2pool because I feel it is the best "weapon" against an increasingly centralized Bitcoin.   Small pools find it difficult to grow due to variance which keeps them small and thus they have higher variance (and the cycle repeats).  Even if a pool does grow there is always the risk it displaces a fallen giant, becomes a giant itself and the risk remains.  Thus I now believe that "medium sized pools" are unstable.  They either decline or grow and while the top few pools may change mining will be dominated by a dynamic of a few super pools and lots of tiny ones.

That being said there are issues which make p2pool non optimal for small miners so pools (hopefully smaller pools) have a role to play.  We talked about it briefly weeks ago but i believe a hybrid pool which uses p2pool as a back end is the missing link.  It is a symbiotic relationship.  The hybrid pool and p2pool gains significantly reduced variance from increased hashing power.  Both are now more competitive at attracting their respective "core audience".  If BitMinter sent hashing power to p2pool then p2pool (and BitMinter) would have an average block generation time of <4 hours.

It is the integration of small & medium sized pools which will move p2pool to the next level.   I believe large farms, hybrid pools and "subp2pools" can form a decentralized backbone.   This is necessary for p2pool to grow beyond that TH/s level and maybe someday provide 51% of the hashing power.

It looks like this guy has taken that idea and run with it:
https://bt.irlbtc.com/view/66202.0

Granted he only has 5GH/s (native) and a crude interface but I admire his spirit.  One thing i would point out is he enjoys about 70% less variance than BitMinter despite his tiny size.  Minters don't jump ship that wasn't the point.  I believe BitMinter could do it better using existing codebase and Dr. H experience as a solid pool ope.  I think BitMinter could grow to be one of the largest pools by offering a decentralized alternative for those who don't want to either use large centralized pools, get slammed by the variance of small pools or run their own p2pool instance.

A hybrid pool involves a few components
a) Running p2pool daemon (pool can provide it a static "vanity" address for payment)*
b) Issue getworks generated by p2pool to users
c) When miners returns shares:
   1) record diff 1 hashes (shares) for reward splits (same as now)
   2) forward share-chain diff hashes (currently ~600) to p2pool network
   3) broadcast full diff hashes (solved blocks) to Bitcoin network (same as now)

* p2pool may need to be either re-coded or integrated into existing pool backend to handle higher getwork volume.

When p2pool finds a block all miners payment addresses are already in the coinbase to payment is secure.  Bitminter would then take the sum it receives and split it among miners based on shares recorded in step b above.  This is very similar to how BitMinter pays out now.   Each day instead of having 1 or 2 blocks where it is the only payee in the coinbase it would have 6 to 12 blocks where it is one of many payees in the coinbase.

An example of the coinbase split can be seen here (this is the reward split in every hash worked by every miner):
http://p2pool.info/  (1p2pool is my vanity address, bitminter could advertise as say 1Minter )

I leave an open offer to assist with building/testing/experimenting with a BitMinterHybrid.  I also understand that it isn't something to be done lightly so a copy of the BitMinter code & database could be moved to a seperate "test instance" to test p2pool integration without interrupting the mainline pool.

I am willing to provide whatever is needed:
a) hashing power pointed at a server including remote access to the rig if needed.  No need for payment, no worries if it has 99% stales, instances crashes, etc.  All my rigs are 2GH/s (no you don't get the rack-mounted water cooled one ).  If we getting it working with one I am willing to donate all of them to provide further testing.
b) funding for servers, equipment, or anything else that is needed
c) coding work (while I can't take the lead I can provide assistance, bug checking, etc)
d) technical details on p2pool and optimizations (I have spent a large amount of time looking over the p2pool codebase)

If Dr. H doesn't want to be involved but would consider selling BitMinter code & database structure (not data) I would be willing to purchase it.  This would be an independent project, fully de-branded, not having any affiliation to BitMinter.  It would just provide a rapid template for building a 200GH/s capable hybrid pool.  I know he has worked extensively on the pool backend with little compensation this would be an opportunity to monetize it.   I will donate an extra 10% for Dr.H to offers as a block bonus to miners of BitMinter.

If Dr. H never wants to take me up on either offer no hard feeling.  If he gets the urge in a week, month, year the offer remains open.  I have enjoyed my time mining here.  I am sincere when I say BitMinter has the best pool op, a great layout, and an awesome community of miners.

Last Edit: 2/29/2012 09:54 - improved clarity of some parts, cut ramblings by 1.8%.

D&T, I agree with what you are saying about variance. I have long been thinking about how this could best be done, pools cooperating in a "pool of pools". I believe that can even things out so that people join pools for their features, not for their size (which is a self-reinforcing thing, causing the biggest pool so stay biggest no matter what).

About 51% attacks on bitcoin and securing against this, I believe what Luke-jr. is doing with expanding on the "getmemorypool" json-rpc interface can be a solution. Also for making pooled mining totally transparent. We've been discussing that in another thread. It would make it impossible for the pool op to hide blocks from miners (stealing the 50 BTC), and could also prevent a pool op from using your hash power in a 51% attack, or using your hash power to pool hop other pools without your knowledge. I intend to implement getmemorypool as an alternative to getwork on BitMinter and build some features around that to make mining more transparent and give miners more control.

I think "getmemorypool" with the expanded functionality will fix a lot of problems with pools. So what remains is the backbone or "pool of pools" which could enable a small pool with better features to beat a bigger pool. It could make variance a non-issue and finally let miners and pool ops focus on everything else.

So what about p2pool? I think p2pool has a couple of problems due to its reward system being based on a chain of proofs of work. I see two problems which is a reason I don't think p2pool is good for end users: A small p2pool, if I understand it correctly, is vulnerable to 51% attack on the reward chain. Someone with 300 GH/s today (a hopper proxy or pool?) could hook up to p2pool and cash out every bitcoin that is created. Everyone else would be mining for free, giving away all their income to the attacker. And a big p2pool, well, the difficulty on the reward chain would approach the difficulty of bitcoin, and make p2pool useless.

What about p2pool as a "pool of pools" backend? The 51% attack could be scary. The difficulty issue makes it useless for the smallest pools. And 10 second long poll means the server is doing long polls all the time. Things like x-roll-ntime become useless. There's no way around it, the server spends all its time pumping out new work to every single miner. You need much faster servers and use a lot of resources simply because of the way p2pool is designed. If you can't do it quickly enough, your stale rate on the reward chain will sky rocket, payouts will drop and the pool dies.

I have some ideas how stales on the reward chain could be reduced. But I see no way to prevent 51% attacks or p2pool creating very high server loads when used by a pool. Otherwise I would have switched already. I would love a tool to make variance less significant.

The only way I see how to fix this is by making a pool backbone that doesn't use a reward chain, but works a little differently. I have a lot of ideas for this. But I'm not sure how much point there is to working on that. I'm not sure if any pools would join a backbone. There isn't much interest for "getmemorypool" either.

I don't want to sell the BitMinter source code. It should be possible to hook up pushpool or poolserverj or something else to p2pool though. And there is already a pool running on top of p2pool.

When there is a split in the share chain, I would assume p2pool nodes will see multiple chains and go with the longest one, just like bitcoin does?

Imagine if I am an evil pool with 310 GH/s and you are the honest p2pool miners with 280 GH/s. I hop over to p2pool with my pool and start mining on top of your shares, entering the share chain. You then try to build shares on top of mine, but every time you do I create a fork. I only build shares on top of my own shares. My forks are always longer than yours, so you never get paid. I am now using 310 GH/s and getting paid for 590 GH/s. You get nothing.

How does p2pool prevent this? If it doesn't go with the fastest-growing fork or something similar, but just stays on the local fork whenever a fork occurs, then it would have forked into a whole bunch of solo miners a long time ago.

Also, I don't see a way to control the difficulty - it's based on the share chain.

Bitcoin block chain technology gives you a single advantage: decentralization (hard for bad guys to shut down). You have to pay for it with 51% attacks, high difficulty locking out small players, and in general very inefficient processing, especially with 10 second block intervals. In this instance I'm not sure if this is a good way forward.

If I'm wrong about 51% attacks, and the 10 second block intervals could be turned into 10 minutes, I'd jump on the bandwagon right now. (The remaining problem, difficulty/variance, is what pools are there to fix)

Quick backend restart. Fixed a memory leak and a couple other issues. The memory leak is what was causing the downtime today and the slow response which gave high stales earlier. Should be back to low stales, quick response and stable server. (knock on wood)


Sure, it becomes like 2 separate p2pools. But I imagine all nodes end on the same fork after a while, to prevent degenerating into everyone solo mining by themselves.

Btw, awesome miner project you have.

Yes, switching to solo mining if p2pool is 51% attacked is a good solution. You can detect it before losing much income, so it should be adequate.

Also, I'm starting to think maybe p2pool's problems with frequent block changes and high difficulty stem from copying bitcoin's block chain technique directly. P2pool has different needs. Maybe this can be fixed by doing things a little differently. Have to think about that.