Why can't the Bitcoin development be more like Linux?
Linux has a huge development team. So many people take part in updating it. If you run Linux on your computer, you will know there are always updates. All these updates are created by the open source community. So why can't Bitcoin development be more like Linux?
What are we missing? Or maybe the development team will grow as big as the team of Linux?
What do you think?
Linux development doesn't deal with billions of dollars of a store of value system
Oh man...you have no idea what you are talking about don't you? Linux is fundation for huge part of internet as far as servers go. If it would have some critical issue, it would be disaster for servers, and anyone who uses linux based systems (hostings, banks etc.). I'd say linux is spine of Internet as far as software goes. Value that rests on linux fundation (only if you add banks that use it) is tens, hundreds or even thousands times larger than current worth of BTC. To that, add value of every other internet server (not server itself, but value of resources that are on that server) that is based on linux and you have value that you cant even pronounce.
I think you misunderstand. All linux "nodes" don't need to run the same linux kernel. Nothing bad happens if they run different versions or even completely incompatible linux kernels. All bitcoin nodes, must at all times, run "bitcoin kernels" (the core protocol) with identical outputs to any input. If some nodes decide chain A is the longest valid chain, and some nodes decide chain B is the longest valid chain well that would be a billion dollar problem. Any deviation no matter how small would cause a catastrophic fork in the network.
What actually makes large scale open source development in bitcoin land difficult is the "bitcoin-core" isn't just the "kernel", it isn't even just a node, it isn't even just a node and wallet. It is the kernel, node, wallet, and and RPC server rolled into this single tightly coupled (never a good idea in software development) unified blob of dense code. Now I am going so say something very heretical to the followers of the Church of Latter Day Satoshis, Satoshi wasn't a very good software developer (or cryptographer). <Pause to collect your thoughts>. He was a brilliant "idea guy". Bitcoin as a concept was some amazing out of the box thinking and truly innovative. The nuts and bolts of the first client however were a complete mess. A lot of his early decisions have really hampered current day developers. The developers can't easily make breaking changes and they need to ensure all future versions of the software are perfectly compatible with older versions of the software.
The Linux equivalent would be if Linux didn't have a kernel and a series of packages and instead it was simply a multi GB single blob of code. There was one repository and you had to use it all or use none of it. If a single update on some package you weren't even using was updated well you had to download and install the new unified Linux software or remain out of date. If the change was a breaking change you were required to upgrade or your server would be forked off the real internet. I think you can how that might keep the Linux development team small and centralized. That is how Bitcoin exists today. Don't blame the current devs, just take a look at Satoshi's first client and you can see where that foundation was built. The reality is the core protocol is never going to have more than a couple dozen developers. The concepts are very difficult. Most people (myself included) simply lack the competencies to do that kind of work. The good news is you probably wouldn't need more than a couple dozen core developers if "core" actually meant the "core protocol" not "the core protocol, and a particular implementation a full node, an a particular implementation of a desktop wallet which uses that full node, and a particular implementation of an RPC server which allows direct integration at the node level".
What the core team could use is probably another couple hundred dedicated testers though. If you are complaining about Bitcoin development and never run testnet for release candidate versions of clients well you have no room to complain.
If the core protocol (the kernel if you will) was separated from the other functionality and developed as a smaller independent project it would be more manageable for a small dedicated core team. This would be much like most "linux developers" aren't working on the linux kernel they are working on packages which rely on the linux kernel. Today if you want to build a wallet you have three choices.
The first would be to fork bitcoin-core and then keep it up to date until the end of time integrating all the changes to the mainline bitcoin-core; if you ever stop updating it then eventually it will be forked off the main network. Also make sure that you all your future changes don't break compatibility with the mainline client.
The second option would be to build a wallet front end which integrates with "bitcoin-core" node via RPC. Sadly some core functionality is missing so you this is less than optimal. You will probably end up reimplementing a lot of low level functionality (like maintaining the UTXO) simply because what you need isn't available via the RPC calls. This is arguably the safest option as the most it is the low level communications between nodes which need to be perfectly compatible and you would still be using bitcoin-core for that.
The third option would be to build the core protocol, node, and wallet functionality from scratch. Sounds easy but now make sure it is bit for bit compatible with bitcoin-core including handling of a million possible edge case scenarios (some of which haven't even occurred yet and the core team may not even be aware of). Also make sure you re-implement all the bugs, weird design choices, and issues with third party packages (openssl encoding bugs, etc). Have fun as not all of these are even well documented. You should be a solid proponent of test driven development.
Obviously none of those options are optimal. However if you could just compile the bitcoin-kernel and consume that in your wallet or other bitcoin related service that would be better right? It would be similar to how redhat consumes the linux kernel in their operating system. Well that option requires breaking the "bitcoin-core" into components so the core protocol is an independent project. It is a huge undertaking to decouple the kernel, node, wallet, and RPC server that Satoshi smashed together into a single unified blob of code, into separate projects which are still perfectly compatible with existing nodes. That process has been painstakingly going on for years now and will probably take many years more.
