Lots of good stuff today.
The new API cache daemon system is running great on the backend. Right now it's only running against Cryptsy and BTC-e (mainly because I haven't updated the other exchanges yet to use the new system). Those two exchanges alone are cacheing 542 API calls in real-time effortlessly. The daemon thread monitor script is doing an excellent job of throttling the API calls(by spawning and removing parallel daemon scripts) to keep the age of the data at the desired setpoint.
Tonight I should be able to update all the exchanges to use the new methods and get a full load test in.
Next on the plate is to get backend plugins running, at which time I'll be able to start storing trade history to the database again.
I've also drastically reduced the number of database queries. Up until this point, I was storing the nonce for each account API in the database, querying to retrieve it each call, then querying again to increment the value. With so many API calls, this was adding up to thousands of queries per second, which are now eliminated due to using a better method based on microtime instead.
The exchange landscape is really looking good lately. I've seen quite a few new exchanges popping up in the last month, almost all of them with a full trade API. I fully expect to support 20+ exchanges by the time Rollerbot launches!
Other good news is that HHVM recently included mysqli support, so soon I will be running everything through the HHVM JIT compiler for a huge speed boost and better memory management.
Here is a screenshot of one of the daemons running in the background. If you look across the bottom, you'll see that the daemon spawned 22 parallel threads, with 5 symbols being queries per thread (this specific daemon is querying cryptsy for trade data). If the queries take too long, then more threads are spawned until the average execution time is where it should be.
