Configuring bitcoin full node for faster RPC calls
I have been working with a bitcoin full node and attempting to parse the blockchain data, whilst storing it in an external database. The script I am using iterates through each block using getblock[hash;2] which populates multiple tables. However in order to get the current blocks input values we have to perform an expensive getRawTransaction each previous txid. We have singled this out as the bottleneck in the code for most time taken, rendering the script very slow. We have also tried using RPC batching but this also takes considerable amount of time, potentially more with a memory overhead. I am wondering if there is a best configuration of the daemon to improve RPC call speed? i.e. using rpcthread/queue, increasing dbcache etc. Another option would be to implement blocks only mode, by reducing the bandwidth could this improve RPC commands? Bitcoin Core Daemon version v0.16.0.0-g4b4d7eb255 Server Details OS-Ubuntu 16.04 32 GB ram
Why is does it take so long to shut down an node used only as a JSON-RPC server?
I'm trying to sync a full node that will only be used as a JSON-RPC server (no mining). I tried to modify the config file and added a service unit, so that the node can run in a low-end VPS with minimum RAM and CPU capabilities. The problem is that the server takes too long to stop, and it's terminated by the system, so it always start rewinding blocks that have been already downloaded. Here is my configuration file:
server=1 daemon=1 #debug=mempool debug=rpc # If run on the test network instead of the real bitcoin network # testnet=1 # You must set rpcuser and rpcpassword to secure the JSON-RPC api # Please make rpcpassword to something secure, `5gKAgrJv8CQr2CGUhjVbBFLSj29HnE6YGXvfykHJzS3k` for example. # Listen for JSON-RPC connections on (default: 8332 or testnet: 18332) rpcuser=myuser rpcpassword=pypassword rpcport=8332 # Enable blocks pruning #prune=550 # Limit dbcache=50 maxconnections=4 rpcthreads=2
And the service unit:
# It is not recommended to modify this file in-place, because it will # be overwritten during package upgrades. If you want to add further # options or overwrite existing ones then use # $ systemctl edit bitcoind.service # See "man systemd.service" for details. # Note that almost all daemon options could be specified in # /etc/bitcoin/bitcoin.conf [Unit] Description=Bitcoin daemon After=network.target [Service] ExecStart=/usbin/bitcoind -daemon=0 -datadir=/home/jsonrpc/bitcoin -conf=/home/jsonrpc/bitcoin/settings.conf ExecStop=/usbin/bitcoin-cli -datadir=/home/jsonrpc/bitcoin -conf=/home/jsonrpc/bitcoin/settings.conf stop # Creates /run/bitcoind owned by bitcoin #RuntimeDirectory=/home/jsonrpc/bitcoin WorkingDirectory=/home/jsonrpc/bitcoin User=jsonrpc Group=jsonrpc TimeoutStopSec=15m #CPUQuota=4% #MemoryLimit=128M #IOReadIOPSMax=10 #IOWriteIOPSMax=10 Type=simple #Restart=on-failure # Hardening measures #################### # Provide a private /tmp and /vatmp. PrivateTmp=true # Mount /usr, /boot/ and /etc read-only for the process. ProtectSystem=full # Disallow the process and all of its children to gain # new privileges through execve(). NoNewPrivileges=true # Use a new /dev namespace only populated with API pseudo devices # such as /dev/null, /dev/zero and /dev/random. PrivateDevices=true # Deny the creation of writable and executable memory mappings. # Commented out as it's not supported on Debian 8 or Ubuntu 16.04 LTS #MemoryDenyWriteExecute=true [Install] WantedBy=multi-user.target
Just for the fun of it..... bitcoin-qt.exe -server=1 -rpcuser=michael -rpcpassword=pass -rpcthreads=4 -rpcallowip=192.168.1.8 -rpcport=8332 -rpcbind=192.168.1.8 bfgminer -o http://192.168.1.8:8332 -u michael -p pass --no-stratum -S opencl:auto --verbose --coinbase-addr 18SHLhskjwg7Sq7c45De8XS6mjke4f3nTb --coinbase-sig "Dan Conrad"
Bitcoind JSON-RPC is very slow for dumping GetRawTransaction even with a high perf computer.
bitcoin.conf is as folow and my query code is parallel for 100 theads. But it still slow and CPU usage is about 20% on a 8700K with 32GB and M2 Samsung 960 SSD! server=1 txindex=1 dbcache=2048 rpcworkqueue=128 rpcthreads=128 rpctimeout=220 Did I miss something ?
After mining successfully for several weeks, I shut down my miner last night after a ~4 day non-stop session for some gaming. Upon trying to fire it back up, I am getting errors with my P2Pool Node that seem to be fairly well documented elsewhere, but that I can nonetheless fail to resolve. Excerpts from the log files are below. It will do this continually, while SGMiner reports "Waiting on work" or "Stratum requested work restart" notifications. Connecting SGMiner to a public pool works fine.
2014-03-23 00:54:16.415000 > >>> Warning: LOST CONTACT WITH BITCOIND for 1.7 minutes! Check that it isn't frozen or dead!
2014-03-23 00:54:16.416000 > ######################################## 2014-03-23 00:54:16.416000 P2Pool: 5784 shares in chain (3108 verified/5790 total) Peers: 7 (0 incoming) 2014-03-23 00:54:16.416000 Local: 0H/s in last 0.0 seconds Local dead on arrival: ??? Expected time to share: ??? 2014-03-23 00:54:16.416000 Shares: 0 (0 orphan, 0 dead) Stale rate: ??? Efficiency: ??? Current payout: 0.0000 DOGE 2014-03-23 00:54:16.417000 Pool: 1365MH/s Stale rate: 22.8% Expected time to block: 44.9 minutes 2014-03-23 00:54:16.603000 Lost peer 184.108.40.206:8555 - Connection to the other side was lost in a non-clean fashion.
2014-03-23 01:31:47.326000 > Unhandled Error
2014-03-23 01:31:47.326000 > Traceback (most recent call last): 2014-03-23 01:31:47.327000 > File "C:\Mining\p2pool-rav-master\p2pool\util\variable.py", line 74, in set 2014-03-23 01:31:47.327000 > self.changed.happened(value) 2014-03-23 01:31:47.327000 > File "C:\Mining\p2pool-rav-master\p2pool\util\variable.py", line 42, in happened 2014-03-23 01:31:47.328000 > func(*event) 2014-03-23 01:31:47.328000 > File "C:\Mining\p2pool-rav-master\p2pool\work.py", line 123, in 2014-03-23 01:31:47.328000 > self.node.best_share_var.changed.watch(lambda _: self.new_work_event.happened()) 2014-03-23 01:31:47.328000 > File "C:\Mining\p2pool-rav-master\p2pool\util\variable.py", line 42, in happened 2014-03-23 01:31:47.328000 > func(*event) 2014-03-23 01:31:47.329000 > --- --- 2014-03-23 01:31:47.329000 > File "C:\Mining\p2pool-rav-master\p2pool\bitcoin\stratum.py", line 38, in _send_work 2014-03-23 01:31:47.329000 > x, got_response = self.wb.get_work(*self.wb.preprocess_request('' if self.username is None else self.username)) 2014-03-23 01:31:47.329000 > File "C:\Mining\p2pool-rav-master\p2pool\work.py", line 179, in preprocess_request 2014-03-23 01:31:47.329000 > raise jsonrpc.Error_for_code(-12345)(u'lost contact with bitcoind') 2014-03-23 01:31:47.330000 > p2pool.util.jsonrpc.NarrowError: -12345 lost contact with bitcoind ...And then that second bit repeats 3-4 times... Also, I can't seem to find it, but part of the repeating error as I saw somewhere in the log last night was that it was trying to connect basically to itself via my internet IP. Nothing about my system or network configuration has changed. What I have tried After the error, I re-downloaded the latest P2Pool-rav-master files and replaced them in my mining folder, leaving the same .conf in my roaming wallet folder. I added (Via a recommendation from this thread, after a suggestion from another that it was due to a memory leak of some kind) the arg "rpcthreads=100" and then changed it to "rpcthreads=250" (more's better, right?) to my wallet .conf file. Neither of these had any effect. Obviously. TL/DR- Miner fine, P2Pool node has started erring after weeks of use with no system or network change. How can I fix or reinstall the node? D5,000 to the person that resolves this issue. Other tips along the way.
本文转自：https://laravel-china.org/index.php/articles/8919/bitcoin-requests-node-data-through-rpc 文章来自本人 [email protected]: ~ $ bitcoind -version head -n 1 Bitcoin Core Daemon version v0.18.0 [email protected]: ~ $ lsb_release -ds Ubuntu 18.04.3 LTS Hardware: ODROID-HC1 hebasto added the Bug label Sep 28, 2019 If you want, you can change this number using the par setting in bitcoin.conf. Most of the threads you see are however not computationally intensive, and are in fact idling most of the time. They're just there to deal with occasional things that need to happen in the background and not interfere with other operations. That includes threads for sending and receiving network messages, for ... Bitcoin Core should also work on most other Unix-like systems but is not frequently tested on them. From 0.17.0 onwards macOS <10.10 is no longer supported. 0.17.0 is built using Qt 5.9.x, which doesn’t support versions of macOS older than 10.10. Known issues. Upgrading from 0.13.0 or older currently results in memory blow-up during the roll-back of blocks to the SegWit activation point. In ... It should be noted that although Bitcoin Core is a multi-threaded program, "the reference Satoshi client is largely single-threaded." Comment by Mike Hearn in BIP 31 (2012) What is meant is that the vast majority of the program's activity takes place in the messaging thread (ThreadMessageHandler - see below.) Almost all of the threads are part of a single, master thread group that is created ...
Support This Channel: https://www.nuancebro.com/join https://www.patreon.com/nuancebro https://www.subscribestar.com/nuancebro https://www.paypal.com/cgi-bin... youtube.com SUBSCRIBE For More Daily Video's - https://www.youtube.com/c/TheSpeedyDiver *EXCLUSIVE MERCH* - https://teespring.com/the-speedy-beetle-exclusive Gaming Chan... I take a look at another type of solvent trap. If you would like to support my channel use one of the links below https://www.patreon.com/user?u=19774761 htt... This video is unavailable. Watch Queue Queue. Watch Queue Queue