This is an abandoned project.

See https://github.com/mempoolexplorer/mempool-explorer-back for a self-hosted version.

BitcoindAdapter

This is a spring boot project to mantain a REST-queriable memPool, and send new transactions and blocks via kafka message broker. It works as a bitcoind adapter which queries bitcoind for mempool transactions and new blocks continuously. The main difference with a simple call to bitcoind's getrawmempool is that transaction data includes input and output addresses, so you don't have to do several queries, bitcoindAdapter does it for you. This is quite useful for building a memPoolExplorer service. It also serves as memPool cache to avoid overloading bitcoind. In order to compare the transactions in mined blocks with the transactions that your mempool would have mined, the result of a getblockTemplate call is also sent via kafka messsage.

BitcoindAdapter uses mainly these bitcoind RPC calls:

• getrawmempool (verbose) to obtain the mempool, and then for each txInput in each transaction:
• getRawTransaction (decoded) is called.
• getBlockHash, getBlock and getBlockCount.
• getBlockTemplate

Mempool changes and new blocks are sent through a configurable kafka topic (memPool.tx.events). Mempool tx changes are guaranteed not to be sent when a new block is mined. That is, if a new block is detected before checking mempool changes, these changes are not sent, but instead the block found is sent. This ensures a message order so message consumers can store the new block transactions before they are removed from mempool in the next message, enabling consumers to compare mined block txs with mempool txs.

BitcoindAdapter is shutdown friendly. That is, if restarted, then signals kafka clients they have to refresh its mempool. This is done by sending kafka messages with numOrder = 0 containing a maximum of 10 tx each (to avoid kafka message size limit).

Requirements

• bitcoind 0.21
• java 11

Usage

This is meant to be used in a docker environment. Go to commands folder to see docker-compose examples

bitcoind configuration

You must have bitcoind running with txindex=1 option and basic authentication. Use a bitcoind.conf file like this in your ~/.bitcoin/bitcoin.conf folder.

# This config should be placed in following path:
# ~/.bitcoin/bitcoin.conf

# [rpc]
# Username for JSON-RPC connections
rpcuser=myuser
# Password for JSON-RPC connections
rpcpassword=mypassword
# Maintain a full transaction index, used by the getrawtransaction rpc call.
txindex=1
# rpc ips allowed (Ip of this service must be included)
rpcallowip=X.X.X.X
# ip of bitcoind
rpcbind=192.168.3.2
# [Wallet]
# Do not load the wallet and disable wallet RPC calls.(optional)
disablewallet=1
# [core]
# Set database cache size in megabytes; machines sync faster with a larger cache. Recommend setting as high as possible based upon machine's available RAM. (optional)
dbcache=4000

.properties configuration

.properties file are loaded by configurationServer service. These properties are almost auto-explicative:

• refreshBTIntervalMilliSec: "5000" Interval in which bitcoindAdapter queries for new BlockTemplate. We does not use "longpoolid" since it does not refresh enough quickly.
• refreshSmartFeesIntervalSec: "60" Interval in which bitcoindAdapter queries for Smart Fee data. By default 1 minute.
• refreshBCIIntervalMilliSec: "60" Interval in which bitcoindAdapter queries for blockChainInfo. By default 1 minute.

REST API

• /blockChain/lastBlocks Returns a list with all last blocks recevived (normally last 3)
• /blockChain/lastBlocksFrom/{epochSecond}/{nano} Returns last blocks from epochSecond/nano
• /blockChain/lastBlocksFrom/{height} Returns last blocks from {height}
• /blockTemplate/blockTemplate Returns last blockTemplate consulted
• /memPool Returns a list with all txIds in mempool
• /memPool/full Returns all mempool if bitcoindadapter.maxMemPoolSizeReturnedInTxNumber property allows it.
• /memPool/{txId} Returns decoded tx with txId given as parameter
• /memPool/changes Returns last N changes in memPool, N defined in bitcoindadapter.memPoolChangesSize property.
• /memPool/changesFrom/{changeCounter} Returns last changes from changeCounter, each change is identified by a changeCounter and a epochSecond/nano.
• /memPool/changesFrom/{epochSecond}/{nano} Returns last changes from epochSecond/nano, each change is identified by a changeCounter and a epochSecond/nano.
• /memPool/state Returns memPool state {STARTING|LOADINGFROMDB|LOADINGFROMBITCOINCLIENT|SAVINGTODB|STARTED}
• /actuator Access to spring boot actuator endpoint. Be aware that actuator port is defined in management.port property.
• /actuator/metrics Access to metrics like mempool.transaction.count or mempool.refresh.time to monitor app performance.
• /alarms/list List of errors or alarms of the service.

ACTUATOR API

You can enter actuator via port 8081 /actuator/ENDPOINT NAME (i.e. health, info... etc)