When debugging another issue, I found a hang-during-startup race condition due to
LoadWallet calling SetMinVersion (via LoadCryptedKey).
Writing to the file that you're in the process of reading is a bad idea.
Use CBlock's vMerkleTree to cache transaction hashes, and pass them
along as argument in more function calls. During initial block download,
this results in every transaction's hash to be only computed once.
During the initial block download (or -loadblock), delay connection
of new blocks a bit, and perform them in a single action. This reduces
the load on the database engine, as subsequent blocks often update an
earlier block's transaction already.
This switches bitcoin's transaction/block verification logic to use a
"coin database", which contains all unredeemed transaction output scripts,
amounts and heights.
The name ultraprune comes from the fact that instead of a full transaction
index, we only (need to) keep an index with unspent outputs. For now, the
blocks themselves are kept as usual, although they are only necessary for
serving, rescanning and reorganizing.
The basic datastructures are CCoins (representing the coins of a single
transaction), and CCoinsView (representing a state of the coins database).
There are several implementations for CCoinsView. A dummy, one backed by
the coins database (coins.dat), one backed by the memory pool, and one
that adds a cache on top of it. FetchInputs, ConnectInputs, ConnectBlock,
DisconnectBlock, ... now operate on a generic CCoinsView.
The block switching logic now builds a single cached CCoinsView with
changes to be committed to the database before any changes are made.
This means no uncommitted changes are ever read from the database, and
should ease the transition to another database layer which does not
support transactions (but does support atomic writes), like LevelDB.
For the getrawtransaction() RPC call, access to a txid-to-disk index
would be preferable. As this index is not necessary or even useful
for any other part of the implementation, it is not provided. Instead,
getrawtransaction() uses the coin database to find the block height,
and then scans that block to find the requested transaction. This is
slow, but should suffice for debug purposes.
Corrupt wallets used to cause a DB_RUNRECOVERY uncaught exception and a
crash. This commit does three things:
1) Runs a BDB verify early in the startup process, and if there is a
low-level problem with the database:
+ Moves the bad wallet.dat to wallet.timestamp.bak
+ Runs a 'salvage' operation to get key/value pairs, and
writes them to a new wallet.dat
+ Continues with startup.
2) Much more tolerant of serialization errors. All errors in deserialization
are reported by tolerated EXCEPT for errors related to reading keypairs
or master key records-- those are reported and then shut down, so the user
can get help (or recover from a backup).
3) Adds a new -salvagewallet option, which:
+ Moves the wallet.dat to wallet.timestamp.bak
+ extracts ONLY keypairs and master keys into a new wallet.dat
+ soft-sets -rescan, to recreate transaction history
This was tested by randomly corrupting testnet wallets using a little
python script I wrote (https://gist.github.com/3812689)
OrderedTxItems returns a multimap of pointers, but needs a place to store the actual CAccountingEntries it points to.
It had been using a stack item, which was clobbered as soon as it returned, resulting in undefined behaviour.
This fixes at least bug #1768.
- Show address receiving the generation, and include it in the correct "account"
- Multiple entries in listtransactions output if the coinbase has multiple outputs to us
This applies on top of the coincontrol listaddressgroupings patch
and makes finding eligible outputs from the groups returned
by listaddressgroupings possible.
Logic:
- If sending a transaction, assign its timestamp to the current time.
- If receiving a transaction outside a block, assign its timestamp to the current time.
- If receiving a block with a future timestamp, assign all its (not already known) transactions' timestamps to the current time.
- If receiving a block with a past timestamp, before the most recent known transaction (that we care about), assign all its (not already known) transactions' timestamps to the same timestamp as that most-recent-known transaction.
- If receiving a block with a past timestamp, but after the most recent known transaction, assign all its (not already known) transactions' timestamps to the block time.
For backward compatibility, new accounting data is stored after a \0 in the comment string.
This way, old versions and third-party software should load and store them, but all actual use (listtransactions, for example) ignores it.
Implement listunspent / getrawtransaction / createrawtransaction /
signrawtransaction, to support creation and
signing-on-multiple-device multisignature transactions.
AvailableCoins() makes a vector of available outputs which is then passed to SelectCoinsMinConf(). This allows unit tests to test the coin selection algorithm without having the whole blockchain available.
This adds a field labelled 'Immature' in the overview section under the 'unconfirmed' field, which shows mined
income that has not yet matured (which is currently not displayed anywhere, even though the transactions
exist in the transaction list). To do that I added a 'GetImmatureBalance' method to the wallet, and connected
that through to the GUI as per the 'GetBalance' and 'GetUnconfirmedBalance' methods. I did a small 'no-op'
change to make the code in adjacent functions a little more readable (imo); it was a change I had made in my
repo earlier...but I thought it wouldn't hurt so left it in. Immature balance comes from mined income that is
at least two blocks deep in the chain (same logic as displayed transactions).
My reasoning is:
- as a miner, it's a critical stat I want to see
- as a miner, and taking into account the label 'immature', the uncertainty is pretty clearly implied
- those numbers are already displayed in the transaction list
- this makes the overview numbers add up to what's in the transaction list
- it's not displayed if the immature balance is 0, so won't bother non-miners
I also 'cleaned' the overview UI a little, moving code to the XML and removing HTML.
This introduces internal types:
* CKeyID: reference (hash160) of a key
* CScriptID: reference (hash160) of a script
* CTxDestination: a boost::variant of the former two
CBitcoinAddress is retrofitted to be a Base58 encoding of a
CTxDestination. This allows all internal code to only use the
internal types, and only have RPC and GUI depend on the base58 code.
Furthermore, the header dependencies are a lot saner now. base58.h is
at the top (right below rpc and gui) instead of at the bottom. For the
rest: wallet -> script -> keystore -> key. Only keystore still requires
a forward declaration of CScript. Solving that would require splitting
script into two layers.
- Signals now go directly from the core to WalletModel/ClientModel.
- WalletModel subscribes to signals on CWallet: Prepares for multi-wallet support, by no longer assuming an implicit global wallet.
- Gets rid of noui.cpp, the few lines that were left are merged into init.cpp
- Rename wxXXX message flags to MF_XXX, to make them UI indifferent.
- ThreadSafeMessageBox no longer returns the value `4` which was never used, converted to void.
Gets rid of `MainFrameRepaint` in favor of specific update functions that tell the UI exactly what changed.
This improves the efficiency of various handlers. Also fixes problems with mined transactions not showing up until restart.
The following notifications were added:
- `NotifyBlocksChanged`: Block chain changed
- `NotifyKeyStoreStatusChanged`: Wallet status (encrypted, locked) changed.
- `NotifyAddressBookChanged`: Address book entry changed.
- `NotifyTransactionChanged`: Wallet transaction added, removed or updated.
- `NotifyNumConnectionsChanged`: Number of connections changed.
- `NotifyAlertChanged`: New, updated or cancelled alert. As this finally makes it possible for the UI to know when a new alert arrived, it can be shown as OS notification.
These notifications could also be useful for RPC clients. However, currently, they are ignored in bitcoind (in noui.cpp).
Also brings back polling with timer for numBlocks in ClientModel. This value updates so frequently during initial download that the number of signals clogs the UI thread and causes heavy CPU usage. And after initial block download, the value changes so rarely that a delay of half a second until the UI updates is unnoticable.
This commit removes the dependency of serialize.h on PROTOCOL_VERSION,
and makes this parameter required instead of implicit. This is much saner,
as it makes the places where changing a version number can have an
influence obvious.
Do not automatically change the wallet format unless the user takes an
explicit action that implies an upgrade (encrypting, for now), or uses
-walletupgrade.
-walletupgrade optionally takes an integer argument: the client version
up to which upgrading is allowed. Without an argument, it is upgraded
to latest supported version. If an argument to -walletupgrade is
provided at the time the wallet is created, the new wallet will initially
not use features beyond that version.
Third, the current wallet version number is reported in getinfo.
so it takes a flag for how to interpret OP_EVAL.
Also increased IsStandard size of scriptSigs to 500 bytes, so
a 3-of-3 multisig transaction IsStandard.
Gavin Andresen