Split off CBlockTreeDB and CCoinsViewDB into txdb-*.{cpp,h} files,
implemented by either LevelDB or BDB.
Based on code from earlier commits by Mike Hearn in his leveldb
branch.
During the initial block download (or -loadblock), delay connection
of new blocks a bit, and perform them in a single action. This reduces
the load on the database engine, as subsequent blocks often update an
earlier block's transaction already.
This switches bitcoin's transaction/block verification logic to use a
"coin database", which contains all unredeemed transaction output scripts,
amounts and heights.
The name ultraprune comes from the fact that instead of a full transaction
index, we only (need to) keep an index with unspent outputs. For now, the
blocks themselves are kept as usual, although they are only necessary for
serving, rescanning and reorganizing.
The basic datastructures are CCoins (representing the coins of a single
transaction), and CCoinsView (representing a state of the coins database).
There are several implementations for CCoinsView. A dummy, one backed by
the coins database (coins.dat), one backed by the memory pool, and one
that adds a cache on top of it. FetchInputs, ConnectInputs, ConnectBlock,
DisconnectBlock, ... now operate on a generic CCoinsView.
The block switching logic now builds a single cached CCoinsView with
changes to be committed to the database before any changes are made.
This means no uncommitted changes are ever read from the database, and
should ease the transition to another database layer which does not
support transactions (but does support atomic writes), like LevelDB.
For the getrawtransaction() RPC call, access to a txid-to-disk index
would be preferable. As this index is not necessary or even useful
for any other part of the implementation, it is not provided. Instead,
getrawtransaction() uses the coin database to find the block height,
and then scans that block to find the requested transaction. This is
slow, but should suffice for debug purposes.
Change the block storage layer again, this time with multiple files
per block, but tracked by txindex.dat database entries. The file
format is exactly the same as the earlier blk00001.dat, but with
smaller files (128 MiB for now).
The database entries track how many bytes each block file already
uses, how many blocks are in it, which range of heights is present
and which range of dates.
Corrupt wallets used to cause a DB_RUNRECOVERY uncaught exception and a
crash. This commit does three things:
1) Runs a BDB verify early in the startup process, and if there is a
low-level problem with the database:
+ Moves the bad wallet.dat to wallet.timestamp.bak
+ Runs a 'salvage' operation to get key/value pairs, and
writes them to a new wallet.dat
+ Continues with startup.
2) Much more tolerant of serialization errors. All errors in deserialization
are reported by tolerated EXCEPT for errors related to reading keypairs
or master key records-- those are reported and then shut down, so the user
can get help (or recover from a backup).
3) Adds a new -salvagewallet option, which:
+ Moves the wallet.dat to wallet.timestamp.bak
+ extracts ONLY keypairs and master keys into a new wallet.dat
+ soft-sets -rescan, to recreate transaction history
This was tested by randomly corrupting testnet wallets using a little
python script I wrote (https://gist.github.com/3812689)
Any problems seen during deserialization will throw an uncaught
exception, crashing the entire bitcoin process. Properly return an
error instead, so that we may at least log the error and gracefully
shutdown other portions of the app.
Cleans up and organizes several scattered functions and variables related to
the BDB env. Class CDBInit() existed to provide a
guaranteed-via-C++-destructor cleanup of the db environment.
A formal CDBEnv class provides all of this inside a single wrapper.
* This is safer than DB_TXN_NOSYNC, and does not appear to impact
performance.
* Applying this to the dbenv is necessary to avoid many fdatasync(2)
calls on db 5.x
* We carefully and thoroughly flush databases upon shutdown and
other important events already.
Add an option -detachdb (and entry in OptionDialog), without which no
lsn_reset is called on addr.dat and blkindex.dat. That means these
files cannot be moved to a new environment, but shutdown can be
significantly faster. The wallet file is always lsn_reset'ed.
-detachdb corresponds to the old behaviour, though it is off by
default now to speed up shutdowns.
This commit removes the dependency of serialize.h on PROTOCOL_VERSION,
and makes this parameter required instead of implicit. This is much saner,
as it makes the places where changing a version number can have an
influence obvious.
Design goals:
* Only keep a limited number of addresses around, so that addr.dat does not grow without bound.
* Keep the address tables in-memory, and occasionally write the table to addr.dat.
* Make sure no (localized) attacker can fill the entire table with his nodes/addresses.
See comments in addrman.h for more detailed information.
so it takes a flag for how to interpret OP_EVAL.
Also increased IsStandard size of scriptSigs to 500 bytes, so
a 3-of-3 multisig transaction IsStandard.
OP_EVAL is a new opcode that evaluates an item on the stack as a script.
It enables a new type of bitcoin address that needs an arbitrarily
complex script to redeem.
In the assert()s take advantage of the fact that string constants
("string") are effectively of type 'const char []', which when used in
an expression yield a non-NULL pointer.
An assertion that should always fail can thus be formulated as:
assert(!"fail);
An assertion where a text message should be added to the expression can
be written as such:
assert("message" && expression);
Signed-off-by: Giel van Schijndel <me@mortis.eu>
This commit adds support for ckeys, or enCrypted private keys, to the wallet.
All keys are stored in memory in their encrypted form and thus the passphrase
is required from the user to spend coins, or to create new addresses.
Keys are encrypted with AES-256-CBC using OpenSSL's EVP library. The key is
calculated via EVP_BytesToKey using SHA512 with (by default) 25000 rounds and
a random salt.
By default, the user's wallet remains unencrypted until they call the RPC
command encryptwallet <passphrase> or, from the GUI menu, Options->
Encrypt Wallet.
When the user is attempting to call RPC functions which require the password
to unlock the wallet, an error will be returned unless they call
walletpassphrase <passphrase> <time to keep key in memory> first.
A keypoolrefill command has been added which tops up the users keypool
(requiring the passphrase via walletpassphrase first).
keypoolsize has been added to the output of getinfo to show the user the
number of keys left before they need to specify their passphrase (and call
keypoolrefill).
Note that walletpassphrase will automatically fill keypool in a separate
thread which it spawns when the passphrase is set. This could cause some
delays in other threads waiting for locks on the wallet passphrase, including
one which could cause the passphrase to be stored longer than expected,
however it will not allow the passphrase to be used longer than expected as
ThreadCleanWalletPassphrase will attempt to get a lock on the key as soon
as the specified lock time has arrived.
When the keypool runs out (and wallet is locked) GetOrReuseKeyFromPool
returns vchDefaultKey, meaning miners may start to generate many blocks to
vchDefaultKey instead of a new key each time.
A walletpassphrasechange <oldpassphrase> <newpassphrase> has been added to
allow the user to change their password via RPC.
Whenever keying material (unencrypted private keys, the user's passphrase,
the wallet's AES key) is stored unencrypted in memory, any reasonable attempt
is made to mlock/VirtualLock that memory before storing the keying material.
This is not true in several (commented) cases where mlock/VirtualLocking the
memory is not possible.
Although encryption of private keys in memory can be very useful on desktop
systems (as some small amount of protection against stupid viruses), on an
RPC server, the password is entered fairly insecurely. Thus, the only main
advantage encryption has for RPC servers is for RPC servers that do not spend
coins, except in rare cases, eg. a webserver of a merchant which only receives
payment except for cases of manual intervention.
Thanks to jgarzik for the original patch and sipa, gmaxwell and many others
for all their input.
Conflicts:
src/wallet.cpp
* A new class CKeyStore manages private keys, and script.cpp depends on access to CKeyStore.
* A new class CWallet extends CKeyStore, and contains all former wallet-specific globals; CWallet depends on script.cpp, not the other way around.
* Wallet-specific functions in CTransaction/CTxIn/CTxOut (GetDebit, GetCredit, GetChange, IsMine, IsFromMe), are moved to CWallet, taking their former 'this' argument as an explicit parameter
* CWalletTx objects know which CWallet they belong to, for convenience, so they have their own direct (and caching) GetDebit/... functions.
* Some code was moved from CWalletDB to CWallet, such as handling of reserve keys.
* Main.cpp keeps a set of all 'registered' wallets, which should be informed about updates to the block chain, and does not have any notion about any 'main' wallet. Function in main.cpp that require a wallet (such as GenerateCoins), take an explicit CWallet* argument.
* The actual CWallet instance used by the application is defined in init.cpp as "CWallet* pwalletMain". rpc.cpp and ui.cpp use this variable.
* Functions in main.cpp and db.cpp that are not used by other modules are marked static.
* The code for handling the 'submitorder' message is removed, as it not really compatible with the idea that a node is independent from the wallet(s) connected to it, and obsolete anyway.
This introduces two new source files, keystore.cpp and wallet.cpp with
corresponding headers. Code is moved from main and db, in a preparation
for a follow-up commit which introduces the classes CWallet and CKeyStore.
Pieter Wuille