The current getblocktxn implementation drops and ignores requests for old
blocks, which causes occasional sync_block timeouts during the
p2p-compactblocks.py test as reported in
https://github.com/bitcoin/bitcoin/issues/8842.
The p2p-compactblocks.py test setup creates many new blocks in a short
period of time, which can lead to getblocktxn requests for blocks below the
hardcoded depth limit of 10 blocks. This commit changes the getblocktxn
handler not to ignore these requests, so the peer nodes in the test setup
will reliably be able to sync.
The protocol change is documented in BIP-152 update "Allow block responses
to getblocktxn requests" at https://github.com/bitcoin/bips/pull/469.
The protocol change is not expected to affect nodes running outside the test
environment, because there shouldn't normally be lots of new blocks being
rapidly added that need to be synced.
Increase wallet-dump RPC timeout from 30 seconds to 1 minute. This avoids a
timeout error that seemed to happen regularly (around 50% of builds) on a
particular jenkins server during the first getnewaddress RPC call made by the
test.
The failing stack trace looked like:
Unexpected exception caught during testing: timeout('timed out',)
File ".../bitcoin/qa/rpc-tests/test_framework/test_framework.py", line 146, in main
self.run_test()
File ".../bitcoin/qa/rpc-tests/wallet-dump.py", line 73, in run_test
addr = self.nodes[0].getnewaddress()
File ".../bitcoin/qa/rpc-tests/test_framework/coverage.py", line 49, in __call__
return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
File ".../bitcoin/qa/rpc-tests/test_framework/authproxy.py", line 145, in __call__
response = self._request('POST', self.__url.path, postdata.encode('utf-8'))
File ".../bitcoin/qa/rpc-tests/test_framework/authproxy.py", line 121, in _request
return self._get_response()
File ".../bitcoin/qa/rpc-tests/test_framework/authproxy.py", line 160, in _get_response
http_response = self.__conn.getresponse()
File "/usr/lib/python3.4/http/client.py", line 1171, in getresponse
response.begin()
File "/usr/lib/python3.4/http/client.py", line 351, in begin
version, status, reason = self._read_status()
File "/usr/lib/python3.4/http/client.py", line 313, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "/usr/lib/python3.4/socket.py", line 374, in readinto
return self._sock.recv_into(b)
Three categories of modifications:
1)
1 instance of 'The Bitcoin Core developers \n',
1 instance of 'the Bitcoin Core developers\n',
3 instances of 'Bitcoin Core Developers\n', and
12 instances of 'The Bitcoin developers\n'
are made uniform with the 443 instances of 'The Bitcoin Core developers\n'
2)
3 instances of 'BitPay, Inc\.\n' are made uniform with the other 6
instances of 'BitPay Inc\.\n'
3)
4 instances where there was no '(c)' between the 'Copyright' and the year
where it deviates from the style of the local directory.
Clear test_node.last_block before requesting blocks in the
compactblocks_not_at_tip test so comparisons won't fail if a blocks were received
before the test started.
The bug doesn't currently cause any problems due to the order tests run, but
this will change in the next commit.
Bug caused the wait_for_block_announcement to be called on the wrong node,
leading to nondeterminism and occasional test failures. Bug was introduced in
merge commit:
d075479 Merge #8882: [qa] Fix race conditions in p2p-compactblocks.py and sendheaders.py
Underlying commits which conflicted were:
27acfc1 [qa] Update p2p-compactblocks.py for compactblocks v2
6976db2 [qa] Another attempt to fix race condition in p2p-compactblocks.py
The first commit changed the test_compactblock_construction function signature
and second commit added code which wasn't updated during the merge to use the
new arguments.
Suhas Daftuar <sdaftuar@chaincode.com> noticed the bug and suggested the fix.
sync_with_ping() only guarantees that the node has processed messages
it's received from the peer, not that block announcements from the node have
made it back to the peer. Replace sync_with_ping() with an explicit check that
the node's tip has been announced.
These are text files but their encoding does not depend on the locale.
Not all of them require utf8 but it is better to fix it at something
to remove potential unpredictability.
This is necessary on FreeBSD where no locale is set by default,
and apparently Python defaults not only the terminal encoding to the locale
but that of every text file. So without LOCALE environment it defaults text
file encoding to ASCII. This causes problems with e.g. `bitcoin.conf`.
Luckily the locale doesn't affect the default encoding for str.encode() and
bytes.decode() on Python 3, so this is the only change necessary.
This improves error reporting if `JSONRPCException` is not specifically caught
and ends up in Python's default backtrace handler.
Before:
```
Traceback (most recent call last):
File "/.../projects/bitcoin/bitcoin/qa/rpc-tests/test_framework/authproxy.py", line 153, in __call__
raise JSONRPCException(response['error'])
test_framework.authproxy.JSONRPCException
```
After:
```
Traceback (most recent call last):
File "/.../projects/bitcoin/bitcoin/qa/rpc-tests/test_framework/authproxy.py", line 152, in __call__
raise JSONRPCException(response['error'])
test_framework.authproxy.JSONRPCException: Unknown named parameter random (-8)
```
Python lambda use was incorrect.
sendcmpct messages need to be synchronized with RPC calls to generate().
Headers need to be synced (eg with getheaders) for cmpctblock announcements
to start.
Last test omitted sending a sendcmpct message.
waitfornewblock waits until a new block is received, or the timeout expires, then
returns the current block height/hash.
waitforblock waits for a specific blockhash, or until the timeout expires, then
returns the current block height/hash. If the target blockhash is the current
tip, it will return immediately.
waitforblockheight waits until the tip has reached a certain height or higher,
then returns the current height and hash.
waitforblockheight is used to avoid polling in the rpc tests.
MarcoFalke