cryptogera/bitcoin
1
0
Fork 0
mirror of https://github.com/bitcoin/bitcoin.git synced 2026-03-14 15:39:09 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge bitcoin/bitcoin#33702: contrib: Remove brittle, confusing and redundant UTF8 encoding from Python IO

Browse Source 
fad61185861a6a9ed806c387aa63d2b31262b1db test: Fix "typo" in written invalid content (MarcoFalke)
fab085c15f7221986f73af7e05e799edf3eadaf0 contrib: Use text=True in subprocess over manual encoding handling (MarcoFalke)
fa71c15f8610816a6ee0426cd396315da3d27c30 scripted-diff: Bump copyright headers after encoding changes (MarcoFalke)
fae612424b3e70acd6011a4459518174463b3424 contrib: Remove confusing and redundant encoding from IO (MarcoFalke)
fa7d72bd1be9a45e8c09525aee68caad1e57963e lint: Drop check to enforce encoding to be specified in Python scripts (MarcoFalke)
faf39d8539c9d563f68071054bbd533157f586ef test: Clarify that Python UTF-8 mode is the default today for most systems (MarcoFalke)
fa83e3a81ddb2170a0d7b0d86b94641a80d026ee lint: Do not allow locale dependent shell scripts (MarcoFalke)

Pull request description:

  Historically, there was an attempt via `test/lint/lint-python-utf8-encoding.py` to enforce explicit UTF8 in every Python IO statement (`open`, `subprocess`, ...). However, the lint check has many problems:

  * The check is incomplete and many IO statements lack the explicit UTF8 specification.
  * It was added at a time when some systems were not UTF8 by default.
  * The check is brittle, as it depends on a fragile regex.

  In theory, now that the minimum Python version is 3.10 (since commit 2123c94448ed142e78942421c597a1f264859c48), the check could be replaced by `PYTHONWARNDEFAULTENCODING=1` from https://docs.python.org/3/whatsnew/3.10.html#optional-encodingwarning-and-encoding-locale-option. However, this comes with many other problems:

  * All our Python scripts already assume and require UTF8 to be set externally. On almost all modern systems, this is already the default. Some Windows versions do not have UTF8 by default and require `PYTHONUTF8=1` to be set for the tests to run already today (with or without the changes in this pull). Also, the CI and many other Bash scripts force UTF8 via `LC_ALL`. Finally, Python 3.15 will likely enable UTF8 on *all* systems by default, per https://peps.python.org/pep-0686/#abstract.
  * So adding UTF8 to every single IO call is redundant, verbose, and confusing, given that it is the expected default.

  So fix all issues, by:

  * Removing the `test/lint/lint-python-utf8-encoding.py` check.
  * Removing the encoding on the individual IO calls.
  * Clarifying the existing docs around the existing UTF8 requirement and assumption.

  Obviously, every IO call is still free to specify UTF8 or any other encoding explicitly, if there is a documented need for it in the future.

ACKs for top commit:
  theStack:
    re-ACK fad61185861a6a9ed806c387aa63d2b31262b1db
  laanwj:
    Re-ACK fad61185861a6a9ed806c387aa63d2b31262b1db

Tree-SHA512: 78025ea3508597d2299490347614f0ee3e4c66e3ba559ff50e498045a9c8bbd92f3a5ced18719d8fcebbd1e47bdbb56a0c85a5b73b425adb0ea4f02fe69c3149
This commit is contained in:
merge-script 2025-12-03 09:54:47 +00:00
commit af0e6a65c9
No known key found for this signature in database
GPG Key ID: 2EEB9F5CC09526C1
63 changed files with 217 additions and 299 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
ci/test/02_run_container.py
View File

@ -26,7 +26,6 @@ def main():
["bash", "-c", "grep export ./ci/test/00_setup_env*.sh"],
stdout=subprocess.PIPE,
text=True,
encoding="utf8",
).stdout.splitlines()
settings = set(l.split("=")[0].split("export ")[1] for l in settings)
# Add "hidden" settings, which are never exported, manually. Otherwise,
@ -42,7 +41,7 @@ def main():
u=os.environ["USER"],
c=os.environ["CONTAINER_NAME"],
)
with open(env_file, "w", encoding="utf8") as file:
with open(env_file, "w") as file:
for k, v in os.environ.items():
if k in settings:
file.write(f"{k}={v}\n")

4
contrib/devtools/circular-dependencies.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2018-2020 The Bitcoin Core developers
# Copyright (c) 2018-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
@ -49,7 +49,7 @@ for arg in sys.argv[1:]:
# TODO: implement support for multiple include directories
for arg in sorted(files.keys()):
module = files[arg]
with open(arg, 'r', encoding="utf8") as f:
with open(arg, 'r') as f:
for line in f:
match = RE.match(line)
if match:

2
contrib/devtools/clang-format-diff.py
View File

@ -169,7 +169,7 @@ def main():
sys.exit(p.returncode)
if not args.i:
with open(filename, encoding="utf8") as f:
with open(filename) as f:
code = f.readlines()
formatted_code = StringIO(stdout).readlines()
diff = difflib.unified_diff(

18
contrib/devtools/copyright_header.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2016-2022 The Bitcoin Core developers
# Copyright (c) 2016-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
@ -54,12 +54,12 @@ GIT_LS_CMD = 'git ls-files --full-name'.split(' ')
GIT_TOPLEVEL_CMD = 'git rev-parse --show-toplevel'.split(' ')
def call_git_ls(base_directory):
out = subprocess.check_output([*GIT_LS_CMD, base_directory])
return [f for f in out.decode("utf-8").split('\n') if f != '']
out = subprocess.check_output([*GIT_LS_CMD, base_directory], text=True)
return [f for f in out.split('\n') if f != '']
def call_git_toplevel():
"Returns the absolute path to the project root"
return subprocess.check_output(GIT_TOPLEVEL_CMD).strip().decode("utf-8")
return subprocess.check_output(GIT_TOPLEVEL_CMD, text=True).strip()
def get_filenames_to_examine(base_directory):
"Returns an array of absolute paths to any project files in the base_directory that pass the include/exclude filters"
@ -140,7 +140,7 @@ def file_has_without_c_style_copyright_for_holder(contents, holder_name):
################################################################################
def read_file(filename):
return open(filename, 'r', encoding="utf8").read()
return open(filename, 'r').read()
def gather_file_info(filename):
info = {}
@ -298,8 +298,8 @@ def report_cmd(argv):
GIT_LOG_CMD = "git log --pretty=format:%%ai %s"
def call_git_log(filename):
out = subprocess.check_output((GIT_LOG_CMD % filename).split(' '))
return out.decode("utf-8").split('\n')
out = subprocess.check_output((GIT_LOG_CMD % filename).split(' '), text=True)
return out.split('\n')
def get_git_change_years(filename):
git_log_lines = call_git_log(filename)
@ -316,12 +316,12 @@ def get_most_recent_git_change_year(filename):
################################################################################
def read_file_lines(filename):
with open(filename, 'r', encoding="utf8") as f:
with open(filename, 'r') as f:
file_lines = f.readlines()
return file_lines
def write_file_lines(filename, file_lines):
with open(filename, 'w', encoding="utf8") as f:
with open(filename, 'w') as f:
f.write(''.join(file_lines))
################################################################################

6
contrib/filter-lcov.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2017-2020 The Bitcoin Core developers
# Copyright (c) 2017-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
@ -16,8 +16,8 @@ pattern = args.pattern
outfile = args.outfile
in_remove = False
with open(tracefile, 'r', encoding="utf8") as f:
with open(outfile, 'w', encoding="utf8") as wf:
with open(tracefile, 'r') as f:
with open(outfile, 'w') as wf:
for line in f:
for p in pattern:
if line.startswith("SF:") and p in line:

6
contrib/linearize/linearize-data.py
View File

@ -2,7 +2,7 @@
#
# linearize-data.py: Construct a linear, no-fork version of the chain.
#
# Copyright (c) 2013-2022 The Bitcoin Core developers
# Copyright (c) 2013-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
#
@ -34,7 +34,7 @@ def get_blk_dt(blk_hdr):
# When getting the list of block hashes, undo any byte reversals.
def get_block_hashes(settings):
blkindex = []
with open(settings['hashlist'], "r", encoding="utf8") as f:
with open(settings['hashlist'], "r") as f:
for line in f:
line = line.rstrip()
if settings['rev_hash_bytes'] == 'true':
@ -267,7 +267,7 @@ if __name__ == '__main__':
print("Usage: linearize-data.py CONFIG-FILE")
sys.exit(1)
with open(sys.argv[1], encoding="utf8") as f:
with open(sys.argv[1]) as f:
for line in f:
# skip comment lines
m = re.search(r'^\s*#', line)

6
contrib/linearize/linearize-hashes.py
View File

@ -2,7 +2,7 @@
#
# linearize-hashes.py: List blocks in a linear, no-fork version of the chain.
#
# Copyright (c) 2013-2022 The Bitcoin Core developers
# Copyright (c) 2013-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
#
@ -87,7 +87,7 @@ def get_block_hashes(settings, max_blocks_per_call=10000):
def get_rpc_cookie():
# Open the cookie file
with open(os.path.join(os.path.expanduser(settings['datadir']), '.cookie'), 'r', encoding="ascii") as f:
with open(os.path.join(os.path.expanduser(settings['datadir']), '.cookie'), 'r') as f:
combined = f.readline()
combined_split = combined.split(":")
settings['rpcuser'] = combined_split[0]
@ -98,7 +98,7 @@ if __name__ == '__main__':
print("Usage: linearize-hashes.py CONFIG-FILE")
sys.exit(1)
with open(sys.argv[1], encoding="utf8") as f:
with open(sys.argv[1]) as f:
for line in f:
# skip comment lines
m = re.search(r'^\s*#', line)

4
contrib/message-capture/message-capture-parser.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2020-2022 The Bitcoin Core developers
# Copyright (c) 2020-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""Parse message capture binary files. To be used in conjunction with -capturemessages."""
@ -205,7 +205,7 @@ def main():
jsonrep = json.dumps(messages)
if output:
with open(str(output), 'w+', encoding="utf8") as f_out:
with open(str(output), 'w+') as f_out:
f_out.write(jsonrep)
else:
print(jsonrep)

10
contrib/seeds/generate-seeds.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2014-2021 The Bitcoin Core developers
# Copyright (c) 2014-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
'''
@ -168,16 +168,16 @@ def main():
g.write(' *\n')
g.write(' * Each line contains a BIP155 serialized (networkID, addr, port) tuple.\n')
g.write(' */\n')
with open(os.path.join(indir,'nodes_main.txt'), 'r', encoding="utf8") as f:
with open(os.path.join(indir,'nodes_main.txt'), 'r') as f:
process_nodes(g, f, 'chainparams_seed_main')
g.write('\n')
with open(os.path.join(indir,'nodes_signet.txt'), 'r', encoding="utf8") as f:
with open(os.path.join(indir,'nodes_signet.txt'), 'r') as f:
process_nodes(g, f, 'chainparams_seed_signet')
g.write('\n')
with open(os.path.join(indir,'nodes_test.txt'), 'r', encoding="utf8") as f:
with open(os.path.join(indir,'nodes_test.txt'), 'r') as f:
process_nodes(g, f, 'chainparams_seed_test')
g.write('\n')
with open(os.path.join(indir,'nodes_testnet4.txt'), 'r', encoding="utf8") as f:
with open(os.path.join(indir,'nodes_testnet4.txt'), 'r') as f:
process_nodes(g, f, 'chainparams_seed_testnet4')
g.write('#endif // BITCOIN_CHAINPARAMSSEEDS_H\n')

4
contrib/seeds/makeseeds.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2013-2022 The Bitcoin Core developers
# Copyright (c) 2013-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
#
@ -211,7 +211,7 @@ def main():
print('Done.', file=sys.stderr)
print('Loading and parsing DNS seeds…', end='', file=sys.stderr, flush=True)
with open(args.seeds, 'r', encoding='utf8') as f:
with open(args.seeds, 'r') as f:
lines = f.readlines()
ips = [parseline(line) for line in lines]
random.shuffle(ips)

8
contrib/verify-binaries/verify.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2020-2021 The Bitcoin Core developers
# Copyright (c) 2020-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""Script for verifying Bitcoin Core release binaries.
@ -246,8 +246,8 @@ def files_are_equal(filename1, filename2):
eq = contents1 == contents2
if not eq:
with open(filename1, 'r', encoding='utf-8') as f1, \
open(filename2, 'r', encoding='utf-8') as f2:
with open(filename1, 'r') as f1, \
open(filename2, 'r') as f2:
f1lines = f1.readlines()
f2lines = f2.readlines()
@ -426,7 +426,7 @@ def verify_shasums_signature(
def parse_sums_file(sums_file_path: str, filename_filter: list[str]) -> list[list[str]]:
# extract hashes/filenames of binaries to verify from hash file;
# each line has the following format: "<hash> <binary_filename>"
with open(sums_file_path, 'r', encoding='utf8') as hash_file:
with open(sums_file_path, 'r') as hash_file:
return [line.split()[:2] for line in hash_file if len(filename_filter) == 0 or any(f in line for f in filename_filter)]

30
contrib/verify-commits/verify-commits.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2018-2022 The Bitcoin Core developers
# Copyright (c) 2018-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""Verify commits against a trusted keys list."""
@ -82,17 +82,17 @@ def main():
# get directory of this program and read data files
dirname = os.path.dirname(os.path.abspath(__file__))
print("Using verify-commits data from " + dirname)
with open(dirname + "/trusted-git-root", "r", encoding="utf8") as f:
with open(dirname + "/trusted-git-root", "r") as f:
verified_root = f.read().splitlines()[0]
with open(dirname + "/trusted-sha512-root-commit", "r", encoding="utf8") as f:
with open(dirname + "/trusted-sha512-root-commit", "r") as f:
verified_sha512_root = f.read().splitlines()[0]
with open(dirname + "/allow-revsig-commits", "r", encoding="utf8") as f:
with open(dirname + "/allow-revsig-commits", "r") as f:
revsig_allowed = f.read().splitlines()
with open(dirname + "/allow-unclean-merge-commits", "r", encoding="utf8") as f:
with open(dirname + "/allow-unclean-merge-commits", "r") as f:
unclean_merge_allowed = f.read().splitlines()
with open(dirname + "/allow-incorrect-sha512-commits", "r", encoding="utf8") as f:
with open(dirname + "/allow-incorrect-sha512-commits", "r") as f:
incorrect_sha512_allowed = f.read().splitlines()
with open(dirname + "/trusted-keys", "r", encoding="utf8") as f:
with open(dirname + "/trusted-keys", "r") as f:
trusted_keys = f.read().splitlines()
# Set commit and variables
@ -140,8 +140,8 @@ def main():
# Check that the commit (and parents) was signed with a trusted key
valid_sig = False
verify_res = subprocess.run([GIT, '-c', 'gpg.program={}/gpg.sh'.format(dirname), 'verify-commit', "--raw", current_commit], capture_output=True)
for line in verify_res.stderr.decode().splitlines():
verify_res = subprocess.run([GIT, '-c', 'gpg.program={}/gpg.sh'.format(dirname), 'verify-commit', "--raw", current_commit], text=True, capture_output=True)
for line in verify_res.stderr.splitlines():
if line.startswith("[GNUPG:] VALIDSIG "):
key = line.split(" ")[-1]
valid_sig = key in trusted_keys
@ -152,7 +152,7 @@ def main():
if prev_commit != "":
print("No parent of {} was signed with a trusted key!".format(prev_commit), file=sys.stderr)
print("Parents are:", file=sys.stderr)
parents = subprocess.check_output([GIT, 'show', '-s', '--format=format:%P', prev_commit]).decode('utf8').splitlines()[0].split(' ')
parents = subprocess.check_output([GIT, 'show', '-s', '--format=format:%P', prev_commit], text=True).splitlines()[0].split(' ')
for parent in parents:
subprocess.call([GIT, 'show', '-s', parent], stdout=sys.stderr)
else:
@ -162,29 +162,29 @@ def main():
# Check the Tree-SHA512
if (verify_tree or prev_commit == "") and current_commit not in incorrect_sha512_allowed:
tree_hash = tree_sha512sum(current_commit)
if ("Tree-SHA512: {}".format(tree_hash)) not in subprocess.check_output([GIT, 'show', '-s', '--format=format:%B', current_commit]).decode('utf8').splitlines():
if ("Tree-SHA512: {}".format(tree_hash)) not in subprocess.check_output([GIT, 'show', '-s', '--format=format:%B', current_commit], text=True).splitlines():
print("Tree-SHA512 did not match for commit " + current_commit, file=sys.stderr)
sys.exit(1)
# Merge commits should only have two parents
parents = subprocess.check_output([GIT, 'show', '-s', '--format=format:%P', current_commit]).decode('utf8').splitlines()[0].split(' ')
parents = subprocess.check_output([GIT, 'show', '-s', '--format=format:%P', current_commit], text=True).splitlines()[0].split(' ')
if len(parents) > 2:
print("Commit {} is an octopus merge".format(current_commit), file=sys.stderr)
sys.exit(1)
# Check that the merge commit is clean
commit_time = int(subprocess.check_output([GIT, 'show', '-s', '--format=format:%ct', current_commit]).decode('utf8').splitlines()[0])
commit_time = int(subprocess.check_output([GIT, 'show', '-s', '--format=format:%ct', current_commit], text=True).splitlines()[0])
check_merge = commit_time > time.time() - args.clean_merge * 24 * 60 * 60 # Only check commits in clean_merge days
allow_unclean = current_commit in unclean_merge_allowed
if len(parents) == 2 and check_merge and not allow_unclean:
current_tree = subprocess.check_output([GIT, 'show', '--format=%T', current_commit]).decode('utf8').splitlines()[0]
current_tree = subprocess.check_output([GIT, 'show', '--format=%T', current_commit], text=True).splitlines()[0]
# This merge-tree functionality requires git >= 2.38. The
# --write-tree option was added in order to opt-in to the new
# behavior. Older versions of git will not recognize the option and
# will instead exit with code 128.
try:
recreated_tree = subprocess.check_output([GIT, "merge-tree", "--write-tree", parents[0], parents[1]]).decode('utf8').splitlines()[0]
recreated_tree = subprocess.check_output([GIT, "merge-tree", "--write-tree", parents[0], parents[1]], text=True).splitlines()[0]
except subprocess.CalledProcessError as e:
if e.returncode == 128:
print("git v2.38+ is required for this functionality.", file=sys.stderr)

2
test/README.md
View File

@ -51,6 +51,8 @@ venv/bin/pip3 install ./pycapnp -C force-bundled-libcapnp=True
venv/bin/python3 build/test/functional/interface_ipc.py
```
The functional tests assume Python UTF-8 Mode, which is the default on most
systems.
On Windows the `PYTHONUTF8` environment variable must be set to 1:
```cmd

2
test/functional/combine_logs.py
View File

@ -148,7 +148,7 @@ def get_log_events(source, logfile):
Log events may be split over multiple lines. We use the timestamp
regex match as the marker for a new log event."""
try:
with open(logfile, 'r', encoding='utf-8') as infile:
with open(logfile, 'r') as infile:
event = ''
timestamp = ''
for line in infile:

4
test/functional/feature_asmap.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2020-2021 The Bitcoin Core developers
# Copyright (c) 2020-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""Test asmap config argument for ASN-based IP bucketing.
@ -97,7 +97,7 @@ class AsmapTest(BitcoinTestFramework):
self.log.info('Test bitcoind -asmap with empty map file')
self.stop_node(0)
empty_asmap = os.path.join(self.datadir, "ip_asn.map")
with open(empty_asmap, "w", encoding="utf-8") as f:
with open(empty_asmap, "w") as f:
f.write("")
msg = f"Error: Could not parse asmap file \"{empty_asmap}\""
self.node.assert_start_raises_init_error(extra_args=[f'-asmap={empty_asmap}'], expected_msg=msg)

42
test/functional/feature_config_args.py
View File

@ -55,7 +55,7 @@ class ConfArgsTest(BitcoinTestFramework):
os.rename(conf_path.with_suffix('.confbkp'), conf_path)
self.log.debug('Verifying includeconf directive pointing to directory is caught')
with open(conf_path, 'a', encoding='utf-8') as conf:
with open(conf_path, 'a') as conf:
conf.write(f'includeconf={self.nodes[0].datadir_path}\n')
self.nodes[0].assert_start_raises_init_error(
extra_args=['-regtest'],
@ -68,12 +68,12 @@ class ConfArgsTest(BitcoinTestFramework):
self.log.info('Disabling configuration via -noconf')
conf_path = self.nodes[0].datadir_path / 'bitcoin.conf'
with open(conf_path, encoding='utf-8') as conf:
with open(conf_path) as conf:
settings = [f'-{line.rstrip()}' for line in conf if len(line) > 1 and line[0] != '[']
os.rename(conf_path, conf_path.with_suffix('.confbkp'))
self.log.debug('Verifying garbage in config can be detected')
with open(conf_path, 'a', encoding='utf-8') as conf:
with open(conf_path, 'a') as conf:
conf.write('garbage\n')
self.nodes[0].assert_start_raises_init_error(
extra_args=['-regtest'],
@ -107,9 +107,9 @@ class ConfArgsTest(BitcoinTestFramework):
expected_msg=conf_in_config_file_err,
)
inc_conf_file_path = self.nodes[0].datadir_path / 'include.conf'
with open(self.nodes[0].datadir_path / 'bitcoin.conf', 'a', encoding='utf-8') as conf:
with open(self.nodes[0].datadir_path / 'bitcoin.conf', 'a') as conf:
conf.write(f'includeconf={inc_conf_file_path}\n')
with open(inc_conf_file_path, 'w', encoding='utf-8') as conf:
with open(inc_conf_file_path, 'w') as conf:
conf.write('conf=some.conf\n')
self.nodes[0].assert_start_raises_init_error(
expected_msg=conf_in_config_file_err,
@ -119,65 +119,65 @@ class ConfArgsTest(BitcoinTestFramework):
expected_msg='Error: Error parsing command line arguments: Invalid parameter -dash_cli=1',
extra_args=['-dash_cli=1'],
)
with open(inc_conf_file_path, 'w', encoding='utf-8') as conf:
with open(inc_conf_file_path, 'w') as conf:
conf.write('dash_conf=1\n')
with self.nodes[0].assert_debug_log(expected_msgs=['Ignoring unknown configuration value dash_conf']):
self.start_node(0)
self.stop_node(0)
with open(inc_conf_file_path, 'w', encoding='utf-8') as conf:
with open(inc_conf_file_path, 'w') as conf:
conf.write('reindex=1\n')
with self.nodes[0].assert_debug_log(expected_msgs=["[warning] reindex=1 is set in the configuration file, which will significantly slow down startup. Consider removing or commenting out this option for better performance, unless there is currently a condition which makes rebuilding the indexes necessary"]):
self.start_node(0)
self.stop_node(0)
with open(inc_conf_file_path, 'w', encoding='utf-8') as conf:
with open(inc_conf_file_path, 'w') as conf:
conf.write('-dash=1\n')
self.nodes[0].assert_start_raises_init_error(expected_msg='Error: Error reading configuration file: parse error on line 1: -dash=1, options in configuration file must be specified without leading -')
if self.is_wallet_compiled():
with open(inc_conf_file_path, 'w', encoding='utf8') as conf:
with open(inc_conf_file_path, 'w') as conf:
conf.write("wallet=foo\n")
self.nodes[0].assert_start_raises_init_error(expected_msg=f'Error: Config setting for -wallet only applied on {self.chain} network when in [{self.chain}] section.')
main_conf_file_path = self.nodes[0].datadir_path / "bitcoin_main.conf"
util.write_config(main_conf_file_path, n=0, chain='', extra_config=f'includeconf={inc_conf_file_path}\n')
with open(inc_conf_file_path, 'w', encoding='utf-8') as conf:
with open(inc_conf_file_path, 'w') as conf:
conf.write('acceptnonstdtxn=1\n')
self.nodes[0].assert_start_raises_init_error(extra_args=[f"-conf={main_conf_file_path}", "-allowignoredconf"], expected_msg='Error: acceptnonstdtxn is not currently supported for main chain')
with open(inc_conf_file_path, 'w', encoding='utf-8') as conf:
with open(inc_conf_file_path, 'w') as conf:
conf.write('nono\n')
self.nodes[0].assert_start_raises_init_error(expected_msg='Error: Error reading configuration file: parse error on line 1: nono, if you intended to specify a negated option, use nono=1 instead')
with open(inc_conf_file_path, 'w', encoding='utf-8') as conf:
with open(inc_conf_file_path, 'w') as conf:
conf.write('server=1\nrpcuser=someuser\nrpcpassword=some#pass')
self.nodes[0].assert_start_raises_init_error(expected_msg='Error: Error reading configuration file: parse error on line 3, using # in rpcpassword can be ambiguous and should be avoided')
with open(inc_conf_file_path, 'w', encoding='utf-8') as conf:
with open(inc_conf_file_path, 'w') as conf:
conf.write('server=1\nrpcuser=someuser\nmain.rpcpassword=some#pass')
self.nodes[0].assert_start_raises_init_error(expected_msg='Error: Error reading configuration file: parse error on line 3, using # in rpcpassword can be ambiguous and should be avoided')
with open(inc_conf_file_path, 'w', encoding='utf-8') as conf:
with open(inc_conf_file_path, 'w') as conf:
conf.write('server=1\nrpcuser=someuser\n[main]\nrpcpassword=some#pass')
self.nodes[0].assert_start_raises_init_error(expected_msg='Error: Error reading configuration file: parse error on line 4, using # in rpcpassword can be ambiguous and should be avoided')
inc_conf_file2_path = self.nodes[0].datadir_path / 'include2.conf'
with open(self.nodes[0].datadir_path / 'bitcoin.conf', 'a', encoding='utf-8') as conf:
with open(self.nodes[0].datadir_path / 'bitcoin.conf', 'a') as conf:
conf.write(f'includeconf={inc_conf_file2_path}\n')
with open(inc_conf_file_path, 'w', encoding='utf-8') as conf:
with open(inc_conf_file_path, 'w') as conf:
conf.write('testnot.datadir=1\n')
with open(inc_conf_file2_path, 'w', encoding='utf-8') as conf:
with open(inc_conf_file2_path, 'w') as conf:
conf.write('[testnet]\n')
self.restart_node(0)
self.nodes[0].stop_node(expected_stderr=f'Warning: {inc_conf_file_path}:1 Section [testnot] is not recognized.{os.linesep}{inc_conf_file2_path}:1 Section [testnet] is not recognized.')
with open(inc_conf_file_path, 'w', encoding='utf-8') as conf:
with open(inc_conf_file_path, 'w') as conf:
conf.write('') # clear
with open(inc_conf_file2_path, 'w', encoding='utf-8') as conf:
with open(inc_conf_file2_path, 'w') as conf:
conf.write('') # clear
def test_config_file_log(self):
@ -522,9 +522,9 @@ class ConfArgsTest(BitcoinTestFramework):
conf_file = default_data_dir / "bitcoin.conf"
# datadir needs to be set before [chain] section
with open(conf_file, encoding='utf8') as f:
with open(conf_file) as f:
conf_file_contents = f.read()
with open(conf_file, 'w', encoding='utf8') as f:
with open(conf_file, 'w') as f:
f.write(f"datadir={new_data_dir}\n")
f.write(conf_file_contents)

6
test/functional/feature_framework_startup_failures.py
View File

@ -43,15 +43,13 @@ class FeatureFrameworkStartupFailures(BitcoinTestFramework):
args = [sys.executable] + sys.argv + [f"--tmpdir={self.options.tmpdir}/{subdir}", f"--internal_test={test.__name__}"] + internal_args
try:
# The subprocess encoding argument gives different results for e.output
# on Linux/Windows, so handle decoding by ourselves for consistency.
output = subprocess.run(args, timeout=60 * self.options.timeout_factor,
stdout=subprocess.PIPE, stderr=subprocess.STDOUT).stdout.decode("utf-8")
stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True).stdout
except subprocess.TimeoutExpired as e:
print("Unexpected child process timeout!\n"
"WARNING: Timeouts like this halt execution of TestNode logic, "
"meaning dangling bitcoind processes are to be expected.\n"
f"<CHILD OUTPUT BEGIN>\n{e.output.decode('utf-8')}\n<CHILD OUTPUT END>",
f"<CHILD OUTPUT BEGIN>\n{e.output}\n<CHILD OUTPUT END>",
file=sys.stderr)
raise

16
test/functional/feature_includeconf.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2018-2021 The Bitcoin Core developers
# Copyright (c) 2018-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""Tests the includeconf argument
@ -24,12 +24,12 @@ class IncludeConfTest(BitcoinTestFramework):
def run_test(self):
# Create additional config files
# - tmpdir/node0/relative.conf
with open(self.nodes[0].datadir_path / "relative.conf", "w", encoding="utf8") as f:
with open(self.nodes[0].datadir_path / "relative.conf", "w") as f:
f.write("uacomment=relative\n")
# - tmpdir/node0/relative2.conf
with open(self.nodes[0].datadir_path / "relative2.conf", "w", encoding="utf8") as f:
with open(self.nodes[0].datadir_path / "relative2.conf", "w") as f:
f.write("uacomment=relative2\n")
with open(self.nodes[0].datadir_path / "bitcoin.conf", "a", encoding="utf8") as f:
with open(self.nodes[0].datadir_path / "bitcoin.conf", "a") as f:
f.write("uacomment=main\nincludeconf=relative.conf\n")
self.restart_node(0)
@ -50,7 +50,7 @@ class IncludeConfTest(BitcoinTestFramework):
)
self.log.info("-includeconf cannot be used recursively. subversion should end with 'main; relative)/'")
with open(self.nodes[0].datadir_path / "relative.conf", "a", encoding="utf8") as f:
with open(self.nodes[0].datadir_path / "relative.conf", "a") as f:
f.write("includeconf=relative2.conf\n")
self.start_node(0)
@ -61,7 +61,7 @@ class IncludeConfTest(BitcoinTestFramework):
self.log.info("-includeconf cannot contain invalid arg")
# Commented out as long as we ignore invalid arguments in configuration files
#with open(self.nodes[0].datadir_path / "relative.conf", "w", encoding="utf8") as f:
#with open(self.nodes[0].datadir_path / "relative.conf", "w") as f:
# f.write("foo=bar\n")
#self.nodes[0].assert_start_raises_init_error(expected_msg="Error: Error reading configuration file: Invalid configuration value foo")
@ -70,11 +70,11 @@ class IncludeConfTest(BitcoinTestFramework):
self.nodes[0].assert_start_raises_init_error(expected_msg="Error: Error reading configuration file: Failed to include configuration file relative.conf")
self.log.info("multiple -includeconf args can be used from the base config file. subversion should end with 'main; relative; relative2)/'")
with open(self.nodes[0].datadir_path / "relative.conf", "w", encoding="utf8") as f:
with open(self.nodes[0].datadir_path / "relative.conf", "w") as f:
# Restore initial file contents
f.write("uacomment=relative\n")
with open(self.nodes[0].datadir_path / "bitcoin.conf", "a", encoding="utf8") as f:
with open(self.nodes[0].datadir_path / "bitcoin.conf", "a") as f:
f.write("includeconf=relative2.conf\n")
self.start_node(0)

6
test/functional/feature_loadblock.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2017-2021 The Bitcoin Core developers
# Copyright (c) 2017-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""Test loadblock option
@ -41,10 +41,10 @@ class LoadblockTest(BitcoinTestFramework):
hash_list = tempfile.NamedTemporaryFile(dir=data_dir,
mode='w',
delete=False,
encoding="utf-8")
)
self.log.info("Create linearization config file")
with open(cfg_file, "a", encoding="utf-8") as cfg:
with open(cfg_file, "a") as cfg:
cfg.write(f"datadir={data_dir}\n")
cfg.write(f"rpcuser={node_url.username}\n")
cfg.write(f"rpcpassword={node_url.password}\n")

6
test/functional/feature_notifications.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2014-2022 The Bitcoin Core developers
# Copyright (c) 2014-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""Test the -alertnotify, -blocknotify and -walletnotify options."""
@ -200,7 +200,7 @@ class NotificationsTest(BitcoinTestFramework):
self.wait_until(lambda: os.path.isfile(self.shutdownnotify_file), timeout=10)
def large_work_invalid_chain_warning_in_alert_file(self):
with open(self.alertnotify_file, 'r', encoding='utf8') as f:
with open(self.alertnotify_file, 'r') as f:
alert_text = f.read()
return LARGE_WORK_INVALID_CHAIN_WARNING in alert_text
@ -213,7 +213,7 @@ class NotificationsTest(BitcoinTestFramework):
fname = os.path.join(self.walletnotify_dir, notify_outputname(self.wallet, tx_id))
# Wait for the cached writes to hit storage
self.wait_until(lambda: os.path.getsize(fname) > 0, timeout=10)
with open(fname, 'rt', encoding='utf-8') as f:
with open(fname, 'rt') as f:
text = f.read()
# Universal newline ensures '\n' on 'nt'
assert_equal(text[-1], '\n')

4
test/functional/feature_startupnotify.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2020-2022 The Bitcoin Core developers
# Copyright (c) 2020-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""Test -startupnotify."""
@ -27,7 +27,7 @@ class StartupNotifyTest(BitcoinTestFramework):
self.log.info("Test -startupnotify is executed once")
def get_count():
with open(tmpdir_file, "r", encoding="utf8") as f:
with open(tmpdir_file, "r") as f:
file_content = f.read()
return file_content.count(FILE_NAME)

4
test/functional/feature_taproot.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2019-2022 The Bitcoin Core developers
# Copyright (c) 2019-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
# Test Taproot softfork (BIPs 340-342)
@ -1396,7 +1396,7 @@ def dump_json_test(tx, input_utxos, idx, success, failure):
sha1 = hashlib.sha1(dump.encode("utf-8")).hexdigest()
dirname = os.environ.get("TEST_DUMP_DIR", ".") + ("/%s" % sha1[0])
os.makedirs(dirname, exist_ok=True)
with open(dirname + ("/%s" % sha1), 'w', encoding="utf8") as f:
with open(dirname + ("/%s" % sha1), 'w') as f:
f.write(dump)
# Data type to keep track of UTXOs, where they were created, and how to spend them.

6
test/functional/feature_versionbits_warning.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2016-2022 The Bitcoin Core developers
# Copyright (c) 2016-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""Test version bits warning system.
@ -32,7 +32,7 @@ class VersionBitsWarningTest(BitcoinTestFramework):
def setup_network(self):
self.alert_filename = os.path.join(self.options.tmpdir, "alert.txt")
# Open and close to create zero-length file
with open(self.alert_filename, 'w', encoding='utf8'):
with open(self.alert_filename, 'w'):
pass
self.extra_args = [[f"-alertnotify=echo %s >> \"{self.alert_filename}\""]]
self.setup_nodes()
@ -55,7 +55,7 @@ class VersionBitsWarningTest(BitcoinTestFramework):
def versionbits_in_alert_file(self):
"""Test that the versionbits warning has been written to the alert file."""
with open(self.alert_filename, 'r', encoding='utf8') as f:
with open(self.alert_filename, 'r') as f:
alert_text = f.read()
return VB_PATTERN.search(alert_text) is not None

4
test/functional/mining_mainnet.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2025 The Bitcoin Core developers
# Copyright (c) 2025-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""Test mining on an alternate mainnet
@ -83,7 +83,7 @@ class MiningMainnetTest(BitcoinTestFramework):
path = os.path.join(os.path.dirname(os.path.realpath(__file__)), self.options.datafile)
prev_hash = node.getbestblockhash()
blocks = None
with open(path, encoding='utf-8') as f:
with open(path) as f:
blocks = json.load(f)
n_blocks = len(blocks['timestamps'])
assert_equal(n_blocks, 2016)

4
test/functional/mocks/invalid_signer.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2018-2022 The Bitcoin Core developers
# Copyright (c) 2018-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
@ -11,7 +11,7 @@ import json
def perform_pre_checks():
mock_result_path = os.path.join(os.getcwd(), "mock_result")
if os.path.isfile(mock_result_path):
with open(mock_result_path, "r", encoding="utf8") as f:
with open(mock_result_path, "r") as f:
mock_result = f.read()
if mock_result[0]:
sys.stdout.write(mock_result[2:])

6
test/functional/mocks/signer.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2018-2022 The Bitcoin Core developers
# Copyright (c) 2018-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
@ -11,7 +11,7 @@ import json
def perform_pre_checks():
mock_result_path = os.path.join(os.getcwd(), "mock_result")
if os.path.isfile(mock_result_path):
with open(mock_result_path, "r", encoding="utf8") as f:
with open(mock_result_path, "r") as f:
mock_result = f.read()
if mock_result[0]:
sys.stdout.write(mock_result[2:])
@ -60,7 +60,7 @@ def signtx(args):
if args.fingerprint != "00000001":
return sys.stdout.write(json.dumps({"error": "Unexpected fingerprint", "fingerprint": args.fingerprint}))
with open(os.path.join(os.getcwd(), "mock_psbt"), "r", encoding="utf8") as f:
with open(os.path.join(os.getcwd(), "mock_psbt"), "r") as f:
mock_psbt = f.read()
if args.fingerprint == "00000001" :

4
test/functional/rpc_createmultisig.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2015-2022 The Bitcoin Core developers
# Copyright (c) 2015-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""Test multisig RPCs"""
@ -184,7 +184,7 @@ class RpcCreateMultiSigTest(BitcoinTestFramework):
def test_sortedmulti_descriptors_bip67(self):
self.log.info('Testing sortedmulti descriptors with BIP 67 test vectors')
with open(os.path.join(os.path.dirname(os.path.realpath(__file__)), 'data/rpc_bip67.json'), encoding='utf-8') as f:
with open(os.path.join(os.path.dirname(os.path.realpath(__file__)), 'data/rpc_bip67.json')) as f:
vectors = json.load(f)
for t in vectors:

4
test/functional/rpc_decodescript.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2015-2022 The Bitcoin Core developers
# Copyright (c) 2015-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""Test decoding scripts via decodescript RPC command."""
@ -266,7 +266,7 @@ class DecodeScriptTest(BitcoinTestFramework):
assert_equal('OP_RETURN 3011020701010101010101020601010101010101', rpc_result['vin'][0]['scriptSig']['asm'])
def decodescript_datadriven_tests(self):
with open(os.path.join(os.path.dirname(os.path.realpath(__file__)), 'data/rpc_decodescript.json'), encoding='utf-8') as f:
with open(os.path.join(os.path.dirname(os.path.realpath(__file__)), 'data/rpc_decodescript.json')) as f:
dd_tests = json.load(f)
for script, result in dd_tests:

6
test/functional/rpc_getblockstats.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2017-2021 The Bitcoin Core developers
# Copyright (c) 2017-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
@ -78,11 +78,11 @@ class GetblockstatsTest(BitcoinTestFramework):
'mocktime': int(mocktime),
'stats': self.expected_stats,
}
with open(filename, 'w', encoding="utf8") as f:
with open(filename, 'w') as f:
json.dump(to_dump, f, sort_keys=True, indent=2)
def load_test_data(self, filename):
with open(filename, 'r', encoding="utf8") as f:
with open(filename, 'r') as f:
d = json.load(f)
blocks = d['blocks']
mocktime = d['mocktime']

6
test/functional/rpc_help.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2018-2022 The Bitcoin Core developers
# Copyright (c) 2018-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""Test RPC help output."""
@ -22,7 +22,7 @@ def process_mapping(fname):
cmds = []
string_params = []
in_rpcs = False
with open(fname, "r", encoding="utf8") as f:
with open(fname, "r") as f:
for line in f:
line = line.rstrip()
if not in_rpcs:
@ -153,7 +153,7 @@ class HelpRpcTest(BitcoinTestFramework):
os.mkdir(dump_dir)
calls = [line.split(' ', 1)[0] for line in self.nodes[0].help().splitlines() if line and not line.startswith('==')]
for call in calls:
with open(os.path.join(dump_dir, call), 'w', encoding='utf-8') as f:
with open(os.path.join(dump_dir, call), 'w') as f:
# Make sure the node can generate the help at runtime without crashing
f.write(self.nodes[0].help(call))

4
test/functional/rpc_psbt.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2018-2022 The Bitcoin Core developers
# Copyright (c) 2018-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""Test the Partially Signed Transaction RPCs.
@ -799,7 +799,7 @@ class PSBTTest(BitcoinTestFramework):
assert_equal(unknown_psbt, unknown_out)
# Open the data file
with open(os.path.join(os.path.dirname(os.path.realpath(__file__)), 'data/rpc_psbt.json'), encoding='utf-8') as f:
with open(os.path.join(os.path.dirname(os.path.realpath(__file__)), 'data/rpc_psbt.json')) as f:
d = json.load(f)
invalids = d['invalid']
invalid_with_msgs = d["invalid_with_msg"]

4
test/functional/rpc_signer.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2017-2022 The Bitcoin Core developers
# Copyright (c) 2017-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""Test external signer.
@ -37,7 +37,7 @@ class RPCSignerTest(BitcoinTestFramework):
self.skip_if_no_external_signer()
def set_mock_result(self, node, res):
with open(os.path.join(node.cwd, "mock_result"), "w", encoding="utf8") as f:
with open(os.path.join(node.cwd, "mock_result"), "w") as f:
f.write(res)
def clear_mock_result(self, node):

6
test/functional/rpc_users.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2015-2022 The Bitcoin Core developers
# Copyright (c) 2015-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""Test multiple RPC users."""
@ -61,11 +61,11 @@ class HTTPBasicsTest(BitcoinTestFramework):
rpcauth3 = lines[1]
self.password = lines[3]
with open(self.nodes[0].datadir_path / "bitcoin.conf", "a", encoding="utf8") as f:
with open(self.nodes[0].datadir_path / "bitcoin.conf", "a") as f:
f.write(rpcauth + "\n")
f.write(rpcauth2 + "\n")
f.write(rpcauth3 + "\n")
with open(self.nodes[1].datadir_path / "bitcoin.conf", "a", encoding="utf8") as f:
with open(self.nodes[1].datadir_path / "bitcoin.conf", "a") as f:
f.write("rpcuser={}\n".format(self.rpcuser))
f.write("rpcpassword={}\n".format(self.rpcpassword))
self.restart_node(0)

6
test/functional/rpc_whitelist.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2017-2019 The Bitcoin Core developers
# Copyright (c) 2017-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""
@ -61,7 +61,7 @@ class RPCWhitelistTest(BitcoinTestFramework):
]
# These commands shouldn't be allowed for any user to test failures
self.never_allowed = ["getnetworkinfo"]
with open(self.nodes[0].datadir_path / "bitcoin.conf", "a", encoding="utf8") as f:
with open(self.nodes[0].datadir_path / "bitcoin.conf", "a") as f:
f.write("\nrpcwhitelistdefault=0\n")
for user in self.users:
f.write("rpcauth=" + user[0] + ":" + user[1] + "\n")
@ -96,7 +96,7 @@ class RPCWhitelistTest(BitcoinTestFramework):
# Replace file configurations
self.nodes[0].replace_in_config([("rpcwhitelistdefault=0", "rpcwhitelistdefault=1")])
with open(self.nodes[0].datadir_path / "bitcoin.conf", 'a', encoding='utf8') as f:
with open(self.nodes[0].datadir_path / "bitcoin.conf", 'a') as f:
f.write("rpcwhitelist=__cookie__:getblockcount,getblockchaininfo,getmempoolinfo,stop\n")
self.restart_node(0)

6
test/functional/test_framework/coverage.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2015-2021 The Bitcoin Core developers
# Copyright (c) 2015-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""Utilities for doing coverage analysis on the RPC interface.
@ -55,7 +55,7 @@ class AuthServiceProxyWrapper():
rpc_method = self.auth_service_proxy_instance._service_name
if self.coverage_logfile:
with open(self.coverage_logfile, 'a+', encoding='utf8') as f:
with open(self.coverage_logfile, 'a+') as f:
f.write("%s\n" % rpc_method)
def __truediv__(self, relative_uri):
@ -107,7 +107,7 @@ def write_all_rpc_commands(dirname: str, node: AuthServiceProxy) -> bool:
if line and not line.startswith('='):
commands.add("%s\n" % line.split()[0])
with open(filename, 'w', encoding='utf8') as f:
with open(filename, 'w') as f:
f.writelines(list(commands))
return True

6
test/functional/test_framework/crypto/ellswift.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2022 The Bitcoin Core developers
# Copyright (c) 2022-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""Test-only Elligator Swift implementation
@ -134,7 +134,7 @@ class TestFrameworkEllSwift(unittest.TestCase):
def test_elligator_encode_testvectors(self):
"""Implement the BIP324 test vectors for ellswift encoding (read from xswiftec_inv_test_vectors.csv)."""
vectors_file = os.path.join(os.path.dirname(os.path.realpath(__file__)), 'xswiftec_inv_test_vectors.csv')
with open(vectors_file, newline='', encoding='utf8') as csvfile:
with open(vectors_file, newline='') as csvfile:
reader = csv.DictReader(csvfile)
for row in reader:
u = FE.from_bytes(bytes.fromhex(row['u']))
@ -150,7 +150,7 @@ class TestFrameworkEllSwift(unittest.TestCase):
def test_elligator_decode_testvectors(self):
"""Implement the BIP324 test vectors for ellswift decoding (read from ellswift_decode_test_vectors.csv)."""
vectors_file = os.path.join(os.path.dirname(os.path.realpath(__file__)), 'ellswift_decode_test_vectors.csv')
with open(vectors_file, newline='', encoding='utf8') as csvfile:
with open(vectors_file, newline='') as csvfile:
reader = csv.DictReader(csvfile)
for row in reader:
encoding = bytes.fromhex(row['ellswift'])

2
test/functional/test_framework/key.py
View File

@ -316,7 +316,7 @@ class TestFrameworkKey(unittest.TestCase):
"""Implement the BIP340 test vectors (read from bip340_test_vectors.csv)."""
num_tests = 0
vectors_file = os.path.join(os.path.dirname(os.path.realpath(__file__)), 'bip340_test_vectors.csv')
with open(vectors_file, newline='', encoding='utf8') as csvfile:
with open(vectors_file, newline='') as csvfile:
reader = csv.reader(csvfile)
next(reader)
for row in reader:

4
test/functional/test_framework/netutil.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2014-2022 The Bitcoin Core developers
# Copyright (c) 2014-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""Linux network utilities.
@ -69,7 +69,7 @@ def netstat(typ='tcp'):
To get pid of all network process running on system, you must run this script
as superuser
'''
with open('/proc/net/'+typ,'r',encoding='utf8') as f:
with open('/proc/net/'+typ,'r') as f:
content = f.readlines()
content.pop(0)
result = []

2
test/functional/test_framework/test_framework.py
View File

@ -751,7 +751,7 @@ class BitcoinTestFramework(metaclass=BitcoinTestMetaClass):
self.log = logging.getLogger('TestFramework')
self.log.setLevel(logging.DEBUG)
# Create file handler to log all messages
fh = logging.FileHandler(self.options.tmpdir + '/test_framework.log', encoding='utf-8')
fh = logging.FileHandler(self.options.tmpdir + '/test_framework.log')
fh.setLevel(logging.DEBUG)
# Create console handler to log messages to stderr. By default this logs only error messages, but can be configured with --loglevel.
ch = logging.StreamHandler(sys.stdout)

4
test/functional/test_framework/test_node.py
View File

@ -506,13 +506,13 @@ class TestNode():
The substitutions are passed as a list of search-replace-tuples, e.g.
[("old", "new"), ("foo", "bar"), ...]
"""
with open(self.bitcoinconf, 'r', encoding='utf8') as conf:
with open(self.bitcoinconf, 'r') as conf:
conf_data = conf.read()
for replacement in replacements:
assert_equal(len(replacement), 2)
old, new = replacement[0], replacement[1]
conf_data = conf_data.replace(old, new)
with open(self.bitcoinconf, 'w', encoding='utf8') as conf:
with open(self.bitcoinconf, 'w') as conf:
conf.write(conf_data)
@property

10
test/functional/test_framework/util.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2014-2022 The Bitcoin Core developers
# Copyright (c) 2014-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""Helpful routines for regression testing."""
@ -528,7 +528,7 @@ def write_config(config_path, *, n, chain, extra_config="", disable_autoconnect=
else:
chain_name_conf_arg = chain
chain_name_conf_section = chain
with open(config_path, 'w', encoding='utf8') as f:
with open(config_path, 'w') as f:
if chain_name_conf_arg:
f.write("{}=1\n".format(chain_name_conf_arg))
if chain_name_conf_section:
@ -594,7 +594,7 @@ def get_temp_default_datadir(temp_dir: pathlib.Path) -> tuple[dict, pathlib.Path
def append_config(datadir, options):
with open(os.path.join(datadir, "bitcoin.conf"), 'a', encoding='utf8') as f:
with open(os.path.join(datadir, "bitcoin.conf"), 'a') as f:
for option in options:
f.write(option + "\n")
@ -603,7 +603,7 @@ def get_auth_cookie(datadir, chain):
user = None
password = None
if os.path.isfile(os.path.join(datadir, "bitcoin.conf")):
with open(os.path.join(datadir, "bitcoin.conf"), 'r', encoding='utf8') as f:
with open(os.path.join(datadir, "bitcoin.conf"), 'r') as f:
for line in f:
if line.startswith("rpcuser="):
assert user is None # Ensure that there is only one rpcuser line
@ -612,7 +612,7 @@ def get_auth_cookie(datadir, chain):
assert password is None # Ensure that there is only one rpcpassword line
password = line.split("=")[1].strip("\n")
try:
with open(os.path.join(datadir, chain, ".cookie"), 'r', encoding="ascii") as f:
with open(os.path.join(datadir, chain, ".cookie"), 'r') as f:
userpass = f.read()
split_userpass = userpass.split(':')
user = split_userpass[0]

8
test/functional/test_runner.py
View File

@ -428,7 +428,7 @@ def main():
# Read config generated by configure.
config = configparser.ConfigParser()
configfile = os.path.abspath(os.path.dirname(__file__)) + "/../config.ini"
config.read_file(open(configfile, encoding="utf8"))
config.read_file(open(configfile))
passon_args.append("--configfile=%s" % configfile)
@ -699,7 +699,7 @@ def print_results(test_results, max_len_name, runtime):
def write_results(test_results, filepath, total_runtime):
with open(filepath, mode="w", encoding="utf8") as results_file:
with open(filepath, mode="w") as results_file:
results_writer = csv.writer(results_file)
results_writer.writerow(['test', 'status', 'duration(seconds)'])
all_passed = True
@ -911,7 +911,7 @@ class RPCCoverage():
if not os.path.isfile(coverage_ref_filename):
raise RuntimeError("No coverage reference found")
with open(coverage_ref_filename, 'r', encoding="utf8") as coverage_ref_file:
with open(coverage_ref_filename, 'r') as coverage_ref_file:
all_cmds.update([line.strip() for line in coverage_ref_file.readlines()])
for root, _, files in os.walk(self.dir):
@ -920,7 +920,7 @@ class RPCCoverage():
coverage_filenames.add(os.path.join(root, filename))
for filename in coverage_filenames:
with open(filename, 'r', encoding="utf8") as coverage_file:
with open(filename, 'r') as coverage_file:
covered_cmds.update([line.strip() for line in coverage_file.readlines()])
return all_cmds - covered_cmds

10
test/functional/tool_signet_miner.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2022 The Bitcoin Core developers
# Copyright (c) 2022-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""Test signet miner tool"""
@ -102,8 +102,8 @@ class SignetMinerTest(BitcoinTestFramework):
'genpsbt',
f'--address={node.getnewaddress()}',
'--poolnum=98',
], check=True, input=json.dumps(template).encode('utf8'), capture_output=True)
psbt = genpsbt.stdout.decode('utf8').strip()
], check=True, text=True, input=json.dumps(template), capture_output=True)
psbt = genpsbt.stdout.strip()
if sign:
self.log.debug("Sign the PSBT")
res = node.walletprocesspsbt(psbt=psbt, sign=True, sighashtype='ALL')
@ -112,8 +112,8 @@ class SignetMinerTest(BitcoinTestFramework):
solvepsbt = subprocess.run(base_cmd + [
'solvepsbt',
f'--grind-cmd={shlex.join(util_argv)}',
], check=True, input=psbt.encode('utf8'), capture_output=True)
node.submitblock(solvepsbt.stdout.decode('utf8').strip())
], check=True, text=True, input=psbt, capture_output=True)
node.submitblock(solvepsbt.stdout.strip())
assert_equal(node.getblockcount(), n_blocks + 1)
def run_test(self):

6
test/functional/tool_utils.py
View File

@ -28,7 +28,7 @@ class ToolUtils(BitcoinTestFramework):
def run_test(self):
self.testcase_dir = Path(self.config["environment"]["SRCDIR"]) / "test" / "functional" / "data" / "util"
self.bins = self.get_binaries()
with open(self.testcase_dir / "bitcoin-util-test.json", encoding="utf8") as f:
with open(self.testcase_dir / "bitcoin-util-test.json") as f:
input_data = json.loads(f.read())
for i, test_obj in enumerate(input_data):
@ -50,7 +50,7 @@ class ToolUtils(BitcoinTestFramework):
# Read the input data (if there is any)
inputData = None
if "input" in testObj:
with open(self.testcase_dir / testObj["input"], encoding="utf8") as f:
with open(self.testcase_dir / testObj["input"]) as f:
inputData = f.read()
# Read the expected output data (if there is any)
@ -60,7 +60,7 @@ class ToolUtils(BitcoinTestFramework):
if "output_cmp" in testObj:
outputFn = testObj['output_cmp']
outputType = os.path.splitext(outputFn)[1][1:] # output type from file extension (determines how to compare)
with open(self.testcase_dir / outputFn, encoding="utf8") as f:
with open(self.testcase_dir / outputFn) as f:
outputData = f.read()
if not outputData:
raise Exception(f"Output data missing for {outputFn}")

6
test/functional/tool_wallet.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2018-2022 The Bitcoin Core developers
# Copyright (c) 2018-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""Test bitcoin-wallet."""
@ -83,7 +83,7 @@ class ToolWalletTest(BitcoinTestFramework):
def read_dump(self, filename):
dump = OrderedDict()
with open(filename, "r", encoding="utf8") as f:
with open(filename, "r") as f:
for row in f:
row = row.strip()
key, value = row.split(',')
@ -98,7 +98,7 @@ class ToolWalletTest(BitcoinTestFramework):
def write_dump(self, dump, filename, magic=None, skip_checksum=False):
if magic is None:
magic = "BITCOIN_CORE_WALLET_DUMP"
with open(filename, "w", encoding="utf8") as f:
with open(filename, "w") as f:
row = ",".join([magic, dump[magic]]) + "\n"
f.write(row)
for k, v in dump.items():

4
test/functional/wallet_backup.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2014-2022 The Bitcoin Core developers
# Copyright (c) 2014-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""Test the wallet backup features.
@ -112,7 +112,7 @@ class WalletBackupTest(BitcoinTestFramework):
def restore_invalid_wallet(self):
node = self.nodes[3]
invalid_wallet_file = self.nodes[0].datadir_path / 'invalid_wallet_file.bak'
open(invalid_wallet_file, 'a', encoding="utf8").write('invald wallet')
open(invalid_wallet_file, "a").write("invalid_wallet_content")
wallet_name = "res0"
not_created_wallet_file = node.wallets_path / wallet_name
error_message = "Wallet file verification failed. Failed to load database path '{}'. Data is not in recognized format.".format(not_created_wallet_file)

4
test/functional/wallet_backwards_compatibility.py
View File

@ -100,7 +100,7 @@ class BackwardsCompatibilityTest(BitcoinTestFramework):
bad_deriv_wallet.dumpwallet(dump_path)
addr = None
seed = None
with open(dump_path, encoding="utf8") as f:
with open(dump_path) as f:
for line in f:
if f"hdkeypath=m/0'/0'/{LAST_KEYPOOL_INDEX}'" in line:
addr = line.split(" ")[4].split("=")[1]
@ -123,7 +123,7 @@ class BackwardsCompatibilityTest(BitcoinTestFramework):
os.unlink(dump_path)
bad_deriv_wallet.dumpwallet(dump_path)
bad_path_addr = None
with open(dump_path, encoding="utf8") as f:
with open(dump_path) as f:
for line in f:
if f"hdkeypath={bad_deriv_path}" in line:
bad_path_addr = line.split(" ")[4].split("=")[1]

8
test/functional/wallet_encryption.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2016-2022 The Bitcoin Core developers
# Copyright (c) 2016-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""Test Wallet encryption"""
@ -134,13 +134,13 @@ class WalletEncryptionTest(BitcoinTestFramework):
do_wallet_tool("-wallet=noprivs", f"-dumpfile={dumpfile_path}", "dump")
# Modify the dump
with open(dumpfile_path, "r", encoding="utf-8") as f:
with open(dumpfile_path, "r") as f:
dump_content = f.readlines()
# Drop the checksum line
dump_content = dump_content[:-1]
# Insert a valid mkey line. This corresponds to a passphrase of "pass".
dump_content.append("046d6b657901000000,300dc926f3b3887aad3d5d5f5a0fc1b1a4a1722f9284bd5c6ff93b64a83902765953939c58fe144013c8b819f42cf698b208e9911e5f0c544fa300000000cc52050000\n")
with open(dumpfile_path, "w", encoding="utf-8") as f:
with open(dumpfile_path, "w") as f:
contents = "".join(dump_content)
f.write(contents)
checksum = hash256(contents.encode())
@ -158,7 +158,7 @@ class WalletEncryptionTest(BitcoinTestFramework):
# Make a new dump and check that there are no mkeys
dumpfile_path = self.nodes[0].datadir_path / "noprivs_enc.dump"
do_wallet_tool("-wallet=noprivs_enc", f"-dumpfile={dumpfile_path}", "dump")
with open(dumpfile_path, "r", encoding="utf-8") as f:
with open(dumpfile_path, "r") as f:
# Check there's nothing with an 'mkey' prefix
assert_equal(all([not line.startswith("046d6b6579") for line in f]), True)

4
test/functional/wallet_multiwallet.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2017-2022 The Bitcoin Core developers
# Copyright (c) 2017-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""Test multiwallet.
@ -180,7 +180,7 @@ class MultiWalletTest(BitcoinTestFramework):
self.nodes[0].assert_start_raises_init_error(['-walletdir=bad'], 'Error: Specified -walletdir "bad" does not exist')
# should not initialize if the specified walletdir is not a directory
not_a_dir = wallet_dir('notadir')
open(not_a_dir, 'a', encoding="utf8").close()
open(not_a_dir, 'a').close()
self.nodes[0].assert_start_raises_init_error(['-walletdir=' + not_a_dir], 'Error: Specified -walletdir "' + not_a_dir + '" is not a directory')
# if wallets/ doesn't exist, datadir should be the default wallet dir

8
test/functional/wallet_signer.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2017-2022 The Bitcoin Core developers
# Copyright (c) 2017-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""Test external signer.
@ -48,7 +48,7 @@ class WalletSignerTest(BitcoinTestFramework):
self.skip_if_no_wallet()
def set_mock_result(self, node, res):
with open(os.path.join(node.cwd, "mock_result"), "w", encoding="utf8") as f:
with open(os.path.join(node.cwd, "mock_result"), "w") as f:
f.write(res)
def clear_mock_result(self, node):
@ -170,7 +170,7 @@ class WalletSignerTest(BitcoinTestFramework):
assert hww.testmempoolaccept([mock_tx])[0]["allowed"]
with open(os.path.join(self.nodes[1].cwd, "mock_psbt"), "w", encoding="utf8") as f:
with open(os.path.join(self.nodes[1].cwd, "mock_psbt"), "w") as f:
f.write(mock_psbt_signed["psbt"])
self.log.info('Test send using hww1')
@ -195,7 +195,7 @@ class WalletSignerTest(BitcoinTestFramework):
mock_psbt_bumped = mock_wallet.psbtbumpfee(orig_tx_id)["psbt"]
mock_psbt_bumped_signed = mock_wallet.walletprocesspsbt(psbt=mock_psbt_bumped, sign=True, sighashtype="ALL", bip32derivs=True)
with open(os.path.join(self.nodes[1].cwd, "mock_psbt"), "w", encoding="utf8") as f:
with open(os.path.join(self.nodes[1].cwd, "mock_psbt"), "w") as f:
f.write(mock_psbt_bumped_signed["psbt"])
self.log.info('Test bumpfee using hww1')

2
test/fuzz/test_runner.py
View File

@ -99,7 +99,7 @@ def main():
# Read config generated by configure.
config = configparser.ConfigParser()
configfile = os.path.abspath(os.path.dirname(__file__)) + "/../config.ini"
config.read_file(open(configfile, encoding="utf8"))
config.read_file(open(configfile))
if not config["components"].getboolean("ENABLE_FUZZ_BINARY"):
logging.error("Must have fuzz executable built")

12
test/lint/check-doc.py
View File

@ -1,13 +1,11 @@
#!/usr/bin/env python3
# Copyright (c) 2015-2022 The Bitcoin Core developers
# Copyright (c) 2015-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
'''
This checks if all command line args are documented.
Return value is 0 to indicate no error.
Author: @MarcoFalke
'''
from subprocess import check_output
@ -27,8 +25,8 @@ SET_DOC_OPTIONAL = set(['-h', '-?', '-dbcrashratio', '-forcecompactdb', '-ipccon
def lint_missing_argument_documentation():
used = check_output(CMD_GREP_ARGS, shell=True).decode('utf8').strip()
docd = check_output(CMD_GREP_DOCS, shell=True).decode('utf8').strip()
used = check_output(CMD_GREP_ARGS, shell=True, text=True).strip()
docd = check_output(CMD_GREP_DOCS, shell=True, text=True).strip()
args_used = set(re.findall(re.compile(REGEX_ARG), used))
args_docd = set(re.findall(re.compile(REGEX_DOC), docd)).union(SET_DOC_OPTIONAL)
@ -46,8 +44,8 @@ def lint_missing_argument_documentation():
def lint_missing_hidden_wallet_args():
wallet_args = check_output(CMD_GREP_WALLET_ARGS, shell=True).decode('utf8').strip()
wallet_hidden_args = check_output(CMD_GREP_WALLET_HIDDEN_ARGS, shell=True).decode('utf8').strip()
wallet_args = check_output(CMD_GREP_WALLET_ARGS, shell=True, text=True).strip()
wallet_hidden_args = check_output(CMD_GREP_WALLET_HIDDEN_ARGS, shell=True, text=True).strip()
wallet_args = set(re.findall(re.compile(REGEX_DOC), wallet_args))
wallet_hidden_args = set(re.findall(re.compile(r' "([^"=]+)'), wallet_hidden_args))

10
test/lint/lint-files.py
View File

@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2021-2022 The Bitcoin Core developers
# Copyright (c) 2021-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
@ -76,7 +76,7 @@ def get_git_file_metadata() -> dict[str, FileMeta]:
'''
Return a dictionary mapping the name of all files in the repository to git tree metadata.
'''
files_raw = check_output(CMD_ALL_FILES).decode("utf8").rstrip("\0").split("\0")
files_raw = check_output(CMD_ALL_FILES, text=True).rstrip("\0").split("\0")
files = {}
for file_spec in files_raw:
meta = FileMeta(file_spec)
@ -169,7 +169,7 @@ def check_shebang_file_permissions(files_meta) -> int:
"""
Checks every file that contains a shebang line to ensure it has an executable permission
"""
filenames = check_output(CMD_SHEBANG_FILES).decode("utf8").strip().split("\n")
filenames = check_output(CMD_SHEBANG_FILES, text=True).strip().split("\n")
# The git grep command we use returns files which contain a shebang on any line within the file
# so we need to filter the list to only files with the shebang on the first line
@ -185,7 +185,7 @@ def check_shebang_file_permissions(files_meta) -> int:
# *.py files which don't contain an `if __name__ == '__main__'` are not expected to be executed directly
if file_meta.extension == "py":
with open(filename, "r", encoding="utf8") as f:
with open(filename, "r") as f:
file_data = f.read()
if not re.search("""if __name__ == ['"]__main__['"]:""", file_data):
continue
@ -198,7 +198,7 @@ def check_shebang_file_permissions(files_meta) -> int:
def main() -> NoReturn:
root_dir = check_output(CMD_TOP_LEVEL).decode("utf8").strip()
root_dir = check_output(CMD_TOP_LEVEL, text=True).strip()
os.chdir(root_dir)
files = get_git_file_metadata()

6
test/lint/lint-include-guards.py
View File

@ -1,6 +1,6 @@
#!/usr/bin/env python3
#
# Copyright (c) 2018-2022 The Bitcoin Core developers
# Copyright (c) 2018-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
@ -31,7 +31,7 @@ def _get_header_file_lst() -> list[str]:
"""
git_cmd_lst = ['git', 'ls-files', '--', '*.h']
header_file_lst = check_output(
git_cmd_lst).decode('utf-8').splitlines()
git_cmd_lst, text=True).splitlines()
header_file_lst = [hf for hf in header_file_lst
if not any(ef in hf for ef
@ -71,7 +71,7 @@ def main():
regex_pattern = f'^#(ifndef|define|endif //) {header_id}'
with open(header_file, 'r', encoding='utf-8') as f:
with open(header_file, 'r') as f:
header_file_contents = f.readlines()
count = 0

18
test/lint/lint-includes.py
View File

@ -1,6 +1,6 @@
#!/usr/bin/env python3
#
# Copyright (c) 2018-2022 The Bitcoin Core developers
# Copyright (c) 2018-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
#
@ -40,13 +40,13 @@ EXPECTED_BOOST_INCLUDES = [
def get_toplevel():
return check_output(["git", "rev-parse", "--show-toplevel"], text=True, encoding="utf8").rstrip("\n")
return check_output(["git", "rev-parse", "--show-toplevel"], text=True).rstrip("\n")
def list_files_by_suffix(suffixes):
exclude_args = [":(exclude)" + dir for dir in EXCLUDED_DIRS]
files_list = check_output(["git", "ls-files", "src"] + exclude_args, text=True, encoding="utf8").splitlines()
files_list = check_output(["git", "ls-files", "src"] + exclude_args, text=True).splitlines()
return [file for file in files_list if file.endswith(suffixes)]
@ -68,7 +68,7 @@ def find_included_cpps():
included_cpps = list()
try:
included_cpps = check_output(["git", "grep", "-E", r"^#include [<\"][^>\"]+\.cpp[>\"]", "--", "*.cpp", "*.h"], text=True, encoding="utf8").splitlines()
included_cpps = check_output(["git", "grep", "-E", r"^#include [<\"][^>\"]+\.cpp[>\"]", "--", "*.cpp", "*.h"], text=True).splitlines()
except CalledProcessError as e:
if e.returncode > 1:
raise e
@ -82,7 +82,7 @@ def find_extra_boosts():
exclusion_set = set()
try:
included_boosts = check_output(["git", "grep", "-E", r"^#include <boost/", "--", "*.cpp", "*.h"], text=True, encoding="utf8").splitlines()
included_boosts = check_output(["git", "grep", "-E", r"^#include <boost/", "--", "*.cpp", "*.h"], text=True).splitlines()
except CalledProcessError as e:
if e.returncode > 1:
raise e
@ -105,7 +105,7 @@ def find_quote_syntax_inclusions():
quote_syntax_inclusions = list()
try:
quote_syntax_inclusions = check_output(["git", "grep", r"^#include \"", "--", "*.cpp", "*.h"] + exclude_args, text=True, encoding="utf8").splitlines()
quote_syntax_inclusions = check_output(["git", "grep", r"^#include \"", "--", "*.cpp", "*.h"] + exclude_args, text=True).splitlines()
except CalledProcessError as e:
if e.returncode > 1:
raise e
@ -120,7 +120,7 @@ def main():
# Check for duplicate includes
for filename in list_files_by_suffix((".cpp", ".h")):
with open(filename, "r", encoding="utf8") as file:
with open(filename, "r") as file:
include_list = [line.rstrip("\n") for line in file if re.match(r"^#include", line)]
duplicates = find_duplicate_includes(include_list)
@ -148,13 +148,13 @@ def main():
if extra_boosts:
for boost in extra_boosts:
print(f"A new Boost dependency in the form of \"{boost}\" appears to have been introduced:")
print(check_output(["git", "grep", boost, "--", "*.cpp", "*.h"], text=True, encoding="utf8"))
print(check_output(["git", "grep", boost, "--", "*.cpp", "*.h"], text=True))
exit_code = 1
# Check if Boost dependencies are no longer used
for expected_boost in EXPECTED_BOOST_INCLUDES:
try:
check_output(["git", "grep", "-q", r"^#include <%s>" % expected_boost, "--", "*.cpp", "*.h"], text=True, encoding="utf8")
check_output(["git", "grep", "-q", r"^#include <%s>" % expected_boost, "--", "*.cpp", "*.h"], text=True)
except CalledProcessError as e:
if e.returncode > 1:
raise e

2
test/lint/lint-locale-dependence.py
View File

@ -219,7 +219,7 @@ def find_locale_dependent_function_uses():
git_grep_output = list()
try:
git_grep_output = check_output(git_grep_command, text=True, encoding="utf8").splitlines()
git_grep_output = check_output(git_grep_command, text=True).splitlines()
except CalledProcessError as e:
if e.returncode > 1:
raise e

8
test/lint/lint-python-dead-code.py
View File

@ -1,6 +1,6 @@
#!/usr/bin/env python3
#
# Copyright (c) 2022 The Bitcoin Core developers
# Copyright (c) 2022-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
@ -24,15 +24,15 @@ def check_vulture_install():
def main():
check_vulture_install()
files = check_output(FILES_ARGS).decode("utf-8").splitlines()
files = check_output(FILES_ARGS, text=True).splitlines()
# --min-confidence 100 will only report code that is guaranteed to be unused within the analyzed files.
# Any value below 100 introduces the risk of false positives, which would create an unacceptable maintenance burden.
vulture_args = ['vulture', '--min-confidence=100'] + files
try:
check_output(vulture_args, stderr=STDOUT)
check_output(vulture_args, stderr=STDOUT, text=True)
except CalledProcessError as e:
print(e.output.decode("utf-8"), end="")
print(e.output, end="")
print("Python dead code detection found some issues")
exit(1)

73
test/lint/lint-python-utf8-encoding.py
View File

@ -1,73 +0,0 @@
#!/usr/bin/env python3
#
# Copyright (c) 2018-2022 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
#
# Make sure we explicitly open all text files using UTF-8 (or ASCII) encoding to
# avoid potential issues on the BSDs where the locale is not always set.
import sys
import re
from subprocess import check_output, CalledProcessError
EXCLUDED_DIRS = ["src/crc32c/", "src/secp256k1/"]
def get_exclude_args():
return [":(exclude)" + dir for dir in EXCLUDED_DIRS]
def check_fileopens():
fileopens = list()
try:
fileopens = check_output(["git", "grep", r" open(", "--", "*.py"] + get_exclude_args(), text=True, encoding="utf8").splitlines()
except CalledProcessError as e:
if e.returncode > 1:
raise e
filtered_fileopens = [fileopen for fileopen in fileopens if not re.search(r"encoding=.(ascii|utf8|utf-8).|open\([^,]*, (\*\*kwargs|['\"][^'\"]*b[^'\"]*['\"])", fileopen)]
return filtered_fileopens
def check_checked_outputs():
checked_outputs = list()
try:
checked_outputs = check_output(["git", "grep", "check_output(", "--", "*.py"] + get_exclude_args(), text=True, encoding="utf8").splitlines()
except CalledProcessError as e:
if e.returncode > 1:
raise e
filtered_checked_outputs = [checked_output for checked_output in checked_outputs if re.search(r"text=True", checked_output) and not re.search(r"encoding=.(ascii|utf8|utf-8).", checked_output)]
return filtered_checked_outputs
def main():
exit_code = 0
nonexplicit_utf8_fileopens = check_fileopens()
if nonexplicit_utf8_fileopens:
print("Python's open(...) seems to be used to open text files without explicitly specifying encoding='utf8':\n")
for fileopen in nonexplicit_utf8_fileopens:
print(fileopen)
exit_code = 1
nonexplicit_utf8_checked_outputs = check_checked_outputs()
if nonexplicit_utf8_checked_outputs:
if nonexplicit_utf8_fileopens:
print("\n")
print("Python's check_output(...) seems to be used to get program outputs without explicitly specifying encoding='utf8':\n")
for checked_output in nonexplicit_utf8_checked_outputs:
print(checked_output)
exit_code = 1
sys.exit(exit_code)
if __name__ == "__main__":
main()

4
test/lint/lint-python.py
View File

@ -1,6 +1,6 @@
#!/usr/bin/env python3
#
# Copyright (c) 2022 The Bitcoin Core developers
# Copyright (c) 2022-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
@ -37,7 +37,7 @@ def check_dependencies():
def main():
check_dependencies()
mypy_files = subprocess.check_output(MYPY_FILES_ARGS).decode("utf-8").splitlines()
mypy_files = subprocess.check_output(MYPY_FILES_ARGS, text=True).splitlines()
mypy_args = ['mypy', '--show-error-codes'] + mypy_files
try:

20
test/lint/lint-shell-locale.py
View File

@ -1,22 +1,19 @@
#!/usr/bin/env python3
#
# Copyright (c) 2018-2022 The Bitcoin Core developers
# Copyright (c) 2018-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""
Make sure all shell scripts are:
a.) explicitly opt out of locale dependence using
"export LC_ALL=C" or "export LC_ALL=C.UTF-8", or
b.) explicitly opt in to locale dependence using the annotation below.
Make sure all shell scripts explicitly opt out of locale dependence using
"export LC_ALL=C" or "export LC_ALL=C.UTF-8", which also enables UTF-8 mode in
Python. See: https://docs.python.org/3/library/os.html#python-utf-8-mode
"""
import subprocess
import sys
import re
OPT_IN_LINE = '# This script is intentionally locale dependent by not setting \"export LC_ALL=C\"'
OPT_OUT_LINES = [
'export LC_ALL=C',
'export LC_ALL=C.UTF-8',
@ -30,10 +27,10 @@ def get_shell_files_list():
'*.sh',
]
try:
return subprocess.check_output(command, stderr = subprocess.STDOUT).decode('utf-8').splitlines()
return subprocess.check_output(command, stderr = subprocess.STDOUT, text=True).splitlines()
except subprocess.CalledProcessError as e:
if e.returncode > 1: # return code is 1 when match is empty
print(e.output.decode('utf-8'), end='')
print(e.output, end='')
sys.exit(1)
return []
@ -44,12 +41,9 @@ def main():
if re.search('src/(secp256k1|minisketch)/', file_path):
continue
with open(file_path, 'r', encoding='utf-8') as file_obj:
with open(file_path, 'r') as file_obj:
contents = file_obj.read()
if OPT_IN_LINE in contents:
continue
non_comment_pattern = re.compile(r'^\s*((?!#).+)$', re.MULTILINE)
non_comment_lines = re.findall(non_comment_pattern, contents)
if not non_comment_lines:

8
test/lint/lint-spelling.py
View File

@ -1,6 +1,6 @@
#!/usr/bin/env python3
#
# Copyright (c) 2022 The Bitcoin Core developers
# Copyright (c) 2022-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
@ -29,13 +29,13 @@ def check_codespell_install():
def main():
check_codespell_install()
files = check_output(FILES_ARGS).decode("utf-8").splitlines()
files = check_output(FILES_ARGS, text=True).splitlines()
codespell_args = ['codespell', '--check-filenames', '--disable-colors', '--quiet-level=7', '--ignore-words={}'.format(IGNORE_WORDS_FILE)] + files
try:
check_output(codespell_args, stderr=STDOUT)
check_output(codespell_args, stderr=STDOUT, text=True)
except CalledProcessError as e:
print(e.output.decode("utf-8"), end="")
print(e.output, end="")
print('^ Warning: codespell identified likely spelling errors. Any false positives? Add them to the list of ignored words in {}'.format(IGNORE_WORDS_FILE))

4
test/lint/lint-submodule.py
View File

@ -1,6 +1,6 @@
#!/usr/bin/env python3
#
# Copyright (c) 2022 The Bitcoin Core developers
# Copyright (c) 2022-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
@ -13,7 +13,7 @@ import sys
def main():
submodules_list = subprocess.check_output(['git', 'submodule', 'status', '--recursive'],
text = True, encoding = 'utf8').rstrip('\n')
text = True).rstrip('\n')
if submodules_list:
print("These submodules were found, delete them:\n", submodules_list)
sys.exit(1)

4
test/lint/lint-tests.py
View File

@ -1,6 +1,6 @@
#!/usr/bin/env python3
#
# Copyright (c) 2018-2022 The Bitcoin Core developers
# Copyright (c) 2018-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
@ -23,7 +23,7 @@ def grep_boost_fixture_test_suite():
"src/test/**.cpp",
"src/wallet/test/**.cpp",
]
return subprocess.check_output(command, text=True, encoding="utf8")
return subprocess.check_output(command, text=True)
def check_matching_test_names(test_suite_list):