Conversation
introduces wildcards into renaming of files to match standard amset format transport_*.json* currently looks to rename transport.json which does not exist.
Reorganize import statements and remove unused imports.
Add test for copy_amset_files function handling transport files.
Updated test to handle gzipped transport files
|
@utf can you please check? |
…Jackbt21/atomate2 into Amset_interpolation_Convergence
| ) | ||
|
|
||
| rename_files({"transport.json": "transport.prev.json"}, allow_missing=True) | ||
| local_transport = next(Path().glob("transport_*.json*"), None) |
There was a problem hiding this comment.
Would a safer wildcard pattern be "transport*.json" so the previous matches would still be there (if applicable)?
There was a problem hiding this comment.
In any of the later iterations (from the 3rd onwards). The previous directory will contain transport.prev.json* and transport_{mesh}.json*. The underscore is here to differentiate between the two so the most recent transport file is selected and not the old one.
| found_file = get_zfile(directory_listing, file, allow_missing=True) | ||
| if found_file is not None: | ||
| files.append(found_file) | ||
| files.append(Path("transport_*.json*")) |
There was a problem hiding this comment.
Change to this? Calling Path with a regex interprets the symbol literally, AFAIK
files.extend(list(Path(".").glob("transport*.json")))There was a problem hiding this comment.
This change was introduced to satisfy the mypy type checking in the test suite, which expects files to contain either Path or None. The path stored inside the Path object is later treated as a globbable string, rather than being subject to any of the class methods, so the Path is essentially just a wrapper.
Alternatively, I found that the mypy type checking can be satisfied by adding a type hint and appending "transport_*.json*" as a string:
# find optional files
files: list[str | Path] = []
for file in (
"settings.yaml",
"vasprun.xml",
"band_structure_data.json",
"wavefunction.h5",
"deformation.h5",
):
found_file = get_zfile(directory_listing, file, allow_missing=True)
if found_file is not None:
files.append(found_file)
files.append("transport_*.json*")
Either way, the wildcards are handled later in common.files.find_and_filter_files, as called by copy_files and gunzip_files.
I am happy to go with either solution.
copy_amset_files()currently looks for a file namedtransport.json. AMSET writes the transport files astransport_{mesh}.jsonby default. Because that file is never found,transport.prev.jsonis never created, socheck_converged()has nothing to compare against on a resubmission, leaving jobs resubmitting weather the interpolation factor is converged or not.Summary
transport.jsonwith the wildcard patterntransport_*.json*, matching the default file naming.rename_files()doesn't support wildcards, so the transport file is renamed with a plain pathlib glob + rename instead of going throughrename_files().allow_missing=Trueintocopy_files, replacing what was inrename_files, to avoid crashing on the first run when no previous file exists.