A handful of new commands, including copy-file, push, and create-sibling-ria, along with various fixes and enhancements Major refactoring and deprecations The no_annex parameter of create, which is exposed in the Python API but not the command line, is deprecated and will be removed in a later release. Use the new annex argument instead, flipping the value. Command-line callers that use --no-annex are unaffected. (#4321) datalad add, which was deprecated in 0.12.0, has been removed. (#4158) (#4319) The following GitRepo and AnnexRepo methods have been removed: get_changed_files, get_missing_files, and get_deleted_files. (#4169) (#4158) The get_branch_commits method of GitRepo and AnnexRepo has been renamed to get_branch_commits_. (#3834) The custom commit method of AnnexRepo has been removed, and AnnexRepo.commit now resolves to the parent method, GitRepo.commit. (#4168) GitPython's git.repo.base.Repo class is no longer available via the .repo attribute of GitRepo and AnnexRepo. (#4172) AnnexRepo.get_corresponding_branch now returns None rather than the current branch name when a managed branch is not checked out. (#4274) The special UUID for git-annex web remotes is now available as datalad.consts.WEB_SPECIAL_REMOTE_UUID. It remains accessible as AnnexRepo.WEB_UUID for compatibility, but new code should use consts.WEB_SPECIAL_REMOTE_UUID (#4460). Fixes Widespread improvements in functionality and test coverage on Windows and crippled file systems in general. (#4057) (#4245) (#4268) (#4276) (#4291) (#4296) (#4301) (#4303) (#4304) (#4305) (#4306) AnnexRepo.get_size_from_key incorrectly handled file chunks. (#4081) create-sibling would too readily clobber existing paths when called with --existing=replace. It now gets confirmation from the user before doing so if running interactively and unconditionally aborts when running non-interactively. (#4147) update (#4159) queried the incorrect branch configuration when updating non-annex repositories. didn't account for the fact that the local repository can be configured as the upstream "remote" for a branch. When the caller included --bare as a git init option, create crashed creating the bare repository, which is currently unsupported, rather than aborting with an informative error message. (#4065) The logic for automatically propagating the 'origin' remote when cloning a local source could unintentionally trigger a fetch of a non-local remote. (#4196) All remaining get_submodules() call sites that relied on the temporary compatibility layer added in v0.12.0 have been updated. (#4348) The custom result summary renderer for get, which was visible with --output-format=tailored, displayed incorrect and confusing information in some cases. The custom renderer has been removed entirely. (#4471) Enhancements and new features The default result renderer learned to elide a chain of results after seeing ten consecutive results that it considers similar, which improves the display of actions that have many results (e.g., saving hundreds of files). (#4337) The default result renderer, in addition to "tailored" result renderer, now triggers the custom summary renderer, if any. (#4338) The new command create-sibling-ria provides support for creating a sibling in a RIA store. (#4124) DataLad ships with a new special remote, git-annex-remote-ora, for interacting with RIA stores and a new command export-archive-ora for exporting an archive from a local annex object store. (#4260) (#4203) The new command push provides an alternative interface to publish for pushing a dataset hierarchy to a sibling. (#4206) The new command copy-file copies files and associated availability information from one dataset to another. (#4430) The command examples have been expanded and improved. (#4091) (#4314) (#4464) The tooling for linking to the DataLad Handbook from DataLad's documentation has been improved. (#4046) The --reckless parameter of clone and install learned two new modes: "ephemeral", where the .git/annex/ of the cloned repository is symlinked to the local source repository's. (#4099) "shared-{group|all|...}" that can be used to set up datasets for collaborative write access. (#4324) clone learned to handle dataset aliases in RIA stores when given a URL of the form ria+<protocol>://<storelocation>#~<aliasname>. (#4459) update (#4167) learned to disallow non-fast-forward updates when ff-only is given to the --merge option. gained a --follow option that controls how --merge behaves, adding support for merging in the revision that is registered in the parent dataset rather than merging in the configured branch from the sibling. now provides a result record for merge events. create-sibling now supports local paths as targets in addition to SSH URLs. (#4187) siblings now shows a warning if the caller requests to delete a sibling that does not exist. (#4257) phrases its warning about non-annex repositories in a less alarming way. (#4323) The rendering of command errors has been improved. (#4157) save now displays a message to signal that the working tree is clean, making it more obvious that no results being rendered corresponds to a clean state. (#4106) provides a stronger warning against using --to-git. (#4290) Calling diff without --recursive but with a path constraint within a subdataset ("<subdataset>/<path>") now traverses into the subdataset, as "<subdataset>/" would, restricting its report to "<subdataset>/<path>". (#4235) New option datalad.annex.retry controls how many times git-annex will retry on a failed transfer. It defaults to 3 and can be set to 0 to restore the previous behavior. (#4382) wtf now warns when the specified dataset does not exist. (#4331) The repr and str output of the dataset and repo classes got a facelift. (#4420) (#4435) (#4439) The DataLad Singularity container now comes with p7zip-full. DataLad shows a log message when the current working is resolved to a different location due to a symlink. This is now logged at the DEBUG rather than WARNING level, as it typically does not indicate a problem. (#4426) DataLad now lets the caller know that git annex init is scanning for unlocked files, as this operation can be slow in some repositories. (#4316) The log_progress helper learned how to set the starting point to a non-zero value and how to update the total of an existing progress bar, two features needed for planned improvements to how some commands display their progress. (#4438) The ExternalVersions object, which is used to check versions of Python modules and external tools (e.g., git-annex), gained an add method that enables DataLad extensions and other third-party code to include other programs of interest. (#4441) All of the remaining spots that use GitPython have been rewritten without it. Most notably, this includes rewrites of the clone, fetch, and push methods of GitRepo. (#4080) (#4087) (#4170) (#4171) (#4175) (#4172) When GitRepo.commit splits its operation across multiple calls to avoid exceeding the maximum command line length, it now amends to initial commit rather than creating multiple commits. (#4156) GitRepo gained a get_corresponding_branch method (which always returns None), allowing a caller to invoke the method without needing to check if the underlying repo class is GitRepo or AnnexRepo. (#4274) A new helper function datalad.core.local.repo.repo_from_path returns a repo class for a specified path. (#4273) New AnnexRepo method localsync performs a git annex sync that disables external interaction and is particularly useful for propagating changes on an adjusted branch back to the main branch. (#4243)