Software repository
A software repository, or repo for short, is a storage location for software packages. Often a table of contents is also stored, along with metadata. A software repository is typically managed by source or version control, or repository managers. Package managers allow automatically installing and updating repositories, sometimes called "packages".
Overview
[edit]Many software publishers and other organizations maintain servers on the Internet for this purpose, either free of charge or for a subscription fee. Repositories may be solely for particular programs, such as CPAN for the Perl programming language, or for an entire operating system. Operators of such repositories typically provide a package management system, tools intended to search for, install and otherwise manipulate software packages from the repositories. For example, many Linux distributions use Advanced Packaging Tool (APT), commonly found in Debian based distributions, or Yellowdog Updater, Modified (yum) found in Red Hat based distributions. There are also multiple independent package management systems, such as pacman, used in Arch Linux and equo, found in Sabayon Linux.
As software repositories are designed to include useful packages, major repositories are designed to be malware free. If a computer is configured to use a digitally signed repository from a reputable vendor, and is coupled with an appropriate permissions system, this significantly reduces the threat of malware to these systems. As a side effect, many systems that have these abilities do not need anti-malware software such as antivirus software.[1]
Most major Linux distributions have many repositories around the world that mirror the main repository.
At client side, a package manager helps installing from and updating the repositories.
Package management system vs. package development process
[edit]A package management system is different from a package development process.
A typical use of a package management system is to facilitate the integration of code from possibly different sources into a coherent stand-alone operating unit. Thus, a package management system might be used to produce a distribution of Linux, possibly a distribution tailored to a specific restricted application.
A package development process, by contrast, is used to manage the co-development of code and documentation of a collection of functions or routines with a common theme, producing thereby a package of software functions that typically will not be complete and usable by themselves. A good package development process will help users conform to good documentation and coding practices, integrating some level of unit testing.
Selected repositories
[edit]The following table lists a few languages with repositories for contributed software. The "Autochecks" column describes the routine checks done.
| Language, purpose | Package development process | Repository | Install methods |
|---|---|---|---|
| C/C++ | vcpkg | ||
| Conan | |||
| Common Lisp | Quicklisp[2] | ||
| D | DUB | dlang.org | dub add <package> |
| Dart | Flutter | pub.dev | flutter pub get <package> |
| Fortran | Fortran Package Manager (fpm) | ||
| Go | go | pkg.go.dev | go get <package> |
| Haskell | Common Architecture for Building Applications and Libraries[3] | Hackage | Cabal |
| Java, Kotlin, Scala, Groovy, Clojure, etc. | Maven[4] | Maven, Apache Ivy, Gradle, sbt | |
| JavaScript, TypeScript, Node.js | node | npm registry[5] | npm install <package> yarn add <package> |
| Bower | bower install <package> | ||
| Julia[6] | |||
| .NET | NuGet | NuGet[7] | dotnet add package <package> |
| Ocaml | OPAM | ||
| Perl | CPAN | PPM[8] | |
| PHP | PEAR, Composer | PECL, Packagist | composer require <package>
pear install <package> |
| PowerShell | PowerShell Gallery | PSResourceGet | |
| Python | Setuptools, Poetry[9] | PyPI | pip, EasyInstall, PyPM, Poetry, uv |
| Conda | |||
| R | R CMD check process[10][11] | CRAN[12] | install.packages[13] remotes[14] |
| Ruby | RubyGems | RubyGems[15] | RubyGems,[15] Bundler[16] |
| Rust | Cargo[17] | crates.io[18] | Cargo[17] |
| Swift | Swift Package Manager | ||
| TeX, LaTeX | CTAN |
(Parts of this table were copied from a "List of Top Repositories by Programming Language" on Stack Overflow[19])
Notable repositories with limited scope include:
Package managers
[edit]Package managers help manage repositories and the distribution of them. If a repository is updated, a package manager will usually allow a user to update that repository through the manager. It also helps to manage things such as dependencies between other repositories. Examples of managers include:
| Package manager | Description |
|---|---|
| npm | Manager for Node.js[20] |
| pip | Installer for Python[21] |
| APT | Manager for Debian[22] |
| Homebrew | Installer for macOS, allows installing packages that Apple did not[23] |
| vcpkg | Manager for C, C++[24][25] |
| yum and DNF | Manager for Fedora, Red Hat Enterprise Linux[26] |
| Pacman | Manager for Arch Linux[27] |
| Composer | Manager for PHP |
Repository managers
[edit]In an enterprise environment, a software repository is usually used to store artifacts, or to mirror external repositories which may be inaccessible due to security restrictions. Such repositories may provide additional functionality, like access control, versioning, security checks for uploaded software, cluster functionality etc. and typically support a variety of formats in one package, so as to cater for all the needs in an enterprise, and thus aiming to provide a single point of truth. One example is Sonatype Nexus Repository.[28]
At server side, a software repository is typically managed by source control or repository managers. Some of the repository managers allow to aggregate other repository location into one URL and provide a caching proxy. When doing continuous builds many artifacts are produced and often centrally stored, so automatically deleting the ones which are not released is important.
Relationship to continuous integration
[edit]As part of the development lifecycle, source code is continuously being built into binary artifacts using continuous integration. This may interact with a binary repository manager much like a developer would by getting artifacts from the repositories and pushing builds there. Tight integration with CI servers enables the storage of important metadata such as:
- Which user triggered the build (whether manually or by committing to revision control)
- Which modules were built
- Which sources were used (commit id, revision, branch)
- Dependencies used
- Environment variables
- Packages installed
Artifacts and packages
[edit]Artifacts and packages inherently mean different things. Artifacts are simply an output or collection of files (ex. JAR, WAR, DLLS, RPM etc.) and one of those files may contain metadata (e.g. POM file). Whereas packages are a single archive file in a well-defined format (ex. NuGet) that contain files appropriate for the package type (ex. DLL, PDB).[29] Many artifacts result from builds but other types are crucial as well. Packages are essentially one of two things: a library or an application.[30]
Compared to source files, binary artifacts are often larger by orders of magnitude, they are rarely deleted or overwritten (except for rare cases such as snapshots or nightly builds), and they are usually accompanied by much metadata such as id, package name, version, license and more.
Metadata
[edit]Metadata describes a binary artifact, is stored and specified separately from the artifact itself, and can have several additional uses. The following table shows some common metadata types and their uses:
| Metadata type | Used for |
|---|---|
| Versions available | Upgrading and downgrading automatically |
| Dependencies | Specify other artifacts that the current artifact depends on |
| Downstream dependencies | Specify other artifacts that depend on the current artifact |
| License | Legal compliance |
| Build date and time | Traceability |
| Documentation | Provide offline availability for contextual documentation in IDEs |
| Approval information | Traceability |
| Metrics | Code coverage, compliance to rules, test results |
| User-created metadata | Custom reports and processes |
See also
[edit]References
[edit]- ↑ itmWEB: Coping with Computer Viruses Archived October 14, 2007, at the Wayback Machine
- ↑ "Quicklisp beta". www.quicklisp.org. Archived from the original on 2019-03-23. Retrieved 2019-03-25.
- ↑ "The Haskell Cabal | Overview". www.haskell.org. Archived from the original on 2019-04-10. Retrieved 2019-03-25.
- ↑ "Maven – Welcome to Apache Maven". maven.apache.org. Archived from the original on 2011-07-24. Retrieved 2019-03-25.
- ↑ "npm". www.npmjs.com. Archived from the original on 2018-04-13. Retrieved 2019-03-25.
- ↑ "Julia Package Listing". pkg.julialang.org. Archived from the original on 2019-01-20. Retrieved 2019-03-25.
- ↑ karann-msft. "NuGet Package Manager UI Reference". docs.microsoft.com. Archived from the original on 2019-03-25. Retrieved 2019-03-25.
- ↑ "Installing Perl Modules - www.cpan.org". www.cpan.org. Archived from the original on 2019-03-14. Retrieved 2019-03-25.
- ↑ "Poetry". python-poetry.org. Archived from the original on 2024-05-22. Retrieved 2024-05-22.
- ↑ Leisch, Friedrich. "Creating R Packages: A Tutorial" (PDF). Archived (PDF) from the original on 2017-12-09. Retrieved 2016-07-19.
- ↑ Graves, Spencer B.; Dorai-Raj, Sundar. "Creating R Packages, Using CRAN, R-Forge, And Local R Archive Networks And Subversion (SVN) Repositories" (PDF). Archived (PDF) from the original on 2017-07-05. Retrieved 2016-07-19.
- ↑ "The Comprehensive R Archive Network". cran.r-project.org. Archived from the original on 2019-01-23. Retrieved 2019-03-25.
- ↑ "R Installation and Administration". cran.r-project.org. Archived from the original on 2015-11-23. Retrieved 2019-03-25.
- ↑ Wickham, Hadley; Bryan, Jenny. "Package structure and state". R Packages. O'Reilly. Archived from the original on 2020-11-09. Retrieved 2020-11-20.
- 1 2 "RubyGems.org your community gem host". rubygems. Archived from the original on 2019-02-13. Retrieved 2022-02-03.
- ↑ "Bundler: The best way to manage a Ruby application's gems". bundler.io. Archived from the original on 2022-01-29. Retrieved 2022-02-03.
- 1 2 "The Cargo Book". Documentation. Rust Programming Language. Archived from the original on 2019-04-28. Retrieved 2019-08-26.
- ↑ "Rust Package Registry". crates.io. Archived from the original on 2019-08-28. Retrieved 2019-08-26.
- ↑ "List of Top Repositories by Programming Language". Stack Overflow. Archived from the original on 2018-12-26. Retrieved 2010-04-14.
- ↑ "npm About". www.npmjs.com. Archived from the original on 2019-11-19. Retrieved 2019-11-21.
- ↑ developers, The pip, pip: The PyPA recommended tool for installing Python packages., archived from the original on 2020-07-14, retrieved 2019-11-21
- ↑ "Apt - Debian Wiki". wiki.debian.org. Archived from the original on 2019-10-19. Retrieved 2019-11-22.
- ↑ "Homebrew". Homebrew. Archived from the original on 2022-10-05. Retrieved 2019-11-22.
- ↑ "Yelp launches Yelp Fusion, Microsoft creates Vcpkg tool, and the new Touch Sense SDK for Android developers". SD Times. September 20, 2016. Archived from the original on November 27, 2020. Retrieved November 19, 2020.
- ↑ "Microsoft's C++ library manager now available for Linux and macOS". SD Times. April 25, 2018. Archived from the original on September 22, 2020. Retrieved November 19, 2020.
- ↑ Chinthaguntla, Keerthi (22 April 2020). "Linux package management with YUM and RPM". Enable Sysadmin. Archived from the original on 2021-04-11. Retrieved 2021-04-11.
- ↑ "pacman - ArchWiki". wiki.archlinux.org. Archived from the original on 2017-08-18. Retrieved 2021-04-11.
- ↑ "Nexus Repository | Software Component Management". Archived from the original on 2021-04-25. Retrieved 2021-04-25.
- ↑ "Linux repository classification schemes". braintickle.blogspot.com. 13 January 2006. Archived from the original on 2007-10-11. Retrieved 2008-03-01.
