PyPI packages in the Package Registry all tiers
- Introduced in GitLab 12.10.
- Moved from GitLab Premium to GitLab Free in 13.3.
Publish PyPI packages in your project’s Package Registry. Then install the packages whenever you need to use them as a dependency.
The Package Registry works with:
For documentation of the specific API endpoints that the pip
and twine
clients use, see the PyPI API documentation.
Learn how to build a PyPI package.
Authenticate with the Package Registry
Before you can publish to the Package Registry, you must authenticate.
To do this, you can use:
- A personal access token
with the scope set to
api
. - A deploy token with the scope set to
read_package_registry
,write_package_registry
, or both. - A CI job token.
Authenticate with a personal access token
To authenticate with a personal access token, edit the ~/.pypirc
file and add:
[distutils]
index-servers =
gitlab
[gitlab]
repository = https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi
username = <your_personal_access_token_name>
password = <your_personal_access_token>
The <project_id>
is either the project’s
URL-encoded
path (for example, group%2Fproject
), or the project’s ID (for example 42
).
Authenticate with a deploy token
To authenticate with a deploy token, edit your ~/.pypirc
file and add:
[distutils]
index-servers =
gitlab
[gitlab]
repository = https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi
username = <deploy token username>
password = <deploy token>
The <project_id>
is either the project’s
URL-encoded
path (for example, group%2Fproject
), or the project’s ID (for example 42
).
Authenticate with a CI job token
Introduced in GitLab 13.4.
To work with PyPI commands within GitLab CI/CD, you
can use CI_JOB_TOKEN
instead of a personal access token or deploy token.
For example:
image: python:latest
run:
script:
- pip install build twine
- python -m build
- TWINE_PASSWORD=${CI_JOB_TOKEN} TWINE_USERNAME=gitlab-ci-token python -m twine upload --repository-url ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/pypi dist/*
You can also use CI_JOB_TOKEN
in a ~/.pypirc
file that you check in to
GitLab:
[distutils]
index-servers =
gitlab
[gitlab]
repository = https://gitlab.example.com/api/v4/projects/${env.CI_PROJECT_ID}/packages/pypi
username = gitlab-ci-token
password = ${env.CI_JOB_TOKEN}
Authenticate to access packages within a group
Follow the instructions above for the token type, but use the group URL in place of the project URL:
https://gitlab.example.com/api/v4/groups/<group_id>/-/packages/pypi
Publish a PyPI package
Prerequisites:
- You must authenticate with the Package Registry.
- Your version string must be valid.
- The maximum allowed package size is 5 GB.
- You can’t upload the same version of a package multiple times. If you try,
you receive the error
400 Bad Request
. - PyPI packages are published using your projectID.
- If your project is in a group, PyPI packages published to your project registry are also available at the group-level registry (see Install from the group level).
You can then publish a package by using twine.
Ensure your version string is valid
If your version string (for example, 0.0.1
) isn’t valid, it gets rejected.
GitLab uses the following regex to validate the version string.
\A(?:
v?
(?:([0-9]+)!)? (?# epoch)
([0-9]+(?:\.[0-9]+)*) (?# release segment)
([-_\.]?((a|b|c|rc|alpha|beta|pre|preview))[-_\.]?([0-9]+)?)? (?# pre-release)
((?:-([0-9]+))|(?:[-_\.]?(post|rev|r)[-_\.]?([0-9]+)?))? (?# post release)
([-_\.]?(dev)[-_\.]?([0-9]+)?)? (?# dev release)
(?:\+([a-z0-9]+(?:[-_\.][a-z0-9]+)*))? (?# local version)
)\z}xi
You can experiment with the regex and try your version strings by using this regular expression editor.
For more details about the regex, review this documentation.
Publish a PyPI package by using twine
To publish a PyPI package, run a command like:
python3 -m twine upload --repository gitlab dist/*
This message indicates that the package was published successfully:
Uploading distributions to https://gitlab.example.com/api/v4/projects/<your_project_id>/packages/pypi
Uploading mypypipackage-0.0.1-py3-none-any.whl
100%|███████████████████████████████████████████████████████████████████████████████████████████| 4.58k/4.58k [00:00<00:00, 10.9kB/s]
Uploading mypypipackage-0.0.1.tar.gz
100%|███████████████████████████████████████████████████████████████████████████████████████████| 4.24k/4.24k [00:00<00:00, 11.0kB/s]
To view the published package, go to your project’s Packages and registries page.
If you didn’t use a .pypirc
file to define your repository source, you can
publish to the repository with the authentication inline:
TWINE_PASSWORD=<personal_access_token or deploy_token> TWINE_USERNAME=<username or deploy_token_username> python3 -m twine upload --repository-url https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi dist/*
If you didn’t follow the steps on this page, ensure your package was properly
built, and that you created a PyPI package with setuptools
.
You can then upload your package by using the following command:
python -m twine upload --repository <source_name> dist/<package_file>
-
<package_file>
is your package filename, ending in.tar.gz
or.whl
. -
<source_name>
is the source name used during setup.
Publishing packages with the same name or version
You cannot publish a package if a package of the same name and version already exists.
You must delete the existing package first.
If you attempt to publish the same package
more than once, a 400 Bad Request
error occurs.
Install a PyPI package
In GitLab 14.2 and later, when a PyPI package is not found in the Package Registry, the request is forwarded to pypi.org.
Administrators can disable this behavior in the Continuous Integration settings.
--index-url
option, do not specify the port if it is a default
port, such as 80
for a URL starting with http
or 443
for a URL starting
with https
.Install from the project level
To install the latest version of a package, use the following command:
pip install --index-url https://<personal_access_token_name>:<personal_access_token>@gitlab.example.com/api/v4/projects/<project_id>/packages/pypi/simple --no-deps <package_name>
-
<package_name>
is the package name. -
<personal_access_token_name>
is a personal access token name with theread_api
scope. -
<personal_access_token>
is a personal access token with theread_api
scope. -
<project_id>
is either the project’s URL-encoded path (for example,group%2Fproject
), or the project’s ID (for example42
).
In these commands, you can use --extra-index-url
instead of --index-url
. However, using
--extra-index-url
makes you vulnerable to dependency confusion attacks because it checks the PyPi
repository for the package before it checks the custom repository. --extra-index-url
adds the
provided URL as an additional registry which the client checks if the package is present.
--index-url
tells the client to check for the package on the provided URL only.
If you were following the guide and want to install the
MyPyPiPackage
package, you can run:
pip install mypypipackage --no-deps --index-url https://<personal_access_token_name>:<personal_access_token>@gitlab.example.com/api/v4/projects/<your_project_id>/packages/pypi/simple
This message indicates that the package was installed successfully:
Looking in indexes: https://<personal_access_token_name>:****@gitlab.example.com/api/v4/projects/<your_project_id>/packages/pypi/simple
Collecting mypypipackage
Downloading https://gitlab.example.com/api/v4/projects/<your_project_id>/packages/pypi/files/d53334205552a355fee8ca35a164512ef7334f33d309e60240d57073ee4386e6/mypypipackage-0.0.1-py3-none-any.whl (1.6 kB)
Installing collected packages: mypypipackage
Successfully installed mypypipackage-0.0.1
Install from the group level
To install the latest version of a package from a group, use the following command:
pip install --index-url https://<personal_access_token_name>:<personal_access_token>@gitlab.example.com/api/v4/groups/<group_id>/-/packages/pypi/simple --no-deps <package_name>
In this command:
-
<package_name>
is the package name. -
<personal_access_token_name>
is a personal access token name with theread_api
scope. -
<personal_access_token>
is a personal access token with theread_api
scope. -
<group_id>
is the group ID.
In these commands, you can use --extra-index-url
instead of --index-url
. However, using
--extra-index-url
makes you vulnerable to dependency confusion attacks because it checks the PyPi
repository for the package before it checks the custom repository. --extra-index-url
adds the
provided URL as an additional registry which the client checks if the package is present.
--index-url
tells the client to check for the package at the provided URL only.
If you’re following the guide and want to install the MyPyPiPackage
package, you can run:
pip install mypypipackage --no-deps --index-url https://<personal_access_token_name>:<personal_access_token>@gitlab.example.com/api/v4/groups/<your_group_id>/-/packages/pypi/simple
Package names
GitLab looks for packages that use
Python normalized names (PEP-503).
The characters -
, _
, and .
are all treated the same, and repeated
characters are removed.
A pip install
request for my.package
looks for packages that match any of
the three characters, such as my-package
, my_package
, and my....package
.
Using requirements.txt
If you want pip to access your private registry, add the --extra-index-url
parameter along with the URL for your registry to your requirements.txt
file.
--extra-index-url https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi/simple
package-name==1.0.0
If this is a private registry, you can authenticate in a couple of ways. For example:
- Using your
requirements.txt
file:
--extra-index-url https://__token__:<your_personal_token>@gitlab.example.com/api/v4/projects/<project_id>/packages/pypi/simple
package-name==1.0.0
- Using a
~/.netrc
file:
machine gitlab.example.com
login __token__
password <your_personal_token>
Troubleshooting
To improve performance, the pip command caches files related to a package. Pip doesn’t remove data by itself. The cache grows as new packages are installed. If you encounter issues, clear the cache with this command:
pip cache purge
Multiple index-url
or extra-index-url
parameters
You can define multiple index-url
and extra-index-url
parameters.
If you use the same domain name (such as gitlab.example.com
) multiple times with different authentication
tokens, pip
may not be able to find your packages. This problem is due to how pip
registers and stores your tokens during commands executions.
To workaround this issue, you can use a group deploy token with the
scope read_package_registry
from a common parent group for all projects or groups targeted by the
index-url
and extra-index-url
values.
Supported CLI commands
The GitLab PyPI repository supports the following CLI commands:
-
twine upload
: Upload a package to the registry. -
pip install
: Install a PyPI package from the registry.