# Using CodeArtifact with Python These topics describe how to use `pip`, the Python package manager, and `twine`, the Python package publishing utility, with CodeArtifact. **Topics** + [Configure and use pip with CodeArtifact](python-configure-pip.md) + [Configure and use twine with CodeArtifact](python-configure-twine.md) + [Python package name normalization](python-name-normalization.md) + [Python compatibility](python-compatibility.md) + [Requesting Python packages from upstreams and external connections](python-upstream-external-connections-request.md) # Configure and use pip with CodeArtifact [pip](https://pypi.org/project/pip/) is the package installer for Python packages. To use pip to install Python packages from your CodeArtifact repository, you must first configure the pip client with your CodeArtifact repository information and credentials. pip can only be used to install Python packages. To publish Python packages, you can use [twine](https://pypi.org/project/twine/). For more information, see [Configure and use twine with CodeArtifact](python-configure-twine.md). ## Configure pip with the `login` command First, configure your AWS credentials for use with the AWS CLI, as described in [Getting started with CodeArtifact](getting-started.md). Then, use the CodeArtifact `login` command to fetch credentials and configure `pip` with them. **Note** If you are accessing a repository in a domain that you own, you don't need to include `--domain-owner`. For more information, see [Cross-account domains](domain-overview.md#domain-overview-cross-account). To configure pip, run the following command. ``` aws codeartifact login --tool pip --domain my_domain --domain-owner 111122223333 --repository my_repo ``` `login` fetches an authorization token from CodeArtifact using your AWS credentials. The `login` command will configure `pip` for use with CodeArtifact by editing `~/.config/pip/pip.conf` to set the `index-url` to the repository specified by the `--repository` option. The default authorization period after calling `login` is 12 hours, and `login` must be called to periodically refresh the token. For more information about the authorization token created with the `login` command, see [Tokens created with the `login` command](tokens-authentication.md#auth-token-login). ## Configure pip without the login command If you cannot use the `login` command to configure `pip`, you can use `pip config`. 1. Use the AWS CLI to fetch a new authorization token. **Note** If you are accessing a repository in a domain that you own, you do not need to include the `--domain-owner`. For more information, see [Cross-account domains](domain-overview.md#domain-overview-cross-account). ``` CODEARTIFACT_AUTH_TOKEN=`aws codeartifact get-authorization-token --domain my_domain --domain-owner 111122223333 --query authorizationToken --output text` ``` 1. Use `pip config` to set the CodeArtifact registry URL and credentials. The following command will update the current environment configuration file only. To update the system-wide configuration file, replace `site` with `global`. ``` pip config set site.index-url https://aws:$CODEARTIFACT_AUTH_TOKEN@my_domain-111122223333.d.codeartifact.region.amazonaws.com/pypi/my_repo/simple/ ``` **Note** To use a dualstack endpoint, use the `codeartifact.region.on.aws` endpoint. **Important** The registry URL must end with a forward slash (/). Otherwise, you cannot connect to the repository. **Example pip configuration file** The following is an example of a `pip.conf` file after setting the CodeArtifact registry URL and credentials. ``` [global] index-url = https://aws:eyJ2ZX...@my_domain-111122223333.d.codeartifact.us-west-2.amazonaws.com/pypi/my_repo/simple/ ``` ## Run pip To run `pip` commands, you must configure `pip` with CodeArtifact. For more information, see the following documentation. 1. Follow the steps in the [Setting up with AWS CodeArtifact](get-set-up-for-codeartifact.md) section to configure your AWS account, tools, and permissions. 1. Configure `twine` by following the steps in [Configure and use twine with CodeArtifact](python-configure-twine.md). Assuming that a package is present in your repository or one of its upstream repositories, you can install it with `pip install`. For example, use the following command to install the `requests` package. ``` pip install requests ``` Use the `-i` option to temporarily revert to installing packages from [https://pypi.org](https://pypi.org) instead of your CodeArtifact repository. ``` pip install -i https://pypi.org/simple requests ``` # Configure and use twine with CodeArtifact [twine](https://pypi.org/project/twine/) is a package publishing utility for Python packages. To use twine to publish Python packages to your CodeArtifact repository, you must first configure twine with your CodeArtifact repository information and credentials. twine can only be used to publish Python packages. To install Python packages, you can use [pip](https://pypi.org/project/pip/). For more information, see [Configure and use pip with CodeArtifact](python-configure-pip.md). ## Configure twine with the `login` command First, configure your AWS credentials for use with the AWS CLI, as described in [Getting started with CodeArtifact](getting-started.md). Then, use the CodeArtifact `login` command to fetch credentials and configure twine with them. **Note** If you are accessing a repository in a domain that you own, you don't need to include `--domain-owner`. For more information, see [Cross-account domains](domain-overview.md#domain-overview-cross-account). To configure twine, run the following command. ``` aws codeartifact login --tool twine --domain my_domain --domain-owner 111122223333 --repository my_repo ``` `login` fetches an authorization token from CodeArtifact using your AWS credentials. The `login` command configures twine for use with CodeArtifact by editing `~/.pypirc` to add the repository specified by the `--repository` option with credentials. The default authorization period after calling `login` is 12 hours, and `login` must be called to periodically refresh the token. For more information about the authorization token created with the `login` command, see [Tokens created with the `login` command](tokens-authentication.md#auth-token-login). ## Configure twine without the `login` command If you cannot use the `login` command to configure twine, you can use the `~/.pypirc` file or environment variables. To use the `~/.pypirc` file, add the following entries to it. The password must be an auth token acquired by the `get-authorization-token` API. ``` [distutils] index-servers = codeartifact [codeartifact] repository = https://my_domain-111122223333.d.codeartifact.us-west-2.amazonaws.com/pypi/my_repo/ password = auth-token username = aws ``` **Note** To use a dualstack endpoint, use the `codeartifact.region.on.aws` endpoint. To use environment variables, do the following. **Note** If you are accessing a repository in a domain that you own, you do not need to include the `--domain-owner`. For more information, see [Cross-account domains](domain-overview.md#domain-overview-cross-account). ``` export TWINE_USERNAME=aws export TWINE_PASSWORD=`aws codeartifact get-authorization-token --domain my_domain --domain-owner 111122223333 --query authorizationToken --output text` export TWINE_REPOSITORY_URL=`aws codeartifact get-repository-endpoint --domain my_domain --domain-owner 111122223333 --repository my_repo --format pypi --query repositoryEndpoint --output text` ``` ## Run twine Before using twine to publish Python package assets, you must first configure CodeArtifact permissions and resources. 1. Follow the steps in the [Setting up with AWS CodeArtifact](get-set-up-for-codeartifact.md) section to configure your AWS account, tools, and permissions. 1. Configure twine by following the steps in [Configure twine with the `login` command](#python-configure-twine-login) or [Configure twine without the `login` command](#python-configure-twine-without-login). After you configure twine, you can run `twine` commands. Use the following command to publish Python package assets. ``` twine upload --repository codeartifact mypackage-1.0.tgz ``` For information about how to build and package your Python application, see [Generating Distribution Archives](https://packaging.python.org/tutorials/packaging-projects/#generating-distribution-archives) on the Python Packaging Authority website. # Python package name normalization CodeArtifact normalizes package names before storing them, which means the package names in CodeArtifact may be different than the name provided when the package was published. For Python packages, when performing normalization the package name is lowercased and all instances of the characters `.`, `-`, and `_` are replaced with a single `-` character. So the package names `pigeon_cli` and `pigeon.cli` are normalized and stored as `pigeon-cli`. The non-normalized name can be used by pip and twine but the normalized name must be used in CodeArtifact CLI or API requests (such as `list-package-versions`) and in ARNs. For more information about Python package name normalization, see [PEP 503](https://www.python.org/dev/peps/pep-0503/#normalized-names) in the Python documentation. # Python compatibility CodeArtifact does not support PyPI's `XML-RPC` or `JSON` APIs. CodeArtifact supports PyPI's `Legacy` APIs, except the `simple` API. While CodeArtifact does not support the `/simple/` API endpoint, it does support the `/simple//` endpoint. For more information, see the following on the Python Packaging Authority's GitHub repository. + [XML-RPC API](https://github.com/pypi/warehouse/blob/main/docs/dev/api-reference/xml-rpc.rst) + [JSON API](https://github.com/pypi/warehouse/blob/main/docs/dev/api-reference/json.rst) + [Legacy API](https://github.com/pypi/warehouse/blob/main/docs/dev/api-reference/legacy.rst) ## pip command support The following sections summarize the pip commands that are supported, by CodeArtifact repositories, in addition to specific commands that are not supported. **Topics** + [Supported commands that interact with a repository](#supported-pip-commands-that-interact-with-a-repository) + [Supported client-side commands](#supported-pip-client-side-commands) ### Supported commands that interact with a repository This section lists `pip` commands where the `pip` client makes one or more requests to the registry it's been configured with. These commands have been verified to function correctly when invoked against a CodeArtifact repository. **** | Command | Description | | --- | --- | | [install](https://pip.pypa.io/en/stable/reference/pip_install/) | Install packages. | | [download](https://pip.pypa.io/en/stable/reference/pip_download/) | Download packages. | CodeArtifact does not implement `pip search`. If you have configured `pip` with a CodeArtifact repository, running `pip search` will search and show packages from [PyPI](https://pypi.org/). ### Supported client-side commands These commands don't require any direct interaction with a repository, so CodeArtifact does not need to do anything to support them. **** | Command | Description | | --- | --- | | [uninstall](https://pip.pypa.io/en/stable/reference/pip_uninstall/) | Uninstall packages. | | [freeze](https://pip.pypa.io/en/stable/reference/pip_freeze/) | Output installed packages in requirements format. | | [list](https://pip.pypa.io/en/stable/reference/pip_list/) | List installed packages. | | [show](https://pip.pypa.io/en/stable/reference/pip_show/) | Show information about installed packages. | | [check](https://pip.pypa.io/en/stable/reference/pip_check/) | Verify installed packages have compatible dependencies. | | [config](https://pip.pypa.io/en/stable/reference/pip_config/) | Manage local and global configuration. | | [wheel](https://pip.pypa.io/en/stable/reference/pip_wheel/) | Build wheels from your requirements. | | [hash](https://pip.pypa.io/en/stable/reference/pip_hash/) | Compute hashes of package archives. | | [completion](https://pip.pypa.io/en/stable/user_guide/#command-completion) | Helps with command completion. | | [debug](https://pip.pypa.io/en/stable/reference/pip_debug/) | Show information useful for debugging. | | help | Show help for commands. | # Requesting Python packages from upstreams and external connections When importing a Python package version from [pypi.org](https://pypi.org/), CodeArtifact will import all the assets in that package version. While most Python packages contain a small number of assets, some contain over 100 assets, typically to support multiple hardware architectures and Python interpreters. It’s common for new assets to be published to pypi.org for an existing package version. For example, some projects publish new assets when new versions of Python are released. When a Python package is installed from CodeArtifact with `pip install`, package versions retained in the CodeArtifact repository are updated to reflect the latest set of assets from pypi.org. Similarly, if new assets are available for a package version in an upstream CodeArtifact repository that are not present in the current CodeArtifact repository, they will be retained in the current repository when `pip install` is run. ## Yanked package versions Some package versions in pypi.org are marked as *yanked*, which communicates to the package installer (such as pip) that the version should not be installed unless it is the only one that matches a version specifier (using either `==` or `===`). See [PEP\$1592](https://peps.python.org/pep-0592/) for more information. If a package version in CodeArtifact was originally fetched from an external connection to [pypi.org](https://pypi.org/), when you install the package version from a CodeArtifact repository, CodeArtifact ensures that the updated yanked metadata of the package version is fetched from pypi.org. ### How to know if a package version is yanked To check if a package version is yanked in CodeArtifact, you can attempt to install it with `pip install packageName===packageVersion`. If the package version is yanked, you will receive a warning message similar to the following: ``` WARNING: The candidate selected for download or install is a yanked version ``` To check if a package version is yanked in [pypi.org](https://pypi.org/), you can visit the package version's pypi.org listing at `https://pypi.org/project/packageName/packageVersion/`. ### Setting yanked status on private packages CodeArtifact does not support setting yanked metadata for packages published directly to CodeArtifact repositories. ## Why is CodeArtifact not fetching the latest yanked metadata or assets for a package version? Normally, CodeArtifact ensures that when a Python package version is fetched from a CodeArtifact repository, the yanked metadata is up-to-date with the latest value on [pypi.org](https://pypi.org/). Additionally, the list of assets in the package version are also kept updated with the latest set on pypi.org and any upstream CodeArtifact repositories. This is true whether you’re installing the package version for the first time and CodeArtifact imports it from pypi.org into your CodeArtifact repository, or if you've installed the package before. However, there are cases when the package manager client, such as pip, won’t pull the latest yanked metadata from pypi.org or upstream repositories. Instead, CodeArtifact will return the data that is already stored in your repository. This section describes the three ways this can occur: **Upstream configuration:** If the external connection to pypi.org is removed from the repository or its upstreams using [disassociate-external-connection](external-connection.md#removing-an-external-connection), yanked metadata will no longer be refreshed from pypi.org. Similarly, if you remove an upstream repository, assets from the removed repository and the removed repository’s upstreams will no longer be available to the current repository. The same is true if you use CodeArtifact [package origin controls](package-origin-controls.md) to prevent new versions of a specific package from being pulled— setting `upstream=BLOCK` will block yanked metadata from being refreshed. **Package version status:** If you set the status of a package version to anything except `Published` or `Unlisted`, yanked metadata and assets of the package version will not be refreshed. Similarly, if you are fetching a specific package version (say `torch 2.0.1`) and the same package version is present in an upstream repository with a status that isn’t `Published` or `Unlisted`, this will also block yanked metadata and asset propagation from the upstream repository to the current repository. This is because other package version statuses are an indication that the versions are not meant to be consumed anymore in any repository. **Direct publishing:** If you publish a specific package version directly into a CodeArtifact repository, this will prevent yanked metadata and asset refresh for the package version from its upstream repositories and pypi.org. For example, say you download an asset from the package version `torch 2.0.1`, such as `torch-2.0.1-cp311-none-macosx_11_0_arm64.whl`, using a web browser and then publish this to your CodeArtifact repository using twine as `torch 2.0.1`. CodeArtifact tracks that the package version entered the domain by direct publishing to your repository, not from an external connection to pypi.org or an upstream repository. In this case, CodeArtifact does not keep the yanked metadata in sync with upstream repositories or pypi.org. The same is true if you publish `torch 2.0.1` into an upstream repository— the presence of the package version will block propagation of yanked metadata and assets to repositories further down the upstream graph.