GitLabCIPython流水线怎么配置

本文目录导读：

1. 基本配置示例
2. 常用优化技巧
3. 配置文件示例 (requirements.txt)
4. 调试技巧

我来详细说明如何配置 GitLab CI/CD 的 Python 流水线。

基本配置示例

基础 Python 项目配置

# .gitlab-ci.yml
image: python:3.9
stages:
  - install
  - lint
  - test
  - build
variables:
  PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
cache:
  paths:
    - .cache/pip/
    - venv/
before_script:
  - python -V
  - pip install --upgrade pip
# 阶段1: 安装依赖
install-dependencies:
  stage: install
  script:
    - pip install -r requirements.txt
    - pip install -r requirements-dev.txt
  artifacts:
    paths:
      - venv/
    expire_in: 1 day
# 阶段2: 代码检查
lint:
  stage: lint
  script:
    - pip install flake8 black mypy
    - flake8 .
    - black --check .
    - mypy .
  needs:
    - install-dependencies
# 阶段3: 测试
test:
  stage: test
  script:
    - pip install pytest pytest-cov
    - pytest --cov=./ --cov-report=xml
  coverage: '/TOTAL.*\s+(\d+%)/'
  artifacts:
    reports:
      cobertura: coverage.xml
    paths:
      - coverage.xml
  needs:
    - install-dependencies
# 阶段4: 构建包
build:
  stage: build
  script:
    - pip install wheel
    - python setup.py bdist_wheel
  artifacts:
    paths:
      - dist/
  only:
    - tags

使用 Poetry 管理依赖

image: python:3.9
stages:
  - test
  - build
  - deploy
variables:
  POETRY_VERSION: "1.4.0"
  PYTHON_VERSION: "3.9"
before_script:
  - pip install "poetry==$POETRY_VERSION"
  - poetry config virtualenvs.create false
  - poetry install
cache:
  key: ${CI_COMMIT_REF_SLUG}
  paths:
    - .cache/pip/
    - ~/.cache/pypoetry/
test:
  stage: test
  script:
    - poetry run pytest --cov=./ --cov-report=html
  artifacts:
    paths:
      - htmlcov/
    expire_in: 30 days
build:
  stage: build
  script:
    - poetry build
  artifacts:
    paths:
      - dist/
  only:
    - tags
deploy:
  stage: deploy
  script:
    - poetry config pypi-token.pypi $PYPI_TOKEN
    - poetry publish
  only:
    - tags

多版本 Python 测试

image: python:3.9
stages:
  - test
.job_template: &python_test
  stage: test
  before_script:
    - python -V
    - pip install -r requirements.txt
    - pip install pytest
  script:
    - pytest
  cache:
    key: $CI_COMMIT_REF_SLUG-$CI_JOB_NAME
    paths:
      - .cache/pip/
python3.8:
  image: python:3.8
  <<: *python_test
python3.9:
  image: python:3.9
  <<: *python_test
python3.10:
  image: python:3.10
  <<: *python_test

Docker 部署的完整示例

image: docker:latest
stages:
  - test
  - build
  - deploy
services:
  - docker:dind
variables:
  DOCKER_DRIVER: overlay2
  DOCKER_TLS_CERTDIR: ""
  CI_REGISTRY: registry.gitlab.com
  CI_REGISTRY_IMAGE: $CI_REGISTRY/$CI_PROJECT_PATH
before_script:
  - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
unit-test:
  stage: test
  image: $CI_REGISTRY_IMAGE:test-$CI_COMMIT_SHORT_SHA
  before_script:
    - echo "Running tests..."
  script:
    - pip install -r requirements.txt
    - pip install pytest pytest-cov
    - pytest --cov=./ --cov-report=xml
  artifacts:
    reports:
      cobertura: coverage.xml
build-image:
  stage: build
  script:
    - docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA .
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
  only:
    - main
    - tags
deploy-staging:
  stage: deploy
  script:
    - docker pull $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
    - docker stop staging-app || true
    - docker rm staging-app || true
    - docker run -d --name staging-app -p 5000:5000 $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
  only:
    - main
deploy-production:
  stage: deploy
  script:
    - docker pull $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
    - docker tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA $CI_REGISTRY_IMAGE:latest
    - docker push $CI_REGISTRY_IMAGE:latest
  only:
    - tags

包含安全扫描的流水线

image: python:3.9
stages:
  - security
  - test
  - build
include:
  - template: Jobs/SAST.gitlab-ci.yml
  - template: Jobs/Secret-Detection.gitlab-ci.yml
security-audit:
  stage: security
  script:
    - pip install safety bandit
    - safety check -r requirements.txt
    - bandit -r .
dependency-check:
  stage: security
  script:
    - pip install pip-audit
    - pip-audit -r requirements.txt
test:
  stage: test
  script:
    - pip install pytest pytest-cov
    - pytest --cov=./ --cov-report=html
build:
  stage: build
  script:
    - python setup.py sdist bdist_wheel
  artifacts:
    paths:
      - dist/

常用优化技巧

缓存优化

cache:
  key: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG"
  paths:
    - .cache/pip/
    - venv/
    - node_modules/  # 如果有前端项目

并行执行任务

parallel:
  matrix:
    - PYTHON_VERSION: ["3.8", "3.9", "3.10"]
    - TEST_TYPE: ["unit", "integration"]

条件执行

job:
  script: echo "This runs only for main branch"
  only:
    - main
job2:
  script: echo "This runs for tags"
  only:
    - tags

环境变量保护

# 在 GitLab CI/CD Settings 中设置
# CI_REGISTRY_PASSWORD: xxx
# DEPLOY_KEY: xxx
# PYPI_TOKEN: xxx
# 在 CI 文件中使用
- echo $PYPI_TOKEN

配置文件示例 (requirements.txt)

fastapi==0.104.1
uvicorn==0.24.0
pydantic==2.5.0
sqlalchemy==2.0.23
redis==5.0.1

调试技巧

1. 查看日志：检查 GitLab CI Job 的详细日志
2. 本地测试：使用 gitlab-runner exec 测试
3. 增量调试：逐步添加阶段

这个流水线配置可以根据你的具体需求进行调整和扩展。