Python site-packages as task input

Hello all.

I’ve been trying to setup a pipeline for a python project.
The thing is I would like to install the app’s dependencies once, output that directory and use it as input for the next task.
Something like this:

jobs:

  • name: hello-world-python-tests
    plan:
    • get: hello-world-python-repo

    • task: setup
      config:
      platform: linux
      image_resource:
      type: docker-image
      source:
      repository: python
      tag: 3.6-alpine
      run:
      dir: hello-world-python-repo
      path: /bin/sh
      args:
      - ./scripts.sh
      - init_workspace # This calls pip install
      inputs:
      - name: hello-world-python-repo
      outputs:
      - name: /usr/local/lib/python3.6/site-packages

    • task: tests
      config:
      platform: linux
      image_resource:
      type: docker-image
      source:
      repository: python
      tag: 3.6-alpine
      run:
      dir: hello-world-python-repo
      path: /bin/sh
      args:
      - ./scripts.sh
      - run_tests # This calls pytest
      inputs:
      - name: hello-world-python-repo
      - name: /usr/local/lib/python3.6/site-packages

I am aware that only relative paths are supported, but is there a way this cloud be achieved for site-packages?
I also tried with pip install -b <some_directory> -r requirements.txt to then move <some_directory> to the path python expects with no luck. I get an “resource is busy” error.

Anyone know a way to do this?
Thanks!

Nvm. Found a way around it.

I run pip install -t dependencies -r app/requirements.txt, then copied the dependencies folder contents to /root/.local/lib/python3.6/site-packages.

run:
      dir: hello-world-python-repo
      path: /bin/sh
      args:
        - -c
        - mkdir -p /root/.local/lib/python3.6/site-packages && cp -r dependencies/* /root/.local/lib/python3.6/site-packages

Used python -c 'import site; print(site.getsitepackages())' to find out which dir to put the packages in.

If anybody knows of a better solution, let me know.

:slight_smile: