Cannot configure pip installation

6 posts / 0 new
Last post
Cannot configure pip installation

After installing IPF with

pip3 install --user ipf==1.7a2

running "ipf_configure_xsede --pip ..." reports (last part of traceback):

File "/home/xdinfo/.local/lib/python3.6/site-packages/ipf/xsede/", line 879, in readWorkflowFile
f = open(path)
FileNotFoundError: [Errno 2] No such file or directory: '/usr/lib/python3.6/site-packages/etc/ipf/workflow/templates/glue2/extmodules.json'

It should be looking under /home/xdinfo/.local/lib/python3.6/site-packages but when running

ipf_configure_xsede --base_dir ~/.local/lib/python3.6/site-packages ...

it fails with

File "/home/xdinfo/.local/lib/python3.6/site-packages/ipf/xsede/", line 640, in writeExtModulesWorkflow
f = open(path, "w")
FileNotFoundError: [Errno 2] No such file or directory: '/home/xdinfo/.local/lib/python3.6/site-packages/etc/ipf/workflow/glue2/expanse-dev_extmodules.json'

because the glue2 subdirectory of /home/xdinfo/.local/lib/python3.6/site-packages/etc/ipf/workflow does not exist. It looks like a well-timed mkdir would be enough to fix this.

Configuration works smoothly using the rpm method.

Delivery Effort Stage: 

Thanks for the report--I had discovered this issue (and fixed it, as you noted) w/r/t the RPM packaging, but hadn't realized it was an issue on the pip side.

I will cut a 1.7a3 release that addresses this, and post here when it is ready.

ipf-1.7a3 is up at  It is a packaging-only update (so, technically should maybe be 1.7a2-2, but that's not how I'm going to do things this time). 

There is no ipf-1.7a3 rpm.

ipf-1.7a3 solves the configuration problem. Thanks.

Followup: after installing ipf-1.7a3 with

pip3 install --user ipf==1.7a3

and then configuring with:

ipf_configure_xsede --base_dir ~/.local/lib/python3.6/site-packages --resource_name expanse-dev --workflows=extmodules,compute --publish_to_xsede --amqp_certificate ~/xdinfo-hostcert.pem --amqp_certificate_key ~/xdinfo-hostcert.pem --scheduler slurm

it runs successfully, but if I replace the --base_dir option with --pip it fails with

FileNotFoundError: [Errno 2] No such file or directory: '/usr/lib/python3.6/site-packages/etc/ipf/workflow/templates/glue2/extmodules.json'

so I think it still is not correctly detecting the installation location.

This is an edge case for installation that (after discussion with JP, we have decided that) we do not support.  Which is to say:  we support using pip install inside a venv/virtualenv or, if one must, as part of the system python install, but there's no code to handle doing a user install on top of the system install (which is what puts things in ~/.local).  The supported course of action for a pip install as a user is to create a venv, and install everything inside the venv.  Install doc will be updated to reflect this.

Log in to post comments