New xdusage 3.x deployment questions for service providers

7 posts / 0 new
Last post
New xdusage 3.x deployment questions for service providers

Early in 2022, XSEDE will release a pure Python xdusage V3.0 that accesses a new XSEDE allocations and accounting database using a RESTful API. We will request that all XSEDE allocated service providers install this version when it is ready.  Please reply to this message to provide feedback on the following two questions:

1) May we continue to wrap xdusage in sudo to protect database/API access credentials? This means that sudo must be available on nodes where users or administrators will run xdusage. If sudo is not acceptable, please suggest an alternate way to prevent users from being able to read the configuration file that xdusage needs to read.

2) Would it be acceptable if xdusage was ONLY delivered as a PyPI package so that it can be installed on a shared cluster-wide file-system? If not, we assume you require an RPM.


-- XSEDE Cyberinfrastructure Integration Team

(1) sudo wrapping is OK for Bridges-2 at PSC. (2) Not sure about the PyPI package - it depends what's in it, and what dependencies it assumes are available in the target environment. RPM may be better for consistency - although we're likely to deploy it centrally and have an LMOD module to add it to users' PATHs. - Derek

(1) Continuing to use sudo wrapping is OK for Stampede2 at TACC.

(2)  I agree with what Derek said in the previous post about PyPI and also the Python language does not provide backward compatibility to allow easy deployment across multiple sites.

1)  Yes, sudo is used for similar issues in other software.

2)  No, we run many different Python environments and different OS versions on different clusters, etc., so even if it were, our staff would almost certainly then take it and build an RPM around it for deployment purposes.

  1. Yes, sudo, as currently used for an unprivileged user, can be supported at SDSC.
  2. Even if provided in PyPI and/or via an RPM we would likely need to rip it apart and redo in order to have control of the Python dependencies and/or to install to our shared filesystem. So, extending with builtin support for install from source, python -m and/or binary package generation would be the most useful in our view.

We should have detailed that our PyPi package would create its own xdusage specific Python3 venv with the Python packages and versions needed by xdusage. Dependencies are very important and relying on system wide dependencies is unnecessarily fragile when Python3 includes native venv support.

Please let us know if this changes your SP answer to question 2) above.

1) Yes, we can continue to support sudo for this at the University of Delaware.

2) This would be fine with us.

Log in to post comments