I recommend a more extensive introduction paragraph the explains the role of IPF in the XSEDE environment, and motivates the SP to configure all the workflows and proactively make sure that IPF is working on their system. Also recommend a visual re-make of this document so that it's more visually appealing perhaps using Markdown.
In Introduction:
- Item 1 and 4 should be combine since there is only one module workflow now.
In Requirements:
- "The module files must be readable" appears twice
- Make "In addition .." a requirement rather that mention it separately
Dependencies:
- Need to specific python-amqp version requirements
RPM Installation:
- Change SD&I to XCI
- Rather than duplicate repo trust setup, point to appropriate online instructions at:
https://software.xsede.org/development/repo/repoconfig.txt
https://software.xsede.org/production/repo/repoconfig.txt
tar.gz Installation
- Should step 3) be done as xdinfo? Recommend this entire section be clear about which user to perform all these steps in.
- Make step 4) about configuring ipf_workflow clearer.
Updating an ipf-xsede.installation
- Mentions json files that haven't been explained before. Recommend some additional explanation of workflows and of the *.json configuration files tach each workflow uses. If appropriate explain this earlier in the doc.
- The /etc/ipf backup best practice appears after RPM/tar installation instructions. This should be mentioned in the installation instructions.
Configuring Torque Logging
- Need to review/expand this to mention all supported schedulers (like slurm)
Configure Service Files
- Remove all Teragrid kit and legacy references which are no longer relevant
- Remove references to mds-stopgap
Best Practices for Software publishing
- Need to explain what to publish into SupportContact, whether field values or URL reference to external information
Testing
- Remove references to deprecated workflows. Make sure the workflow name is consistent with the name at the beginning of the document.
Hi Eric,
I think your revised installation document addresses most of JP's feedback above except for the comment
"Mentions json files that haven't been explained before. Recommend some additional explanation of workflows and of the *.json configuration files tach each workflow uses. If appropriate explain this earlier in the doc.".
I think this could be really helpful in the introduction of the document to describe what does deploying IPF look like. E.g., multiple daemons, each with a JSON workflow file and a init script. Also workflow is such an overloaded term, it might be good to clarify what it means in the IPF context before using it. Derek also had a request at one point to understand what was installed by IPF and where to find errors. The first mention of logging is in the Testing area.
I'm also wondering -- is it a best practice to have one initd script per daemon? Or could it be consolidated into one script and simplify the steps the sysadmin has to do?
Also, I rendered an HTML version of the markdown file for now because I was having trouble reading the document. I installed a python markdown implementation via Yum, markdown_py. I tried to configure it in Apache but it didn't work and I'm having trouble locating where the error is being logged (asked Kate for help). If it turns out to be time consuming, how hard would it be to update your build scripts to run the markdown_py script on the md file?
Thanks,
Shava
Thanks for the comments.
I will take another look at revising how we try to explain the workflows.
With regard to one initd script per daemon, it is that way for two reasons: 1) there is actually a separate process running for each periodic workflow, not a single daemon that fires off multiple kinds of workflows, and 2) not all sites run the same set of workflows.
Given these, I don't think it's worth it to try to consolidate init.d scripts for this release, though I'm certainly willing to think for the future about creating a configurable daemon that fires off the individual workflows.
It would not be hard to add markdown_py to the build and upload scripts; sounds like a good idea. So we'll have an INSTALL.md and an INSTALL.html. Then there's no need to do the apache module.
Eric
I've added an overview section to the Install document, that covers what an IPF workflow is, how it is defined, and how it is invoked. INSTALL.md has been committed to SVN, and INSTALL.md and INSTALL.html have been uploaded to the rc2 dir on software.xsede.org. Please let me know if you think this is good now, or if further revisions are advisable.
Hi Eric,
This looks great except that %VER%-%REL% did not get resolved. Can you fix before testing is completed?
Thanks,
Shava
Fixed.